TileDB arrays support complex attribute structures with different data types.
How to run this tutorial
You can run this tutorial in two ways:
Locally on your machine.
On TileDB Cloud.
However, since TileDB Cloud has a free tier, we strongly recommend that you sign up and run everything there, as that requires no installations or deployment.
This tutorial shows how to create arrays with multiple attributes and introduces some basic notions. For more detail about attributes, visit Key Concepts: Attributes.
First, import the necessary libraries, set the array URI (that is, its path, which in this tutorial will be on local storage), and delete any previously created arrays with the same name.
# Import necessary librariesimport os.pathimport shutilimport numpy as npimport tiledb# Set array URIarray_uri = os.path.expanduser("~/multiple_attributes")# Delete array if it already existsif os.path.exists(array_uri): shutil.rmtree(array_uri)
# Import necessary librarieslibrary(tiledb)# Set array URIarray_uri <-path.expand("~/multiple_attributes_r")# Delete array if it already existsif (file.exists(array_uri)) {unlink(array_uri, recursive =TRUE)}
Next, create the array by specifying its schema, which accepts two attributes. The array in this tutorial is dense, but everything covered here applies to sparse arrays as well.
# Create the two dimensionsd1 = tiledb.Dim(name="d1", domain=(1, 4), tile=2, dtype=np.int32)d2 = tiledb.Dim(name="d2", domain=(1, 4), tile=2, dtype=np.int32)# Create a domain using the two dimensionsdom = tiledb.Domain(d1, d2)# Order of the dimensions matters when slicing subarrays.# Remember to give priority to more selective dimensions to# maximize the pruning power during slicing.# Create two attributes, one integer, one floata1 = tiledb.Attr(name="a1", dtype=np.int32)a2 = tiledb.Attr(name="a2", dtype=np.float32)# Create the array schema, setting `sparse=False` to indicate a dense arraysch = tiledb.ArraySchema(domain=dom, sparse=False, attrs=[a1, a2])# Create the array on disk (it will initially be empty)tiledb.Array.create(array_uri, sch)
# Create the two dimensionsd1 <-tiledb_dim("d1", c(1L, 4L), 2L, "INT32")d2 <-tiledb_dim("d2", c(1L, 4L), 2L, "INT32")# Create a domain using the two dimensionsdom <-tiledb_domain(dims =c(d1, d2))# Order of the dimensions matters when slicing subarrays.# Remember to give priority to more selective dimensions to# maximize the pruning power during slicing.# Create an attributea1 <-tiledb_attr("a1", type ="INT32")a2 <-tiledb_attr("a2", type ="FLOAT64")# Create the array schema, setting `sparse = FALSE` to indicate a dense arraysch <-tiledb_array_schema(dom, c(a1, a2), sparse =FALSE)# Create the array on disk (it will initially be empty)arr <-tiledb_array_create(array_uri, sch)
Populate the TileDB array with two 2D input arrays, one for each attribute.
# Prepare some data in two NumPy arrays, one for each attributea1_data = np.array( [[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12], [13, 14, 15, 16]], dtype=np.int32)a2_data = np.array( [ [1.1, 2.2, 3.3, 4.4], [5.5, 6.6, 7.7, 8.8], [9.9, 10.10, 11.11, 12.12], [13.13, 14.14, 15.15, 16.16], ], dtype=np.float32,)# Write data to the arraywith tiledb.open(array_uri, "w") as A: A[:] = {"a1": a1_data, "a2": a2_data}
# Prepare some data in two arrays, one for each attributea1_data <-t(array(1:16, dim =c(4, 4)))a2_data <-array(c(1.1, 2.2, 3.3, 4.4,5.5, 6.6, 7.7, 8.8,9.9, 10.10, 11.11, 12.12,13.13, 14.14, 15.15, 16.16 ),dim =c(4L, 4L))# Open the array for writing and write data to the arrayarr <-tiledb_array(uri = array_uri,query_type ="WRITE",return_as ="data.frame")arr[] <-list(a1 = a1_data,a2 = a2_data)# Close the arrayarr <-tiledb_array_close(arr)
The array is a folder in the path specified in array_uri. You can learn about the different contents of the array folder in other sections of the Academy.
Note the two separate files storing the array data (a0.tdb and a1.tdb), one for each attribute.
# Open the array in read modeA = tiledb.open(array_uri, "r")# Show the entire arrayprint("Entire array: ")print(A[:])print("\n")# Show the 'a1' attribute of the arrayprint("Attribute 'a1': ")print(A[:]["a1"])print("\n")# Show the 'a2' attribute of the arrayprint("Attribute 'a2': ")print(A[:]["a2"])# Remember to close the arrayA.close()
# Open the array in read modeinvisible(tiledb_array_open(arr, type ="READ"))# Show the entire arraycat("Entire array:\n")print(arr[])# Show the 'a1' attribute of the arraycat("Attribute 'a1':\n")print(arr[]["a1"])# Show the 'a1' attribute of the arraycat("Attribute 'a2':\n")print(arr[]["a2"])# Close the arrayinvisible(tiledb_array_close(arr))
In Python, you can read the data by specifying a subset of the attributes in a query. This instructs TileDB to not retrieve any data at all for attributes not specified in the query. This is a much faster operation than bringing data for all attributes from storage to main memory, and then subselect on the attribute values. For more information on performance tuning, visit the Performance section.
# Open the array in read modeA = tiledb.open(array_uri, "r")# Prepare queryq = A.query(attrs=["a2"])# Show the entire array, but use qprint("Entire array on 'a2': ")print(q[:])print("\n")# Slice, but use q - returns only 'a2' valuesprint("Slice [1:3), [1:2): ")print(q[1:3, 1:2])# Remember to close the arrayA.close()