Multiple Attributes

TileDB arrays support complex attribute structures with different data types.
How to run this tutorial

You can run this tutorial in two ways:

  1. Locally on your machine.
  2. On TileDB Cloud.

However, since TileDB Cloud has a free tier, we strongly recommend that you sign up and run everything there, as that requires no installations or deployment.

This tutorial shows how to create arrays with multiple attributes and introduces some basic notions. For more detail about attributes, visit Key Concepts: Attributes.

First, import the necessary libraries, set the array URI (that is, its path, which in this tutorial will be on local storage), and delete any previously created arrays with the same name.

# Import necessary libraries
import tiledb
import numpy as np
import shutil
import os.path

# Set array URI
array_uri = os.path.expanduser("~/multiple_attributes")

# Delete array if it already exists
if os.path.exists(array_uri):
    shutil.rmtree(array_uri)
# Import necessary libraries
library(tiledb)

# Set array URI
array_uri <- path.expand("~/multiple_attributes_r")

# Delete array if it already exists
if (file.exists(array_uri)) {
  unlink(array_uri, recursive = TRUE)
}

Next, create the array by specifying its schema, which accepts two attributes. The array in this tutorial is dense, but everything covered here applies to sparse arrays as well.

# Create the two dimensions
d1 = tiledb.Dim(name="d1", domain=(1, 4), tile=2, dtype=np.int32)
d2 = tiledb.Dim(name="d2", domain=(1, 4), tile=2, dtype=np.int32)

# Create a domain using the two dimensions
dom = tiledb.Domain(d1, d2)
# Order of the dimensions matters when slicing subarrays.
# Remember to give priority to more selective dimensions to
# maximize the pruning power during slicing.

# Create two attributes, one integer, one float
a1 = tiledb.Attr(name="a1", dtype=np.int32)
a2 = tiledb.Attr(name="a2", dtype=np.float32)

# Create the array schema, setting `sparse=False` to indicate a dense array
sch = tiledb.ArraySchema(domain=dom, sparse=False, attrs=[a1, a2])

# Create the array on disk (it will initially be empty)
tiledb.Array.create(array_uri, sch)
# Create the two dimensions
d1 <- tiledb_dim("d1", c(1L, 4L), 2L, "INT32")
d2 <- tiledb_dim("d2", c(1L, 4L), 2L, "INT32")

# Create a domain using the two dimensions
dom <- tiledb_domain(dims = c(d1, d2))
# Order of the dimensions matters when slicing subarrays.
# Remember to give priority to more selective dimensions to
# maximize the pruning power during slicing.

# Create an attribute
a1 <- tiledb_attr("a1", type = "INT32")
a2 <- tiledb_attr("a2", type = "FLOAT64")

# Create the array schema, setting `sparse = FALSE` to indicate a dense array
sch <- tiledb_array_schema(dom, c(a1, a2), sparse = FALSE)

# Create the array on disk (it will initially be empty)
arr <- tiledb_array_create(array_uri, sch)

Populate the TileDB array with two 2D input arrays, one for each attribute.

# Prepare some data in two NumPy arrays, one for each attribute
a1_data = np.array(
    [[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12], [13, 14, 15, 16]], dtype=np.int32
)
a2_data = np.array(
    [
        [1.1, 2.2, 3.3, 4.4],
        [5.5, 6.6, 7.7, 8.8],
        [9.9, 10.10, 11.11, 12.12],
        [13.13, 14.14, 15.15, 16.16],
    ],
    dtype=np.float32,
)

# Write data to the array
with tiledb.open(array_uri, "w") as A:
    A[:] = {"a1": a1_data, "a2": a2_data}
# Prepare some data in two arrays, one for each attribute
a1_data <- t(array(1:16, dim = c(4, 4)))

a2_data <- array(
  c(
    1.1, 2.2, 3.3, 4.4,
    5.5, 6.6, 7.7, 8.8,
    9.9, 10.10, 11.11, 12.12,
    13.13, 14.14, 15.15, 16.16
  ),
  dim = c(4L, 4L)
)
# Open the array for writing and write data to the array
arr <- tiledb_array(
  uri = array_uri,
  query_type = "WRITE",
  return_as = "data.frame"
)

arr[] <- list(
  a1 = a1_data,
  a2 = a2_data
)

# Close the array
arr <- tiledb_array_close(arr)

The array is a folder in the path specified in array_uri. You can learn about the different contents of the array folder in other sections of the Academy.

Note the two separate files storing the array data (a0.tdb and a1.tdb), one for each attribute.

/Users/stavrospapadopoulos/multiple_attributes
├── __commits
│   └── __1715099712235_1715099712235_674ec8e52f10e893f678f429d504b9c3_21.wrt
├── __fragment_meta
├── __fragments
│   └── __1715099712235_1715099712235_674ec8e52f10e893f678f429d504b9c3_21
│       ├── __fragment_metadata.tdb
│       ├── a0.tdb
│       └── a1.tdb
├── __labels
├── __meta
└── __schema
    ├── __1715099712229_1715099712229_00000002d59b8f49a2495a3957b1bc5c
    └── __enumerations

9 directories, 5 files
/Users/stavrospapadopoulos/multiple_attributes
├── __commits
│   └── __1715099712235_1715099712235_674ec8e52f10e893f678f429d504b9c3_21.wrt
├── __fragment_meta
├── __fragments
│   └── __1715099712235_1715099712235_674ec8e52f10e893f678f429d504b9c3_21
│       ├── __fragment_metadata.tdb
│       ├── a0.tdb
│       └── a1.tdb
├── __labels
├── __meta
└── __schema
    ├── __1715099712229_1715099712229_00000002d59b8f49a2495a3957b1bc5c
    └── __enumerations

9 directories, 5 files

Read the data.

# Open the array in read mode
A = tiledb.open(array_uri, "r")

# Show the entire array
print("Entire array: ")
print(A[:])
print("\n")

# Show the 'a1' attribute of the array
print("Attribute 'a1': ")
print(A[:]["a1"])
print("\n")

# Show the 'a2' attribute of the array
print("Attribute 'a2': ")
print(A[:]["a2"])

# Remember to close the array
A.close()
Entire array: 
OrderedDict({'a1': array([[ 1,  2,  3,  4],
       [ 5,  6,  7,  8],
       [ 9, 10, 11, 12],
       [13, 14, 15, 16]], dtype=int32), 'a2': array([[ 1.1 ,  2.2 ,  3.3 ,  4.4 ],
       [ 5.5 ,  6.6 ,  7.7 ,  8.8 ],
       [ 9.9 , 10.1 , 11.11, 12.12],
       [13.13, 14.14, 15.15, 16.16]], dtype=float32)})


Attribute 'a1': 
[[ 1  2  3  4]
 [ 5  6  7  8]
 [ 9 10 11 12]
 [13 14 15 16]]


Attribute 'a2': 
[[ 1.1   2.2   3.3   4.4 ]
 [ 5.5   6.6   7.7   8.8 ]
 [ 9.9  10.1  11.11 12.12]
 [13.13 14.14 15.15 16.16]]
# Open the array in read mode
invisible(tiledb_array_open(arr, type = "READ"))

# Show the entire array
cat("Entire array:\n")
print(arr[])

# Show the 'a1' attribute of the array
cat("Attribute 'a1':\n")
print(arr[]["a1"])

# Show the 'a1' attribute of the array
cat("Attribute 'a2':\n")
print(arr[]["a2"])

# Close the array
invisible(tiledb_array_close(arr))
Entire array:
   d1 d2 a1    a2
1   1  1  1  1.10
2   2  1  5  2.20
3   3  1  9  3.30
4   4  1 13  4.40
5   1  2  2  5.50
6   2  2  6  6.60
7   3  2 10  7.70
8   4  2 14  8.80
9   1  3  3  9.90
10  2  3  7 10.10
11  3  3 11 11.11
12  4  3 15 12.12
13  1  4  4 13.13
14  2  4  8 14.14
15  3  4 12 15.15
16  4  4 16 16.16
Attribute 'a1':
   a1
1   1
2   5
3   9
4  13
5   2
6   6
7  10
8  14
9   3
10  7
11 11
12 15
13  4
14  8
15 12
16 16
Attribute 'a2':
      a2
1   1.10
2   2.20
3   3.30
4   4.40
5   5.50
6   6.60
7   7.70
8   8.80
9   9.90
10 10.10
11 11.11
12 12.12
13 13.13
14 14.14
15 15.15
16 16.16

In Python, you can read the data by specifying a subset of the attributes in a query. This instructs TileDB to not retrieve any data at all for attributes not specified in the query. This is a much faster operation than bringing data for all attributes from storage to main memory, and then subselect on the attribute values. For more information on performance tuning, visit the Performance section.

# Open the array in read mode
A = tiledb.open(array_uri, "r")

# Prepare query
q = A.query(attrs=["a2"])

# Show the entire array, but use q
print("Entire array on 'a2': ")
print(q[:])
print("\n")

# Slice, but use q - returns only 'a2' values
print("Slice [1:3), [1:2): ")
print(q[1:3, 1:2])

# Remember to close the array
A.close()
Entire array on 'a2': 
OrderedDict({'a2': array([[ 1.1 ,  2.2 ,  3.3 ,  4.4 ],
       [ 5.5 ,  6.6 ,  7.7 ,  8.8 ],
       [ 9.9 , 10.1 , 11.11, 12.12],
       [13.13, 14.14, 15.15, 16.16]], dtype=float32)})


Slice [1:3), [1:2): 
OrderedDict({'a2': array([[1.1],
       [5.5]], dtype=float32)})

Clean up in the end by deleting the array.

# Delete the array
if os.path.exists(array_uri):
    shutil.rmtree(array_uri)
if (file.exists(array_uri)) {
  unlink(array_uri, recursive = TRUE)
}