String Dimensions

TileDB sparse arrays support the use of string dimensions. Learn how to work with string dimensions effectively in this tutorial.

How to run this tutorial

You can run this tutorial in two ways:

Locally on your machine.
On TileDB Cloud.

However, since TileDB Cloud has a free tier, we strongly recommend that you sign up and run everything there, as that requires no installations or deployment.

This tutorial explains how to use string dimensions. Note that this applies only to sparse arrays.

First, import the necessary libraries, set the array URI (that is, its path, which in this tutorial will be on local storage), and delete any previously created arrays with the same name.

Python
R

# Import necessary libraries
import tiledb
import numpy as np
import shutil
import os.path

# Set array URI
array_uri = os.path.expanduser("~/string_dimensions")

# Delete array if it already exists
if os.path.exists(array_uri):
    shutil.rmtree(array_uri)

# Import necessary libraries
library(tiledb)

# Set array URI
array_uri <- path.expand("~/string_dimensions_r")

# Delete array if it already exists
if (file.exists(array_uri)) {
  unlink(array_uri, recursive = TRUE)
}

Next, create the array by specifying its schema. Notice how a string dimension defines its domain.

Python
R

# Create the two dimensions, one string, one int64
d_str = tiledb.Dim(name="str_dim", domain=(None, None), dtype=np.bytes_)
d_int = tiledb.Dim(name="int64_dim", domain=(0, 100), tile=10, dtype=np.int64)

# Create a domain using the two dimensions
dom = tiledb.Domain(d_str, d_int)

# Create an attribute
a = tiledb.Attr(name="a", dtype=np.int32)

# Create the array schema, setting `sparse=True` to indicate a sparse array
sch = tiledb.ArraySchema(domain=dom, sparse=True, attrs=[a])

# Create the array on disk (it will initially be empty)
tiledb.Array.create(array_uri, sch)

# Create string, int64, and float64 dimensions
d_str <- tiledb_dim("str_dim", NULL, NULL, "ASCII")
d_int <- tiledb_dim("int32_dim", c(1L, 100L), 10L, "INT32")

# Create a domain using the above dimensions
dom <- tiledb_domain(c(d_str, d_int))

# Create an attribute
att <- tiledb_attr("a", "INT32")

# Create a sparse schema
schema <- tiledb_array_schema(dom, att, sparse = TRUE)

# Create the array on disk
arr <- tiledb_array_create(array_uri, schema)

Populate the TileDB array by passing 1D input arrays in the coordinate (COO) format.

Python
R

# Define some data
data = [1, 2, 3, 4]
c_str = [b"aa", b"bbb", b"c", b"dddd"]
c_int64 = [0, 10, 20, 30]

# Write data to the array
with tiledb.open(array_uri, "w") as A:
    A[c_str, c_int64] = data

data <- c(1L, 2L, 3L, 4L)
c_str <- c("aa", "bbb", "c", "dddd")
c_int32 <- c(10L, 20L, 30L, 40L)

# write data to the array
arr <- tiledb_array(array_uri, return_as = "data.frame")
arr[] <- data.frame(
  str_dim = c_str,
  int32_dim = c_int32,
  a = data
)

Read the data by using the different slicing methods supported in TileDB.

Python
R

# Slice data
with tiledb.open(array_uri) as A:
    print("Read full array:")
    print(A[:])

    print("\nRead string slice A['c':'dddd']:")
    print(A["c":"dddd"])

    print("\nRead A[:, 10]: ")
    print(A["aa":"bbb"])

    print("\nRead A.multi_index['aa':'c', 0:10]")
    print(A.multi_index["aa":"c", 0:10])

Read full array:
OrderedDict({'a': array([1, 2, 3, 4], dtype=int32), 'str_dim': array([b'aa', b'bbb', b'c', b'dddd'], dtype=object), 'int64_dim': array([ 0, 10, 20, 30])})

Read string slice A['c':'dddd']:
OrderedDict({'a': array([3, 4], dtype=int32), 'str_dim': array([b'c', b'dddd'], dtype=object), 'int64_dim': array([20, 30])})

Read A[:, 10]: 
OrderedDict({'a': array([1, 2], dtype=int32), 'str_dim': array([b'aa', b'bbb'], dtype=object), 'int64_dim': array([ 0, 10])})

Read A.multi_index['aa':'c', 0:10]
OrderedDict({'str_dim': array([b'aa', b'bbb'], dtype=object), 'int64_dim': array([ 0, 10]), 'a': array([1, 2], dtype=int32)})

# Open array as a data.frame
arr <- tiledb_array(array_uri, return_as = "data.frame")
df <- arr[]

# Print the whole dataframe
cat("Whole dataframe:\n")
print(df)

# Get a summary of the data
cat("Summary:\n")
print(summary(df))

# Get string dimension
cat("String dimension:\n")
print(df$str_dim)

Whole dataframe:
  str_dim int32_dim a
1      aa        10 1
2     bbb        20 2
3       c        30 3
4    dddd        40 4
Summary:
   str_dim            int32_dim          a       
 Length:4           Min.   :10.0   Min.   :1.00  
 Class :character   1st Qu.:17.5   1st Qu.:1.75  
 Mode  :character   Median :25.0   Median :2.50  
                    Mean   :25.0   Mean   :2.50  
                    3rd Qu.:32.5   3rd Qu.:3.25  
                    Max.   :40.0   Max.   :4.00  
String dimension:
[1] "aa"   "bbb"  "c"    "dddd"

Clean up in the end by deleting the array.

Python
R

# Delete the array
if os.path.exists(array_uri):
    shutil.rmtree(array_uri)

if (file.exists(array_uri)) {
  unlink(array_uri, recursive = TRUE)
}