Multi-Range Reads

You can read array data that spans multiple ranges, rather than running several individual read queries.
How to run this tutorial

You can run this tutorial in two ways:

  1. Locally on your machine.
  2. On TileDB Cloud.

However, since TileDB Cloud has a free tier, we strongly recommend that you sign up and run everything there, as that requires no installations or deployment.

This tutorial shows how to perform multi-range slices. Although it focuses on a dense array, mutli-range slicing applies to sparse arrays as well. Visit the Key Concepts: Reads section for more details on multi-range slicing.

First, import the necessary libraries, set the array URI (that is, its path, which in this tutorial will be on local storage), and delete any previously created arrays with the same name.

# Import necessary libraries
import tiledb
import numpy as np
import shutil
import os.path

# Set array URI
array_uri = os.path.expanduser("~/multi_range_reads")

# Delete array if it already exists
if os.path.exists(array_uri):
    shutil.rmtree(array_uri)
# Import necessary libraries
library(tiledb)

# Set array URI
array_uri <- path.expand("~/multi_range_reads_r")

# Delete array if it already exists
if (file.exists(array_uri)) {
  unlink(array_uri, recursive = TRUE)
}

Next, create the array by specifying its schema.

# Create the two dimensions
d1 = tiledb.Dim(name="d1", domain=(1, 4), tile=2, dtype=np.int32)
d2 = tiledb.Dim(name="d2", domain=(1, 4), tile=2, dtype=np.int32)

# Create a domain using the two dimensions
dom = tiledb.Domain(d1, d2)

# Create an attribute
a = tiledb.Attr(name="a", dtype=np.int32)

# Create the array schema, setting `sparse=False` to indicate a dense array
sch = tiledb.ArraySchema(domain=dom, sparse=False, attrs=[a])

# Create the array on disk (it will initially be empty)
tiledb.Array.create(array_uri, sch)
# Create the two dimensions
d1 <- tiledb_dim("d1", c(1L, 4L), 2L, "INT32")
d2 <- tiledb_dim("d2", c(1L, 4L), 2L, "INT32")

# Create a domain using the two dimensions
dom <- tiledb_domain(dims = c(d1, d2))

# Create an attribute
a <- tiledb_attr("a", type = "INT32")

# Create the array schema, setting `sparse = FALSE` to indicate a dense array
sch <- tiledb_array_schema(dom, a, sparse = FALSE)

# Create the array on disk (it will initially be empty)
arr <- tiledb_array_create(array_uri, sch)

Populate the TileDB array with a 2D input array.

# Prepare some data in a NumPy array
data = np.array(
    [[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12], [13, 14, 15, 16]], dtype=np.int32
)

# Write data to the array
with tiledb.open(array_uri, "w") as A:
    A[:] = data
# Prepare some data in an array
data <- t(array(1:16, dim = c(4, 4)))

# Open the array for writing and write data to the array
arr <- tiledb_array(uri = array_uri, query_type = "WRITE", return_as = "data.frame")
arr[] <- data

# Close the array
arr <- tiledb_array_close(arr)

Use multi-range slicing to read data from the array.

# Open the array in read mode
A = tiledb.open(array_uri, "r")

# Show the entire array
print("Entire array: ")
print(A[:])
print("\n")

# Slice a multi-range subarray.
# Note that `multi_index` uses closed ranges.
print("Multi-range, rows 1,2 and 4, and columns 1-3: ")
print(A.multi_index[[slice(1, 2), 4], slice(1, 3)]["a"])

# Remember to close the array
A.close()
Entire array: 
OrderedDict({'a': array([[ 1,  2,  3,  4],
       [ 5,  6,  7,  8],
       [ 9, 10, 11, 12],
       [13, 14, 15, 16]], dtype=int32)})


Multi-range, rows 1,2 and 4, and columns 1-3: 
[[ 1  2  3]
 [ 5  6  7]
 [13 14 15]]
# Define selected ranges
selected_ranges <- list(
  d1 = matrix(c(
    1, 2,
    4, 4
  ), 2, 2, byrow = TRUE),
  d2 = cbind(1, 3)
)

# Open the array using the selected ranges
arr <- tiledb_array(
  array_uri,
  selected_ranges = selected_ranges,
  return_as = "data.frame"
)
print(arr[])
  d1 d2  a
1  1  1  1
2  2  1  5
3  4  1 13
4  1  2  2
5  2  2  6
6  4  2 14
7  1  3  3
8  2  3  7
9  4  3 15

Clean up in the end by deleting the array.

# Delete the array
if os.path.exists(array_uri):
    shutil.rmtree(array_uri)
if (file.exists(array_uri)) {
  unlink(array_uri, recursive = TRUE)
}