Encryption

Learn how to encrypt your arrays with AES-256 key for secure storage.
How to run this tutorial

You can run this tutorial in two ways:

  1. Locally on your machine.
  2. On TileDB Cloud.

However, since TileDB Cloud has a free tier, we strongly recommend that you sign up and run everything there, as that requires no installations or deployment.

This tutorial explains how to create, write, and read encrypted arrays. For more information, visit the Key Concepts: Encryption section.

TileDB supports the Advanced Encryption Standard (AES). To create an encrypted array, you need to generate an AES-256 key and pass it to the corresponding array statements in your application.

Important

This page uses a sample AES-256 key 0123456789abcdeF0123456789abcdeF directly in its code examples for illustrative purposes only. Do not use this key to encrypt your arrays.

Additionally, do not store encryption keys within the code of your application.

First, import the necessary libraries, set the array URI (that is, its path, which in this tutorial will be on local storage), and delete any previously created arrays with the same name.

# Import necessary libraries
import tiledb
import numpy as np
import shutil
import os.path

# Set array URI
array_uri = os.path.expanduser("~/encryption")

# Delete array if it already exists
if os.path.exists(array_uri):
    shutil.rmtree(array_uri)
library(tiledb)

array_uri <- path.expand("~/datetimes_r")

if (dir.exists(array_uri)) {
  unlink(array_uri, recursive = TRUE)
}

Next, create the array by specifying its schema. This is like the case where no encryption used, except you pass an encryption key in different methods and functions.

When working with encrypted arrays in Python, you pass the encryption key as a parameter into the commands that create and open the array. With R, you always pass the encryption key into a config object, which you pass into a context object. You use the context object when creating the array schema with tiledb_array_schema() and when creating or opening the array with tiledb_array().

# Create the two dimensions
d1 = tiledb.Dim(name="d1", domain=(0, 3), tile=2, dtype=np.int32)
d2 = tiledb.Dim(name="d2", domain=(0, 3), tile=2, dtype=np.int32)

# Create a domain using the two dimensions
dom = tiledb.Domain(d1, d2)

# Create an attribute
# The filter list we created above is passed into the `filters` parameter
a = tiledb.Attr(name="a", dtype=np.int32)

# Create the array schema with `sparse=False`
sch = tiledb.ArraySchema(domain=dom, sparse=False, attrs=[a])

# Create an encryption key
encryption_key = "0123456789abcdeF0123456789abcdeF"

# Create the array on disk (it will initially be empty).
# Observe that the encryption key is passed into `key`.
# This indicates that the array will utilize encryption.
tiledb.Array.create(array_uri, sch, key=encryption_key)
# Create an encryption key
encryption_key <- "0123456789abcdeF0123456789abcdeF"

# define a config object to house the encryption_key
config <- tiledb_config()
config["sm.encryption_type"] <- "AES_256_GCM"
config["sm.encryption_key"] <- encryption_key

# set the context
ctx <- tiledb_ctx(config)

# Create the two dimensions
d1 <- tiledb_dim("d1", c(1L, 4L), 2L, "INT32")
d2 <- tiledb_dim("d2", c(1L, 4L), 2L, "INT32")

# Create a domain using the two dimensions
dom <- tiledb_domain(dims = c(d1, d2))

# Create an attribute
a <- tiledb_attr("a", type = "INT32")

# Create the array schema with `sparse = FALSE`
sch <- tiledb_array_schema(dom, a, sparse = FALSE, ctx = ctx)

# Create the array on disk (it will initially be empty).
# Observe that the encryption key is passed into
# `encryption_key`. This indicates that the array will
# utilize encryption.
cat("Creating array...")
arr <- tiledb_array_create(array_uri, sch)

Populate the TileDB array with a 2D input array, like in the unencrypted case.

# Prepare some data in a NumPy array
data = np.array(
    [[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12], [13, 14, 15, 16]], dtype=np.int32
)

# Write data to the array.
# This time you need to pass the encryption key when
# opening the array, otherwise you will get an error.
with tiledb.open(array_uri, "w", key=encryption_key) as A:
    A[:] = data
# Prepare some data in an array
data <- t(array(1:16, dim = c(4, 4)))

# Open the array for writing and write data to the array
arr <- tiledb_array(
  uri = array_uri,
  query_type = "WRITE",
  ctx = ctx,
  return_as = "data.frame"
)
arr[] <- data

# Close the array
arr <- tiledb_array_close(arr)

Now read the data in the array:

# Open the array in read mode.
# This time you need to pass the encryption key when
# opening the array, otherwise you will get an error.
A = tiledb.open(array_uri, "r", key=encryption_key)

# Read the entire array
print(A[:])

# Close the array
A.close()
OrderedDict({'a': array([[ 1,  2,  3,  4],
       [ 5,  6,  7,  8],
       [ 9, 10, 11, 12],
       [13, 14, 15, 16]], dtype=int32)})
arr <- tiledb_array(
  array_uri,
  query_type = "READ",
  ctx = ctx,
  return_as = "data.frame"
)

ints <- arr[]
print(ints)

# Be sure to close the array
invisible(tiledb_array_close(arr))
   d1 d2  a
1   1  1  1
2   2  1  5
3   3  1  9
4   4  1 13
5   1  2  2
6   2  2  6
7   3  2 10
8   4  2 14
9   1  3  3
10  2  3  7
11  3  3 11
12  4  3 15
13  1  4  4
14  2  4  8
15  3  4 12
16  4  4 16

Clean up in the end by deleting the array.

# Delete the array
if os.path.exists(array_uri):
    shutil.rmtree(array_uri)
if (dir.exists(array_uri)) {
  unlink(array_uri, recursive = TRUE)
}