Slice Data
TileDB’s architecture allows for elastic and scalable data access. The serverless nature and the underlying storage format enables for a completely lock-free, multi-reader/multi-writer system, which provides massive scalability.
Efficiency
When data is sliced, the request is sent to TileDB Cloud where all filtering and computations happen server side and only the results are returned to the client. This minimizes data movement and allows for high performance with lower egress costs and lower
Multi-region redirection
TileDB Cloud SaaS is deployed to five AWS regions currently listed below. The goal of running TileDB Cloud in each region is to allow computations to take place in the same region as the data. This allows for faster performance and reduced AWS costs for intra-region data transfers.
us-east-1us-west-2eu-west-1eu-west-2ap-southeast-1
Automatic redirection
To facilitate always having compute in the same region as the data, TileDB supports automatic redirection of your queries and tasks to the proper supported region.
How it works
TileDB Cloud uses Cloudflare workers for automatic redirection, which will look up the region of the array being accessed and issue a 302 temporary redirect to the HTTP request. TileDB Embedded or the TileDB Cloud APIs then honors the redirection and sends the request to the TileDB Cloud service in the proper region.
If your array lives in a region which TileDB Cloud does not run, TileDB Cloud sends the request to the nearest region in which a TileDB Cloud instance is available.
Use automatic redirection
By default, TileDB Cloud automatically redirects you to the closest region, so you do not need to specify a server address for cases where latency is not a large concern.
For situations where you wish to minimize the latency of your queries, you can specify a specific region directly.
Access a region directly
You can access a region directly, bypassing the automatic redirection. This is helpful if you want to avoid the slight increase in latency that the redirection adds.
To access a region directly the domain is of the scheme: <region>.aws.api.tiledb.com
The five existing domains are:
us-east-1.aws.api.tiledb.comus-west-2.aws.api.tiledb.comeu-west-1.aws.api.tiledb.comeu-west-2.aws.api.tiledb.comap-southeast-1.aws.api.tiledb.com
Just like with the redirection domain, you can manually set the domain to send a request directly to a region.
import tiledb, tiledb.sql
import pandas
# Create the configuration parameters
config = tiledb.Config()
config["rest.username"] = "my_username"
config["rest.password"] = "my_password"
# or, more preferably, config["rest.token"] = "my_token"
# Manually set the server address to the redirection URL
config["rest.server_address"] = "https://eu-west-2.aws.api.tiledb.com"
# This is the array URI format in TileDB Cloud
array_name = "tiledb://TileDB-Inc/quickstart_sparse-eu-west-2"
# Write code exactly as in TileDB Developer
with tiledb.SparseArray(array_name, "r", ctx=tiledb.Ctx(config)) as A:
data = A[:]
print(pandas.DataFrame(data))
# Using embedded SQL, you need to pass the username/password as config parameters as well as the server_addres in `init_command`
db = tiledb.sql.connect(
db="test",
init_command="set mytile_tiledb_config='rest.username=xxx,rest.password=xxx,rest.server_address=https://eu-west-2.aws.api.tiledb.com,...'",
)
pandas.read_sql(sql="select * from `<array_uri>`", con=db)import tiledb, tiledb.cloud, numpy
def median(numpy_ordered_dictionary):
return numpy.median(numpy_ordered_dictionary["a"])
tiledb.cloud.login(
username="my_username",
password="my_password",
host="https://eu-west-2.aws.api.tiledb.com",
)
# or tiledb.cloud.login(token="my_token", host="https://eu-west-2.aws.api.tiledb.com")
with tiledb.DenseArray(
"tiledb://TileDB-Inc/quickstart_sparse-eu-west-2", ctx=tiledb.cloud.Ctx()
) as A:
# apply on subarray [1,4]x[1,4]
res = A.apply(median, [(1, 4), (1, 4)], attrs=["a"])
print(res)