Object Stores

TileDB works seamlessly with cloud object stores, so your data can live under your control.

Object stores (such as Amazon S3, Google Cloud Storage, etc.) provide several benefits to organizations that are looking to store and analyze large quantities of data, such as “infinite” scalability and high throughput at a relatively low cost. However, object stores introduce a few challenges:

• Every object (i.e., file) is immutable. That means that, even if you wish to change a single byte, you will need to change the entire object.
• Certain operations, such as listing objects, may be subject to different consistency guarantees.
• File locking isn’t a concept known to or supported by object stores.
• Object stores may incur high latency when reading the various objects.

The above characteristics greatly affect the way data systems implement their various operations, such as updates, indexing, etc. TileDB takes appropriate care to optimize I/O for each object store using the corresponding SDK, and nicely abstracts the backends, enabling future support for new backends. Visit the Tutorials: Backends section for more information on the object stores that TileDB supports.