Deployment Methods

ai/ml

vector search

foundation

Learn about about the deployment methods of the variety of vector search algorithms supported by TileDB.

This section covers different ways in which TileDB-Vector-Search can be run.

Methods

TileDB-Vector-Search implements the following execution methods for vector ingestion and similarity search:

Local machine, main-memory: The client machine performs the execution locally, assuming it has unlimited main memory. This can be your laptop or a single server that you manage. Main-memory execution can achieve low latency. However, it fails to handle larger datasets when the client machine has insufficient resources.
Local machine, out-of-core: The execution is performed locally on the client machine, respecting a user provided main-memory limit. This can be your laptop or a single server that you manage. Out-of-core implementations use disk I/O and chunking of execution to perform the computation within the limited main-memory budget. They typically have larger latency than main-memory based implementations. However, they can handle arbitrarily large inputs.
Serverless, main-memory: The execution is similar to Local machine, main-memory but it is executed within a TileDB Cloud Serverless UDF. This allows you to specify the exact amount of resources required for each task execution and allocate them on demand with TileDB Cloud, without having to manage servers on your side and paying only for the resources utilized during the execution. Another advantage of this method is that it allows the execution server to be collocated with data stored in cloud object stores, taking advantage of network latency and throughput improvements for data access.
Serverless, out-of-core: The execution is similar to Local machine, out-of-core but it is executed within a TileDB Cloud serverless UDF, providing the serverless execution benefits discussed previously.
Distributed: The execution is represented as a TileDB Cloud task graph, a DAG (Directed Acyclic Graph) of tasks, and executed with on-demand provisioned resources in TileDB Cloud. This allows you to handle larger datasets by leveraging distributed resources to parallelize the task execution.

Index type support

Not all of the TileDB-Vector-Search indexing algorithms support the same capabilities. This section lays out support for each index type for both ingestion and similarity searches:

Ingestion

This table lays out support for each index type for ingestion:

	`FLAT`	`IVF_FLAT`	`VAMANA`
Local machine, main-memory	✅	✅	✅
Local machine, out-of-core	✅	✅	❌
Serverless, main memory	✅	✅	✅
Serverless, out-of-core	✅	✅	❌
Distributed	❌	✅	❌

To configure these deployment techniques, use the mode and input_vectors_per_work_item parameters in the ingest() function. To learn more about how to configure these parameters and how they will affect performance, visit Key Concepts: Performance.

Similarity search

This table lays out support for each index type for similarity searches:

	`FLAT`	`IVF_FLAT`	`VAMANA`
Local machine, main-memory	✅	✅	✅
Local machine, out-of-core	✅	✅	❌
Serverless, main memory	✅	✅	✅
Serverless, out-of-core	✅	✅	❌
Distributed	❌	✅	❌

To configure these deployment techniques, use the memory_budget parameter in the Index class constructor as well as the mode and driver_mode parameters in the query() function. To learn more about how to configure these parameters and how they will affect performance, visit Key Concepts: Performance.