Task Graphs

Description of TileDB catalog features for task graphs.

TileDB has a powerful distributed computing infrastructure based on task graphs.

TileDB allows you to create, manage, and execute task graphs inside its secure infrastructure. Each task is essentially code to be executed, and the graph defines a parallel framework that connects the tasks through dependencies. The TileDB execution engine computes all the tasks in parallel, respecting their dependencies. Task graphs provide a powerful way to implement distributed workflows and algorithms for unprecedented scale and performance.

Add task graph

Section Scale includes all the information you need to know about how to create, register, and execute task graphs.

Once created, your task graph will appear under Assets -> Code -> Task graphs.

Overview

The Overview tab provides basic information about a task graph:

• Description - If you provided a description to the task graph (e.g., from Settings), it is visible here. The description is indexed and searchable in the catalog. Therefore, it’s recommended to add a meaningful description for all your assets.
• TileDB URI - The unique resource identifier for TileDB, based on which you can refer to the task graph. It contains the namespace and the UUID of the asset.
• UUID - The unique identifier for the task graph.
• Original URI - The location on cloud storage where the asset is stored. This property is visible only to the admin of the asset.
• Permissions - What rights the current user has on this asset.
• License - If available, under which license the asset is available. Editable through Settings, if you are the admin of the asset.
• Tags - Any tags on the asset, if available, which will be searchable in the TileDB catalog.
• Author - The user who created and registered the task graph to the catalog.

Preview

You can see a human-readable rendering of the task graph under the Preview tab.

Sharing & Activity

The Sharing screen allows you to securely share your task graph with other TileDB users, whereas the Activity screen shows you the various accesses performed on the task graph by you or any other user with whom you have shared your task graph. They are both covered in detail in the Collaborate section.

Settings

In the task graph settings, you can modify the following:

• Add a description - Note that this is indexed and, thus, searchable in the TileDB catalog.
• License - The type of license for the task graph, especially if you are making this publicly available.
• Tags - These can be used for efficient search in the catalog.
• Make task graph read-only - This is useful if you want to prevent any task graph changes by you or someone with whom you shared the task graph.
• Make public - If you wish to share the task graph with all the TileDB users. This will appear in the Marketplace tab in the left navigation menu. If you make a task graph public, you can easily change it back to private in the same manner.
• Change cloud credentials - Credentials should be provided so that TileDB can securely access the task graph on the cloud store where it is physically stored.
• Rename task graph - Read the Rename task graph subsection below.
• Delete task graph - Read the Delete task graph subsection below.

You can programmatically update some task graph settings with the following command:

Python

tiledb.cloud.asset.update_info(
    uri="tiledb://<account>/<task_graph_name>",
    description=None,  # Optional - A new description
    name=None,  # Optional - A new name for the task graph
    tags=None,  # Optional - Task graph tags that will be searchable in the catalog
    access_credentials_name=None,  # Optional - The cloud credentials that access the task graph (should already exist in your account settings)
)

To make a task graph public programmatically, run the following:

Python

tiledb.cloud.asset.share(
    "tiledb://<account>/<task_graph_name>", namespace="public", permissions="read"
)

Rename task graph

You can rename a task graph from the Settings tab. This action does not alter or copy the contents of the task graph; it just registers the asset in the catalog under a different name.

You can programmatically rename a task graph as follows:

Python

tiledb.cloud.asset.update_info(
    "`tiledb://<account>/<previous_name>`", name="<new_name>"
)

Warning

Take caution when renaming task graphs, as any URIs including the previous task graph name will no longer work.

Delete task graph

When deleting a task graph, you have two options:

• Unregister: This operation removes the task graph from the TileDB catalog, but it does not physically remove it from the object store. Since the task graph will persist on storage, you can register it again in the TileDB catalog in the future.
• Delete: This operation both unregisters and physically removes the task graph from storage. Note that this operation cannot be undone.

You can delete the task graph from the Settings tab, which will prompt you to choose among the two operations above.

You can also programmatically delete or unregister a task graph as follows:

Python

# Unregister a task graph
tiledb.cloud.asset.deregister(uri="tiledb://<account>/<task_graph_name>")

# Delete a task graph
tiledb.cloud.asset.delete(uri="tiledb://<account>/<task_graph_name>")