Task Log

TileDB Cloud logs details about all tasks executed in the system.

In TileDB Cloud, you can review any compute run by any user of the current namespace, if it is a serverless compute action (array, SQL, UDF), or if someone has launched a server or a task graph from the Task Log. You can find the Task Log by opening Monitor > Logs > Tasks.

Task Log table view

The table view has the following information:

  • The dot implies the status of this task: It may be blue (COMPLETED), red (FAILED) or orange (UNKNOWN when the user doesn’t define the type of error returned). You can hover over the dot to see the status value.
  • User - Who launched the task.
  • Type - The type of the task, on which you can filter.
  • Duration - The duration of the task to be completed.
  • Cost - The total cost of this task.
  • Start time - When this task was initiated. By manually adding the Duration to the Start time, you can get the end time.
  • Origin - This is the method from the code that launched this task.

This list has a complete view of the namespace tasks. To focus on what matters, you may:

  • Search by username or task id if you want to find more details.
  • Sort by start time or origin name.
  • Filter by Created by (Notebook, UDF, or Task Graph), Range (last 7, 30, or 90 days), Status (Running, Completed, or Failed), or Type (refer to Task types).

A screenshot of the task log table view. A screenshot of the task log table view.

Task types

You’ll find the following types of tasks in the Task Log:

  • Serverless compute tasks:
    • SQL - This is a serverless SQL task run on Cloud.
    • UDF - This is a serverless UDF task run directly, without any task graph.
    • QUERY - This is a serverless array access task. As all the code assets are wrapped as arrays, apart from access on an array, you will find this type when you also launch a notebook or a dashboard.
  • Task graph UDF tasks:
    • GENERIC UDF - This is a UDF spun off by a generic task graph.
    • BATCH UDF - This is a UDF spun off by a batch task graph.

Task Log detailed view

By selecting an entry in the Task Log, you can see even more details for a task. Apart from the information you can see in the table view, you can see in detail:

  • The exact Runtime duration, in milliseconds.
  • A breakdown of the query cost, split between Task duration cost and Data charge cost.
  • The total Memory and the number of CPUs used for this task.
  • The exact Finished time, apart from the Started time and the Duration of the task.
  • The exact code executed with this task.

Certain task types also have a field with details about the origin of the task. You can drill down to gather more information about the task:

  • Array used - A link to the array accessed to find an asset, including launching a notebook or dashboard.
  • Notebook used - A link to the notebook, with indication of the specific method used. This is the notebook that launched the task, not the action of opening the notebook itself (which is an array query task).
  • Parent Task Graph - The ID for the parent task graph.

A screenshot of the task log detailed view. A screenshot of the task log detailed view.

Note

A notebook launch has an Array used origin, while a notebook execution has a Notebook used origin.

Get tasks programmatically

You can get all the tasks programmatically by using the tiledb.cloud.fetch_tasks method in Python and the tasksGet method in Java. From that list, you can navigate to each task type and get more information.

# List all tasks
all_tasks = tiledb.cloud.fetch_tasks()
print(all_tasks)

# List only tasks on a specific array
array_tasks = tiledb.cloud.fetch_tasks(array="tiledb://TileDB-Inc/quickstart_sparse")
print(array_tasks)

# Lists tasks within a specific time period
import datetime

ninety_days_ago = datetime.datetime.utcnow() - datetime.timedelta(days=90)
datetime_tasks = tiledb.cloud.fetch_tasks(
    array="tiledb://TileDB-Inc/quickstart_sparse", start=ninety_days_ago
)
print(datetime_tasks)

# Filter tasks by status, valid statuses are RUNNING, FAILED, COMPLETED
running_tasks = tiledb.cloud.fetch_tasks(status="RUNNING")
print(running_tasks)
TasksApi apiInstance = new TasksApi(defaultClient);
String namespace = "<NAMESPACE"; // String | namespace to filter
String createdBy = null; // String | username to filter
String array = "<ARRAY_URI>"; // String | name/uri of array that is url-encoded to filter
Integer start = null; // Integer | start time for tasks to filter by
Integer end = null; // Integer | end time for tasks to filter by
Integer page = null; // Integer | pagination offset
Integer perPage = null; // Integer | pagination limit
String type = null; // String | task type, \"QUERY\", \"SQL\", \"UDF\", \"GENERIC_UDF\"
List<String> excludeType = Arrays.asList(); // List<String> | task_type to exclude matching array in results, more than one can be included
List<String> fileType = Arrays.asList(); // List<String> | match file_type of task array, more than one can be included
List<String> excludeFileType = Arrays.asList(); // List<String> | exclude file_type of task arrays, more than one can be included
String status = null; // String | Filter to only return these statuses
String search = null; // String | search string that will look at name, namespace or description fields
String orderby = null; // String | sort by which field valid values include start_time, name
try {
    ArrayTaskData result = apiInstance.tasksGet(namespace, createdBy, array, start, end, page, perPage, type, excludeType, fileType, excludeFileType, status, search, orderby);
    System.out.println(result);
} catch (ApiException e) {
    System.err.println("Exception when calling TasksApi#tasksGet");
    System.err.println("Status code: " + e.getCode());
    System.err.println("Reason: " + e.getResponseBody());
    System.err.println("Response headers: " + e.getResponseHeaders());
    e.printStackTrace();
}

For convenience, you can also see the last SQL or UDF task:

# Get last SQL task
tiledb.cloud.last_sql_task()
# Get last UDF task
tiledb.cloud.last_udf_task()

Or you can get a specific task with a given task ID (which can be found on the UI Console):

task = tiledb.cloud.task(id=task_id)