1. Accounts
  2. Individual Accounts
  3. Profile
  4. Cloud Credentials
  • Home
  • What is TileDB?
  • Get Started
  • Explore Content
  • Accounts
    • Individual Accounts
      • Apply for the Free Tier
      • Profile
        • Overview
        • Cloud Credentials
        • Storage Paths
        • REST API Tokens
        • Credits
    • Organization Admins
      • Create an Organization
      • Profile
        • Overview
        • Members
        • Cloud Credentials
        • Storage Paths
        • Billing
      • API Tokens
    • Organization Members
      • Organization Invitations
      • Profile
        • Overview
        • Members
        • Cloud Credentials
        • Storage Paths
        • Billing
      • API Tokens
  • Catalog
    • Introduction
    • Data
      • Arrays
      • Tables
      • Single-Cell (SOMA)
      • Genomics (VCF)
      • Biomedical Imaging
      • Vector Search
      • Files
    • Code
      • Notebooks
      • Dashboards
      • User-Defined Functions
      • Task Graphs
      • ML Models
    • Groups
    • Marketplace
    • Search
  • Collaborate
    • Introduction
    • Organizations
    • Access Control
      • Introduction
      • Share Assets
      • Asset Permissions
      • Public Assets
    • Logging
    • Marketplace
  • Analyze
    • Introduction
    • Slice Data
    • Multi-Region Redirection
    • Notebooks
      • Launch a Notebook
      • Usage
      • Widgets
      • Notebook Image Dependencies
    • Dashboards
      • Dashboards
      • Streamlit
    • Preview
    • User-Defined Functions
    • Task Graphs
    • Serverless SQL
    • Monitor
      • Task Log
      • Task Graph Log
  • Scale
    • Introduction
    • Task Graphs
    • API Usage
  • Structure
    • Why Structure Is Important
    • Arrays
      • Introduction
      • Quickstart
      • Foundation
        • Array Data Model
        • Key Concepts
          • Storage
            • Arrays
            • Dimensions
            • Attributes
            • Cells
            • Domain
            • Tiles
            • Data Layout
            • Compression
            • Encryption
            • Tile Filters
            • Array Schema
            • Schema Evolution
            • Fragments
            • Fragment Metadata
            • Commits
            • Indexing
            • Array Metadata
            • Datetimes
            • Groups
            • Object Stores
          • Compute
            • Writes
            • Deletions
            • Consolidation
            • Vacuuming
            • Time Traveling
            • Reads
            • Query Conditions
            • Aggregates
            • User-Defined Functions
            • Distributed Compute
            • Concurrency
            • Parallelism
        • Storage Format Spec
      • Tutorials
        • Basics
          • Basic Dense Array
          • Basic Sparse Array
          • Array Metadata
          • Compression
          • Encryption
          • Data Layout
          • Tile Filters
          • Datetimes
          • Multiple Attributes
          • Variable-Length Attributes
          • String Dimensions
          • Nullable Attributes
          • Multi-Range Reads
          • Query Conditions
          • Aggregates
          • Deletions
          • Catching Errors
          • Configuration
          • Basic S3 Example
          • Basic TileDB Cloud
          • fromDataFrame
          • Palmer Penguins
        • Advanced
          • Schema Evolution
          • Advanced Writes
            • Write at a Timestamp
            • Get Fragment Info
            • Consolidation
              • Fragments
              • Fragment List
              • Consolidation Plan
              • Commits
              • Fragment Metadata
              • Array Metadata
            • Vacuuming
              • Fragments
              • Commits
              • Fragment Metadata
              • Array Metadata
          • Advanced Reads
            • Get Fragment Info
            • Time Traveling
              • Introduction
              • Fragments
              • Array Metadata
              • Schema Evolution
          • Array Upgrade
          • Backends
            • Amazon S3
            • Azure Blob Storage
            • Google Cloud Storage
            • MinIO
            • Lustre
          • Virtual Filesystem
          • User-Defined Functions
          • Distributed Compute
          • Result Estimation
          • Incomplete Queries
        • Management
          • Array Schema
          • Groups
          • Object Management
        • Performance
          • Summary of Factors
          • Dense vs. Sparse
          • Dimensions vs. Attributes
          • Compression
          • Tiling and Data Layout
          • Tuning Writes
          • Tuning Reads
      • API Reference
    • Tables
      • Introduction
      • Quickstart
      • Foundation
        • Data Model
        • Key Concepts
          • Indexes
          • Columnar Storage
          • Compression
          • Data Manipulation
          • Optimize Tables
          • ACID
          • Serverless SQL
          • SQL Connectors
          • Dataframes
          • CSV Ingestion
      • Tutorials
        • Basics
          • Ingestion with SQL
          • CSV Ingestion
          • Basic S3 Example
          • Running Locally
        • Advanced
          • Scalable Ingestion
          • Scalable Queries
      • API Reference
    • AI & ML
      • Vector Search
        • Introduction
        • Quickstart
        • Foundation
          • Data Model
          • Key Concepts
            • Vector Search
            • Vector Databases
            • Algorithms
            • Distance Metrics
            • Updates
            • Deployment Methods
            • Architecture
            • Distributed Compute
          • Storage Format Spec
        • Tutorials
          • Basics
            • Ingestion & Querying
            • Updates
            • Deletions
            • Basic S3 Example
            • Running Locally
          • Advanced
            • Versioning
            • Time Traveling
            • Consolidation
            • Distributed Compute
            • RAG LLM
            • LLM Memory
            • File Search
            • Image Search
            • Protein Search
          • Performance
        • API Reference
      • ML Models
        • Introduction
        • Quickstart
        • Foundation
          • Basics
          • Storage
          • Cloud Execution
          • Why TileDB for Machine Learning
        • Tutorials
          • Ingestion
            • Data Ingestion
              • Dense Datasets
              • Sparse Datasets
            • ML Model Ingestion
          • Management
            • Array Schema
            • Machine Learning: Groups
            • Time Traveling
    • Life Sciences
      • Single-cell
        • Introduction
        • Quickstart
        • Foundation
          • Data Model
          • Key Concepts
            • Data Structures
            • Use of Apache Arrow
            • Join IDs
            • State Management
            • TileDB Cloud URIs
          • SOMA API Specification
        • Tutorials
          • Data Ingestion
          • Bulk Ingestion Tutorial
          • Data Access
          • Distributed Compute
          • Basic S3 Example
          • Multi-Experiment Queries
          • Appending Data to a SOMA Experiment
          • Add New Measurements
          • SQL Queries
          • Running Locally
          • Shapes in TileDB-SOMA
          • Drug Discovery App
        • Spatial
          • Introduction
          • Foundation
            • Spatial Data Model
            • Data Structures
          • Tutorials
            • Spatial Data Ingestion
            • Access Spatial Data
            • Manage Coordinate Spaces
        • API Reference
      • Population Genomics
        • Introduction
        • Quickstart
        • Foundation
          • Data Model
          • Key Concepts
            • The N+1 Problem
            • Architecture
            • Arrays
            • Ingestion
            • Reads
            • Variant Statistics
            • Annotations
            • User-Defined Functions
            • Tables and SQL
            • Distributed Compute
          • Storage Format Spec
        • Tutorials
          • Basics
            • Basic Ingestion
            • Basic Queries
            • Export to VCF
            • Add New Samples
            • Deleting Samples
            • Basic S3 Example
            • Basic TileDB Cloud
          • Advanced
            • Scalable Ingestion
            • Scalable Queries
            • Query Transforms
            • Handling Large Queries
            • Annotations
              • Finding Annotations
              • Embedded Annotations
              • External Annotations
              • Annotation VCFs
              • Ingesting Annotations
            • Variant Statistics
            • Tables and SQL
            • User-Defined Functions
            • Sample Metadata
            • Split VCF
          • Performance
        • API Reference
          • Command Line Interface
          • Python API
          • Cloud API
      • Biomedical Imaging
        • Introduction
        • Foundation
          • Data Model
          • Key Concepts
            • Arrays
            • Ingestion
            • Reads
            • User Defined Functions
          • Storage Format Spec
        • Quickstart
        • Tutorials
          • Basics
            • Ingestion
            • Read
              • OpenSlide
              • TileDB-Py
          • Advanced
            • Batched Ingestion
            • Chunked Ingestion
            • Machine Learning
              • PyTorch
            • Napari
    • Files
  • API Reference
  • Self-Hosting
    • Installation
    • Upgrades
    • Administrative Tasks
    • Image Customization
      • Customize User-Defined Function Images
      • AWS ECR Container Registry
      • Customize Jupyter Notebook Images
    • Single Sign-On
      • Configure Single Sign-On
      • OpenID Connect
      • Okta SCIM
      • Microsoft Entra
  • Glossary

On this page

  • Prerequisites
  • Access keys
  • AWS AssumeRole
    • Generate an AWS Principal and External ID
    • Create or use an existing Amazon S3 bucket
    • Add a bucket policy to the bucket
    • Create a new IAM policy to match the bucket policy
    • Create a new IAM role
    • Attach the ARN to the cloud credential
  1. Accounts
  2. Individual Accounts
  3. Profile
  4. Cloud Credentials

Cloud Credentials

accounts
remote access
administration
To create, register, and access arrays through TileDB Cloud, you need to set up access credentials.

To create, register, and access arrays through the TileDB Cloud SaaS service, you need to set up access credentials. For S3-compatible object stores, TileDB Cloud SaaS supports both IAM roles and access credential key pairs. TileDB Cloud SaaS securely stores all keys in an encrypted database and never grants your keys to any other user. TileDB Cloud SaaS uses your keys in containerized stateless workers, which are under TileDB’s full control and inaccessible by any other user’s code (e.g., SQL or UDFs).

You can add multiple AWS keys to TileDB Cloud SaaS, register different arrays with different keys, select a key to be your default key, and revoke any key at any time, all from the Cloud credentials section of your profile:

The 'Cloud credentials' section of a user's TileDB Cloud profile. The 'Cloud credentials' section of a user's TileDB Cloud profile.

Prerequisites

This page assumes you have done the following:

  • Sign up for a TileDB account.
  • Work with your IT team to set up your AWS account with S3 and IAM access. You need to be able to create S3 buckets. You also need to create IAM policies and IAM roles if you are using AWS AssumeRole.

Access keys

To add an access key, perform the following:

  1. Navigate to the Cloud credentials section of your profile. Select Add credentials or Add cloud credentials, depending on if a credential already exists.

  2. Select Access and secret key as the format. Select Next.

    The Add Credentials modal of the 'Cloud credentials' section of TileDB Cloud. The 'Access and secret key' option is highlighted. The Add Credentials modal of the 'Cloud credentials' section of TileDB Cloud. The 'Access and secret key' option is highlighted.

  3. Add a Friendly Name of your choice.

  4. Add the AWS Access Key Id and AWS Secret Access Key.

  5. Optional: Specify a Custom endpoint URL if using a cloud storage provider other than AWS.

  6. Optional: Toggle Set as default on to set the current key as your default key.

    The Add Credentials modal of the 'Cloud credentials' section of TileDB Cloud, where you can specify a friendly name, an AWS Access Key ID, an AWS secret access key, a custom endpoint, and whether this key should be the default credential. The Add Credentials modal of the 'Cloud credentials' section of TileDB Cloud, where you can specify a friendly name, an AWS Access Key ID, an AWS secret access key, a custom endpoint, and whether this key should be the default credential.

  7. Select Add credentials to save the credentials to your profile.

AWS AssumeRole

With an AWS AssumeRole policy, you can enable AWS cross-account access, so that a role in one account can access a bucket in a separate account.

When using AWS AssumeRole, the Service Token Service (STS) creates temporary keys that the deployment party (in this case, the TileDB Cloud SaaS Console) can use to access the bucket. This means you don’t need to create an AWS IAM user for every user logging in to the TileDB Cloud SaaS Console and generate key pairs. Instead, after a user authenticates to TileDB, the AssumeRole functionality grants the TileDB Cloud SaaS Console access to the bucket on behalf of a user. This allows many users in the same organization who need access to the same cloud storage bucket to reuse those credentials.

For example, consider two accounts: account A and account B. Account A signs up with access to TileDB Cloud SaaS to access one or more bucket’s in account B’s AWS account. The most common setup is to create an IAM role for TileDB Cloud SaaS to use and then allow it to access a specific bucket within an Amazon S3 bucket policy. This is done by linking an Amazon Resource Name (ARN) generated for account B to account A. Requests for access to the bucket will only be granted coming from our AWS account with our external ID.

Creating credentials that use a trusted IAM role involves the following high-level steps:

  1. Generate an AWS Principal and External ID in TileDB for use in AWS.
  2. Create a bucket, if one doesn’t exist, or use an existing bucket.
  3. Add a bucket policy to the bucket.
  4. Create a new IAM policy to match the bucket policy.
  5. Create an IAM role using the AWS Principal and External ID from TileDB and attach the IAM policy to the IAM role.
  6. Collect the ARN from the IAM role.
  7. Attach the ARN to the cloud credential in TileDB.
  8. Optionally set up KMS for the target bucket.

Generate an AWS Principal and External ID

  1. Navigate to the Cloud credentials section of your profile.

  2. Select Add cloud credentials.

  3. Select ARN role as the format. Select Next.

    The Add Credentials modal of the 'Cloud credentials' section of TileDB Cloud. The ARN Role option is highlighted. The Add Credentials modal of the 'Cloud credentials' section of TileDB Cloud. The ARN Role option is highlighted.

  4. A brief description about ARN IAM roles as they relate to TileDB Cloud SaaS will appear. Select Next.

  5. The Existing role tab contains the AWS Principal and External ID needed for your bucket policy. Make note of these.

    The 'Add credentials' modal of the 'Cloud credentials' section of TileDB Cloud, showing the AWS Principal and the External ID of the existing role. The 'Add credentials' modal of the 'Cloud credentials' section of TileDB Cloud, showing the AWS Principal and the External ID of the existing role.

  6. Select the New role tab of the modal. This JSON string will be the trust policy for the IAM role you will be assuming. Included within the object are the AWS Principal and External ID from the Existing role tab. Copy the trust policy, and save it somewhere.

    The 'Add credentials' modal of the 'Cloud credentials' section of TileDB Cloud, showing details about the new role in a JSON object. The 'Add credentials' modal of the 'Cloud credentials' section of TileDB Cloud, showing details about the new role in a JSON object.

An example JSON trust policy is below:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::{PROVIDED IN UI}:root"
      },
      "Action": "sts:AssumeRole",
      "Condition": {
        "StringEquals": {
          "sts:ExternalId": "{PROVIDED IN UI}"
        }
      }
    }
  ]
}

Create or use an existing Amazon S3 bucket

  1. Log in to the AWS Console as account B or an administrator.

  2. In the Services menu, open S3. You can also search for S3 in the search bar.

    The 'S3' section of the Amazon AWS Console. The 'S3' section of the Amazon AWS Console.

  3. Select Buckets.

  4. Identify the bucket you will use for your TileDB resources:

    • If using an existing bucket, search for the bucket name under General purpose buckets.

    • If creating a new bucket, follow these instructions (you can also refer to Amazon’s guide on creating an S3 bucket):

      1. In S3, select Create bucket. This will open the Create bucket window.

      2. Give the bucket a meaningful Bucket name. Under Bucket type, leave the General purpose bucket type selected.

        The 'Create bucket' section of S3 in the Amazon AWS Console. The 'Create bucket' section of S3 in the Amazon AWS Console.

      3. If you’re new to S3, you can leave all other settings as default. Many of these settings can be changed after the bucket is created.

      4. Select Create bucket.

Add a bucket policy to the bucket

  1. Open the S3 bucket you wish to use for the ARN.

  2. Select the Permissions tab.

  3. Locate the Bucket policy section, and select Edit.

    The 'Permissions' window of an S3 bucket in the Amazon AWS Console. The 'Permissions' window of an S3 bucket in the Amazon AWS Console.

  4. Within the Edit bucket policy window, provide a list of Statements in JSON format containing the necessary permissions you wish your TileDB account to have on your bucket:

    The 'Edit bucket policy' window within the AWS Console, showing a JSON object representing the policy to be created. The 'Edit bucket policy' window within the AWS Console, showing a JSON object representing the policy to be created.

    An example JSON bucket policy is below, which lets you list objects in your bucket and have full control over all objects in your bucket:

    {
      "Version": "2012-10-17",
      "Statement": [
        {
          "Sid": "ListObjectsInBucket",
          "Effect": "Allow",
          "Principal": {
            "AWS": "arn:aws:iam::<aws-principal>:root"
          },
          "Action": ["s3:ListBucket", "s3:ListBucketMultipartUploads"],
          "Resource": "arn:aws:s3:::<bucket-name>"
        },
        {
          "Sid": "AllObjectActions",
          "Effect": "Allow",
          "Principal": {
            "AWS": "arn:aws:iam::<aws-principal>:root"
          },
          "Action": [
            "s3:GetObject",
            "s3:PutObject",
            "s3:AbortMultipartUpload",
            "s3:ListMultipartUploadParts",
            "s3:DeleteObject"
          ],
          "Resource": "arn:aws:s3:::<bucket-name>/*"
        }
      ]
    }

    Replace <aws-principal> with the value of the AWS Principal you generated from Generate an AWS Principal and External ID, and replace <bucket-name> with the name of your bucket. This example uses tiledb-docs-example.

  5. Select Save changes to save your changes.

Create a new IAM policy to match the bucket policy

  1. From the AWS Console, open IAM. You can also search for IAM in the search bar.

  2. Under Access management, select Policies.

    The 'Policies' section of IAM within the AWS Console. The 'Policies' section of IAM within the AWS Console.

  3. Create a policy by selecting Create policy.

  4. Use either the Visual editor to manually define policies, or use the JSON editor to paste a JSON policy. You can use the same policy you defined on your bucket if it fits your use case. Just make sure you remove the Principal key.

    The 'Create policy' window within IAM in the AWS Console, showing a JSON object representing the policy to be created. The 'Create policy' window within IAM in the AWS Console, showing a JSON object representing the policy to be created.

  5. Supply a meaningful Policy name. Optionally add a Description and Tags**.

    The 'Review and create' window within the 'Create policy' workflow in IAM in the AWS Console. The 'Review and create' window within the 'Create policy' workflow in IAM in the AWS Console.

  6. Select Create policy.

Create a new IAM role

  1. Under Access management, select Roles.

  2. Select Create role.

  3. Under Trusted entity type, select Custom trust policy.

  4. In the Custom trust policy section, paste the trust policy JSON generated from Generate an AWS Principal and External ID. Select Next.

    The 'Select trusted entity' window within the 'Create role' workflow in IAM in the AWS Console. The 'Review and create' window within the 'Create role' workflow in IAM in the AWS Console.

  5. In the Add permissions window, search for and attach the IAM policy you created in the previous section.

    The 'Add permissions' window within the 'Create role' workflow in IAM in the AWS Console. The 'Add permissions' window within the 'Create role' workflow in IAM in the AWS Console.

  6. In the Name, review, and create window, add a meaningful Role name and an optional Description.

    The 'Name, review, and create' window within the 'Create role' workflow in IAM in the AWS Console. The 'Name, review, and create' window within the 'Create role' workflow in IAM in the AWS Console.

  7. Select Create role.

  8. Reopen the role you created and copy the ARN.

    The default screen after opening an IAM role in the AWS Console. The default screen after opening an IAM role in the AWS Console.

Attach the ARN to the cloud credential

  1. Return to Cloud credentials in the TileDB Cloud Console.

  2. Add a Friendly Name of your choice.

  3. Add the ARN from earlier to the Amazon Resource Name (ARN) section of the modal.

  4. Optionally specify a Custom endpoint

  5. Optionally specify this credential to Allow to run tasks and code or Set as default.

    Allow to run tasks and code will allow TileDB to inject the credential into a task graph environment. This is only applicable to ARN-type roles. Set as default just makes this credential the default credential used for the given namespace.

    The 'Add credentials' modal of the 'Cloud credentials' section of TileDB Cloud, showing text boxes to set the Friendly Name and the Amazon Resource Name (ARN). The 'Add credentials' modal of the 'Cloud credentials' section of TileDB Cloud, showing text boxes to set the Friendly Name and the Amazon Resource Name (ARN).

  6. Select Test connection to test the connection. Once the test connection succeeds, you may Add the new credential to your TileDB Cloud SaaS profile.

    The 'Add credentials' modal of the 'Cloud credentials' section of TileDB Cloud, showing text boxes to set the Friendly Name and the Amazon Resource Name (ARN). The 'Add credentials' modal of the 'Cloud credentials' section of TileDB Cloud, showing text boxes to set the Friendly Name and the Amazon Resource Name (ARN).

Note

After saving and re-editing the cloud credential, the ARN display will be truncated to the first 20 characters for security purposes.

Overview
Storage Paths