Spatial Data Ingestion

Learn how to ingest 10X Visium Space Ranger data with TileDB-SOMA.

TileDB-SOMA’s support for spatial omics data includes the ability to convert output files from 10X’s Space Ranger pipeline to a SOMA Experiment.

Tip

Learn more about the process of converting datasets into TileDB-SOMA’s storage format in the data ingestion tutorial.

Setup

This tutorial relies on tiledbsoma and a few other packages.

Python

import os
import tempfile
from pathlib import Path

import tiledb.cloud
import tiledbsoma
import tiledbsoma.io.spatial
from scanpy.datasets._datasets import (
    _download_visium_dataset as download_visium_dataset,
)

tiledbsoma.show_package_versions()

Define a few variables to use throughout the tutorial. The DATASET_NAME and SPACERANGER_VERSION variables identify the 10X dataset to downloaded.

Python

# 10X Example dataset
EXPERIMENT_NAME = "CytAssist_FFPE_Mouse_Brain_Rep2"
SPACERANGER_VERSION = "2.0.0"

# SOMA experiment parameters
MEASUREMENT_NAME = "RNA"  # Dataset modality name
SCENE_NAME = "scene0"  # Name of scene where spatial data is stored

Prerequisites

While you can run this tutorial locally, note that this tutorial relies on remote resources to run correctly.

You must create a REST API token and create an environment variable named $TILEDB_REST_TOKEN set to the value of your generated token.

However, this is not necessary when running from inside a TileDB workspace where the API token is automatically generated and configured for you.

To proceed with this tutorial, you will need to update the following variables to correspond to your TileDB namespace and destination S3 bucket:

Python

tiledb_namespace = os.environ["TILEDB_ACCOUNT"]
s3_bucket = os.environ["S3_BUCKET"]

Dataset

The dataset you will use comes from the coronal section of a formalin-fixed, paraffin-embedded (FFPE) mouse brain, generated by 10X Genomics while using their Visium CytAssist Spatial Gene Expression platform. Space Ranger, 10X’s software for Visium data, processed the raw data. To view the data, visit 10X Genomics.

To create a spatial SOMA experiment you need to have a directory with Space Ranger output files from a 10X Visium experiment.

Download Space Ranger output files for the specified dataset by using the handy download_visium_dataset() function from scanpy.

Python

dataset_dir = download_visium_dataset(
    sample_id=EXPERIMENT_NAME,
    spaceranger_version=SPACERANGER_VERSION,
    base_dir=Path(tempfile.gettempdir()),
)

Take a look at the structure of the downloaded dataset.

Python

!tree {dataset_dir}

TileDB-SOMA will ingest the following information from the Space Ranger output files:

• The filtered gene expression matrix,filtered_feature_bc_matrix.h5, stores the expression matrix where rows are genes and columns are barcodes corresponding to spatial spots.
• The tissue positions list, tissue_positions_list.csv, stores the xy-coordinates and spot status (tissue or not).
• The scale factors JSON, scalefactors_json.json, stores scale factors for aligning the image at different resolutions.
• The tissue images tissue_lowres_image.png and tissue_hires_image.png.

Ingest

Construct the URI of where to create the new SOMA experiment.

Python

experiment_uri = f"tiledb://{tiledb_namespace}/{s3_bucket}/{EXPERIMENT_NAME}"
experiment_uri

You can use a local filepath or an S3 URI. In this case, use a TileDB URI, which will direct SOMA to create the experiment in the specified S3 bucket and register it in TileDB’s data catalog under the specified user namespace.

The ingestion function, from_visium(), requires the following parameters:

• experiment_uri: URI (or local file path) for the new experiment.
• input_path: Local path to the directory with the Space Ranger output files.
• measurement_name: Name of the SOMA Measurement (in this case, it’s “RNA”).
• scene_name: Name of the SOMA Scene collection that will store all the spatial data components.

The image_channel_first parameter controls the ordering of the dimensions of the tissue images. In the default setting, the image channel is the first dimension (that is, "CYX"), but setting image_channel_first=False will change the order to "YXC". Note that different image and machine learning libraries use different conventions for image dimension order. For example, matplotlib expects channel to be the last dimension, whereas pytorch expects the opposite order.

Calling the from_visium() function will create a new SOMA experiment at the specified URI, populated with the data from the Space Ranger output files.

Python

tiledbsoma.io.spatial.from_visium(
    experiment_uri=experiment_uri,
    input_path=dataset_dir,
    measurement_name=MEASUREMENT_NAME,
    scene_name=SCENE_NAME,
    image_channel_first=False,
)

Once the ingestion completes successfully, you can open the new SOMA experiment to explore the data.

Python

with tiledbsoma.Experiment.open(experiment_uri) as exp:
    print(exp)

Clean-up

Now that you have successfully ingested the Visium dataset into a SOMA experiment, you can clean it up by deleting it from S3 and deregistering it from your TileDB workspace.

Python

tiledb.cloud.asset.delete(experiment_uri, recursive=True)