Array Schema

Just like TileDB arrays, you can inspect various aspects of a ML model schema, including its members, domain, attributes, and filters.

This guide shows how to inspect the array schema, domain, dimensions, attributes, and filters of Machine Learning (ML) asset.

Note

The following tutorial assumes that you have successfully completed the tutorials for ingesting ML datasets and models in TileDB.

Inspect the array schema

You can get the schema of an existing array as follows:

# load the schema directly
# optionally pass `key=...` for encrypted arrays
schema = tiledb.ArraySchema.load(array_name)

# or access the schema from an open array object
A = tiledb.Array(array_name)
schema = A.schema

You can also inspect the different array schema members:

# ... get array schema

# Check if array is sparse
schema.sparse

# Get tile capacity
schema.capacity

# Cell and tile order return 'row-major', 'col-major', or 'global'
# Get tile order
schema.tile_order

# Get cell order
schema.cell_order

# Get coordinates filter list
coords_filters = schema.coords_filters

# Get offsets filter list
offsets_filters = schema.offsets_filters

# Get the array domain
domain = schema.domain

# Get number of attributes
attr_num = schema.nattr

# Get attribute by index (0 <= idx < attr_num)
attr = schema.attr(0)

# Check if the named attribute exists
schema.has_attr("features")

# Get attribute by name
attr = schema.attr("features")

None- Array type: dense
- Cell order: row-major
- Tile order: row-major
- Capacity: 10000
- Allows duplicates: false
- Coordinates filters: 1
  > ZSTD: COMPRESSION_LEVEL=-1
- Offsets filters: 1
  > ZSTD: COMPRESSION_LEVEL=-1
- Validity filters: 1
  > RLE: COMPRESSION_LEVEL=-1

### Dimension ###
- Name: dim_0
- Type: INT32
- Cell val num: 1
- Domain: [0, 59999]
- Tile extent: 64
- Filters: 0

### Dimension ###
- Name: dim_1
- Type: INT32
- Cell val num: 1
- Domain: [0, 27]
- Tile extent: 28
- Filters: 0

### Dimension ###
- Name: dim_2
- Type: INT32
- Cell val num: 1
- Domain: [0, 27]
- Tile extent: 28
- Filters: 0

### Attribute ###
- Name: features
- Type: UINT8
- Nullable: false
- Cell val num: 1
- Filters: 0
- Fill value: �
 

Inspect the domain

# ... get array schema
# ... get domain from schema

# Get the domain data type (i.e., the datatype of all dimensions)
# note: types are automatically converted to NumPy dtypes
domain_dtype = domain.dtype

# Get number of dimensions
dim_num = domain.ndim

# Get dimension by index (0 <= idx < dim_num)
dim = domain.dim(0)

# Get dimension by name
dim = domain.dim("dim_1")

# Check if domain has named dimension
domain.has_dim("dim_1")

# Print the domain in ASCII format
domain.dump()

Inspect array dimensions

# ... get array schema
# ... get domain
# ... get dimension by index

# Get dimension name
dim_name = dims[0].name

# Get dimension datatype
dim_dtype = dims[0].dtype

# Get dimension domain
dim_domain = dims[0].domain

# Get tile extent
tile_extent = dims[0].tile

# Get dimension filter list
filter_list = dims[0].filters

Inspect attributes

# ... create context
# ... get array schema
# ... get attribute by index or name

# Get attribute name
attr_name = attr.name

# Get attribute datatype
attr_dtype = attr.dtype

# Get filter list
filter_list = attr.filters

# Check if attribute is variable-length
var_length = attr.isvar

# Get number of values per cell
# note: for variable-length attributes, ncells == typemax(np.uint32)
num = attr.ncells

# Get datatype size (bytes) for this attribute
cell_size = attr.dtype.itemsize

Inspect filters

# ... get filter list

# Get number of filters
num_filters = len(filter_list)

print(len(filter_list))
# Get the maximum tile chunk size
max_chunk_size = filter_list.chunksize

# Get a filter by index (0 <= idx < num_filters)
filter = filter_list[0]

# Get filter type
filter.__class__

# Get filter option (depends on the filter: `filter.{option_name}`)
level = filter.level