Groups

Learn how to organize your TileDB arrays in a virtual hierarchy, which is especially useful when working with S3-based object storage.

How to run this tutorial

You can run this tutorial in two ways:

1. Locally on your machine.
2. On TileDB Cloud.

However, since TileDB Cloud has a free tier, we strongly recommend that you sign up and run everything there, as that requires no installations or deployment.

This tutorial shows all the functionality of groups. For more information, visit the Key Concepts: Groups section.

First, import the necessary libraries, set the array and group URIs (that is, their paths, which in this tutorial will be on local storage), and delete any previously created arrays and groups with the same name.

Python

# Import necessary libraries
import os.path
import shutil

import numpy as np
import tiledb

# Set array URI and name
array_uri = os.path.expanduser("~/groups_array")
array_name = "my_array"

# Set group paths and names
grp_root_uri = os.path.expanduser("~/groups_grp_root")
grp1_uri = os.path.expanduser("~/groups_grp1")
grp2_uri = os.path.expanduser("~/groups_grp2")
grp3_uri = os.path.expanduser("~/groups_grp3")
grp1_name = "grp1"
grp2_name = "grp2"
grp3_name = "grp3"

# Delete all paths if they already exist
if os.path.exists(array_uri):
    shutil.rmtree(array_uri)
if os.path.exists(grp_root_uri):
    shutil.rmtree(grp_root_uri)
if os.path.exists(grp1_uri):
    shutil.rmtree(grp1_uri)
if os.path.exists(grp2_uri):
    shutil.rmtree(grp2_uri)
if os.path.exists(grp3_uri):
    shutil.rmtree(grp3_uri)

R

# Import necessary libraries
library(tiledb)

# Set array URI and name
array_uri <- path.expand("~/groups_array_r")
array_name <- "my_array"

# Set group paths and names
grp_root_uri <- path.expand("~/groups_grp_root_r")
grp1_uri <- path.expand("~/groups_grp1_r")
grp2_uri <- path.expand("~/groups_grp2_r")
grp3_uri <- path.expand("~/groups_grp3_r")
grp1_name <- "grp1"
grp2_name <- "grp2"
grp3_name <- "grp3"

# Delete array if it already exists
if (file.exists(array_uri)) unlink(array_uri, recursive = TRUE)
if (file.exists(grp_root_uri)) unlink(grp_root_uri, recursive = TRUE)
if (file.exists(grp1_uri)) unlink(grp1_uri, recursive = TRUE)
if (file.exists(grp2_uri)) unlink(grp2_uri, recursive = TRUE)
if (file.exists(grp3_uri)) unlink(grp3_uri, recursive = TRUE)

Create groups

You can create a group as follows.

Python

# This creates an empty group
tiledb.Group.create(grp_root_uri)

R

# This creates an empty group
tiledb_group_create(grp_root_uri)

The group has the following directory structure on storage.

/Users/stavrospapadopoulos/groups_grp_root
├── __group
├── __meta
└── __tiledb_group.tdb

3 directories, 1 file

Add group members

First, add an array to the group. The following code sample creates a dense array, but what follows applies to any array.

Python

# Create the two dimensions
d1 = tiledb.Dim(name="d1", domain=(1, 4), tile=2, dtype=np.int32)
d2 = tiledb.Dim(name="d2", domain=(1, 4), tile=2, dtype=np.int32)

# Create a domain using the two dimensions
dom = tiledb.Domain(d1, d2)
# Order of the dimensions matters when slicing subarrays.
# Remember to give priority to more selective dimensions to
# maximize the pruning power during slicing.

# Create an attribute
a = tiledb.Attr(name="a", dtype=np.int32)

# Create the array schema, setting `sparse=False` to indicate a dense array
sch = tiledb.ArraySchema(domain=dom, sparse=False, attrs=[a])

# Create the array on disk (it will initially be empty)
tiledb.Array.create(array_uri, sch)

R

# Create the two dimensions
d1 <- tiledb_dim("d1", c(1L, 4L), 2L, "INT32")
d2 <- tiledb_dim("d2", c(1L, 4L), 2L, "INT32")

# Create a domain using the two dimensions
dom <- tiledb_domain(dims = c(d1, d2))

# Create an attribute
a <- tiledb_attr("a", type = "INT32")

# Create the array schema, setting `sparse = FALSE` to indicate a dense array
sch <- tiledb_array_schema(dom, a, sparse = FALSE)

# Create the array on disk (it will initially be empty)
arr <- tiledb_array_create(array_uri, sch)

Next, add the array to the group.

Python

# Open the group for writing
grp_root = tiledb.Group(grp_root_uri, "w")

# Add an array to the group and give it some name.
# NOTE: The array name may be different from the physical path.
# NOTE: The physical path of the array does not need to be
# nested inside the physical path of the group.
grp_root.add(array_uri, array_name)

# Remember to close the group when done
grp_root.close()

R

# Open the group for writing
grp_root <- tiledb_group(grp_root_uri, type = "WRITE")

# Add an array to the group and give it some name.
# NOTE: The array name may be different from the physical path.
# NOTE: The physical path of the array does not need to be
# nested inside the physical path of the group.
# The `relative` flag dictates whether the `array_uri`
# is relative to the `grp_root_uri`.
tiledb_group_add_member(
  grp = grp_root,
  uri = array_uri,
  relative = FALSE,
  name = array_name
)

Note that you don’t need to nest the physical path of the array under the physical path of the group. The group acts as a virtual folder.

/Users/stavrospapadopoulos/groups_grp_root
├── __group
│   └── __1715199165273_1715199165273_0a368c13be4bcd88c9bf9b3b82d3e9de_2
├── __meta
└── __tiledb_group.tdb

3 directories, 2 files

Subsequently, add two subgroups.

Python

with tiledb.Group(grp_root_uri, "w") as grp:
    tiledb.group_create(grp1_uri)
    grp.add(grp1_uri, grp1_name)
    tiledb.group_create(grp2_uri)
    grp.add(grp2_uri, grp2_name)

R

tiledb_group_create(grp1_uri)
tiledb_group_add_member(grp_root, grp1_uri, FALSE, grp1_name)

tiledb_group_create(grp2_uri)
tiledb_group_add_member(grp_root, grp2_uri, FALSE, grp2_name)

# Remember to close the group when done. Changes are not committed
# to disk until the group is closed.
grp_root <- tiledb_group_close(grp_root)

This creates some unique files in the group folder, but again the subgroups don’t need to be physically nested under the root group.

/Users/stavrospapadopoulos/groups_grp_root
├── __group
│   ├── __1715199165273_1715199165273_0a368c13be4bcd88c9bf9b3b82d3e9de_2
│   └── __1715199165406_1715199165406_2badabf703dba706e3b87836a5828366_2
├── __meta
└── __tiledb_group.tdb

3 directories, 3 files

Check members

You can check the contents of a group as follows.

Python

# Open the group in read mode
grp_root = tiledb.Group(grp_root_uri, "r")

# Show the directory structure of the group
print(grp_root)

# Iterate through the group members
for i in range(0, len(grp_root)):
    print(f"URI: {grp_root[i].uri}, Type: {grp_root[i].type}, Name: {grp_root[i].name}")

# Remember to close the group when done
grp_root.close()

groups_grp_root GROUP
|-- grp1 GROUP
|-- grp2 GROUP
|-- my_array ARRAY

URI: file:///Users/stavrospapadopoulos/groups_grp1, Type: <class 'tiledb.group.Group'>, Name: grp1
URI: file:///Users/stavrospapadopoulos/groups_grp2, Type: <class 'tiledb.group.Group'>, Name: grp2
URI: file:///Users/stavrospapadopoulos/groups_array, Type: <class 'tiledb.libtiledb.Array'>, Name: my_array

R

# Open the group in read mode
grp_root <- tiledb_group_open(grp_root, "READ")

# Show the directory structure of the group
print(grp_root)

# You can also do the same with tiledb_group_member_dump
# NOTE: tiledb_group_member_dump can search recursively,
# but recursive listings on remote object stores may be
# an expensive or slow operation
writeLines(tiledb_group_member_dump(grp_root, recursive = TRUE))

# Iterate through the group members
for (i in 0:(tiledb_group_member_count(grp_root) - 1)) {
  print(
    paste0(
      "URI: ", tiledb_group_member(grp_root, i)[2],
      ", Type: ", tiledb_group_member(grp_root, i)[1],
      ", Name: ", tiledb_group_member(grp_root, i)[3]
    )
  )
}

# Remember to close the group when done
grp_root <- tiledb_group_close(grp_root)

groups_grp_root_r GROUP
|-- my_array ARRAY
|-- grp1 GROUP
|-- grp2 GROUP
groups_grp_root_r GROUP
|-- my_array ARRAY
|-- grp1 GROUP
|-- grp2 GROUP

[1] "URI: file:///Users/nickv/groups_array_r, Type: ARRAY, Name: my_array"
[1] "URI: file:///Users/nickv/groups_grp1_r, Type: GROUP, Name: grp1"
[1] "URI: file:///Users/nickv/groups_grp2_r, Type: GROUP, Name: grp2"

Move and delete members

You can move group members by deleting them and then adding them in a new path.

Python

# Move the array into a subgroup.
# NOTE: The process is the same for groups.
with tiledb.Group(grp_root_uri, "w") as grp_root:
    # The following does not physically delete the array
    grp_root.remove(array_uri)
with tiledb.Group(grp1_uri, "w") as grp1:
    grp1.add(array_uri, array_name)

# Print the new directory structure of the root group
with tiledb.Group(grp_root_uri, "r") as grp_root:
    print(grp_root)
    for i in range(0, len(grp_root)):
        print(
            f"URI: {grp_root[i].uri}, Type: {grp_root[i].type}, Name: {grp_root[i].name}"
        )

groups_grp_root GROUP
|-- grp1 GROUP
|------ my_array ARRAY
|-- grp2 GROUP

URI: file:///Users/stavrospapadopoulos/groups_grp1, Type: <class 'tiledb.group.Group'>, Name: grp1
URI: file:///Users/stavrospapadopoulos/groups_grp2, Type: <class 'tiledb.group.Group'>, Name: grp2

R

# Open the root group in write mode
grp_root <- tiledb_group_open(grp_root, "WRITE")

# Move the array into a subgroup.
# NOTE: The process is the same for groups.
# The following does not physically delete the array
tiledb_group_remove_member(grp_root, array_uri)

# Remember to close the group when done
grp_root <- tiledb_group_close(grp_root)

# Open the grp1 group in write mode
grp1 <- tiledb_group(grp1_uri, "WRITE")

# Add the array to the new group
tiledb_group_add_member(grp1, array_uri, FALSE, array_name)

# Close grp1 to commit the change
grp1 <- tiledb_group_close(grp1)

# Reopen grp_root for reading
grp_root <- tiledb_group_open(grp_root, "READ")

# Print the new directory structure of the group
writeLines(tiledb_group_member_dump(grp_root, recursive = TRUE))

# Iterate through the group members
for (i in 0:(tiledb_group_member_count(grp_root) - 1)) {
  print(
    paste0(
      "URI: ", tiledb_group_member(grp_root, i)[2],
      ", Type: ", tiledb_group_member(grp_root, i)[1],
      ", Name: ", tiledb_group_member(grp_root, i)[3]
    )
  )
}

# Remember to close the group when done
grp_root <- tiledb_group_close(grp_root)

groups_grp_root_r GROUP
|-- grp1 GROUP
|------ my_array ARRAY
|-- grp2 GROUP

[1] "URI: file:///Users/nickv/groups_grp1_r, Type: GROUP, Name: grp1"
[1] "URI: file:///Users/nickv/groups_grp2_r, Type: GROUP, Name: grp2"

You can delete group members as follows.

Python

# Delete a group (the process is the same for arrays as well).
with tiledb.Group(grp_root_uri, "w") as grp_root:
    grp_root.remove(grp1_name)

# Need to physically delete the files
shutil.rmtree(grp1_uri)

# The subgroup, along with the array therein, got removed
# from the group.
with tiledb.Group(grp_root_uri, "r") as grp_root:
    print(grp_root)
    for i in range(0, len(grp_root)):
        print(
            f"URI: {grp_root[i].uri}, Type: {grp_root[i].type}, Name: {grp_root[i].name}"
        )

groups_grp_root GROUP
|-- grp2 GROUP

URI: file:///Users/stavrospapadopoulos/groups_grp2, Type: <class 'tiledb.group.Group'>, Name: grp2

R

# Open the root group in write mode
grp_root <- tiledb_group_open(grp_root, "WRITE")

# Delete a group (the process is the same for arrays as well).
tiledb_group_remove_member(grp_root, grp1_uri)

# Need to remove the files as well
unlink(grp1_uri, recursive = TRUE)

# Close the group and reopen in read mode
grp_root <- tiledb_group_close(grp_root)
grp_root <- tiledb_group_open(grp_root, "READ")

# The subgroup, along with the array therein, got removed
# from the group.
writeLines(tiledb_group_member_dump(grp_root, recursive = TRUE))

# Iterate through the group members
for (i in 0:(tiledb_group_member_count(grp_root) - 1)) {
  print(
    paste0(
      "URI: ", tiledb_group_member(grp_root, i)[2],
      ", Type: ", tiledb_group_member(grp_root, i)[1],
      ", Name: ", tiledb_group_member(grp_root, i)[3]
    )
  )
}

# Remember to close the group when done
grp_root <- tiledb_group_close(grp_root)

groups_grp_root_r GROUP
|-- grp2 GROUP

[1] "URI: file:///Users/nickv/groups_grp2_r, Type: GROUP, Name: grp2"

The group internal structure after these modifications is as follows.

/Users/stavrospapadopoulos/groups_grp_root
├── __group
│   ├── __1715199165273_1715199165273_0a368c13be4bcd88c9bf9b3b82d3e9de_2
│   ├── __1715199165406_1715199165406_2badabf703dba706e3b87836a5828366_2
│   ├── __1715199165553_1715199165553_1f20eb90c84468eef3489d6aea9439ef_2
│   └── __1715199165559_1715199165559_52f1c8aa804df35017186d5ed391969a_2
├── __meta
└── __tiledb_group.tdb

3 directories, 5 files

Group metadata

Handling group metadata is similar to Array Metadata.

Python

# Write some metadata
with tiledb.Group(grp_root_uri, "w") as grp_root:
    grp_root.meta["ints"] = [1, 2, 3]
    grp_root.meta["str"] = "string_metadata"

# Show group metadata
with tiledb.Group(grp_root_uri, "r") as grp_root:
    print(grp_root.meta["ints"])
    print(dict(grp_root.meta._iter(keys_only=False)))

# Delete group metadata
with tiledb.Group(grp_root_uri, "w") as grp_root:
    del grp_root.meta["ints"]

# Show group metadata again
with tiledb.Group(grp_root_uri, "r") as grp_root:
    print(dict(grp_root.meta._iter(keys_only=False)))

(1, 2, 3)
{'ints': (1, 2, 3), 'str': 'string_metadata'}
{'str': 'string_metadata'}

R

# Open the root group in write mode
grp_root <- tiledb_group_open(grp_root, "WRITE")

# Write some metadata
success <- tiledb_group_put_metadata(grp_root, "ints", c(1L, 2L, 3L))
success <- tiledb_group_put_metadata(grp_root, "str", "string_metadata")

# Remember to close the group when done
grp_root <- tiledb_group_close(grp_root)

# Open the root group in write mode
grp_root <- tiledb_group_open(grp_root, "READ")

# Get all metadata
print(tiledb_group_get_all_metadata(grp_root))

# Get metadata from key
print(tiledb_group_get_metadata(grp_root, "ints"))

# Get metadata from index
# 0 <= idx < length(tiledb_group_get_all_metadata(grp_root))
print(tiledb_group_get_metadata_from_index(grp_root, 1))

# Close the group and reopen for writing
grp_root <- tiledb_group_close(grp_root)
grp_root <- tiledb_group_open(grp_root, "WRITE")

# Delete group metadata
print("Deleting metadata with key 'ints'...")
tiledb_group_delete_metadata(grp_root, "ints")

# Close the group and reopen for reading
grp_root <- tiledb_group_close(grp_root)
grp_root <- tiledb_group_open(grp_root, "READ")

# Get all metadata
print(tiledb_group_get_all_metadata(grp_root))

# Remember to close the group
grp_root <- tiledb_group_close(grp_root)

$ints
[1] 1 2 3
attr(,"key")
[1] "ints"

$str
[1] "string_metadata"
attr(,"key")
[1] "str"

[1] 1 2 3
attr(,"key")
[1] "ints"
[1] "string_metadata"
attr(,"key")
[1] "str"
[1] "Deleting metadata with key 'ints'..."
$str
[1] "string_metadata"
attr(,"key")
[1] "str"

Observe the files written inside the group folder after the group metadata changes.

/Users/stavrospapadopoulos/groups_grp_root
├── __group
│   ├── __1715199165273_1715199165273_0a368c13be4bcd88c9bf9b3b82d3e9de_2
│   ├── __1715199165406_1715199165406_2badabf703dba706e3b87836a5828366_2
│   ├── __1715199165553_1715199165553_1f20eb90c84468eef3489d6aea9439ef_2
│   └── __1715199165559_1715199165559_52f1c8aa804df35017186d5ed391969a_2
├── __meta
│   ├── __1715199165713_1715199165713_49376877310549716f58c3fd19c763f2
│   └── __1715199165715_1715199165715_6e85b548b8fb2448de1a74a5b17e6780
└── __tiledb_group.tdb

3 directories, 7 files

Write at a timestamp

Groups support the same writing at a timestamp functionality as the Arrays Write at a Timestamp functionality.

Python

# Create another group
tiledb.Group.create(grp3_uri)

# You can write at an arbitrary timestamp by
# passing an appropriate config when opening the group.
with tiledb.Group(
    grp_root_uri, "w", config=tiledb.Config({"sm.group.timestamp_end": 1})
) as grp_root:
    grp_root.add(grp3_uri, grp3_name)

# Same is true for writing metadata
with tiledb.Group(
    grp_root_uri, "w", config=tiledb.Config({"sm.group.timestamp_end": 2})
) as grp_root:
    grp_root.meta["ts"] = "2"

R

# Create another group
tiledb_group_create(grp3_uri)

# Define config objects with different timestamps
ts1_cfg <- tiledb_config(c(sm.group.timestamp_end = "1"))

ts2_cfg <- tiledb_config(c(sm.group.timestamp_end = "2"))

# You can write at an arbitrary timestamp by
# passing an appropriate config when opening the group.
grp_root_ts1 <- tiledb_group(grp_root_uri, "WRITE", cfg = ts1_cfg)
tiledb_group_add_member(grp_root_ts1, grp3_uri, FALSE, grp3_name)
grp_root_ts1 <- tiledb_group_close(grp_root_ts1)

# The same is true for writing metadata
grp_root_ts2 <- tiledb_group(grp_root_uri, "WRITE", cfg = ts2_cfg)
success <- tiledb_group_put_metadata(grp_root_ts2, "ts", 2)
grp_root_ts2 <- tiledb_group_close(grp_root_ts2)

Now the written timestamps appear as prefixes of some files in the folder physical structure.

/Users/stavrospapadopoulos/groups_grp_root
├── __group
│   ├── __1715199165273_1715199165273_0a368c13be4bcd88c9bf9b3b82d3e9de_2
│   ├── __1715199165406_1715199165406_2badabf703dba706e3b87836a5828366_2
│   ├── __1715199165553_1715199165553_1f20eb90c84468eef3489d6aea9439ef_2
│   ├── __1715199165559_1715199165559_52f1c8aa804df35017186d5ed391969a_2
│   └── __1_1_3edb27acc1cb01441ef5e33f69c1e83d_2
├── __meta
│   ├── __1715199165713_1715199165713_49376877310549716f58c3fd19c763f2
│   ├── __1715199165715_1715199165715_6e85b548b8fb2448de1a74a5b17e6780
│   └── __2_2_705aeb4ecb9e35d5c21f78ef5dcc07c4
└── __tiledb_group.tdb

3 directories, 9 files

This is also particularly useful for showing versioning and time traveling in the next subsection.

Time traveling

Time traveling works in a similar manner to the Arrays Time Traveling functionality.

Python

# In timestamp range [0,0], the group should appear as empty,
# as nothing was written in that range.
with tiledb.Group(
    grp_root_uri, "r", config=tiledb.Config({"sm.group.timestamp_end": 0})
) as grp_root:
    print("In timestamp range [0,0]:")
    print(dict(grp_root.meta._iter(keys_only=False)))
    print(grp_root)

# In timestamp range [0,1], the group contains one subgroup,
# but no metadata (as those were written at timestamp 2).
with tiledb.Group(
    grp_root_uri, "r", config=tiledb.Config({"sm.group.timestamp_end": 1})
) as grp_root:
    print("In timestamp range [0,1]:")
    print(dict(grp_root.meta._iter(keys_only=False)))
    print(grp_root)

# In timestamp range [0,2], the group contains one subgroup,
# and also the metadata you wrote at timestamp 2.
with tiledb.Group(
    grp_root_uri, "r", config=tiledb.Config({"sm.group.timestamp_end": 2})
) as grp_root:
    print("In timestamp range [0,2]:")
    print(dict(grp_root.meta._iter(keys_only=False)))
    print(grp_root)

In timestamp range [0,0]:
{}
groups_grp_root GROUP

In timestamp range [0,1]:
{}
groups_grp_root GROUP
|-- grp3 GROUP

In timestamp range [0,2]:
{'ts': '2'}
groups_grp_root GROUP
|-- grp3 GROUP

R

# Define ts=0 cfg object
ts0_cfg <- tiledb_config(
  c("sm.group.timestamp_end" = "0")
)

# In timestamp range [0,0], the group should appear as empty,
# as nothing was written in that range.
grp_root_ts0 <- tiledb_group(grp_root_uri, "READ", cfg = ts0_cfg)
cat("In timestamp range [0,0]:\n")
print(tiledb_group_get_all_metadata(grp_root_ts0))
writeLines(tiledb_group_member_dump(grp_root_ts0, recursive = TRUE))
grp_root_ts0 <- tiledb_group_close(grp_root_ts0)

# In timestamp range [0,1], the group contains one subgroup,
# but no metadata (as those were written at timestamp 2).
grp_root_ts1 <- tiledb_group_open(grp_root_ts1, "READ")
cat("In timestamp range [0,1]:\n")
print(tiledb_group_get_all_metadata(grp_root_ts1))
writeLines(tiledb_group_member_dump(grp_root_ts1, recursive = TRUE))
grp_root_ts1 <- tiledb_group_close(grp_root_ts1)

# In timestamp range [0,2], the group contains one subgroup,
# and also the metadata you wrote at timestamp 2.
grp_root_ts2 <- tiledb_group_open(grp_root_ts2, "READ")
cat("In timestamp range [0,2]:\n")
print(tiledb_group_get_all_metadata(grp_root_ts2))
writeLines(tiledb_group_member_dump(grp_root_ts2, recursive = TRUE))
grp_root_ts2 <- tiledb_group_close(grp_root_ts2)

In timestamp range [0,0]:
list()
groups_grp_root_r GROUP

In timestamp range [0,1]:
list()
groups_grp_root_r GROUP
|-- grp3 GROUP

In timestamp range [0,2]:
$ts
[1] 2
attr(,"key")
[1] "ts"

groups_grp_root_r GROUP
|-- grp3 GROUP

Consolidation and vacuuming

Warning

Consolidation and vacuuming of information about group members isn’t supported yet. Contact us if you would like us to accelerate this feature. However, TileDB supports consolidation and vacuuming of group metadata.

You can consolidate group metadata like you would consolidate array metadata as follows.

Python

# You can consolidate group metadata as follows
tiledb.Group.consolidate_metadata(grp_root_uri)

R

# Consolidating group metadata is not currently possible
# with the R API. Please use another API.

Observe that TileDB created a new file in the meta subfolder.

/Users/stavrospapadopoulos/groups_grp_root
├── __group
│   ├── __1715199165273_1715199165273_0a368c13be4bcd88c9bf9b3b82d3e9de_2
│   ├── __1715199165406_1715199165406_2badabf703dba706e3b87836a5828366_2
│   ├── __1715199165553_1715199165553_1f20eb90c84468eef3489d6aea9439ef_2
│   ├── __1715199165559_1715199165559_52f1c8aa804df35017186d5ed391969a_2
│   └── __1_1_3edb27acc1cb01441ef5e33f69c1e83d_2
├── __meta
│   ├── __1715199165713_1715199165713_49376877310549716f58c3fd19c763f2
│   ├── __1715199165715_1715199165715_6e85b548b8fb2448de1a74a5b17e6780
│   ├── __2_1715199165715_646db5df320f2392463a1fdf3f08216b
│   ├── __2_1715199165715_646db5df320f2392463a1fdf3f08216b.vac
│   └── __2_2_705aeb4ecb9e35d5c21f78ef5dcc07c4
└── __tiledb_group.tdb

3 directories, 11 files

You can vacuum group metadata like you would vacuum array metadata as follows.

Python

# You can vacuum group metadata as follows
tiledb.Group.vacuum_metadata(grp_root_uri)

R

# Vacuuming consolidated group metadata is not currently possible
# with the R API. Please use another API.

Note that the meta subfolder has only one file, which has the metadata information across all timestamps.

/Users/stavrospapadopoulos/groups_grp_root
├── __group
│   ├── __1715199165273_1715199165273_0a368c13be4bcd88c9bf9b3b82d3e9de_2
│   ├── __1715199165406_1715199165406_2badabf703dba706e3b87836a5828366_2
│   ├── __1715199165553_1715199165553_1f20eb90c84468eef3489d6aea9439ef_2
│   ├── __1715199165559_1715199165559_52f1c8aa804df35017186d5ed391969a_2
│   └── __1_1_3edb27acc1cb01441ef5e33f69c1e83d_2
├── __meta
│   └── __2_1715199165715_646db5df320f2392463a1fdf3f08216b
└── __tiledb_group.tdb

3 directories, 7 files

Remember to clean up in the end.

Python

# Delete all paths
if os.path.exists(array_uri):
    shutil.rmtree(array_uri)
if os.path.exists(grp_root_uri):
    shutil.rmtree(grp_root_uri)
if os.path.exists(grp1_uri):
    shutil.rmtree(grp1_uri)
if os.path.exists(grp2_uri):
    shutil.rmtree(grp2_uri)
if os.path.exists(grp3_uri):
    shutil.rmtree(grp3_uri)

R

# Delete array if it already exists
if (file.exists(array_uri)) {
  unlink(array_uri, recursive = TRUE)
}
if (file.exists(grp_root_uri)) {
  unlink(grp_root_uri, recursive = TRUE)
}
if (file.exists(grp1_uri)) {
  unlink(grp1_uri, recursive = TRUE)
}
if (file.exists(grp2_uri)) {
  unlink(grp2_uri, recursive = TRUE)
}
if (file.exists(grp3_uri)) {
  unlink(grp3_uri, recursive = TRUE)
}

Groups on TileDB Cloud

Groups play an important role in managing assets on TileDB Cloud. For more information on the other group capabilities, visit the Catalog: Groups section.