Commits

Each commit in a TileDB array corresponds to a successfully-written fragment and enables you to read data from that fragment.

TileDB supports multiple concurrent readers and writers. But how does it guarantee that a read operation does not access a partially complete write (which can lead to inconsistencies and incorrect results)?

TileDB achieves this with the concept of commits. Effectively, when TileDB writes a fragment, the fragment is invisible to the read operations. A read operation considers a fragment as complete, only if a corresponding commit file is written in the __commits subfolder inside the array folder (visit the Storage Format Spec section for more details on the commit file format). The commit file is written by the write operation as the very last step and only when all the fragment files have been successfully generated. This guarantees write atomicity and consistency, even if a fragment write fails with partially written files.