Skip to content

Commit 9d9d637

Browse files
authored
Document ops log (#2036)
Ops part of #1958 not sure what about the txn log?
1 parent 3eb19e8 commit 9d9d637

File tree

1 file changed

+72
-0
lines changed

1 file changed

+72
-0
lines changed

icechunk-python/docs/docs/repository-features.md

Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -58,3 +58,75 @@ repo.set_metadata(dict(test=True, team="science"))
5858
repo.update_metadata(dict(number_of_bugs=42))
5959
print(repo.get_metadata())
6060
```
61+
## Operations Log
62+
63+
Who changed what, and when? As repositories grow and multiple collaborators commit, branch, tag, and run maintenance tasks, it's easy to lose track of what happened. Traditional object storage gives you no history of structural changes—you'd need to build your own audit trail.
64+
65+
Icechunk records every repository mutation in an **operations log** (ops log). Each entry captures the operation type, a timestamp, and any relevant details like branch names or snapshot IDs. The log is ordered newest-first and covers the full lifetime of the repository.
66+
67+
68+
### Reading the Log
69+
70+
```python exec="on" session="opslog" source="material-block" result="code"
71+
import icechunk as ic
72+
import zarr
73+
import numpy as np
74+
75+
# Create a repository and make some changes
76+
repo = ic.Repository.create(ic.in_memory_storage())
77+
session = repo.writable_session("main")
78+
root = zarr.group(session.store)
79+
root.create_array("temperature", shape=(100,), dtype="f4")
80+
session.commit("Add temperature array")
81+
82+
session = repo.writable_session("main")
83+
arr = zarr.open_array(session.store, path="temperature")
84+
arr[:] = np.random.randn(100).astype("f4")
85+
session.commit("Write temperature data")
86+
87+
repo.create_branch("develop", repo.lookup_branch("main"))
88+
89+
for update in repo.ops_log():
90+
print(f"{update.updated_at} {update.kind}")
91+
```
92+
93+
Each entry is an [`Update`](reference/ops.md) with three fields:
94+
95+
- `kind` — an [`UpdateType`](reference/ops.md) variant describing what happened
96+
- `updated_at` — a `datetime.datetime` timestamp (UTC, microsecond precision)
97+
- `backup_path` — internal storage detail (you can ignore this)
98+
99+
### Update Types
100+
101+
Every repository mutation creates exactly one log entry. The `kind` field tells you what happened:
102+
103+
#### Commits
104+
105+
| Type | Fields | Triggered by |
106+
|------|--------|-------------|
107+
| `UpdateType.NewCommit` | `branch`, `new_snap_id` | Committing a session |
108+
| `UpdateType.CommitAmended` | `branch`, `previous_snap_id`, `new_snap_id` | Committing with amend |
109+
| `UpdateType.NewDetachedSnapshot` | `new_snap_id` | Flushing a session (anonymous snapshot) |
110+
111+
#### Branches and Tags
112+
113+
| Type | Fields | Triggered by |
114+
|------|--------|-------------|
115+
| `UpdateType.BranchCreated` | `name` | Creating a branch |
116+
| `UpdateType.BranchDeleted` | `name`, `previous_snap_id` | Deleting a branch |
117+
| `UpdateType.BranchReset` | `name`, `previous_snap_id` | Resetting a branch to a different snapshot |
118+
| `UpdateType.TagCreated` | `name` | Creating a tag |
119+
| `UpdateType.TagDeleted` | `name`, `previous_snap_id` | Deleting a tag |
120+
121+
#### Repository Administration
122+
123+
| Type | Fields | Triggered by |
124+
|------|--------|-------------|
125+
| `UpdateType.RepoInitialized` || Creating a new repository |
126+
| `UpdateType.ConfigChanged` || Saving repository configuration |
127+
| `UpdateType.MetadataChanged` || Setting or updating repository metadata |
128+
| `UpdateType.RepoStatusChanged` | `status` | Changing repository status (e.g., read-only) |
129+
| `UpdateType.FeatureFlagChanged` | `id`, `new_value` | Changing a feature flag |
130+
| `UpdateType.GCRan` || Running garbage collection |
131+
| `UpdateType.ExpirationRan` || Running snapshot expiration |
132+
| `UpdateType.RepoMigrated` | `from_version`, `to_version` | Upgrading from an older format version |

0 commit comments

Comments
 (0)