Skip to content

[tracking] catalog managed tables supportΒ #1132

@zachschuermann

Description

@zachschuermann

tracking read/write support for new catalogManaged tables. first targeting reads, items tracked below.

Milestone 1: catalog-managed reads in kernel (with log_tail of commits)

πŸ› οΈ Snapshot APIs

Description Issue/PR Status Notes
txnId in commit info: required for catalog-managed tables #1148 βœ…
prefactor: move out listed log files #1150 βœ…
prefactor: move out last checkpoint hint #1154 βœ…
add experimental UC client #1164 βœ…
add catalogManaged table features and catalog-managed feature flag #1165 βœ…
prefactor: LogSegmentBuilder #1159 ⚠️ Paused, just modified constructors directly for now.
prefactor: push down _last_checkpoint read into LogSegment #1204 βœ…
prefactor: SnapshotBuilder #1189 βœ…
Add log_tail to listing #1194 βœ…
prefactor: move Snapshot::try_new_from into SnapshotBuilder #1289 βœ…
Add staged commits to ParsedLogPath #1305 βœ…
SnapshotBuilder.with_log_tail #1290 βœ…

⚠️ Paused: TableChanges APIs

Description Issue/PR Status Notes
Plumb through log_tail to TableChanges APIs #1292 πŸ› οΈ

Milestone 2: catalog-managed reads via high-level APIs (uc-catalog crate)

Description Issue/PR Status Notes
uc-catalog crate with load_table API #1324 πŸ› οΈ
uc-catalog in-mem tests TODO πŸ› οΈ
uc-catalog FFI TODO πŸ› οΈ

Milestone 3: catalog-managed writes in kernel

Description Issue/PR Status Notes
prefactor: CommitResult #1343 πŸ“
new Committer API with filesystem-based Committer TODO πŸ› οΈ

Milestone 4: catalog-managed writes via high-level APIs (uc-catalog crate)

todo

Metadata

Metadata

Labels

No labels
No labels

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions