Skip to content

Provide a non-collecting iterator for history() #3753

@chiragjn

Description

@chiragjn

pub async fn history(&self, limit: Option<usize>) -> Result<Vec<CommitInfo>, DeltaTableError> {
let infos = self
.snapshot()?
.snapshot
.snapshot()
.commit_infos(&self.log_store(), limit)
.await?
.try_collect::<Vec<_>>()
.await?;
Ok(infos.into_iter().flatten().collect())
}

Since the history commits are yielded in reverse chronological order it would be beneficial for use-cases where we want to break early and not scan lots of small .json files from object store

We can also work with a time filter to just get commits since a given timestamp - I can open a separate issue for this if it sounds reasonable

Metadata

Metadata

Assignees

Labels

binding/rustIssues for the Rust crate

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions