Skip to content

[RNTuple] add LZ4 compression for RNTuple writing and default to it#439

Merged
Moelf merged 2 commits into
RNTuple_fablefrom
RNTuple_compression
Jun 16, 2026
Merged

[RNTuple] add LZ4 compression for RNTuple writing and default to it#439
Moelf merged 2 commits into
RNTuple_fablefrom
RNTuple_compression

Conversation

@Moelf

@Moelf Moelf commented Jun 13, 2026

Copy link
Copy Markdown
Member

this goes on top of #438 (I don't have Stacked PR so no UI sugar :()

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds ROOT-style write-time compression for UnROOT’s RNTuple writer, defaulting output to LZ4 (ROOT fCompress 404), and expands CI validation to cross-check written files with C++ ROOT via a data-driven sidecar spec.

Changes:

  • Implement ROOT compression block framing for RNTuple writing (LZ4/ZSTD/ZLIB) and apply it to header/footer/page-link envelopes and per-page payloads.
  • Default write_rntuple(...; compression=...) to LZ4 (404) and propagate fCompress into the ROOT file header.
  • Replace the previous ad-hoc ROOT validation script with a JSON-driven validator and run it in CI for multiple compression settings.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
src/RNTuple/Writing/compression.jl Adds ROOT block framing and per-algorithm compression helpers for the writer.
src/RNTuple/Writing/TFileWriter.jl Plumbs compression through writing, compresses envelopes/pages, and records on-disk sizes in locators/anchor.
src/UnROOT.jl Imports zlib compression APIs and includes the new compression implementation.
test/RNTupleWriting/highlevel.jl Adds round-trip tests across compression modes and checks default LZ4 behavior/size impact.
test/RNTupleWriting/output_sample.jl Generates deterministic sample data plus a JSON sidecar describing expected values and compression.
test/RNTupleWriting/validate_rntuple.py New ROOT-based validator that checks readability and exact value correctness from the JSON sidecar.
test/RNTupleWriting/test1.py Removes the old minimal validation script.
.github/workflows/cvmfs.yml Updates CI to generate/validate files for multiple compression settings using C++ ROOT.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/RNTuple/Writing/compression.jl
@Moelf Moelf requested a review from tamasgal June 13, 2026 17:16
@tamasgal

Copy link
Copy Markdown
Member

Let's go ;)

@Moelf Moelf merged commit 0b79244 into RNTuple_fable Jun 16, 2026
1 check passed
@Moelf Moelf deleted the RNTuple_compression branch June 16, 2026 01:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants