Replace CEL-based NoSQL maintenance retention with commit count configuration

[iting0321]

Summary

Fixes #3847 .

This PR replaces CEL-based retention rules in NoSQL metastore maintenance with simple positive integer configuration values.

Each *-retain property now specifies the number of latest commits to retain. All properties default to 1, preserving the current behavior of retaining only the latest commit.

This PR also:

• Removes the CEL retention module and dependencies.
• Adds validation and tests for commit-count retention.
• Updates configuration documentation and license files.
• Adds migration guidance: replace *-retain=false with *-retain=1.

CEL values such as true and expressions using ageDays, ageHours, or ageMinutes are no longer supported.

Migration

This is a user-facing configuration change. Existing CEL values must be replaced with commit counts.

Previous value	New value
false	1
commits <= N	N+1
true	No exact unbounded equivalent, configure an explicit sufficiently large count
Expressions using ageDays, ageHours, or ageMinutes	No time-based equivalent, configure an appropriate commit count

For example:

# Before
polaris.persistence.nosql.maintenance.catalog.catalog-state-retain=false

# After
polaris.persistence.nosql.maintenance.catalog.catalog-state-retain=1

This PR does not introduce a catalog-state-retain-days property. The current behavior was deliberately changed by #3483 to retain only the latest catalog state, so adding a 30-day default would restore previously removed behavior rather than preserve the existing behavior.

Checklist

• 🛡️ Don't disclose security issues! (contact security@apache.org)
• 🔗 Clearly explained why the changes are needed, or linked related issues: Fixes Use simple config instead of CEL expressions in NoSQL maintenance CLI #3847
• 🧪 Added/updated tests with good coverage, or manually tested (and explained how)
• 💡 Added comments for complex logic
• 🧾 Updated CHANGELOG.md (if needed)
• 📚 Updated documentation in site/content/in-dev/unreleased (if needed)

[Copilot]

Pull request overview

This PR replaces CEL-expression-based retention rules for NoSQL metastore maintenance with a simpler commit-count configuration, and removes the now-unused CEL retention module/dependencies across the build and distribution artifacts.

Changes:

• Replace *-retain CEL expressions with @Min(1) integer commit-count settings (defaulting to 1) for catalog maintenance retention.
• Remove the CEL retention module and its dependency wiring (Gradle projects, BOM, versions catalog), and tighten Checkstyle’s relocated-package ban accordingly.
• Add commit-count retention tests and update user documentation/migration guidance.

Reviewed changes

Copilot reviewed 20 out of 20 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
site/content/in-dev/unreleased/configuration/config-sections/smallrye-polaris_persistence_nosql_maintenance_catalog.md	Updates the config reference docs to describe *-retain as int commit counts with default 1.
site/content/in-dev/unreleased/admin-tool.md	Adds an “important” upgrade note explaining the migration away from CEL expressions.
runtime/server/distribution/LICENSE	Removes the bundled dependency notice for Projectnessie CEL.
runtime/admin/distribution/LICENSE	Removes the bundled dependency notice for Projectnessie CEL.
persistence/nosql/persistence/metastore-maintenance/src/testFixtures/java/org/apache/polaris/persistence/nosql/metastore/maintenance/MutableCatalogsMaintenanceConfig.java	Updates test-fixture config wrapper to return int retention values.
persistence/nosql/persistence/metastore-maintenance/src/test/java/org/apache/polaris/persistence/nosql/metastore/maintenance/TestCommitRetention.java	Adds focused tests for commit-count retention behavior and invalid values.
persistence/nosql/persistence/metastore-maintenance/src/test/java/org/apache/polaris/persistence/nosql/metastore/maintenance/TestCatalogMaintenance.java	Updates existing maintenance tests to configure retention via commit counts.
persistence/nosql/persistence/metastore-maintenance/src/main/java/org/apache/polaris/persistence/nosql/metastore/maintenance/CatalogsMaintenanceConfig.java	Switches config mapping from optional CEL strings to validated int commit-count properties with defaults.
persistence/nosql/persistence/metastore-maintenance/src/main/java/org/apache/polaris/persistence/nosql/metastore/maintenance/CatalogRetainedIdentifier.java	Replaces CEL-based “continue” predicates with a commit-count predicate for identifying retained objects.
persistence/nosql/persistence/metastore-maintenance/build.gradle.kts	Drops the CEL maintenance module dependency.
persistence/nosql/persistence/maintenance/spi/src/main/java/org/apache/polaris/persistence/nosql/maintenance/spi/RetainedCollector.java	Removes references to the CEL predicate from SPI documentation/imports.
persistence/nosql/persistence/maintenance/spi/build.gradle.kts	Removes compileOnly dependency on the CEL maintenance module.
persistence/nosql/persistence/maintenance/retain-cel/src/test/java/org/apache/polaris/maintenance/cel/TestCelReferenceContinuePredicate.java	Deletes CEL predicate tests (module removal).
persistence/nosql/persistence/maintenance/retain-cel/src/main/java/org/apache/polaris/maintenance/cel/CelReferenceContinuePredicate.java	Deletes the CEL-based reference continue predicate implementation.
persistence/nosql/persistence/maintenance/retain-cel/src/jmh/java/org/apache/polaris/maintenance/cel/CelReferenceContinuePredicateBench.java	Deletes the CEL predicate benchmark (module removal).
persistence/nosql/persistence/maintenance/retain-cel/build.gradle.kts	Removes the Gradle module definition for the CEL retention project.
gradle/projects.main.properties	Removes the CEL module from the Gradle project map.
gradle/libs.versions.toml	Removes the CEL BOM entry from the versions catalog.
codestyle/checkstyle.xml	Removes the CEL-related exception and bans all *.relocated.* imports again.
bom/build.gradle.kts	Removes the CEL module from the published BOM dependencies.

[dimas-b]

Thanks for picking up #3847, @iting0321 !

The PR LGTM with a couple of minor comments.

[dimas-b]

This looks like a reasonable compromise for now.

[dimas-b]

This is a bit misleading. I'd expect three test calls to return true 🤔

E.g. the old CEL expression commits <= 3 would be true on three calls, right?

[iting0321]

Thanks for your feedback!
The behavior for RetainedCollector.refRetain() retains the current commit before evaluating
the predicate:

var obj = iter.next();

retainedObjConsumer.accept(obj); // current commit is retained first

if (!continuePredicate.test(obj)) {
    break;
  }

Therefore, true, true, false retains exactly three commits the predicate determines whether traversal should continue to the next commit.
I agree the name is misleading so I rename the method to continueWhileMoreCommitsRemain.

[dimas-b]

This can underflow (after many calls).

[iting0321]

fixed!

[snazy]

I don’t think this is ready to merge yet.

NoSQL pagination tokens contain a reference to historic state, subsequent requests need that exact object. If maintenance has already collected it, pagination would break in a subtle way, after an arbitrarily short time.

Could we define the expected token behavior across maintenance and add a test covering this interaction? Depending on the outcome, we may need to preserve a time-based retention dimension, though not necessarily CEL itself.

Please also verify the migration mapping for commits <= N: since the current commit is retained before evaluating the predicate, the old expression appears to retain N + 1, not N.

Requesting changes pending that investigation.

[flyrain]

Is @Min actually checked here (maybe via Quarkus)?

[dimas-b]

Yes, Quarkus should enforce validation annotations on its managed config objects (worth double-checking, though 😅 )

[iting0321]

I don’t think this is ready to merge yet.

NoSQL pagination tokens contain a reference to historic state, subsequent requests need that exact object. If maintenance has already collected it, pagination would break in a subtle way, after an arbitrarily short time.

Could we define the expected token behavior across maintenance and add a test covering this interaction? Depending on the outcome, we may need to preserve a time-based retention dimension, though not necessarily CEL itself.

Please also verify the migration mapping for commits <= N: since the current commit is retained before evaluating the predicate, the old expression appears to retain N + 1, not N.

Requesting changes pending that investigation.

Hi @snazy, thanks for your feedback!
I’ve updated the PR accordingly.

For pagination, I defined the following behavior:

• A NoSQL pagination token remains usable for at least the configured pagination-token-retention duration after its referenced snapshot is superseded.
• The new setting applies globally to all maintained container histories, rather than only catalog state, and defaults to P30D.
• Commit-count and time-based retention are combined with OR semantics: a snapshot is retained while either condition requires it.
• After the retention window expires and maintenance collects the snapshot, reuse of the token fails explicitly with 400 Bad Request (Invalid or expired NoSQL pagination token) instead of silently returning an empty page.

I added an integration test that creates a pagination token, supersedes its snapshot, runs maintenance, and verifies that:

1. pagination remains snapshot-consistent within the retention window
2. newly created data does not appear in the historic page
3. the token is rejected explicitly after the window expires and the snapshot is collected.

You were also correct about the migration mapping. Because the current commit is retained before the predicate is evaluated:

• commits < N maps to N
• commits <= N maps to N + 1

I corrected this in both the changelog and migration documentation.