Skip to content

fix(ci): read the ANTLR archive with whichever tool the host provides - #30695

Open
sonika-shah wants to merge 2 commits into
mainfrom
fix/antlr-cli-optional-jar-check
Open

fix(ci): read the ANTLR archive with whichever tool the host provides#30695
sonika-shah wants to merge 2 commits into
mainfrom
fix/antlr-cli-optional-jar-check

Conversation

@sonika-shah

@sonika-shah sonika-shah commented Jul 30, 2026

Copy link
Copy Markdown
Collaborator

Problem

make install_antlr_cli fails on any image that carries a JRE but not a JDK. The download and checksum both succeed; the build then dies on the archive check that follows them:

Distro ANTLR candidate '4.7.2-5' is not 4.9.2; using pinned download.
/tmp/tmp.KRcI3pmOjh: OK          <- pinned SHA-256 verified
/bin/sh: 51: jar: not found      <- actual failure
ANTLR 4.9.2 download failed from https://repo1.maven.org/maven2/... (attempt 1)
...
Failed to download a valid ANTLR 4.9.2 CLI after 3 attempts
make: *** [Makefile:93: install_antlr_cli] Error 1

jar is a JDK tool. When it is missing, jar tf fails, the enclosing && chain reports the download as bad, and the recipe retries three times before giving up — on a payload that verified clean the first time.

The motivating casualty is the Collate local ingestion webserver image. Its base installs default-jre-headless (ingestion/Dockerfile:31), which provides java but no jar, and its Dockerfile runs this recipe during the build.

Branches carrying jar tf (main, 2.0, 2.0.0-rc1) are affected; the 1.13.x / 1.12.x line predates it and is unaffected.

Change

Read the archive with whichever tool the host provides, rather than insisting on jar:

&& { if   command -v jar   > /dev/null 2>&1; then jar tf "$jar_file" > /dev/null; \
     elif command -v unzip > /dev/null 2>&1; then unzip -tqq "$jar_file" > /dev/null; \
     else :; fi; }
Host Behaviour
JDK present (dev machines, Maven CI) jar tf — unchanged from today
JRE + unzip (slim runtime images) unzip -tcheck stays active
Neither reader falls through to the pinned checksum alone

A reader that is present and rejects the archive still fails the build, and there is no second attempt with the other tool — a rejection is a rejection.

The skip branch is not a coverage gap being accepted. A host with neither reader has no java either (verified: debian:bookworm-slim has none of jar, unzip, java), and the recipe's output is a #!/usr/bin/java -jar wrapper — so such a host cannot run the CLI regardless of what this check does.

Nothing else in the recipe needs a JDK: it uses printf/cat/chmod/mv.

Verification

Run in the real affected image (collate-customers-ingestion:om-2.0.0-rc1-cl-2.0.0-rc1), which reports jar: ABSENT, unzip: /usr/bin/unzip, java: openjdk 17.0.20, and distro antlr4 candidate 4.7.2-5 — so it exercises the pinned-download path.

Baseline (main) — reproduces the CI failure exactly, including exit code:

/tmp/tmp.jlBnvuLis1: OK
/bin/sh: 51: jar: not found        (x3)
Failed to download a valid ANTLR 4.9.2 CLI after 3 attempts
BASELINE EXIT=2

With this change:

Case Result
Real image (no jar, unzip present) EXIT=0; antlr4ANTLR Parser Generator Version 4.9.2
↳ which branch ran? UNZIP-BRANCH-RAN (args: -tqq /tmp/tmp.Wyjo8GsG0w) — delegating wrapper confirms the real unzip ran and passed
unzip present but rejects EXIT=2 — build fails, as it must
jar present and passes EXIT=0; JAR-BRANCH-RANjar takes precedence over unzip
jar present and rejects EXIT=2 — no fallback to unzip, even though unzip was present and would have passed
Neither reader EXIT=0 — checksum alone

Re-running is idempotent (ANTLR 4.9.2 already available at /usr/local/bin/antlr4; nothing to do.), and the installed CLI generates the real grammars:

$ antlr4 -Dlanguage=JavaScript -o /out openmetadata-spec/.../{Fqn,EntityLink,JdbcUri}.g4
FqnLexer.js  FqnParser.js  FqnListener.js
EntityLinkParser.js  EntityLinkListener.js
JdbcUriLexer.js  JdbcUriParser.js

FqnLexer.js / FqnParser.js are what the webserver UI imports.

Cherry-pick

Needs 2.0 and 2.0.0-rc1 — both carry the line and both feed local webserver builds.

Greptile Summary

Updates the ANTLR CLI installation recipe to validate downloaded archives with jar when available, fall back to unzip, and rely on the pinned checksum when neither archive reader exists.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename Overview
Makefile Replaces the unconditional JDK-dependent archive check with a portable reader fallback while preserving checksum verification and rejection behavior.

Reviews (3): Last reviewed commit: "Merge branch 'main' into fix/antlr-cli-o..." | Re-trigger Greptile

Copilot AI review requested due to automatic review settings July 30, 2026 10:34

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@github-actions

Copy link
Copy Markdown
Contributor

❌ PR checklist incomplete

This PR cannot be merged until the following are addressed on its linked issue:

  • No GitHub issue is linked. Link an issue in the Development section of the PR (or add Fixes #12345 to the description). For a same-org cross-repo issue, add Fixes open-metadata/<repo>#123 to the description.

The fields live on the linked issue in the Shipping project (open the issue → right sidebar → Projects). After you set them, re-run this check (or push a commit) — issue/project changes do not re-trigger it automatically.

Maintainers can bypass this check by adding the skip-pr-checks label.

@github-actions github-actions Bot added backend safe to test Add this label to run secure Github workflows on PRs labels Jul 30, 2026
@github-actions

github-actions Bot commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

✅ Playwright Results — workflow succeeded

Validated commit e4d6a45c508edf27489edde4f179006271f3cb7e in Playwright run 30545775418, attempt 1.

✅ 538 passed · ❌ 0 failed · 🟡 1 flaky · ⏭️ 3 skipped · 🧰 0 lifecycle flaky

Performance

Blocking targets: ✅ met · Optimization targets: 🟡 in progress

Shard-job maxima below are not the full workflow wall time; the linked run includes build, fixture, planning, and reporting.

🕒 Full workflow signal wall (to summary) 48m 30s

⏱️ Max setup 2m 44s · max shard execution 14m 38s · max shard-job elapsed before upload 17m 38s · reporting 4s

🌐 202.93 requests/attempt · 2.85 app boots/UI scenario · 19.02% common-shard skew

Optimization targets still in progress:

  • Common shard skew was 19.02% (convergence target: at most 15%).
  • Browser traffic was 202.93 requests per attempt (convergence target: fewer than 200).
  • Application boot ratio was 2.85 per UI scenario (1601 boots / 561 scenarios; convergence target: at most 1).
Shard Passed Failed Flaky Skipped Lifecycle failed Lifecycle flaky
✅ Shard chromium-01 82 0 0 0 0 0
✅ Shard chromium-02 101 0 0 0 0 0
✅ Shard chromium-03 114 0 0 0 0 0
🟡 Shard chromium-04 101 0 1 3 0 0
✅ Shard data-asset-rules-01 61 0 0 0 0 0
✅ Shard domain-isolation-01 14 0 0 0 0 0
✅ Shard global-state-01 23 0 0 0 0 0
✅ Shard ingestion-01 1 0 0 0 0 0
✅ Shard reindex-01 2 0 0 0 0 0
✅ Shard search-01 10 0 0 0 0 0
✅ Shard search-rbac-01 29 0 0 0 0 0
🟡 1 flaky test(s) (passed on retry)
  • Pages/Entity.spec.tsDomain Propagation (shard chromium-04, 1 retry)

📦 Download artifacts

How to debug locally
# Download playwright-test-results-<shard> artifact and unzip
npx playwright show-trace path/to/trace.zip    # view trace

install_antlr_cli validated the downloaded ANTLR jar with `jar tf` on top of the
pinned SHA-256. `jar` is a JDK tool, so on images that carry only a JRE the check
fails with `jar: not found`, the surrounding && chain reports the download as bad,
and the recipe retries three times before exiting 1 -- even though the download
and checksum both succeeded.

Read the archive with `jar` when present, else `unzip -t`, else fall through to the
checksum alone. A reader that is present and rejects the archive still fails the
build; only the case where no reader exists is skipped, and such a host has no java
either, so it could not run the installed CLI regardless.

This keeps the check active everywhere it runs today. The Collate local ingestion
webserver base image is the motivating case: it ships default-jre-headless (java,
no jar) but installs unzip for the Oracle instant client, so it now verifies via
unzip instead of failing.
@sonika-shah
sonika-shah force-pushed the fix/antlr-cli-optional-jar-check branch from a1e7abc to 26f9075 Compare July 30, 2026 11:27
Copilot AI review requested due to automatic review settings July 30, 2026 11:27

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@sonika-shah sonika-shah changed the title fix(ci): skip the ANTLR archive check when the JDK jar tool is absent fix(ci): read the ANTLR archive with whichever tool the host provides Jul 30, 2026
@sonika-shah sonika-shah added the skip-pr-checks Bypass PR metadata validation check label Jul 30, 2026
Copilot AI review requested due to automatic review settings July 30, 2026 13:11

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@gitar-bot

gitar-bot Bot commented Jul 30, 2026

Copy link
Copy Markdown
Code Review ✅ Approved

Makes the ANTLR CLI installation recipe skip the jar tf archive check on JRE-only base images when the JDK jar tool is absent, while strictly preserving SHA-256 verification and full validation when jar is present. No issues found.

Options

Display: compact → Showing less information.

Comment with these commands to change the behavior for this request:

Compact
gitar display:verbose         

Was this helpful? React with 👍 / 👎 | Gitar | Powered by Gitar — free for open source

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backend safe to test Add this label to run secure Github workflows on PRs skip-pr-checks Bypass PR metadata validation check

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants