Skip to content

ci: publish engine base image to GHCR - #91

Merged
jiaoew1991 merged 4 commits into
developfrom
enwei/publish-engine-ghcr
Jun 9, 2026
Merged

ci: publish engine base image to GHCR #91
jiaoew1991 merged 4 commits into
developfrom
enwei/publish-engine-ghcr

Conversation

@jiaoew1991

Copy link
Copy Markdown
Collaborator

No description provided.

jiaoew1991 and others added 2 commits June 9, 2026 12:44
Build engine/Dockerfile (ubuntu-based: py3.12, JDK17, Ray, RayDP spark4.1/scala2.13,
lance-spark, pylance) and push to ghcr.io/lumalabs/nurion-engine so downstream images
can FROM it without cross-account AWS ECR.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
lance-spark 0.5.1 is the latest (DataFrame write support); pin pylance to 7.0.0 to
match lance-spark 0.5.1's lance-core (unified 7.0.0 version line).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@cursor

cursor Bot commented Jun 9, 2026

Copy link
Copy Markdown

PR Summary

Medium Risk
Pinned pylance and lance-spark versions can change query/data behavior; the Dockerfile build-path change must succeed in CI or the new GHCR image will not build.

Overview
Adds a GitHub Actions workflow that builds engine/Dockerfile and pushes the nurion engine base image to GHCR (ghcr.io/lumalabs/nurion-engine:develop and a SHA tag), with disk cleanup and Buildx so downstream services can FROM it without cross-account ECR.

Dockerfile changes fix the in-image build layout: lib/anvil-rs and lib/raydp/java are copied under /app/build/lib/ so RayDP’s Maven protoSourceRoot still resolves sibling anvil-rs/proto. anvil-rs wheel build and RayDP JAR packaging are merged into one RUN layer with shared proto usage and a single cleanup pass. Runtime pins bump lance-spark-bundle to 0.5.1 and pylance to 7.0.0 (from a minimum version).

Reviewed by Cursor Bugbot for commit 10654a9. Bugbot is set up for automated code reviews on this repo. Configure here.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Autofix Details

Bugbot Autofix prepared a fix for the issue found in the latest run.

  • ✅ Fixed: Develop tag wrong branch
    • Updated workflow to run on main and develop branches instead of the feature branch, ensuring the :develop tag remains current after PR merge.

Create PR

Or push these changes by commenting:

@cursor push f561177c8b
Preview (f561177c8b)
diff --git a/.github/workflows/publish-engine-base.yaml b/.github/workflows/publish-engine-base.yaml
--- a/.github/workflows/publish-engine-base.yaml
+++ b/.github/workflows/publish-engine-base.yaml
@@ -7,7 +7,7 @@
 
 on:
     push:
-        branches: [enwei/publish-engine-ghcr]
+        branches: [main, develop]
     workflow_dispatch:
 
 permissions:

You can send follow-ups to the cloud agent here.


on:
push:
branches: [enwei/publish-engine-ghcr]

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Develop tag wrong branch

Medium Severity

The workflow publishes ghcr.io/lumalabs/nurion-engine:develop but only runs on pushes to enwei/publish-engine-ghcr. After merge, pushes to develop or main will not rebuild that tag, so downstream FROM consumers can pull a :develop image that does not match the repository’s develop branch.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit dd9248f. Configure here.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

There are 2 total unresolved issues (including 1 from previous review).

Fix All in Cursor

Bugbot Autofix prepared a fix for the issue found in the latest run.

  • ✅ Fixed: Pylance pin unavailable on PyPI
    • Changed pylance==7.0.0 to pylance>=4.0.0 to match available PyPI versions and align with other configuration files.

Create PR

Or push these changes by commenting:

@cursor push 4b6d245a1d
Preview (4b6d245a1d)
diff --git a/engine/Dockerfile b/engine/Dockerfile
--- a/engine/Dockerfile
+++ b/engine/Dockerfile
@@ -89,7 +89,7 @@
     pandas>=2.0.0 \
     click>=8.1.7 \
     "fsspec[s3]>=2024.6.0" \
-    pylance==7.0.0 \
+    "pylance>=4.0.0" \
     s3fs>=2024.6.0 \
     grpcio>=1.68.0

You can send follow-ups to the cloud agent here.

Reviewed by Cursor Bugbot for commit 3b05556. Configure here.

Comment thread engine/Dockerfile
click>=8.1.7 \
"fsspec[s3]>=2024.6.0" \
pylance>=0.38.0 \
pylance==7.0.0 \

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pylance pin unavailable on PyPI

High Severity

The engine base image install step pins pylance==7.0.0 from the default PyPI index, but that exact release does not appear among versions resolved elsewhere in the repo from pypi.org, so uv pip install in the Docker build will likely fail and block the new publish workflow.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 3b05556. Configure here.

@github-actions

github-actions Bot commented Jun 9, 2026

Copy link
Copy Markdown

Coverage Report

Overall: 55%

Diff Coverage (changed files only)

Diff Coverage

Diff: origin/develop...HEAD, staged and unstaged changes

No lines with coverage information in this diff.

The RayDP fork's protobuf-maven-plugin reads its gRPC proto from
${project.basedir}/../../../anvil-rs/proto (the sibling lib/anvil-rs/proto).
The base-image Dockerfile flattened the COPY layout (/app/build/anvil-rs,
/app/build/java) so that relative path missed, and it deleted anvil-rs
before the RayDP maven build. With no proto, protobuf-maven-plugin
generated nothing and scalac failed with 6 "not found:
PushRequest/PushResponse/AnvilGrpc" errors.

Copy the sources into /app/build/lib/{anvil-rs,raydp/java} to preserve
the relative proto path, and merge the anvil-rs + RayDP builds into one
layer so the proto survives through the maven build. Reverts the -e
debug flag added while diagnosing.

Verified locally: mvn -P scala-2.13 clean package -> BUILD SUCCESS,
AnvilGrpc.java generated under target/generated-sources, all three
raydp jars produced.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@jiaoew1991 jiaoew1991 changed the title Enwei/publish engine ghcr ci: publish engine base image to GHCR Jun 9, 2026
@jiaoew1991
jiaoew1991 merged commit 103599f into develop Jun 9, 2026
16 of 18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant