ci: publish engine base image to GHCR - #91
Conversation
Build engine/Dockerfile (ubuntu-based: py3.12, JDK17, Ray, RayDP spark4.1/scala2.13, lance-spark, pylance) and push to ghcr.io/lumalabs/nurion-engine so downstream images can FROM it without cross-account AWS ECR. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
lance-spark 0.5.1 is the latest (DataFrame write support); pin pylance to 7.0.0 to match lance-spark 0.5.1's lance-core (unified 7.0.0 version line). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
PR SummaryMedium Risk Overview Dockerfile changes fix the in-image build layout: Reviewed by Cursor Bugbot for commit 10654a9. Bugbot is set up for automated code reviews on this repo. Configure here. |
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
Autofix Details
Bugbot Autofix prepared a fix for the issue found in the latest run.
- ✅ Fixed: Develop tag wrong branch
- Updated workflow to run on main and develop branches instead of the feature branch, ensuring the :develop tag remains current after PR merge.
Or push these changes by commenting:
@cursor push f561177c8b
Preview (f561177c8b)
diff --git a/.github/workflows/publish-engine-base.yaml b/.github/workflows/publish-engine-base.yaml
--- a/.github/workflows/publish-engine-base.yaml
+++ b/.github/workflows/publish-engine-base.yaml
@@ -7,7 +7,7 @@
on:
push:
- branches: [enwei/publish-engine-ghcr]
+ branches: [main, develop]
workflow_dispatch:
permissions:You can send follow-ups to the cloud agent here.
|
|
||
| on: | ||
| push: | ||
| branches: [enwei/publish-engine-ghcr] |
There was a problem hiding this comment.
Develop tag wrong branch
Medium Severity
The workflow publishes ghcr.io/lumalabs/nurion-engine:develop but only runs on pushes to enwei/publish-engine-ghcr. After merge, pushes to develop or main will not rebuild that tag, so downstream FROM consumers can pull a :develop image that does not match the repository’s develop branch.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit dd9248f. Configure here.
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
There are 2 total unresolved issues (including 1 from previous review).
Bugbot Autofix prepared a fix for the issue found in the latest run.
- ✅ Fixed: Pylance pin unavailable on PyPI
- Changed pylance==7.0.0 to pylance>=4.0.0 to match available PyPI versions and align with other configuration files.
Or push these changes by commenting:
@cursor push 4b6d245a1d
Preview (4b6d245a1d)
diff --git a/engine/Dockerfile b/engine/Dockerfile
--- a/engine/Dockerfile
+++ b/engine/Dockerfile
@@ -89,7 +89,7 @@
pandas>=2.0.0 \
click>=8.1.7 \
"fsspec[s3]>=2024.6.0" \
- pylance==7.0.0 \
+ "pylance>=4.0.0" \
s3fs>=2024.6.0 \
grpcio>=1.68.0You can send follow-ups to the cloud agent here.
Reviewed by Cursor Bugbot for commit 3b05556. Configure here.
| click>=8.1.7 \ | ||
| "fsspec[s3]>=2024.6.0" \ | ||
| pylance>=0.38.0 \ | ||
| pylance==7.0.0 \ |
There was a problem hiding this comment.
Pylance pin unavailable on PyPI
High Severity
The engine base image install step pins pylance==7.0.0 from the default PyPI index, but that exact release does not appear among versions resolved elsewhere in the repo from pypi.org, so uv pip install in the Docker build will likely fail and block the new publish workflow.
Reviewed by Cursor Bugbot for commit 3b05556. Configure here.
Coverage ReportOverall: 55% Diff Coverage (changed files only)Diff CoverageDiff: origin/develop...HEAD, staged and unstaged changesNo lines with coverage information in this diff. |
The RayDP fork's protobuf-maven-plugin reads its gRPC proto from
${project.basedir}/../../../anvil-rs/proto (the sibling lib/anvil-rs/proto).
The base-image Dockerfile flattened the COPY layout (/app/build/anvil-rs,
/app/build/java) so that relative path missed, and it deleted anvil-rs
before the RayDP maven build. With no proto, protobuf-maven-plugin
generated nothing and scalac failed with 6 "not found:
PushRequest/PushResponse/AnvilGrpc" errors.
Copy the sources into /app/build/lib/{anvil-rs,raydp/java} to preserve
the relative proto path, and merge the anvil-rs + RayDP builds into one
layer so the proto survives through the maven build. Reverts the -e
debug flag added while diagnosing.
Verified locally: mvn -P scala-2.13 clean package -> BUILD SUCCESS,
AnvilGrpc.java generated under target/generated-sources, all three
raydp jars produced.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>



No description provided.