-
Notifications
You must be signed in to change notification settings - Fork 2k
Open
Description
Background
Currently, H2O-3 development Docker images are built and pushed to AWS ECR using a Jenkins pipeline. We want to migrate this process to GitHub Actions for better maintainability, transparency, and integration with the GitHub ecosystem.
Purpose
- Automate the build and push of all H2O-3 development Docker images to AWS ECR.
- Maintain image dependency order to ensure reproducibility.
- Enable optional caching, forced rebuilds, and selective publishing to ECR.
Images Built
Images are built in dependency order across multiple tiers:
Tier 1: dev-base (foundation image)
Tier 2: dev-jdk-8-base, dev-jdk-others-base
Tier 3: dev-python-base
Tier 4: dev-python-3.7 to dev-python-3.11, dev-r-base
Tier 5: dev-r-{3.4.1, 3.5.3, 4.0.2, 4.4.0, 4.5.2}, dev-jdk-{8,11,17}, dev-mojocompat
Tier 6: dev-build-base
Tier 7: dev-build-hadoop
Tier 8: dev-release
Tier 9: dev-python-{version}-jdk-{version}, dev-r-{version}-jdk-{version}
Workflow Features
- Branch input: Builds can be triggered for a specific H2O-3 branch.
- Force build: Option to rebuild images even if they already exist in ECR.
- Publish: Option to push built images to AWS ECR.
- No-cache: Option to build images without using Docker cache.
- AWS Roles: Separate roles for ECR and S3 downloads.
- Multi-version support: Python and JDK versions are handled via matrix builds.
Current Status
- GitHub Actions workflow has been drafted for all image tiers.
- ECR login, S3 download, and Docker build/push steps are implemented.
- Matrix strategies for Python and JDK versions are included.
- Conditional logic is in place to skip builds if images already exist and
force_buildisfalse.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels