Skip to content

Migrate the H2O-3 Docker Image Builds Automation to GitHub Actions #16764

@silvarohan18

Description

@silvarohan18

Background

Currently, H2O-3 development Docker images are built and pushed to AWS ECR using a Jenkins pipeline. We want to migrate this process to GitHub Actions for better maintainability, transparency, and integration with the GitHub ecosystem.

Purpose

  • Automate the build and push of all H2O-3 development Docker images to AWS ECR.
  • Maintain image dependency order to ensure reproducibility.
  • Enable optional caching, forced rebuilds, and selective publishing to ECR.

Images Built

Images are built in dependency order across multiple tiers:

Tier 1: dev-base (foundation image)
Tier 2: dev-jdk-8-base, dev-jdk-others-base
Tier 3: dev-python-base
Tier 4: dev-python-3.7 to dev-python-3.11, dev-r-base
Tier 5: dev-r-{3.4.1, 3.5.3, 4.0.2, 4.4.0, 4.5.2}, dev-jdk-{8,11,17}, dev-mojocompat
Tier 6: dev-build-base
Tier 7: dev-build-hadoop
Tier 8: dev-release
Tier 9: dev-python-{version}-jdk-{version}, dev-r-{version}-jdk-{version}

Workflow Features

  • Branch input: Builds can be triggered for a specific H2O-3 branch.
  • Force build: Option to rebuild images even if they already exist in ECR.
  • Publish: Option to push built images to AWS ECR.
  • No-cache: Option to build images without using Docker cache.
  • AWS Roles: Separate roles for ECR and S3 downloads.
  • Multi-version support: Python and JDK versions are handled via matrix builds.

Current Status

  • GitHub Actions workflow has been drafted for all image tiers.
  • ECR login, S3 download, and Docker build/push steps are implemented.
  • Matrix strategies for Python and JDK versions are included.
  • Conditional logic is in place to skip builds if images already exist and force_build is false.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions