[DENG-8809] Migrate CircleCI to Github Action #469
Conversation
scholtzan
force-pushed
the
migrate-to-gha
branch
3 times, most recently
from
fc7dd4d to
eb1c888
Compare
scholtzan
changed the title
scholtzan
force-pushed
the
migrate-to-gha
branch
3 times, most recently
from
2bb09a3 to
53c9b08
Compare
scholtzan
force-pushed
the
migrate-to-gha
branch
2 times, most recently
from
a6db4c1 to
92fd8c4
Compare
scholtzan
force-pushed
the
migrate-to-gha
branch
from
92fd8c4 to
db7abce
Compare
scholtzan
force-pushed
the
migrate-to-gha
branch
from
c62b85d to
3234854
Compare
BenWu
left a comment
BenWu
left a comment
There was a problem hiding this comment.
The pull request template should also be updated to remove the circleci references
| uses: actions/checkout@v6 | ||
| with: | ||
| persist-credentials: false | ||
| - name: Build the Docker image |
There was a problem hiding this comment.
Just a general question: is there a standard way to use the docker cache across jobs in a workflow? The test job builds the image from scratch. I don't think the circleci version shared the cache either but I'm wondering if there's a GHA has something built for that
There was a problem hiding this comment.
Yeah, good point. I changed the config to store the image artifacts so they can be re-used in the deploy jobs.
| push: | ||
| branches: | ||
| - '**' | ||
| pull_request: |
There was a problem hiding this comment.
I guess we only need to build on push when changes get merged into main, I changed that in the config. Pushing to another branch won't trigger a build, unless a PR gets opened. Which I would expect to be the case in this repo anyway.
scholtzan
force-pushed
the
migrate-to-gha
branch
from
ad72115 to
a1442d1
Compare
| run: | | ||
| docker build jobs/{{ job_name }} -t us-docker.pkg.dev/moz-fx-data-artifacts-prod/docker-etl/{{ job_name }}:latest | ||
| # yamllint enable | ||
| - name: Save Docker image |
There was a problem hiding this comment.
Storing the image seems to sometimes take longer than building it so maybe this isn't that useful. Also it would only need to store it on main unless someone adds another job.
Does caching described here work? https://docs.docker.com/build/ci/github-actions/cache/
In any case, I think we can leave this as something to look into in the future and take this out for now
https://mozilla-hub.atlassian.net/browse/DENG-8809
Needs to be merged with mozilla/telemetry-airflow#2308
This migrates the CircleCI configs to Github Actions. This change will allow users that create a PR to merge this PR without running into issues with credentials/contexts for pushing the docker images to GAR.
I tested the deploy in 3234854 and it pushed the image to https://console.cloud.google.com/artifacts/docker/moz-fx-data-artifacts-prod/us/docker-etl/bq2sftp/sha256:e9b80133ad7cbfd71057d52d746e3248fcde9a454212f0971d7bb190dde6d155?project=moz-fx-data-artifacts-prod
I'll disable the CircleCI Pipeline once this has been approved and will reconfigure the required build check to the GHA one.
See DENG-8850 for additional discussion.
Checklist for reviewer:
referenced, the pull request should include the bug number in the title)
.circleci/config.yml) will cause environment variables (particularlycredentials) to be exposed in test logs
telemetry-airflow
responsibly.
Note for deployments: In order to push images built by this PR, the user who merges the PR
must be in the telemetry Github team.
This is because deploys depend on the
data-eng-airflow-gcr CircleCI context.
See DENG-8850 for additional discussion.