-
Notifications
You must be signed in to change notification settings - Fork 4.8k
ci: make cdk connector compatibility test create gh issue #67611
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Changes from 5 commits
c5a4b7c
7b0b600
1991d71
4020792
1f06c0c
5521d9c
4080874
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -114,12 +114,41 @@ jobs: | |
working-directory: airbyte-integrations/connectors/${{ matrix.connector }} | ||
run: poe test-integration-tests | ||
|
||
- name: Check if move team should be notified | ||
id: check-move-notify | ||
if: always() | ||
run: | | ||
echo "should-notify=${{ | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. setting condition here so that I don't repeat it everywhere |
||
github.event_name == 'schedule' && | ||
startsWith(matrix.connector, 'destination-') && | ||
(steps.run-unit-tests.outcome == 'failure' || steps.run-integration-tests.outcome == 'failure') | ||
}}" >> $GITHUB_OUTPUT | ||
|
||
- name: Slack Notification on Failure | ||
if: github.event_name == 'schedule' && failure() && (steps.run-unit-tests.outcome == 'failure' || steps.run-integration-tests.outcome == 'failure') | ||
if: always() && steps.check-move-notify.outputs.should-notify == 'true' | ||
uses: slackapi/slack-github-action@91efab103c0de0a537f72a35f6b8cda0ee76bf0a # v2.1.1 | ||
with: | ||
token: ${{ secrets.SLACK_BOT_TOKEN_AIRBYTE_TEAM }} | ||
method: chat.postMessage | ||
payload: | | ||
channel: C09H56MT8J2 # jose-alert-test channel. The use of this channel is meant to be temporary while we evaluate the usefulness of these notifications. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I renamed the channel to |
||
channel: C09H56MT8J2 | ||
text: "`${{ matrix.connector }}` connector tests failed when using local CDK! <https://github.com/${{github.repository}}/actions/runs/${{github.run_id}}|View Test Logs>" | ||
|
||
- name: Authenticate as GitHub App | ||
if: always() && steps.check-move-notify.outputs.should-notify == 'true' | ||
uses: actions/create-github-app-token@67018539274d69449ef7c02e8e71183d1719ab42 # v2 | ||
id: app-token | ||
with: | ||
owner: "airbytehq" | ||
repositories: "airbyte-internal-issues" | ||
app-id: ${{ secrets.OCTAVIA_BOT_APP_ID }} | ||
private-key: ${{ secrets.OCTAVIA_BOT_PRIVATE_KEY }} | ||
|
||
- name: Create Github Issue on Failure | ||
if: always() && steps.check-move-notify.outputs.should-notify == 'true' | ||
env: | ||
GH_TOKEN: ${{ steps.app-token.outputs.token }} | ||
run: tools/bin/bulk-cdk-compatibility-test/create-gh-issue.sh \ | ||
${{ matrix.connector }} \ | ||
${{ github.sha }} \ | ||
"https://github.com/${{github.repository}}/actions/runs/${{github.run_id}}" |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,51 @@ | ||
#!/usr/bin/env bash | ||
|
||
set -euo pipefail | ||
|
||
# Creates a GitHub issue for CDK connector compatibility test failures. | ||
# If an issue for the connector already exists, appends a new entry to the existing issue | ||
# instead of creating a new issue. | ||
# | ||
# This is intended to be used by the move team since it has specific logic to how the move team manages oncall issues. | ||
# | ||
# Usage: tools/bin/bulk-cdk-compatibility-test/create-gh-issue.sh <connector_name> <job_url> <commit_sha> | ||
# Example: tools/bin/bulk-cdk-compatibility-test/create-gh-issue.sh destination-dev-null www.myjoburl.com abcdef123456 | ||
|
||
# default values are specific to how the move team manages issues | ||
ISSUES_REPOSITORY="${ISSUES_REPOSITORY:-airbytehq/airbyte-internal-issues}" | ||
PROJECT_NUMBER="${PROJECT_NUMBER:-98}" | ||
STATUS_FIELD_ID="${STATUS_FIELD_ID:-PVTSSF_lADOA4_XW84Am4WkzgetXZM}" | ||
ONCALL_OPTION_ID="${ONCALL_OPTION_ID:-3ecf8bb4}" | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. using all these ids is not pretty, but afaik this is the only way to deal creating issues and editing their fields There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. can you add a comment explaining how to find these IDs? I actually don't know where to find them 😅 |
||
|
||
connector_name=$1 | ||
job_url=$2 | ||
commit_sha=$3 | ||
|
||
title="[CDK Connector Compatibility Test] Failures for $connector_name" | ||
|
||
existing_issue_number=$(gh issue list --state open --search "$title in:title" -R $ISSUES_REPOSITORY --json number --jq '.[0].number') | ||
|
||
# Build new entry | ||
timestamp="$(date -u +'%Y-%m-%d %H:%M:%SZ')" | ||
entry=$'#### ❌ '"$timestamp"$'\n'"Run: $job_url"$'\n'"Commit: ${commit_sha}" | ||
|
||
if [ -n "${existing_issue_number:-}" ]; then | ||
# add new entry to existing issue | ||
existing_issue_body="$(gh issue view "$existing_issue_number" -R "$ISSUES_REPOSITORY" --json body --jq .body)" | ||
updated_issue_body="$existing_issue_body"$'\n'"${entry}" | ||
gh issue edit "$existing_issue_number" -R "$ISSUES_REPOSITORY" -b "$updated_issue_body" | ||
echo "Updated issue #${existing_issue_number} with new failure entry." | ||
else | ||
# create new issue | ||
INITIAL_BODY=$'# Test failure history\n\n'"${entry}" | ||
issue_url=$(gh issue create -t "$title" -b "$INITIAL_BODY" -R $ISSUES_REPOSITORY) | ||
echo "Created issue ${issue_url}." | ||
item_id=$(gh project item-add $PROJECT_NUMBER --owner "airbytehq" --url "$issue_url" --format json --jq '.id') | ||
project_id=$(gh project view $PROJECT_NUMBER --owner "airbytehq" --format json --jq '.id') | ||
gh project item-edit \ | ||
--id $item_id \ | ||
--project-id $project_id \ | ||
--field-id $STATUS_FIELD_ID \ | ||
--single-select-option-id "$ONCALL_OPTION_ID" | ||
echo "Created new issue for $connector_name." | ||
fi |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
note that this is only to notify the move team. The logic to create a gh issue changes quite a bit depending on how teams manage their oncall tickets so I thought it made sense to just do it for the ones we own (and also by doing this we are not making other team sign up to how we want to deal with these issues).