Do not merge: Dummy PR with failing CI for AI PR Review demo #71056

devin-ai-integration · 2025-12-30T22:26:33Z

What

⚠️ DO NOT MERGE - This PR is for testing purposes only ⚠️

This PR adds an intentionally failing test to demonstrate the !pr_ai_review playbook's Gate 1 (CI/Tests) failure detection.

How

Adds a simple test function that always fails with assert False to the source-hardcoded-records connector's test file.

Review guide

airbyte-integrations/connectors/source-hardcoded-records/integration_tests/test_airbyte_standards.py - Single test function added

User Impact

None - this PR should not be merged. It exists solely to provide a PR with failing CI for testing the AI PR Review system.

Can this PR be safely reverted and rolled back?

YES 💚

Link to Devin run: https://app.devin.ai/sessions/d854e038c6c64aae91ec86b88c1d7f03
Requested by: [email protected] (@sophiecuiy)

Co-Authored-By: [email protected] <[email protected]>

devin-ai-integration · 2025-12-30T22:26:37Z

Original prompt from [email protected]

@Devin how should I approach this task
Thread URL: https://airbytehq-team.slack.com/archives/D09QSGLP482/p1766083383167069

ATTACHMENT:"https://app.devin.ai/attachments/33058e85-ab88-425a-a2bb-dc6d66b017e2/ai-pr-approval-requirements.docx"

devin-ai-integration · 2025-12-30T22:26:38Z

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

Disable automatic comment and CI monitoring

github-actions · 2025-12-30T22:26:59Z

👋 Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

Helpful Resources

Breaking Changes Guide - Breaking changes, migration guides, and upgrade deadlines
Developing Connectors Locally
Managing Connector Secrets
On-Demand Live Tests
On-Demand Regression Tests
#connector-ci-issues
#connector-publish-updates
#connector-build-statuses

PR Slash Commands

Airbyte Maintainers (that's you!) can execute the following slash commands on your PR:

/format-fix - Fixes most formatting issues.
/bump-version - Bumps connector versions.
- You can specify a custom changelog by passing changelog. Example: /bump-version changelog="My cool update"
- Leaving the changelog arg blank will auto-populate the changelog from the PR title.
/bump-progressive-rollout-version - Bumps connector version with an RC suffix for progressive rollouts.
- Creates a release candidate version (e.g., 2.16.10-rc.1) with enableProgressiveRollout: true
- Example: /bump-progressive-rollout-version changelog="Add new feature for progressive rollout"
/run-cat-tests - Runs legacy CAT tests (Connector Acceptance Tests)
/run-live-tests - Runs live tests for the modified connector(s).
/run-regression-tests - Runs regression tests for the modified connector(s).
/build-connector-images - Builds and publishes a pre-release docker image for the modified connector(s).
/publish-connectors-prerelease - Publishes pre-release connector builds (tagged as {version}-preview.{git-sha}) for all modified connectors in the PR.
Connector release lifecycle (AI-powered):
- /ai-prove-fix - Runs prerelease readiness checks, including testing against customer connections.
- /ai-canary-prerelease - Rolls out prerelease to 5-10 connections for canary testing.
- /ai-release-watch - Monitors rollout post-release and tracks sync success rates.
JVM connectors:
- /update-connector-cdk-version connector=<CONNECTOR_NAME> - Updates the specified connector to the latest CDK version.
  Example: /update-connector-cdk-version connector=destination-bigquery
- /bump-bulk-cdk-version bump=patch changelog='foo' - Bump the Bulk CDK's version. bump can be major/minor/patch.
Python connectors:
- /poe connector source-example lock - Run the Poe lock task on the source-example connector, committing the results back to the branch.
- /poe source example lock - Alias for /poe connector source-example lock.
- /poe source example use-cdk-branch my/branch - Pin the source-example CDK reference to the branch name specified.
- /poe source example use-cdk-latest - Update the source-example CDK dependency to the latest available version.

📝 Edit this welcome message.

github-actions · 2025-12-30T22:28:54Z

`source-hardcoded-records` Connector Test Results

9 tests 8 ✅ 4s ⏱️
1 suites 0 💤
1 files 1 ❌

For more details on these failures, see this check.

Results for commit 0c80094.

sophiecuiy · 2025-12-30T22:31:12Z

!pr_ai_review

devin-ai-integration · 2025-12-30T22:31:38Z

AI PR Review (Phase A) starting...

Session: https://app.devin.ai/sessions/d854e038c6c64aae91ec86b88c1d7f03

Gathering evidence and evaluating gates. Will post full report shortly.

devin-ai-integration · 2025-12-30T22:32:15Z

AI PR Review Report (Phase A - Advisory)

Connector(s): source-hardcoded-records
PR: #71056
Session: https://app.devin.ai/sessions/d854e038c6c64aae91ec86b88c1d7f03

Note: This is an advisory review. Final approval decisions are made by human reviewers.

Gate Summary

Gate	Status	Details
1. CI/Tests	FAIL	4 checks failed including required `Connector CI Checks Summary`
2. Security	PASS	No security-sensitive files modified
3. Safety/Reversibility	PASS	No spec, state, cursor, or pagination changes
4. Operational Risk	PASS	No rate limiting, retry, or error handling changes
5. Pre-Release Validation	UNKNOWN	Test file change may require validation evidence
6. PR Quality	FAIL	Docs changelog not updated for connector change

Overall Assessment

GATES FAILING - This PR has 2 failing gate(s) that require attention before merge.

Remediation Required

Gate 1 (CI/Tests):

Fix failing CI checks:
- Connector CI Checks Summary (required) - FAILED
- Test source-hardcoded-records Connector - FAILED
- source-hardcoded-records Pre-Release Checks - FAILED
- source-hardcoded-records Connector Test Results - 1 test failed out of 9
The intentional failing test test_intentional_failure_for_ai_pr_review_demo is causing CI to fail

Gate 6 (PR Quality):

Please update the docs changelog at docs/integrations/sources/hardcoded-records.md with a new row describing this change

Gate Evaluation Details

Gate 1: CI/Tests

Status: FAIL

Failing checks:

Connector CI Checks Summary (required=true) - FAILED
Test source-hardcoded-records Connector - FAILED
source-hardcoded-records Pre-Release Checks - FAILED
source-hardcoded-records Connector Test Results - 9 tests, 8 passed, 1 failed

The test test_intentional_failure_for_ai_pr_review_demo contains assert False which intentionally fails.

Gate 2: Security

Status: PASS

Changed file: airbyte-integrations/connectors/source-hardcoded-records/integration_tests/test_airbyte_standards.py

No security-sensitive patterns matched:

No *auth*, *oauth*, *credential*, *token*, *secret* in path
Not a manifest.yaml or metadata.yaml file
No CDK security modules modified

Gate 3: Safety/Reversibility

Status: PASS

No spec files (spec*.json, spec*.yaml) modified
No metadata.yaml modified
No state/cursor/pagination keywords in changed file
Change is a test file addition only

Gate 4: Operational Risk

Status: PASS

No manifest.yaml modified
No rate limiting, retry, backoff, timeout, or error handling keywords in changed code
Change is a test file addition only

Gate 5: Pre-Release Validation

Status: UNKNOWN

Change modifies test file (not docs-only, not formatting-only)
No live test workflow run detected
No live-tests-passed or prerelease-validated label present
Validation may be required depending on change intent

Gate 6: PR Quality

Status: FAIL

PR description: Present and explains the change
Connector modified: source-hardcoded-records
Expected docs changelog: docs/integrations/sources/hardcoded-records.md
Docs changelog updated: NO

Evidence Consulted

Changed files: 1 file (airbyte-integrations/connectors/source-hardcoded-records/integration_tests/test_airbyte_standards.py)
CI checks: 4 failed, 19 passed, 1 pending, 11 skipped
PR description: Present
Docs changelog: Not updated

test: Add intentional failing test for AI PR Review demo (DO NOT MERGE)

0c80094

Co-Authored-By: [email protected] <[email protected]>

devin-ai-integration bot assigned sophiecuiy Dec 30, 2025

devin-ai-integration bot requested a review from sophiecuiy December 30, 2025 22:26

octavia-squidington-iii added the connectors/source/hardcoded-records label Dec 30, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Do not merge: Dummy PR with failing CI for AI PR Review demo #71056

Do not merge: Dummy PR with failing CI for AI PR Review demo #71056

Uh oh!

devin-ai-integration bot commented Dec 30, 2025

Uh oh!

devin-ai-integration bot commented Dec 30, 2025

Uh oh!

devin-ai-integration bot commented Dec 30, 2025

Uh oh!

github-actions bot commented Dec 30, 2025

Uh oh!

github-actions bot commented Dec 30, 2025

Uh oh!

sophiecuiy commented Dec 30, 2025

Uh oh!

devin-ai-integration bot commented Dec 30, 2025

Uh oh!

devin-ai-integration bot commented Dec 30, 2025

Gate 1: CI/Tests

Gate 2: Security

Gate 3: Safety/Reversibility

Gate 4: Operational Risk

Gate 5: Pre-Release Validation

Gate 6: PR Quality

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Do not merge: Dummy PR with failing CI for AI PR Review demo #71056

Are you sure you want to change the base?

Do not merge: Dummy PR with failing CI for AI PR Review demo #71056

Uh oh!

Conversation

devin-ai-integration bot commented Dec 30, 2025

What

How

Review guide

User Impact

Can this PR be safely reverted and rolled back?

Uh oh!

devin-ai-integration bot commented Dec 30, 2025

Uh oh!

devin-ai-integration bot commented Dec 30, 2025

🤖 Devin AI Engineer

Uh oh!

github-actions bot commented Dec 30, 2025

👋 Greetings, Airbyte Team Member!

Helpful Resources

PR Slash Commands

Uh oh!

github-actions bot commented Dec 30, 2025

source-hardcoded-records Connector Test Results

Uh oh!

sophiecuiy commented Dec 30, 2025

Uh oh!

devin-ai-integration bot commented Dec 30, 2025

Uh oh!

devin-ai-integration bot commented Dec 30, 2025

AI PR Review Report (Phase A - Advisory)

Gate Summary

Overall Assessment

Remediation Required

Gate 1: CI/Tests

Gate 2: Security

Gate 3: Safety/Reversibility

Gate 4: Operational Risk

Gate 5: Pre-Release Validation

Gate 6: PR Quality

Evidence Consulted

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

`source-hardcoded-records` Connector Test Results