Skip to content

Conversation

aaronsteers
Copy link
Contributor

@aaronsteers aaronsteers commented Sep 4, 2025

ci: Add GitHub action linting workflow

Summary

Adds actionlint workflow to validate GitHub Actions workflow files using reviewdog for PR annotations. The workflow runs conditionally only when .github/workflows/** files are modified, providing automated linting feedback directly on pull requests.

Key features:

  • Uses reviewdog/[email protected] for actionlint + PR annotations
  • Conditional triggers via paths filter to run only on workflow file changes
  • Reporter set to github-pr-review for inline PR comments
  • Includes built-in shellcheck and pyflakes integration
  • Follows existing repo patterns for runner selection

Review & Testing Checklist for Human

  • Verify conditional path triggers work correctly - Test that the workflow triggers when modifying workflow files and doesn't trigger for other changes
  • Test reviewdog PR annotations - Create a workflow file with intentional actionlint errors to verify reviewdog posts inline comments on the PR
  • Check permissions - Ensure secrets.GITHUB_TOKEN has sufficient permissions for reviewdog to post PR comments
  • Validate runner selection - Confirm ubuntu-latest (PyAirbyte) vs ubuntu-24.04 (CDK/MCP) runner choices match repo conventions

Test Plan

  1. Create a test PR that modifies a workflow file with actionlint violations
  2. Verify the actionlint workflow triggers automatically
  3. Confirm reviewdog posts PR review comments highlighting the issues
  4. Test that non-workflow changes don't trigger the actionlint workflow

Notes

  • This workflow will only run when workflow files are modified, keeping noise low
  • actionlint includes enhanced analysis with shellcheck and pyflakes by default
  • First implementation of reviewdog in these repositories - monitoring needed for any permission/integration issues

Requested by: @aaronsteers
Link to Devin run: https://app.devin.ai/sessions/064c9d5b289f41539aa6e5f6acfbb8cf

Summary by CodeRabbit

  • Chores
    • Added automated linting for GitHub Actions workflows, running on pushes to default branch and on PRs that modify workflow files.
    • Reports issues directly in PR reviews with error-level enforcement to surface workflow syntax problems early.
    • Includes a test step that intentionally introduces a malformed workflow entry to verify the linter flags errors; may cause CI to report failures.
    • No user-facing UI changes; improves CI quality and reduces deployment risks.

Copy link
Contributor

Original prompt from AJ Steers
Received message in Slack channel #ask-devin-ai:

@Devin - apparently there's an 'actionlint' tool for linting GitHub actions. Can you research it and then research a GitHub action we can use to run it? 

I'd like to use this as an opportunity to test a very popular action that does linting and formatting and test annotations directly on the PR. I don't remember what that tool/action was called but if you find it I may recognize the name.

Copy link
Contributor

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

Copy link

github-actions bot commented Sep 4, 2025

👋 Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

Testing This PyAirbyte Version

You can test this version of PyAirbyte using the following:

# Run PyAirbyte CLI from this branch:
uvx --from 'git+https://github.com/airbytehq/PyAirbyte.git@devin/1757017385-add-actionlint-workflow' pyairbyte --help

# Install PyAirbyte from this branch for development:
pip install 'git+https://github.com/airbytehq/PyAirbyte.git@devin/1757017385-add-actionlint-workflow'

Helpful Resources

PR Slash Commands

Airbyte Maintainers can execute the following slash commands on your PR:

  • /fix-pr - Fixes most formatting and linting issues
  • /poetry-lock - Updates poetry.lock file
  • /test-pr - Runs tests with the updated PyAirbyte

Community Support

Questions? Join the #pyairbyte channel in our Slack workspace.

📝 Edit this welcome message.

Copy link

github-actions bot commented Sep 4, 2025

PyTest Results (Fast Tests Only, No Creds)

301 tests  +25   301 ✅ +26   5m 1s ⏱️ + 1m 43s
  1 suites ± 0     0 💤 ± 0 
  1 files   ± 0     0 ❌  -  1 

Results for commit 38e1618. ± Comparison against base commit 43e5af7.

♻️ This comment has been updated with latest results.

Copy link
Contributor

coderabbitai bot commented Sep 4, 2025

Warning

Rate limit exceeded

@devin-ai-integration[bot] has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 4 minutes and 11 seconds before requesting another review.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

📥 Commits

Reviewing files that changed from the base of the PR and between d0ea7f6 and 38e1618.

📒 Files selected for processing (1)
  • .github/workflows/actionlint.yml (1 hunks)
📝 Walkthrough

Walkthrough

Adds a new GitHub Actions workflow file .github/workflows/actionlint.yml named "GitHub action linting" that triggers on pushes to main and PRs affecting .github/workflows/**, runs actionlint via reviewdog, and includes an extra step that references a nonexistent action to produce an actionlint error.

Changes

Cohort / File(s) Summary
CI: actionlint workflow
.github/workflows/actionlint.yml
Adds "GitHub action linting" workflow: triggers on push to main and PRs touching .github/workflows/**; permissions contents: read, pull-requests: write; job actionlint on ubuntu-latest; steps: checkout (actions/checkout@v4), run reviewdog/[email protected] with github_token: ${{ secrets.GITHUB_TOKEN }}, reporter: github-pr-review, level: error; adds a "Test step with actionlint error" that uses nonexistent/invalid-action@v999 with fake_parameter: test to intentionally surface actionlint errors.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  actor Dev as Developer
  participant GH as GitHub
  participant WF as Workflow: actionlint.yml
  participant RD as reviewdog/action-actionlint
  participant PR as PR Review

  Dev->>GH: Push to `main` or open PR affecting `.github/workflows/**`
  GH-->>WF: Trigger workflow
  WF->>WF: Checkout repository (actions/checkout@v4)
  WF->>RD: Invoke actionlint via reviewdog (reporter=github-pr-review, level=error)
  alt Workflow steps valid but action definitions invalid
    RD-->>PR: Post actionlint findings as PR review comments
    Note right of PR: Reviewdog flags invalid action usage (e.g., nonexistent/invalid-action@v999)
  else Workflow fails to parse
    WF-->>GH: Workflow parsing fails / job does not run
    GH-->>Dev: Notify of workflow parse error
  end
Loading

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Would you like to also enforce this on all branches or add a scheduled run to catch issues outside PRs, wdyt?

✨ Finishing Touches
🧪 Generate unit tests
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch devin/1757017385-add-actionlint-workflow

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

‼️ IMPORTANT
Auto-reply has been disabled for this repository in the CodeRabbit settings. The CodeRabbit bot will not respond to your replies unless it is explicitly tagged.

  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

CodeRabbit Commands (Invoked using PR/Issue comments)

Type @coderabbitai help to get the list of available commands.

Other keywords and placeholders

  • Add @coderabbitai ignore or @coderabbit ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai or @coderabbitai title anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Status, Documentation and Community

  • Visit our Status Page to check the current availability of CodeRabbit.
  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (4)
.github/workflows/actionlint.yml (4)

3-11: Event triggers: drop push or add conditional

Given the reporter is github-pr-review, running on push to main won’t post annotations. Would you prefer we: (a) remove the push trigger, or (b) keep it but rely on the job-level if: github.event_name == 'pull_request' added above, wdyt?


19-21: Pin actions to commit SHAs for supply-chain hardening

Would you be open to pinning actions/checkout@v4 and reviewdog/[email protected] to their immutable commit SHAs (and rely on Dependabot to bump), wdyt?


1-1: Nit: workflow name

Minor: consider a crisper name like “Actionlint” or “Lint GitHub Actions workflows” for readability, wdyt?


21-25: Forked PRs: do we need annotations there?

If you want review comments on PRs from forks, we may need a pull_request_target variant with safe checkout (and keep linting strictly to workflow files). Do we want to support that now or defer until needed, wdyt?

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

💡 Knowledge Base configuration:

  • MCP integration is disabled by default for public repositories
  • Jira integration is disabled by default for public repositories
  • Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between 43e5af7 and c22c2a0.

📒 Files selected for processing (1)
  • .github/workflows/actionlint.yml (1 hunks)
🧰 Additional context used
🪛 GitHub Check: CodeQL
.github/workflows/actionlint.yml

[warning] 15-25: Workflow does not contain permissions
Actions job or workflow does not limit the permissions of the GITHUB_TOKEN. Consider setting an explicit permissions block, using the following as a minimal starting point: {{contents: read}}

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (7)
  • GitHub Check: Pytest (All, Python 3.11, Ubuntu)
  • GitHub Check: Pytest (All, Python 3.11, Windows)
  • GitHub Check: Pytest (All, Python 3.10, Ubuntu)
  • GitHub Check: Pytest (All, Python 3.10, Windows)
  • GitHub Check: MyPy Check
  • GitHub Check: Pytest (No Creds)
  • GitHub Check: Pytest (Fast)
🔇 Additional comments (1)
.github/workflows/actionlint.yml (1)

16-16: Stick with ubuntu-latest here; only welcome-message.yml uses ubuntu-24.04 (line 27). Should we update that instead for consistency, wdyt?

Likely an incorrect or invalid review comment.

Copy link

github-actions bot commented Sep 4, 2025

PyTest Results (Full)

364 tests  ±0   348 ✅ ±0   20m 58s ⏱️ -40s
  1 suites ±0    16 💤 ±0 
  1 files   ±0     0 ❌ ±0 

Results for commit 38e1618. ± Comparison against base commit 43e5af7.

♻️ This comment has been updated with latest results.

devin-ai-integration bot and others added 3 commits September 5, 2025 00:12
…testing

- Add permissions block with contents:read and pull-requests:write
- Replace invalid parameter with YAML syntax error to test reviewdog annotations
- Addresses GitHub Advanced Security bot suggestion about missing permissions

Co-Authored-By: AJ Steers <[email protected]>
- Fix YAML syntax to allow workflow parsing by GitHub Actions
- Use nonexistent action reference that actionlint can detect and report
- This should trigger proper reviewdog PR annotations for testing

Co-Authored-By: AJ Steers <[email protected]>
@aaronsteers aaronsteers merged commit d0d5268 into main Sep 9, 2025
22 checks passed
@aaronsteers aaronsteers deleted the devin/1757017385-add-actionlint-workflow branch September 9, 2025 19:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant