feat: add poe task to run GitHub Actions workflows locally with wrkflw #753
Conversation
- Add run-workflow poe task that accepts workflow filename parameter - Uses wrkflw tool in emulation mode for local CI job execution - Tested with python_lint.yml workflow (successfully parses and executes) - Created scripts/run_workflow.py with proper error handling and help - Note: Workflow execution success depends on local environment setup Co-Authored-By: AJ Steers <[email protected]>
Original prompt from AJ Steers |
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
👋 Greetings, Airbyte Team Member!Here are some helpful tips and reminders for your convenience. Testing This PyAirbyte VersionYou can test this version of PyAirbyte using the following: # Run PyAirbyte CLI from this branch:
uvx --from 'git+https://github.com/airbytehq/PyAirbyte.git@devin/1755389443-test-wrkflw-tool' pyairbyte --help
# Install PyAirbyte from this branch for development:
pip install 'git+https://github.com/airbytehq/PyAirbyte.git@devin/1755389443-test-wrkflw-tool'Helpful ResourcesPR Slash CommandsAirbyte Maintainers can execute the following slash commands on your PR:
Community SupportQuestions? Join the #pyairbyte channel in our Slack workspace. |
There was a problem hiding this comment.
Pull Request Overview
This PR integrates the wrkflw tool into PyAirbyte's development workflow by adding a new poe task for running GitHub Actions workflows locally. The implementation provides a user-friendly interface with argument validation, error handling, and automatic workflow discovery.
- Adds a new
run-workflowpoe task that accepts workflow filenames as arguments - Creates a Python script with comprehensive error handling and user-friendly messages
- Enables local execution of GitHub Actions workflows using wrkflw's emulation mode
Reviewed Changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
scripts/__init__.py |
Creates scripts package for development tools |
scripts/run_workflow.py |
Main script implementing workflow execution logic with validation and error handling |
pyproject.toml |
Adds new poe task entry point for the workflow runner |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
|
Note Other AI code review bot(s) detectedCodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review. 📝 WalkthroughWalkthroughAdds a Poe include in Changes
Sequence Diagram(s)sequenceDiagram
participant User
participant Poe
participant Shell
participant wrkflw
User->>Poe: poe wrkflw / ci-test / ci-test-workflow / ci-validate [args]
Poe->>Shell: execute task shell script
Shell->>Shell: check PATH for wrkflw
alt wrkflw missing
Shell->>Shell: install wrkflw via brew
end
Shell->>wrkflw: run wrkflw with provided args or workflow-specific command
wrkflw-->>User: output/results
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Would you like to add a quick validation step (e.g., fail fast if the referenced workflow file in 📜 Recent review detailsConfiguration used: CodeRabbit UI 💡 Knowledge Base configuration:
You can enable these sources in your CodeRabbit configuration. 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (7)
✨ Finishing Touches🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. CodeRabbit Commands (Invoked using PR/Issue comments)Type Other keywords and placeholders
CodeRabbit Configuration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (4)
scripts/run_workflow.py (4)
18-20: Accept both .yml and .yaml extensionsIf a user passes “.yaml”, we currently append “.yml”, yielding “.yaml.yml”. Should we accept both suffixes to make this more robust, wdyt?
- if not workflow_file.endswith(".yml"): + if not (workflow_file.endswith(".yml") or workflow_file.endswith(".yaml")): workflow_file += ".yml"Try:
poe run-workflow python_lint.yamlafter the change.
23-30: Print errors to stderr and list .yaml workflows as wellRouting error messages to stderr makes CLI behavior friendlier in pipelines. Also, should we list “.yaml” files too for discovery, wdyt?
- if not workflow_path.exists(): - print(f"Error: Workflow file {workflow_path} does not exist") - print("Available workflows:") + if not workflow_path.exists(): + print(f"Error: Workflow file {workflow_path} does not exist", file=sys.stderr) + print("Available workflows:", file=sys.stderr) workflows_dir = Path(".github/workflows") if workflows_dir.exists(): - for yml_file in workflows_dir.glob("*.yml"): - print(f" {yml_file.name}") + # List both .yml and .yaml files + candidates = list(workflows_dir.glob("*.yml")) + list(workflows_dir.glob("*.yaml")) + for wf in sorted(candidates): + print(f" {wf.name}", file=sys.stderr) sys.exit(1)
34-43: Route failure messages to stderr and preserve exit codesMinor ergonomics: print errors to stderr while preserving original exit codes. Would you consider this tweak, wdyt?
print(f"Running: {' '.join(cmd)}") try: result = subprocess.run(cmd, check=True) sys.exit(result.returncode) except subprocess.CalledProcessError as e: - print(f"Error running workflow: {e}") + print(f"Error running workflow: {e}", file=sys.stderr) sys.exit(e.returncode) except FileNotFoundError: - print("Error: wrkflw command not found. Please ensure wrkflw is installed.") + print("Error: wrkflw command not found. Please ensure wrkflw is installed.", file=sys.stderr) sys.exit(1)
9-47: Optional: Use argparse for UX and to avoid argv countingTotally optional, but
argparsegives built-in help, clearer errors, and avoids PLR2004 entirely. Would you like to switch to argparse, wdyt?Example refactor (for reference only):
import argparse def main() -> None: parser = argparse.ArgumentParser(description="Run a GitHub Actions workflow locally using wrkflw.") parser.add_argument("workflow_filename", help="Workflow filename under .github/workflows (with or without .yml/.yaml)") args = parser.parse_args() workflow_file = args.workflow_filename if not (workflow_file.endswith(".yml") or workflow_file.endswith(".yaml")): workflow_file += ".yml" # ... carry on as beforeHappy to convert the file to argparse and update messages if you’d like me to push a follow-up commit.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
💡 Knowledge Base configuration:
- MCP integration is disabled by default for public repositories
- Jira integration is disabled by default for public repositories
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (3)
pyproject.toml(1 hunks)scripts/__init__.py(1 hunks)scripts/run_workflow.py(1 hunks)
🧰 Additional context used
🪛 GitHub Actions: Run Linters
scripts/run_workflow.py
[error] 1-1: poetry run ruff check: EXE001 Shebang is present but file is not executable.
[error] 1-1: poetry run ruff check: CPY001 Missing copyright notice at top of file.
[error] 9-9: poetry run ruff check: ANN201 Missing return type annotation for public function main.
[error] 11-11: poetry run ruff check: PLR2004 Magic value used in comparison, consider replacing 2 with a constant variable.
scripts/__init__.py
[error] 1-1: poetry run ruff check: CPY001 Missing copyright notice at top of file.
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (6)
- GitHub Check: Pytest (All, Python 3.10, Ubuntu)
- GitHub Check: Pytest (All, Python 3.11, Ubuntu)
- GitHub Check: Pytest (All, Python 3.10, Windows)
- GitHub Check: Pytest (All, Python 3.11, Windows)
- GitHub Check: Pytest (Fast)
- GitHub Check: Pytest (No Creds)
🔇 Additional comments (3)
scripts/run_workflow.py (2)
8-14: Define a named constant for the expected argv length (PLR2004)
Ruff flags the literal2in thelen(sys.argv) != 2check. Could we introduce a module-level constant to address PLR2004? For example:from pathlib import Path +EXPECTED_ARGC = 2 # script name + workflow filename + def main() -> None: """Run a GitHub Actions workflow locally using wrkflw.""" - if len(sys.argv) != 2: + if len(sys.argv) != EXPECTED_ARGC: print("Usage: poe run-workflow <workflow-filename>") print("Example: poe run-workflow python_lint.yml") sys.exit(1)Wdyt?
[optional_refactors_recommended]
1-3: Shebang & Header Alignment in scripts/run_workflow.py
It looks like this file still has a#!/usr/bin/env python3but isn’t marked executable (mode 100644), and we haven’t adopted SPDX headers anywhere else in the repo. Could you:
- Either remove the shebang (to satisfy EXE001) or mark the file executable (
chmod +x) if it’s meant to be run directly?- Decide on a consistent copyright header style—do we want to start embedding
at the top of Python modules, or stick with the existing “© YYYY Airbyte, Inc., all rights reserved.” pattern we see in tests?# Copyright (c) 2025 Airbyte, Inc. and contributors # SPDX-License-Identifier: Apache-2.0WDYT?
pyproject.toml (1)
199-199: Poe task wiring looks goodThe new poe task uses the
scriptrunner with a helpful description. This should integrate cleanly withpoe run-workflow .... LGTM, wdyt?
- Add return type annotation to main() function - Add copyright notice header - Replace magic number with EXPECTED_ARGS constant - Fix import sorting with ruff --fix Co-Authored-By: AJ Steers <[email protected]>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Full Execution Log for
|
…rkflw - Remove scripts/run_workflow.py and scripts/__init__.py - Update poe task to use shell command with Docker + cargo install - Simplifies dependency management by using official rust Docker image - Maintains same usage pattern: poe run-workflow workflow=python_lint.yml Co-Authored-By: AJ Steers <[email protected]>
…w installation - Replace rust:latest with python:3.12 base image to provide Python/Poetry dependencies - Install Rust via rustup and wrkflw via cargo within Python container - Fix shell sourcing issue by using dot notation instead of source command - Successfully tested with python_lint.yml workflow - all jobs pass - Usage: WORKFLOW=python_lint.yml poe run-workflow Co-Authored-By: AJ Steers <[email protected]>
- Remote branch still had scripts/__init__.py from earlier commits - Local branch correctly removed scripts/ directory when switching to Docker bash approach - Confirming deletion to complete merge resolution Co-Authored-By: AJ Steers <[email protected]>
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (4)
pyproject.toml (4)
199-199: Heavyweight Docker-in-Docker install path; consider faster, repeatable alternatives.Installing rustup+cargo+wrkflw on every invocation will be slow and network-flaky. Would you consider either (a) running wrkflw directly on host if available, (b) pointing to the Python runner which can check for wrkflw and provide guidance, or (c) using a prebuilt image with wrkflw preinstalled to avoid repeated setup, wdyt?
Here’s a lighter-weight variant if you want to keep the Docker path but avoid redundant installs (assumes you publish/standardize an image that contains wrkflw):
-run-workflow = { shell = "docker run --rm -v \"$(pwd)\":/workspace -w /workspace python:3.12 sh -c \"curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y && . ~/.cargo/env && cargo install wrkflw && pip install poetry && wrkflw run .github/workflows/$WORKFLOW --runtime emulation\"", help = "Run a GitHub Actions workflow locally using wrkflw (e.g., WORKFLOW=python_lint.yml poe run-workflow)" } +run-workflow = { shell = "docker run --rm -v \"$(pwd)\":/workspace -w /workspace ghcr.io/your-org/wrkflw:latest sh -c \"wrkflw run .github/workflows/$WORKFLOW --runtime emulation\"", help = "Run a GitHub Actions workflow locally using wrkflw (e.g., WORKFLOW=python_lint.yml poe run-workflow)" }
199-199: Cross-platform concern: $(pwd) and shell quoting may break on Windows.Using
$(pwd) assumes a POSIX shell. On Windows (non-WSL), Poe may invoke PowerShell/cmd where $ (pwd) won’t expand. Would you prefer routing to the Python runner (which avoids Docker entirely per PR goals), or at minimum documenting that this task requires WSL/macOS/Linux, wdyt?
199-199: Remove unnecessary “pip install poetry” inside the container.Poetry isn’t used in the container for running wrkflw. Dropping this will shave time and reduce surface area. Shall we remove it, wdyt?
Apply:
-docker run --rm -v "$(pwd)":/workspace -w /workspace python:3.12 sh -c "curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y && . ~/.cargo/env && cargo install wrkflw && pip install poetry && wrkflw run .github/workflows/$WORKFLOW --runtime emulation" +docker run --rm -v "$(pwd)":/workspace -w /workspace python:3.12 sh -c "curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y && . ~/.cargo/env && cargo install wrkflw && wrkflw run .github/workflows/$WORKFLOW --runtime emulation"
199-199: Consider parameterizing secrets/env passthrough for workflows that need them.If any local workflows require env vars (e.g., GITHUB_TOKEN) the container won’t see them. Would you like to pass through a small allowlist of envs or support an --env-file option in the Python runner, wdyt?
For Docker-based approach:
-docker run --rm -v "$(pwd)":/workspace -w /workspace ... +docker run --rm \ + -e GITHUB_TOKEN \ + --env-file .env.local \ + -v "$(pwd)":/workspace -w /workspace ...
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
💡 Knowledge Base configuration:
- MCP integration is disabled by default for public repositories
- Jira integration is disabled by default for public repositories
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (1)
pyproject.toml(1 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (6)
- GitHub Check: Pytest (All, Python 3.11, Ubuntu)
- GitHub Check: Pytest (All, Python 3.11, Windows)
- GitHub Check: Pytest (All, Python 3.10, Windows)
- GitHub Check: Pytest (All, Python 3.10, Ubuntu)
- GitHub Check: Pytest (Fast)
- GitHub Check: Pytest (No Creds)
🔇 Additional comments (2)
pyproject.toml (2)
199-199: Nice: defaults to emulation runtime and includes a helpful usage hint.The default to --runtime emulation aligns with the goal of avoiding Docker-based workflow runners, and the help string clarifies usage. Thanks for the DX touch!
199-199: Alignrun-workflowwith our Python runner?I wasn’t able to find
scripts/run_workflow.pyin the repo—are we planning to add it as part of this PR? Once that helper script exists, we could swap out the long Docker+wrkflw shell wrapper inpyproject.toml(line 199) for a direct Python call, e.g.:-run-workflow = { shell = "docker run --rm … wrkflw run .github/workflows/$WORKFLOW …", help = "Run a GitHub Actions workflow locally using wrkflw (e.g., WORKFLOW=python_lint.yml poe run-workflow)" } +run-workflow = { cmd = "python scripts/run_workflow.py", help = "Run a GitHub Actions workflow locally (e.g., poe run-workflow python_lint)" }That would give us built-in argparse support,
.ymlauto-append, workflow listing/discovery, and friendlier errors out of the box—wdyt?
- Create poe-tasks/wrkflw-poe-tasks.toml for reusability across repos - Update pyproject.toml to include external task file - Rename 'run-workflow' to 'wrkflw' with multi-line shell syntax - Add 'ci-test' as passthrough wrapper (opens TUI by default) - Add 'ci-test-workflow' for workflow filename behavior - Use WRKFLW_ARGS environment variable for argument passing - Successfully tested with WORKFLOW=python_lint.yml poe ci-test-workflow Co-Authored-By: AJ Steers <[email protected]>
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (2)
poe-tasks/wrkflw-poe-tasks.toml (2)
2-10: Container wrapper: quote PWD, pass env into container, and consider reproducibility/cachingA few tweaks can improve reliability and performance:
- Quote the bind mount path to handle spaces.
- Pass WRKFLW_ARGS via docker env instead of relying on host-side expansion.
- Optional: pin versions (poetry, wrkflw), ensure curl/build tools are present, and/or cache cargo/pip to avoid cold installs each run. Would you consider these for faster, more reproducible runs, wdyt?
Here’s a minimal patch for quoting and env passthrough:
-wrkflw = { shell = """ -docker run --rm -v $(pwd):/workspace -w /workspace python:3.12 sh -c " +wrkflw = { shell = """ +docker run --rm -v "$PWD":/workspace -w /workspace -e WRKFLW_ARGS python:3.12 sh -c " curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y && . ~/.cargo/env && - cargo install wrkflw && - pip install poetry && - wrkflw ${WRKFLW_ARGS:-} + cargo install wrkflw && + pip install poetry && + wrkflw ${WRKFLW_ARGS:-} " """, help = "Run wrkflw commands in Docker container with Rust/Poetry setup (use WRKFLW_ARGS env var for arguments)" }If you’re open to a bigger refactor, I can propose: (a) a small custom image pre-loading rustup, wrkflw, and poetry, or (b) named volumes like -v wrkflw_cargo:/root/.cargo and -v wrkflw_pip:/root/.cache/pip to persist caches. wdyt?
14-14: Make WORKFLOW arg handling more robust (auto-suffix, .yaml support, quoting, empty-check)Would you like to accept both names with/without extension, support .yml and .yaml, and quote the path to be safe? Also, consider aligning the task name with the PR objective (run-workflow) for clarity. Here’s an inline tweak that stays single-line:
-ci-test-workflow = { shell = "WRKFLW_ARGS=\"run .github/workflows/$WORKFLOW --runtime emulation\" poe wrkflw", help = "Run a GitHub Actions workflow locally (e.g., WORKFLOW=python_lint.yml poe ci-test-workflow)" } +ci-test-workflow = { shell = "WF=\"$WORKFLOW\"; [ -n \"${WF}\" ] || { echo 'ERROR: WORKFLOW env var is required'; exit 2; }; case \"$WF\" in *.yml|*.yaml) : ;; *) WF=\"${WF}.yml\" ;; esac; WRKFLW_ARGS=\"run .github/workflows/${WF} --runtime emulation\" poe wrkflw", help = "Run a GitHub Actions workflow locally (e.g., WORKFLOW=python_lint or WORKFLOW=python_lint.yml; supports .yml/.yaml)" }If preferred, I can also add a separate alias task named run-workflow that mirrors this behavior for discoverability. wdyt?
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
💡 Knowledge Base configuration:
- MCP integration is disabled by default for public repositories
- Jira integration is disabled by default for public repositories
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (2)
poe-tasks/wrkflw-poe-tasks.toml(1 hunks)pyproject.toml(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
- pyproject.toml
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (6)
- GitHub Check: Pytest (All, Python 3.11, Ubuntu)
- GitHub Check: Pytest (All, Python 3.11, Windows)
- GitHub Check: Pytest (No Creds)
- GitHub Check: Pytest (All, Python 3.10, Windows)
- GitHub Check: Pytest (All, Python 3.10, Ubuntu)
- GitHub Check: Pytest (Fast)
- Add ci-validate convenience wrapper around wrkflw's validate CLI command - Uses same environment variable pattern as other wrkflw tasks - Successfully tested with WORKFLOW=python_lint.yml poe ci-validate - Completes the wrkflw integration with all requested task variants Co-Authored-By: AJ Steers <[email protected]>
- Fixed shell command quoting to prevent TOML parsing errors - Changed from double quotes with escaping to single quotes - Successfully tested with poe ci-test-workflow python_lint.yml - All 3 linting jobs (ruff-lint-check, mypy-check, ruff-format-check) pass Co-Authored-By: AJ Steers <[email protected]>
- Use conditional brew install pattern: if ! command -v wrkflw; then brew install wrkflw; fi
- Fix variable substitution using ${workflow} syntax in poe tasks
- Add --runtime emulation flag for reliable execution without Docker dependencies
- All three tasks now working: ci-test, ci-test-workflow, ci-validate
- Successfully tested workflow validation and TUI interface
Co-Authored-By: AJ Steers <[email protected]>
There was a problem hiding this comment.
Actionable comments posted: 0
♻️ Duplicate comments (1)
poe-tasks/wrkflw-poe-tasks.toml (1)
11-19: Route via the wrkflw wrapper to avoid duplication and ensure consistent install behavior?This duplicates the install-check logic and still calls the host binary directly. If we always go through the
wrkflwwrapper task, we keep logic DRY and consistent. Could we delegate to the wrapper, wdyt?Apply this diff:
[tool.poe.tasks.ci-test] help = "Run wrkflw (opens TUI if no args provided)" -shell = """ -if ! command -v wrkflw &> /dev/null; then - echo "Installing wrkflw via brew..." - brew install wrkflw -fi -wrkflw -""" +shell = "poe wrkflw"
🧹 Nitpick comments (5)
poe-tasks/wrkflw-poe-tasks.toml (5)
3-9: Brew-only install will fail off macOS; add cross-platform fallback and explicit arg pass-through?Relying solely on Homebrew will break for Linux/Windows devs and for macOS machines without brew. Also, the use of "$@" assumes Poe forwards arbitrary args to shell tasks, which can be brittle without explicit args config. Shall we add a cargo fallback and make arg passing explicit via Poe task args, wdyt?
Apply this diff to improve portability and clarity:
[tool.poe.tasks.wrkflw] help = "Run wrkflw commands with automatic brew installation if needed" +args = [ + { name = "args", positional = true, variadic = true, help = "Arguments passed through to wrkflw" } +] shell = """ -if ! command -v wrkflw &> /dev/null; then - echo "Installing wrkflw via brew..." - brew install wrkflw -fi -wrkflw "$@" +set -e +if ! command -v wrkflw &> /dev/null; then + if command -v brew &> /dev/null; then + echo "Installing wrkflw via brew..." + brew install wrkflw + elif command -v cargo &> /dev/null; then + echo "Installing wrkflw via cargo..." + cargo install wrkflw + else + echo "Error: wrkflw not found and neither brew nor cargo is installed." >&2 + echo "Please install wrkflw: https://github.com/prefix-dev/wrkflw#installation" >&2 + exit 1 + fi +fi +wrkflw ${args} """
22-29: Auto-append “.yml”, validate existence, and reuse wrapper for DRYness?Per the PR objectives, the filename should accept bare names and auto-append “.yml”. Also, checking the file exists and listing available workflows would improve UX. Can we centralize invocation through the wrapper and add these niceties, wdyt?
Apply this diff:
[tool.poe.tasks.ci-test-workflow] -help = "Run a GitHub Actions workflow locally (e.g., poe ci-test-workflow python_lint.yml)" -shell = """ -if ! command -v wrkflw &> /dev/null; then - echo "Installing wrkflw via brew..." - brew install wrkflw -fi -wrkflw run --runtime emulation .github/workflows/${workflow} -""" +help = "Run a GitHub Actions workflow locally (e.g., poe ci-test-workflow python_lint or python_lint.yml)" +shell = """ +set -e +workflow="${workflow}" +case "$workflow" in + *.yml|*.yaml) ;; # keep as-is + *) workflow="${workflow}.yml" ;; +esac +path=".github/workflows/${workflow}" +if [ ! -f "$path" ]; then + echo "Workflow '${workflow}' not found at ${path}" >&2 + echo "Available workflows:" >&2 + ls -1 .github/workflows >&2 || true + exit 1 +fi +poe wrkflw run --runtime emulation "$path" +"""
35-42: Mirror the workflow UX in validate: add “.yml” auto-append, existence check, and wrapper reuse?For consistency, could we apply the same auto-append and validation behavior here, and delegate to the wrapper, wdyt?
Apply this diff:
[tool.poe.tasks.ci-validate] -help = "Validate a GitHub Actions workflow file (e.g., poe ci-validate python_lint.yml)" -shell = """ -if ! command -v wrkflw &> /dev/null; then - echo "Installing wrkflw via brew..." - brew install wrkflw -fi -wrkflw validate .github/workflows/${workflow} -""" +help = "Validate a GitHub Actions workflow file (e.g., poe ci-validate python_lint or python_lint.yml)" +shell = """ +set -e +workflow="${workflow}" +case "$workflow" in + *.yml|*.yaml) ;; + *) workflow="${workflow}.yml" ;; +esac +path=".github/workflows/${workflow}" +if [ ! -f "$path" ]; then + echo "Workflow '${workflow}' not found at ${path}" >&2 + echo "Available workflows:" >&2 + ls -1 .github/workflows >&2 || true + exit 1 +fi +poe wrkflw validate "$path" +"""
28-28: Quote the workflow path to avoid globbing/whitespace issues?Quoting the path is safer even if filenames don’t contain spaces. Shall we wrap it in quotes, wdyt?
Apply this diff:
-wrkflw run --runtime emulation .github/workflows/${workflow} +wrkflw run --runtime emulation ".github/workflows/${workflow}"
1-46: Consider wiring the Python helper for workflow tasks?I noticed the PR description mentions a
scripts/run_workflow.pyhelper that handles automatic “.yml” suffixing and friendly listing on invalid filenames, but our verification shows:
scripts/run_workflow.pydoesn’t exist in the repo- The TOML tasks in
poe-tasks/wrkflw-poe-tasks.tomlstill use pure shell logic without suffix/appending or discovery UXpyproject.tomlcorrectly includes thewrkflw-poe-tasks.tomlfileDo you have the Python script wired up elsewhere, or would you prefer folding that suffix and discovery behavior into these tasks via a new
run-workflowhelper? wdyt? I’m happy to add a small Python stub plus apoe run-workflowtask in a follow-up PR if that helps.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
💡 Knowledge Base configuration:
- MCP integration is disabled by default for public repositories
- Jira integration is disabled by default for public repositories
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (1)
poe-tasks/wrkflw-poe-tasks.toml(1 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (6)
- GitHub Check: Pytest (All, Python 3.11, Ubuntu)
- GitHub Check: Pytest (No Creds)
- GitHub Check: Pytest (All, Python 3.10, Ubuntu)
- GitHub Check: Pytest (All, Python 3.11, Windows)
- GitHub Check: Pytest (All, Python 3.10, Windows)
- GitHub Check: Pytest (Fast)
|
/test-pr
|
|
/format-pr |
- Variable substitution now working correctly with parameter - Default container runtime provides more accurate CI simulation - Successfully tested with python_lint.yml workflow execution Co-Authored-By: AJ Steers <[email protected]>
|
/test-pr
|
|
/fix-pr
|
|
Closing this PR as we've decided to go with the 'act' GitHub workflow tool instead of wrkflw. Why we're switching to 'act'After further evaluation, we've determined that 'act' (https://github.com/nektos/act) is a better fit for our local GitHub Actions workflow execution needs:
Next StepsWe'll implement 'act' integration in a future PR, which will include:
Thanks for the work on this wrkflw integration - the patterns and structure will be valuable for the 'act' implementation. Requested by @aaronsteers |
1 participant
feat: add poe task to run GitHub Actions workflows locally with wrkflw
Summary
This PR integrates the
wrkflwtool into PyAirbyte's development workflow by adding a new poe task that allows developers to run GitHub Actions workflows locally. The implementation includes:run-workflowthat accepts workflow filename as an argumentscripts/run_workflow.pywith argument parsing, validation, and error handlingThe tool was successfully tested with PyAirbyte's workflow structure and demonstrates proper integration, though workflow execution success depends on local environment setup and code quality compliance.
Review & Testing Checklist for Human
poe run-workflow python_lint.ymland verify wrkflw executes (success/failure of the workflow itself is less important than proper tool integration)poe run-workflow python_lint(without .yml) and verify automatic extension addition worksRecommended test plan:
cargo install wrkflwpoe run-workflow python_lint.ymlpoe run-workflow nonexistent.ymlDiagram
%%{ init : { "theme" : "default" }}%% flowchart TD pyproject["pyproject.toml<br/>[tool.poe.tasks]"]:::minor-edit init["scripts/__init__.py"]:::major-edit script["scripts/run_workflow.py<br/>main()"]:::major-edit workflows[".github/workflows/<br/>python_lint.yml<br/>python_pytest.yml"]:::context wrkflw["wrkflw tool<br/>(external dependency)"]:::context pyproject -->|"calls script"| script script -->|"imports from"| init script -->|"validates & runs"| workflows script -->|"executes with --runtime emulation"| wrkflw subgraph Legend L1[Major Edit]:::major-edit L2[Minor Edit]:::minor-edit L3[Context/No Edit]:::context end classDef major-edit fill:#90EE90 classDef minor-edit fill:#87CEEB classDef context fill:#FFFFFFNotes
Session details: Requested by AJ Steers (@aaronsteers) - Devin session
Summary by CodeRabbit