-
Notifications
You must be signed in to change notification settings - Fork 24
Implement pre/post processing framework for multi-target workflows #86
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
This PR implements issue #61 by adding support for pre-processing and post-processing phases to Roast workflows. This allows workflows that operate on multiple targets to: - Execute setup tasks once before processing any targets - Aggregate results from all workflow executions - Generate comprehensive reports after all targets are processed ## Changes - Added `pre_processing` and `post_processing` arrays to workflow schema - Updated ConfigurationLoader to extract pre/post processing steps - Modified StepLoader to support phase-specific directories (pre_processing/, post_processing/) - Enhanced WorkflowRunner to execute phases and collect workflow results - Added comprehensive test coverage for the new functionality - Created example workflow demonstrating the feature - Updated README with documentation ## Example Usage ```yaml name: test_improvement_workflow tools: [read_file, update_files, cmd] target: "$(find . -name '*_test.rb' -type f)" pre_processing: - gather_baseline_metrics - setup_test_environment steps: - analyze_test_file - improve_test_coverage - optimize_test_performance - validate_changes post_processing: - aggregate_metrics - generate_summary_report - cleanup_environment ``` The post-processing steps have access to aggregated results from all workflow executions via `{{ workflow_results }}`. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
- Change pre_processing data structure to have step results directly accessible - Update workflow runner to flatten the data structure when passing to workflows - Update README documentation to show the cleaner syntax - Now you can access pre_processing results with: pre_processing_data.step_name instead of pre_processing_data.output.step_name 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
…ples Instead of accessing private instance variables, refactored the test to verify observable behavior: that workflows are executed for each target file. The test now tracks which files were processed rather than examining internal state. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
- Removed all internal state manipulation (stubs for state, transcript, output_manager) - Replaced complex mock workflows with real workflow files and configurations - Tests now verify observable behavior through actual output and execution - Simplified test setup by using temporary directories and real workflows - Fixed all tests to use command execution ($(echo)) instead of string prompts - Improved test clarity by focusing on public interfaces and outcomes All tests now follow black box testing principles: - No access to private instance variables - No stubbing of internal implementation details - Testing through observable behavior and public APIs only 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR implements a new pre-/post-processing framework for multi-target workflows, allowing one-time setup and teardown tasks as well as aggregated reporting across all targets.
- Add
phase
support inWorkflowExecutor
andStepLoader
to load phase-specific steps - Introduce
WorkflowExecutionContext
to collect and expose pre-processing and per-target outputs - Update configuration, base workflow, examples, and documentation to wire in pre/post processing steps
Reviewed Changes
Copilot reviewed 27 out of 27 changed files in this pull request and generated 2 comments.
Show a summary per file
File | Description |
---|---|
lib/roast/workflow/workflow_executor.rb | Added phase keyword to initialize and pass to StepLoader |
lib/roast/workflow/step_loader.rb | Lookup of steps under pre_processing/ and post_processing/ dirs |
lib/roast/workflow/workflow_execution_context.rb | New class to manage and aggregate outputs across phases |
lib/roast/workflow/configuration_loader.rb | Extract pre_processing and post_processing arrays |
lib/roast/workflow/configuration.rb | Store new pre/post arrays on Configuration |
lib/roast/workflow/base_workflow.rb | Inject pre_processing_data into workflow instances |
examples/pre_post_processing/* | New example workflow and step prompts |
README.md | Document the new pre/post-processing framework |
Comments suppressed due to low confidence (1)
lib/roast/workflow/workflow_execution_context.rb:1
- Add unit tests for
WorkflowExecutionContext
, coveringadd_target_output
,to_h
, andgenerate_target_key
behavior to ensure outputs are aggregated correctly.
# frozen_string_literal: true
@@ -21,6 +21,8 @@ jobs: | |||
continue-on-error: ${{ matrix.ruby == 'ruby-head' }} | |||
steps: | |||
- uses: actions/checkout@v4 | |||
- name: Install ripgrep |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In a future PR, we probably want to adopt the dev
way here and have a key users can use to verify a requirement has been met and not need to run this command every time.
The test was failing because it didn't stub the output_manager method that's now being called in workflow_runner.rb for the pre/post processing feature.
bc000c9
to
d4e6eb9
Compare
- Add verbose output for command execution in workflows - Show command outputs when using --verbose flag - Display command outputs in conditional branches when verbose - Add user-friendly error messages when commands or steps fail - Show ❌ indicators and helpful context for failures - Display command output directly on failure (no verbose needed) - Improve exit handler messages with actionable suggestions - Update tests to support verbose attribute on workflow mocks This helps users who may not be familiar with Roast understand what's happening during workflow execution and debug issues more easily. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
- Improved verbose mode and error reporting - Automatic workflow discovery by name - Configurable base URI for API endpoints - Fixed search file tool path handling - ActiveSupport 7.0+ compatibility 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
- Changed from using frozen WorkflowContext to a mock object to avoid stubbing errors - Fixed exit_on_error? stubbing by using .with() to specify specific return values - Rubocop auto-corrected trailing whitespace and spacing issues - All tests now pass with the updated pre/post processing code
Summary
This PR implements issue #61 by adding support for pre-processing and post-processing phases to Roast workflows. This allows workflows that operate on multiple targets to:
Changes
pre_processing
andpost_processing
arrays to workflow schemaExample Usage
The post-processing steps have access to aggregated results from all workflow executions via
{{ workflow_results }}
.Testing
Closes #61
🤖 Generated with Claude Code