Skip to content

Implement pre/post processing framework for multi-target workflows #86

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 18 commits into from
May 29, 2025

Conversation

obie
Copy link
Contributor

@obie obie commented May 26, 2025

Summary

This PR implements issue #61 by adding support for pre-processing and post-processing phases to Roast workflows. This allows workflows that operate on multiple targets to:

  • Execute setup tasks once before processing any targets
  • Aggregate results from all workflow executions
  • Generate comprehensive reports after all targets are processed

Changes

  • Added pre_processing and post_processing arrays to workflow schema
  • Updated ConfigurationLoader to extract pre/post processing steps
  • Modified StepLoader to support phase-specific directories (pre_processing/, post_processing/)
  • Enhanced WorkflowRunner to execute phases and collect workflow results
  • Added comprehensive test coverage for the new functionality
  • Created example workflow demonstrating the feature
  • Updated README with documentation

Example Usage

name: test_improvement_workflow
tools: [read_file, update_files, cmd]

target: "$(find . -name '*_test.rb' -type f)"

pre_processing:
  - gather_baseline_metrics
  - setup_test_environment

steps:
  - analyze_test_file
  - improve_test_coverage
  - optimize_test_performance
  - validate_changes

post_processing:
  - aggregate_metrics
  - generate_summary_report
  - cleanup_environment

The post-processing steps have access to aggregated results from all workflow executions via {{ workflow_results }}.

Testing

  • Added comprehensive unit tests for all new functionality
  • Created example workflow with documentation
  • All tests passing
  • RuboCop clean

Closes #61

🤖 Generated with Claude Code

obie and others added 4 commits May 26, 2025 16:33
This PR implements issue #61 by adding support for pre-processing and post-processing phases to Roast workflows. This allows workflows that operate on multiple targets to:

- Execute setup tasks once before processing any targets
- Aggregate results from all workflow executions
- Generate comprehensive reports after all targets are processed

## Changes

- Added `pre_processing` and `post_processing` arrays to workflow schema
- Updated ConfigurationLoader to extract pre/post processing steps
- Modified StepLoader to support phase-specific directories (pre_processing/, post_processing/)
- Enhanced WorkflowRunner to execute phases and collect workflow results
- Added comprehensive test coverage for the new functionality
- Created example workflow demonstrating the feature
- Updated README with documentation

## Example Usage

```yaml
name: test_improvement_workflow
tools: [read_file, update_files, cmd]

target: "$(find . -name '*_test.rb' -type f)"

pre_processing:
  - gather_baseline_metrics
  - setup_test_environment

steps:
  - analyze_test_file
  - improve_test_coverage
  - optimize_test_performance
  - validate_changes

post_processing:
  - aggregate_metrics
  - generate_summary_report
  - cleanup_environment
```

The post-processing steps have access to aggregated results from all workflow executions via `{{ workflow_results }}`.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
- Change pre_processing data structure to have step results directly accessible
- Update workflow runner to flatten the data structure when passing to workflows
- Update README documentation to show the cleaner syntax
- Now you can access pre_processing results with: pre_processing_data.step_name
  instead of pre_processing_data.output.step_name

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
…ples

Instead of accessing private instance variables, refactored the test to verify
observable behavior: that workflows are executed for each target file. The test
now tracks which files were processed rather than examining internal state.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
- Removed all internal state manipulation (stubs for state, transcript, output_manager)
- Replaced complex mock workflows with real workflow files and configurations
- Tests now verify observable behavior through actual output and execution
- Simplified test setup by using temporary directories and real workflows
- Fixed all tests to use command execution ($(echo)) instead of string prompts
- Improved test clarity by focusing on public interfaces and outcomes

All tests now follow black box testing principles:
- No access to private instance variables
- No stubbing of internal implementation details
- Testing through observable behavior and public APIs only

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
@obie obie requested review from Copilot and parruda May 29, 2025 00:34
@obie obie self-assigned this May 29, 2025
@obie obie added the enhancement New feature or request label May 29, 2025
@obie obie requested a review from jonallured May 29, 2025 00:35
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR implements a new pre-/post-processing framework for multi-target workflows, allowing one-time setup and teardown tasks as well as aggregated reporting across all targets.

  • Add phase support in WorkflowExecutor and StepLoader to load phase-specific steps
  • Introduce WorkflowExecutionContext to collect and expose pre-processing and per-target outputs
  • Update configuration, base workflow, examples, and documentation to wire in pre/post processing steps

Reviewed Changes

Copilot reviewed 27 out of 27 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
lib/roast/workflow/workflow_executor.rb Added phase keyword to initialize and pass to StepLoader
lib/roast/workflow/step_loader.rb Lookup of steps under pre_processing/ and post_processing/ dirs
lib/roast/workflow/workflow_execution_context.rb New class to manage and aggregate outputs across phases
lib/roast/workflow/configuration_loader.rb Extract pre_processing and post_processing arrays
lib/roast/workflow/configuration.rb Store new pre/post arrays on Configuration
lib/roast/workflow/base_workflow.rb Inject pre_processing_data into workflow instances
examples/pre_post_processing/* New example workflow and step prompts
README.md Document the new pre/post-processing framework
Comments suppressed due to low confidence (1)

lib/roast/workflow/workflow_execution_context.rb:1

  • Add unit tests for WorkflowExecutionContext, covering add_target_output, to_h, and generate_target_key behavior to ensure outputs are aggregated correctly.
# frozen_string_literal: true

@obie obie marked this pull request as ready for review May 29, 2025 14:57
@@ -21,6 +21,8 @@ jobs:
continue-on-error: ${{ matrix.ruby == 'ruby-head' }}
steps:
- uses: actions/checkout@v4
- name: Install ripgrep
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In a future PR, we probably want to adopt the dev way here and have a key users can use to verify a requirement has been met and not need to run this command every time.

obie added 2 commits May 29, 2025 14:13
The test was failing because it didn't stub the output_manager method
that's now being called in workflow_runner.rb for the pre/post processing
feature.
@obie obie force-pushed the feature/pre-post-processing branch from bc000c9 to d4e6eb9 Compare May 29, 2025 18:33
obie and others added 4 commits May 29, 2025 14:37
- Add verbose output for command execution in workflows
- Show command outputs when using --verbose flag
- Display command outputs in conditional branches when verbose
- Add user-friendly error messages when commands or steps fail
- Show ❌ indicators and helpful context for failures
- Display command output directly on failure (no verbose needed)
- Improve exit handler messages with actionable suggestions
- Update tests to support verbose attribute on workflow mocks

This helps users who may not be familiar with Roast understand
what's happening during workflow execution and debug issues more
easily.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
- Improved verbose mode and error reporting
- Automatic workflow discovery by name
- Configurable base URI for API endpoints
- Fixed search file tool path handling
- ActiveSupport 7.0+ compatibility

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
- Changed from using frozen WorkflowContext to a mock object to avoid stubbing errors
- Fixed exit_on_error? stubbing by using .with() to specify specific return values
- Rubocop auto-corrected trailing whitespace and spacing issues
- All tests now pass with the updated pre/post processing code
@obie obie merged commit b51c7db into main May 29, 2025
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Pre/Post Processing Framework for Multi-Target Workflows
2 participants