Skip to content

Fix nested step dependency and argument inheritance issues#258

Merged
jimsynz merged 1 commit intomainfrom
fix/element-in-compose
Aug 10, 2025
Merged

Fix nested step dependency and argument inheritance issues#258
jimsynz merged 1 commit intomainfrom
fix/element-in-compose

Conversation

@jimsynz
Copy link
Copy Markdown
Contributor

@jimsynz jimsynz commented Aug 8, 2025

Summary

This PR resolves critical issues with nested step dependencies and argument inheritance in Reactor, addressing problems where steps inside nested contexts couldn't properly access parent scope results and where compose steps in maps couldn't inherit map arguments.

Issues Fixed

Closes #206 - Steps preceding a switch only get executed once
Closes #252 - Compose step inside map can't use map's arguments

Root Cause Analysis

The core issues stemmed from two related problems:

  1. Missing Dependency Tracking: The planner wasn't identifying when steps inside nested contexts (around, group, switch, map) depended on results from parent scope steps, leading to incomplete dependency graphs and steps executing out of order.

  2. Argument Inheritance Gaps: Compose steps inside maps couldn't access the map's arguments, requiring manual workarounds and breaking the expected inheritance model.

Key Changes

1. Enhanced Dependency Resolution (lib/reactor/planner.ex)

  • Added extract_nested_dependencies() function to identify cross-scope dependencies
  • Implemented proper tracking when nested steps reference parent scope results via result() calls
  • Enhanced dependency graph building to include nested step relationships
  • Ensures parent steps execute before container steps when nested steps depend on them

2. New Step Protocol Extension (lib/reactor/step.ex)

  • Added optional nested_steps/1 callback for steps that contain other steps
  • Provides infrastructure for the planner to discover nested step dependencies
  • Includes default empty implementation via use Reactor.Step

3. Map Step Improvements (lib/reactor/step/map.ex)

  • Implemented nested_steps/1 callback to expose contained steps to planner
  • Enhanced argument inheritance for nested compose steps
  • Added support for cross-scope element references
  • Fixed element argument resolution for nested contexts

4. Map DSL Context Passing (lib/reactor/dsl/map.ex)

  • Added context passing mechanism for argument inheritance
  • Compose steps now automatically inherit map arguments when not explicitly provided
  • Supports nested map contexts with proper argument propagation
  • Maintains explicit argument precedence while enabling inheritance

5. Runtime Support (lib/reactor/executor/step_runner.ex)

  • Added nested dependency collection and context passing
  • Enables runtime access to cross-scope dependencies
  • Provides nested dependency information to step execution context

6. Improved Error Handling (lib/reactor/builder/compose.ex)

  • Better error reporting when both extra and missing arguments exist
  • More informative error messages for argument resolution issues

Examples

Issue #206 - Dependency Resolution (Before: Broken)

# This would cause parent steps to execute multiple times
switch :decision do
  on result(:parent_step)  # parent_step would execute multiple times
  
  matches? &(&1 == true) do
    step :branch_step do
      argument :data, result(:another_parent_step, :key)  # Dependency not tracked!
    end
  end
end

Issue #206 - Dependency Resolution (After: Fixed)

# Now properly tracks that switch depends on both parent steps
switch :decision do
  on result(:parent_step)  # parent_step executes once
  
  matches? &(&1 == true) do
    step :branch_step do
      argument :data, result(:another_parent_step, :key)  # Dependency properly tracked!
    end
  end
end

Issue #252 - Map Argument Inheritance (Before: Broken)

map :process_items do
  source input(:items)
  argument :config, result(:get_config)
  
  compose :process_item, ProcessReactor do
    argument :item, element(:process_items)
    # Had to manually add this workaround:
    argument :config, value(nil)  # ❌ Required workaround
  end
end

Issue #252 - Map Argument Inheritance (After: Fixed)

map :process_items do
  source input(:items)
  argument :config, result(:get_config)
  
  compose :process_item, ProcessReactor do
    argument :item, element(:process_items)
    # config is now automatically inherited from the map! ✅
  end
end

Important Notes

For cross-scope dependencies: You still need to explicitly declare arguments that reference parent scope results using result(). What we fixed is ensuring the planner properly tracks these dependencies so execution order is correct.

For map argument inheritance: Compose steps now automatically inherit arguments from their containing map, eliminating the need for manual workarounds.

Test Coverage

  • Comprehensive nested dependency tests covering around, group, and switch steps
  • Map argument inheritance tests for compose steps
  • Cross-scope dependency scenarios with complex nesting
  • Error handling validation for edge cases
  • Backwards compatibility verification ensuring existing code continues to work

Backwards Compatibility

Fully backwards compatible

  • All existing functionality preserved
  • New features are opt-in via the nested_steps/1 callback
  • No breaking changes to existing APIs
  • Existing reactors continue to work without modification

Performance Impact

  • Minimal planning overhead: Only processes steps that implement nested_steps/1
  • No runtime performance impact: Dependency resolution happens at planning time
  • Memory efficient: Uses existing graph structures with additional edge labels

Testing

All tests pass including:

  • Existing test suite (313 tests)
  • New nested dependency tests
  • Map argument inheritance tests
  • Cross-scope dependency scenarios
mix test
# 8 doctests, 313 tests, 0 failures

Migration Guide

No migration required! This is a pure enhancement that:

  • Fixes existing broken scenarios automatically
  • Adds new capabilities without breaking changes
  • Maintains all existing behavior

For users who were working around the compose-in-map issue, you can now remove manual argument passing workarounds as they'll be inherited automatically.

@jimsynz jimsynz requested a review from a team August 8, 2025 01:40
This commit addresses multiple related issues with nested step dependencies
and argument inheritance in Reactor:

**Core Issues Fixed:**
- Steps inside nested contexts (around, group, switch, map) couldn't depend on parent scope results
- Compose steps inside maps couldn't access the map's arguments
- Missing dependency tracking for cross-scope references

**Key Changes:**

1. **Enhanced Planner (lib/reactor/planner.ex)**:
   - Added extract_nested_dependencies() to identify cross-scope dependencies
   - Added support for nested step dependency tracking via new nested_steps/1 callback
   - Properly handles result() references from nested steps to parent scope

2. **Improved Map Step (lib/reactor/step/map.ex)**:
   - Implemented nested_steps/1 callback to expose contained steps to planner
   - Enhanced argument inheritance for nested compose steps
   - Added support for cross-scope element references
   - Fixed element argument resolution for nested contexts

3. **Enhanced Step Protocol (lib/reactor/step.ex)**:
   - Added optional nested_steps/1 callback for steps containing other steps
   - Provides default empty implementation via use Reactor.Step

4. **Map DSL Improvements (lib/reactor/dsl/map.ex)**:
   - Added context passing for argument inheritance
   - Compose steps now automatically inherit map arguments when not explicitly provided
   - Supports nested map contexts with proper argument propagation

5. **Runtime Support (lib/reactor/executor/step_runner.ex)**:
   - Added nested dependency collection and context passing
   - Enables runtime access to cross-scope dependencies

6. **Compose Builder Fix (lib/reactor/builder/compose.ex)**:
   - Improved error handling for missing arguments
   - Better error reporting when both extra and missing arguments exist

**Test Coverage:**
- Added comprehensive tests for nested dependency resolution
- Added tests for compose steps in maps with argument inheritance
- Added tests for cross-scope dependency scenarios
- Verified proper error handling for invalid patterns

**Backwards Compatibility:**
- All changes are backwards compatible
- Existing functionality unchanged
- New features are opt-in via the nested_steps callback

Fixes issues with compose steps in maps not having access to map arguments
and resolves dependency planning issues for nested step contexts.
@jimsynz jimsynz force-pushed the fix/element-in-compose branch from 38ff044 to 14d97d5 Compare August 10, 2025 21:12
@jimsynz jimsynz merged commit 8672743 into main Aug 10, 2025
22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

compose step inside of a map cant use maps arguments unless a dummy value is passed Getting element of nested inner map causes unreachable error

2 participants