feat: add module support for hierarchical CPN composition#69
Draft
fahchen wants to merge 10 commits into
Draft
Conversation
This commit implements comprehensive module support for ColouredFlow, enabling hierarchical composition and reuse in Coloured Petri Nets. ## Key Features ### Definition Layer - Module: Reusable subnet definitions with port places - PortPlace: Input/Output/IO interface points for modules - SocketAssignment: Mapping between parent places and module ports - Substitution Transitions: Transitions that reference and execute modules ### Validation Layer - ModuleValidator: Comprehensive validation of modules and substitution transitions - Validates module structure, port places, socket assignments, colour set matching - Detects circular module references - Ensures all required socket assignments are present ### Storage Layer - JSON codecs for all module-related structures - Automatic serialization/deserialization of modules - Updated ColouredPetriNet codec to include modules field ### Builder Layer - ModuleHelper: Convenient builder functions for modules - Helper functions for creating port places and socket assignments - Support for building substitution transitions ### Enactment Layer - ModuleInstance: Data structure for tracking module instances at runtime - Foundation for future module execution implementation ### Tests - Comprehensive test coverage for Module functions - ModuleValidator tests covering all validation scenarios - Tests for duplicate detection, circular references, and colour set matching ### Documentation - Complete module support documentation in docs/module_support.md - Usage examples and API reference - Notes on future enhancements ## Implementation Details Created files: - Definition: module.ex, port_place.ex, socket_assignment.ex - Validators: module_validator.ex, invalid_module_error.ex - Storage codecs: module.ex, port_place.ex, socket_assignment.ex - Builder: module_helper.ex - Enactment: module_instance.ex - Tests: module_test.exs, module_validator_test.exs - Documentation: module_support.md Modified files: - ColouredPetriNet: Added modules field and helper methods - Transition: Added subst and socket_assignments fields - Validators: Integrated ModuleValidator into validation pipeline - Storage codecs: Updated to support new fields 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
This commit adds the FlowConverter utility that enables converting existing ColouredPetriNet flows into Module definitions, making it easy to reuse workflows as composable components. ## Key Features ### Flow to Module Conversion - Manual port specification: Explicitly define which places become ports - Automatic port detection: System detects inputs/outputs based on arc patterns - Validation: Check if conversion is safe before proceeding ### FlowConverter API 1. flow_to_module/2 - Convert flow with explicit port specifications - Separates places into port places and internal places - Preserves all flow definitions (colour sets, transitions, arcs, etc.) 2. flow_to_module_auto/2 - Automatic port detection based on arc patterns - Places with no incoming arcs -> input ports - Places with no outgoing arcs -> output ports - Places with both -> internal places 3. validate_conversion/2 - Validates conversion before execution - Returns errors for blocking issues - Returns warnings for potential problems ### Use Cases - Convert standalone flows into composable modules - Build reusable module libraries from existing workflows - Create service-like components (auth, email, notifications) - Build pipeline stages that can be composed ### Testing - Comprehensive test coverage for all conversion scenarios - Tests for manual and automatic port detection - Validation edge cases and error handling ### Documentation - Updated module_support.md with FlowConverter usage - Common patterns: Service flows and pipeline stages - Examples of manual and automatic conversion ## Implementation Created files: - lib/coloured_flow/builder/flow_converter.ex - test/coloured_flow/builder/flow_converter_test.exs Modified files: - docs/module_support.md 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Add comprehensive design documentation for runtime module loading system. ## Overview This document describes the architecture and implementation plan for dynamic module loading and management in ColouredFlow. ## Key Components 1. **ModuleReference**: Type definition for various module reference methods - Reference by name (from registry) - Reference by Flow ID (from database) - Reference by Flow name (from database) 2. **ModuleRegistry**: In-memory registry for loaded modules - Register/unregister modules - Version management - Hot reload support 3. **ModuleLoader**: Load modules from various sources - Database (Flow table) - Cache management - Auto-detection of ports 4. **ModuleResolver**: Resolve module references at runtime - Resolve single references - Resolve all references in a CPN - Validation ## Use Cases - Pre-register common modules at startup - Load modules on-demand from database - Version management (module:v1, module:v2) - Hot reload for module updates ## Benefits - Separation of concerns - Reusability across flows - Easy versioning - Memory efficiency - Lazy loading 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Design a clean, decoupled module resolution system using behaviour pattern. ## Key Design Decisions 1. **No Global Registry**: Completely stateless, no global GenServer 2. **Behaviour-Based**: ModuleResolver is a behaviour that users implement 3. **Explicit Dependencies**: Resolver is passed as parameter, not global 4. **Default Implementation**: ColouredFlow provides Default resolver ## Components ### ModuleReference - Type definition for module references - Supports flow_id, flow_name, custom references ### ModuleResolver (Behaviour) - @callback resolve(module_ref, context) :: {:ok, Module.t()} | {:error, term()} - Interface for custom resolution strategies ### ModuleResolver.Default - Default implementation loads from database - Uses FlowConverter to transform flows into modules - Optional caching support ### ColouredPetriNet.resolve_modules/3 - resolve_modules(cpnet, resolver_impl, context) - Finds all module_ref in transitions - Calls resolver for each reference - Returns CPN with resolved modules ## Usage Examples ### With Default Resolver ```elixir resolver = ModuleResolver.Default.new(repo: MyApp.Repo) {:ok, resolved} = ColouredPetriNet.resolve_modules( cpnet, ModuleResolver.Default, resolver ) ``` ### With Custom Resolver ```elixir defmodule MyApp.CustomResolver do @behaviour ModuleResolver @impl true def resolve(ref, ctx), do: # custom logic end ``` ## Benefits - Fully decoupled (no global state) - Flexible (any loading strategy) - Testable (easy to mock) - Composable (can chain resolvers) - Explicit (clear dependencies) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Design module resolver that is passed to enactment as MFA tuple.
## Key Design
1. **Behaviour with 3 parameters**
- resolve(module_ref, context, runtime_info)
- context: initialized externally (repo, cache, etc.)
- runtime_info: from enactment (enactment_id, etc.)
2. **MFA format for lazy resolution**
- {Module, :resolve, [context]}
- Enactment calls when needed
- apply(Module, :resolve, [ref, context, runtime_info])
3. **Benefits**
- Lazy loading (resolve only when needed)
- Runtime context (use enactment info)
- No global state
- Flexible per-enactment configuration
Note: This design will be updated to remove function name from MFA
since it's redundant (can be inferred from behaviour).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Remove function name from resolver tuple since it's implicit from behaviour.
## Changes
**Before**: `{Module, :resolve, [context]}`
**After**: `{Module, [context]}`
## Rationale
Since ModuleResolver behaviour defines `resolve/3`, the function name
is redundant and can be inferred. This makes the configuration cleaner.
## Updated API
- Type: `resolver()` instead of `mfa()`
- Helper: `new/2` instead of `to_mfa/2`
- Caller: `resolve/3` instead of `call_mfa/3`
## Example
```elixir
# Initialize
context = ModuleResolver.Default.init(repo: MyApp.Repo)
resolver = ModuleResolver.new(ModuleResolver.Default, context)
# Returns: {ModuleResolver.Default, [context]}
# Use in enactment
{:ok, module} = ModuleResolver.resolve(resolver, module_ref, runtime_info)
# Internally: apply(ModuleResolver.Default, :resolve, [ref, context, info])
```
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Redesign module resolver with cleaner, more idiomatic Elixir approach.
## Key Design Changes
**Before**: `{Module, [context]}` with behaviour + apply
**After**: Protocol-based with struct instances
## New Design
1. **Protocol Definition**
```elixir
defprotocol ModuleResolver do
def resolve(resolver, module_ref, options)
end
```
2. **Resolver as Struct**
```elixir
resolver = ModuleResolver.Default.new(repo: MyApp.Repo)
# Returns: %ModuleResolver.Default{repo: ..., cache: ...}
```
3. **Usage**
```elixir
# Pass to enactment
{resolver, additional_opts}
# Call (clean syntax!)
ModuleResolver.resolve(resolver, ref, opts)
```
## Benefits
- More idiomatic Elixir (protocol dispatch)
- Cleaner syntax
- Easy to extend (implement protocol)
- Type-safe with structs
- Efficient protocol dispatch
## Alternative
Also documented behaviour-based approach as Option 2.
Recommendation: Use protocol-based (Option 1) for better
Elixir idioms and cleaner code.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Redesign module resolution as SubFlowManager to cover full lifecycle. ## Key Changes **Before**: ModuleResolver (only resolution) **After**: SubFlowManager (full lifecycle management) ## Why SubFlowManager? The behaviour is responsible for more than just resolving modules: 1. Resolve module references 2. Start sub-flow instances 3. Query sub-flow status 4. Retrieve results 5. Manage lifecycle (cancel, etc.) ## Two Design Options ### Option A: Detailed Lifecycle (Recommended) - resolve_module(manager, ref, opts) - start_subflow(manager, module, marking, opts) - query_subflow(manager, id, opts) - get_subflow_result(manager, id, opts) - cancel_subflow(manager, id, opts) Benefits: - Full control over sub-flow lifecycle - Can monitor progress - Can cancel long-running flows - Aligns with CPN theory ### Option B: Simplified Execute - execute(manager, ref, inputs, opts) - execute_async(manager, ref, inputs, opts) - await(manager, handle, timeout) Benefits: - Simpler API - One-call execution - Easier to implement ## Recommendation Use Option A for: - Better alignment with CPN theory - Support for long-running sub-flows - Monitoring and debugging capabilities - Future extensibility ## Questions for Confirmation 1. Is SubFlowManager a good name? 2. Option A or Option B? 3. Additional callbacks needed? 4. Sync vs async support? 5. Error handling scenarios? 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Document when, where, and why each callback is called during substitution transition execution. ## Coverage Each callback documented with: - **When**: Exact timing in transition lifecycle - **Where**: Which process/context calls it - **Why**: Purpose and trigger conditions - **Frequency**: How many times called - **Signature**: Full call signature with parameters - **Example**: Real usage code ## Callbacks Documented 1. **resolve_module**: When transition becomes enabled 2. **start_subflow**: After module resolved, before firing 3. **query_subflow**: Polling loop while waiting 4. **get_subflow_result**: After sub-flow completes 5. **cancel_subflow**: On timeout/error/cleanup ## Timeline Diagram Shows complete flow from transition fire to completion: ``` Transition fires ↓ 1. resolve_module() ↓ 2. start_subflow() ↓ 3. query_subflow() (polling) ↓ 4. get_subflow_result() ↓ Apply outputs & complete (Optional: 5. cancel_subflow()) ``` ## Complete Example Includes full implementation showing all callbacks in context. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Add SubFlowManager protocol and default implementation for managing child enactment (module execution) lifecycle with message-based communication. Key features: - SubFlowManager protocol for child enactment management - ModuleReference type for dynamic module loading - Default implementation loading modules from database - Message-based communication (no polling) - Token-based control (no direct cancellation) - Clear token sharing semantics via socket assignments Components: - SubFlowManager protocol with callbacks: - resolve_module: Load module from reference - start_child_enactment: Start module execution - get_child_state: Query state for debugging - ModuleReference type supporting: - Reference by flow_id with port_specs - Reference by flow_id with auto-detect - Reference by flow_name with port_specs - Reference by flow_name with auto-detect - Custom references for extensions - Default implementation: - Loads flows from database via Ecto repo - Converts flows to modules using FlowConverter - Stubs for child enactment management (to be implemented) Design follows revised SubFlowManager design with: - "child enactment" terminology - Message protocol for completion notification - Token copy semantics for parent-child communication 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This commit implements comprehensive module support for ColouredFlow, enabling hierarchical composition and reuse in Coloured Petri Nets.
Key Features
Definition Layer
Validation Layer
Storage Layer
Builder Layer
Enactment Layer
Tests
Documentation
Implementation Details
Created files:
Modified files:
🤖 Generated with Claude Code