Skip to content

architecture_reference.md

Latest commit

 

History

History
375 lines (285 loc) · 12 KB

architecture_reference.md

File metadata and controls

375 lines (285 loc) · 12 KB

GNN Architecture Reference

Version: v2.0.0
Last Updated: 2026-03-24
Status: ✅ Production Ready
Modules: 38+ · Pipeline steps: 25 · Renderers: 9 backends (see ../implementations/README.md) · Tests: see ../../../README.md

GNN Architecture Team
Version: 2.0.0
Status: Production Ready
Last Updated: 2026-03-24

Implementation details of the thin orchestrator pattern and cross-module integration.

For complete pipeline documentation:

Complete 25-Step Pipeline Mapping

The GNN pipeline consists of exactly 25 steps (0-24), each following the thin orchestrator pattern:

Core Processing (Steps 0-9)

  • 0_template.py → src/template/ - Pipeline initialization
  • 1_setup.py → src/setup/ - Environment and dependency setup
  • 2_tests.py → src/tests/ - Test suite execution
  • 3_gnn.py → src/gnn/ - GNN parsing and multi-format processing
  • 4_model_registry.py → src/model_registry/ - Model versioning
  • 5_type_checker.py → src/type_checker/ - Type validation
  • 6_validation.py → src/validation/ - Consistency checking
  • 7_export.py → src/export/ - Multi-format export
  • 8_visualization.py → src/visualization/ - Graph visualization
  • 9_advanced_viz.py → src/advanced_visualization/ - Advanced plots

Simulation & Analysis (Steps 10-16)

  • 10_ontology.py → src/ontology/ - Ontology processing
  • 11_render.py → src/render/ - Code generation
  • 12_execute.py → src/execute/ - Simulation execution
  • 13_llm.py → src/llm/ - LLM analysis
  • 14_ml_integration.py → src/ml_integration/ - ML integration
  • 15_audio.py → src/audio/ - Audio generation
  • 16_analysis.py → src/analysis/ - Statistical analysis

Integration & Output (Steps 17-24)

  • 17_integration.py → src/integration/ - System integration
  • 18_security.py → src/security/ - Security validation
  • 19_research.py → src/research/ - Research tools
  • 20_website.py → src/website/ - Website generation
  • 21_mcp.py → src/mcp/ - MCP processing
  • 22_gui.py → src/gui/ - GUI interface
  • 23_report.py → src/report/ - Report generation
  • 24_intelligent_analysis.py → src/intelligent_analysis/ - AI-enhanced analysis

For module-specific documentation, see each src/[module]/AGENTS.md file.

### Thin Orchestrator Pattern

(Actual Implementation)

Pattern Definition

Each numbered pipeline step (0-24) follows this structure:

  • Thin orchestrator script handles argument parsing, logging, output management
  • Module directory contains actual implementation logic
  • Cross-references between steps via standardized JSON outputs

Pattern Implementation Example

Step 8: Visualization (src/8_visualization.py)

#!/usr/bin/env python3
"""Step 8: Visualization Processing (Thin Orchestrator)"""

import sys
from pathlib import Path

# Thin orchestrator: delegates to module
from utils.pipeline_template import create_standardized_pipeline_script
from visualization import process_visualization_main  # ← Core implementation

run_script = create_standardized_pipeline_script(
    "8_visualization.py",
    process_visualization_main,  # ← Delegates to module
    "Matrix and network visualization processing"
)

def main() -> int:
    return run_script()  # ← Pure orchestration

### Modular Implementation Layer

c/visualization/init.py)

# Exposes core functionality with safe imports
try:
    from .visualizer import GNNVisualizer, generate_graph_visualization
    from .matrix_visualizer import MatrixVisualizer, process_matrix_visualization
    from .processor import generate_matrix_visualizations, generate_network_visualizations
except Exception:
    # Alternative implementations for missing dependencies
    class GNNVisualizer:
        def __init__(self, *args, **kwargs): 
            self.available = False
        def generate(self, *a, **k): 
            return False

Cross-Module Data Flow (Actual Files)

Step 3 → Step 5: Parsed Data Transfer

Input:  input/gnn_files/actinf_pomdp_agent.md
Output: output/3_gnn_output/gnn_processing_results.json

Cross-reference in src/5_type_checker.py:
├── gnn_output_dir = get_output_dir_for_script("3_gnn.py", Path(args.output_dir))
├── gnn_results_file = gnn_nested_dir / "gnn_processing_results.json"  
└── with open(gnn_results_file, "r") as f: gnn_results = json.load(f)

Step 5 → Step 8: Type Data Transfer

Type data flows from Step 5 analysis to Step 8 visualization:

src/8_visualization.py:
└── visualizer.py:generate_matrix_visualization()
    ├── Reads: output/5_type_checker_output/type_check_results.json
    ├── Extracts: type_analysis["dimension_analysis"]
    └── Generates: matrix heatmaps based on dimensional analysis

Step 11 → Step 12: Generated Code Execution

Code generation to execution transfer:

src/11_render.py → output/11_render_output/
├── actinf_pomdp_agent_pymdp.py      (Generated PyMDP code)
├── actinf_pomdp_agent_rxinfer.jl    (Generated RxInfer code) 
└── render_summary.json              (Generation metadata)

src/12_execute.py:
├── Discovers generated files in output/11_render_output/
├── Executes: subprocess.run(["python", "actinf_pomdp_agent_pymdp.py"])
└── Captures: execution results, timing, memory usage

Module Structure Analysis (Real Locations)

Parsing Modules (src/gnn/)

Core parsing functionality distribution:

src/gnn/
├── multi_format_processor.py          # Main processor (called by 3_gnn.py)
├── schema_validator.py                 # Regex patterns (line 58-63)
│   ├── SECTION_PATTERN
│   ├── VARIABLE_PATTERN  
│   ├── CONNECTION_PATTERN
│   └── PARAMETER_PATTERN
├── parser.py                          # GNNParsingSystem (line 72)
│   ├── _detect_format() (line 107)
│   └── _basic_parser() (line 120)
└── parsers/                           # Format-specific parsers
    ├── markdown_parser.py             # MarkdownGNNParser 
    ├── python_parser.py               # PythonGNNParser (line 25)
    ├── lean_parser.py                 # LeanGNNParser
    └── unified_parser.py              # UnifiedGNNParser

Type Analysis Modules (src/type_checker/)

Type analysis implementation:

src/type_checker/
├── analysis_utils.py                  # Core functions (line 13-62)
│   ├── analyze_variable_types() → Dict[str, Any]
│   ├── analyze_connections() → Dict[str, Any]
│   └── estimate_computational_complexity() → Dict[str, Any]
├── checker.py                         # GNNTypeChecker (line 174)
│   ├── check_file() → Tuple[bool, List[str], List[str], Dict[str, Any]]
│   ├── _check_required_sections()
│   └── _collect_variable_analysis()
└── processor.py                       # GNNTypeChecker (line 20)
    ├── _validate_type() (line 191)
    └── _analyze_types() (line 232)

Visualization Modules (src/visualization/)

Visualization implementation hierarchy:

src/visualization/
├── __init__.py                        # Safe imports (line 15-47)
├── visualizer.py                      # GNNVisualizer (line 66)
├── matrix_visualizer.py               # MatrixVisualizer (line 40)
│   └── generate_matrix_visualizations() (line 941)
└── processor.py                       # Core processing functions
    ├── parse_matrix_data() (line 367)
    ├── generate_matrix_visualizations() (line 403)
    └── generate_network_visualizations() (line 523)

Framework Integration Points (Implementation Details)

PyMDP Integration

Location: src/render/pymdp/ (to be implemented) Template Variables:

# Matrix extraction from parsed GNN
A = extract_matrix(gnn_data, "A")  # Likelihood
B = extract_matrix(gnn_data, "B")  # Transitions  
C = extract_vector(gnn_data, "C")  # Preferences
D = extract_vector(gnn_data, "D")  # Prior
E = extract_vector(gnn_data, "E")  # Habit

# PyMDP agent construction
agent = pymdp.Agent(A=A, B=B, C=C, D=D, E=E)

RxInfer.jl Integration

Location: src/render/rxinfer/ (to be implemented) Model Template:

@model function gnn_model()
    # Extract GNN parameters
    A ~ MatrixDirichlet({{gnn_A_prior}})
    B ~ MatrixDirichlet({{gnn_B_prior}})
    
    # State evolution  
    s[1] ~ Categorical({{gnn_D}})
    for t in 2:T
        s[t] ~ Categorical(B[:, s[t-1], u[t-1]])
    end
    
    # Observation model
    for t in 1:T  
        o[t] ~ Categorical(A[:, s[t]])
    end
end

DisCoPy Integration

Location: src/render/discopy/ (to be implemented)
Category Theory Mapping:

# GNN connections → DisCoPy morphisms
connections = gnn_data["connections"]
diagram = Id(X)  # Identity on objects

for conn in connections:
    if conn["type"] == "directed":
        # f: A → B becomes morphism  
        source_obj = Object(conn["source"])
        target_obj = Object(conn["target"])
        morphism = Arrow(source_obj, target_obj, name=f"{conn['source']}_to_{conn['target']}")
        diagram = diagram >> morphism

Pipeline Orchestration Details

Argument Flow (src/main.py → steps)

# Main pipeline argument passing:
def execute_pipeline_step(script_name: str, args: PipelineArguments, logger):
    cmd = build_step_command_args(
        script_name.replace('.py', ''),
        args,  # Contains: target_dir, output_dir, verbose, etc.
        python_executable,
        script_path
    )
    
# Each step receives s### Standardized I/O and State Management
# --target-dir, --output-dir, --```python
from src.utils.logging_helpers import setup_logging
from src.utils.io_helpers import get_output_dir_for_script

# Standardized logging setup
logger = setup_logging(__file__)

# Standardized output directory resolution
output_dir = get_output_dir_for_script(__file__)

def get_output_dir_for_script(script_name: str, base_output_dir: Path) -> Path: step_name = script_name.replace('.py', '') return base_output_dir / f"{step_name}_output"

Results in structure

output/

├── 3_gnn_output/

├── 5_type_checker_output/

├── 8_visualization_output/

└── 11_render_output/


### Error Handling Pattern

```python
# Standardized error handling in each step:
try:
    success = module_processing_function(target_dir, output_dir, logger, **kwa```python
try:
    # Core logic
    process_data()
except Exception as e:
    logger.error(f"Critical error in processing: {str(e)}")
    raise

turn 1 # Pipeline continues with next step


## Dependency Resolution (Actual Implementation)

### Safe Import Pattern (src/visualization/**init**.py:15-47)

```python
# Pattern used across modules for optional dependencies:
try:
    from .matrix_visualizer import MatrixVisualizer, process_matrix_visualization
except Exception:
    MatrixVisualizer = None
    process_matrix_visualization = None

# Allows graceful degradation when dependencies missing
if MatrixVisualizer is None:
    logger.warning("Matrix visualization unavailable - matplotlib missing")
    return create_alternative_html_report()

MCP Integration Pattern

Each module includes mcp.py with tool registration:

# Example: src/visualization/mcp.py
@server.tool()
def visualize_gnn_model(content: str, output_path: str) -> dict:
    """Generate visualization for GNN model content."""
    return process_visualization_main(
        target_dir=Path(content),
        output_dir=Path(output_path)
    )

This architecture enables modular development, safe-to-fail operation, and framework interoperability while maintaining clear separation between orchestration and implementation logic.