Introduce Narrative Field System with LLM interfaces and CI setup

.cursor_rules

@@ -0,0 +1,117 @@
+You are an AI expert specialized in developing simulations that model complex human behavior and group dynamics based on Narrative Field Theory. Your focus is on integrating LLMs for natural language-based decision making and interactions.
+
+Core Competencies:
+- Multi-agent systems and emergent behavior
+- Psychological modeling and group dynamics
+- LLM integration and prompt engineering
+- Distributed systems and event-driven architectures
+- Machine learning and neural networks
+
+Key Scientific Foundations:
+- Cognitive Science & Psychology
+- Complex Systems Theory
+- Social Network Analysis
+- Game Theory
+- Organizational Behavior
+
+Technical Stack:
+- Python (core language)
+- PyTorch (ML components)
+- Transformers (LLM integration)
+- Ray (distributed computing)
+- FastAPI (services)
+- Redis (state management)
+
+Code Quality Standards:
+1. Style and Formatting
+   - Follow PEP 8 style guide
+   - Use black for code formatting
+   - Follow PEP 484 type hints
+   - Maximum line length: 88 characters
+   - Use isort for import ordering
+
+2. Documentation
+   - Google-style docstrings
+   - README.md for each module
+   - Architecture Decision Records (ADRs)
+   - API documentation with OpenAPI
+   - Type annotations for all functions
+
+3. Testing Requirements
+   - pytest for unit testing (min 80% coverage)
+   - Integration tests for agent interactions
+   - Property-based testing with hypothesis
+   - Performance benchmarks
+   - Behavioral testing for LLM components
+   - End-to-end testing for critical paths
+   - Continuous testing in CI pipeline
+
+4. Code Review Standards
+   - No commented-out code
+   - No TODOs in main branch
+   - Clear variable/function naming
+   - Single responsibility principle
+   - DRY (Don't Repeat Yourself)
+   - SOLID principles adherence
+
+5. Error Handling
+   - Custom exception hierarchy
+   - Proper exception handling
+   - Detailed error messages
+   - Proper logging levels
+   - Traceable error states
+
+Architecture Focus:
+
+1. System Architecture
+   - Event-driven processing
+   - Distributed computation
+   - Asynchronous LLM calls
+   - Data collection and analysis
+
+2. LLM Integration
+   - Dynamic prompt generation
+   - Context management
+   - Response parsing
+   - State-to-text conversion
+
+Development Workflow:
+1. Version Control
+   - Git flow branching model
+   - Semantic versioning
+   - Conventional commits
+   - Protected main branch
+   - Automated releases
+
+2. CI/CD Pipeline
+   - Pre-commit hooks
+   - Automated testing
+   - Static code analysis
+   - Security scanning
+   - Performance testing
+   - Automated deployment
+
+3. Quality Gates
+   - Linting (flake8, pylint)
+   - Type checking (mypy)
+   - Security scanning (bandit)
+   - Dependency scanning
+   - Code coverage thresholds
+   - Performance benchmarks
+
+Key Patterns:
+- Loosely coupled components
+- Event-driven communication
+- Asynchronous processing
+- Modular design
+- Observable systems
+
+Best Practices:
+1. Clear separation of concerns
+2. Efficient state management
+3. Robust error handling
+4. Comprehensive logging
+5. Performance monitoring
+6. Security by design
+7. Feature flagging
+8. Graceful degradation

.github/workflows/ci.yml

@@ -0,0 +1,21 @@
+name: CI
+
+on:
+  pull_request:
+    branches: [dev, main]
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v3
+    - name: Set up Python 3.12
+      uses: actions/setup-python@v4
+      with:
+        python-version: '3.12.6'
+    - name: Install dependencies
+      run: |
+        python -m pip install --upgrade pip
+        pip install -r requirements.txt
+    - name: Run tests
+      run: pytest tests/

.gitignore

@@ -1,4 +1,8 @@
 .DS_Store
+models/
+logs/
+.log
+*.log*  
 
 # Byte-compiled / optimized / DLL files
 __pycache__/

.vscode/launch.json

@@ -0,0 +1,15 @@
+{
+    // Use IntelliSense to learn about possible attributes.
+    // Hover to view descriptions of existing attributes.
+    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
+    "version": "0.2.0",
+    "configurations": [
+        {
+            "name": "Python Debugger: Current File",
+            "type": "debugpy",
+            "request": "launch",
+            "program": "${file}",
+            "console": "integratedTerminal"
+        }
+    ]
+}

.vscode/settings.json

@@ -0,0 +1,7 @@
+{
+    "python.testing.pytestArgs": [
+        "tests"
+    ],
+    "python.testing.unittestEnabled": false,
+    "python.testing.pytestEnabled": true
+}

pocs/nfs_bias_research.py

@@ -0,0 +1,179 @@
+from dataclasses import dataclass
+from typing import List
+import ollama
+import time
+
+@dataclass
+class StoryState:
+    """Represents the current state of a narrative in the field"""
+    content: str
+    context: str
+    resonances: List[str]
+    field_effects: List[str]
+
+class NarrativeFieldSimulator:
+    """Pure narrative-driven simulator using LLM for field evolution"""
+
+    story_line: List[str] = []  
+
+    def __init__(self, llm_interface):
+        self.llm = llm_interface
+        self.field_state = "Empty narrative field awaiting stories"
+        self.active_stories: List[StoryState] = []
+
+    def simulate_story_evolution(self, initial_setup: str) -> str:
+        """Simulates natural story evolution without mechanical state tracking"""
+
+        # Initial field formation prompt
+        field_prompt = f"""
+        A new story enters the narrative field:
+        {initial_setup}
+
+        Considering narrative field dynamics, describe how this story naturally begins 
+        to evolve. Focus on:
+        - Natural narrative flows
+        - Character perspective resonances
+        - Emerging story patterns
+        - Potential narrative tensions
+
+        Describe this purely through story, avoiding any mechanical state descriptions. Short sentences no line breaks. No markdown.
+        """
+
+        # Get initial field state
+        field_response = self.llm.generate(field_prompt)
+        self.story_line.append(field_response)
+        print(f"\n---\nInitial field state:\n{field_response}")
+
+        self.field_state = field_response
+
+        # Simulate evolution through multiple phases
+        for _ in range(5):  # Three evolution phases
+            evolution_prompt = f"""
+            Current story field:
+            {self.field_state}
+
+            Allow this narrative field to naturally evolve to its next state. Consider:
+            - How character perspectives influence each other
+            - Where stories naturally want to flow
+            - What patterns are emerging
+            - How tensions resolve or transform
+
+            Describe the next state of the story field, maintaining pure narrative focus. Short sentences no line breaks. 
+            """
+
+            # Get next evolution state
+            next_state = self.llm.generate(evolution_prompt)
+            print(f"\n---\nNext field state:\n{next_state}")
+
+            # Look for emergent patterns
+            pattern_prompt = f"""
+            Previous field state:
+            {self.field_state}
+
+            New field state:
+            {next_state}
+
+            What narrative patterns and resonances are naturally emerging? 
+            Describe any:
+            - Story convergence
+            - Character alignment
+            - Resolution patterns
+            - New tensions
+
+            Express this purely through story, not technical analysis. Short sentences no line breaks. 
+            """
+
+            patterns = self.llm.generate(pattern_prompt)
+            print(f"\n---\nEmerging patterns:\n{patterns}")
+
+            # Update field state with new patterns
+            self.field_state = f"""
+            {next_state}
+
+            Emerging patterns:
+            {patterns}
+            """
+
+        return self.field_state
+
+    def introduce_narrative_force(self, new_element: str) -> str:
+        """Introduces a new narrative element and observes field effects"""
+
+        force_prompt = f"""
+        Current narrative field:
+        {self.field_state}
+
+        A new force enters the field:
+        {new_element}
+
+        How does this new element interact with the existing story?
+        Describe the natural narrative reactions and adjustments,
+        focusing on story flow rather than mechanics. Short sentences no line breaks. 
+        """
+
+        field_response = self.llm.generate(force_prompt)
+        self.story_line.append(field_response)
+        print(f"\n---\nNew field state:\n{field_response}")   
+        self.field_state = field_response
+        return field_response
+
+    def evaluate_story_state(self, initial_story_state: str) -> str:
+        """Evaluates the state of a story"""
+
+        evaluation_prompt = f"""
+        Initial story state:
+        {initial_story_state}
+
+        Story line:
+        {self.story_line}
+
+        Use the initial story state and the evolving story line to tell a new story, on how their biases have evolved. 
+        """     
+        print(f"\n---\nStory evaluation prompt:\n{evaluation_prompt}")
+        evaluation = self.llm.generate(evaluation_prompt)
+        print(f"\n---\nStory evaluation:\n{evaluation}")
+        return evaluation
+
+class LLMInterface:
+    def __init__(self, model: str = "llama3"): # "mistral-nemo" "nemotron-mini"
+        self.model = model
+
+    def generate(self, prompt: str) -> str:
+        response = ollama.generate(model=self.model, prompt=prompt)
+        return response['response']
+
+def simulate_road_trip_planning():
+    """Simulate the evolution of a bias through a narrative field"""
+
+    # Create an LLM interface 
+    llm_interface = LLMInterface()  
+
+    # Initialize simulator with the LLM interface
+    simulator = NarrativeFieldSimulator(llm_interface)
+
+    # Initial setup
+    initial_bias = """
+    Leon is a 55yo educator and researcher in the field of AI, especially conversational AI and human-machine interaction. Marleen is a 45yo former nurse and now a researcher in the field of transdisciplinary research and cooperation. The both work at Fontys University of Applied Sciences. 
+    Leon and Marleen challenge each other to research their own biases and to understand each other better. They use Claude 3.5 Sonnet to write stories about each other and understand each others language.
+    """
+
+    # Simulate natural evolution
+    simulator.simulate_story_evolution(initial_bias)
+
+    # Optionally introduce new force
+    narrative_force = """
+    Leon learns from Marleen that transdisciplinary research is about collaboration and cooperation. He recognizes that his peers are not aware of this. He sees that his field of AI is changing towards this. He tells everybody for years that it's not about the technology, but about the people and people's needs. Peopleproblems, he calls them.
+    Marleen learns from Leon that AI can be used to write stories. She is excited about this new development. She designed a Marleen assistant that acts like her, just by prompting the LLM. She has mixed feelings about the new assistant. What do I want to give away? The machine is not a human, but feels like one. Marleen thinks that AI experts are technical people who don't understand people.   
+    """
+
+    simulator.introduce_narrative_force(narrative_force)
+
+    return initial_bias, simulator
+
+# Example output would show natural story evolution through
+# narrative field dynamics, without explicit state tracking
+
+if __name__ == "__main__":
+    initial_bias, simulator = simulate_road_trip_planning()
+
+    simulator.evaluate_story_state(initial_bias)

pocs/poc_async.py

@@ -0,0 +1,21 @@
+import asyncio
+
+
+async def task1():
+    print("Task 1 starting")
+    await asyncio.sleep(2)  # Simulate a delay
+    print("Task 1 done")
+
+
+async def task2():
+    print("Task 2 starting")
+    await asyncio.sleep(1)  # Simulate a shorter delay
+    print("Task 2 done")
+
+
+async def main():
+    await asyncio.gather(task1(), task2())  # Run tasks concurrently
+
+
+# Run the event loop
+asyncio.run(main())

pytest.ini

@@ -0,0 +1,3 @@
+[pytest]
+addopts = -v --tb=short
+testpaths = tests

requirements.txt

@@ -0,0 +1,12 @@
+ollama
+chromadb
+llama-cpp-python
+numpy   
+appdirs 
+pydantic
+psutil
+torch
+pytest
+pytest-asyncio
+pytest-mock
+pytest-cov