Skip to content

Releases: cgbarlow/machine-dream_AG

Machine Dream v0.1.0-alpha: LLM Learning & Batch Testing Framework

12 Jan 22:25
@cgbarlow cgbarlow
v0.1.0-alpha
de77301

Choose a tag to compare

View all tags
Machine Dream v0.1.0-alpha: LLM Learning & Batch Testing Framework Pre-release
Pre-release

Release Date: 2026-01-13
Status: Alpha (Pre-release)
Focus: LLM learning capabilities, batch testing infrastructure, production-ready CLI

🎯 Overview

This alpha release establishes Machine Dream as a functional continuous machine cognition research platform. The system demonstrates LLM learning through experience consolidation and provides comprehensive batch testing tools for evaluating learning effectiveness across multiple AI models.

Research Basis:

✨ Major Features

🤖 LLM Integration & Learning

Pure LLM Sudoku Player

  • No deterministic fallbacks - authentic learning through struggle
  • Multi-provider support: LM Studio, OpenAI, Anthropic, Ollama, OpenRouter
  • Profile management system for multiple AI model configurations
  • Smart model loading with state detection (loading/loaded/not-loaded)
  • Automatic model lifecycle management via LM Studio API

Learning Units

  • Discrete knowledge packages created through experience consolidation
  • Iterative learning: units absorb new experiences over time
  • Unit comparison and A/B testing support
  • Export/import for sharing learned strategies
  • Profile-specific learning isolation

Dual Consolidation

  • Standard mode: 3-5 consolidated strategies
  • Enhanced mode (-2x): 6-10 strategies for deeper learning
  • Adaptive clustering with keyword depth adjustment
  • Grid context-based subdivision for fine-grained clustering

🧪 Batch Testing Framework

Comprehensive Test Suite (comprehensive-test-suite.sh)

  • Tests all profiles in size order (largest model first)
  • Multiple modes: standard, AISP, AISP-full
  • Creates dual learning units (standard + -2x)
  • Live move-by-move output with --visualize-basic
  • Smart model loading (detects loading state, prevents unnecessary reloads)
  • Profile exclusion support (--exclude)

Validation Testing (batch-test-learning-unit.sh)

  • Test specific learning units without dream cycle
  • Profile-specific learning units (profile:unit pairs)
  • Multi-mode support for comprehensive validation
  • Compare standard vs -2x units
  • Automated result tracking with CSV summary

A/B Testing (ab-test-learning.sh)

  • Compare learning ON vs OFF (baseline vs learning)
  • Dream consolidation between test phases
  • Statistical comparison of solve rates
  • Session isolation and cleanup

Iterative Learning (iterative-learning.sh)

  • Track improvement over time
  • Batch-based learning with periodic consolidation
  • Incremental knowledge accumulation

🌙 Dreaming Pipeline

5-Phase Consolidation

  1. Capture - Gather successful experiences
  2. Triage - Filter by importance and correctness
  3. Compression - Cluster similar reasoning patterns
  4. Abstraction - Generate few-shot examples
  5. Verification - Validate and store learning units

Enhanced Clustering

  • Adaptive keyword depth (2-4 keywords based on target)
  • Content-based subdivision for meeting strategy targets
  • Grid context awareness (row/column/box regions)
  • Reasoning signature extraction with configurable granularity

📋 AISP Integration

AI Specification Protocol Support

  • --aisp mode: Convert prompts to AISP syntax
  • --aisp-full mode: End-to-end AISP communication
  • Experimentation with formal specifications vs natural language
  • Potential for reduced parsing errors and improved multi-model consistency

💻 Production-Ready CLI

310 Passing Tests (100% pass rate)

  • Unit tests: Puzzle engine, GRASP loop, memory, consolidation
  • Integration tests: CLI commands, LLM player, profiles
  • Zero TypeScript errors
  • Zero mock implementations in critical paths

Command Categories

  • llm play - AI model puzzle solving with learning
  • llm profile - Multi-provider profile management
  • llm learning - Learning unit management
  • llm dream - Experience consolidation
  • memory - AgentDB memory operations
  • system - System health and administration
  • puzzle - Puzzle generation and validation

📚 Documentation

Comprehensive Documentation Suite

  • Main README with research basis and quick start
  • User Guide (52K) - Installation, concepts, workflows
  • CLI Reference (24K) - Complete command documentation
  • Scripts Guide - Batch testing workflows and examples
  • 15 Formal Specifications (Spec 01-15)
  • 10 Architecture Decision Records (ADRs)
  • Archive of 41 historical development documents

Documentation Organization

  • docs/ reduced from 41 to 4 essential files
  • Archived historical development docs with comprehensive index
  • Cross-referenced specs and ADRs
  • Role-based navigation (users, developers, researchers)

🚀 Key Improvements (This Release)

Smart Model Loading

  • Detects models in "loading" state and waits (up to 60s)
  • Prevents unnecessary unload/reload cycles
  • Reduces "Operation canceled" errors in LM Studio

Live Move Output

  • --visualize-basic flag shows compact move-by-move results
  • Real-time feedback during puzzle solving
  • Parse failure display with correct move counter synchronization

Size-Based Profile Ordering

  • Profiles tested from largest to smallest model
  • Uses profile tags to determine model size
  • Optimizes testing workflow for large models

Profile Exclusion

  • --exclude option to skip specific profiles
  • Useful for excluding unstable or slow models
  • Maintains testing efficiency

Enhanced .gitignore

  • Properly ignores test result directories
  • ab-test-results/, comprehensive-results/, batch-results/
  • Removed 96+ tracked log files from repository

📊 Statistics

  • Version: 0.1.0-alpha
  • Tests: 310 passing (100% pass rate)
  • TypeScript Errors: 0
  • Documentation: 75+ documents (~1.6MB)
  • Specifications: 15 formal specs
  • ADRs: 10 architectural decisions
  • Scripts: 10 batch testing scripts
  • Commits: 82 since Jan 1, 2026

🎮 Component Status

Component Status Notes
LLM Integration ✅ Stable Pure LLM player with profiles, AISP support
Learning Units ✅ Stable Discrete knowledge packages with consolidation
Dreaming Pipeline ✅ Stable 5-phase consolidation (standard + dual -2x)
Batch Testing ✅ Stable Comprehensive suite + validation scripts
CLI Interface ✅ Stable 310 tests passing, production-ready
Profile Management ✅ Stable Multi-provider connection management
AgentDB ✅ Stable SQLite-based persistence layer
Puzzle Engine ✅ Stable Generation, validation, rules
TUI ⚠️ Experimental Under active development, may not work properly

🔬 Research Applications

Measuring LLM Learning

  • Baseline vs learning comparisons
  • Session-based experience accumulation
  • Consolidation effectiveness tracking
  • Cross-model learning transferability

Testing Hypotheses

  • Does dreaming consolidation improve performance?
  • Are doubled strategies more effective than standard?
  • Does AISP reduce parsing errors?
  • Can learning units transfer between models?

Experimental Workflows

  1. Generate learning units via comprehensive test suite
  2. Validate units with batch testing
  3. Compare standard vs -2x consolidation
  4. Test cross-model compatibility
  5. Measure iterative improvement

📖 Quick Start

# Installation
git clone https://github.com/your-org/machine-dream.git
cd machine-dream_AG
npm install
npm run build
npm link

# Create AI model profile
machine-dream llm profile add \
  --name qwen3-coder \
  --provider lmstudio \
  --base-url http://localhost:1234/v1 \
  --model qwen3-30b-instruct \
  --set-default

# Run comprehensive test suite (generate learning units)
./scripts/comprehensive-test-suite.sh --runs 3

# Validate specific learning units
./scripts/batch-test-learning-unit.sh \
  --profiles qwen3:unit_standard,qwen3:unit_standard-2x \
  --runs 10

🔧 Technical Requirements

  • Node.js: v20+ (v24 fully supported)
  • npm: Latest version
  • LM Studio: Optional (for local LLM inference)
  • OS: Linux, macOS, Windows (WSL tested)

⚠️ Known Limitations (Alpha)

TUI Status

The Terminal UI is experimental and under active development. Use CLI commands for stable interactions.

Model Support

Tested primarily with LM Studio local models. Cloud provider support (OpenAI, Anthropic) available but less extensively tested.

Performance

Large models (120B+) require significant VRAM. Testing with multiple large models may require sequential execution.

AISP Integration

AISP support is experimental. Benefits are theoretical for Machine Dream's specific use cases - active area of research.

📝 Documentation

🤝 Contributing

This is a research project. Contributions welcome for:

  • Theoretical improvements to cognitive architecture
  • Memory system optimizations
  • LLM integration enhancements
  • Documentation improvements
  • Bug reports and fixes

See specifications in docs/specs/ and ADRs in docs/adr/ for architectural guidance.

📄 License

MIT

🙏 Acknowledgments

**Research Foundations:...

Read more
Loading