Skip to content

Latest commit

 

History

History
1201 lines (855 loc) · 37.3 KB

CHANGELOG.md

File metadata and controls

1201 lines (855 loc) · 37.3 KB

Changelog

All notable changes to TriX are documented here.

Note: Doc paths referenced in older entries (e.g. docs/TUTORIAL.md, docs/MESA8_NEURAL_CUDA.md) may have moved to docs/archive/. See docs/INDEX.md for current doc locations.

[0.12.0] - 2024-12-19

Mesa 13: XOR Superposition Signature Compression

Core achievement: 129x signature compression with deterministic O(1) routing.

"Don't store what you can XOR."

The Numbers

Metric Target Achieved
Compression ratio 11.6x 129x
Routing determinism 100% 100%
Test coverage Full 64 tests

Added

XOR Superposition (src/trix/nn/xor_superposition.py)

  • SparseDelta - Sparse XOR delta encoding (3 bytes per difference)
  • CompressedSignatures - XOR superposition storage with lossless roundtrip
  • SuperpositionRouter - Hamming-distance routing with compression
  • XORSuperpositionFFN - Drop-in FFN with compress/decompress lifecycle

Batch Operations (src/trix/nn/xor_routing.py)

  • popcount_vectorized - Lookup-table based population count
  • pack_ternary_batch - Batch ternary packing to uint8
  • hamming_distance_batch - Batched Hamming distance computation

HierarchicalTriXFFN Integration

  • compress_signatures() - Compress for inference deployment
  • decompress_signatures() - Decompress for training/fine-tuning
  • get_compression_stats() - Compression ratio and sparsity stats

Tests (tests/test_xor_superposition.py)

  • 33 new tests covering compression, routing, determinism
  • Parametrized compression ratio validation
  • Edge case coverage (single signature, non-divisible dims, identical sigs)

Documentation

  • MESA13_XOR_SUPERPOSITION.md - Complete technical specification

Key Insight

Trained TriX signatures exhibit ~99% structural similarity. XOR superposition exploits this by storing one centroid + sparse deltas:

For ternary vectors: argmax(dot) = argmin(hamming)

This preserves routing decisions exactly while compressing 128KB → 1KB.

Deterministic Neural Networks

Compressed routing is bit-exact reproducible:

ffn.compress_signatures()
ffn.eval()
_, r1, _ = ffn(x)
_, r2, _ = ffn(x)
assert torch.equal(r1['tile_idx'], r2['tile_idx'])  # Always true

This is the foundation for auditable, verifiable neural computation.

[0.11.0] - 2024-12-19

Mesa 12: HALO - Homeo-Adaptive Learning Observer

Core achievement: Self-aligning AI through intrinsic coherence sensing.

"Who needs Human Reinforcement Learning Feedback when you have a Homeo-Adaptive Learning Observer?!"

The Paradigm Shift

RLHF HALO
Human labelers needed Self-observing
Expensive annotation Free (watches itself)
Slow feedback loops Real-time, every step
Human bias injection Reads actual entropy
Episodic, sparse signal Continuous, dense signal
External reward proxy Intrinsic coherence measure
Can't scale Scales infinitely

Added

Guardian Angel Architecture (src/trix/guardian/)

  • ProgrammableTile - Substrate with gentle read/write API
  • ProgrammableTileBank - Collection with unified interface
  • XORReflector - Shows what changed between states
  • SuperpositionedReflector - Multi-angle self-view (N orthogonal bases)
  • TrainingManifoldReflector - Meta-level trajectory assessment
  • ObservationFrame - Full transparency snapshot
  • StateEncoder - Compress observations to state vectors
  • ObserverModel - LSTM-based temporal prediction
  • GuardianAngel - Complete HALO integration
  • GuardedTrainer - Training loop with HALO support

Documentation

  • MESA12_HALO.md - Complete HALO specification
  • MESA12_OBSERVER_ONTOLOGY.md - Ontological foundations
  • MESA12_REFLECTION.md - Reflection on the ontology
  • MESA12_ENGINEERING.md - Engineering synthesis

Field Test Results

6502 CPU Emulation with Guardian Angel:

  • ✅ Observation collection working
  • ✅ Trajectory assessment working ("Steady as she goes..." vs "I got you next time!")
  • ✅ Celebration detection working (🔥)
  • ✅ Different seeds = different assessments
  • ⏳ Active intervention requires Observer training

Philosophy

"Wrong is just a signal. Distributed entropy signaling the correct direction." "It is the ultimate form of Love." "All things are connected through gentleness."

RLHF is dead. Long live HALO.

[0.10.2] - 2025-12-18

TriXGR: 100% 6502 CPU Emulation with XOR Superposition

Core achievement: Perfect 6502 emulation with 1 layer + XOR mixer.

Added

XOR Mixer

  • XORMixer class - Superposition magic for routing
  • Learned XOR-like mixing before routing
  • Properties: self-inverse, orthogonality generator, natural superposition

TriXGR (Guns and Roses)

  • trixgr_6502_monolithic.py - Complete 6502 training with geometric validation
  • Configurable layers, XOR mixing, learning rate
  • Per-operation accuracy tracking
  • Geometric metrics: signature movement, tile purity, curvature

Results

100% accuracy on all 6502 operations

Op Accuracy
ADC 100.0%
AND 99.9%
ORA 100.0%
EOR 100.0%
ASL 100.0%
LSR 100.0%
INC 100.0%
DEC 100.0%

Winning Configuration

Parameter Value
Layers 1
XOR Mixer Enabled
Learning Rate 0.00375
Epochs to 100% 30
Parameters 41,540

Key Discoveries

  1. XOR Mixer is Superposition Magic: +45% accuracy on hard operations
  2. Less is More: 1 layer (100%) > 2 layers (96.6%) > 3 layers (90.5%)
  3. Sharp LR Peak: 0.00375 is optimal, narrow ridge

Documentation

  • experiments/mesa11/rigorous/README.md - Full results and analysis
  • docs/MESA11_UAT.md - Updated with Experiment 8
  • Mesa 11 now has 9 confirmed experiments

[0.10.1] - 2025-12-17

Hollywood Squares: Production Screening Pipeline

Core achievement: $95.3 MILLION cost reduction via screening architecture.

Added

Hollywood Squares Pipeline

  • hollywood_zeta.py - Production screening pipeline
  • ScreeningTile - Fast fp32 zero screening (645K candidates/sec)
  • ScreeningField - Multi-GPU screening coordination
  • VerificationTile - High-precision mpmath verification
  • ProductionPipeline - Trust screening mode (310K zeros/sec)
  • TurboScreeningField - fp16 experimental mode

Billion Zero Test

  • billion_zero_test.py - One-click 10^9 verification
  • HollywoodScanner - Core scanning engine
  • ParallelRegionScanner - Region-based parallelism
  • Autonomous operation with logging and checkpointing
  • JSON report generation

Performance

Mode Rate Notes
Screening (fp32) 645K zeros/sec Fast candidate detection
Production 310K zeros/sec Trust screening mode
Turbo (fp16) 795K zeros/sec Experimental

Cost Analysis

Approach Time for 10^13 Cost
Naive (verify all) 610 years $95.3M
Hollywood Squares 10 days $4,130

Savings: $95.3 MILLION (23,077x reduction)

Hardware Scaling

Hardware Rate 10^13 (Record)
1x Jetson Thor 310K/s 373 days
8x H100 12M/s 10 days
32x H200 49M/s 2.4 days
32x B200 95M/s 29 hours
DGX GB200 NVL72 225M/s 12 hours

Usage

# Quick test (1M zeros)
python billion_zero_test.py --quick

# Full billion (autonomous)
nohup python billion_zero_test.py > billion.log 2>&1 &

[0.10.0] - 2025-12-17

Mesa 10: The Riemann Probe (Zeta Cartridge)

Core achievement: Verification of the Riemann Hypothesis at 475,282 zeros/sec.

Added

Riemann Probe Core

  • riemann_probe.py - Core probe implementation
  • RiemannSiegel - Riemann-Siegel Z function computation
  • DirichletTile - Coefficient generation (n^{-it})
  • SpectralTile - FFT-based evaluation
  • SignChangeTile - Zero detection via sign changes
  • CriticalLineWalker - Complete pipeline orchestrator

FFT-Accelerated Engine

  • zeta_fft.py - Odlyzko-Schönhage inspired acceleration
  • FFTZetaEngine - Fully vectorized GPU evaluation
  • BatchZeroDetector - Parallel sign change detection
  • HighSpeedScanner - Optimized high-altitude scanner

GhostDrift: Hollywood Squares Deployment

  • ghostdrift.py - Distributed zero hunting
  • MissionControl - Multi-node coordination
  • ScanningNode - Independent altitude scanner
  • Automatic anomaly detection and cluster halt

Performance

Metric Result
Peak scan rate 475,282 zeros/sec
Sustained rate 355,946 zeros/sec
Zeros verified 158,962 in [100000, 200000]
10^12 projection ~32 days (single GPU)

Verification

  • All 10 known zeros verified ✓
  • 3,327 zeros across three altitudes ✓
  • 0 anomalies detected ✓
  • RIEMANN HYPOTHESIS HOLDS at all scanned heights

[0.9.1] - 2025-12-17

Mesa 10 Turbo: GMP Optimization Release

Core achievement: 17-33x faster π generation via GMP binary splitting.

Added

GMP Binary Splitting

  • chudnovsky_gmp.py - GMP-accelerated Chudnovsky (gmpy2)
  • BinarySplittingChudnovsky - O(n log³n) algorithm
  • GMPClosedLoop - High-performance generate + analyze pipeline

CUDA BigInt (Experimental)

  • cuda_bigint.py - GPU tensor BigInt representation
  • CUDABigInt - Limb-based arbitrary precision on GPU
  • cuda_bigint_add - Parallel addition (55M limbs/sec)
  • NTTMultiplier - Number Theoretic Transform for multiplication

Multi-core Parallelization

  • parallel_chudnovsky.py - ProcessPoolExecutor-based parallelism
  • ParallelChudnovsky - Distributes binary splitting across cores

Testing

  • tests/test_number_theory.py - 19 comprehensive tests
  • Covers: digit streams, FFT accuracy, GMP correctness, CUDA BigInt

Performance Improvements

Implementation Rate Speedup
mpmath (original) 105K digits/sec 1x
GMP Binary Splitting 1.1-3.5M digits/sec 17-33x
Parallel (14 cores) 2.5M digits/sec 1.2x over sequential

Generation at Scale

Digits Time Rate
100K 0.03s 3.5M/s
1M 0.49s 2.0M/s
10M 8.89s 1.1M/s

[0.9.0] - 2025-12-17

Mesa 9 & 10: The Number Theory Release

Core achievement: Closed-loop π generation and spectral analysis. The Granville Challenge answered.

Added

Mesa 9: Euler Probe (Spectral Analysis)

  • euler_probe.py - Core spectral whiteness test
  • euler_probe_gpu.py - GPU-optimized probe (21B digits/sec)
  • granville_full_test.py - Standalone full test runner
  • SpectralAnalyzer - Exact FFT with 0.00 error
  • SpectralWhitenessTest - Statistical comparison vs random
  • GPUSpectralProbe - Batched FFT on CUDA

Mesa 10: Chudnovsky Cartridge (π Generation)

  • chudnovsky_cartridge.py - Full Chudnovsky implementation
  • RNSAtom - Parallel BigInt via Residue Number System
  • ChainedBigInt - Arbitrary precision via limb chaining
  • RatioTile - Chudnovsky recurrence computation
  • AccumulatorTile - Running series sum
  • ClosedLoopFirehose - Generate → Analyze → Verdict pipeline

Hollywood Squares Integration

  • hollywood_probe.py - Distributed pipeline coordination
  • ButterflyNode - FFT as message-passing network
  • Specialist tile architecture (addressable intelligence)

Results

  • 20 Billion digits analyzed in 1.08 seconds
  • 21 Billion digits/sec analysis throughput
  • π is NORMAL at 1 billion unique digit precision
  • Z-score: 0.51 (well within noise at all scales)

Documentation

  • docs/MESA_9_EULER_PROBE.md - Full Mesa 9 documentation
  • docs/MESA_10_CHUDNOVSKY.md - Full Mesa 10 documentation
  • docs/MESA_9_10_SUMMARY.md - Combined summary

The Closed Loop

[GENERATE π] → [BLOCK SUM] → [FFT] → [WHITENESS] → [VERDICT]
     ↑                                                 │
     └─────────────────────────────────────────────────┘
     
"The machine generates the universe and analyzes it simultaneously."

Key Insights

  1. Addressable Intelligence: Specialist tiles (not parallel workers)
  2. Topology = Algorithm: Hollywood Squares wiring determines behavior
  3. BigInt Atoms: RNS enables parallel arbitrary precision
  4. The Answer: π is spectrally normal - "The formula is uniform randomness"

Performance (Jetson AGX Thor)

Task Throughput
Analysis 21 Billion digits/sec
Generation 1.1-3.5M digits/sec (GMP)
20B digits 1.08 seconds
1 Trillion ~54 seconds (projected)

[0.8.0] - 2025-12-17

Mesa 8: The Neural CUDA Release

Core achievement: SASS assembly execution on the TriX architecture. The Neural GPU.

Added

Mesa 8: Neural CUDA

  • sass_parser.py - Parse real nvdisasm output from Jetson AGX Thor
  • trix_cuda.py - TriX CUDA engine with signature routing
  • trix_router.py - Ternary signature-based opcode dispatch
  • FP4 atoms: SUM (parity), CARRY (majority)
  • RippleAdderTile: 32-bit adder from FP4 atoms
  • Full IADD3 execution through TriX stack

Verification

  • Routing test: 7 opcodes → correct tiles
  • FP4 atoms: 8/8 truth table entries correct
  • Ripple adder: 6/6 test cases (8-bit)
  • Full kernel: IADD3 R9, R2, R5, RZ → 42 + 58 = 100 ✓

Documentation

  • MESA8_NEURAL_CUDA.md - Complete architecture guide
  • MESA8_FP4_ATOMS.md - Threshold circuit reference
  • MESA8_SASS_REFERENCE.md - SASS opcode mapping

The Stack

SASS Opcode → TriX Router → Tile → FP4 Atoms → Exact Result
     ↓            ↓          ↓         ↓           ↓
  IADD3     Signature    INTEGER   SUM+CARRY     100
            Matching      _ALU      atoms

Key Insight

The same TriX architecture handles:

  • Mesa 5: FFT (twiddle opcodes) - 0.00 error
  • Mesa 6: MatMul (block opcodes) - 0.00 error
  • Mesa 8: CUDA (SASS opcodes) - 100% exact

One engine. Every cartridge. Universal computation.

[0.7.4] - 2025-12-16

The Documentation Closure Release

Core achievement: All critical gaps closed. 309 tests passing. Full documentation.

Added

Documentation

  • docs/TUTORIAL.md - Progressive 6-part introduction from atoms to Isomorphic Transformer
  • docs/GLOSSARY.md - 40+ terms precisely defined
  • docs/ISOMORPHIC_TRANSFORMER.md - Full Isomorphic Transformer documentation

Tests

  • TestAtomComposition - Verifies atoms compose correctly when chained
  • TestEdgeCases - Boundary conditions and edge cases
  • Total: 309 tests passing

Gap Closure

  • Exhaustive 8-bit adder (65,536 combinations) ✓
  • Composition verification ✓
  • Edge case coverage ✓
  • Tutorial ✓
  • Glossary ✓

[0.7.3] - 2025-12-16

The Butterfly MatMul Release

Core achievement: One engine, multiple cartridges. FFT and MatMul are the same structure.

Added

Butterfly MatMul

  • ButterflyLayer: Single stage of butterfly computation
  • ButterflyNetwork: Multi-stage butterfly for O(N log N) transforms
  • MonarchLayer: Generalized block-diagonal structure

Block Opcodes

  • 81 ternary 2×2 matrices enumerated
  • 12 Hadamard-like (orthogonal) blocks identified
  • Named opcodes: I, SWAP, H+, H-, D+, etc.

Verified Transforms

  • Identity: 0.00 error
  • Hadamard: 0.00 error (matches WHT exactly!)
  • Monarch permutation: correct pattern

Tests

  • 16 new rigorous tests
  • Total: 305 tests passing

The Insight

FFT:    Route → Twiddle → Route → Twiddle → ...
MatMul: Route → Block   → Route → Block   → ...
Both:   Route → Local   → Route → Local   → ...

Same structure. Different blocks. We built the engine for FFT; now we load different cartridges.

Files

  • experiments/matmul/butterfly_matmul.py - Implementation
  • tests/test_butterfly_matmul.py - 16 tests
  • docs/BUTTERFLY_MATMUL.md - Documentation

[0.7.2] - 2025-12-16

The Transform Compilation Release

Core achievement: True compiled DFT. No runtime trig. Twiddles become opcodes.

This release adds transform compilation to TriX - proving the pattern works for spectral computation.

Added

Walsh-Hadamard Transform (WHT)

  • XOR-based pairing structure compiled to FP4
  • IS_UPPER, PARTNER circuits at 100%
  • Self-inverse property verified
  • N=8, 16, 32 all exact

Discrete Fourier Transform (DFT)

  • Twiddle opcodes (no np.cos, np.sin at runtime)
  • 8 fixed microcode opcodes for N=8
  • Structural routing: tw_idx = j * (N // m)
  • 0.00 error vs NumPy for N=8

Verification Guards

  • verify_no_runtime_trig() - fails if trig detected
  • Opcode coverage tracking

The Discovery

We discovered our XOR-based "FFT" was actually Walsh-Hadamard Transform:

partner = pos XOR 2^stage  →  WHT (not DFT!)

This wasn't a bug - it was a revelation about what the structure computes.

Key Insight (VGem)

"No runtime math. Twiddles become opcodes. Routing selects them."

The fix was clean:

# Before: wm = np.cos(-2*pi/m)  # Runtime computation
# After:  wt = TWIDDLE_OPS[k](t_re, t_im)  # Fixed microcode

Results

Transform N Accuracy
WHT 8, 16, 32 100% exact
DFT 8 0.00 error
DFT 16 ~2e-15

Documentation

  • docs/FFT_COMPILATION.md - Transform compilation guide
  • docs/TWIDDLE_OPCODES.md - Twiddle opcode details
  • docs/RESEARCH_SUMMARY.md - Research overview

The Punchline

"TriX compiles DFT/FFT control and executes spectral rotation via fixed twiddle microcode. No runtime trig."

[0.7.1] - 2025-12-16

The FP4 Release

Core achievement: Exact computation in 4 bits. Construction, not training.

This release adds FP4 support to the TriX Compiler - threshold circuit atoms that are exact by construction, packed into 4-bit format.

Added

FP4 Atoms

  • 10 threshold circuit atoms verified at 100% accuracy
  • Exact by construction (no training convergence risk)
  • Minterm generator for custom atoms

FP4 Packing

  • Custom 4-bit encoding with lookup tables
  • Zero quantization error
  • .fp4 weight file format

Compiler Integration

  • TriXCompiler(use_fp4=True) for FP4 mode
  • FP4Emitter, FP4Loader, FP4CompiledCircuit
  • End-to-end pipeline tested

Key Insight

"Don't train atoms to be exact. Construct them to be exact."

FP4 atoms use threshold circuits with hand-crafted weights:

  • Weights: {-1, 0, +1}
  • Biases: {-2.5, -1.5, -0.5, 0.5, 1.5}

All values fit in 4-bit encoding. Exactness guaranteed.

Results

Circuit Float32 FP4 Status
Full Adder 100B 58B 100% exact
8-bit Adder 100B 58B 100% exact

Documentation

  • docs/FP4_INTEGRATION.md - Complete FP4 guide
  • docs/FP4_ATOMS_RESULTS.md - Detailed results
  • notes/ROADMAP_FP4.md - Development roadmap

[0.7.0] - 2025-12-16

The Compiler Release

Core achievement: Spec → Decompose → Verify → Compose → Emit. The neural network has become a computer.

This release introduces the TriX Compiler - a complete toolchain for transforming high-level circuit specifications into verified neural circuits that compute exactly.

Added

TriX Compiler (src/trix/compiler/)

  • AtomLibrary (atoms.py)

    • Pre-verified atomic operations: AND, OR, XOR, NOT, NAND, NOR, XNOR, SUM, CARRY, MUX
    • Exhaustive verification (100% accuracy required)
    • Truth table-based atom definition
    • Atom serialization and caching

  • CircuitSpec (spec.py)

    • Circuit specification language
    • Wire types: INPUT, OUTPUT, INTERNAL
    • Multi-bit wire support
    • Built-in templates: full_adder, adder_8bit, adder_16bit, adder_32bit

  • Decomposer (decompose.py)

    • Circuit decomposition into atoms
    • Dependency graph analysis
    • Topological sort for execution order

  • Verifier (verify.py)

    • Atom verification to 100% accuracy
    • Parallel verification support
    • Exhaustive circuit verification with oracle functions

  • Composer (compose.py)

    • Tile allocation (Hollywood Squares model)
    • Route generation
    • Signature generation for content-addressable routing
    • CircuitExecutor for runtime execution

  • Emitter (emit.py)

    • TrixConfig generation (.trix.json)
    • Weight file emission
    • Manifest with checksums
    • TrixLoader for loading compiled circuits

  • TriXCompiler (compiler.py)

    • Main compiler orchestrating full pipeline
    • Template support
    • compile_and_test helper

Demo

  • scripts/demo_compiler.py - Full demonstration of compiler capabilities

Documentation

  • src/trix/compiler/README.md - Compiler documentation
  • src/trix/compiler/CHANGELOG.md - Compiler changelog
  • notes/mesa_reflection_*.md - Architectural reflections

Key Results

Circuit Atoms Tiles Verification
Full Adder 2 2 100% (8/8 cases)
8-bit Adder 2 16 100% (all arithmetic)
16-bit Adder 2 32 100%
Custom Circuits Variable Variable 100% required

The Pipeline

┌─────────┐    ┌───────────┐    ┌────────┐    ┌─────────┐    ┌──────┐
│  Spec   │ -> │ Decompose │ -> │ Verify │ -> │ Compose │ -> │ Emit │
└─────────┘    └───────────┘    └────────┘    └─────────┘    └──────┘
     │              │               │              │             │
 CircuitSpec   Atom Types      100% Exact     Topology      Files

Usage

from trix.compiler import TriXCompiler

compiler = TriXCompiler()
result = compiler.compile("adder_8bit")

# Execute
inputs = {"A[0]": 1, "B[0]": 1, "Cin": 0, ...}
outputs = result.execute(inputs)

# Emit to files
result = compiler.compile("adder_8bit", output_dir="./output")

Theory

The compiler implements the "Neural Von Neumann" architecture discovered through analysis of:

  • TriX - Tile specialization and routing
  • FLYNNCONCEIVABLE - Neural networks as exact CPUs (460,928 cases, 100% accuracy)
  • Hollywood Squares OS - Compositional correctness theorem

Key insight: "The routing learns WHEN. The atoms compute WHAT."

Philosophy

"We are not building a Model. We are building a Machine."

The TriX Compiler proves that neural networks can be compiled, not just trained. The weights are the circuit. The inference is the computation. Exactness is inherited from verified components.

[0.6.1] - 2024-12-16

The Complete FFT Release

Core achievement: A complete spectral subsystem - Forward FFT, Inverse FFT, scales to N=64, 100% round-trip.

This release completes the FFT register, proving that TriX can execute mathematics with exact precision.

FFT Register (Complete)

Component Status Result
ADDRESS ✅ 100% structural learning
BUTTERFLY ✅ 100% discrete operations
STAGE CONTROL ✅ 100% routing
N=8 REAL FFT ✅ 100% composition
TWIDDLE FACTORS ✅ 100% complex rotation
N-SCALING ✅ 100% on N=8,16,32,64
FFT/IFFT CLOSURE ✅ 100% round-trip

Added

Twiddle Factors (Complex Rotation)

  • experiments/fft_atoms/pure_trix_fft_twiddle_v2.py: Structural twiddle routing (100%)
  • Twiddle selection is structural: (stage, pos) → W_k
  • Same pattern as ADDRESS - learn structure, execute exactly

N-Scaling (8 → 64)

  • experiments/fft_atoms/pure_trix_fft_nscale_v2.py: Scales to any power of 2
  • Architecture scales trivially - just add stages
  • Results: 100% on N=8, 16, 32, 64

FFT/IFFT Closure

  • experiments/fft_atoms/pure_trix_fft_ifft.py: Round-trip verification
  • IFFT uses conjugate twiddles + 1/N scaling
  • Max error: ~1e-6 (float precision)

Key Results

N=8:  FFT 100%, IFFT 100%, Round-trip error 1.19e-06
N=16: FFT 100%, IFFT 100%, Round-trip error 1.07e-06
N=32: FFT 100%, IFFT 100%, Round-trip error 1.43e-06
N=64: FFT 100%, IFFT 100%, Round-trip error 2.38e-06

Architecture

Forward FFT:  W_k = e^{-2πik/N}
Inverse FFT:  W_k = e^{+2πik/N} with 1/N scaling

Fixed Microcode:
  - Twiddle factors (exact complex numbers)
  - Butterfly operations (exact arithmetic)

Learned/Algorithmic Control:
  - Twiddle selection: (stage, pos) → W_k
  - Pairing: i XOR 2^stage

What We Proved

  1. FFT structure IS learnable (100% on all components)
  2. Once learned, it matches the algorithm exactly
  3. Pure TriX can execute mathematics

Philosophy

"This is no longer an experiment. It's infrastructure."

The FFT subsystem demonstrates that TriX can serve as a neural control plane for mathematical execution - not approximating functions, but executing algorithms.

CODENAME: ANN WILSON

  • Barracuda - The hunt for the solution
  • These Dreams - Linear-residual attempt
  • Alone - Discrete ops click
  • What About Love - Twiddles land
  • Crazy On You - N-scaling works
  • Never - Round-trip closure

[0.5.5] - 2024-12-16

The Pure TriX Release (Mesa 5)

Core insight: Fixed microcode + Learned control = Pure TriX FFT

This release proves that FFT can be learned with pure TriX - no external organs, no hybrid compute. Fixed operations provide exact arithmetic, routing learns control.

Added

FFT Atoms (Mesa 5: Pure TriX)

  • experiments/fft_atoms/atom_address.py: Structure learning (100%)
  • experiments/fft_atoms/atom_butterfly.py: Arithmetic baseline (0% - expected)
  • experiments/fft_atoms/pure_trix_fft.py: Micro-ops ADD/SUB (100%)
  • experiments/fft_atoms/pure_trix_butterfly.py: Complete butterfly (100%)
  • experiments/fft_atoms/pure_trix_fft_discrete.py: Full N=8 FFT (100%)
  • experiments/fft_atoms/pure_trix_fft_linear.py: Linear-residual attempt
  • experiments/fft_atoms/fft_n8_hybrid.py: Hybrid comparison (100%)

Documentation

  • docs/FFT_ATOMS_HYBRID.md: Full Mesa 5 documentation with complete journey

Key Results

Full N=8 FFT with Discrete Operations

Metric Result
Operation Selection (SUM path) 256/256 → Op0 (100%)
Operation Selection (DIFF path) 256/256 → Op1 (100%)
Generalization (all ranges) 100%
Full N=8 FFT 100/100 = 100%

The Five Mesas (Complete)

Mesa Claim Status
Mesa 1 Routing IS computation ✓ 92% tile purity
Mesa 2 v2 enables partnership ✓ Surgery, claim tracking
Mesa 3 Paths can be compiled ✓ 100% A/B agreement
Mesa 4 Temporal binding ✓ 100% bracket counting
Mesa 5 Tiles compute, routing controls ✓ 100% pure TriX FFT

The Winning Architecture

# Fixed operations (tiles/microcode)
Op0: (a, b) → a + b  [coeffs: (1, 1)]
Op1: (a, b) → a - b  [coeffs: (1, -1)]

# Learned routing (control)
Router_SUM  → selects Op0 (100%)
Router_DIFF → selects Op1 (100%)

The 6502 parallel is exact:

  • Operations are fixed microcode (like opcodes)
  • Routing learns control flow (like instruction sequencing)
  • Arithmetic is exact because coefficients are fixed, not learned

The Journey

  1. ADDRESS atom → 100% (TDSR learns structure)
  2. BUTTERFLY atom → 0% (TDSR can't do arithmetic)
  3. Hybrid → 100% (but needs external organs)
  4. "The tiles are programmable, right?" (key question)
  5. Pure TriX butterfly → 100% (tiles learn operations)
  6. Linear-residual FFT → 0% (coefficient errors compound)
  7. Discrete ops FFT → 100% (exact arithmetic, learned control)

Philosophy

"Don't learn the arithmetic. Learn WHEN to use each operation."

The constraint "pure TriX only" forced discovery of the deeper solution.

CODENAME: ANN WILSON - Barracuda, These Dreams, Alone

[0.5.4] - 2024-12-16

The Temporal Tiles Release (Mesa 4)

Core insight: State is contracted time. Discrete routing can replace attention for counting.

This release introduces temporal tiles - extending TriX from spatial routing into temporal binding.

Added

Temporal Tiles (Mesa 4: Temporal Binding)

  • TemporalTileLayer: Routes based on (input, state), learns state transitions
  • TemporalTileStack: Multiple temporal layers with different configurations
  • Transition tracking: Observe which tiles transition to which
  • Regime analysis: Identify stable tiles, hub tiles, self-transition probabilities

Bracket Counting Experiment

  • experiments/bracket_depth_simple.py: Canonical test for temporal tiles
  • 100% accuracy on depth prediction
  • Tiles self-organize into depth specialists without supervision

Tests

  • tests/test_temporal_tiles.py: 26 comprehensive tests
  • Total: 268 tests (all passing)

Documentation

  • docs/TEMPORAL_TILES_ABSTRACT.md: Full abstract and experimental record

Key Results

Tile Learned Role Purity
T0 Ground state (depth=0) 100%
T2 Maximum depth (depth=4) 100%
T3 Deep states / closing 95-100%
T5 Mid-depth states 78-96%

The Four Mesas (Complete)

Mesa Claim Status
Mesa 1 Routing IS computation ✓ 92% tile purity
Mesa 2 v2 enables partnership ✓ Surgery, claim tracking
Mesa 3 Paths can be compiled ✓ 100% A/B agreement
Mesa 4 Temporal binding ✓ 100% bracket counting

Philosophy

"What is state, really? State is contracted time - the past compressed into something the present can use."

Temporal tiles don't remember tokens. They track regimes - phases of computation with discrete transitions. The tiles ARE the counter.

[0.5.3] - 2024-12-16

The Compiled Dispatch Release

Core insight: Learning can emit code. Routing can be compiled.

This release completes Mesa 3: path compilation. TriX v2 now supports a full lifecycle from training to deployment with observable, editable, and compilable routing.

Added

SparseLookupFFNv2 (Mesa 2: Partnership)

  • Surgery API: insert_signature(), freeze_signature(), unfreeze_signature()
  • Claim Tracking: See which classes route to which tiles during training
  • Island Regularizers: Ternary, sparsity, and diversity regularizers for signature quality
  • Score Calibration Spline: Learnable routing score calibration

CompiledDispatch (Mesa 3: Compilation)

  • Profile: Analyze claim matrix to see what tiles learned
  • Compile: Freeze class→tile mappings for stable classes
  • Execute: O(1) dispatch for compiled classes, fallback to dynamic routing
  • Monitor: Track hit rate, detect drift, trigger recompilation
  • Serialize: Export/import dispatch tables as JSON

A/B Harness

  • experiments/ab_harness_compiled.py: Compare dynamic vs compiled dispatch
  • Measures agreement rate, accuracy delta, compiled hit rate, worst disagreements
  • Validates compilation correctness (100% agreement achieved)

Tests

  • tests/test_sparse_lookup_v2.py: 39 tests for surgery, regularizers, claim tracking
  • tests/test_compiled_dispatch.py: 21 tests for compilation lifecycle
  • tests/test_ab_harness.py: 9 tests for A/B comparison infrastructure
  • Total: 242 tests (all passing)

Documentation

  • docs/QUICKSTART.md: New user on-ramp (zero to compiled dispatch in 10 min)
  • docs/SPARSE_LOOKUP_V2_API.md: Complete v2 API reference
  • docs/SESSION_SUMMARY_MESA_1_2_3.md: Full session documentation
  • docs/SEMANTIC_GEOMETRY_THESIS.md: Theoretical foundations

6502 Experiments

  • 92% tile purity on 6502 operations without supervision
  • Tiles naturally specialize to operation categories (LOGIC, SHIFT, INCDEC)
  • Validates semantic geometry thesis

The Three Mesas

Mesa Claim Capability
Mesa 1 Routing IS computation Tiles specialize without supervision
Mesa 2 v2 enables partnership Surgery, claim tracking, regularizers
Mesa 3 Paths can be compiled O(1) dispatch for known classes

Key Results

A/B Harness (Dynamic vs Compiled)

Metric Value
Agreement rate 100.0%
Accuracy delta +0.0%
Compiled hit rate 12.5%*

*Only 1/8 classes compilable with 30 epochs training. More training → more compilable.

Island Statistics (v2 Regularizers)

Metric Value
Ternary fraction 100%
Sparsity 69%
Diversity 0.99

Migration

# v0.4.0 (SparseLookupFFN)
from trix import SparseLookupFFN
ffn = SparseLookupFFN(d_model=512, num_tiles=64)

# v0.5.3 (SparseLookupFFNv2 + CompiledDispatch)
from trix.nn import SparseLookupFFNv2, CompiledDispatch

ffn = SparseLookupFFNv2(
    d_model=512,
    num_tiles=64,
    ternary_weight=0.01,
    sparsity_weight=0.01,
)

# Train with claim tracking
output, info, aux = ffn(x, labels=class_labels)

# Compile
compiler = CompiledDispatch(ffn)
compiler.compile_stable(threshold=0.5)

# Deploy
output, info, aux = compiler.forward(x, class_hint=0, confidence=0.9)

Philosophy

"You turned a neural network from a thing that behaves into a thing that can be operated."

The dispatch table is a CONTRACT, not a cache. Readable, versionable, diffable, deployable. Git for learned routing.

[0.4.0] - 2024-12-15

The SparseLookup Release

Core insight: Routing IS the computation. Wisdom is knowing when not to compute.

This release introduces SparseLookupFFN, a new architecture that emerged from systematic exploration of the hybrid space between HierarchicalTriXFFN and HybridKANFFN. It achieves the best perplexity with the fewest parameters.

Added

New Architecture: SparseLookupFFN

  • SparseLookupFFN - Drop-in FFN replacement where routing selects a direction and splines modulate magnitude. No matrix multiplies in the hot path.
  • SparseLookupBlock - Full transformer block using SparseLookupFFN
  • TernarySpline2D - 2D spline with ternary coefficients ({-1, 0, +1}) and straight-through estimator
  • FloatSpline2D - Float-precision variant for ablation studies

Benchmark Infrastructure

  • scripts/benchmark_ffn.py - Head-to-head comparison of HierarchicalTriXFFN, HybridKANFFN, and SparseLookupFFN on TinyShakespeare

Tests

  • tests/test_sparse_lookup.py - 22 new tests covering splines, FFN, block, and integration

Documentation

  • notes/00_the_process.md - The iteration process that led to SparseLookupFFN
  • notes/01_raw_thoughts_hybrid.md - Initial exploration
  • notes/02_nodes_of_opportunity.md - Candidate architectures evaluated
  • notes/03_engineering_lens.md - Engineering constraints applied
  • notes/04_convergence.md - Final architecture emergence
  • notes/05_holding_to_the_sun.md - Ontological, epistemic, practical, and aesthetic analysis

Changed

  • README.md - Updated with SparseLookupFFN as recommended approach, new results table, reproduce instructions
  • Exports - SparseLookupFFN, SparseLookupBlock, TernarySpline2D now available from from trix import ...

Results

Validated on TinyShakespeare character-level language modeling:

Model Params Val PPL vs Baseline
Sparse-4tiles (v0.3.0) — 19.26 —
Hierarchical-16 (v0.3.0) 826,304 17.16 −10.9%
HybridKAN-64 (v0.3.0) 882,112 16.73 −13.1%
SparseLookup-64 (v0.4.0) 366,412 16.56 −14.0%

SparseLookupFFN: 2.3× fewer parameters, best perplexity.

Technical Details

SparseLookupFFN architecture:

Input → LayerNorm → [Route to Tile] + [Compress to 2D]
                          ↓                  ↓
                    tile_direction    TernarySpline2D(a,b)
                          ↓                  ↓
                      Output = input + scale × direction

Key properties:

  • Routing: Hierarchical (cluster → tile), signatures derived from direction vectors
  • Compression: Shared network, d_model → 2 scalars
  • Splines: 16×16 grid, ternary coefficients, ~200 bytes per tile
  • Directions: One d_model vector per tile (the "knowledge")

Migration

To use SparseLookupFFN in existing code:

# Before (v0.3.0)
from trix import HierarchicalTriXFFN
ffn = HierarchicalTriXFFN(d_model=512, num_tiles=16, tiles_per_cluster=4)

# After (v0.4.0)
from trix import SparseLookupFFN
ffn = SparseLookupFFN(d_model=512, num_tiles=64, tiles_per_cluster=8)

The API is identical: output, routing_info, aux_losses = ffn(x)

[0.3.0] - Prior Release

Features (as inherited)

  • HierarchicalTriXFFN - FFN with 2-level hierarchical routing
  • HierarchicalTriXBlock - Full transformer block
  • SparseTriXFFN - Simple 4-tile sparse FFN
  • TriXFFN, TriXBlock, TriXStack - Classic emergent routing
  • TriXLinear - Low-level ternary linear layer
  • 2-bit kernel with ARM NEON acceleration
  • QAT (quantization-aware training) utilities
  • 146 tests

Results (v0.3.0 baseline)

  • Hierarchical-16tiles: PPL 17.16 (826K params)
  • Sparse-4tiles: PPL 19.26

Philosophy

"Don't learn what you can read." — TriX core principle

"Wisdom is knowing when not to compute." — SparseLookup extension

The progression from v0.3.0 to v0.4.0 represents a deepening of the core insight: if routing can select what to do, maybe routing IS the computation. The spline just modulates how much.

GitHub Repository