Skip to content

v0.7.0 - Enhanced Randomness Parameters

Choose a tag to compare

View all tags
@scouzi1966 scouzi1966 released this 23 Sep 02:26
· 561 commits to main since this release
v0.7.0
05e56b8

🚀 afm v0.7.0 - Enhanced Randomness Parameters with Advanced Sampling Support (AFM Generation Options)

There is no evidence of this because I didn't keep historical test data but my gut feeling is that Apple recently tuned defaults of the model to be more deterministic than earlier MacOS beta versions. For example, it outputs the exact same response for the same prompt. They may have added some cacheing. Who knows.

I've seen the same on IOS devices as well.

This release unlocks official parameters to shape the response beyond Apple's default.

You can read Apple's Doc on what they call 'Generation options'if you're interested:

https://developer.apple.com/documentation/foundationmodels/generationoptions

This release introduces comprehensive enhancements to the randomness parameter system, providing advanced sampling control while maintaining full backward compatibility.

✨ Major New Features

🎯 Enhanced Randomness Parameters

  • Nucleus Sampling (Top-P): random:top-p=<0.0-1.0> - Controls diversity by probability threshold
  • Top-K Sampling: random:top-k=<positive integer> - Limits selection to K most likely tokens
  • Seeded Random: random:seed=<value> - Reproducible results with specific seeds
  • Combined Modes: random:top-p=0.9:seed=42 - Mix parameters for precise control
  • Greedy sampling: greedy setting - more deterministic mode
  • Temperature setting: -t <temperature> between 0 and 1
  • **Available in all methods **: Works in both single and server modes

🧪 Comprehensive Testing Suite

  • New Test Script: test-go.sh with 24+ test cases covering all parameter combinations
  • Flexible Configuration: Command-line options for binary path and test prompts
  • Complete Validation: Tests for conflict detection and error handling
  • Timestamped Logging: Detailed test results with full audit trail

🛡️ Enhanced Validation & Error Handling

  • Conflict Detection: Properly rejects invalid parameter combinations
  • Clear Error Messages: Helpful feedback explaining Apple Foundation Models API constraints
  • Comprehensive Validation: Range checking for all parameter types

📝 Usage Examples

CLI Usage

# Basic usage (backward compatible)
./afm -r greedy -s "Tell me a story"
./afm -r random -s "Tell me a story"

# Advanced sampling modes
./afm -r "random:top-p=0.9" -s "Tell me a story"           # Nucleus sampling
./afm -r "random:top-k=50" -s "Tell me a story"            # Top-k sampling
./afm -r "random:seed=42" -s "Tell me a story"             # Seeded random
./afm -r "random:top-p=0.9:seed=42" -s "Tell me a story"   # Combined mode

# Server mode with advanced parameters
./afm -r "random:top-p=0.9:seed=42" -p 9999

Testing

# Run comprehensive test suite
./test-go.sh

# Test with custom configuration
./test-go.sh -b /path/to/afm -s "Custom test prompt"

🔧 Technical Improvements

Code Quality Enhancements

  • Centralized Debug Logger: Sources/MacLocalAPI/Utils/DebugLogger.swift eliminates code duplication
  • Enhanced Documentation: Comprehensive comments explaining Apple Foundation Models API constraints
  • Better Error Handling: Clear validation messages for invalid parameter combinations

Parameter Validation

  • Apple API Compliance: Enforces Foundation Models constraints (single sampling method only)
  • Range Validation: top-p (0.0-1.0), top-k (positive integers), seeds (non-negative integers)
  • Conflict Detection: Rejects invalid combinations like random:top-p=0.9:top-k=50

📋 Requirements

  • macOS 26+ (Tahoe) with Apple Intelligence enabled
  • Apple Silicon Mac (M1/M2/M3/M4 series)
  • Foundation Models framework available

🔄 Backward Compatibility

✅ Fully Backward Compatible - All existing usage patterns continue to work:

  • --randomness greedy and --randomness random work exactly as before
  • All existing scripts and integrations remain functional
  • No breaking changes to existing functionality

📦 Installation

Download & Run

# Download and extract
curl -L https://github.com/scouzi1966/maclocal-api/releases/download/v0.7.0/afm-v0.7.0-arm64.tar.gz | tar -xz
chmod +x afm

# Test the installation
./test-go.sh

Homebrew (Formula will be updated shortly)

brew tap scouzi1966/afm
brew install afm

🚨 Important Notes

Parameter Constraints (per Apple Foundation Models API)

  • Single Sampling Method: Cannot combine top-p and top-k in one request
  • Clear Error Messages: Invalid combinations are rejected with helpful explanations
  • Seed Compatibility: Seeds can be used with any sampling method for reproducibility

Examples of Invalid Usage

# These will be rejected with clear error messages
./afm -r "random:top-p=0.9:top-k=50" -s "test"    # Cannot combine sampling methods
./afm -r "random:top-p=1.5" -s "test"             # top-p out of range  
./afm -r "random:top-k=-5" -s "test"              # Invalid top-k value

🔗 Links

SHA256: 11afe70d44987c5cc96ed7b4e6eb8725b289ccb6ab4e3b67b60527c373c2bb57

Assets 3
Loading