Skip to content

Installation Guide

Jump to bottom
Michael Kuykendall edited this page Sep 8, 2025 · 1 revision

Installation Guide

Complete installation instructions for all platforms.

🚀 Quick Installation

Pre-built Binaries (Recommended)

Linux

curl -L https://github.com/Michael-A-Kuykendall/shimmy/releases/latest/download/shimmy-linux-x86_64 -o shimmy
chmod +x shimmy
sudo mv shimmy /usr/local/bin/

macOS

# Intel Macs
curl -L https://github.com/Michael-A-Kuykendall/shimmy/releases/latest/download/shimmy-macos-intel -o shimmy
chmod +x shimmy
sudo mv shimmy /usr/local/bin/

# Apple Silicon (M1/M2)
curl -L https://github.com/Michael-A-Kuykendall/shimmy/releases/latest/download/shimmy-macos-arm64 -o shimmy
chmod +x shimmy
sudo mv shimmy /usr/local/bin/

Windows

# PowerShell
Invoke-WebRequest -Uri "https://github.com/Michael-A-Kuykendall/shimmy/releases/latest/download/shimmy-windows-x86_64.exe" -OutFile "shimmy.exe"

# Or with curl
curl -L https://github.com/Michael-A-Kuykendall/shimmy/releases/latest/download/shimmy.exe -o shimmy.exe

Package Managers

Rust/Cargo (All Platforms)

# Basic installation
cargo install shimmy

# With GPU support (CUDA/Metal)
cargo install shimmy --features llama

Homebrew (macOS)

Coming soon

Chocolatey (Windows)

Coming soon

🐳 Docker Installation

# Pull official image
docker pull shimmy/shimmy:latest

# Run with GPU support
docker run --runtime=nvidia --gpus all -p 3000:3000 shimmy/shimmy:latest

# Docker Compose
curl -O https://raw.githubusercontent.com/Michael-A-Kuykendall/shimmy/main/docker-compose.yml
docker-compose up -d

⚙️ Post-Installation Setup

1. Verify Installation

shimmy --version
shimmy discover  # Find available models

2. Download Models

# Place GGUF models in supported directories
mkdir -p ~/.shimmy/models
# OR use existing Ollama models (auto-discovered)

3. Start Server

shimmy serve --bind 0.0.0.0:3000

4. Test API

curl http://localhost:3000/v1/models

🔧 Build from Source

Requirements

  • Rust 1.70+
  • Git
  • CUDA Toolkit (optional, for GPU support)

Build Steps

git clone https://github.com/Michael-A-Kuykendall/shimmy.git
cd shimmy

# Basic build
cargo build --release

# With GPU support
cargo build --release --features llama

# Binary location
./target/release/shimmy

Cross-Compilation

# Linux from macOS/Windows
rustup target add x86_64-unknown-linux-gnu
cargo build --release --target x86_64-unknown-linux-gnu

# macOS from Linux/Windows  
rustup target add x86_64-apple-darwin
cargo build --release --target x86_64-apple-darwin

🚨 Troubleshooting

Windows Security Warning

Windows Defender may flag the binary as a false positive. This is common for new Rust binaries. Solutions:

  1. Add exception in Windows Defender
  2. Use cargo install shimmy instead
  3. Build from source

macOS Gatekeeper

# If macOS blocks execution
sudo xattr -r -d com.apple.quarantine shimmy

Linux Permissions

# If permission denied
chmod +x shimmy

# If binary won't execute
ldd shimmy  # Check dependencies

📊 Installation Verification

After installation, verify everything works:

# Check version
shimmy --version

# Test model discovery
shimmy discover

# Start server (background)
shimmy serve --bind 127.0.0.1:3000 &

# Test API compatibility
curl -X POST http://localhost:3000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "your-model-name",
    "messages": [{"role": "user", "content": "Hello!"}],
    "max_tokens": 50
  }'

⚡ Performance Tips

  • GPU Acceleration: Build with --features llama for CUDA/Metal support
  • Memory: Ensure sufficient RAM for your models (4GB+ recommended)
  • SSD: Use SSD storage for better model loading performance
  • Network: Use --bind 0.0.0.0:3000 for network access

Next Steps: Quick Start Guide | Configuration

Clone this wiki locally