Complete Lightspeed AI-Powered Assistance Implementation with Platform-Based GPU Detection

[tsebastiani]

Summary

This PR introduces a comprehensive Lightspeed AI-powered assistance feature for krknctl, implementing intelligent chaos engineering command suggestions using Retrieval-Augmented Generation (RAG) with automatic GPU detection and acceleration. The implementation includes a complete redesign of the GPU detection system, multi-platform container support, and robust error handling.

🚀 Major Implementation Tasks Completed

1. GPU Detection System Redesign

Previous System: Complex container-based GPU detection using test images ❌

• Removed complex GPU check implementation using container images
• Eliminated GetSupportedGPUTypes() and container-based testing approach

New System: Platform-based automatic detection ✅

• macOS arm64: Automatically assumes Apple Silicon GPU support (Metal via libkrun)
• Linux with NVIDIA devices: Detects physical NVIDIA devices (/dev/nvidia0, /dev/nvidiactl, /dev/nvidia-uvm)
• Generic fallback: CPU-only mode for all other platforms
• Added --no-gpu flag to force CPU-only mode without device mounting

2. Container Runtime Support

• Podman Only: Lightspeed exclusively supports Podman container runtime
• Docker Blocking: Commands fail gracefully with helpful error messages when Docker is detected
• Error Handling: Provides links to Podman GPU documentation (https://podman-desktop.io/docs/podman/gpu)

3. Container Architecture

Three Specialized Containers:

• Apple Silicon (rag-model-apple-silicon): Vulkan backend for Apple M1/M2/M3/M4 GPUs
• NVIDIA (rag-model-nvidia): CUDA backend for NVIDIA GPUs
• Generic (rag-model-generic): CPU-only fallback for all other platforms

Container Selection Logic:

• Uses PlatformGPUDetector.GetLightspeedImageURI() to select appropriate container
• Tag construction follows {rag_model_tag}-{architecture} pattern from config
• Device mounting handled by PlatformGPUDetector.GetDeviceMounts()

4. Multi-Stage Container Build Fix

Problem: Documentation indexing failed in builder stage of multi-stage builds

• Root Cause: Git and Python dependencies not fully available during builder stage
• Solution: Moved documentation indexing from builder stage to runtime stage
• Impact: Fixed NVIDIA and Generic containers (Apple single-stage already worked)

5. Documentation Indexing System

Sources Indexed:

• Local krknctl help documentation
• Live krkn-chaos/website repository (chaos engineering guides)
• Live krkn-chaos/krkn-hub repository (scenario definitions)

Indexing Process:

• Build Time: Creates cached indices for offline/airgapped environments
• Runtime: Can rebuild indices with fresh documentation or use cached versions
• Verification: Automatic validation of indexed document sources and counts

🔧 Technical Implementation

Core Components

• pkg/gpucheck/gpucheck.go: Platform-based GPU detection logic
• cmd/lightspeed_check.go: Lightspeed commands with Docker runtime blocking
• cmd/lightspeed.go: RAG model deployment with GPU-specific container selection
• pkg/config/config.go: Enhanced with Lightspeed-specific configuration methods

Container Files

• containers/lightspeed-rag/Containerfile.apple-silicon: Single-stage Vulkan build
• containers/lightspeed-rag/Containerfile.nvidia: Multi-stage CUDA build
• containers/lightspeed-rag/Containerfile.generic: Multi-stage CPU-only build

Key Functions

• DetectGPUAcceleration(): Platform-based GPU type detection
• deployRAGModelWithGPUType(): GPU-aware container deployment
• HandleContainerError(): Enhanced error reporting with helpful suggestions

🎯 New CLI Commands

Lightspeed Check

# Automatic GPU detection and validation
krknctl lightspeed check

# Force CPU-only mode
krknctl lightspeed check --no-gpu

Lightspeed Run

# AI-powered assistance with auto-detected GPU
krknctl lightspeed run

# Force CPU-only mode (no GPU acceleration)
krknctl lightspeed run --no-gpu

# Offline mode for airgapped environments  
krknctl lightspeed run --offline

🏗️ Configuration Integration

• Config-Based Tags: Uses rag_model_tag from pkg/config/config.json to construct container tags
• Centralized Settings: All RAG service parameters (ports, endpoints, timeouts) in configuration
• Private Registry Support: Full integration with existing private registry authentication

📊 User Experience Improvements

Progress Feedback

• Spinner with dynamic progress messages during container image pulls
• Real-time feedback during RAG model deployment
• Health checking with automatic retry and timeout handling

Error Handling

• Platform-specific error messages with actionable solutions
• Automatic fallback from live indexing to cached documentation
• Container cleanup on deployment failures

🧪 Testing

✅ Comprehensive Test Coverage

• Updated test suite to use new PlatformGPUDetector API
• Platform detection tests for all supported GPU types
• Container deployment tests with mock orchestrator
• Error handling tests for various failure scenarios

✅ Manual Testing Verified

• All three container types build and run successfully
• GPU detection works correctly on Apple Silicon and NVIDIA systems
• Documentation indexing includes all expected sources (krknctl + website + krkn-hub)
• Interactive RAG service provides accurate responses about chaos engineering

📁 Files Changed

New Files

• pkg/gpucheck/gpucheck.go - Platform-based GPU detection logic
• pkg/gpucheck/gpucheck_test.go - Comprehensive unit tests
• cmd/lightspeed_check.go - Lightspeed commands implementation
• cmd/lightspeed_check_test.go - CLI command tests
• cmd/lightspeed.go - RAG model deployment functions
• containers/lightspeed-rag/Containerfile.apple-silicon - Single-stage Vulkan build
• containers/lightspeed-rag/Containerfile.nvidia - Multi-stage CUDA build
• containers/lightspeed-rag/Containerfile.generic - Multi-stage CPU-only build
• containers/lightspeed-rag/rag_service.py - FastAPI RAG service
• containers/lightspeed-rag/index_docs.py - Documentation indexing script
• containers/lightspeed-rag/entrypoint.sh - Container entrypoint
• containers/lightspeed-rag/requirements.txt - Python dependencies

Modified Files

• pkg/config/config.json - Added Lightspeed configuration parameters
• pkg/config/config.go - Added Lightspeed configuration methods
• cmd/root.go - Integrated lightspeed command into CLI structure
• CLAUDE.md - Added comprehensive Lightspeed documentation

🌟 Key Benefits

Intelligent Assistance

• Natural language queries about chaos engineering scenarios
• Smart command suggestions based on user intent
• Comprehensive documentation search across all krkn projects

Automatic GPU Optimization

• Zero configuration GPU detection and utilization
• Cross-platform compatibility (Apple Silicon, NVIDIA, generic CPU)
• Graceful degradation when GPU acceleration is unavailable

Developer Experience

• Simple CLI interface following krknctl conventions
• Comprehensive error messages with actionable solutions
• Offline support for airgapped environments

🚀 Future Enhancements Ready

This foundation enables future integrations with the krkn-lightspeed repository for:

• Enhanced RAG model capabilities
• Advanced chaos engineering assistance
• Integration with additional AI/ML features

🏁 Build and Test Instructions

# Build the project
go build -tags containers_image_openpgp -ldflags="-w -s" .

# Run tests
go test -tags containers_image_openpgp ./pkg/gpucheck ./pkg/config ./cmd

# Test Lightspeed commands
krknctl lightspeed check
krknctl lightspeed run --help

Assisted-by: Claude Sonnet 4
Signed-off-by: Tullio Sebastiani tsebasti@redhat.com

[paigerube14]

tested with the gpu alpha release and works great!!

% krknctl-gpu lightspeed run
📣📦 a newer version of krknctl, v0.10.4-beta, is currently available, check it out! https://github.com/krkn-chaos/krknctl/releases/latest


container runtime: Podman

🔍 Detecting GPU acceleration...
✅ GPU acceleration: Apple Silicon GPU (M1, M2, M3, M4 with Metal via libkrun)

🚀 Deploying lightspeed model...
🚀 RAG container started: bf84c3cb5460e5ea1cdc6850fa544ef4fb9dfad8c9fd7b52cfbeff006ca96151
📡 Port mapping: localhost:8080 -> container:8080

🩺 Performing health check...
🩺 Health checking Lightspeed service at http://localhost:8080/health...
⏳ Waiting for service to become ready... (1/60)
⏳ Waiting for service to become ready... (2/60)
✅ Service healthy: llama-cpp-python:Llama-3.2-1B-Instruct-Q4_K_M.gguf with 199 documents indexed
✅ Lightspeed service is ready!

🤖 Starting interactive Lightspeed service on port 8080...
Type your chaos engineering questions and get intelligent krknctl command suggestions!
Type 'exit' or 'quit' to stop.
🤖 AI Assistant ready! Ask me about krknctl commands or chaos engineering:
📍 Service available at: http://localhost:8080
💡 Try asking: 'How do I run a pod deletion scenario?'
🚪 Type 'exit', 'quit', or press Ctrl+C to stop.

> how can I run a pod scenario on my namespace test-app-1

🤖 To run a pod scenario on your namespace test-app-1, use the following command: 

🎯 krknctl run pod-scenarios --namespace test-app-1

This command will target the pods in the "test-app-1" namespace. If you don't specify a namespace, it will target all pods in the namespace, including those matching the label "app: test". The default value for the "disruption-count" flag is 1. The default value for the "kill-timeout" flag is 180 seconds. If you want to run the scenario for a specific duration, you can specify it as an argument, e.g., `krknctl run pod-scenarios --chaos-duration 600`. Remember that the "source: krkn-hub" data provides the authoritative flags, so make sure to use it over the official documentation. If you're unsure about the flags, please ask and I'll help you out. 🎯

> exit
👋 Goodbye!

🧹 Cleaning up Lightspeed service...
✅ Lightspeed service stopped successfully

[paigerube14]

should we take out any litmus related docs?