Skip to content

PACKAGE.md

Latest commit

 

History

History
229 lines (168 loc) · 4.66 KB

PACKAGE.md

File metadata and controls

229 lines (168 loc) · 4.66 KB
Raw

LLMocal Package Distribution Guide

This document explains how to build, test, and publish the LLMocal Python package.

Overview

LLMocal is now set up as a modern Python package that can be:

  • Installed via pip install llmocal
  • Installed via uv add llmocal
  • Used programmatically with import llmocal
  • Used from command line with llmocal chat

Package Structure

The package follows modern Python packaging standards with:

  • pyproject.toml: Modern package configuration
  • MANIFEST.in: File inclusion rules for distribution
  • CHANGELOG.md: Version history tracking
  • examples/: Usage examples for developers
  • High-level API: Easy-to-use LLMocal class for quick integration

Building the Package

1. Prerequisites

  • Python 3.11+
  • uv (for dependency management)
  • twine (for PyPI uploads)
# Install build dependencies
uv add --dev twine build

2. Build Process

# Clean previous builds
rm -rf build/ dist/ *.egg-info/

# Build the package
uv build

# This creates:
# - dist/llmocal-X.X.X.tar.gz (source distribution)
# - dist/llmocal-X.X.X-py3-none-any.whl (wheel)

3. Test the Built Package

# Test imports and basic functionality
uv run python -c "
import llmocal
client = llmocal.LLMocal()
print('Package works correctly!')
"

# Test the CLI command
uv run llmocal --help

Publishing to PyPI

Automated Publishing

Use the provided script:

./scripts/publish.sh

This script will:

  1. Clean previous builds
  2. Run tests
  3. Build the package
  4. Validate the package
  5. Show package info
  6. Ask for confirmation
  7. Upload to PyPI

Manual Publishing

# 1. Build the package
uv build

# 2. Check the package
uv run python -m twine check dist/*

# 3. Upload to PyPI (requires API token)
uv run python -m twine upload dist/*

# Or upload to test PyPI first
uv run python -m twine upload --repository testpypi dist/*

Version Management

Before publishing a new version:

  1. Update version in pyproject.toml
  2. Update CHANGELOG.md with new changes
  3. Test thoroughly in different environments
  4. Create a git tag for the version
# Example version update workflow
git checkout main
git pull origin main

# Edit pyproject.toml and CHANGELOG.md
# Test everything works

git add .
git commit -m "Release v1.0.5"
git tag v1.0.5
git push origin main --tags

Package Usage Examples

For End Users (Simple)

import llmocal

# Quick start - downloads model automatically
client = llmocal.LLMocal()
client.setup()
response = client.chat("Hello, how are you?")
print(response)

For Developers (Advanced)

import llmocal
from llmocal import LLMocalConfig, ModelManager

# Custom configuration
config = LLMocalConfig(n_ctx=8192, n_threads=8)
client = llmocal.LLMocal(config=config)

# Custom model
client = llmocal.LLMocal(
    repo_id="TheBloke/CodeLlama-7B-Instruct-GGUF",
    filename="codellama-7b-instruct.Q4_K_M.gguf"
)

# Direct access to components
model_manager = ModelManager()
engine = client.engine  # After setup()

Command Line Usage

# Install the package
pip install llmocal

# Use the CLI
llmocal chat
llmocal chat --repo-id "TheBloke/Llama-2-7B-Chat-GGUF" --filename "llama-2-7b-chat.Q4_K_M.gguf"

Development Installation

For development work:

# Clone the repository
git clone https://github.com/alexnicita/llmocal.git
cd llmocal

# Install in development mode with uv
uv sync

# Run in development mode
uv run python -m llmocal.cli chat

Package Features

✅ Modern packaging - Uses pyproject.toml with proper metadata
✅ Easy installation - pip install llmocal or uv add llmocal
✅ Simple API - High-level LLMocal class for quick usage
✅ Advanced API - Access to all internal components
✅ CLI commands - llmocal chat command available
✅ Model flexibility - Support for any GGUF model from Hugging Face
✅ Documentation - Complete README with examples
✅ Type hints - Full type annotation support
✅ Cross-platform - Works on macOS, Linux, and Windows

Troubleshooting

Build Issues

# Clear cache and retry
rm -rf build/ dist/ *.egg-info/ .uv-cache/
uv clean
uv build

Import Issues

# Test in clean environment
uv venv test-env
source test-env/bin/activate
pip install dist/llmocal-*.whl
python -c "import llmocal; print('OK')"

Dependency Issues

# Update lock file
uv lock --upgrade
uv sync

Support