pyrattler-recipe-autogen

Automatically generates recipe.yaml files for rattler-build directly from your Python project's pyproject.toml, eliminating the need for manual recipe creation. Features intelligent auto-detection, advanced dependency management, and comprehensive integration support.

✨ Features

🚀 Core Functionality

Automatic Recipe Generation: Creates complete recipe.yaml files from pyproject.toml
Dynamic Version Resolution: Supports setuptools_scm, hatch-vcs, and other dynamic versioning
Intelligent Source Detection: Auto-detects PyPI, Git repositories, and local sources
Smart Dependency Mapping: Converts Python dependencies to conda equivalents

🔧 Advanced Build Configuration

Multi-Backend Support: Works with hatchling, setuptools, flit, poetry backends
Entry Point Detection: Automatically configures console scripts and GUI applications
Platform-Specific Builds: Handles conditional dependencies and platform variants
Custom Build Scripts: Detects and configures custom build processes

🧪 Intelligent Testing

Test Auto-Detection: Finds pytest, unittest, and custom test configurations
Import Testing: Generates appropriate import tests for your package
Test Dependency Management: Separates test requirements from runtime dependencies
Coverage Integration: Supports pytest-cov and coverage.py configurations

🌍 Platform & Environment Support

Cross-Platform Builds: Supports Linux, macOS, Windows, and other platforms
Architecture Detection: Handles x86_64, ARM64, and other architectures
Python Version Matrix: Supports multiple Python version requirements
Conditional Dependencies: Platform-specific and environment-specific packages

📊 Advanced Context Variables

Package Intelligence: Auto-detects package structure and metadata
Build System Analysis: Identifies build backend and configuration
Dependency Categorization: Classifies runtime, development, and optional dependencies
Development Environment: Detects CI/CD, documentation, and tooling setup

🎨 Output Customization

Multiple Formats: Generate YAML or JSON output
Template Customization: Custom templates for different recipe sections
Section Control: Include/exclude specific recipe sections
Validation Framework: Built-in recipe validation and best practice checks
Formatting Options: Configurable indentation, sorting, and style

🔗 Integration Enhancements

Pixi Integration: Deep integration with pixi environment management
CI/CD Detection: Supports GitHub Actions, GitLab CI, Travis CI, and more
Pre-commit Hooks: Integration with pre-commit configuration
Development Workflow: Optimization suggestions for development setup
Tool Detection: Identifies pytest, mypy, ruff, and other development tools

🔒 Security & Quality Assurance

Security Scanning: Bandit code security analysis and Safety dependency vulnerability checks
Code Quality: Comprehensive linting with Ruff and type checking with MyPy
Test Coverage: High test coverage requirements with pytest and coverage reporting
Pre-commit Hooks: Automated quality checks on every commit

📋 Requirements

Python 3.9 or later
PyYAML for YAML processing
PyYAML for YAML processing
tomli for TOML parsing (Python < 3.11)

🚀 Installation

From PyPI (when published)

pip install pyrattler-recipe-autogen

From Source

git clone https://github.com/millsks/pyrattler-recipe-autogen.git
cd pyrattler-recipe-autogen
pip install -e .

Using Pixi (Recommended)

git clone https://github.com/millsks/pyrattler-recipe-autogen.git
cd pyrattler-recipe-autogen
pixi install
pixi shell

📖 Usage

Command Line Interface

Basic usage:

# Generate recipe.yaml from pyproject.toml in current directory
pyrattler-recipe-autogen

# Specify input and output files
pyrattler-recipe-autogen -i path/to/pyproject.toml -o path/to/recipe.yaml

# Overwrite existing recipe file
pyrattler-recipe-autogen --overwrite

🎮 Try the Demo

Want to see what pyrattler-recipe-autogen can do? Try the interactive demo:

# Run all demos (shows simple, scientific, webapp, and current project)
python -m pyrattler_recipe_autogen.demo

# Try specific package types
python -m pyrattler_recipe_autogen.demo --type simple
python -m pyrattler_recipe_autogen.demo --type scientific
python -m pyrattler_recipe_autogen.demo --type webapp

# Analyze the current project
python -m pyrattler_recipe_autogen.demo --type current

# See complete recipes (instead of previews)
python -m pyrattler_recipe_autogen.demo --full

With pixi (if you have the source):

pixi run demo          # All demos
pixi run demo-simple   # Simple package demo
pixi run demo-current  # Current project demo
pixi run demo-full     # Full output

The demo shows how the tool works with:

🔧 Simple packages: Basic Python projects with standard dependencies
🔬 Scientific packages: NumPy/SciPy/matplotlib-based projects
🌐 Web applications: FastAPI/Django-style projects
📦 Real projects: Analysis of actual pyproject.toml files

Programmatic API

Basic recipe generation:

from pathlib import Path
from pyrattler_recipe_autogen import generate_recipe

# Generate recipe from pyproject.toml
generate_recipe(
    pyproject_path=Path("pyproject.toml"),
    output_path=Path("recipe.yaml"),
    overwrite=True
)

Advanced configuration with custom output:

from pyrattler_recipe_autogen.core import (
    generate_recipe_with_config,
    OutputConfig,
    IntegrationConfig,
    _detect_integration_enhancements
)

# Configure output options
output_config = OutputConfig(
    output_format="json",           # "yaml" or "json"
    yaml_style="block",            # "default", "block", or "flow"
    include_comments=True,         # Include helpful comments
    sort_keys=True,               # Sort dictionary keys
    indent=4,                     # Indentation level
    exclude_sections=["test"],    # Skip test section
    validate_output=True          # Validate generated recipe
)

# Generate with custom configuration
generate_recipe_with_config(
    pyproject_path=Path("pyproject.toml"),
    output_path=Path("recipe.yaml"),
    config=output_config
)

Integration analysis:

from pyrattler_recipe_autogen.core import (
    _detect_integration_enhancements,
    _load_integration_config,
    load_pyproject_toml
)

# Analyze project integrations
toml_data = load_pyproject_toml(Path("pyproject.toml"))
integration_config = _load_integration_config(toml_data)
integration_info = _detect_integration_enhancements(
    Path("."), toml_data, integration_config
)

# Access detected information
print(f"Pixi detected: {integration_info.pixi_detected}")
print(f"CI/CD systems: {integration_info.ci_cd_systems}")
print(f"Development tools: {integration_info.dev_tools}")
print(f"Suggestions: {integration_info.workflow_suggestions}")

⚙️ Configuration

Basic Configuration in pyproject.toml

You can customize recipe generation by adding configuration to your pyproject.toml:

[tool.conda.recipe]
# Basic recipe metadata
name = "my-package"
version = "1.0.0"

# Source configuration
[tool.conda.recipe.source]
url = "https://github.com/user/repo/archive/v{{ version }}.tar.gz"
sha256 = "abc123..."

# Build configuration
[tool.conda.recipe.build]
script = "python -m pip install . -vv --no-build-isolation"
number = 0
skip = ["py<39"]

# Requirements
[tool.conda.recipe.requirements]
build = ["pip", "setuptools", "wheel"]
host = ["python", "pip"]
run = ["python", "numpy", "pandas"]

# Test configuration
[tool.conda.recipe.test]
imports = ["my_package"]
commands = ["my-package --help"]
requires = ["pytest"]

# About section
[tool.conda.recipe.about]
summary = "A fantastic Python package"
license = "MIT"
license_file = "LICENSE"
homepage = "https://github.com/user/repo"

Advanced Output Configuration

[tool.conda.recipe.output]
format = "yaml"              # "yaml" or "json"
yaml_style = "block"         # "default", "block", or "flow"
include_comments = true      # Add helpful comments
sort_keys = false           # Sort dictionary keys
indent = 2                  # Indentation level
validate_output = true      # Validate generated recipe
include_sections = []       # Only include these sections (empty = all)
exclude_sections = ["test"] # Exclude these sections
json_indent = 2            # JSON indentation level

# Custom templates for specific sections
[tool.conda.recipe.output.custom_templates]
package = "Custom package template"

Integration Enhancement Configuration

[tool.conda.recipe.integration]
pixi_integration = true          # Detect pixi configuration
ci_cd_detection = true          # Detect CI/CD systems
precommit_integration = true    # Detect pre-commit setup
dev_workflow_optimization = true # Analyze development workflow
suggest_improvements = true     # Generate improvement suggestions

Context Variables and Platform Support

[tool.conda.recipe.context]
# Define custom context variables
python_min = "3.9"
package_name = "my-package"

# Platform-specific configurations
[tool.conda.recipe.platforms]
linux = ["linux-64", "linux-aarch64"]
macos = ["osx-64", "osx-arm64"]
windows = ["win-64"]

# Architecture-specific dependencies
[tool.conda.recipe.requirements.run.linux]
additional = ["linux-specific-package"]

[tool.conda.recipe.requirements.run.macos]
additional = ["macos-specific-package"]

🏗️ Generated Recipe Structure

The tool generates comprehensive recipe.yaml files with the following sections:

context:
  name: my-package
  version: 1.0.0
  python_min: "3.9"
  # ... additional context variables

package:
  name: ${{ name }}
  version: ${{ version }}

source:
  url: https://pypi.org/packages/source/m/my-package/my_package-1.0.0.tar.gz
  # or git repository, local path, etc.

build:
  script: python -m pip install . -vv --no-build-isolation
  number: 0
  # ... build configuration

requirements:
  build: []
  host: [python, pip]
  run: [python, numpy, pandas]

test:
  python:
    imports: [my_package]
  commands: [my-package --help]

about:
  summary: A fantastic Python package
  license: MIT
  license_file: LICENSE
  homepage: https://github.com/user/repo

🔧 Advanced Features

Dynamic Version Resolution

Supports multiple dynamic versioning systems:

setuptools_scm: Git tag-based versioning
hatch-vcs: Modern Git versioning for hatch
poetry: Poetry dynamic versioning
setuptools: Simple dynamic versions

Smart Dependency Mapping

Converts Python package names to conda equivalents
Handles version constraints and markers
Separates runtime, build, and test dependencies
Manages optional and extra dependencies

Platform-Specific Builds

Detects platform-specific dependencies
Configures appropriate selectors
Handles architecture requirements
Manages Python version matrices

Integration Analysis

Pixi Detection: Analyzes pixi.toml and environments
CI/CD Systems: Detects GitHub Actions, GitLab CI, Travis CI, etc.
Development Tools: Finds pytest, mypy, ruff, black, and more
Pre-commit: Analyzes .pre-commit-config.yaml
Workflow Suggestions: Recommends improvements and best practices
Workflow Suggestions: Recommends improvements and best practices

📚 Examples

Simple Python Package

For a basic Python package with the following pyproject.toml:

[project]
name = "my-simple-package"
version = "1.0.0"
description = "A simple Python package"
dependencies = ["requests", "click"]
requires-python = ">=3.9"

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

Generated recipe:

context:
  name: my-simple-package
  version: 1.0.0
  python_min: "3.9"

package:
  name: ${{ name }}
  version: ${{ version }}

source:
  url: https://pypi.org/packages/source/m/my-simple-package/my_simple_package-1.0.0.tar.gz

build:
  script: python -m pip install . -vv --no-build-isolation
  number: 0

requirements:
  host: [python, pip]
  run: [python, requests, click]

test:
  python:
    imports: [my_simple_package]

Complex Package with Pixi

For a project using pixi with the following structure:

# pyproject.toml
[project]
name = "advanced-package"
dynamic = ["version"]
dependencies = ["numpy>=1.20", "pandas>=1.3"]
optional-dependencies.dev = ["pytest", "mypy", "ruff"]

[tool.hatch.version]
source = "vcs"

# pixi.toml
[project]
name = "advanced-package"
channels = ["conda-forge", "bioconda"]
platforms = ["linux-64", "osx-64", "osx-arm64"]

[dependencies]
python = ">=3.9"
numpy = ">=1.20"
pandas = ">=1.3"

[feature.dev.dependencies]
pytest = "*"
mypy = "*"
ruff = "*"

Generated recipe includes pixi-aware dependency management and development tool detection.

🚨 Troubleshooting

Common Issues

Issue: ModuleNotFoundError: No module named 'tomllib'
Solution: Install tomli for Python < 3.11: pip install tomli

Issue: Recipe generation fails with "No version found"
Solution: Ensure version is specified in [project] or configure dynamic versioning properly

Issue: Dependencies not mapping correctly
Solution: Check for conda-forge equivalents or specify custom mappings in [tool.conda.recipe.requirements]

Issue: Build script not detected
Solution: Specify custom build script in [tool.conda.recipe.build]

Debug Mode

Enable verbose logging to debug issues:

import logging
logging.basicConfig(level=logging.DEBUG)

from pyrattler_recipe_autogen import generate_recipe
# Your code here

Validation Errors

The tool includes built-in validation. Common validation warnings:

Missing recommended sections: Add missing package, source, or requirements sections
Package version missing: Ensure version is specified or dynamic versioning is configured
Undefined context variables: Check template variables in your configuration
Unused context variables: Remove unused variables from context section

🧪 Development

Setup Development Environment

Using pixi (recommended):

git clone https://github.com/millsks/pyrattler-recipe-autogen.git
cd pyrattler-recipe-autogen
pixi install
pixi shell

Using pip:

git clone https://github.com/millsks/pyrattler-recipe-autogen.git
cd pyrattler-recipe-autogen
pip install -e ".[dev]"

Development Commands

# Run tests
pixi run test

# Run tests with coverage
pixi run test-cov

# Format code
pixi run format

# Run linting
pixi run lint

# Run type checking
pixi run type-check

# Run security checks
pixi run security-check

# Run all checks (lint + type-check)
pixi run check

# Run all checks including security (lint + type-check + security-check)
pixi run check-all

# Build package
pixi run build

# Clean build artifacts
pixi run clean

# Check current version
pixi run version

Running the Full CI Pipeline

pixi run ci  # Runs format + check + test-cov (includes security checks)

Project Structure

pyrattler-recipe-autogen/
├── src/
│   └── pyrattler_recipe_autogen/
│       ├── __init__.py
│       ├── _version.py          # Auto-generated
│       ├── cli.py              # Command-line interface
│       └── core.py             # Core recipe generation logic
├── tests/
│   ├── test_cli.py            # CLI tests
│   ├── test_core.py           # Core functionality tests
│   └── test_init.py           # Package initialization tests
├── pyproject.toml             # Project configuration
├── pixi.toml                 # Pixi environment configuration
├── README.md                 # This file
└── LICENSE                   # MIT license

🔄 Versioning

This project uses hatch-vcs for automatic version management:

Tagged releases: Use the tag version (e.g., v0.1.0 → 0.1.0)
Development builds: Include commit info (e.g., 0.1.0.dev5+g1234567)
Dirty working directories: Append .dirty to version

Check current version: pixi run version

🤝 Contributing

We welcome contributions! Please see our Contributing Guidelines for details.

📚 Documentation

Release Process: Complete guide for creating and publishing releases
Contributing Guidelines: Development setup and contribution workflow
Security Policy: How to report security vulnerabilities
Changelog: Release history and version changes

Quick Start for Contributors

Fork and Clone:

git clone https://github.com/yourusername/pyrattler-recipe-autogen.git
cd pyrattler-recipe-autogen

Setup Development Environment:

pixi install
pixi run dev-setup  # Installs pre-commit hooks

Create a Branch:

git checkout -b feature/your-feature-name

Make Changes and ensure tests pass:

pixi run format       # Format code
pixi run check-all    # Run linting, type checking, and security checks
pixi run test-cov     # Run tests with coverage

Commit with Conventional Commits:

git commit -m "feat: add new feature"
# or
git commit -m "fix: resolve issue with dependency mapping"

Push and Create PR:

git push origin feature/your-feature-name

Code Quality Standards

Test Coverage: Maintain >95% test coverage
Type Hints: All functions must have proper type annotations
Documentation: Add docstrings for all public functions and classes
Conventional Commits: Use conventional commit format for all commits
Pre-commit Hooks: All hooks must pass before committing

Development Workflow

Setup: Use pixi install and pixi run dev-setup
Development: Use pixi run commands for all tasks
Testing: Write tests for new features in tests/
Documentation: Update README and docstrings
Quality: Ensure all pre-commit hooks pass
Pull Request: Create PR with clear description

📊 Project Statistics

Test Coverage: 97% (core.py), 95% (overall)
Code Quality: Pre-commit hooks with ruff, mypy, bandit
Documentation: Comprehensive docstrings and examples
CI/CD: GitHub Actions with automated testing and quality checks
Dependencies: Minimal runtime dependencies for reliability

🗺️ Roadmap

Completed Features ✅

✅ Enhanced Build Configuration: Multi-backend support and intelligent detection
✅ Advanced Requirements Management: Smart dependency mapping and pixi integration
✅ Test Section Intelligence: Auto-detection and configuration
✅ Platform/Variant Support: Cross-platform builds and architecture handling
✅ Advanced Context Variables: Package intelligence and environment detection
✅ Output Customization: Multiple formats and validation framework
✅ Integration Enhancements: CI/CD, pixi, and development tool detection
✅ Documentation Enhancement: Comprehensive docs and examples

Future Enhancements 🚀

🔄 GUI Interface: Desktop application for recipe generation
🔄 Recipe Templates: Pre-built templates for common package types
🔄 Batch Processing: Generate recipes for multiple packages
🔄 Cloud Integration: Direct integration with conda-forge/feedstocks
🔄 Package Analysis: Deep dependency analysis and optimization suggestions

🐛 Known Issues

Windows path handling in some edge cases
Large dependency trees may cause slow generation
Some proprietary build systems not fully supported

See Issues for current bug reports and feature requests.

📖 API Documentation

Full API documentation is available in the source code docstrings. Key classes and functions:

Core Functions

generate_recipe(): Main recipe generation function
generate_recipe_with_config(): Advanced generation with custom configuration
load_pyproject_toml(): Load and parse pyproject.toml files

Configuration Classes

OutputConfig: Configure output format and validation
IntegrationConfig: Configure integration detection
IntegrationInfo: Results of integration analysis

Detection Functions

_detect_pixi_integration(): Analyze pixi configuration
_detect_ci_cd_systems(): Find CI/CD systems
_detect_dev_tools(): Identify development tools

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

Rattler - Fast conda package management
Pixi - Modern Python package management
conda-forge - Community-driven conda package repository
PyPA - Python Packaging Authority standards

📞 Support

Documentation: Check this README and source code docstrings
Issues: Report bugs on GitHub Issues
Discussions: Ask questions in GitHub Discussions
Contributing: See CONTRIBUTING.md for development guidelines

Happy Recipe Generating! 🚀

Development

Prerequisites

Pixi - Modern package management for Python

Environment Setup

# Clone the repository
git clone https://github.com/millsks/pyrattler-recipe-autogen.git
cd pyrattler-recipe-autogen

# Install dependencies (includes editable install of the project)
pixi install

# Set up development environment (install pre-commit hooks)
pixi run dev-setup

Pre-commit Hooks

This project uses pre-commit to ensure code quality and consistency. The hooks run automatically on every commit and include:

Code Formatting: Ruff formatting and linting
Type Checking: MyPy static type analysis
Security Scanning: Bandit security linter
File Validation: YAML, TOML, JSON syntax checking
Git Best Practices: Large file detection, merge conflict detection
Commit Messages: Conventional commit format validation
Documentation: Markdown linting and formatting
GitHub Actions: Workflow file validation

# Install hooks (done automatically by dev-setup)
pixi run pre-commit-install

# Run all hooks manually
pixi run pre-commit-run

# Run specific hook
pixi run pre-commit run ruff --all-files

Development Tasks

Pixi provides convenient commands for development tasks:

# The project is automatically installed in editable mode when you run pixi install

# Run tests
pixi run test

# Run tests with coverage
pixi run test-cov

# Format code with ruff
pixi run format

# Run linting
pixi run lint

# Run type checking
pixi run type-check

# Run security checks
pixi run security-check

# Run all checks (lint + type-check)
pixi run check

# Run all checks including security (lint + type-check + security-check)
pixi run check-all

# Build package
pixi run build

# Clean build artifacts
pixi run clean

# Check current version
pixi run version

# Run the full CI pipeline (format + check + test-cov)
pixi run ci

# Generate changelog
pixi run changelog

# Preview unreleased changes
pixi run changelog-unreleased

# Preview latest version changes
pixi run changelog-latest

# Preview what the next release would look like
pixi run release-preview

Changelog Generation

This project uses git-cliff to automatically generate changelogs based on conventional commits:

# Generate complete changelog
pixi run changelog

# See unreleased changes
pixi run changelog-unreleased

# Preview the next release
pixi run release-preview

The changelog follows Keep a Changelog format and uses Conventional Commits for automated categorization.


### Using Different Environments

Pixi supports multiple environments for different purposes:

```bash
# Use the default development environment
pixi shell

# Use the test-only environment
pixi shell -e test

# Use the lint-only environment
pixi shell -e lint

# Run tasks in specific environments
pixi run -e test test
pixi run -e lint lint

Package Structure

The project follows a src layout:

pyrattler-recipe-autogen/
├── src/
│   └── pyrattler_recipe_autogen/
│       ├── __init__.py          # Package initialization and exports
│       ├── core.py              # Core business logic
│       └── cli.py               # Command line interface
├── tests/                       # Test suite
├── pyproject.toml              # Package configuration
├── generate_conda_recipe.py    # Legacy wrapper script
└── dev.py                      # Development utility script

Configuration

The tool reads configuration from your pyproject.toml file and supports additional conda-specific configuration under [tool.conda.recipe.*] sections:

[tool.conda.recipe.extra_context]
# Additional context variables

[tool.conda.recipe.about]
# Override about section fields

[tool.conda.recipe.build]
# Override build configuration

[tool.conda.recipe.source]
# Override source configuration

[tool.conda.recipe.test]
# Test configuration

[tool.conda.recipe.extra]
# Extra recipe sections

Releases and Contributing

Versioning and Releases

This project uses hatch-vcs for automatic version management based on git tags:

Development versions: Generated automatically as 0.1.devN (where N is the number of commits since the last tag)
Release versions: Created by pushing git tags in the format v1.0.0

Creating a Release

For maintainers: Use the streamlined release workflow in GitHub Actions.

Go to Actions → "Release and Publish" workflow
Click "Run workflow" and fill in the form:
- Version (e.g., 1.0.0)
- Pre-release checkbox (for alpha/beta)
- Publishing options (PyPI, Test PyPI)
Wait for completion - everything is automated!

For complete details: See RELEASE_PROCESS.md

Test Publishing

The release workflow includes optional Test PyPI publishing for verification before production release.

Contributing

Fork the repository
Create a feature branch
Make your changes
Run tests: pixi run test
Run checks: pixi run check
Submit a pull request

License