🥔 Farmore

"Mirror every repo you own — in one command."

Farmore is a comprehensive Python CLI tool for backing up GitHub repositories and their associated data. Clone repositories, export issues, download releases, backup wikis, and more — all with a single command.

Version: 0.9.0 License: MIT Python: 3.10+

✨ Features

🔄 Repository Management

Bulk backups - Clone all repos for a user or organization in one command
Single repo backups - Backup individual repositories with the repo command
Repository search - Search GitHub and clone matching repositories by keyword
Smart updates - Automatically pulls updates for existing repositories
Parallel processing - Fast backups with configurable worker threads
SSH/HTTPS support - Tries SSH first, falls back to HTTPS with token
Bare/Mirror clones - Use --bare flag for true 1:1 repository backups
Git LFS support - Use --lfs flag for repositories with large files
Name regex filtering - Filter repositories by name pattern with --name-regex
Incremental backups - Track backup state with --incremental flag

📊 Data Export

Issues export - Export all issues to JSON/YAML with optional comments
Pull requests export - Export PRs with metadata and comments
Workflows backup - Backup GitHub Actions workflows and run history
Releases download - Download releases with metadata and binary assets
Wiki backup - Clone repository wikis as git repositories
Labels export - Backup repository labels with colors
Milestones export - Backup milestone configuration and progress
Gists backup - Clone all your GitHub gists
Attachments download - Download images/files from issues and PRs

🔐 Access & Security

Private repository support - Full access with GitHub Personal Access Tokens
Organization repos - Backup all organization repositories
Starred & watched repos - Mirror repositories you've starred or are watching
Secrets export - Export repository secret names (values are never exposed)
Followers/Following - Export user's social connections
Webhooks export - Backup webhook configurations

🎯 Advanced Features

Flexible filtering - By visibility (public/private/all), forks, archived status
Rate limit handling - Automatic retries with exponential backoff
Organized structure - Clean directory organization separating code from data
Cross-platform - Works on Linux, macOS, and Windows
Beautiful CLI - Powered by Typer and Rich with progress bars
GitHub Enterprise - Support via --github-host flag
Discussions backup - Export repository discussions (GraphQL API)
Projects backup - Export Projects v2 data (GraphQL API)

📦 Installation

Requirements

Python Version: 3.10 or higher is required.

Additional Requirements:

Git installed and available in PATH
GitHub Personal Access Token (for private repos and higher rate limits)

🎯 From PyPI (Recommended)

The easiest way to install Farmore is from the Python Package Index (PyPI):

pip install farmore

This is the recommended method for end users. Once installed, the farmore command will be available globally.

Verify installation:

farmore --version
farmore --help

🧪 From TestPyPI (Pre-release Testing)

To test pre-release versions before they're published to PyPI:

pip install --index-url https://test.pypi.org/simple/ farmore

When to use this:

Testing new features before official release
Helping with beta testing
Verifying bug fixes in development versions

Note: TestPyPI packages may not have all dependencies available. You might need to install dependencies from regular PyPI separately.

📥 From GitHub Releases (Specific Versions)

Download a specific version from the GitHub Releases page:

Download the .whl file from the release you want (e.g., farmore-0.3.0-py3-none-any.whl)
Install the downloaded file:

pip install farmore-0.3.0-py3-none-any.whl

When to use this:

You need a specific version
You want to verify package integrity
You're installing in an offline environment (download first, install later)

Alternative - Source distribution:

# Download the .tar.gz file instead
pip install farmore-0.3.0.tar.gz

🔧 From Source (Development)

For developers who want to contribute or modify the code:

# Clone the repository
git clone https://github.com/miztizm/farmore.git
cd farmore

# Install in editable mode (changes to code take effect immediately)
pip install -e .

# Or install with development dependencies (recommended for contributors)
pip install -e ".[dev]"

Development dependencies include:

pytest - Testing framework
pytest-cov - Code coverage
ruff - Linting and formatting
mypy - Type checking
Additional testing utilities

Verify installation:

farmore --version
farmore --help

🔄 Upgrading

To upgrade to the latest version:

# From PyPI
pip install --upgrade farmore

# From TestPyPI
pip install --upgrade --index-url https://test.pypi.org/simple/ farmore

🗑️ Uninstalling

To remove Farmore:

pip uninstall farmore

🚀 Quick Start

# Backup all repos for a user
farmore user miztizm

# Backup a single repository
farmore repo microsoft/vscode

# Search and clone repositories by keyword
farmore search "nuxt laravel" --limit 10

# Backup with issues and pull requests
farmore repo miztizm/hello-world --include-issues --include-pulls

# Backup everything for a repository
farmore repo python/cpython --all

# Backup all repos for an organization
farmore org github --include-issues --include-pulls

# With authentication (recommended)
export GITHUB_TOKEN=ghp_your_token_here
farmore user miztizm

"They say privacy is dead. Prove them wrong. Use a token." — schema.cx

🔑 Authentication

Farmore uses GitHub Personal Access Tokens (PAT) for authentication. Tokens provide:

Access to private repositories
Higher rate limits (5,000 vs 60 requests/hour)
Organization repository access

Creating a Token

⭐ Recommended: Use Classic Personal Access Token

Create a Classic PAT: https://github.com/settings/tokens
- Click "Tokens (classic)" → "Generate new token (classic)"
- Give it a name: farmore-backup
- Select scope: ✅ repo (required for private repositories)
- Optional: ✅ delete_repo (only if using farmore delete command)
- Click "Generate token" and copy it immediately
Set environment variable:

# Linux/macOS
export GITHUB_TOKEN=ghp_your_token_here

# Windows PowerShell
$env:GITHUB_TOKEN="ghp_your_token_here"

# Or create a .env file in the project directory
echo "GITHUB_TOKEN=ghp_your_token_here" > .env

Why Classic PAT?

✅ Simple setup (just check repo scope)
✅ Works with all repository types (personal + organization)
✅ Proven reliability
✅ Broader API compatibility

Rate Limits

Mode	Requests/Hour	Use Case
❌ Unauthenticated	60	Small public repos only
✅ Authenticated	5,000	Production, private repos

Security Best Practices

✅ Use environment variables or .env files
✅ Set token expiration (90 days recommended)
✅ Use minimal required permissions
❌ Never commit tokens to version control
❌ Avoid --token flag (exposes in shell history)

📚 Commands

Farmore provides 14 commands organized into 4 categories:

🔄 Repository Backup

`farmore user <username>`

Backup all private and public repositories for a GitHub user.

# Backup all repos
farmore user miztizm

# Backup with data exports
farmore user miztizm --include-issues --include-pulls

# Filter by visibility
farmore user miztizm --visibility public

# Dry run
farmore user miztizm --dry-run

Key Options: --visibility, --include-forks, --include-archived, --include-issues, --include-pulls, --include-workflows, --include-releases, --include-wikis, --max-workers, --dry-run

`farmore org <orgname>`

Backup all repositories for an organization. Same options as user.

`farmore repo <owner>/<repo>`

Backup a single repository with optional data exports.

# Just clone
farmore repo microsoft/vscode

# Clone + data
farmore repo microsoft/vscode --include-issues --include-pulls

# Everything
farmore repo python/cpython --all

Key Options: --include-issues, --include-pulls, --include-workflows, --include-releases, --include-wikis, --all

`farmore search <query>` 🆕

Search GitHub repositories by keyword and clone matching results.

# Basic search - clone top 10 results
farmore search "smsbomber"

# Search with filters
farmore search "machine learning" --language python --min-stars 1000 --limit 20

# Search and auto-confirm (skip prompt)
farmore search "react components" --limit 5 --yes

# Custom output directory
farmore search "awesome-python" --output-dir ./my-collections --limit 15

# Sort by stars (descending)
farmore search "cli tools" --language go --sort stars --order desc --limit 25

# Flat structure (no owner subdirectories)
farmore search "react hooks" --flat-structure --limit 10

Key Options:

--limit (1-100): Maximum repositories to clone (default: 10)
--language: Filter by programming language (e.g., "python", "javascript", "go")
--min-stars: Minimum number of stars required
--sort: Sort order - "best-match" (default), "stars", "forks", or "updated"
--order: Sort direction - "desc" (default) or "asc"
--yes / -y: Skip confirmation prompt
--output-dir: Custom output directory (default: ./search-results/<query>/)
--flat-structure: Clone repos directly without owner subdirectories
--workers: Number of parallel workers for cloning (default: 4)

Output Structure (Default):

search-results/
└── <sanitized-query>/
    ├── <owner1>/
    │   └── <repo1>/
    ├── <owner2>/
    │   └── <repo2>/
    └── <owner3>/
        └── <repo3>/

Output Structure (Flat - with --flat-structure):

search-results/
└── <sanitized-query>/
    ├── <repo1>/
    ├── <repo2>/
    └── <repo3>/

Note: When using --flat-structure, repositories with duplicate names will have their owner appended (e.g., repo-owner) to avoid conflicts.

Rate Limits:

Search API: 30 requests/minute (authenticated users)
Unauthenticated: 10 requests/minute

Note: The search command uses GitHub's search API with support for advanced search qualifiers. You can also use GitHub's native search syntax directly in the query (e.g., "language:python stars:>1000").

📊 Data Export

`farmore issues <owner>/<repo>`

Export issues to JSON/YAML.

farmore issues microsoft/vscode
farmore issues miztizm/hello-world --state open --include-comments

Options: --format [json|yaml], --state [all|open|closed], --include-comments

`farmore pulls <owner>/<repo>`

Export pull requests to JSON/YAML.

farmore pulls microsoft/vscode
farmore pulls miztizm/hello-world --state open --include-comments

Options: --format [json|yaml], --state [all|open|closed], --include-comments

`farmore workflows <owner>/<repo>`

Backup GitHub Actions workflows.

farmore workflows microsoft/vscode
farmore workflows actions/checkout --include-runs

Options: --include-runs

`farmore releases <owner>/<repo>`

Download releases and assets.

farmore releases microsoft/vscode
farmore releases nodejs/node --download-assets

Options: --download-assets

`farmore wiki <owner>/<repo>`

Clone repository wiki.

farmore wiki python/cpython

🔍 Profile & Discovery

`farmore profile [username]`

Export user profile.

farmore profile              # Your profile
farmore profile miztizm      # Another user's profile

Options: --format [json|yaml]

`farmore starred [username]`

Mirror starred repositories.

farmore starred              # Your starred repos
farmore starred miztizm      # Another user's starred repos

`farmore watched [username]`

Mirror watched repositories.

farmore watched

🔐 Security & Management

`farmore secrets <owner>/<repo>`

Export repository secret names (values never exposed).

farmore secrets miztizm/farmore

Options: --format [json|yaml]

`farmore delete <owner>/<repo>`

Delete a repository (requires confirmation).

farmore delete miztizm/old-project
farmore delete miztizm/test-repo --force  # Skip confirmation

⚠️ Warning: Requires delete_repo scope in your GitHub token.

📂 Directory Structure

Farmore organizes backups with a clean structure that separates code from metadata:

backups/
└── <username>/
    ├── profile.json                          # User profile
    ├── repos/                                # Git repositories
    │   ├── private/<owner>/<repo>/
    │   ├── public/<owner>/<repo>/
    │   ├── starred/<owner>/<repo>/
    │   ├── watched/<owner>/<repo>/
    │   ├── organizations/<owner>/<repo>/
    │   └── forks/<owner>/<repo>/
    └── data/                                 # Metadata
        ├── issues/<owner>_<repo>_issues.json
        ├── pulls/<owner>_<repo>_pulls.json
        ├── workflows/<owner>_<repo>/
        ├── releases/<owner>_<repo>/
        ├── wikis/<owner>_<repo>.wiki/
        └── secrets/<owner>_<repo>_secrets.json

Example:

# After: farmore user miztizm --include-issues --include-pulls
backups/miztizm/
├── profile.json
├── repos/
│   ├── public/miztizm/farmore/
│   └── private/miztizm/secret-project/
└── data/
    ├── issues/
    │   ├── miztizm_farmore_issues.json
    │   └── miztizm_secret-project_issues.json
    └── pulls/
        ├── miztizm_farmore_pulls.json
        └── miztizm_secret-project_pulls.json

🔧 How It Works

Discovery - Uses GitHub API to find repositories
Filtering - Applies visibility, fork, and archived filters
Parallel Processing - Processes multiple repos simultaneously
Smart Cloning - Tries SSH first, falls back to HTTPS
Progress Reporting - Real-time progress with rich output
Summary - Final statistics and error reporting

Key Features:

Incremental backups - Updates existing repos, clones new ones
Error resilience - Continues even if individual repos fail
Rate limit aware - Handles GitHub API limits automatically
Organized output - Auto-categorizes by type

🛠️ Development

Setup

git clone https://github.com/miztizm/farmore.git
cd farmore
python -m venv venv
source venv/bin/activate  # Windows: venv\Scripts\activate
pip install -e ".[dev]"

Testing

pytest                                    # Run all tests
pytest --cov=farmore --cov-report=html   # With coverage
pytest tests/test_github_api.py          # Specific test

Code Quality

black farmore/        # Format
ruff check farmore/   # Lint
mypy farmore/         # Type check

📄 License

MIT License - see LICENSE file for details.

🤝 Contributing

Contributions welcome! Please:

Check existing issues and PRs
Follow the existing code style
Add tests for new features
Update documentation

💬 Support

🐛 For bugs, Issues
📩 For questions, Email

🌟 Acknowledgments

Built with Typer, Rich, Requests, and PyYAML.

"Control is an illusion. But backups? Those are real." — schema.cx

Made with 🥔 by miztizm

Name		Name	Last commit message	Last commit date
Latest commit History 25 Commits
.github		.github
farmore		farmore
scripts		scripts
tests		tests
.dockerignore		.dockerignore
.env.example		.env.example
.gitignore		.gitignore
CHANGELOG.md		CHANGELOG.md
Dockerfile		Dockerfile
LICENSE		LICENSE
README.md		README.md
debug.log		debug.log
pyproject.toml		pyproject.toml

License

miztizm/farmore

Folders and files

Latest commit

History

Repository files navigation

🥔 Farmore

✨ Features

🔄 Repository Management

📊 Data Export

🔐 Access & Security

🎯 Advanced Features

📦 Installation

Requirements

🎯 From PyPI (Recommended)

🧪 From TestPyPI (Pre-release Testing)

📥 From GitHub Releases (Specific Versions)

🔧 From Source (Development)

🔄 Upgrading

🗑️ Uninstalling

🚀 Quick Start

🔑 Authentication

Creating a Token

Why Classic PAT?

Rate Limits

Security Best Practices

📚 Commands

🔄 Repository Backup

farmore user <username>

farmore org <orgname>

farmore repo <owner>/<repo>

farmore search <query> 🆕

📊 Data Export

farmore issues <owner>/<repo>

farmore pulls <owner>/<repo>

farmore workflows <owner>/<repo>

farmore releases <owner>/<repo>

farmore wiki <owner>/<repo>

🔍 Profile & Discovery

farmore profile [username]

farmore starred [username]

farmore watched [username]

🔐 Security & Management

farmore secrets <owner>/<repo>

farmore delete <owner>/<repo>

📂 Directory Structure

🔧 How It Works

🛠️ Development

Setup

Testing

Code Quality

📄 License

🤝 Contributing

💬 Support

🌟 Acknowledgments

About

Topics

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases 9

Languages

`farmore user <username>`

`farmore org <orgname>`

`farmore repo <owner>/<repo>`

`farmore search <query>` 🆕

`farmore issues <owner>/<repo>`

`farmore pulls <owner>/<repo>`

`farmore workflows <owner>/<repo>`

`farmore releases <owner>/<repo>`

`farmore wiki <owner>/<repo>`

`farmore profile [username]`

`farmore starred [username]`

`farmore watched [username]`

`farmore secrets <owner>/<repo>`

`farmore delete <owner>/<repo>`