Refactor multi ruleset design

.github/workflows/ci.yml

@@ -62,10 +62,11 @@ jobs:
       - name: Run basic functionality test
         run: |
           uv run rulebook-ai --help
-          uv run rulebook-ai list-rules
+          uv run rulebook-ai packs list
           mkdir -p test_project
-          uv run rulebook-ai install --rule-set light-spec --project-dir test_project
-          yes | uv run rulebook-ai clean-all --project-dir test_project
+          uv run rulebook-ai packs add light-spec --project-dir test_project
+          uv run rulebook-ai project sync --pack light-spec --project-dir test_project
+          yes | uv run rulebook-ai project clean --project-dir test_project
 
       - name: Run tests with pytest
         run: uv run pytest --cov=rulebook_ai

.gitignore

@@ -103,7 +103,9 @@ GEMINI.md
 .github/copilot-instructions.md
 .idx/
 project_rules/
+.gemini/GEMINI.md
 
+.rulebook-ai/
 
 # vscode extension
 ### VS Code extension package

memory/docs/features/community_packs/ADR-001_Pack_Naming_and_Index_Cache.md

@@ -0,0 +1,36 @@
+# ADR-001: Community Pack Naming and Index Cache Strategy
+
+## Status
+
+Accepted
+
+## Context
+
+Community packs allow third-party `Rule Packs` to be installed via the `rulebook-ai` CLI. Because packs are copied into `.rulebook-ai/packs/<name>`, two challenges arise:
+
+1. **Name collisions**: Different repositories may publish packs with the same `manifest.yaml` `name`.
+2. **Shared discovery data**: The CLI needs an index of available community packs, and this index should be available across all user projects.
+
+## Decision
+
+* **Global name enforcement (alias deferred)**
+  * The `manifest.yaml` `name` is treated as a globally unique identifier.
+  * If an unlisted pack's `name` conflicts with an existing local pack, the CLI aborts and no alias is offered in the MVP.
+  * Installed packs use this `name` as the directory under `.rulebook-ai/packs/`. The source slug, decomposed as `username/repo[/path]`, is recorded in metadata for traceability.
+
+* **Shared Local Index Cache**
+  * The community index `packs.json` is cached inside the installed Python package under `rulebook_ai/community/index_cache/packs.json`.
+  * The `rulebook-ai packs update` command refreshes this cache. Because it lives in the package directory, all repositories on the same machine share a single copy.
+
+## Consequences
+
+* Users can keep working with short, memorable pack names while avoiding accidental overwrites.
+* Conflicting names result in an error until upstream naming is resolved, keeping behavior predictable.
+* A single Local Index Cache avoids repeated downloads and keeps different projects in sync.
+* Future features (e.g., alias support, per-repo caches, or namespace support) can build on this foundation without breaking the MVP design.
+
+## Alternatives Considered
+
+* **Installing packs by slug** – This avoids collisions but forces users to remember long identifiers. Rejected for poorer user experience.
+* **Per-repository index cache** – Would duplicate data and require each project to update separately. Rejected to keep MVP simple and predictable.
+

memory/docs/features/community_packs/TDD_plan.md

@@ -0,0 +1,47 @@
+# TDD Plan: Community Pack Ecosystem
+
+This document outlines the test strategy for introducing community-maintained rule packs.
+
+## Test Strategy & Environment
+
+- **Test Type**: The tests for this feature are primarily **`integration tests`**. They verify the integration of the `rulebook-ai` CLI with external systems like the local filesystem and the `Git` version control system.
+- **Test Dependencies**:
+    - To test the `packs add <slug>` functionality, tests will rely on local mock `Git` repositories created under the `tests/fixtures/` directory. This approach ensures tests are fast, reliable, and independent of network connectivity.
+    - To test the `packs update` functionality, tests will use a local mock `packs.json` index file, rather than fetching from a live production URL.
+
+---
+
+## Phase 1: Core Engine (Add by Source)
+- [x] `test_add_pack_by_github_slug_installs_to_folder`: installing `github:username/repo` places files under `.rulebook-ai/packs/<name>`.
+- [x] `test_add_pack_by_local_path_installs_to_folder`: installing `local:path/to/pack` places files under `.rulebook-ai/packs/<name>`.
+- [x] `test_add_pack_conflicting_name_fails`: adding a pack whose `manifest.yaml` `name` already exists aborts.
+- [x] `test_add_pack_invalid_structure_fails`: missing required files triggers validation error.
+- [x] `test_add_pack_user_decline_aborts`: user choosing "no" cancels installation for remote packs.
+- [x] `test_add_pack_by_github_slug_installs_to_folder`: installing `github:username/repo` places files under `.rulebook-ai/packs/<name>`.
+- [x] `test_add_pack_by_local_path_installs_to_folder`: installing `local:path/to/pack` places files under `.rulebook-ai/packs/<name>`.
+- [x] `test_add_pack_conflicting_name_fails`: adding a pack whose `manifest.yaml` `name` already exists aborts.
+- [x] `test_add_pack_invalid_structure_fails`: missing required files triggers validation error.
+- [x] `test_add_pack_user_decline_aborts`: user choosing "no" cancels installation for remote packs.
+- [x] `test_add_pack_by_github_slug_installs_to_folder`: installing `github:username/repo` places files under `.rulebook-ai/packs/<name>`.
+- [x] `test_add_pack_by_local_path_installs_to_folder`: installing `local:path/to/pack` places files under `.rulebook-ai/packs/<name>`.
+- [x] `test_add_pack_conflicting_name_fails`: adding a pack whose `manifest.yaml` `name` already exists aborts.
+- [x] `test_add_pack_invalid_structure_fails`: missing required files triggers validation error.
+- [x] `test_add_pack_user_decline_aborts`: user choosing "no" cancels installation for remote packs.
+
+## Phase 2: Community Index
+- [x] `test_packs_update_refreshes_cache`: `packs update` replaces `rulebook_ai/community/index_cache/packs.json` when fetch succeeds.
+- [x] `test_packs_update_invalid_json_retains_old_cache`: malformed index leaves previous cache untouched.
+- [x] `test_add_pack_by_name_uses_cache`: installing by `name` pulls metadata from the cache and clones the correct repository.
+- [x] `test_add_unknown_pack_name_fails`: unknown `name` emits "pack not found" error.
+- [x] `test_installed_pack_records_slug_metadata`: installed pack stores slug metadata and shows `(community)` in `packs list`.
+- [x] `test_add_pack_name_mismatch_fails`: manifest `name` differing from index entry aborts install.
+
+## Phase 3: Listing and Visibility
+- [x] `test_packs_list_shows_builtin_and_community`: output merges built-in packs with entries from the cache and labels community packs.
+- [x] `test_packs_list_does_not_hit_network`: running `packs list` uses only local data.
+
+## Phase 4: Ecosystem Infrastructure
+Tests in this phase run inside the separate public index repository's CI.
+
+- [ ] `test_index_ci_validation_checks_name_alignment`: CI workflow rejects pull requests when `manifest.yaml` `name` differs from index entry.
+- [ ] `test_index_ci_validation_detects_missing_files`: CI workflow fails when repository lacks required structure.

memory/docs/features/community_packs/implementation_plan.md

@@ -0,0 +1,96 @@
+# Implementation Plan: Community Pack Ecosystem
+
+### Guiding Principle
+
+The most fundamental action is **installing a single, known pack from a direct source**. Everything else—discovery, listing, updating an index—is a layer of abstraction on top of that core capability.
+
+---
+
+### Phase 1: The Core Engine - Installing a Pack by Direct URL
+
+**Objective:** To create a robust, secure mechanism for adding a single pack from a specified Git repository. This phase completely ignores the community index.
+
+1.  **Task: Implement the "Add by Slug" Logic**
+    *   Modify the `packs add` command to recognize a GitHub slug (e.g., `username/repository` or `username/repository/path`).
+    *   Parse the slug to determine the repository URL.
+
+2.  **Task: Develop the Secure Fetch-and-Validate Workflow**
+    *   Create a function that, given a repository URL:
+        a. Clones the repository's default branch into a secure, temporary directory.
+        b. **Validates** the contents of the temporary directory against the `pack_developer_guide.md` (e.g., `manifest.yaml` and `rules/` exist).
+        c. **Verifies Identity:** Parses the `manifest.yaml` and confirms its `name` is valid, not reserved by built-in packs, and not already installed from a different source.
+        d. **Returns** the validated pack's temporary path or throws an error.
+        e. **Reusable Validation:** expose the structure/name checks as a helper so the index CI can import the same logic.
+    *   *Testing Note: This workflow will be verified via `integration tests` using local mock Git repositories created under `tests/fixtures/`.*
+
+3.  **Task: Implement User Confirmation and Installation**
+    *   If validation succeeds, present a clear warning to the user that they are installing un‑audited code from a direct URL.
+    *   Require explicit user confirmation (`y/n`).
+    *   Before moving files, check whether `.rulebook-ai/packs/<name>` already exists; abort if it comes from a different source.
+    *   Upon confirmation, move the pack from the temporary directory into `.rulebook-ai/packs/<name>`.
+    *   Ensure the temporary directory is always cleaned up, especially on error.
+
+4.  **Task: Persist Source Metadata**
+    *   After installation, record the pack's slug and commit hash in `pack.json` and in `.rulebook-ai/selection.json` so future commands can trace provenance.
+
+**Outcome of Phase 1:** A user can reliably and securely install any compatible pack if they know its GitHub URL slug (e.g., `rulebook-ai packs add my-org/my-cool-pack`).
+
+---
+
+### Phase 2: The Discovery Layer - The Community Index
+
+**Objective:** To introduce the concept of a community index that users can refresh, enabling installation by name instead of by URL.
+
+1.  **Task: Implement the Local Cache**
+    *   Store the local index cache inside the Python package at `rulebook_ai/community/index_cache/packs.json` so all repositories share one updated index.
+    *   Create functions to read from and write to this cache file.
+
+2.  **Task: Implement the `packs update` Command**
+    *   This is the only command that will access the network for the index.
+    *   Fetch the `packs.json` from the hardcoded official Index Repository URL.
+    *   Validate the JSON structure and required fields.
+    *   Retry the fetch once on failure; if it still fails or validation fails, retain the previous cache and surface a clear error message.
+
+3.  **Task: Enhance `packs add` to Use the Index**
+    *   Update the command's logic:
+        a. First, check if the given `<input>` is a slug. If so, use the Phase 1 workflow.
+        b. If not a slug, search for a pack with a matching `name` in the local index cache.
+        c. If found, use the `repo`, `commit`, or `tag` from the index entry to fuel the secure fetch-and-validate workflow from Phase 1.
+        d. Abort if the resolved `name` already exists locally from a different source.
+        e. Adjust warning messages based on whether the pack is pinned to a commit or tag, warning that unpinned installs may change unexpectedly.
+
+**Outcome of Phase 2:** A user can run `packs update` to discover new packs and then install them by their simple name (e.g., `rulebook-ai packs add python-pro-pack`).
+
+---
+
+### Phase 3: The UI Layer - Listing and Visibility
+
+**Objective:** To provide users with a clear, unified view of all packs available to them.
+
+1.  **Task: Implement Built-in Pack Discovery**
+    *   Create a mechanism for the CLI to be aware of its own bundled, built-in packs.
+
+2.  **Task: Implement the `packs list` Command**
+    *   This command will **not** access the network.
+    *   It will read the list of built-in packs.
+    *   It will read the list of community packs from the local cache (from Phase 2).
+    *   It will present a single, merged list to the user, clearly distinguishing between `(built-in)` and `(community)` packs and showing metadata like the summary.
+
+**Outcome of Phase 3:** A user can run `packs list` to see all installable packs from all sources at a glance.
+
+---
+
+### Phase 4: The Ecosystem - Contributor Tooling
+
+**Objective:** To set up the external infrastructure needed for the community to thrive.
+
+1.  **Task: Create the Public Index GitHub Repository**
+    *   This codebase ships a template under `community-index/` containing `packs.json`, `README.md`, and `CONTRIBUTING.md`.
+    *   To publish it, run `git init`, commit the files, and push to a new public GitHub repository (e.g., `rulebook-ai-community/index`).
+
+2.  **Task: Implement the CI Validation Workflow**
+    *   The template includes `.github/workflows/validate.yml` and `scripts/validate_index.py`.
+    *   The workflow runs on pull requests and clones each referenced pack to verify required files and `manifest.yaml` name alignment.
+
+**Outcome of Phase 4:** The community has a place to submit packs and a clear, automated process for validating their submissions, completing the ecosystem loop.
+ing the ecosystem loop.