Merge pull request #8 from londogard/copilot/implement-s3-import-command

Add streaming S3 import command with metadata identity and path filter support

Author: Lundez

.github/FUNDING.yml

@@ -0,0 +1 @@
+github: [londogard]

.github/workflows/ci.yml

@@ -0,0 +1,61 @@
+name: CI
+
+on:
+    push:
+    pull_request:
+
+jobs:
+    test-and-build:
+        runs-on: ubuntu-latest
+
+        steps:
+            - name: Check out repository
+              uses: actions/checkout@v4
+
+            - name: Set up Python
+              uses: actions/setup-python@v5
+              with:
+                  python-version: "3.12"
+
+            - name: Set up uv
+              uses: astral-sh/setup-uv@v4
+
+            - name: Install dependencies
+              run: uv sync --group dev
+
+            - name: Reject macOS junk files
+              run: |
+                  if git ls-files | grep -E '(^__MACOSX/|(^|/)\.DS_Store$|(^|/)\._)'; then
+                    echo "Remove macOS archive/junk files before merging."
+                    exit 1
+                  fi
+
+            - name: Run non-integration tests
+              run: uv run pytest tests -m "not integration"
+
+            - name: Smoke-test CLI help
+              run: uv run fluxel --help
+
+            - name: Build package
+              run: uv build
+
+    s3-integration:
+        runs-on: ubuntu-latest
+
+        steps:
+            - name: Check out repository
+              uses: actions/checkout@v4
+
+            - name: Set up Python
+              uses: actions/setup-python@v5
+              with:
+                  python-version: "3.12"
+
+            - name: Set up uv
+              uses: astral-sh/setup-uv@v4
+
+            - name: Install dependencies
+              run: uv sync --group dev
+
+            - name: Run S3 integration tests against MiniStack
+              run: bash scripts/run_s3_integration.sh

.gitignore

@@ -2,6 +2,9 @@
 __pycache__/
 *.py[cod]
 *$py.class
+.DS_Store
+._*
+__MACOSX/
 
 # C extensions
 *.so

CHANGELOG.md

@@ -0,0 +1,27 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on Keep a Changelog, and Fluxel currently tracks changes before its first public beta release.
+
+## Unreleased
+
+### Added
+
+- Streaming S3 import with metadata identity mode and repeatable path filters.
+- Manifest sidecar indexes for exact-path and prefix-based lookups.
+- Staged add support for arbitrary local files, directories, S3 objects, and S3 prefixes.
+- Real S3 integration coverage for remote repository flows.
+- AGPL-3.0-or-later licensing, attribution notice, and funding metadata for the first public beta.
+
+### Changed
+
+- CLI examples and tests now prefer `--repo` repository selection semantics.
+- Client-local state writes now use atomic replace semantics for HEAD and staging payloads.
+- Manifest index prefix iteration now streams rows instead of materializing full result sets.
+
+### Fixed
+
+- Manifest parsing now validates entry shape, digests, and metadata-only invariants with line-aware errors.
+- CLI commands now return clean `... error:` messages for common validation, filesystem, and object-storage failures instead of raw tracebacks.
+- Corrupt S3-hosted manifest lines now surface actionable validation errors.

CONTRIBUTING.md

@@ -0,0 +1,49 @@
+# Contributing
+
+## Development Setup
+
+Fluxel uses `uv` for dependency management and local commands.
+
+```bash
+uv sync --group dev
+uv run fluxel --help
+```
+
+## Local Validation
+
+Run the full local suite:
+
+```bash
+uv run pytest tests
+```
+
+Run only the real S3 integration tests against Ministack:
+
+```bash
+export FLUXEL_MINISTACK_ENDPOINT=http://127.0.0.1:9000
+export FLUXEL_MINISTACK_ACCESS_KEY=ministack
+export FLUXEL_MINISTACK_SECRET_KEY=ministack123
+export FLUXEL_MINISTACK_REGION=us-east-1
+
+uv run pytest tests/test_s3_integration.py -m integration
+```
+
+If `FLUXEL_MINISTACK_ENDPOINT` is unset or unreachable, the integration tests skip automatically.
+
+## Project Expectations
+
+- Fluxel is licensed under AGPL-3.0-or-later; preserve the license and notice files in redistributions.
+- Prefer Python type hints by default.
+- Keep Fluxel client-first: no server, daemon, or central database.
+- Keep canonical blob storage simple and immutable.
+- Metadata-only operations must not read blob payloads.
+- Prefer stream-safe, O(1)-memory patterns when working with manifests and large imports.
+- Use Blake3 for Fluxel content and identity hashing.
+
+## Pull Requests
+
+- Keep changes focused and explain user-facing behavior changes clearly.
+- Add or update tests for every functional change.
+- Update [README.md](README.md) and [CHANGELOG.md](CHANGELOG.md) when the CLI, packaging, or documented workflows change.
+- Keep [NOTICE](NOTICE) aligned with project attribution and sponsorship guidance.
+- Prefer `--repo` in docs and examples for repository selection.

ISSUES.MD

@@ -87,7 +87,7 @@ Implement `fluxel log` using commit parent links only.
 - Optional JSON output mode.
 
 **Acceptance Criteria**
-- `fluxel log --root <root> main` prints deterministic history.
+- `fluxel log --repo <repo> main` prints deterministic history.
 - No blob reads occur during history traversal.
 - Unit tests cover empty/single/multi-commit chains.