[roadmap] Add Evidence Integrity & Disposition Phase 1

[ewilhelm1979-netizen]

Problem and Goal

ISCY already stores, versions, hashes, rates, and links Evidence. This PR adds Evidence Integrity & Disposition Phase 1 so existing Evidence can be checked, placed under Legal Hold, and disposition decisions can be documented without introducing a new Evidence engine or physically deleting files.

Changed Functions

• Add tenant-scoped Evidence integrity overview data.
• Add manual and bounded batch SHA-256 re-checks for stored Evidence artifacts.
• Add integrity status, calculated hash, mismatch, safe error class, quarantine/review metadata, and review notes.
• Add Legal Hold set/release metadata.
• Add disposition and retention decision metadata.
• Add audit events for integrity checks, mismatches, missing artifacts, Legal Hold changes, and disposition decisions.

Affected Files

• rust/iscy-backend/src/db_admin.rs
• rust/iscy-backend/src/evidence_store.rs
• rust/iscy-backend/src/lib.rs
• rust/iscy-backend/tests/http_tests.rs
• CHANGELOG.md
• docs/ISCY_STRATEGIC_ROADMAP.md
• docs/ISCY_Handbuch.md
• docs/ISCY_Handbuch.pdf

Database Migrations

• Adds additive migration 0033_rust_evidence_integrity_disposition for SQLite and PostgreSQL.
• Extends evidence_evidenceitem with integrity, Legal Hold, and disposition metadata.
• Adds evidence_integrity_event for tenant-scoped audit history.
• Adds indexes for integrity status, Legal Hold status, disposition status, and event lookups.
• Existing Evidence data is preserved; no destructive migration is included.

New or Changed API Endpoints

• GET /api/v1/evidence/integrity
• POST /api/v1/evidence/{evidence_id}/integrity-check
• POST /api/v1/evidence/integrity-checks
• GET /api/v1/evidence/{evidence_id}/integrity-events
• POST /api/v1/evidence/{evidence_id}/legal-hold
• POST /api/v1/evidence/{evidence_id}/legal-hold/release
• POST /api/v1/evidence/{evidence_id}/disposition

GUI Changes

• Adds /evidence/integrity/ Web UI.
• Shows safe metadata for Evidence title, type, quality status, hash presence, integrity status, Legal Hold, retention/disposition dates, last check, and disposition state.
• Read-only roles see safe metadata only.
• Admin/editor roles get actions for Re-Hash, Legal Hold set/release, and metadata-only disposition decisions.

Tenant and Permission Boundaries

• Every read, write, event, and target lookup is tenant-scoped in SQL.
• Foreign Evidence IDs are hidden as not found.
• Write operations require authenticated admin/editor-style write permissions.
• Read-only roles can inspect safe metadata and audit history but cannot trigger checks or change Legal Hold/disposition state.
• Batch checks are bounded and tenant-local.

Security Decisions

• Re-hash values are computed server-side; client-provided hashes are not trusted.
• Stored Evidence paths are resolved under the configured media root with safe canonical containment checks.
• Responses and events do not expose absolute media paths, secrets, authorization material, raw file payloads, SQL details, or foreign tenant IDs.
• disposition_completed_metadata_only is intentionally metadata-only and never deletes files.
• No dependency, CI, CodeQL, cargo-audit, cargo-deny, Docker, nginx, Postgres, Nix, or platform upgrade is included.

Tests Executed

• cargo fmt --manifest-path rust/iscy-backend/Cargo.toml -- --check
• cargo clippy --locked --manifest-path rust/iscy-backend/Cargo.toml --all-targets -- -D warnings
• cargo test --locked --manifest-path rust/iscy-backend/Cargo.toml
• cargo audit --file rust/iscy-backend/Cargo.lock --ignore RUSTSEC-2023-0071 via nix run nixpkgs#cargo-audit
• cargo deny --manifest-path rust/iscy-backend/Cargo.toml check advisories licenses sources via nix run nixpkgs#cargo-deny inside the dev shell
• make rust-smoke
• make rust-restore-smoke
• nix flake check
• make rust-postgres-restore-drill initially reported SKIP in the Codex environment because disposable PostgreSQL URLs were not configured
• make docs-pdf
• git diff --check

Additional PostgreSQL Live Drill

After the initial Draft PR validation, a local PostgreSQL live restore drill was executed successfully on the maintainer workstation.

• Branch: feature/evidence-integrity-disposition
• PostgreSQL version: PostgreSQL 16.14
• Source database: disposable local PostgreSQL database, postgres://iscy@127.0.0.1:24987/iscy_source
• Restore database: disposable local PostgreSQL database, postgres://iscy@127.0.0.1:24987/iscy_restore
• Result: Rust PostgreSQL restore drill OK
• Migration 0033_rust_evidence_integrity_disposition was applied successfully on PostgreSQL and confirmed in the restored database.
• git status remained clean.
• No files were changed.
• No commits were created.
• The temporary PostgreSQL instance was stopped, the port no longer answered, and the temporary data directory was deleted.

This closes the previously noted local limitation around PostgreSQL live migration validation.

Known Limitations

• Docker Compose validation and hardened Docker build were not executed locally because Docker/Podman is not available in this environment. GitHub CI covered these paths successfully.
• cargo audit completed with exit 0 after loading RustSec advisories; it warned that the crates.io index could not be updated locally.

Deliberately Not Implemented

• No S3/object-storage backend.
• No physical Evidence deletion or technical data destruction.
• No complex scheduler or periodic Re-Hash worker.
• No new Evidence, Risk, Control, or Notification engine.
• No CA/PKI/CSR work.
• No signed agent packages or release provenance.
• No dependency or platform upgrades.

AI Assistance

Implemented with OpenAI Codex assistance. Human review remains required for correctness, security, legal interpretation, and release approval.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2e7b51b470

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Require authentication for integrity metadata view

When this page is requested without a session or bearer token, web_context_from_request falls back to caller-supplied tenant_id/user_id query parameters, so /evidence/integrity/?tenant_id=...&user_id=... returns that tenant's evidence integrity metadata to anyone who can guess a tenant id. This new view exposes evidence titles, legal-hold/disposition state, hashes, and check results; unlike the new API and write paths it should require authenticated_tenant_context or restrict the query-parameter fallback to a development-only path.

Useful? React with 👍 / 👎.