Skip to content

FEATURE_MATRIX.md

Latest commit

 

History

History
251 lines (182 loc) · 7.49 KB

FEATURE_MATRIX.md

File metadata and controls

251 lines (182 loc) · 7.49 KB

FEATURE MATRIX — AELITIUM

Status: DRAFT Rule: If not explicitly implemented or verifiable in the current code surface, it is not claimed.

1. Capture — LiteLLM auto-instrumentation

  • Feature: LiteLLM auto-capture via enable_litellm()

  • Interface: Python API

  • Source: engine/capture/litellm.py, aelitium/__init__.py

  • Status: implemented

  • Guarantees:

    • Exposes enable_litellm() as a public API entrypoint
    • Enables capture for supported LiteLLM call paths handled by the integration
    • Captured artifacts include deterministic hashes such as request_hash, response_hash, and binding_hash

  • Non-guarantees:

    • Does not capture calls made before instrumentation is enabled
    • Does not guarantee coverage of every LLM call path in a codebase
    • Does not verify provider-side behavior or output truth
    • Does not guarantee identical outputs across runs

2. Capture — OpenAI (non-streaming)

  • Feature: OpenAI non-streaming capture

  • Interface: Python API

  • Source: engine/capture/openai.py, aelitium/__init__.py

  • Status: implemented

  • Guarantees:

    • Captures supported OpenAI request/response pairs
    • Produces deterministic hashes for captured request and response artifacts
    • Binds captured request and recorded response through binding_hash

  • Non-guarantees:

    • Does not capture unsupported SDK usage patterns or unwrapped call paths
    • Does not guarantee completeness of provider metadata in every case
    • Does not assert semantic correctness of model output

3. Capture — OpenAI (streaming)

  • Feature: OpenAI streaming capture

  • Interface: Python API

  • Source: engine/capture/openai.py, aelitium/__init__.py

  • Status: implemented

  • Guarantees:

    • Captures supported streaming responses as recorded evidence artifacts
    • Produces deterministic hashes over the recorded artifacts generated by the capture path

  • Non-guarantees:

    • Does not claim replay semantics beyond the recorded artifacts
    • Does not reconstruct provider-internal state
    • Does not assert semantic correctness of streamed output

4. Capture — Anthropic Messages

  • Feature: Anthropic Messages capture

  • Interface: Python API

  • Source: engine/capture/anthropic.py, aelitium/__init__.py

  • Status: implemented

  • Guarantees:

    • Captures supported Anthropic Messages request/response pairs
    • Produces deterministic request, response, and binding hashes for captured artifacts

  • Non-guarantees:

    • Does not capture unsupported SDK usage patterns or unwrapped call paths
    • Does not guarantee completeness of provider metadata in every case
    • Does not assert semantic correctness of model output

5. Canonicalization

  • Feature: canonicalize

  • Interface: CLI

  • Source: engine/ai_canonical.py, engine/ai_cli.py

  • Status: implemented

  • Guarantees:

    • Exposes CLI canonicalization for supported request inputs
    • Produces a normalized representation used by the current hashing flow

  • Non-guarantees:

    • Does not claim stability across future canonicalization changes unless explicitly versioned
    • Does not include provider-side hidden parameters not present in captured inputs

6. Evidence hashing

  • Feature: Deterministic hashing of captured artifacts

  • Interface: Internal / artifact-level

  • Source: capture modules in engine/capture/*

  • Status: implemented

  • Guarantees:

    • Same canonical artifact content produces the same hash
    • binding_hash is derived from the canonical request scope and recorded response artifact in the implemented capture flow

  • Non-guarantees:

    • Does not prove external truth of the captured content
    • Does not eliminate trust in the capture environment
    • Does not prevent tampering before capture or before hashing

7. Bundle packaging

  • Feature: pack

  • Interface: CLI

  • Source: engine/ai_pack.py, engine/ai_cli.py

  • Status: implemented

  • Guarantees:

    • Exposes bundle packaging through the CLI
    • Packages supported evidence artifacts into the project's bundle structure

  • Non-guarantees:

    • Does not by itself validate semantic correctness of bundle contents
    • Does not guarantee completeness of capture

8. Bundle verification

  • Feature: verify-bundle

  • Interface: CLI

  • Source: engine/ai_cli.py

  • Status: implemented

  • Guarantees:

    • Verifies bundle integrity against included hashed artifacts
    • Returns an invalid verification result when verification checks fail
    • Can be run offline against local bundle contents

  • Non-guarantees:

    • Does not prove when the bundle was generated
    • Does not prove what a model "actually said"
    • Does not verify facts outside the included artifacts
    • Does not validate full schema correctness of the canonical payload

9. Receipt verification

  • Feature: verify-receipt

  • Interface: CLI

  • Source: engine/ai_cli.py

  • Status: implemented

  • Guarantees:

    • Validates receipt JSON structure and required fields handled by the verifier
    • Verifies subject hash when provided to the verifier flow
    • Verifies Ed25519 signature checks in supported receipt verification paths

  • Non-guarantees:

    • Does not prove external event authenticity beyond the receipt contents and verification inputs
    • Does not establish a real-world timeline by itself
    • Does not prove what a model "actually said"

10. Compare

  • Feature: compare

  • Interface: CLI

  • Source: engine/ai_cli.py

  • Status: implemented

  • Guarantees:

    • Compares supported evidence inputs through the CLI
    • Returns explicit comparison outcomes including UNCHANGED, CHANGED, NOT_COMPARABLE, and INVALID_BUNDLE

  • Non-guarantees:

    • Does not explain why a change occurred
    • Does not attribute causality to provider, model, prompt, or environment changes

11. Scan / check

  • Feature: scan (check alias)

  • Interface: CLI

  • Source: engine/ai_cli.py

  • Status: implemented

  • Guarantees:

    • Exposes scan functionality for identifying potential uncaptured LLM call sites
    • Exposes check as an alias of the scan path

  • Non-guarantees:

    • Does not guarantee full coverage of all call paths
    • May produce false positives or false negatives
    • Does not prove that a flagged site executed at runtime

12. Evidence log

  • Feature: EvidenceLog / append-only local logging / chain verification

  • Interface: Python API / internal logging support

  • Source: engine/capture/log.py, aelitium/__init__.py

  • Status: implemented

  • Guarantees:

    • Supports append-only local evidence logging in the implemented log format
    • Exposes chain verification via verify_chain() for the log structure implemented in code

  • Non-guarantees:

    • Does not provide distributed consistency
    • Does not guarantee durability beyond the local storage environment
    • Does not prove external truth of logged content

13. EU AI Act Art. 12 export

  • Feature: export_eu_ai_act_art12

  • Interface: Python API

  • Source: engine/compliance.py, aelitium/__init__.py

  • Status: implemented

  • Guarantees:

    • Returns a structured export dictionary derived from processed bundle files
    • Exposes the exporter through the public Python API

  • Non-guarantees:

    • Does not guarantee legal compliance
    • Does not constitute certification, legal advice, or regulatory acceptance
    • Does not guarantee that every required external record exists outside the processed bundle files

Global non-guarantees

  • Does not prove what a model "actually said"
  • Does not guarantee exact timing of events
  • Does not eliminate trust in the capture environment
  • Does not provide tamper-proof guarantees
  • Does not ensure completeness of all LLM interactions