feat: local-first tamper-evident audit log callback (asqav)

[jagmarques]

Relevant issues

Closes #25329. Follows up on the local-first question from discussion #25237.

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all unit tests on make test-unit
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Screenshots / Proof of Fix

Captured from a live proxy on localhost:4000. The model response uses mock_response (no provider key on this machine), which only stubs the upstream LLM call. The proxy routing, the callback hooks, and the audit file writes are the real code paths.

Config:

model_list:
  - model_name: mock-gpt-4o
    litellm_params:
      model: openai/gpt-4o
      api_key: not-needed
      mock_response: "Audit me."

litellm_settings:
  callbacks: ["asqav"]

Start the proxy and run three calls through it:

$ ASQAV_LOG_PATH=/tmp/litellm_asqav_audit.jsonl python litellm/proxy/proxy_cli.py --config /tmp/asqav_proof_config.yaml --port 4000

$ for i in 1 2 3; do curl -s http://localhost:4000/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d "{\"model\": \"mock-gpt-4o\", \"messages\": [{\"role\": \"user\", \"content\": \"audit test $i\"}]}" \
    | python3 -c "import json,sys; r=json.load(sys.stdin); print(r['id'], r['choices'][0]['message']['content'])"; done
chatcmpl-f81dafa7-d861-41ab-8156-5e86202a2cae Audit me.
chatcmpl-d16e724d-d720-4a04-8ec9-2e27bd32900c Audit me.
chatcmpl-f5dc5a85-2a40-4d4d-a2f0-a05b0011a5cf Audit me.

One record per call lands in the audit file, each chained to the previous one:

$ python3 -c "
import json
for line in open('/tmp/litellm_asqav_audit.jsonl'):
    r = json.loads(line)
    print(r['seq'], r['model'], r['status'], 'prev=' + r['prev_hash'][:12], 'hash=' + r['record_hash'][:12])
"
0 gpt-4o success prev=000000000000 hash=b13da1d5b912
1 gpt-4o success prev=b13da1d5b912 hash=fdfdf752fe43
2 gpt-4o success prev=fdfdf752fe43 hash=5a978d02ca65

First record in full (content stored as digests, not plaintext):

{
    "seq": 0,
    "ts": "2026-06-12T04:20:49.787547+00:00",
    "prev_hash": "0000000000000000000000000000000000000000000000000000000000000000",
    "call_id": "chatcmpl-f81dafa7-d861-41ab-8156-5e86202a2cae",
    "model": "gpt-4o",
    "status": "success",
    "latency_ms": 22,
    "prompt_tokens": 10,
    "completion_tokens": 20,
    "total_tokens": 30,
    "finish_reason": "stop",
    "provider_request_id": null,
    "messages_digest": "1820df7315201252b6424d32388298fb06949371a53fbf95dd1074667828a828",
    "response_content_digest": "b8b49bde4d35a98d569025820c23e34fd11090437764327c0d8894f8fdc89c9b",
    "metadata": {},
    "record_hash": "b13da1d5b9120e7b2854cff10471081a2ccbd50a3a50dac70b51bc77ace491c6"
}

Verification passes on the clean log and fails after flipping one byte:

$ ASQAV_LOG_PATH=/tmp/litellm_asqav_audit.jsonl python -c "
from litellm.integrations.asqav import AsqavLogger
ok, msg = AsqavLogger().verify_chain()
print('verify_chain:', ok, msg)
"
verify_chain: True ok

$ sed 's/success/SUCCESS/' /tmp/litellm_asqav_audit.jsonl > /tmp/tampered.jsonl
$ python -c "
from litellm.integrations.asqav import AsqavLogger
ok, msg = AsqavLogger().verify_chain('/tmp/tampered.jsonl')
print('verify_chain (tampered):', ok, msg)
"
verify_chain (tampered): False line 1: hash mismatch (stored=b13da1d5b912, computed=ae375f2576a1)

Type

New Feature

Changes

#25329 closed after krrish asked whether signing could be done on device without a network call per LLM invocation. This PR does that.

AsqavLogger is a CustomLogger that appends one JSON record per call to a local JSONL file (~/.litellm_asqav_audit.jsonl by default, overridable with ASQAV_LOG_PATH). Each record carries a SHA-256 chain hash over its canonical fields plus the hash of the previous record, so the log verifies offline with stdlib. Nothing external runs on the per-call path.

The record append happens under the chain lock, so concurrent callbacks always land on disk in chain order, and the async hooks hand the write to asyncio.to_thread to keep the event loop free. On restart the logger re-reads the tail of the log with a widening window, so records of any size resume the chain correctly.

Content is hashed rather than stored in plaintext, which matters for compliance environments that cannot retain raw prompts. There's an optional cloud checkpoint, off by default. It runs on a background thread, only activates when ASQAV_API_KEY is set, and sends only the chain head and sequence number, so it has no effect on the proxy's hot path and leaks nothing about the local filesystem.

For proxy config, "asqav" is registered in _custom_logger_compatible_callbacks_literal, CustomLoggerRegistry, and _init_custom_logger_compatible_class. Users can set callbacks: ["asqav"] in config.yaml.

24 tests: 23 in tests/test_litellm/integrations/asqav/test_asqav.py covering chain integrity, tamper detection (modified hash, deleted record), restart resume including a record over 4 KB, concurrent-write ordering, checkpoint payload contents, content hashing behavior, and the invariant that no background thread starts without an API key, plus a callback registration test in tests/test_litellm/litellm_core_utils/test_litellm_logging.py. pytest tests/test_litellm/integrations/asqav/test_asqav.py -v

Docs page at docs/my-website/docs/observability/asqav_integration.md

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

[codecov]

Codecov Report

❌ Patch coverage is 83.08458% with 34 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/integrations/asqav/asqav.py	82.10%	34 Missing ⚠️

📢 Thoughts on this report? Let us know!

[veria-ai]

PR overview

This PR adds an ASQAV integration callback that records a local-first, tamper-evident audit log for LiteLLM activity. The touched code maintains audit-log sequencing and hash chaining for generated records.

There is one open security concern: the audit chain state is currently process-local, so deployments with multiple proxy workers can generate conflicting sequence and hash-chain values. That can make legitimate audit records fail verification when requests are handled by different workers, weakening the reliability of the tamper-evidence mechanism in multi-process setups. Four issues have already been addressed, so the remaining risk is focused on making the chain state safe across workers.

Open issues (1)

• Medium: Audit chain state is process-local — litellm/integrations/asqav/asqav.py:209

Fixed/addressed: 4 · PR risk: 5/10

[greptile-apps]

Greptile Summary

This PR introduces AsqavLogger, a local-first tamper-evident audit callback that appends one SHA-256-chained JSONL record per LLM call with no runtime dependencies beyond stdlib. Registration plumbing (__init__.py, custom_logger_registry.py, litellm_logging.py) is clean and follows the existing singleton pattern for in-memory loggers.

• Chain integrity — seq/prev_hash assignment and the file write are all within self._lock, keeping the on-disk chain consistent under concurrent threads; asyncio.to_thread keeps async hooks off the event loop.
• Restart recovery — _read_tail doubles its window until a complete last line is captured, and _load_chain_tail restores both _prev_hash and _call_count so seq numbering continues correctly.
• Redact-content opt-out — when ASQAV_REDACT_CONTENT=false, both plaintext content fields (messages, response_content) and their SHA-256 digest counterparts (messages_digest, response_content_digest) end up in the record; the digests should be removed when plaintext is stored instead.

Confidence Score: 5/5

The new integration is entirely additive, fail-soft on every code path, and does not touch any existing request-handling logic.

The chain locking, async offloading, tail-read doubling, and seq-counter restoration are all correctly implemented and backed by 24 tests. The only findings are minor — a redundant-field inconsistency when redact_content=False and two style nits — none of which affect call reliability or chain verification.

litellm/integrations/asqav/asqav.py — the redact_content=False branch and the inline frozenset constants.

Important Files Changed

Filename	Overview
litellm/integrations/asqav/asqav.py	New tamper-evident audit logger: chain logic, locking, and file I/O are correct; minor issues with redact_content=False storing both plaintext and digest, inline frozenset constants recreated on every call
litellm/litellm_core_utils/litellm_logging.py	Adds "asqav" branch to _init_custom_logger_compatible_class with correct singleton guard; no issues
tests/test_litellm/integrations/asqav/test_asqav.py	23 comprehensive mock-only tests covering chain integrity, tamper detection, concurrent writes, large records, and restart resumption; all local, no network calls

_{Reviews (5): Last reviewed commit: "style: apply black formatting to asqav i..." | Re-trigger Greptile}

[Sameerlite]

Thanks for the contribution, @jagmarques! A few things to address before this is ready:

1. Greptile raised some concerns (scored 3/5) — specifically a race condition where file writes happen outside the lock (causing out-of-order writes under concurrent callbacks), a hardcoded 4 KB tail-read buffer that can silently truncate audit entries, and missing __all__ / __repr__. These are worth addressing before merge.
2. CI is failing — could you check the failing checks and either fix them or comment if they're pre-existing/unrelated?
3. Proof of working — the "Screenshots / Proof of Fix" section has instructions but no captured output. A quick log snippet or demo of the audit log being written and verified would be great.

Happy to help if you have questions!

[jagmarques]

Pushed 51b0e1d with all three addressed.

The file append now happens inside the chain lock in _build_and_append (litellm/integrations/asqav/asqav.py:288), so seq assignment and the write are atomic and records land on disk in chain order. Chain state only advances after a successful write, and the async hooks hand the write to asyncio.to_thread so the event loop never blocks on disk I/O.

The fixed 4 KB tail buffer is replaced by _read_tail (asqav.py:58), which doubles its window until the whole last line fits, so a record of any size resumes the chain after a restart. Also added all and repr (repr reports config but never the API key) and dropped log_path from the checkpoint payload that greptile's security note flagged.

On CI, both failing checks were ours. lint was black formatting on asqav.py, now applied. codecov/patch came from the untested checkpoint and plaintext paths, which the new tests cover. There are 7 new tests, including a concurrency test (8 threads x 25 records each, ordering plus verify_chain) and a restart test with a 10 KB record. Both fail on the previous commit and pass now.

Proof is captured in the PR body: three curls through a live proxy on localhost:4000, the chained records, verify_chain returning ok, and a tampered copy failing at line 1.

@greptileai

Thanks for the detailed pointers Sameerlite.

[jagmarques]

Checking in on this one. The three points from your review are in (race fix inside the chain lock, the resizable tail read, and all / repr), CI is green across the legs, and the proof run is in the PR body. Happy to adjust anything else whenever you get a chance to look.

[Sameerlite]

Thanks for the PR! Triggering Greptile for a code review:

@greptileai

[Sameerlite]

we have a get httpx client method. Please use that

[jagmarques]

Outdated - the checkpoint code that used httpx was removed in commit 96c19ee. The current integration has no network calls and no httpx dependency; it is stdlib-only.

[Sameerlite]

This should not be here. It should be in litellm-docs

[jagmarques]

Outdated - the docs file was removed from this PR in a prior commit. The docs will be submitted separately to the litellm-docs repo.

[Sameerlite]

Thanks for the update — CI is now green and the proof output in the PR body is great! Greptile reviewed the current HEAD at 4/5 with a few open items:

• seq counter not restored on restart (P1): _load_chain_tail restores _prev_hash but leaves _call_count at 0, so the seq field silently resets to 0 after every restart. Greptile provided a one-line fix: self._call_count = last_record.get("seq", -1) + 1.
• Documentation file in wrong repo (P2): the docs page at docs/my-website/docs/observability/asqav_integration.md should go in the litellm-docs repo, not here.
• Sameerlite's open threads: please also use get_httpx_client() instead of creating your own HTTP client, and move the docs as noted above.

Once those are addressed and Greptile reaches 5/5 with no open threads we'll take another look — nearly there!

@greptileai

[CLAassistant]

All committers have signed the CLA.

[jagmarques]

Pushed 14f6407 addressing the open review items:

F401 lint - removed the unused httpxSpecialProvider import. It was imported but never referenced in the file.

seq restore on restart - _load_chain_tail already sets self._call_count = last_record.get("seq", -1) + 1; the existing test_seq_counter_restored_after_restart test asserts this and it passes.

get_httpx_client - the checkpoint path was the only place a custom HTTP client appeared. Rather than just swapping the client, I removed the cloud checkpoint feature entirely (see next point). No raw httpx.Client(...) calls remain.

Cloud checkpoint removed - checked the Asqav cloud source: there is no /v1/checkpoints or /api/v1/checkpoints route. The API prefix is /api/v1 and the route list covers agents, sessions, signatures, observations, and others, but no checkpoint endpoint. Shipping a path that always 404s is worse than no path, so I removed _schedule_checkpoint, the api_key/checkpoint_interval constructor params, and the env vars ASQAV_API_KEY / ASQAV_CHECKPOINT_INTERVAL. The local JSONL audit log (the PR's core feature) is unaffected.

Docs (P2) - no docs file was in this PR diff. Created a separate PR to BerriAI/litellm-docs: BerriAI/litellm-docs#376

Tests: 23 passed, ruff clean.

[jagmarques]

Rebased onto current litellm_internal_staging (conflict was in tests/test_litellm/litellm_core_utils/test_litellm_logging.py — upstream added two new partial-spend tests; resolved by keeping both upstream tests and our asqav singleton test). Also fixed the strict-rule budget violations: replaced Dict/Tuple from typing with the modern built-in forms (dict/tuple) across all annotations in asqav.py (11 UP006 violations, now 0). Strict gate and ruff both pass locally; 23 asqav unit tests pass. Ready for Greptile re-review.

[jagmarques]

All review feedback is addressed at the latest head and CI is green: the integration uses litellm's shared get_httpx_client, the seq counter is restored on restart with a regression test, file permissions are tightened to 0600, and proxy identity metadata is captured with sensitive values filtered. The documentation page lives in litellm-docs (PR #376). The inline review threads are resolved. Could you take another look when you have a moment? Thanks.

[Sameerlite]

@greptileai