Skip to content

README.md

Latest commit

 

History

History
83 lines (62 loc) · 3.33 KB

README.md

File metadata and controls

83 lines (62 loc) · 3.33 KB
Raw

AgentFresh

AgentFresh is a reproducible implementation of workload-aware, dependency-closed maintenance for materialized LLM-agent workflows when a black-box model deployment changes. The release contains the implementation, locked public workload snapshots, sanitized final-run databases, and the report files used to produce the manuscript's numerical results.

The release does not contain API credentials, provider endpoints, private research records, exploratory runs, manuscript files, rendered figures, or logs. The included DuckDB files preserve parsed states and execution provenance while removing raw model responses and error payloads.

Environment

Use Python 3.12 or newer. Install dependencies from the repository root:

python -m venv .venv
.\.venv\Scripts\python.exe -m pip install -r requirements.txt

The public result package can be inspected without any API configuration:

& .\.venv\Scripts\python.exe -m unittest discover -s tests -v
& .\.venv\Scripts\python.exe -m agentfresh status --run-id final-massive-20260821

Final result package

The two sanitized databases and their reports are under artifacts/final/runs/. They contain the MASSIVE and BFCL final collections, transition diagnostics, maintenance frontiers, paired bootstrap summaries, closure ablations, scale measurements, and BFCL transition certificates. No model calls are needed to inspect or reproduce the offline analysis from these materialized checkpoints. The replay command writes a new isolated run and leaves the published reports untouched:

& .\.venv\Scripts\python.exe -m agentfresh replay --source-run-id final-massive-20260821 --replay-run-id replay-massive
& .\.venv\Scripts\python.exe -m agentfresh replay --source-run-id final-bfcl-v4-v2-20260823 --replay-run-id replay-bfcl

Recompute from public data

MASSIVE is the official Amazon Science v1.1 test archive. The archive is provided in data/raw/massive/ through Git LFS. BFCL files are pinned to the public Gorilla repository commit recorded in configs/bfcl_v2.json and can be restored with:

& .\.venv\Scripts\python.exe -m agentfresh download-bfcl --config configs\bfcl_v2.json

To perform new API collection, copy .env.example to .env, fill in the credentials and model names, then use the commands below. These calls are not required for the published result package.

& .\.venv\Scripts\python.exe -m agentfresh prepare --run-id new-massive --config configs\massive_v1.json
& .\.venv\Scripts\python.exe -m agentfresh smoke --run-id new-massive
& .\.venv\Scripts\python.exe -m agentfresh collect --run-id new-massive
& .\.venv\Scripts\python.exe -m agentfresh evaluate --run-id new-massive

Every collection is checkpointed and resumable. The progress log records local timestamps, phase, stage, throughput, elapsed time, ETA, active work, and invalid-result counts.

Citation

Citation metadata is provided in CITATION.cff. Until publication metadata is available, cite the submitted manuscript by title and authors.

AgentFresh: Incremental View Maintenance for Materialized LLM-Agent Workflows under Black-Box Model Evolution. Yunjiao Shi and Rui Song. Submitted manuscript, 2026.

License

Code is released under the MIT License. Benchmark files retain their upstream licenses; consult their source notices before redistribution.