-
Notifications
You must be signed in to change notification settings - Fork 2
Expand file tree
/
Copy path.env.template
More file actions
141 lines (126 loc) · 8.91 KB
/
Copy path.env.template
File metadata and controls
141 lines (126 loc) · 8.91 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
# CliniCARE-Bench — configuration
#
# Copy to `.env` and fill in. Keep it out of git (.gitignore already covers it).
# The file is plain KEY=VAL, so it must be EXPORTED, not merely sourced:
#
# set -a; source .env; set +a
#
# `harbor run` reads these from the process environment; each task.toml pulls what it
# needs through `[environment.env]` substitution.
# ── Data: MIMIC-IV ────────────────────────────────────────────────────────────
# You must be credentialed on PhysioNet for MIMIC-IV. See DATA_ACCESS.md.
#
# duckdb — a local copy of MIMIC-IV converted to Parquet. Recommended: no per-query
# cost, no auth expiry, and a frozen snapshot so results stay reproducible.
# bigquery — MIMIC-IV on PhysioNet's BigQuery. Queries are billed to YOUR project, and
# physionet-data is unclustered, so a full sweep can get expensive. Read the
# cost section of DATA_ACCESS.md before choosing this.
MIMIC_BACKEND=duckdb
# Root of the converted Parquet tree — expects <root>/{hosp,icu,ed,note}/<table>.parquet.
# Required for the duckdb backend. On Docker this is the path on the HOST; the sidecar compose
# bind-mounts it read-only into the one container allowed to touch patient data.
MIMIC_DATA_ROOT=/path/to/mimic-parquet
# Refuse to start when the conversion is incomplete (default). A missing table would otherwise
# make tools return empty, which an agent reports as "Indeterminate - lack of data" — a
# plausible wrong answer rather than an error. Set to 1 only to poke at a partial dataset.
# MIMIC_ALLOW_PARTIAL=0
# BigQuery backend only. Queries are billed here; physionet-data cannot be billed.
# MIMIC_BQ_BILLING_PROJECT=your-gcp-project
# Defaults target MIMIC-IV v3.1 — override only if your dataset names differ.
# MIMIC_BQ_PROJECT=physionet-data
# MIMIC_BQ_DATASET_HOSP=mimiciv_3_1_hosp
# MIMIC_BQ_DATASET_ICU=mimiciv_3_1_icu
# MIMIC_BQ_DATASET_ED=mimiciv_ed
# MIMIC_BQ_DATASET_NOTE=mimiciv_note
#
# Multi-tenant only: require every request to carry the caller's own PhysioNet-linked OAuth
# token instead of using the sidecar's credentials. Leave OFF for a self-hosted sidecar — a
# forwarded access token has no refresh material and expires after ~1 hour, which will kill a
# long sweep mid-run. Application Default Credentials refresh themselves and do not.
# MIMIC_BQ_PER_REQUEST_AUTH=0
# ── The cohort bundle (gated) ─────────────────────────────────────────────────
# full750.csv + dev148.csv. Distributed separately to PhysioNet-credentialed users;
# they are not in this repo and must not be committed to it. See DATA_ACCESS.md.
CLINICARE_COHORT_DIR=/path/to/clinicare-cohort
# ── The policy corpus (public download) ───────────────────────────────────────
# 71 clinical guideline documents backing the policy-grounding axis. A public release asset,
# not repo content — download and verify against its SHA256SUMS, then point here.
# See benchmark/corpus/README.md.
CLINICARE_CORPUS_DIR=/path/to/clinicare-corpus
# ── Trial image + egress allowlist ────────────────────────────────────────────
# Where the trial container image lives. A bare name is a LOCAL Docker image (the default, and
# all you need): build it with `bash scripts/build_image.sh`. Set a registry path only if a
# remote backend must pull it. The TAG is never configured — env_image.py derives it from a
# content hash of the Dockerfile + mimic_tools, so the image and the tasks can't disagree.
# CLINICARE_IMAGE_REPO=clinicare-env
# The agent container is egress-locked to these hosts and nothing else. Defaults to the three
# vendor APIs. You rarely need to set this: the mimic-tools sidecar is appended automatically, and
# so is the host of any *_BASE_URL below — a gateway works without being named here. Set it only
# to allow something neither of those covers. Dotted hostnames only (some Harbor backends reject a
# bare label). Baked in at build_tasks.py time, so rebuild the tasks after changing it.
# CLINICARE_ALLOWED_HOSTS=api.anthropic.com,api.openai.com,generativelanguage.googleapis.com
# ── The mimic-tools sidecar (running trials requires Docker) ──────────────────
# Data access is credential-isolated: the agent container holds no data credentials and reaches
# MIMIC only by proxying to a sidecar that does. The MCP server is proxy-only: there is no
# in-container path to the data, because a code-capable agent could otherwise read the credentials
# out of its own environment and query the raw tables, bypassing the governed tool surface. So a
# sidecar is mandatory, not optional. See packages/mimic_tools/SIDECAR.md.
#
# On Docker the sidecar is just another compose service, started for you — no hosting required:
# harbor run ... --extra-docker-compose <tasks>/_sidecar.compose.yml
#
# (Exploring the tool surface directly — smoke_test.py, the Streamlit console — imports
# mimic_tools in-process and needs none of this. The sidecar is only for running trials.)
# Shared secret between the agent's proxy shim and the sidecar. Generate one:
#
# echo "MIMIC_API_SECRET=$(openssl rand -hex 32)" >> .env
#
# Required — the sidecar refuses to start without one. Why, given that a local Docker run is
# already private: an unset secret disables auth rather than failing, and a self-hosted sidecar
# binds a port, so the two together are an open MIMIC endpoint on any cloud VM. Set
# MIMIC_REQUIRE_API_SECRET=0 to opt out on a genuinely local-only instance.
MIMIC_API_SECRET=
# Leave UNSET for local Docker runs: docker-compose supplies the compose service address, and a
# value here would override it and silently send the agent somewhere else. Set it only if you
# host a sidecar yourself — which is what a non-Docker Harbor backend (e.g. Modal) requires,
# since the compose overlay does not apply there.
# MIMIC_TOOLS_URL=
# ── Models under test ─────────────────────────────────────────────────────────
# Four harnesses: Claude Code, Codex and Gemini CLI each read their own vendor's key; opencode
# (which carries the six open-weight arms) uses the OpenAI-compatible pair below. Set only the
# ones whose arms you intend to run.
ANTHROPIC_API_KEY=
OPENAI_API_KEY=
GEMINI_API_KEY=
# Point an arm at an OpenAI-compatible gateway instead of the vendor endpoint. Mind the
# route suffixes — they differ by client and getting them wrong reads as a model outage:
# Claude Code appends /v1/messages, so its base takes NO /v1 suffix
# Codex / OpenAI-compatible clients DO want the /v1 route
#
# REQUIRED for the six open-weight arms. Ten of the paper's sixteen systems are vendor CLIs on
# their own models and need only the keys above; the other six are open-weight models on the
# opencode harness, addressed as `openai/fireworks_ai/<model>`. No vendor endpoint serves those —
# they are routed by a gateway, so OPENAI_BASE_URL must point at yours. Same for the two
# harness-ablation arms (qwen-coder, mini-swe-agent). build_tasks.py allowlists whatever host you
# set here, so build the tasks with these exported and rebuild if you change one.
# ANTHROPIC_BASE_URL=
# OPENAI_BASE_URL=
# GOOGLE_GEMINI_BASE_URL=
# ── LLM judge ─────────────────────────────────────────────────────────────────
# HOST-SIDE ONLY. Every LLM-judged metric runs OUTSIDE the trial, from the artifacts the trial
# persisted (benchmark/eval/metrics/{process,policy_support}.py and eval/metrics/grounding/).
# The in-container verifier is deterministic — no LLM, no credentials, no network — so the
# agent container never needs these. See benchmark/_template/tests/test.sh.
#
# One variable, read by every judge call site. The judge speaks the OpenAI SDK, so
# JUDGE_BASE_URL needs the /v1 route.
JUDGE_BASE_URL=
JUDGE_API_KEY=
JUDGE_MODEL=openai/gpt-5.5
# ── MCP timeouts ──────────────────────────────────────────────────────────────
# The mimic MCP server cold-starts DuckDB over the whole table set, and the first note read can
# be slow. Too tight a window presents as the agent having no tools, which it then reports as
# "Indeterminate - lack of data" rather than as an error. Each CLI reads a different variable.
MCP_STARTUP_TIMEOUT_SEC=120 # codex (seconds)
MCP_TIMEOUT=120000 # claude-code handshake (ms)
MCP_TOOL_TIMEOUT=600000 # claude-code per-call (ms); gemini already defaults to 600s