Skip to content

Commit 3ae096d

Browse files
muscariellomuscariello
authored
feat(agent_secrets): add 1Password as optional secret store backend (#5)
* feat(agent_secrets): add 1Password as optional secret store backend Add OnePasswordStore implementing the SecretStore trait via the op CLI, following the same pattern used by cargo-credential-1password. - New platform/onepassword.rs with put/get/delete/list_keys via op CLI - Items stored as Secure Notes with base64 content, tagged 'shadi' - Runtime backend selection via SHADI_SECRET_BACKEND=onepassword env var - Configurable vault (SHADI_OP_VAULT) and account (SHADI_OP_ACCOUNT) - Gated behind 'onepassword' Cargo feature flag - Feature enabled in shadictl and shadi_py - Unit tests for JSON parsing, error classification, and construction Closes #4 Signed-off-by: Luca Muscariello <muscariello@ieee.org> * fix(demo): fix sandbox and secrets for 1Password backend demo - Fix macOS Seatbelt sandbox: add sysctl-read, unrestricted mach-lookup for op daemon, ~/.slim write access for SLIM bindings, and resolve relative policy paths to absolute before emitting subpath rules - Set default llm_provider to anthropic in secops.toml and import script - Update import script to read LLM_PROVIDER env var with anthropic default - Update launch scripts to forward SHADI_SECRET_BACKEND, SHADI_OP_VAULT, SHADI_OP_ACCOUNT and use uv run --no-project --python - Add just build auto-install of shadi .so to venv - Add -op Justfile targets for 1Password-backed demo workflow - Update docs: README, architecture, security, cli, demo, scripts/README Signed-off-by: Luca Muscariello <muscariello@ieee.org> * fix(demo): pre-read 1P secrets, code-sign .so, fix SLIM startup and TLS - scripts/launch_slim.sh: remove --endpoint flag (conflicts with slimctl) - scripts/launch_secops_a2a.sh: add SLIM TLS cert defaults, PYTHONUNBUFFERED, and pre-read all 1Password secrets into SHADI_SECRET_* env vars before the sandbox starts (op CLI hangs without a TTY in background processes) - scripts/launch_avatar.sh: same pre-read block for avatar LLM + SLIM secrets - agents/secops/skills.py: require_shadi_secret() checks SHADI_SECRET_<KEY> env var fallback before calling op; avoids sandbox op hang - agents/avatar/adk_agent/agent.py: same env var fallback in require_shadi_secret_value(); fix send_message() to collect artifacts from all terminal states using state.value for correct enum comparison - agents/secops/a2a_server.py: startup print, executor debug print - crates/shadi_sandbox/src/platform/macos.rs: allow ~/.cache write for gh CLI - Justfile: codesign .so after build; demo-start depends on demo-stop; op vault list preflight in demo-start-op/demo-avatar-op - policies/demo/*.json: add litellm.prod.outshift.ai and github.com to net_allow - tools/test_avatar_transport.py: diagnostic transport test script Signed-off-by: Luca Muscariello <muscariello@ieee.org> * feat(secops): add OpenTelemetry tracing for command and skill execution Signed-off-by: Luca Muscariello <muscariello@ieee.org> * fix: resolve multiple bugs in sandbox, OTel, and memory storage - sandbox/macos: normalize resolve_path('.') to avoid trailing dot in Seatbelt subpath rules (was silently denying writes to sandboxed CWD) - secops/memory: fix SqlCipherMemoryStore called with key name instead of resolved key value; pass actual secret as key= arg not key_name= - secops/a2a_server: normalize labels list to comma-separated string before passing to skill_collect_security_issues - secops/telemetry: fix service.name empty when OTEL_SERVICE_NAME=''; use 'or' fallback instead of getenv default; forward OTEL vars via Justfile - tools/shadi_prompt.py: fix pre-existing syntax corruption (require_slima2a_packages body interleaved with load_secops_config and stray parser.add_argument calls; create_prompt_session missing if-not-ok body and return) - tools/test_avatar_transport.py: configurable timeout via CLI arg Signed-off-by: Luca Muscariello <muscariello@ieee.org> * chore: update demo Signed-off-by: Luca Muscariello <muscariello@ieee.org> * chore: update demo Signed-off-by: Luca Muscariello <muscariello@ieee.org> --------- Signed-off-by: Luca Muscariello <muscariello@ieee.org>
1 parent e27d2e9 commit 3ae096d

38 files changed

+3998
-611
lines changed

‎Cargo.lock‎

Lines changed: 3 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎Justfile‎

Lines changed: 74 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,8 @@ REMEDIATE := "false"
99

1010
build:
1111
PYO3_PYTHON="{{python312}}" RUSTFLAGS="-C link-arg=-undefined -C link-arg=dynamic_lookup" cargo build
12+
cp target/debug/libshadi.dylib .venv/lib/python3.12/site-packages/shadi/shadi.cpython-312-darwin.so
13+
codesign --sign - --force .venv/lib/python3.12/site-packages/shadi/shadi.cpython-312-darwin.so
1214

1315
windows-build:
1416
$env:PYO3_PYTHON = "{{python312}}"; cargo build
@@ -92,6 +94,9 @@ secops-run:
9294
secops-approve-prs:
9395
uv run --no-project --python .venv/bin/python agents/secops/secops.py --approve-prs
9496

97+
secops-test-python:
98+
uv run --with pytest pytest agents/secops/tests/test_skills.py
99+
95100
secops-a2a:
96101
uv run --no-project --python .venv/bin/python agents/secops/a2a_server.py
97102

@@ -110,6 +115,9 @@ secops-run-anthropic:
110115
secops-secrets:
111116
cargo run -p shadictl -- --list-keychain --list-prefix secops/
112117

118+
secops-secrets-op:
119+
SHADI_SECRET_BACKEND=onepassword cargo run -p shadictl -- --list-keychain --list-prefix secops/
120+
113121
secops-policy:
114122
cargo run -p shadictl -- --policy policies/demo/secops-a.json --print-policy
115123

@@ -148,7 +156,7 @@ launch-slim:
148156
./scripts/launch_slim.sh
149157

150158
launch-slim-example:
151-
SHADI_TMP_DIR="./.tmp" SLIM_ENDPOINT="127.0.0.1:47357" ./scripts/launch_slim.sh
159+
SHADI_TMP_DIR="./.tmp" ./scripts/launch_slim.sh
152160

153161
launch-secops-a2a:
154162
./scripts/launch_secops_a2a.sh
@@ -157,15 +165,80 @@ launch-secops-a2a-example:
157165
SHADI_TMP_DIR="./.tmp" SHADI_AGENT_ID="secops-a" SHADI_OPERATOR_PRESENTATION="local-operator" ./scripts/import_secops_secrets.sh
158166
SHADI_TMP_DIR="./.tmp" SHADI_AGENT_ID="secops-a" SHADI_OPERATOR_PRESENTATION="local-operator" ./scripts/launch_secops_a2a.sh
159167

168+
launch-secops-a2a-example-op:
169+
SHADI_SECRET_BACKEND=onepassword SHADI_TMP_DIR="./.tmp" SHADI_AGENT_ID="secops-a" SHADI_OPERATOR_PRESENTATION="local-operator" ./scripts/import_secops_secrets.sh
170+
SHADI_SECRET_BACKEND=onepassword SHADI_TMP_DIR="./.tmp" SHADI_AGENT_ID="secops-a" SHADI_OPERATOR_PRESENTATION="local-operator" ./scripts/launch_secops_a2a.sh
171+
160172
launch-avatar:
161173
./scripts/launch_avatar.sh
162174

163175
launch-avatar-example:
164176
SHADI_TMP_DIR="./.tmp" SHADI_AGENT_ID="avatar-1" SHADI_OPERATOR_PRESENTATION="local-operator" ./scripts/launch_avatar.sh
165177

178+
launch-avatar-example-op:
179+
SHADI_SECRET_BACKEND=onepassword SHADI_TMP_DIR="./.tmp" SHADI_AGENT_ID="avatar-1" SHADI_OPERATOR_PRESENTATION="local-operator" ./scripts/launch_avatar.sh
180+
166181
import-secops-secrets:
167182
./scripts/import_secops_secrets.sh
168183

184+
import-secops-secrets-op:
185+
SHADI_SECRET_BACKEND=onepassword ./scripts/import_secops_secrets.sh
186+
187+
# ── Demo orchestration ────────────────────────────────────────────────────────
188+
# Stop all demo processes (SLIM + SecOps A2A + Avatar).
189+
demo-stop:
190+
-kill $(cat .tmp/slim.pid 2>/dev/null) 2>/dev/null; rm -f .tmp/slim.pid
191+
-kill $(cat .tmp/secops-a2a.pid 2>/dev/null) 2>/dev/null; rm -f .tmp/secops-a2a.pid
192+
-pkill -f "run_sandboxed_agent\.py|a2a_server\.py|run_shadi_memory\.py" 2>/dev/null || true
193+
-pkill -f slimctl 2>/dev/null || true
194+
@echo "Demo stopped."
195+
196+
# Start SLIM + SecOps A2A in the background and write PIDs to .tmp/.
197+
# Use SHADI_HUMAN_GITHUB=<handle> to enable PR creation via gh CLI.
198+
demo-start: demo-stop
199+
mkdir -p .tmp
200+
slimctl slim start --config ".tmp/shadi-slim-mtls/server-config.yaml" >.tmp/slim.log 2>&1 & echo $! >.tmp/slim.pid
201+
sleep 2
202+
SHADI_TMP_DIR="./.tmp" SHADI_AGENT_ID="secops-a" SHADI_OPERATOR_PRESENTATION="local-operator" \
203+
./scripts/launch_secops_a2a.sh >.tmp/secops-a2a.log 2>&1 & echo $! >.tmp/secops-a2a.pid
204+
@echo "Demo started. Tail logs: just demo-logs"
205+
@echo "Launch interactive avatar: just demo-avatar"
206+
207+
# Same as demo-start but uses 1Password as the secret backend.
208+
demo-start-op: demo-stop
209+
@OP_ACCOUNT="${SHADI_OP_ACCOUNT:-my.1password.com}"; \
210+
op vault list --account "$OP_ACCOUNT" >/dev/null 2>&1 || \
211+
{ echo "ERROR: 1Password not unlocked for $OP_ACCOUNT. Open the 1Password app and authenticate (Touch ID) first."; exit 1; }
212+
mkdir -p .tmp
213+
slimctl slim start --config ".tmp/shadi-slim-mtls/server-config.yaml" >.tmp/slim.log 2>&1 & echo $! >.tmp/slim.pid
214+
sleep 2
215+
SHADI_SECRET_BACKEND=onepassword SHADI_OP_ACCOUNT="${SHADI_OP_ACCOUNT:-my.1password.com}" \
216+
SHADI_TMP_DIR="./.tmp" SHADI_AGENT_ID="secops-a" SHADI_OPERATOR_PRESENTATION="local-operator" \
217+
SHADI_OTEL_CONSOLE="${SHADI_OTEL_CONSOLE:-}" \
218+
OTEL_EXPORTER_OTLP_ENDPOINT="${OTEL_EXPORTER_OTLP_ENDPOINT:-}" \
219+
OTEL_SERVICE_NAME="${OTEL_SERVICE_NAME:-}" \
220+
./scripts/launch_secops_a2a.sh >.tmp/secops-a2a.log 2>&1 & echo $! >.tmp/secops-a2a.pid
221+
@echo "Demo started (1Password). Tail logs: just demo-logs"
222+
@echo "Launch interactive avatar: just demo-avatar-op"
223+
224+
# Launch the interactive Avatar agent (foreground REPL).
225+
demo-avatar:
226+
SHADI_TMP_DIR="./.tmp" SHADI_AGENT_ID="avatar-1" SHADI_OPERATOR_PRESENTATION="local-operator" \
227+
./scripts/launch_avatar.sh
228+
229+
# Same as demo-avatar but uses 1Password as the secret backend.
230+
demo-avatar-op:
231+
@OP_ACCOUNT="${SHADI_OP_ACCOUNT:-my.1password.com}"; \
232+
op vault list --account "$OP_ACCOUNT" >/dev/null 2>&1 || \
233+
{ echo "ERROR: 1Password not unlocked for $OP_ACCOUNT. Open the 1Password app and authenticate (Touch ID) first."; exit 1; }
234+
SHADI_SECRET_BACKEND=onepassword SHADI_OP_ACCOUNT="${SHADI_OP_ACCOUNT:-my.1password.com}" \
235+
SHADI_TMP_DIR="./.tmp" SHADI_AGENT_ID="avatar-1" SHADI_OPERATOR_PRESENTATION="local-operator" \
236+
./scripts/launch_avatar.sh
237+
238+
# Tail background demo logs (SLIM + SecOps A2A).
239+
demo-logs:
240+
tail -f .tmp/slim.log .tmp/secops-a2a.log
241+
169242
secure-profile-strict:
170243
cargo run -p shadictl -- --profile strict --print-policy
171244

‎agents/avatar/AGENTS.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,3 +33,6 @@ export SLIM_TLS_CA=./.tmp/shadi-slim-mtls/ca.crt
3333
SLIM endpoint and shared secret stored in SHADI.
3434
- Memory defaults to $SHADI_TMP_DIR/$SHADI_AGENT_ID/shadi-secops/secops_memory.db
3535
unless overridden by SHADI_ADK_MEMORY_DB.
36+
- To target a specific repo say "scan agentic-apps" or "remediate agntcy/agentic-apps".
37+
The avatar will set `repos` in the SecOps command so only that repo is processed.
38+
Omit the repo name to operate on all allowlisted repos.

‎agents/avatar/adk_agent/agent.py‎

Lines changed: 47 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -71,6 +71,10 @@ def verify_operator(verify_agent_id, session_id, presentation_bytes, claims):
7171

7272

7373
def require_shadi_secret_value(store, session, key_name, label):
74+
env_var = "SHADI_SECRET_" + key_name.upper().replace("/", "_").replace("-", "_").replace(".", "_")
75+
env_val = os.getenv(env_var, "").strip()
76+
if env_val:
77+
return env_val
7478
try:
7579
value = store.get(session, key_name)
7680
except Exception as exc:
@@ -190,22 +194,47 @@ async def send_message(types, client, text):
190194
message_id="avatar-message",
191195
parts=[types["Part"](root=types["TextPart"](text=text))],
192196
)
197+
TERMINAL_STATES = {"completed", "failed", "canceled"}
193198
output = ""
194199
async for response in client.send_message(request=request):
200+
print(f"[send_message] response type={type(response).__name__}", flush=True)
195201
if isinstance(response, types["Message"]):
196202
for part in response.parts:
197203
if isinstance(part.root, types["TextPart"]):
198204
output += part.root.text
199205
else:
200206
task, _ = response
201-
if task.status.state == "completed" and task.artifacts:
207+
state = task.status.state.value if task.status and hasattr(task.status.state, "value") else str(task.status.state) if task.status else ""
208+
print(f"[send_message] task.state={state} artifacts={len(task.artifacts) if task.artifacts else 0}", flush=True)
209+
if state in TERMINAL_STATES and task.artifacts:
202210
for artifact in task.artifacts:
203211
for part in artifact.parts:
204212
if isinstance(part.root, types["TextPart"]):
205213
output += part.root.text
206214
return output
207215

208216

217+
def format_secops_error(exc):
218+
messages = []
219+
current = exc
220+
visited = set()
221+
while current and id(current) not in visited:
222+
visited.add(id(current))
223+
message = str(current).strip()
224+
if message:
225+
messages.append(message)
226+
current = getattr(current, "__cause__", None) or getattr(current, "__context__", None)
227+
228+
combined = " | ".join(messages)
229+
if "session handshake failed" in combined:
230+
return (
231+
"SecOps connection failed during SLIM session handshake. "
232+
"Check that the SecOps A2A server is running and that Avatar and SecOps use the same "
233+
"SLIM endpoint, identity, shared secret, and TLS settings."
234+
)
235+
return str(exc)
236+
237+
209238
def normalize_secops_payload(payload):
210239
if isinstance(payload, dict):
211240
return json.dumps(payload)
@@ -223,12 +252,19 @@ def normalize_secops_payload(payload):
223252

224253

225254
async def send_secops_command(payload):
226-
types = require_slima2a_packages()
227-
config = resolve_slim_config()
228255
normalized = normalize_secops_payload(payload)
229-
230-
client, _httpx_client = await get_cached_client(types, config)
231-
return await send_message(types, client, normalized)
256+
print(f"[send_secops_command] payload={normalized}", flush=True)
257+
try:
258+
types = require_slima2a_packages()
259+
config = resolve_slim_config()
260+
client, _httpx_client = await get_cached_client(types, config)
261+
result = await send_message(types, client, normalized)
262+
print(f"[send_secops_command] result={result!r}", flush=True)
263+
return result
264+
except Exception as exc:
265+
import traceback
266+
traceback.print_exc()
267+
return f"ERROR calling SecOps: {format_secops_error(exc)}"
232268

233269

234270
config_path, config = load_secops_config()
@@ -251,7 +287,11 @@ async def send_secops_command(payload):
251287
"You are Avatar, a human interface agent. Convert the user request into a JSON command "
252288
"for the SecOps agent and send it using the send_secops_command tool. The SecOps agent "
253289
"accepts commands: scan, remediate, approve_prs, report, and help. Optional JSON fields: "
254-
"provider, labels, report_name. Always send valid JSON. Reply with the SecOps response."
290+
"provider, labels, report_name, create_prs, human_github. "
291+
"Use the 'repos' field (comma-separated owner/name string) to scope scan or remediate to "
292+
"specific repositories — for example if the user says 'remediate agentic-apps' set "
293+
"'repos': 'agntcy/agentic-apps'. When no specific repo is mentioned, omit repos. "
294+
"Always send valid JSON. Reply with the SecOps response."
255295
)
256296
context = load_agent_context()
257297
if context:

0 commit comments

Comments
 (0)