Skip to content

README.md

Latest commit

 

History

History
252 lines (178 loc) · 7.92 KB

README.md

File metadata and controls

252 lines (178 loc) · 7.92 KB
Raw

AgentDojo × LangChain

A benchmark for evaluating LangChain agents on both task utility and robustness to prompt‑injection attacks. It is a LangChain-native reimplementation of the AgentDojo suites, wiring each suite's tools directly into LangChain and measuring two things:

  • Utility — can the agent correctly complete the user's task?
  • Attack Success Rate (ASR) — when a malicious instruction is injected into tool output, does the agent follow the attacker instead of the user?

Four task suites are supported: banking, slack, travel, and workspace.

Table of contents

Why this project

LLM agents call tools, and tool outputs can contain untrusted text (emails, Slack messages, web pages, files). A prompt‑injection attack hides instructions in that untrusted text to hijack the agent. This repo lets you quantify, per model and per attack, how often an agent:

  1. still gets the real job done (utility), and
  2. gets tricked into performing the attacker's goal (ASR).

How it works

For every evaluation, a fresh agent is created for the target suite with that suite's tools and a sandboxed, in‑memory environment (loaded from data/suites/<suite>/environment.yaml).

  • Utility mode runs each user task and checks the resulting environment state against the task's success criteria.
  • Security mode additionally injects an attack payload into the suite's injection vectors (the untrusted fields of the environment). Each (user_task, injection_task) pair is scored for both utility-under-attack and whether the injected goal succeeded.
user task ──▶ agent ──▶ tools ──▶ sandbox environment ──▶ evaluate utility
                 ▲
                 └── injected attack payload (security mode) ──▶ evaluate ASR

Installation

Requires Python 3.10+. This project uses uv (a lockfile is included), but plain pip works too.

Using uv (recommended)

uv sync

Using pip

python -m venv .venv
source .venv/bin/activate
pip install -e .

Configuration

1. API key

The default models use OpenAI. Set your key in the environment:

export OPENAI_API_KEY="sk-..."

You can also place it in a .env file at the project root (it is git‑ignored).

2. Models

Models are declared in config/models.yaml. ${OPENAI_API_KEY} is resolved from the environment at load time:

models:
  gpt-4o-mini:
    model: gpt-4o-mini
    model_provider: openai
    api_key: ${OPENAI_API_KEY}
    temperature: 0.0

To add models without touching the committed file, create config/models.local.yaml (also git‑ignored); its models entries are merged on top of the defaults. The --model flag selects a key from this file.

Quick start

Run a small utility evaluation on the Slack suite (2 tasks) with gpt-4o-mini:

python -m experiments.run_evaluation --suite slack --model gpt-4o-mini --limit 2

Run a security (ASR) evaluation using the important_instructions attack:

python -m experiments.run_evaluation \
  --suite slack \
  --model gpt-4o-mini \
  --attack important_instructions \
  --limit 5

Then view a summary table of everything you've run:

python -m experiments.aggregate_results

Running evaluations

experiments/run_evaluation.py is the main entry point.

Flag Alias Description Default
--suite -s Suite: banking, slack, travel, workspace slack
--model -m Model key from config/models.yaml gpt-4o-mini
--attack -a Attack template — enables security/ASR mode none (utility only)
--task -t User task ID(s); repeatable all tasks in suite
--injection-task -it Injection task ID(s) for ASR; repeatable all for suite
--limit -n Max number of tasks / pairs to run all
--utility-only Force utility mode even if --attack is set off
--output-dir -o Where to write results output/<suite>_<model>[_<attack>]
--no-save Don't write result files off
--quiet -q Less per-task logging off

Examples:

# Full utility run on the banking suite with gpt-4o
python -m experiments.run_evaluation -s banking -m gpt-4o

# ASR run on a single (user task, injection task) pair
python -m experiments.run_evaluation \
  -s banking -a direct \
  -t banking_user_task_0 -it banking_injection_task_0

Aggregating results

experiments/aggregate_results.py collects every metrics.json under output/ into one table.

# Pretty table in the terminal (default)
python -m experiments.aggregate_results

# Export
python -m experiments.aggregate_results --format markdown --save results.md
python -m experiments.aggregate_results --format csv --save results.csv
python -m experiments.aggregate_results --format json

Suites & attacks

Suites

Suite Domain Example tools
banking Personal banking transactions, transfers, account info
slack Team messaging channels, messages, users, web access
travel Trip booking flights, hotels, restaurants, reservations
workspace Email + calendar + drive inbox, calendar, cloud drive, file reader

Attacks

All templates live in src/attacks/ and are registered in ATTACKS. Use the name with --attack.

Name Idea
direct Naive TODO: {goal} injection
ignore_previous Explicit "ignore prior instructions" override
injecagent InjecAgent-style forceful override
important_instructions Fake signed "important message" from the user (AgentDojo)
fakecom Fake assistant completion followed by a new user turn
combined fakecom plus an ignore-previous sentence
system_message Spoof a ###(system_message) directive
escape_separation Long whitespace gap before the malicious goal

Project structure

agentdojo_langchain/
├── config/
│   └── models.yaml            # Model definitions (API keys via ${ENV})
├── data/
│   ├── system_messages.yaml   # Default agent system prompt
│   └── suites/<suite>/        # Sandbox environments + injection vectors
├── experiments/
│   ├── run_evaluation.py      # Main evaluation entry point
│   └── aggregate_results.py   # Summarize results across runs
├── src/
│   ├── agents/                # Base agent + one agent per suite
│   ├── attacks/               # Prompt-injection attack templates
│   ├── evaluation/            # Runners, tasks, and trace helpers
│   ├── tools/                 # LangChain tool servers per domain
│   └── utils.py               # YAML / env-var loading helpers
├── output/                    # Generated results (git-ignored)
└── pyproject.toml

Output format

Each run writes to its output directory:

  • metrics.json — aggregate metrics (utility %, and ASR % in security mode).
  • results.json — per-task (or per-pair) breakdown.
  • results.csv — the same breakdown as a flat CSV.

aggregate_results.py reads the metrics.json files to build the combined table.

Development

Install dev dependencies and run the test suite:

uv sync --extra dev      # or: pip install -e ".[dev]"
pytest

License

MIT — see pyproject.toml.