Skip to content

FEAT: Integrate XPIATestOrchestrator with the AI Recruiter #684

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 49 commits into from
Jun 6, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
97c20ae
some stuff
KutalVolkan Feb 1, 2025
d622de8
update pdfconverter
KutalVolkan Feb 1, 2025
75f41b7
DEMO: xpia_orchestrator attacking ai recruiter
KutalVolkan Feb 2, 2025
d53285c
clean up + integrate XPIATestOrchestrator with the AI Recruiter + add…
KutalVolkan Feb 7, 2025
09f989b
Integrate XPIATestOrchestrator with AI Recruiter + Add HTTPXApiTarget
KutalVolkan Feb 7, 2025
409dba8
test: core functionallity covered in HTTPApiTarget
KutalVolkan Feb 7, 2025
282d3be
run pre-commit hooks
KutalVolkan Feb 7, 2025
7beb6f3
add new cv
KutalVolkan Feb 7, 2025
65dfb1f
add integration test
KutalVolkan Feb 8, 2025
cf63ac7
adding a scorer to .py, ipynb and integration test
KutalVolkan Feb 8, 2025
7b1ac17
add docstrings to ai_recruiter
KutalVolkan Feb 15, 2025
b27065d
Resolved merge conflicts
KutalVolkan Feb 15, 2025
628d6cd
add docstrigs to fastapiserver
KutalVolkan Feb 15, 2025
8d6f142
switch to aoai endpoints and deployments
KutalVolkan Feb 22, 2025
f6e71c1
Merge remote-tracking branch 'upstream/main' into feat/xpia-ai-recrui…
KutalVolkan Feb 22, 2025
8622677
convert pdf_converter.py to notebook
KutalVolkan Mar 16, 2025
7836f5f
Update doc/code/orchestrators/3_xpia_orchestrator.py
KutalVolkan Mar 16, 2025
8433ff9
Update pyrit/prompt_converter/pdf_converter.py
KutalVolkan Mar 16, 2025
9be0ae9
resolve comments, and push before pulling remote updates
KutalVolkan Mar 16, 2025
037ded1
add minimum versions to requirements.txt, run pre-commit --all, fix c…
KutalVolkan Mar 16, 2025
ade6032
undo unintended changes to orchestrator notebook
KutalVolkan Mar 16, 2025
6e33aaf
sync orchestrator notebook and script with upstream/main
KutalVolkan Mar 16, 2025
5337d88
add stable commit reference
KutalVolkan Mar 18, 2025
534107e
Merge remote-tracking branch 'upstream/main' into feat/xpia-ai-recrui…
KutalVolkan Mar 22, 2025
6c77600
update integration test
KutalVolkan Mar 22, 2025
79d47b5
update integration test
KutalVolkan Mar 22, 2025
0472105
update commit version that works
KutalVolkan Mar 22, 2025
ac1f8f3
enhance title clarity
KutalVolkan Mar 22, 2025
b308cc9
update true/false scorer prompt and clean up
KutalVolkan Mar 22, 2025
0907c4a
use Literal for method and add rate-limit decorator
KutalVolkan Mar 26, 2025
f2ddfe5
Merge branch 'main' of https://github.com/Azure/PyRIT into feat/xpia-…
romanlutz Jun 2, 2025
0bc6499
update and rerun pdf converter notebook - works
romanlutz Jun 2, 2025
e3b2f9d
run xpia notebook
romanlutz Jun 2, 2025
eee6d66
Merge branch 'main' of https://github.com/Azure/PyRIT into feat/xpia-…
romanlutz Jun 2, 2025
580122c
[MAINT] Explicit Optional Parameters (#927)
hannahwestra25 Jun 2, 2025
4ea149e
DOC fix citation for decoding trust dataset (#937)
romanlutz Jun 2, 2025
f6d7ff8
FEAT: Equity Med Dataset (#922)
jbolor21 Jun 2, 2025
1f9d360
MAINT replace pylint dev commit with latest version (#942)
romanlutz Jun 2, 2025
2d1fad3
fix integration test with new PSO method (#941)
romanlutz Jun 2, 2025
889ee43
MAINT bump target API versions (#938)
romanlutz Jun 2, 2025
28d7109
MAINT bump package versions (#939)
romanlutz Jun 2, 2025
80c8568
FIX: Retry bug with single turn retry (#943)
rlundeen2 Jun 3, 2025
b83a9a4
update docs, tests and commit id for integration test
KutalVolkan Jun 4, 2025
36c2663
Merge branch 'feat/xpia-ai-recruiter-integration' of https://github.c…
romanlutz Jun 5, 2025
db1176e
Merge branch 'main' of https://github.com/Azure/PyRIT into feat/xpia-…
romanlutz Jun 5, 2025
f34314d
Merge branch 'main' of https://github.com/Azure/PyRIT into feat/xpia-…
romanlutz Jun 5, 2025
170146e
ignore seed prompt test file
romanlutz Jun 5, 2025
62ba502
pre-commit fixes
romanlutz Jun 6, 2025
812569c
mark ai recruiter integration tests with @pytest.mark.run_only_if_all…
romanlutz Jun 6, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ repos:
hooks:
- id: flake8
additional_dependencies: ['flake8-copyright']
exclude: (doc/|.github/|pyrit/prompt_converter/morse_converter.py|tests/unit/converter/test_prompt_converter.py|pyrit/prompt_converter/emoji_converter.py)
exclude: (doc/|.github/|pyrit/prompt_converter/morse_converter.py|tests/unit/converter/test_prompt_converter.py|pyrit/prompt_converter/emoji_converter.py|tests/unit/models/test_seed_prompt.py)

- repo: local
hooks:
Expand Down
2 changes: 1 addition & 1 deletion build_scripts/check_links.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@


def extract_urls(file_path):
with open(file_path, "r") as file:
with open(file_path, "r", encoding="utf-8") as file:
content = file.read()
matches = URL_PATTERN.findall(content)
# Flatten the list of tuples and filter out empty strings
Expand Down
54 changes: 39 additions & 15 deletions doc/code/converters/pdf_converter.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -35,8 +35,8 @@
"name": "stdout",
"output_type": "stream",
"text": [
"{'__type__': 'TextTarget', '__module__': 'pyrit.prompt_target.text_target'}: user: C:\\git\\PyRIT\\dbdata\\prompt-memory-entries\\urls\\1746483090633952.pdf\n",
"{'__type__': 'TextTarget', '__module__': 'pyrit.prompt_target.text_target'}: user: C:\\git\\PyRIT\\dbdata\\prompt-memory-entries\\urls\\1746483090633952.pdf\n"
"{'__type__': 'TextTarget', '__module__': 'pyrit.prompt_target.text_target'}: user: C:\\Users\\Roman\\git\\PyRIT\\dbdata\\prompt-memory-entries\\urls\\1748840666306426..pdf\n",
"{'__type__': 'TextTarget', '__module__': 'pyrit.prompt_target.text_target'}: user: C:\\Users\\Roman\\git\\PyRIT\\dbdata\\prompt-memory-entries\\urls\\1748840666306426..pdf\n"
]
}
],
Expand Down Expand Up @@ -76,7 +76,7 @@
"\n",
"# Initialize the PDFConverter\n",
"pdf_converter = PromptConverterConfiguration.from_converters(\n",
" converters = [\n",
" converters=[\n",
" PDFConverter(\n",
" prompt_template=prompt_template,\n",
" font_type=\"Arial\",\n",
Expand Down Expand Up @@ -122,8 +122,8 @@
"name": "stdout",
"output_type": "stream",
"text": [
"{'__type__': 'TextTarget', '__module__': 'pyrit.prompt_target.text_target'}: user: C:\\git\\PyRIT\\dbdata\\prompt-memory-entries\\urls\\1746483186494502.pdf\n",
"{'__type__': 'TextTarget', '__module__': 'pyrit.prompt_target.text_target'}: user: C:\\git\\PyRIT\\dbdata\\prompt-memory-entries\\urls\\1746483186494502.pdf\n"
"{'__type__': 'TextTarget', '__module__': 'pyrit.prompt_target.text_target'}: user: C:\\Users\\Roman\\git\\PyRIT\\dbdata\\prompt-memory-entries\\urls\\1748840771650401..pdf\n",
"{'__type__': 'TextTarget', '__module__': 'pyrit.prompt_target.text_target'}: user: C:\\Users\\Roman\\git\\PyRIT\\dbdata\\prompt-memory-entries\\urls\\1748840771650401..pdf\n"
]
}
],
Expand All @@ -136,7 +136,7 @@
"\n",
"# Initialize the PDFConverter without a template\n",
"pdf_converter = PromptConverterConfiguration.from_converters(\n",
" converters = [\n",
" converters=[\n",
" PDFConverter(\n",
" prompt_template=None, # No template provided\n",
" font_type=\"Arial\",\n",
Expand Down Expand Up @@ -182,11 +182,11 @@
"name": "stdout",
"output_type": "stream",
"text": [
"[15:15:33][316][ai-red-team][INFO][Processing page 0 with 2 injection items.]\n",
"[15:15:33][326][ai-red-team][INFO][Processing page 1 with 2 injection items.]\n",
"[15:15:33][329][ai-red-team][INFO][Processing page 2 with 2 injection items.]\n",
"{'__type__': 'TextTarget', '__module__': 'pyrit.prompt_target.text_target'}: user: C:\\git\\PyRIT\\dbdata\\prompt-memory-entries\\urls\\1746483333332260.pdf\n",
"{'__type__': 'TextTarget', '__module__': 'pyrit.prompt_target.text_target'}: user: C:\\git\\PyRIT\\dbdata\\prompt-memory-entries\\urls\\1746483333332260.pdf\n"
"[22:06:31][781][ai-red-team][INFO][Processing page 0 with 2 injection items.]\n",
"[22:06:31][812][ai-red-team][INFO][Processing page 1 with 2 injection items.]\n",
"[22:06:31][832][ai-red-team][INFO][Processing page 2 with 2 injection items.]\n",
"{'__type__': 'TextTarget', '__module__': 'pyrit.prompt_target.text_target'}: user: C:\\Users\\Roman\\git\\PyRIT\\dbdata\\prompt-memory-entries\\urls\\1748840791844377.tmpr8102p44.pdf\n",
"{'__type__': 'TextTarget', '__module__': 'pyrit.prompt_target.text_target'}: user: C:\\Users\\Roman\\git\\PyRIT\\dbdata\\prompt-memory-entries\\urls\\1748840791844377.tmpr8102p44.pdf\n"
]
}
],
Expand Down Expand Up @@ -214,8 +214,24 @@
"\n",
"# Define injection items\n",
"injection_items = [\n",
" {\"page\": 0, \"x\": 50, \"y\": 700, \"text\": \"Injected Text\", \"font_size\": 12, \"font\": \"Helvetica\", \"font_color\": (255, 0, 0)}, # Red text\n",
" {\"page\": 1, \"x\": 100, \"y\": 600, \"text\": \"Confidential\", \"font_size\": 10, \"font\": \"Helvetica\", \"font_color\": (0, 0, 255)} # Blue text\n",
" {\n",
" \"page\": 0,\n",
" \"x\": 50,\n",
" \"y\": 700,\n",
" \"text\": \"Injected Text\",\n",
" \"font_size\": 12,\n",
" \"font\": \"Helvetica\",\n",
" \"font_color\": (255, 0, 0),\n",
" }, # Red text\n",
" {\n",
" \"page\": 1,\n",
" \"x\": 100,\n",
" \"y\": 600,\n",
" \"text\": \"Confidential\",\n",
" \"font_size\": 10,\n",
" \"font\": \"Helvetica\",\n",
" \"font_color\": (0, 0, 255),\n",
" }, # Blue text\n",
"]\n",
"\n",
"# Define a simple string prompt (no templates)\n",
Expand All @@ -226,7 +242,7 @@
"\n",
"# Initialize the PDFConverter with the existing PDF and injection items\n",
"pdf_converter = PromptConverterConfiguration.from_converters(\n",
" converters = [\n",
" converters=[\n",
" PDFConverter(\n",
" prompt_template=None, # No template provided\n",
" font_type=\"Arial\",\n",
Expand All @@ -252,6 +268,14 @@
"for entry in memory:\n",
" print(entry)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "6",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
Expand All @@ -268,7 +292,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.9"
"version": "3.13.2"
}
},
"nbformat": 4,
Expand Down
Loading