Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
125 commits
Select commit Hold shift + click to select a range
08677ff
Running into concurrency issues with claude code, using the concurren…
LastS3cond Mar 21, 2026
4ef8b86
Removing excess skill from OfficeQA
LastS3cond Mar 24, 2026
98ae05c
Better error handling/tracing
LastS3cond Mar 24, 2026
d0b473d
Added logic to eval to hold out the test set when given the --held-ou…
LastS3cond Mar 24, 2026
af065bd
Merge pull request #4 from LastS3cond/main
LastS3cond Mar 25, 2026
cb26b0a
feat: Add Opencode SDK support and SEAL-QA evaluation with Gemini
noahpro99 Mar 25, 2026
f1b755d
feat: Add Opencode SDK support, SEAL-QA eval with skills toggle, and …
noahpro99 Mar 25, 2026
50785c5
created single eval entrypoint script
Mar 25, 2026
2a5d41c
Merge branch 'feature/sealqa-eval' into cookies
Kuzoto Mar 25, 2026
32fb1ea
feat: Add Opencode SDK support, SEAL-QA eval with skills toggle, and …
noahpro99 Mar 25, 2026
9cdd75a
fix: Update default train/val ratios to 0.12 in loop script
noahpro99 Mar 25, 2026
bff8f4a
Merge branch 'feature/sealqa-eval' into cookies
Kuzoto Mar 25, 2026
3ecabe8
fix: Update default train/val ratios to 0.12 in eval script
noahpro99 Mar 25, 2026
b4cce8d
Merge branch 'feature/sealqa-eval' into cookies
Kuzoto Mar 25, 2026
710a91e
resolved circular import and updated single eval entry
Kuzoto Mar 25, 2026
f936c23
changed script
Kuzoto Mar 25, 2026
c494ae1
feat: Add Kimi K2.5 model to ARC provider in opencode config
noahpro99 Mar 25, 2026
c9da309
renamed eval script
Kuzoto Mar 25, 2026
54a59dd
opencode tool for web search and fetch
LastS3cond Mar 25, 2026
efba327
feat: Isolated run dirs for eval to avoid skill conflicts between runs
noahpro99 Mar 25, 2026
f3a0ace
Merge pull request #5 from LastS3cond/main
LastS3cond Mar 25, 2026
7ca5c4f
feat: Add run_loop_comb.py for multi-benchmark skill evolution loop
noahpro99 Mar 25, 2026
c7632bd
fix: Update default train/val ratios to 0.13 for 14/14/83 split
noahpro99 Mar 25, 2026
068368b
chore: Remove redundant single-benchmark scripts replaced by comb ver…
noahpro99 Mar 25, 2026
149fd1d
chore: Replace sbatch scripts with two generic ones (eval + loop)
noahpro99 Mar 25, 2026
7c20079
feat: Add sbatch script for vLLM server
noahpro99 Mar 25, 2026
6764259
Merge branch 'feature/sealqa-eval' into cookies
noahpro99 Mar 25, 2026
8904c36
fix: Always copy meta-skills (skill-creator) to run dirs including ba…
noahpro99 Mar 25, 2026
72ecf38
skill-creator keep
noahpro99 Mar 25, 2026
6400e49
seal qa prompt
noahpro99 Mar 26, 2026
4b38d41
added livecodebench prompt
Kuzoto Mar 26, 2026
9c3e99e
fix: Unignore .claude/skills/ so loop can git-add evolved skills
noahpro99 Mar 26, 2026
7a4a18a
fix: Unset CLAUDECODE env var in sbatch scripts to avoid nested sessi…
noahpro99 Mar 26, 2026
981484a
Running into concurrency issues with claude code, using the concurren…
LastS3cond Mar 21, 2026
8ea379c
Removing excess skill from OfficeQA
LastS3cond Mar 24, 2026
9cb2d18
Better error handling/tracing
LastS3cond Mar 24, 2026
9fdc7f0
Added logic to eval to hold out the test set when given the --held-ou…
LastS3cond Mar 24, 2026
e7431d0
opencode tool for web search and fetch
LastS3cond Mar 25, 2026
887aae1
Removing the model choices to allow more flexibility
LastS3cond Mar 25, 2026
7bcdbeb
Changing base prompt for sealQA
LastS3cond Mar 22, 2026
8cea719
Scaffolding for baseline osworld run
LastS3cond Mar 19, 2026
a82d3c9
Changes post-merge + save full conversation
LastS3cond Mar 20, 2026
0ec41df
Adding my fork of OSWorld as a submodule
LastS3cond Mar 26, 2026
e63ace8
Add vms to .gitignore
LastS3cond Mar 26, 2026
ab3938d
Merge https://github.com/salzubi401/deep_agents
LastS3cond Mar 26, 2026
41b3ba9
feat: Loop and eval share isolated .evoskill-runs/<session>/ dirs
noahpro99 Mar 26, 2026
21ac51d
added specialized prompt and provider support to livecodebench agent
Kuzoto Mar 26, 2026
28337ad
Merge remote-tracking branch 'refs/remotes/origin/cookies' into cookies
Kuzoto Mar 26, 2026
9d77f4b
feat: Fully isolate loop in .evoskill-runs/<session>/ dir
noahpro99 Mar 26, 2026
d18f623
fix: Set cwd directly on agent options instead of deepcopy
noahpro99 Mar 26, 2026
5384088
chore: Remove redundant run_eval_livecodebench.py (use run_eval_comb.py)
noahpro99 Mar 26, 2026
16552e2
fix: Replace Docker sandbox with subprocess for LiveCodeBench scorer
noahpro99 Mar 26, 2026
6d7cceb
Merge branch 'cookies' of https://github.com/noahpro99/deep_agents
LastS3cond Mar 26, 2026
3923e62
fixed timeouts
Kuzoto Mar 26, 2026
d3b0b30
Add Together AI provider with GPT-OSS-120B model to opencode.json
noahpro99 Mar 27, 2026
4a6832e
feat: Add --dataset_slice to truncate dataset before train/val/test s…
noahpro99 Mar 27, 2026
54cbd94
fix: Symlink data dirs (treasury docs, DABstep) into run dirs
noahpro99 Mar 27, 2026
c8cb525
Adjusted stratified sampling to take a max_examples var
LastS3cond Mar 27, 2026
02fe9da
Merge branch 'main' of https://github.com/noahpro99/deep_agents
LastS3cond Mar 27, 2026
f321fe8
GDPVal scaffolding
LastS3cond Mar 27, 2026
b587b94
Merge pull request #2 from LastS3cond:main
noahpro99 Mar 27, 2026
132e4a0
chore: Remove redundant run_loop.py (replaced by run_loop_comb.py)
noahpro99 Mar 27, 2026
423ee5d
rename: run_eval_comb.py -> run_eval.py, run_loop_comb.py -> run_loop.py
noahpro99 Mar 27, 2026
ead7f51
fix: Resume retries empty outputs, not just errors
noahpro99 Mar 27, 2026
12f40aa
Temporarily disabling scoring + small changes
LastS3cond Mar 27, 2026
f2f69da
added gepa improvement loop
Kuzoto Mar 27, 2026
21fc5ae
corrected branch selection
Kuzoto Mar 27, 2026
46d4c6a
refactor: Remove include_skills param from prepare_run_dir
noahpro99 Mar 27, 2026
c15486d
fix: Increase output token limits (3072->32768 for OSS, 8192->32768 f…
noahpro99 Mar 27, 2026
4f91231
feat: Add OpenRouter provider for Gemini 3.1 Flash Lite
noahpro99 Mar 27, 2026
4fd795c
fixed provider for arc dspy
Kuzoto Mar 27, 2026
fa19cac
Merge branch 'gepa_test'
Kuzoto Mar 27, 2026
6db40b0
Changes to run the frames dataset
Mar 27, 2026
2b7bf25
fix: Symlink .opencode (custom tools) into run dirs
noahpro99 Mar 27, 2026
3cd7574
chore: Remove redundant run_loop.py (replaced by run_loop_comb.py)
noahpro99 Mar 27, 2026
e527afe
rename: run_eval_comb.py -> run_eval.py, run_loop_comb.py -> run_loop.py
noahpro99 Mar 27, 2026
3d0a2f0
fix: Resume retries empty outputs, not just errors
noahpro99 Mar 27, 2026
d3b22b4
refactor: Remove include_skills param from prepare_run_dir
noahpro99 Mar 27, 2026
692d931
fix: Increase output token limits (3072->32768 for OSS, 8192->32768 f…
noahpro99 Mar 27, 2026
c2f9462
feat: Add OpenRouter provider for Gemini 3.1 Flash Lite
noahpro99 Mar 27, 2026
f8c5cf7
added gepa improvement loop
Kuzoto Mar 27, 2026
0ccc958
corrected branch selection
Kuzoto Mar 27, 2026
ee76357
fixed provider for arc dspy
Kuzoto Mar 27, 2026
09fcc1f
fix: Symlink .opencode (custom tools) into run dirs
noahpro99 Mar 27, 2026
dbd3840
Consolidating num_samples and dataset_slice, adjusting opencode tool
LastS3cond Mar 28, 2026
3c77b44
Merge branch 'main' of https://github.com/noahpro99/deep_agents
LastS3cond Mar 28, 2026
0cbd37a
fix: Never overwrite good results with bad ones on resume
noahpro99 Mar 28, 2026
8357546
fix: Add missing proposer/generator imports to run_loop.py
noahpro99 Mar 28, 2026
a3dc999
feat: Add score field to IndexedEvalResult for caching grader scores
noahpro99 Mar 28, 2026
c6d1a21
fixed gepa_runner and added per session checkpoints, feedback history
Kuzoto Mar 28, 2026
935af75
Merge branch 'gepa-fix'
noahpro99 Mar 28, 2026
4ffcdad
added support for officeqa with opencode and tweaked gepa
Kuzoto Mar 29, 2026
62da02e
fix: export_best_skills reads from session dir, not main repo
noahpro99 Mar 29, 2026
b5664e6
reverted gepa runner back to prompt optimizer
Kuzoto Mar 29, 2026
b8751ba
Frames modifications, structured output fixes
LastS3cond Mar 29, 2026
65d1c26
Merge branch 'main' of https://github.com/noahpro99/deep_agents
LastS3cond Mar 29, 2026
fbdf015
fixed gepaloop and added gepa eval
Kuzoto Mar 30, 2026
73fb11c
Merge remote-tracking branch 'refs/remotes/origin/main'
Kuzoto Mar 30, 2026
98d590f
added arg to disable and enable caching for gepa
Kuzoto Mar 30, 2026
3d4ca8a
fixed running gepa on sealqa
Kuzoto Mar 30, 2026
0697bd2
refactor: Centralize stratified split in split_held_out()
noahpro99 Mar 30, 2026
9d13316
fix: Pass dataset_slice to stratified_split in loop's build_train_val
noahpro99 Mar 30, 2026
5bc48fc
ignore opencode josn
noahpro99 Mar 30, 2026
7578df6
fix: Set cwd on Claude SDK options so eval uses session dir skills
noahpro99 Mar 30, 2026
f1767dd
Merged my code
LastS3cond Mar 30, 2026
35590e7
fix: Scorer uses ARC first (OpenRouter out of credits), remove .lower…
noahpro99 Mar 30, 2026
61ea923
fix: Increase eval timeout from 17 to 40 minutes
noahpro99 Mar 30, 2026
f270761
Merge branch 'main' of https://github.com/noahpro99/deep_agents
LastS3cond Mar 30, 2026
34e1858
changed gepa runner to use opencode.json
Kuzoto Mar 30, 2026
44506d6
eval time out increase
LastS3cond Mar 30, 2026
032d1d1
gepa runner
LastS3cond Mar 30, 2026
5c03035
Revert "fix: Increase eval timeout from 17 to 40 minutes"
noahpro99 Mar 31, 2026
cfe1721
livecodebench prompt
noahpro99 Mar 31, 2026
ccc10c0
added websearch for gepa
Kuzoto Mar 31, 2026
4c964f8
Merge branch 'main' of https://github.com/noahpro99/deep_agents
LastS3cond Mar 31, 2026
e120a84
changed time to 21 minutes + QOL changes
LastS3cond Mar 31, 2026
d5fac33
Adding back opencode formatting just in case
LastS3cond Mar 31, 2026
561080f
QOL changes to correctly identify skills in eval runs
LastS3cond Mar 31, 2026
c3d444e
browse comp
noahpro99 Mar 31, 2026
e0ff5a2
feat: Add BrowseComp benchmark support
noahpro99 Mar 31, 2026
ea22f1f
Merge branch 'main' of https://github.com/noahpro99/deep_agents
LastS3cond Mar 31, 2026
ccf24f7
feat: All loop agents use same model/provider as base agent
noahpro99 Apr 1, 2026
f4a011a
fixed opencode proposer and generator
Kuzoto Apr 1, 2026
619e83a
fixed edge case with skill writing
Kuzoto Apr 1, 2026
4e04fe8
vllm provider + sbatch script
noahpro99 Apr 3, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions .claude/program.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
name: base
parent: null
generation: 0
system_prompt:
type: preset
preset: claude_code
append: You are an expert analyst working at a top office firm. You are given a
question and a set of documents. Your job is to answer the question based on the
documents. You are also given a set of tools that you can use to answer the question.
You can use the tools to read the documents, write the answer, or run code.
allowed_tools:
- Read
- Write
- Bash
- Glob
- Grep
- Edit
- WebFetch
- WebSearch
- TodoWrite
- BashOutput
- Skill
output_format:
type: json_schema
schema:
properties:
final_answer:
title: Final Answer
type: string
reasoning:
title: Reasoning
type: string
required:
- final_answer
- reasoning
title: AgentResponse
type: object
metadata:
created_at: '2026-03-26T03:01:01.482305'
111 changes: 0 additions & 111 deletions .claude/skills/brainstorming/SKILL.md

This file was deleted.

18 changes: 17 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,13 @@ wheels/
treasury_bulletins_parsed/
*.skill
*.log

# Claude Code config (ignore most, but track program state)
.claude/*
!.claude/program.yaml
!.claude/skills/
results/
job-outputs/
DABstep-data/

# Virtual environments
Expand All @@ -23,7 +25,21 @@ DABstep-data/

# Cache
.cache/
*.txt
output.txt
tests/
scratch/

# Vmware for OSWorld
vmware_vm_data/
.vmware_vms

# Ignore loose Python scripts in repo root
/*.py
.evoskill-runs/
*.csv
*.zip

# Relevant data directories that don't need to be in the commit history
data_directories/
opencode.json
archive/
3 changes: 3 additions & 0 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[submodule "OSWorld"]
path = OSWorld
url = https://github.com/LastS3cond/OSWorld.git
Loading