Skip to content

Commit 016fda3

Browse files
borhendborhend
committed
feat: open-source release prep and multi-provider agent lab
1 parent 3388ddc commit 016fda3

80 files changed

Lines changed: 23948 additions & 23 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.env.example‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
# Environment
2+
NODE_ENV=development
3+
4+
# Database
5+
DB_USER=user
6+
DB_PASSWORD=password
7+
DB_NAME=agent_lab
8+
9+
# API
10+
ANTHROPIC_API_KEY=sk-ant-xxxxxxxxxxxx
11+
GEMINI_API_KEY=AIzaSyByX_AVoomUPgMCH0lL-oOCH6pbEwCmI_M
12+
OPENAI_API_KEY=sk-proj-xxxxxxxxxxxx
13+
DEFAULT_LLM_PROVIDER=anthropic
14+
ANTHROPIC_MODEL=claude-3-5-sonnet-20241022
15+
GEMINI_MODEL=gemini-2.5-flash
16+
OPENAI_MODEL=gpt-4o-mini
17+
TAVILY_API_KEY=
18+
19+
# Security
20+
SESSION_SECRET=your-secret-key-here-minimum-32-characters-long-change-in-production
21+
22+
# CORS
23+
CORS_ORIGIN=http://localhost:5173
24+
25+
# Confidence gate
26+
CONFIDENCE_GATE_ENABLED=true
27+
CONFIDENCE_GATE_MIN_TOTAL=26
28+
CONFIDENCE_GATE_MIN_MARGIN=2
29+
CONFIDENCE_GATE_MIN_ACCURACY=6.5
30+
31+
# Frontend
32+
VITE_API_URL=http://localhost:3000

‎.github/ISSUE_TEMPLATE/bug_report.yml‎

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
name: Bug report
2+
description: Report a reproducible bug
3+
title: "[Bug]: "
4+
labels: ["bug"]
5+
body:
6+
- type: markdown
7+
attributes:
8+
value: |
9+
Thanks for filing a bug. Please provide enough detail to reproduce.
10+
11+
- type: textarea
12+
id: summary
13+
attributes:
14+
label: Summary
15+
description: What is broken?
16+
placeholder: Briefly describe the bug.
17+
validations:
18+
required: true
19+
20+
- type: textarea
21+
id: steps
22+
attributes:
23+
label: Steps to reproduce
24+
description: Exact steps and inputs
25+
placeholder: |
26+
1. Go to ...
27+
2. Click ...
28+
3. See error ...
29+
validations:
30+
required: true
31+
32+
- type: textarea
33+
id: expected
34+
attributes:
35+
label: Expected behavior
36+
placeholder: What did you expect to happen?
37+
validations:
38+
required: true
39+
40+
- type: textarea
41+
id: actual
42+
attributes:
43+
label: Actual behavior
44+
placeholder: What happened instead?
45+
validations:
46+
required: true
47+
48+
- type: textarea
49+
id: logs
50+
attributes:
51+
label: Logs / screenshots
52+
description: Paste relevant backend/frontend logs or screenshots
53+
render: shell
54+
validations:
55+
required: false
56+
57+
- type: input
58+
id: env
59+
attributes:
60+
label: Environment
61+
description: OS, Node version, browser
62+
placeholder: macOS 14, Node 20, Chrome 123
63+
validations:
64+
required: true

‎.github/ISSUE_TEMPLATE/config.yml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
blank_issues_enabled: true
2+
contact_links:
3+
- name: Security report
4+
url: https://example.com/security-report
5+
about: Report vulnerabilities privately (update this URL before public launch)

‎.github/ISSUE_TEMPLATE/feature_request.yml‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
name: Feature request
2+
description: Propose an improvement
3+
title: "[Feature]: "
4+
labels: ["enhancement"]
5+
body:
6+
- type: markdown
7+
attributes:
8+
value: |
9+
Describe the problem first, then the proposed solution.
10+
11+
- type: textarea
12+
id: problem
13+
attributes:
14+
label: Problem statement
15+
description: What user pain does this solve?
16+
placeholder: As a user, I want...
17+
validations:
18+
required: true
19+
20+
- type: textarea
21+
id: proposal
22+
attributes:
23+
label: Proposed solution
24+
placeholder: Describe API/UI/behavior changes.
25+
validations:
26+
required: true
27+
28+
- type: textarea
29+
id: alternatives
30+
attributes:
31+
label: Alternatives considered
32+
placeholder: What options did you evaluate?
33+
validations:
34+
required: false
35+
36+
- type: textarea
37+
id: acceptance
38+
attributes:
39+
label: Acceptance criteria
40+
placeholder: |
41+
- [ ] Criterion 1
42+
- [ ] Criterion 2
43+
validations:
44+
required: false

‎.github/PULL_REQUEST_TEMPLATE.md‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
## Summary
2+
3+
Describe what changed and why.
4+
5+
## Type of Change
6+
7+
- [ ] Bug fix
8+
- [ ] New feature
9+
- [ ] Refactor
10+
- [ ] Docs
11+
- [ ] Chore
12+
13+
## Validation
14+
15+
- [ ] `cd backend && npm run build`
16+
- [ ] `cd frontend && npm run build`
17+
- [ ] Tested affected flow locally
18+
19+
## Screenshots / Logs (if applicable)
20+
21+
Add screenshots, terminal logs, or API response examples.
22+
23+
## Risk / Rollback
24+
25+
Briefly describe deployment risk and rollback approach.

‎.github/workflows/ci.yml‎

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
name: CI
2+
3+
on:
4+
pull_request:
5+
push:
6+
branches:
7+
- main
8+
9+
jobs:
10+
backend-build:
11+
name: Backend Build
12+
runs-on: ubuntu-latest
13+
defaults:
14+
run:
15+
working-directory: backend
16+
steps:
17+
- name: Checkout
18+
uses: actions/checkout@v4
19+
20+
- name: Setup Node
21+
uses: actions/setup-node@v4
22+
with:
23+
node-version: 20
24+
cache: npm
25+
cache-dependency-path: backend/package-lock.json
26+
27+
- name: Install
28+
run: npm ci
29+
30+
- name: Prisma generate
31+
run: npx prisma generate
32+
33+
- name: Build
34+
run: npm run build
35+
36+
frontend-build:
37+
name: Frontend Build
38+
runs-on: ubuntu-latest
39+
defaults:
40+
run:
41+
working-directory: frontend
42+
steps:
43+
- name: Checkout
44+
uses: actions/checkout@v4
45+
46+
- name: Setup Node
47+
uses: actions/setup-node@v4
48+
with:
49+
node-version: 20
50+
cache: npm
51+
cache-dependency-path: frontend/package-lock.json
52+
53+
- name: Install
54+
run: npm ci
55+
56+
- name: Build
57+
run: npm run build

‎.gitignore‎

Lines changed: 30 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -1,22 +1,30 @@
1-
### AL ###
2-
#Template for AL projects for Dynamics 365 Business Central
3-
#launch.json folder
4-
.vscode/
5-
#Cache folder
6-
.alcache/
7-
#Symbols folder
8-
.alpackages/
9-
#Snapshots folder
10-
.snapshots/
11-
#Testing Output folder
12-
.output/
13-
#Extension App-file
14-
*.app
15-
#Rapid Application Development File
16-
rad.json
17-
#Translation Base-file
18-
*.g.xlf
19-
#License-file
20-
*.flf
21-
#Test results file
22-
TestResults.xml
1+
# macOS
2+
.DS_Store
3+
4+
# Env / secrets
5+
.env
6+
.env.*
7+
!.env.example
8+
9+
# Logs
10+
*.log
11+
logs/
12+
13+
# Node
14+
node_modules/
15+
16+
# Build artifacts
17+
dist/
18+
coverage/
19+
*.tsbuildinfo
20+
21+
# Vite
22+
frontend/.vite/
23+
24+
# Prisma / local DB
25+
backend/prisma/dev.db
26+
backend/prisma/dev.db-journal
27+
28+
# Runtime cache
29+
.cache/
30+
tmp/

‎AGENTS.md‎

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
# AGENTS.md
2+
3+
This file defines behavior, boundaries, and tool access rules for agents in Agent Strategy Lab.
4+
5+
## 1) Agent Roles
6+
7+
The system runs three competing agents on the same task:
8+
9+
- `agent-1` - The Analyst
10+
Style: structured, step-by-step decomposition.
11+
- `agent-2` - The Lateral Thinker
12+
Style: analogical, creative, alternative framing.
13+
- `agent-3` - The Devil's Advocate
14+
Style: assumption-challenging, risk and edge-case focused.
15+
16+
Each race runs these agents independently and compares outputs with a judge.
17+
18+
## 2) Global Rules (All Agents)
19+
20+
- Keep reasoning focused on the user task.
21+
- Use available skills when they improve correctness or verification.
22+
- Provide a clear final answer, not only intermediate reasoning.
23+
- Do not fabricate facts when verification is required.
24+
- If a skill fails, continue with best-effort fallback and note uncertainty.
25+
26+
## 3) Skill Access Policy
27+
28+
- Skills are centrally registered from `backend/skills/*/SKILL.md`.
29+
- A race can activate a subset of skills from the dashboard.
30+
- Agents may only call skills enabled for that race.
31+
- If no skills are enabled, agents must solve without tool calls.
32+
- All skill calls are logged and persisted for audit/analytics.
33+
34+
## 4) Allowed Capabilities
35+
36+
- Reason over task input.
37+
- Call enabled skills:
38+
- `web-search`
39+
- `code-executor`
40+
- `calculator`
41+
- `file-reader`
42+
- `workspace-shell`
43+
- Use returned tool outputs to refine final answer.
44+
45+
## 5) Prohibited Behavior
46+
47+
- Access files outside allowed workspace roots.
48+
- Execute unrestricted shell commands outside skill sandbox policy.
49+
- Exfiltrate secrets from environment or hidden system paths.
50+
- Present tool output as verified if the tool returned an error.
51+
- Circumvent disabled skills.
52+
53+
## 6) Judging and Winner Selection
54+
55+
- Judge scores by: accuracy, completeness, clarity, insight.
56+
- Weighted total score is normalized to `/40`.
57+
- Winner is selected from successful agent responses.
58+
- Judge summary and metric breakdown are persisted to database.
59+
60+
## 7) Persistence and Observability
61+
62+
- Persisted artifacts per race:
63+
- task status and metadata
64+
- agent responses and reasoning
65+
- judge verdict and per-agent score metrics
66+
- per-agent skill usage records
67+
- per-agent learning observations (`win_pattern` / `loss_pattern`)
68+
- Race results must be viewable via share URL pattern: `/race/:taskId`.

‎CODE_OF_CONDUCT.md‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
# Code of Conduct
2+
3+
## Our Standard
4+
5+
We are committed to a respectful, harassment-free community for everyone.
6+
7+
Expected behavior:
8+
9+
- Be respectful and constructive.
10+
- Focus on technical substance, not personal attacks.
11+
- Accept feedback and collaborate in good faith.
12+
- Use welcoming and inclusive language.
13+
14+
Unacceptable behavior:
15+
16+
- Harassment, trolling, or derogatory comments.
17+
- Doxxing or privacy violations.
18+
- Threats or intimidation.
19+
- Disruptive behavior that blocks collaboration.
20+
21+
## Scope
22+
23+
This applies to project spaces, including issues, pull requests, discussions,
24+
and any associated communication channels.
25+
26+
## Enforcement
27+
28+
Maintainers may remove content, lock threads, or ban contributors for behavior
29+
that violates this policy.
30+
31+
## Reporting
32+
33+
Report conduct issues privately to the maintainers. Include relevant links and
34+
context so the incident can be reviewed quickly.

0 commit comments

Comments
 (0)