Skip to content

Commit c006389

Browse files
RajjjAryanCopilot
RajjjAryan
and
Copilot
authored
feat: setup script, evaluation walkthrough, 10 portals, batch retry (#33)
- Add setup.sh one-liner that installs all dependencies (#25 replacement) - Add docs/evaluation-walkthrough.md with full A-F scoring example (#7) - Add 10 company portals: Stripe, Datadog, Cloudflare, Figma, Notion, Databricks, Scale AI, Supabase, Dagger, Modal (#2) - Add rate limiting and retry logic to batch mode: 2s between batches, 10s backoff on 429, max 3 attempts per URL, non-retryable errors (#26) - Update README with setup.sh option Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent ef50b41 commit c006389

5 files changed

Lines changed: 417 additions & 10 deletions

File tree

‎README.md‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -53,7 +53,11 @@ AI-powered job search pipeline built on **GitHub Copilot CLI**. Evaluate offers,
5353
### Setup
5454

5555
```bash
56-
# 1. Clone and install
56+
# Option A: One-liner setup
57+
git clone <your-repo-url>
58+
cd career-copilot && bash setup.sh
59+
60+
# Option B: Manual setup
5761
git clone <your-repo-url>
5862
cd career-copilot && npm install
5963
npx playwright install chromium # Required for PDF generation

‎docs/evaluation-walkthrough.md‎

Lines changed: 205 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,205 @@
1+
# Evaluation Walkthrough
2+
3+
> A step-by-step example of how career-copilot evaluates a job offer, showing what each block produces and how the scoring works.
4+
5+
---
6+
7+
## Starting the Evaluation
8+
9+
Paste a job URL into Copilot CLI:
10+
11+
```
12+
Evaluate this: https://boards.greenhouse.io/example/jobs/12345
13+
```
14+
15+
career-copilot reads your `cv.md`, `config/profile.yml`, `modes/_profile.md`, and `modes/_shared.md`, then runs the full A-F evaluation pipeline.
16+
17+
---
18+
19+
## Block A — Role Summary
20+
21+
The AI extracts structured metadata from the JD:
22+
23+
```markdown
24+
## A — Role Summary
25+
26+
| Field | Value |
27+
|-------|-------|
28+
| **Company** | Acme Corp |
29+
| **Role** | Senior Backend Engineer |
30+
| **Archetype** | AI Platform / LLMOps |
31+
| **Domain** | Developer Tools |
32+
| **Function** | Engineering |
33+
| **Seniority** | Senior |
34+
| **Remote** | Hybrid (NYC, 3 days/week) |
35+
| **Location** | New York, USA |
36+
| **Team size** | 8-person platform team |
37+
| **URL** | https://boards.greenhouse.io/example/jobs/12345 |
38+
39+
**TL;DR**: Build and maintain the AI inference platform serving 10M+ daily requests.
40+
The team owns model deployment, observability, and cost optimization.
41+
```
42+
43+
The **archetype** classification determines which scoring weights and profile sections apply.
44+
45+
---
46+
47+
## Block B — CV Match (Score: 4.2/5)
48+
49+
A requirement-by-requirement comparison against your CV:
50+
51+
```markdown
52+
## B — CV Match (Score: 4.2/5)
53+
54+
### Requirements Mapping
55+
56+
| # | JD Requirement | CV Evidence | Strength |
57+
|---|---------------|-------------|----------|
58+
| 1 | 5+ years backend engineering | 7 years across 3 companies | ✅ Strong |
59+
| 2 | Kubernetes & container orchestration | Led K8s migration for 200-service platform | ✅ Strong |
60+
| 3 | ML model serving (TensorFlow Serving, Triton) | Deployed PyTorch models via custom pipeline | ⚠️ Partial |
61+
| 4 | Observability (Datadog, Prometheus) | Built Grafana dashboards, Prometheus alerting | ✅ Strong |
62+
| 5 | Cost optimization at scale | No direct evidence | ❌ Gap |
63+
64+
### Gaps & Mitigation
65+
66+
| Gap | Severity | Mitigation Strategy |
67+
|-----|----------|---------------------|
68+
| ML serving frameworks (Triton) | Medium | Adjacent: custom serving pipeline + quick Triton ramp-up |
69+
| Cloud cost optimization | Low | Portfolio project on spot instance optimization |
70+
```
71+
72+
**How scoring works**: Each requirement is weighted by how prominently it appears in the JD. Strong matches contribute fully, partial matches at 50%, gaps at 0%. The weighted average gives the block score.
73+
74+
---
75+
76+
## Block C — Level & Strategy (Score: 4.0/5)
77+
78+
Compares your seniority to what the JD expects:
79+
80+
```markdown
81+
## C — Level & Strategy (Score: 4.0/5)
82+
83+
**JD Level**: Senior (IC4-IC5 equivalent)
84+
**Candidate Level**: Senior-to-Staff trajectory
85+
**Assessment**: At-level to slightly over-leveled
86+
87+
### Positioning Strategy
88+
- Lead with: Platform ownership and cross-team impact
89+
- Key proof points: Led K8s migration (cv.md §3), mentored 4 engineers (cv.md §5)
90+
- Risk: None — strong at-level match with growth headroom
91+
```
92+
93+
---
94+
95+
## Block D — Comp & Market (Score: 3.8/5)
96+
97+
Market research via web search:
98+
99+
```markdown
100+
## D — Comp & Market (Score: 3.8/5)
101+
102+
### Market Compensation
103+
| Source | Range |
104+
|--------|-------|
105+
| levels.fyi | $180K-$240K (Senior, NYC) |
106+
| Glassdoor | $170K-$220K |
107+
| H1B data | $195K (median, similar roles) |
108+
109+
### JD Stated Comp
110+
$175K-$210K base + equity (if mentioned)
111+
112+
### Assessment
113+
Slightly below market median for NYC Senior Backend. Equity could compensate.
114+
Negotiate toward $200K+ base if offer comes in.
115+
```
116+
117+
---
118+
119+
## Block E — Red Flags & Cultural Signals (Score: 4.5/5)
120+
121+
Proactive detection of concerns:
122+
123+
```markdown
124+
## E — Red Flags & Cultural Signals (Score: 4.5/5)
125+
126+
### 🟢 Positive Signals
127+
- Clear team structure and reporting line mentioned
128+
- Concrete technical challenges (not vague "fast-paced")
129+
- Remote flexibility (hybrid, not forced 5-day)
130+
131+
### 🟡 Watch
132+
- "Startup mentality" in a 500-person company — could mean under-resourced
133+
134+
### 🔴 Red Flags
135+
- None detected
136+
```
137+
138+
---
139+
140+
## Block F — Final Verdict
141+
142+
The weighted global score and recommendation:
143+
144+
```markdown
145+
## F — Final Verdict
146+
147+
| Dimension | Score | Weight |
148+
|-----------|-------|--------|
149+
| CV Match | 4.2 | 30% |
150+
| Level & Strategy | 4.0 | 20% |
151+
| Comp & Market | 3.8 | 20% |
152+
| Cultural Signals | 4.5 | 15% |
153+
| North Star Alignment | 4.0 | 15% |
154+
155+
### Global Score: 4.1 / 5 — Grade: B+
156+
157+
**Recommendation**: Good match, worth applying. Strong technical alignment
158+
with moderate comp upside. The platform ownership scope matches your
159+
growth trajectory.
160+
161+
### Suggested Next Steps
162+
1. Apply with tailored CV (run `generate PDF` mode)
163+
2. Prepare STAR stories for system design and K8s migration
164+
3. Research the team lead on LinkedIn for culture fit signals
165+
```
166+
167+
---
168+
169+
## Score Interpretation
170+
171+
| Score Range | Grade | Meaning |
172+
|------------|-------|---------|
173+
| 4.5+ | A | Strong match — apply immediately |
174+
| 4.0–4.4 | B | Good match — worth applying |
175+
| 3.5–3.9 | C | Decent but not ideal — apply only with specific reason |
176+
| Below 3.5 | D-F | Recommend against applying |
177+
178+
---
179+
180+
## Output Files
181+
182+
After evaluation, career-copilot creates:
183+
184+
| File | Location | Content |
185+
|------|----------|---------|
186+
| Evaluation report | `reports/001-acme-corp-2026-04-10.md` | Full A-F analysis |
187+
| Tailored PDF | `output/cv-acme-corp-2026-04-10.pdf` | ATS-optimized resume |
188+
| Tracker entry | `data/applications.md` | Pipeline row with score and status |
189+
190+
---
191+
192+
## Try It Yourself
193+
194+
```bash
195+
# 1. Setup (if not done)
196+
bash setup.sh
197+
198+
# 2. Open Copilot CLI in this directory
199+
200+
# 3. Paste any job URL
201+
# "Evaluate this: https://boards.greenhouse.io/company/jobs/123456"
202+
203+
# 4. For batch evaluation, add URLs to batch/batch-input.tsv and run:
204+
# "Process the batch"
205+
```

‎modes/batch.md‎

Lines changed: 27 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -145,7 +145,13 @@ task(
145145
- Wait for all to complete before dispatching next batch of 3
146146
- Update `batch/batch-state.tsv` after each completion
147147

148-
### Step 4 — Collect Results
148+
**Rate limiting:**
149+
- Wait **2 seconds** between dispatching each batch of workers to avoid overwhelming job board APIs
150+
- If a `web_fetch` or `browser_navigate` returns a 429 (rate limited), wait **10 seconds** then retry
151+
- If fetching from the same domain (e.g., multiple Greenhouse URLs), space requests **3 seconds** apart
152+
- Maximum **3 concurrent workers** — do not increase even if system resources allow it
153+
154+
### Step 4 — Collect Results & Retry
149155

150156
After each worker completes:
151157

@@ -154,6 +160,14 @@ After each worker completes:
154160
3. Update `batch/batch-state.tsv` with status, score, paths
155161
4. Log any errors
156162

163+
**Retry logic for failed items:**
164+
- After all items in the batch are processed, check `batch-state.tsv` for `failed` items
165+
- Retry failed items up to **2 additional times** (3 total attempts)
166+
- Increment the `retries` column in `batch-state.tsv` each attempt
167+
- Wait **5 seconds** before each retry attempt
168+
- If an item fails 3 times, mark it as `failed` permanently with the last error message
169+
- Do NOT retry items that failed due to: login required, page not found (404), or job listing removed
170+
157171
### Step 5 — Merge Tracker
158172

159173
After all workers complete:
@@ -166,19 +180,23 @@ This consolidates `batch/tracker-additions/*.tsv` into `data/applications.md`.
166180

167181
---
168182

169-
## Error Handling
183+
## Error Handling & Retry Policy
170184

171-
| Error | Recovery |
172-
|-------|----------|
173-
| URL inaccessible | Worker fails → conductor marks `failed`, continues |
174-
| JD behind login | Use `browser_navigate` to render page, extract JD from snapshot. If login required → `failed` |
175-
| Worker crashes | Mark `failed`, continue. Re-run batch to retry failed items |
176-
| PDF generation fails | Report .md saved. PDF marked as ❌ in tracker |
177-
| All workers fail | Check `npm run doctor` for setup issues |
185+
| Error | Recovery | Retryable? |
186+
|-------|----------|-----------|
187+
| URL inaccessible / timeout | Wait 5s, retry up to 2 more times | ✅ Yes |
188+
| Rate limited (429) | Wait 10s, retry | ✅ Yes |
189+
| JD behind login | Use `browser_navigate` first. If login wall persists → `failed` | ❌ No |
190+
| Page not found (404) | Mark `failed`, listing likely removed | ❌ No |
191+
| Worker crashes | Mark `failed`, retry in next pass | ✅ Yes |
192+
| PDF generation fails | Report .md saved. PDF marked as ❌ in tracker | ✅ Yes (PDF only) |
193+
| All workers fail | Check `npm run doctor` for setup issues | — |
178194

179195
- **Individual failures do NOT block other items** — each URL is independent
180196
- **Resumability**: Running batch again skips completed items, retries failed ones
181197
- **Timeout**: If a worker exceeds 5 minutes, mark as `failed`
198+
- **Max retries**: 3 total attempts per item (1 initial + 2 retries)
199+
- **Backoff**: 5 second delay between retry attempts, 10 seconds after rate limit
182200

183201
---
184202

‎setup.sh‎

Lines changed: 112 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,112 @@
1+
#!/usr/bin/env bash
2+
set -euo pipefail
3+
4+
# career-copilot setup script
5+
# One command: curl -sL <repo-url>/setup.sh | bash
6+
# Or locally: bash setup.sh
7+
8+
GREEN='\033[0;32m'
9+
RED='\033[0;31m'
10+
YELLOW='\033[1;33m'
11+
NC='\033[0m'
12+
13+
pass() { echo -e " ${GREEN}✅ $1${NC}"; }
14+
fail() { echo -e " ${RED}❌ $1${NC}"; exit 1; }
15+
warn() { echo -e " ${YELLOW}⚠️ $1${NC}"; }
16+
info() { echo -e " $1"; }
17+
18+
echo ""
19+
echo "🚀 career-copilot setup"
20+
echo "========================"
21+
echo ""
22+
23+
# 1. Check Node.js
24+
echo "1. Checking Node.js..."
25+
if command -v node &>/dev/null; then
26+
NODE_VERSION=$(node -v | sed 's/v//')
27+
NODE_MAJOR=$(echo "$NODE_VERSION" | cut -d. -f1)
28+
if [ "$NODE_MAJOR" -ge 18 ]; then
29+
pass "Node.js v${NODE_VERSION}"
30+
else
31+
fail "Node.js >= 18 required (found v${NODE_VERSION}). Install from https://nodejs.org"
32+
fi
33+
else
34+
fail "Node.js not found. Install from https://nodejs.org (v18+)"
35+
fi
36+
37+
# 2. Install npm dependencies
38+
echo ""
39+
echo "2. Installing npm dependencies..."
40+
if [ -f package.json ]; then
41+
npm install --no-fund --no-audit 2>&1 | tail -1
42+
pass "npm packages installed"
43+
else
44+
fail "package.json not found — run this from the career-copilot directory"
45+
fi
46+
47+
# 3. Install Playwright Chromium
48+
echo ""
49+
echo "3. Installing Playwright Chromium (for PDF generation)..."
50+
npx playwright install chromium 2>&1 | tail -3
51+
pass "Playwright Chromium installed"
52+
53+
# 4. Check Go (optional — for TUI dashboard)
54+
echo ""
55+
echo "4. Checking Go (optional — for TUI dashboard)..."
56+
if command -v go &>/dev/null; then
57+
GO_VERSION=$(go version | awk '{print $3}' | sed 's/go//')
58+
pass "Go ${GO_VERSION} — dashboard available"
59+
info " Build with: cd dashboard && go build -o career-dashboard ."
60+
else
61+
warn "Go not installed — TUI dashboard won't build (everything else works fine)"
62+
info " Install from https://go.dev/dl/ if you want the dashboard"
63+
fi
64+
65+
# 5. Setup config files
66+
echo ""
67+
echo "5. Setting up configuration..."
68+
if [ ! -f config/profile.yml ]; then
69+
if [ -f config/profile.example.yml ]; then
70+
cp config/profile.example.yml config/profile.yml
71+
warn "Created config/profile.yml from template — edit with your details"
72+
else
73+
warn "No profile template found — create config/profile.yml manually"
74+
fi
75+
else
76+
pass "config/profile.yml exists"
77+
fi
78+
79+
if [ ! -f portals.yml ]; then
80+
if [ -f templates/portals.example.yml ]; then
81+
cp templates/portals.example.yml portals.yml
82+
warn "Created portals.yml from template — customize for your job search"
83+
else
84+
warn "No portals template found"
85+
fi
86+
else
87+
pass "portals.yml exists"
88+
fi
89+
90+
if [ ! -f cv.md ]; then
91+
warn "cv.md not found — create it with your CV in markdown format"
92+
info " See examples/ for reference CV formats"
93+
else
94+
pass "cv.md exists"
95+
fi
96+
97+
# 6. Run doctor
98+
echo ""
99+
echo "6. Running doctor check..."
100+
echo ""
101+
node doctor.mjs 2>&1 || true
102+
103+
echo ""
104+
echo "========================"
105+
echo -e "${GREEN}✅ Setup complete!${NC}"
106+
echo ""
107+
echo "Next steps:"
108+
echo " 1. Edit config/profile.yml with your details"
109+
echo " 2. Create cv.md with your CV in markdown"
110+
echo " 3. Open GitHub Copilot CLI in this directory"
111+
echo " 4. Paste a job URL to evaluate it"
112+
echo ""

0 commit comments

Comments
 (0)