Skip to content

Commit d64bfc3

Browse files
webmaxruwebmaxru
authored
feat: AgentRC Readiness Scanner Web App (#90)
* feat: add webapp with report rendering, Docker support, and CI/CD * fix: update Trivy action version and improve Dockerfile for backend dependencies * chore: initialize frontend package with vitest for testing * fix: update Dockerfile to ignore scripts during npm install and simplify docker-compose context * fix: harden frontend tests with dedicated vitest config and static imports * fix: make Bicep secrets conditional for empty GH token * fix: update achievedLevel validation to accept 0 and adjust related tests * fix: enhance report validation and rendering by adding safe class handling for status, impact, and effort * fix: improve report validation and enhance frontend theme handling * fix: enhance share button functionality and add tooltip for better user guidance * feat: add Azure Container Registry resource and update container image handling * fix: update storage account naming convention to ensure uniqueness and lowercase formatting * fix: update container image handling and improve resource naming conventions for Azure deployment * fix: refine rate limiter to only skip OPTIONS requests and update allowed signal status in report rendering * fix: rename GitHub token parameter for consistency in scanning configuration * fix: remove existing env storage before Bicep deploy (Container Apps PUT limitation) * fix: use ARM REST API for storage removal with propagation delay * fix: delete container app before storage to allow Bicep recreation * fix: import GHCR image into ACR, add GHCR auth for security scan, retry smoke tests - Image was pushed to GHCR but Bicep pulls from ACR added az acr import steps - Security scan lacked GHCR auth added docker/login-action + packages:read - Smoke test had no retries after cold-start restarts added retry loop * fix: enhance report validation for areaReports and policies, add comprehensive tests * fix: improve error handling in SPA route and enhance report validation logic * feat(apm): add APM configuration checks and integrate into readiness criteria * feat(report-validator): enhance validation logic for pillars, levels, and criteria arrays fix(report): update report rendering logic to handle edge cases in passed and total values style(progress): replace progress bar with spinner for better UX during repository cloning fix(config): remove appInsightsConnectionString from public config response fix(Dockerfile): ensure core package symlink is recreated after removal fix(bicep): disable admin user for Azure Container Registry and add AcrPull role assignment chore(package-lock): update dependencies and remove unnecessary dev dependencies * fix: remove duplicate APM criteria already merged in main via PR #92 * fix: improve URL parsing and handle empty segments in owner/repo format fix: resolve frontend path using fileURLToPath for better compatibility fix: enhance theme toggle functionality to handle localStorage errors gracefully * refactor: update build process and add esbuild configuration - Changed the start script to run the bundled server from the dist directory. - Added a build script to bundle the application using esbuild. - Introduced a new esbuild configuration file to handle the bundling of the server. - Updated dependencies to include esbuild and adjusted the location of @agentrc/core. * fix(scanner): improve error handling for clone timeout and sanitize error messages * fix(Dockerfile): correct paths for backend files and improve directory structure * fix(report-validator): enhance validation and sanitization of criteria fields to prevent XSS fix(report): use safe number handling for app and area summaries in report rendering fix(Dockerfile): include node_modules from deps for backend build * feat(storage): add report cleanup functionality and integrate with server startup * fix(Dockerfile): simplify directory creation and ownership setup fix(cleanup): streamline error handling in removeTempDir function * fix(bicep): enforce constraints on name prefix parameters for resource naming fix(scanner): encode GitHub token in clone URL to prevent issues with special characters
1 parent 94836f7 commit d64bfc3

43 files changed

Lines changed: 9631 additions & 0 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.dockerignore‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
# Dependencies (re-installed inside Docker)
2+
node_modules
3+
4+
# Build outputs
5+
dist
6+
*.tgz
7+
8+
# Git / CI
9+
.git
10+
.github
11+
.husky
12+
13+
# CLI source (not needed for webapp)
14+
src
15+
plugin
16+
infra
17+
18+
# VS Code extension
19+
vscode-extension
20+
21+
# Docs / examples
22+
docs
23+
examples
24+
25+
# Non-essential root files
26+
eval-results.*
27+
readiness-report.html
28+
CHANGELOG.md
29+
CODE_OF_CONDUCT.md
30+
CONTRIBUTING.md
31+
SECURITY.md
32+
SUPPORT.md
33+
CODEOWNERS
34+
LICENSE
35+
AGENTS.md
36+
agentrc.eval.json
37+
eslint.config.js
38+
vitest.config.ts
39+
tsconfig.json
40+
tsup.config.ts
41+
.vscode

‎.github/prompts/align-cli-webapp-reports.prompt.md‎

Lines changed: 161 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,161 @@
1+
---
2+
description: Compare CLI visual readiness report with local webapp report for a given repo, identify differences in checks/rendering/scoring, and fix them
3+
---
4+
5+
You are debugging consistency between two readiness report outputs for the **AgentRC** project:
6+
7+
1. **CLI visual report** — generated by `npm run dev -- readiness --visual` from the repo root (produces an HTML file)
8+
2. **Webapp report** — rendered by the local dev server at `http://localhost:3000/{owner}/{repo}`
9+
10+
Both should produce identical results for the same repository because they share the same core engine (`packages/core/src/services/readiness.ts`). In practice they can diverge due to rendering differences or scoring logic bugs.
11+
12+
## Architecture Reference
13+
14+
### Shared Core (source of truth for checks)
15+
16+
- `packages/core/src/services/readiness.ts` — All criteria definitions, `countStatus()`, `buildCriteria()`, `buildExtras()`, pillar/level aggregation
17+
- Criteria scopes: `repo` (always), `app` (per-application), `area` (only with `--per-area`)
18+
- `countStatus()` **excludes** skipped checks from the denominator when computing pillar pass/total
19+
- Extras (bonus checks) are **not scored** — they don't affect levels or totals
20+
21+
### CLI Rendering
22+
23+
- `packages/core/src/services/visualReport.ts` — Generates the standalone HTML report
24+
- `calculateAiToolingData()` — Aggregates AI criteria across repos (counts all including skipped in the hero display)
25+
- Total checks: `report.pillars.reduce((s, p) => s + p.total, 0)`
26+
- Does **not** render bonus/extras section in HTML output
27+
28+
### Webapp Backend
29+
30+
- `webapp/backend/src/services/scanner.js` — Clones repo, calls `runReadinessReport()` from `@agentrc/core`
31+
- `webapp/backend/src/routes/scan.js` — `POST /api/scan` endpoint
32+
- Returns the raw `ReadinessReport` JSON (same shape as CLI)
33+
- Uses `@agentrc/core` as a `file:../../packages/core` dependency — always uses local source code
34+
35+
### Webapp Frontend
36+
37+
- `webapp/frontend/src/report.js` — Independent rendering implementation (NOT shared with CLI)
38+
- `buildHero()` — Total from `report.pillars.reduce((s, p) => s + p.total, 0)`
39+
- `buildAiToolingHero()` — Renders all AI criteria (including skipped) with pass/total count
40+
- `buildPillarDetails()` — Shows per-pillar expandable cards
41+
- **Does** render bonus checks section (unlike CLI)
42+
- Has Service Information section (policy chain, engine signals) — CLI doesn't
43+
44+
### Known Inconsistency Patterns
45+
46+
- **AI Hero vs Pillar scoring**: Both implementations count skipped checks as non-passing in the AI Hero but exclude them from pillar denominator via `countStatus()`
47+
- **Rendering gaps**: Webapp shows bonus checks + service info; CLI doesn't
48+
- **Icon mapping**: CLI uses HTML entities via `getAiCriterionIcon()`; webapp uses emoji via `AI_ICONS` map — new criteria IDs may get fallback icon (`🔧`) in webapp
49+
50+
## Step-by-Step Procedure
51+
52+
### Phase 0: Start Local Webapp
53+
54+
1. Start the webapp backend dev server (from the repo root):
55+
56+
```
57+
cd webapp/backend && npm run dev
58+
```
59+
60+
This starts the Express server at `http://localhost:3000` with the local `@agentrc/core` source.
61+
62+
2. Optionally serve the frontend for full visual testing:
63+
```
64+
cd webapp/frontend && npx vite --port 5173
65+
```
66+
67+
### Phase 1: Generate Both Reports
68+
69+
3. Run the CLI against the target repo to produce the visual HTML report:
70+
71+
```
72+
npm run dev -- readiness --visual --repo {owner}/{repo}
73+
```
74+
75+
Save the output HTML (typically `readiness-report.html`).
76+
77+
4. Hit the local webapp API to get the raw JSON:
78+
79+
```
80+
POST http://localhost:3000/api/scan
81+
Body: {"repo_url":"https://github.com/{owner}/{repo}"}
82+
```
83+
84+
Example with PowerShell:
85+
86+
```powershell
87+
$response = Invoke-RestMethod -Uri "http://localhost:3000/api/scan" -Method POST -ContentType "application/json" -Body '{"repo_url":"https://github.com/{owner}/{repo}"}' -TimeoutSec 120
88+
```
89+
90+
5. Also open the local webapp page for visual comparison: `http://localhost:5173/{owner}/{repo}` (if frontend dev server is running) or `http://localhost:3000/{owner}/{repo}` (if backend serves static files).
91+
92+
### Phase 2: Compare Data Layer
93+
94+
6. Extract from CLI HTML: total checks, per-pillar passed/total, AI hero passed/total/percentage, criteria list with statuses, achieved level, fix-first items.
95+
96+
7. Extract from webapp JSON: same fields. Use:
97+
98+
```powershell
99+
$pillars = $response.pillars
100+
$totalPassed = ($pillars | Measure-Object -Property passed -Sum).Sum
101+
$totalChecks = ($pillars | Measure-Object -Property total -Sum).Sum
102+
Write-Host "Total: $totalPassed of $totalChecks checks"
103+
Write-Host "Criteria count: $($response.criteria.Count)"
104+
```
105+
106+
8. Diff the two — check for:
107+
- **Missing criteria** in either side (criteria list length mismatch)
108+
- **Status mismatches** for the same criterion ID
109+
- **Total check count** differences (pillar aggregation)
110+
- **AI Tooling hero** percentage/label differences
111+
- **Achieved level** and next-level calculation differences
112+
- **Fix-first** list ordering differences
113+
114+
### Phase 3: Compare Rendering Layer
115+
116+
9. Compare how both renderers handle:
117+
- Skipped checks display (icon, text, inclusion in totals)
118+
- Bonus/extras section presence
119+
- Pillar grouping (repo-health vs ai-setup)
120+
- AI criterion icons for new/unknown IDs
121+
- Score thresholds for labels (Excellent/Good/Fair/Getting Started/Not Started)
122+
123+
### Phase 4: Root Cause & Fix
124+
125+
10. For each difference found, classify as:
126+
- **Rendering divergence** → Fix in either `visualReport.ts` (CLI) or `report.js` (webapp) to align
127+
- **Scoring logic bug** → Fix in `readiness.ts` (core) which fixes both
128+
- **Icon/label mapping gap** → Update the icon map in the affected renderer
129+
130+
11. Implement the fixes directly in the source files.
131+
132+
12. After fixing, restart the webapp dev server and re-run Phase 1-2 to verify alignment.
133+
134+
### Phase 5: Validate
135+
136+
13. Confirm both reports show identical:
137+
- Total check count (e.g., "11 of 20")
138+
- Per-pillar passed/total
139+
- AI Tooling hero percentage and label
140+
- Achieved maturity level
141+
- Fix-first items (same set, same order)
142+
143+
14. Note any **acceptable differences** that are by-design (e.g., webapp shows bonus checks, CLI doesn't).
144+
145+
15. Run existing tests to ensure no regressions:
146+
```
147+
npm test
148+
cd webapp/backend && npm test
149+
```
150+
151+
## Output Format
152+
153+
Produce a comparison table:
154+
155+
| Aspect | CLI Value | Webapp Value | Match? | Root Cause | Fix |
156+
| ------------ | --------- | ------------ | ------ | ---------- | --- |
157+
| Total checks | X of Y | X of Z | ... | ... | ... |
158+
| AI Hero % | ...% | ...% | ... | ... | ... |
159+
| ... | ... | ... | ... | ... | ... |
160+
161+
Then implement the fixes and verify.

‎.github/workflows/webapp-cd.yml‎

Lines changed: 178 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,178 @@
1+
name: Webapp CD
2+
3+
on:
4+
workflow_dispatch:
5+
push:
6+
branches: [main]
7+
paths:
8+
- "webapp/**"
9+
- "packages/core/**"
10+
- "Dockerfile.webapp"
11+
- "infra/webapp/**"
12+
13+
concurrency:
14+
group: agentrc-webapp-production
15+
cancel-in-progress: false
16+
17+
permissions:
18+
contents: read
19+
packages: write
20+
id-token: write
21+
22+
env:
23+
REGISTRY: ghcr.io
24+
IMAGE_NAME: ${{ github.repository_owner }}/agentrc-webapp
25+
RESOURCE_GROUP: agentrc-webapp-rg
26+
BICEP_FILE: infra/webapp/main.bicep
27+
28+
jobs:
29+
build-push:
30+
runs-on: ubuntu-latest
31+
outputs:
32+
image-tag: ${{ steps.meta.outputs.version }}
33+
image-full: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ steps.meta.outputs.version }}
34+
steps:
35+
- uses: actions/checkout@v4
36+
37+
- name: Log in to GHCR
38+
uses: docker/login-action@v3
39+
with:
40+
registry: ${{ env.REGISTRY }}
41+
username: ${{ github.actor }}
42+
password: ${{ secrets.GITHUB_TOKEN }}
43+
44+
- name: Docker metadata
45+
id: meta
46+
uses: docker/metadata-action@v5
47+
with:
48+
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
49+
tags: |
50+
type=sha,prefix=sha-
51+
type=raw,value=latest
52+
53+
- name: Build and push
54+
uses: docker/build-push-action@v6
55+
with:
56+
context: .
57+
file: Dockerfile.webapp
58+
push: true
59+
tags: ${{ steps.meta.outputs.tags }}
60+
labels: ${{ steps.meta.outputs.labels }}
61+
62+
security-scan:
63+
runs-on: ubuntu-latest
64+
needs: build-push
65+
permissions:
66+
security-events: write
67+
packages: read
68+
steps:
69+
- uses: actions/checkout@v4
70+
- name: Log in to GHCR
71+
uses: docker/login-action@v3
72+
with:
73+
registry: ${{ env.REGISTRY }}
74+
username: ${{ github.actor }}
75+
password: ${{ secrets.GITHUB_TOKEN }}
76+
- name: Run Trivy vulnerability scanner
77+
uses: aquasecurity/trivy-action@v0.35.0
78+
with:
79+
image-ref: ${{ needs.build-push.outputs.image-full }}
80+
format: sarif
81+
output: trivy-results.sarif
82+
severity: CRITICAL,HIGH
83+
- name: Upload Trivy scan results
84+
uses: github/codeql-action/upload-sarif@v3
85+
if: always()
86+
with:
87+
sarif_file: trivy-results.sarif
88+
89+
deploy:
90+
runs-on: ubuntu-latest
91+
needs: [build-push, security-scan]
92+
environment: production
93+
steps:
94+
- uses: actions/checkout@v4
95+
96+
- name: Azure Login (OIDC)
97+
uses: azure/login@v2
98+
with:
99+
client-id: ${{ secrets.AZURE_CLIENT_ID }}
100+
tenant-id: ${{ secrets.AZURE_TENANT_ID }}
101+
subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
102+
103+
- name: Ensure resource group
104+
run: |
105+
az group create \
106+
--name ${{ env.RESOURCE_GROUP }} \
107+
--location eastus \
108+
--tags application=agentrc-webapp managedBy=bicep
109+
110+
- name: Push image to ACR
111+
run: |
112+
ACR_NAME=$(az acr list --resource-group ${{ env.RESOURCE_GROUP }} --query "[0].name" -o tsv)
113+
if [ -z "$ACR_NAME" ]; then
114+
echo "ACR not yet provisioned — will be created by Bicep and image imported after"
115+
else
116+
az acr import \
117+
--name "$ACR_NAME" \
118+
--source ${{ needs.build-push.outputs.image-full }} \
119+
--image agentrc-webapp:${{ needs.build-push.outputs.image-tag }} \
120+
--image agentrc-webapp:latest \
121+
--force
122+
fi
123+
124+
- name: Pre-deploy cleanup (Container Apps storage update limitation)
125+
run: |
126+
az containerapp delete \
127+
--name agentrc-webapp \
128+
--resource-group ${{ env.RESOURCE_GROUP }} \
129+
--yes 2>&1 || echo "Container app does not exist yet"
130+
SUB_ID=$(az account show --query id -o tsv)
131+
az rest --method DELETE \
132+
--url "https://management.azure.com/subscriptions/${SUB_ID}/resourceGroups/${{ env.RESOURCE_GROUP }}/providers/Microsoft.App/managedEnvironments/agentrc-env/storages/reportsshare?api-version=2024-03-01" \
133+
2>&1 || echo "Storage mount does not exist yet"
134+
sleep 10
135+
136+
- name: Deploy infrastructure
137+
uses: azure/arm-deploy@v2
138+
with:
139+
resourceGroupName: ${{ env.RESOURCE_GROUP }}
140+
template: ${{ env.BICEP_FILE }}
141+
parameters: >
142+
containerImageTag=${{ needs.build-push.outputs.image-tag }}
143+
ghTokenForScan=${{ secrets.GH_TOKEN_FOR_SCAN }}
144+
145+
- name: Ensure image in ACR
146+
run: |
147+
ACR_NAME=$(az acr list --resource-group ${{ env.RESOURCE_GROUP }} --query "[0].name" -o tsv)
148+
az acr import \
149+
--name "$ACR_NAME" \
150+
--source ${{ needs.build-push.outputs.image-full }} \
151+
--image agentrc-webapp:${{ needs.build-push.outputs.image-tag }} \
152+
--image agentrc-webapp:latest \
153+
--force
154+
155+
- name: Restart container app to pick up new image
156+
run: |
157+
az containerapp revision restart \
158+
--name agentrc-webapp \
159+
--resource-group ${{ env.RESOURCE_GROUP }} \
160+
--revision "$(az containerapp revision list --name agentrc-webapp --resource-group ${{ env.RESOURCE_GROUP }} --query '[0].name' -o tsv)"
161+
162+
- name: Smoke test
163+
run: |
164+
APP_URL=$(az containerapp show \
165+
--name agentrc-webapp \
166+
--resource-group ${{ env.RESOURCE_GROUP }} \
167+
--query "properties.configuration.ingress.fqdn" -o tsv)
168+
echo "Testing https://${APP_URL}"
169+
for i in 1 2 3 4 5; do
170+
if curl -sf "https://${APP_URL}/api/health" | grep -q '"ok"'; then
171+
break
172+
fi
173+
echo "Attempt $i failed, retrying in 15s..."
174+
sleep 15
175+
done
176+
curl -sf "https://${APP_URL}/api/health" | grep -q '"ok"'
177+
curl -sf "https://${APP_URL}/" | grep -q "AgentRC"
178+
echo "Smoke tests passed!"

0 commit comments

Comments
 (0)