Note: This is an early example evaluation created during Scout's development. Future evaluations will follow a more standardized format driven by the evaluation agent's intake flow.
PRIMITIVES.md— A filled-in primitives document defining 17 experience primitives for evaluating the Azure Portal. Shows what a real primitives spec looks like with definitions, portal delivery assessments, and MCP gap analysis.output/report.md— A completed evaluation report comparing Azure Portal vs MCP tools across 17 primitives. Shows the report format: TL;DR, methodology, primitive-by-primitive scoring, competitive analysis, and prioritized recommendations.
This evaluation asked: "Can MCP tools deliver Azure management experiences that match or exceed the Azure Portal?"
The evaluation used 3 signals:
- Walkthrough — hands-on assessment of all 17 primitives in Azure Portal + MCP tool inventory
- Research — GitHub issues, Reddit, competitor CLI analysis (Vercel, Railway, DigitalOcean, etc.)
- Data — telemetry schema analysis across Kusto clusters (schema-level, not query-level for this version)
- Portal average rating: 2.76 / 5 across 17 primitives
- MCP leapfrogs the Portal on 5 of 17 primitives — the ones the Portal is worst at
- 6 prioritized recommendations (P0-P2) with specific implementation guidance