Skip to content

Latest commit

 

History

History
702 lines (532 loc) · 17.3 KB

File metadata and controls

702 lines (532 loc) · 17.3 KB

ARM Reviewer Workflow Guide

This guide demonstrates how to use the ARM Reviewer MCP server in your development workflow to validate Azure Resource Manager (ARM) specifications before creating PRs.

Table of Contents


Quick Start

Installation

  1. Install the MCP server:

    cd mcp-arm-reviewer
    uv sync
  2. Configure for Claude Desktop (optional):

    On Windows: %APPDATA%/Claude/claude_desktop_config.json

    On MacOS: ~/Library/Application Support/Claude/claude_desktop_config.json

    {
      "mcpServers": {
        "mcp-arm-reviewer": {
          "command": "uv",
          "args": [
            "--directory",
            "C:\\repos\\mcp-arm-reviewer\\mcp-arm-reviewer",
            "run",
            "mcp-arm-reviewer"
          ]
        }
      }
    }
  3. Test the installation:

    cd mcp-arm-reviewer
    uv run mcp-arm-reviewer

Pre-PR Development Workflow

Workflow Overview

┌─────────────────────┐
│ Write TypeSpec/API  │
│   Specifications    │
└──────────┬──────────┘
           │
           ▼
┌─────────────────────┐
│  Analyze File/      │
│   Workspace         │
└──────────┬──────────┘
           │
           ▼
┌─────────────────────┐
│  Review Issues      │
│  (Errors/Warnings)  │
└──────────┬──────────┘
           │
           ├─── Issues Found ───┐
           │                    │
           │                    ▼
           │         ┌─────────────────────┐
           │         │  Get Fix Suggestions│
           │         └──────────┬──────────┘
           │                    │
           │                    ▼
           │         ┌─────────────────────┐
           │         │   Apply Fixes       │
           │         └──────────┬──────────┘
           │                    │
           │                    └────────┐
           │                             │
           ▼                             ▼
┌─────────────────────┐      ┌─────────────────────┐
│  No Issues Found    │      │   Re-analyze        │
│  ✅ Ready for PR    │      │   (Loop until clean)│
└─────────────────────┘      └─────────────────────┘

Step-by-Step Process

Step 1: Create/Modify Your API Specifications

Create your TypeSpec or OpenAPI/Swagger specifications:

// example.tsp
import "@azure-tools/typespec-azure-resource-manager";
import "@typespec/rest";

@armProviderNamespace
namespace Microsoft.Example;

@doc("Example resource")
model ExampleResource is TrackedResource<ExampleResourceProperties> {
  @doc("Resource name")
  @pattern("^[a-z0-9]+$")
  @key("resourceName")
  @segment("resources")
  name: string;
}

@doc("Properties of the example resource")
model ExampleResourceProperties {
  @doc("Description")
  description?: string;
  
  @doc("Reference to related resource")
  relatedResourceId?: ResourceIdentifier<[{type: "Microsoft.Network/virtualNetworks"}]>;
}

Step 2: Analyze Your File (In Claude Desktop)

Using Claude Desktop with MCP:

  1. Open Claude Desktop
  2. Ask: "Analyze my ARM specification file"
  3. Claude will use the analyze-file tool:
Tool: analyze-file
Input:
{
  "file_path": "example.tsp",
  "content": "<your file content>"
}

Output Example:

ARM Analysis Complete for example.tsp

Total Issues: 0
✅ No ARM compliance issues found!

View detailed results using the 'armreview://review/file_example_tsp' resource.

Step 3: Analyze Entire Workspace

For comprehensive analysis of multiple files:

In Claude Desktop: Ask: "Analyze my entire workspace at C:\repos\my-api-specs"

Tool: analyze-workspace
Input:
{
  "workspace_path": "C:\\repos\\my-api-specs"
}

Output Example:

ARM Workspace Analysis Complete

Files Analyzed: 15
Total Issues: 8

Issues by Severity:
- Error: 3
- Warning: 4
- Info: 1

Top Issue Types:
- MISSING_ARM_ID_DETAILS: 3
- API_VERSION_FORMAT: 2
- MISSING_PAGINATION: 2
- CONSIDER_LONG_RUNNING_OPERATION: 1

Use 'arm-compliance-summary' prompt for detailed breakdown.

Step 4: Get Compliance Summary

In Claude Desktop: Ask: "Show me the ARM compliance summary for errors only"

Prompt: arm-compliance-summary
Arguments:
{
  "severity": "error"
}

Output Example:

ARM Compliance Summary

Total Issues Found: 3
Files Analyzed: 15

Issues by Category:

**ERROR: MISSING_ARM_ID_DETAILS**
- Count: 2
- Affected files: example.json, network.json
- Examples:
  • Property 'virtualNetworkId' in 'NetworkSettings' has format 'arm-id' but missing 'x-ms-arm-id-details'
  • Property 'subnetId' in 'SubnetReference' has format 'arm-id' but missing 'x-ms-arm-id-details'

**ERROR: UNIFORM_VERSIONING_VIOLATION**
- Count: 1
- Affected files: readme.md
- Examples:
  • Multiple API versions detected in default tag: ['2023-12-01', '2024-01-01']. All resources in a service should use the same API version.

Step 5: Get Fix Suggestions

In Claude Desktop: Ask: "How do I fix the MISSING_ARM_ID_DETAILS errors?"

Prompt: arm-fix-suggestions
Arguments:
{
  "error_code": "MISSING_ARM_ID_DETAILS"
}

Output Example:

ARM Compliance Fix Suggestions

## MISSING_ARM_ID_DETAILS

**Issue**: Property 'virtualNetworkId' in 'NetworkSettings' has format 'arm-id' but missing 'x-ms-arm-id-details'

**Fix Suggestions**:
1. Add the x-ms-arm-id-details extension to your arm-id formatted string properties
2. Specify allowedResources with appropriate type and scopes

Example:
```json
"resourceId": {
  "type": "string",
  "format": "arm-id",
  "x-ms-arm-id-details": {
    "allowedResources": [
      {
        "scopes": ["ResourceGroup"],
        "type": "Microsoft.Network/virtualNetworks"
      }
    ]
  }
}

Affected locations:

  • example.json: Property 'virtualNetworkId' in 'NetworkSettings' has format 'arm-id' but missing 'x-ms-arm-id-details'
  • network.json: Property 'subnetId' in 'SubnetReference' has format 'arm-id' but missing 'x-ms-arm-id-details'

#### Step 6: Apply Fixes and Re-analyze

1. Apply the suggested fixes to your files
2. Re-run the analysis to verify fixes
3. Repeat until all critical issues are resolved

#### Step 7: Ready for PR

Once analysis shows no errors:

✅ No ARM compliance issues found! Ready to create your PR!


---

## Integration Options

### Option 1: Interactive Development (Claude Desktop)

**Best for:** Individual developers writing specifications

**Workflow:**
1. Write/edit specifications
2. Ask Claude to analyze: "Check my ARM spec for compliance"
3. Get immediate feedback with fix suggestions
4. Iterate until clean
5. Create PR

**Example Claude Prompts:**
- "Analyze my TypeSpec file for ARM compliance"
- "Check all my API specs in the workspace"
- "What ARM errors do I have and how do I fix them?"
- "Show me only the critical ARM issues"
- "Give me fix suggestions for MISSING_PAGINATION errors"

### Option 2: VS Code Extension

**Best for:** Developers who prefer IDE integration

1. Install MCP extension for VS Code
2. Configure workspace settings
3. Get real-time validation as you type
4. Click on errors for fix suggestions

### Option 3: Command-Line Validation

**Best for:** Pre-commit hooks and local scripts

Create a validation script:

```python
# validate_specs.py
import asyncio
import json
import sys
from pathlib import Path
from mcp_arm_reviewer.server import ARMReviewer

async def validate_workspace(workspace_path):
    """Validate all specs in workspace"""
    workspace = Path(workspace_path)
    reviewer = ARMReviewer()
    
    errors = []
    for file_path in workspace.rglob("*.json"):
        with open(file_path, 'r') as f:
            content = f.read()
        
        file_errors = reviewer.analyze_file(str(file_path), content)
        errors.extend([e for e in file_errors if e.severity == "error"])
    
    if errors:
        print(f"❌ Found {len(errors)} ARM compliance errors:")
        for error in errors:
            print(f"  {error.file_path}: {error.message}")
        sys.exit(1)
    else:
        print("✅ All specs pass ARM compliance checks!")
        sys.exit(0)

if __name__ == "__main__":
    asyncio.run(validate_workspace(sys.argv[1]))

Run before commit:

python validate_specs.py ./specifications

Option 4: Git Pre-commit Hook

Best for: Automatic validation before commits

Create .git/hooks/pre-commit:

#!/bin/bash

echo "Running ARM compliance checks..."

# Find changed spec files
CHANGED_FILES=$(git diff --cached --name-only --diff-filter=ACM | grep -E '\.(json|yaml|yml|tsp)$')

if [ -z "$CHANGED_FILES" ]; then
    echo "No spec files changed"
    exit 0
fi

# Run validation
cd mcp-arm-reviewer
uv run python validate_specs.py ../

if [ $? -ne 0 ]; then
    echo "❌ ARM compliance check failed!"
    echo "Fix the issues above before committing."
    exit 1
fi

echo "✅ ARM compliance check passed!"
exit 0

Make it executable:

chmod +x .git/hooks/pre-commit

Common Use Cases

Use Case 1: New API Development

Scenario: Creating a new Azure service API from scratch

Workflow:

  1. Set up TypeSpec project structure
  2. Write initial resource definitions
  3. Analyze workspace → Get early feedback
  4. Iterate on design with real-time validation
  5. Before first PR: Full workspace analysis
  6. Address all errors and warnings
  7. Submit PR with confidence

Time Saved: ~4-6 hours of review iterations

Use Case 2: API Version Update

Scenario: Adding a new API version to existing service

Workflow:

  1. Copy previous version files
  2. Make changes for new version
  3. Analyze new files → Check for breaking changes
  4. Update readme.md with new version
  5. Analyze readme → Ensure uniform versioning
  6. Get compliance summary → Verify no regressions
  7. Submit PR

Key Check: Uniform versioning across all files

Use Case 3: OpenAPI to TypeSpec Migration

Scenario: Migrating existing OpenAPI specs to TypeSpec

Workflow:

  1. Analyze original OpenAPI → Baseline current issues
  2. Convert to TypeSpec
  3. Analyze TypeSpec version → Compare issue count
  4. Fix migration-introduced issues
  5. Ensure all original compliance is maintained
  6. Submit migration PR

Benefit: Ensure migration doesn't introduce new compliance issues

Use Case 4: PR Review Preparation

Scenario: Final check before submitting PR

Workflow:

# 1. Analyze all changed files
git diff --name-only origin/main | grep -E '\.(json|yaml|tsp)$' > changed_files.txt

# 2. Run ARM review on each
for file in $(cat changed_files.txt); do
    # Analyze via MCP or command-line
done

# 3. Generate summary report
# 4. Fix all errors
# 5. Document any accepted warnings
# 6. Submit PR with clean ARM compliance

PR Template Addition:

## ARM Compliance
- [x] Analyzed with ARM Reviewer MCP
- [x] 0 errors, 0 warnings
- [x] All `arm-id` properties have `x-ms-arm-id-details`
- [x] API versions follow YYYY-MM-DD format
- [x] Uniform versioning maintained

CI/CD Integration

GitHub Actions Workflow

Create .github/workflows/arm-compliance.yml:

name: ARM Compliance Check

on:
  pull_request:
    paths:
      - '**.json'
      - '**.yaml'
      - '**.yml'
      - '**.tsp'
      - '**/readme.md'

jobs:
  arm-compliance:
    runs-on: ubuntu-latest
    
    steps:
      - name: Checkout code
        uses: actions/checkout@v3
      
      - name: Install uv
        uses: astral-sh/setup-uv@v1
      
      - name: Install ARM Reviewer
        run: |
          cd mcp-arm-reviewer
          uv sync
      
      - name: Analyze Changed Files
        run: |
          # Get changed spec files
          git fetch origin main
          CHANGED_FILES=$(git diff --name-only origin/main...HEAD | grep -E '\.(json|yaml|yml|tsp)$' || true)
          
          if [ -z "$CHANGED_FILES" ]; then
            echo "No spec files changed"
            exit 0
          fi
          
          # Analyze each file
          cd mcp-arm-reviewer
          uv run python ../scripts/validate_pr.py $CHANGED_FILES
      
      - name: Upload Results
        if: failure()
        uses: actions/upload-artifact@v3
        with:
          name: arm-compliance-report
          path: arm-compliance-report.json

Azure DevOps Pipeline

Create azure-pipelines.yml:

trigger:
  branches:
    include:
      - main
      - feature/*
  paths:
    include:
      - '**/*.json'
      - '**/*.yaml'
      - '**/*.tsp'

pool:
  vmImage: 'ubuntu-latest'

steps:
- task: UsePythonVersion@0
  inputs:
    versionSpec: '3.13'
  displayName: 'Use Python 3.13'

- script: |
    curl -LsSf https://astral.sh/uv/install.sh | sh
    source $HOME/.cargo/env
  displayName: 'Install uv'

- script: |
    cd mcp-arm-reviewer
    uv sync
  displayName: 'Install ARM Reviewer'

- script: |
    cd mcp-arm-reviewer
    uv run python ../scripts/validate_workspace.py $(Build.SourcesDirectory)
  displayName: 'Run ARM Compliance Check'

- task: PublishTestResults@2
  condition: succeededOrFailed()
  inputs:
    testResultsFormat: 'JUnit'
    testResultsFiles: '**/arm-compliance-results.xml'

Best Practices

1. Early and Often

  • Run analysis after every significant change
  • Don't wait until PR time to check compliance
  • Catch issues when they're easy to fix

2. Understand Error Priorities

  • Errors: Must fix before PR
  • Warnings: Should fix, document if intentional
  • Info: Consider for best practices

3. Use Fix Suggestions

  • Don't guess at fixes
  • Use the arm-fix-suggestions prompt
  • Follow provided examples

4. Maintain Clean Baseline

  • Start new features with clean compliance
  • Don't add to technical debt
  • Regular workspace scans to prevent drift

5. Document Exceptions

If you must suppress a warning:

// Note: MISSING_PAGINATION suppressed - single-item operation
"x-ms-arm-reviewer-suppress": ["MISSING_PAGINATION"]

Troubleshooting

No Issues Found But PR Review Found Problems

Cause: Analysis only checks what you tell it to analyze

Solution:

  • Use analyze-workspace instead of analyze-file
  • Ensure you're analyzing the final generated OpenAPI, not just source
  • Check if TypeSpec compilation is generating different output

Too Many False Positives

Cause: Rules may need tuning for your specific service

Solution:

  • Review warning vs error classifications
  • Use filters: severity: "error" to focus on critical issues
  • Provide feedback for rule improvements

Performance Issues on Large Workspaces

Cause: Analyzing hundreds of files can be slow

Solution:

  • Use analyze-file for individual file checks during development
  • Use analyze-workspace only for pre-PR validation
  • Consider caching results and only analyzing changed files

Quick Reference

Common Commands

# Analyze single file
analyze-file: {file_path: "spec.json", content: "..."}

# Analyze workspace
analyze-workspace: {workspace_path: "./specifications"}

# Get error summary
arm-compliance-summary: {severity: "error"}

# Get fix suggestions for specific error
arm-fix-suggestions: {error_code: "MISSING_ARM_ID_DETAILS"}

# Get fixes for specific file
arm-fix-suggestions: {file_path: "example.json"}

# Clear cached results
clear-analysis: {}

Error Code Quick Reference

Code Fix Time Breaking? Action
MISSING_ARM_ID_DETAILS 5 min No Add extension with resource types
UNIFORM_VERSIONING_VIOLATION Hours Maybe Align all API versions or split service
API_VERSION_FORMAT 2 min No Change to YYYY-MM-DD format
MISSING_AZURE_RESOURCE 2 min No Add x-ms-azure-resource: true
MISSING_PAGINATION 10 min No Add x-ms-pageable extension
MISSING_REQUIRED_PROPERTY 5 min Maybe Add id/name/type properties

Support & Resources


Summary

The ARM Reviewer MCP server fits into your development workflow as an early warning system that catches compliance issues before they become PR blockers. By integrating it into your daily development process—whether through Claude Desktop, command-line tools, or CI/CD pipelines—you ensure ARM compliance from day one and save hours in the review process.

Key Workflow Principle: Analyze early, fix immediately, submit with confidence. ✅