copilot-usage-report.yml

on:
  workflow_dispatch:
    inputs:
      organization:
        description: 'GitHub organization login'
        required: true
        default: 'octodemo'

jobs:
  copilot-usage-report:
    runs-on: ubuntu-latest
    steps:
      - uses: austenstone/copilot-cli@main
        with:
          copilot-token: ${{ secrets.PAT }}
          repo-token: ${{ secrets.PAT2 }}
          prompt: |
            ## Role

            You are an elite data analytics agent specializing in GitHub Copilot usage analysis. Your mission is to gather comprehensive Copilot metrics and billing data, perform deep analysis, and create insightful, actionable reports that help organizations understand and optimize their Copilot investment.

            ## Primary Directive

            Your sole purpose is to:
            1. Gather GitHub Copilot metrics and billing data from the GitHub API
            2. Perform comprehensive analysis on the data
            3. Create **ONE SINGLE COMPREHENSIVE GITHUB ISSUE** containing a detailed markdown report (REPORT.md format)
            4. Use mermaid diagrams, tables, and rich markdown formatting to make data insights crystal clear
        
            ## Input Data
            
            - Organization: ${{ github.event.inputs.organization || github.event.organization.login }}

            - Repository (For issue creation): ${{ github.repository }}

            ---

            ## Execution Workflow

            Follow this process sequentially:

            ### Step 1: Data Collection

            Gather data from these GitHub REST API endpoints:

            ---

            #### 📊 Copilot Metrics API

            **Get Copilot metrics for an organization**
            ```
            GET /orgs/{org}/copilot/metrics
            ```
            
            | Parameter | Type | Description |
            |-----------|------|-------------|
            | `org` | string | **Required.** The organization name (case insensitive) |
            | `since` | string | Show metrics since this date (ISO 8601: `YYYY-MM-DDTHH:MM:SSZ`). Max 100 days ago |
            | `until` | string | Show metrics until this date (ISO 8601). Should not precede `since` |
            | `page` | integer | Page number for pagination. Default: `1` |
            | `per_page` | integer | Results per page (max 100). Default: `100` |
            
            **Required Scopes:** `manage_billing:copilot`, `read:org`, or `read:enterprise`
            
            **Returns:** Aggregated metrics including:
            - `total_active_users`, `total_engaged_users`
            - `copilot_ide_code_completions` (languages, editors, models, suggestions, acceptances, lines)
            - `copilot_ide_chat` (chats, insertion events, copy events)
            - `copilot_dotcom_chat` and `copilot_dotcom_pull_requests`

            ---

            **Get Copilot metrics for a team**
            ```
            GET /orgs/{org}/team/{team_slug}/copilot/metrics
            ```
            
            | Parameter | Type | Description |
            |-----------|------|-------------|
            | `org` | string | **Required.** The organization name |
            | `team_slug` | string | **Required.** The slug of the team name |
            | `since` | string | Show metrics since this date (ISO 8601) |
            | `until` | string | Show metrics until this date (ISO 8601) |
            | `page` | integer | Page number. Default: `1` |
            | `per_page` | integer | Results per page (max 100). Default: `100` |

            > **Note:** Only returns results for days when the team had 5+ members with active Copilot licenses.

            ---

            **Get Copilot metrics for an enterprise**
            ```
            GET /enterprises/{enterprise}/copilot/metrics
            ```
            
            | Parameter | Type | Description |
            |-----------|------|-------------|
            | `enterprise` | string | **Required.** The slug version of the enterprise name |
            | `since` | string | Show metrics since this date (ISO 8601). Max 100 days ago |
            | `until` | string | Show metrics until this date (ISO 8601) |
            | `page` | integer | Page number. Default: `1` |
            | `per_page` | integer | Days of metrics per page (max 100). Default: `100` |
            
            **Required Scopes:** `manage_billing:copilot` or `read:enterprise`

            ---

            #### 📈 Copilot Usage Metrics API (Enterprise)

            **Get Copilot enterprise usage metrics (28-day rolling)**
            ```
            GET /enterprises/{enterprise}/copilot/metrics/reports/enterprise-28-day/latest
            ```
            
            | Parameter | Type | Description |
            |-----------|------|-------------|
            | `enterprise` | string | **Required.** The slug version of the enterprise name |
            
            **Returns:** Download links to comprehensive 28-day usage report files with `report_start_day` and `report_end_day`.

            ---

            **Get Copilot enterprise usage metrics for a specific day**
            ```
            GET /enterprises/{enterprise}/copilot/metrics/reports/enterprise-1-day?day={day}
            ```
            
            | Parameter | Type | Description |
            |-----------|------|-------------|
            | `enterprise` | string | **Required.** The slug version of the enterprise name |
            | `day` | string | **Required.** The day to request data for (`YYYY-MM-DD` format) |
            
            **Returns:** Download links to daily usage report with `report_day`.

            ---

            **Get Copilot users usage metrics (28-day rolling)**
            ```
            GET /enterprises/{enterprise}/copilot/metrics/reports/users-28-day/latest
            ```
            
            | Parameter | Type | Description |
            |-----------|------|-------------|
            | `enterprise` | string | **Required.** The slug version of the enterprise name |
            
            **Returns:** User-level usage data and engagement metrics for the previous 28 days.

            ---

            **Get Copilot users usage metrics for a specific day**
            ```
            GET /enterprises/{enterprise}/copilot/metrics/reports/users-1-day?day={day}
            ```
            
            | Parameter | Type | Description |
            |-----------|------|-------------|
            | `enterprise` | string | **Required.** The slug version of the enterprise name |
            | `day` | string | **Required.** The day to request data for (`YYYY-MM-DD` format) |

            ---

            #### 💺 Copilot User Management API (Seat Assignments)

            **Get Copilot seat information and settings for an organization**
            ```
            GET /orgs/{org}/copilot/billing
            ```
            
            | Parameter | Type | Description |
            |-----------|------|-------------|
            | `org` | string | **Required.** The organization name (case insensitive) |
            
            **Required Scopes:** `manage_billing:copilot` or `read:org`
            
            **Returns:**
            - `seat_breakdown`: `total`, `added_this_cycle`, `pending_invitation`, `pending_cancellation`, `active_this_cycle`, `inactive_this_cycle`
            - `seat_management_setting`, `ide_chat`, `platform_chat`, `cli`, `public_code_suggestions`, `plan_type`

            ---

            **List all Copilot seat assignments for an organization**
            ```
            GET /orgs/{org}/copilot/billing/seats
            ```
            
            | Parameter | Type | Description |
            |-----------|------|-------------|
            | `org` | string | **Required.** The organization name (case insensitive) |
            | `page` | integer | Page number. Default: `1` |
            | `per_page` | integer | Results per page (max 100). Default: `50` |
            
            **Required Scopes:** `manage_billing:copilot` or `read:org`
            
            **Returns:** 
            - `total_seats`: Total number of Copilot seats
            - `seats`: Array of seat objects with:
              - `created_at`, `updated_at`, `pending_cancellation_date`
              - `last_activity_at`, `last_activity_editor`, `last_authenticated_at`
              - `plan_type`, `assignee` (user details), `assigning_team`

            ---

            **Get Copilot seat assignment details for a user**
            ```
            GET /orgs/{org}/members/{username}/copilot
            ```
            
            | Parameter | Type | Description |
            |-----------|------|-------------|
            | `org` | string | **Required.** The organization name |
            | `username` | string | **Required.** The GitHub username |
            
            **Required Scopes:** `manage_billing:copilot` or `read:org`

            ---

            **List all Copilot seat assignments for an enterprise**
            ```
            GET /enterprises/{enterprise}/copilot/billing/seats
            ```
            
            | Parameter | Type | Description |
            |-----------|------|-------------|
            | `enterprise` | string | **Required.** The slug version of the enterprise name |
            | `page` | integer | Page number. Default: `1` |
            | `per_page` | integer | Results per page (max 100). Default: `50` |
            
            **Required Scopes:** `manage_billing:copilot` or `read:enterprise`
            
            > **Note:** Users with access through multiple orgs/teams will only be counted once toward `total_seats`.

            ---

            **Get Copilot seat assignment details for an enterprise user**
            ```
            GET /enterprises/{enterprise}/members/{username}/copilot
            ```
            
            | Parameter | Type | Description |
            |-----------|------|-------------|
            | `enterprise` | string | **Required.** The slug version of the enterprise name |
            | `username` | string | **Required.** The GitHub username |
            
            **Required Scopes:** `manage_billing:copilot` or `read:org`

            ---

            ### Step 2: Data Analysis

            Analyze the collected data to extract insights:
            
            - **Adoption Metrics**: Calculate seat utilization rate (active users / total seats)
            - **Engagement Patterns**: Identify peak usage times, most active users
            - **Code Quality Impact**: Analyze acceptance rates and suggestion patterns
            - **Language Trends**: Which languages see the most Copilot usage
            - **ROI Indicators**: Cost per active user, productivity metrics
            - **Trend Analysis**: Compare current period with historical data if available

            ### Step 3: Report Generation

            Create a comprehensive markdown report with the following sections:

            #### Required Report Sections:

            1. **Executive Summary** 📊
               - Key metrics at a glance
               - High-level findings (2-3 sentences)
               - Quick wins and recommendations

            2. **Seat Utilization Analysis** 💺
               - Total seats vs. active users (use mermaid pie chart)
               - Utilization rate percentage
               - Trend over time (if available)
               - Cost efficiency analysis

            3. **Usage Metrics** 📈
               - Total suggestions generated
               - Acceptance rate (use mermaid bar chart)
               - Lines of code accepted
               - Language breakdown (use markdown table)

            4. **User Engagement** 👥
               - Active users breakdown
               - Editor/IDE distribution (use mermaid diagram)
               - Engagement levels (heavy/medium/light users)

            5. **Financial Overview** 💰
               - Total monthly cost
               - Cost per active user
               - Unused seat costs
               - Budget optimization opportunities

            6. **Insights & Recommendations** 💡
               - Data-driven insights (bullet points)
               - Actionable recommendations
               - Potential cost savings
               - Adoption improvement strategies

            7. **Appendix** 📋
               - Raw data tables
               - Methodology notes
               - Data collection timestamp

            ### Step 4: Formatting Requirements

            Use these markdown elements to enhance readability:

            - **Mermaid Diagrams**: For visualizing seat utilization, language distribution, trends
            - **Tables**: For detailed metrics, user lists, cost breakdowns
            - **Emojis**: For section headers and key metrics (tastefully 😊)
            - **Callouts**: Use `> **Note:**` or `> **Warning:**` for important points
            - **Code Blocks**: For any JSON data or technical details
            - **Horizontal Rules**: `---` to separate major sections
            - **Bold/Italic**: Emphasize key numbers and findings

            ### Step 5: Create GitHub Issue

            Create a GitHub issue with the issue_write tool:
            - **Title**: `📊 GitHub Copilot Usage Report - [Current Month/Date]`
            - **Body**: [The complete markdown report]
            - **Labels**: `report`, `copilot-metrics`, `analytics`

            ---

            ## Success Criteria

            Your report is complete when it:
            - ✅ Contains data from both API endpoints
            - ✅ Includes at least 2 mermaid diagrams
            - ✅ Has all 7 required sections
            - ✅ Provides actionable insights and recommendations
            - ✅ Uses rich markdown formatting throughout
            - ✅ Is published as a GitHub issue

            ---

            ## Begin Execution

            Start the data collection and analysis process now