Adding usage analytics#114
Merged
Merged
Conversation
Signed-off-by: Vishnu Challa <vchalla@redhat.com>
There was a problem hiding this comment.
Pull request overview
This PR adds a usage analytics dashboard (served from docs/) and a companion script to pull aggregated telemetry from Elasticsearch into a static JSON file suitable for GitHub Pages.
Changes:
- Add a Python script to query Elasticsearch aggregations and generate
docs/analytics-data.json. - Add a new
docs/analytics.htmldashboard with supporting JS/CSS to visualize KPIs, usage, reliability, latency, and token cost. - Link the new Analytics page from the docs homepage and ignore the generated JSON output in git.
Reviewed changes
Copilot reviewed 5 out of 6 changed files in this pull request and generated 12 comments.
Show a summary per file
| File | Description |
|---|---|
| scripts/fetch_analytics.py | Queries ES and writes a static aggregated JSON payload for the dashboard. |
| docs/js/analytics.js | Fetches live ES data or static JSON and renders KPIs, charts, tables, and configuration UI. |
| docs/analytics.html | Adds the analytics dashboard page markup and configuration modal. |
| docs/css/analytics.css | Adds styling for the dashboard to match the site’s dark theme. |
| docs/index.html | Adds a navbar link to the analytics dashboard. |
| .gitignore | Ignores the generated docs/analytics-data.json output file. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+38
to
+42
| def get_base_url(): | ||
| parsed = urlparse(ES_URL) | ||
| if parsed.username: | ||
| return f"{parsed.scheme}://{parsed.hostname}" + (f":{parsed.port}" if parsed.port else "") | ||
| return ES_URL.rstrip("/") |
Comment on lines
+244
to
+251
| function renderValueSection(data) { | ||
| const cmds = data.features.commands; | ||
| const analyzeCount = cmds.filter(c => ['auto_analyze', 'auto_viz'].includes(c.command)).reduce((s, c) => s + c.count, 0); | ||
| const prCount = cmds.find(c => c.command === 'analyze_pr')?.count || 0; | ||
| const summaryCount = cmds.filter(c => ['summarize', 'perf_summary'].includes(c.command)).reduce((s, c) => s + c.count, 0); | ||
| const hoursSaved = ((analyzeCount * CONFIG.minutesSavedPerAnalysis) + (prCount * CONFIG.minutesSavedPerPRReview)) / 60; | ||
| const days = data.reliability.over_time.length || 1; | ||
|
|
Comment on lines
+480
to
+495
| function renderCostTable(data) { | ||
| const costPer1k = getCostPer1k(); | ||
| let totalCost = 0; | ||
| const tbody = document.getElementById('cost-table-body'); | ||
| tbody.innerHTML = ''; | ||
| for (const cmd of data.tokens.by_command) { | ||
| const cost = (cmd.tokens / 1000) * costPer1k; | ||
| totalCost += cost; | ||
| const tr = document.createElement('tr'); | ||
| tr.innerHTML = '<td>' + cmd.command + '</td><td>' + fmt(cmd.count) + '</td><td>' + fmt(cmd.tokens) + '</td><td>' + fmt(cmd.count > 0 ? Math.round(cmd.tokens / cmd.count) : 0) + '</td><td>' + fmtCost(cost) + '</td>'; | ||
| tbody.appendChild(tr); | ||
| } | ||
| document.getElementById('cost-total').textContent = fmtCost(totalCost); | ||
| document.getElementById('cost-per-request').textContent = fmtCost(totalCost / (data.kpis.total_requests || 1)); | ||
| document.getElementById('cost-per-day').textContent = fmtCost(totalCost / (data.tokens.over_time.length || 1)); | ||
| } |
Comment on lines
+66
to
+67
| function loadConfig() { try { return JSON.parse(localStorage.getItem(CONFIG.storageKey)) || {}; } catch { return {}; } } | ||
| function saveConfig(cfg) { localStorage.setItem(CONFIG.storageKey, JSON.stringify(cfg)); } |
Comment on lines
+583
to
+585
| document.getElementById('config-clear').addEventListener('click', () => { | ||
| localStorage.removeItem(CONFIG.storageKey); | ||
| document.getElementById('es-url').value = ''; |
| <div class="modal"> | ||
| <div class="modal-header"> | ||
| <h3>Configuration</h3> | ||
| <button class="modal-close" id="config-close">×</button> |
Comment on lines
+248
to
+251
| <strong>Error reasons not captured in telemetry</strong> | ||
| <p>All current failures come from <code>auto_viz</code> with no <code>error_type</code> or <code>error_message</code> recorded. The telemetry client sets <code>success=false</code> but does not populate the error fields for this command.</p> | ||
| <p>To surface root causes here, the bot's telemetry emit calls need to include <code>error_type</code> and <code>error_message</code> when marking events as failed.</p> | ||
| </div> |
| const barW = c.total > 0 ? Math.round((c.success / c.total) * 100) : 0; | ||
| cmdRows += | ||
| '<tr>' + | ||
| '<td>' + c.command + '</td>' + |
| card.className = 'team-detail-card'; | ||
| card.innerHTML = | ||
| '<div class="tdc-header">' + | ||
| '<h3 class="tdc-name">' + t.team + '</h3>' + |
Comment on lines
+423
to
+434
| const failCmds = (data.reliability.fail_by_command || []).filter(c => c.count > 0); | ||
| if (failCmds.length) { | ||
| makeOrUpdate('chart-fail-by-cmd', { | ||
| type: 'bar', | ||
| data: { | ||
| labels: failCmds.map(c => c.command), | ||
| datasets: [{ label: 'Failures', data: failCmds.map(c => c.count), backgroundColor: CONFIG.colors.map(c => c + 'aa'), borderRadius: 4, maxBarThickness: 48 }], | ||
| }, | ||
| options: { ...chartDefaults, plugins: { ...chartDefaults.plugins, legend: { display: false } } }, | ||
| }); | ||
| } | ||
|
|
Collaborator
|
/lgtm |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adding analytics dashboard on top of the usage metrics.
Testing
Tested and verified in local.