Agent Debug: Converge debug data with JSONL as single source of truth by vijayupadya · Pull Request #4954 · microsoft/vscode-copilot-chat

vijayupadya · 2026-04-03T05:35:46Z

Converges the two independent chat debug event systems (in-memory span store for debug panel + JSONL file logger for troubleshoot skill) into a single architecture where JSONL is the source of truth and the debug panel reads from it.

Chat Debug Event System: Convergence

Problem (Before)

Two independent systems consumed OTel debug events via separate subscriptions and separate conversion paths:

OTelChatDebugLogProvider — in-memory 10K span array → Debug Panel UI
ChatDebugFileLoggerService — JSONL on disk → Troubleshoot Skill

Architecture (Before)

┌─────────────────────────────────────────────────────────────────────────────┐
│                        IOTelService (Event Source)                          │
│                                                                             │
│          onDidCompleteSpan              onDidEmitSpanEvent                  │
│               │    │                        │    │                          │
└───────────────┼────┼────────────────────────┼────┼──────────────────────────┘
                │    │                        │    │
       ┌────────┘    └──────────┐    ┌────────┘    └──────────┐
       │                        │    │                        │
       ▼                        ▼    ▼                        ▼
┌──────────────────────────────────────┐  ┌──────────────────────────────────┐
│  OTelChatDebugLogProvider            │  │  ChatDebugFileLoggerService      │
│  (Debug Panel)                       │  │  (File Logger)                   │
│                                      │  │                                  │
│  ┌──────────────────────────┐        │  │  ┌──────────────────────────┐    │
│  │ Direct OTel Subscription │        │  │  │ Direct OTel Subscription │    │
│  └────────────┬─────────────┘        │  │  └────────────┬─────────────┘    │
│               │                      │  │               │                  │
│               ▼                      │  │               ▼                  │
│  ┌──────────────────────────┐        │  │  ┌──────────────────────────┐    │
│  │ _allSpans[]              │        │  │  │ _spanToEntry()           │    │
│  │ (in-memory, max 10K)     │        │  │  └────────────┬─────────────┘    │
│  │                          │        │  │               │                  │
│  └────────────┬─────────────┘        │  │               ▼                  │
│               │                      │  │  ┌──────────────────────────┐    │
│  ┌────────────┼─────────────┐        │  │  │ Buffer (4s flush)        │    │
│  │ _sessionSpanIndices      │        │  │  └────────────┬─────────────┘    │
│  │ _spanIdIndex             │        │  │               │                  │
│  │ _sessionOrder            │        │  │               ▼                  │
│  └────────────┬─────────────┘        │  │  ┌──────────────────────────┐    │
│               │                      │  │  │ JSONL on disk            │    │
│  ┌────────────┼─────────────┐        │  │  │ + system_prompt_N.json   │    │
│  │ LRU Eviction +           │        │  │  │ + tools_N.json           │    │
│  │ Async Compaction         │        │  │  │ + models.json            │    │
│  │                          │        │  │  └────────────┬─────────────┘    │
│  └────────────┬─────────────┘        │  │               │                  │
│               │                      │  └───────────────┼──────────────────┘
│               ▼                      │                  │
│  ┌──────────────────────────┐        │                  ▼
│  │ completedSpanToDebugEvent│        │  ┌──────────────────────────────────┐
│  │ extractConversationEvents│        │  │  Troubleshoot Skill              │
│  │ spanEventToUserMessage   │        │  │  (reads JSONL via grep/jq)       │
│  └────────────┬─────────────┘        │  └──────────────────────────────────┘
│               │                      │
│               ▼                      │
│  ┌──────────────────────────┐        │   ┌─────────────────────────────────┐
│  │ Debug Panel UI           │        │   │ VS Code Core API                │
│  └──────────────────────────┘        │   │ onDidReceiveChatDebugEvent      │
│                                      │   │ (discovery, skills)             │
└──────────────────────────────────────┘   │         │                       │
                                           └─────────┼───────────────────────┘
                                                     │
                                                     └──► File Logger only
                                                          (panel doesn't get these)

Architecture (After)

┌─────────────────────────────────────────────────────────────────────────────┐
│                        IOTelService (Event Source)                          │
│          onDidCompleteSpan              onDidEmitSpanEvent                  │
│                    │                           │                            │
└────────────────────┼───────────────────────────┼────────────────────────────┘
                     │                           │
                     └──────────┬────────────────┘
                                │
                                ▼
┌─────────────────────────────────────────────────────────────────────────────┐
│  ChatDebugFileLoggerService (unchanged OTel subscription)                   │
│                                                                             │
│  ┌──────────────────────────┐     ┌──────────────────────────────────────┐  │
│  │ OTel Subscription        │     │ VS Code Core API                     │  │
│  │ (same as before)         │     │ onDidReceiveChatDebugEvent           │  │
│  └────────────┬─────────────┘     │ (discovery events)                   │  │
│               │                   └──────────────┬───────────────────────┘  │
│               │                                  │                          │
│               ▼                                  │                          │
│  ┌──────────────────────────┐                    │                          │
│  │ _spanToEntry()           │◄───────────────────┘                          │
│  └────────────┬─────────────┘                                               │
│               │                                                             │
│               ▼                                                             │
│  ┌──────────────────────────┐     ┌──────────────────────────────────┐      │
│  │ _bufferEntry()           │────►│ onDidEmitEntry(sessionId, entry) │      │
│  └────────────┬─────────────┘     │ (Emitter, fires sync pre-flush)  │      │
│               │                   └────────────────┬─────────────────┘      │
│               ▼                                    │                        │
│  ┌──────────────────────────┐                      │                        │
│  │ Flush → JSONL on disk    │                      │                        │
│  │ + system_prompt_N.json   │                      │                        │
│  │ + tools_N.json           │                      │                        │
│  └──────────────────────────┘                      │                        │
│                                                    │                        │
│  Read APIs:                                        │                        │
│  ┌──────────────────────────┐                      │                        │
│  │ readTailEntries(sid, N)  │  ← reads last N      │                        │
│  │ streamEntries(sid, cb)   │  ← streaming parser  │                        │
│  │ readEntries(sid)         │  ← full read         │                        │
│  └────────────┬─────────────┘                      │                        │
│               │                                    │                        │
└───────────────┼────────────────────────────────────┼────────────────────────┘
                │                                    │
                │  (on panel open:                   │  (while panel open:
                │   read history from JSONL)         │   live events)
                │                                    │
                ▼                                    ▼
┌─────────────────────────────────────────────────────────────────────┐
│  OTelChatDebugLogProvider (STATELESS)                               │
│                                                                     │
│        1 — Fast initial load:                                       │
│    readTailEntries(sid, 500) → latest 500 events → return to core   │
│                                                                     │
│        2 — Background backfill:                                     │
│    streamEntries(sid, cb) → stream older events via progress        │
│    (deduplicated; core binary-inserts by timestamp)                 │
│                                                                     │
│        3 — Live events:                                             │
│    onDidEmitEntry → filter by session → stream to UI                │
│                                                                     │
│  Detail resolution:                                                 │
│    LRU cache (1000 entries) → on miss, scan JSONL on disk           │
│    Reads details (e.g. system_prompt_N.json / tools_N.json on demand|
│                                                                     │
│  On panel close: clear all caches → zero memory                     │
└─────────────────────────────────────────────────────────────────────┘

Unchanged:
┌──────────────────────────────────┐
│  Troubleshoot Skill              │
│  (reads JSONL via grep/jq)       │
│  (same files, no changes)        │
└──────────────────────────────────┘

Extensibility

Future consumer requires one subscription:

//IDebugLogEntry is now the canonical event format. Future consumers (e.g., SQLite projection) subscribe with one line:
fileLogger.onDidEmitEntry(({ sessionId, entry }) => store.insert(sessionId, entry));

Copilot

Pull request overview

Converges agent/chat debug logging onto a single JSONL-on-disk source of truth by extending the debug file logger contract (read/stream + live entry events) and updating the VS Code chat debug panel provider to read/stream from those JSONL logs instead of maintaining its own in-memory span store.

Changes:

Extends IChatDebugFileLoggerService with onDidEmitEntry plus read/stream APIs and promotes IDebugLogEntry to the platform layer as the canonical schema.
Refactors the chat debug panel provider to load history from JSONL (tail-first + background backfill) and stream live events from the logger’s new event.
Updates the node logger implementation to support entry streaming/tail reads and to better handle resumed sessions and early tool definition events.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 5 comments.

File	Description
src/platform/chat/common/chatDebugFileLoggerService.ts	Adds canonical `IDebugLogEntry` type and new logger APIs for live emission + reading/streaming JSONL.
src/extension/chat/vscode-node/chatDebugFileLoggerService.ts	Implements `onDidEmitEntry` + JSONL read/tail/stream methods; adjusts session resume + tools handling.
src/extension/trajectory/vscode-node/otelSpanToChatDebugEvent.ts	Adds converters from `IDebugLogEntry` to VS Code debug panel event types and detail resolvers.
src/extension/trajectory/vscode-node/otelChatDebugLogProvider.ts	Replaces in-memory span storage with JSONL-backed loading/streaming, deduping, and detail resolution from disk.

src/extension/trajectory/vscode-node/otelChatDebugLogProvider.ts

src/extension/chat/vscode-node/chatDebugFileLoggerService.ts

src/extension/trajectory/vscode-node/otelChatDebugLogProvider.ts

src/extension/chat/vscode-node/chatDebugFileLoggerService.ts

OrenMe · 2026-04-03T13:08:57Z

Will this ensure that sessions are kept even between restarts so they can be viewed in the agent debug logs?

vijayupadya · 2026-04-03T18:23:34Z

Will this ensure that sessions are kept even between restarts so they can be viewed in the agent debug logs?

Yes, that is one of the main benefits of this change.

OrenMe · 2026-04-03T19:03:40Z

Nice, thanks for working on this!
I originally created https://marketplace.visualstudio.com/items?itemName=OrenMelamed.chat-workflow-viewer

It is using the jsonl files so you can access the data as long as files are there so this is a really good addition

Copilot

Pull request overview

This PR converges the chat debug event system so JSONL files become the single source of truth for both the Debug Panel UI and the troubleshoot/diagnostics flows, removing the prior split between in-memory span storage and file logging.

Changes:

Adds a canonical IDebugLogEntry schema plus realtime + read APIs (onDidEmitEntry, readEntries, readTailEntries, streamEntries) to the chat debug file logger service.
Refactors OTelChatDebugLogProvider to be stateless and backed by JSONL (fast tail load + streamed backfill + live streaming via onDidEmitEntry).
Updates settings UX to deprecate the old toggle and document the canonical file-logging toggle.

Show a summary per file

File	Description
src/platform/configuration/common/configurationService.ts	Marks debug logging toggle relationship and deprecates the old setting during transition.
src/platform/chat/common/chatDebugFileLoggerService.ts	Extends the service interface with canonical entry schema + live/read APIs.
src/extension/chat/vscode-node/chatDebugFileLoggerService.ts	Implements the new APIs, adds realtime emission, and resume behavior changes.
src/extension/chat/vscode-node/test/chatDebugFileLoggerService.spec.ts	Adds unit coverage for the new read/stream APIs and realtime emitter.
src/extension/trajectory/vscode-node/otelSpanToChatDebugEvent.ts	Adds converters/resolvers from `IDebugLogEntry` to VS Code debug panel events.
src/extension/trajectory/vscode-node/otelChatDebugLogProvider.ts	Switches debug panel provider to read/stream from JSONL and subscribe to live entries.
src/extension/trajectory/vscode-node/test/debugLogEntryConverters.spec.ts	Adds tests for the new entry→event converter and dedup key behavior.
package.nls.json	Updates setting descriptions and introduces deprecation messaging text.
package.json	Adds a deprecationMessage for the deprecated setting.

Copilot's findings

Files reviewed: 9/9 changed files
Comments generated: 2

src/extension/trajectory/vscode-node/otelChatDebugLogProvider.ts

src/extension/chat/vscode-node/chatDebugFileLoggerService.ts

Copilot

Pull request overview

This PR converges the agent/chat debug pipeline so JSONL log files become the single source of truth: the debug panel now reads historical + live events from the file logger instead of maintaining its own in-memory span store.

Changes:

Extends IChatDebugFileLoggerService with live-entry events and read APIs (onDidEmitEntry, readEntries, readTailEntries, streamEntries) and updates the node implementation accordingly.
Reworks OTelChatDebugLogProvider to be stateless: initial tail load + background backfill from JSONL, plus live streaming via onDidEmitEntry, with on-demand detail resolution from disk.
Adds child-session routing attributes for tool/subagent spans and updates settings metadata (deprecation messaging + canonical toggle).

Show a summary per file

File	Description
src/platform/configuration/common/configurationService.ts	Marks legacy debug setting as deprecated; clarifies canonical toggle.
src/platform/chat/common/chatDebugFileLoggerService.ts	Expands file logger interface to support live streaming + read APIs; defines canonical `IDebugLogEntry`.
src/extension/chat/vscode-node/chatDebugFileLoggerService.ts	Implements new interface APIs, child-session routing, resume/run-index stamping, and emits entries pre-flush.
src/extension/chat/vscode-node/test/chatDebugFileLoggerService.spec.ts	Adds/adjusts tests to cover the new read/stream/emit behaviors.
src/extension/trajectory/vscode-node/otelSpanToChatDebugEvent.ts	Adds converters from `IDebugLogEntry` → VS Code debug panel events + detail resolution helpers.
src/extension/trajectory/vscode-node/otelChatDebugLogProvider.ts	Switches provider to JSONL-backed loading/streaming and disk-based detail resolution; keeps OTLP export/import support.
src/extension/trajectory/vscode-node/test/debugLogEntryConverters.spec.ts	Adds unit coverage for entry conversion/dedup key behavior.
src/extension/tools/vscode-node/toolsService.ts	Stamps parent session / label attributes onto tool spans for child-session routing.
src/extension/intents/node/toolCallingLoop.ts	Registers subagent sessions as child sessions with the file logger prior to span completion.
package.nls.json	Updates setting descriptions and adds deprecation text.
package.json	Adds deprecation message to legacy setting contribution.

Copilot's findings

Files reviewed: 11/11 changed files
Comments generated: 4

src/extension/trajectory/vscode-node/otelSpanToChatDebugEvent.ts

src/extension/chat/vscode-node/chatDebugFileLoggerService.ts

src/extension/trajectory/vscode-node/otelChatDebugLogProvider.ts

converge debug data

12d204d

vijayupadya requested a review from Copilot April 3, 2026 05:35

Copilot started reviewing on behalf of vijayupadya April 3, 2026 05:36 View session

Copilot AI reviewed Apr 3, 2026

View reviewed changes

Feedback updates

f2679be

add ver field

d2d5bc7

exp update

76a6577

vijayupadya requested a review from Copilot April 3, 2026 20:33

Copilot started reviewing on behalf of vijayupadya April 3, 2026 20:34 View session

Copilot AI reviewed Apr 3, 2026

View reviewed changes

src/extension/trajectory/vscode-node/otelChatDebugLogProvider.ts Outdated Show resolved Hide resolved

src/extension/chat/vscode-node/chatDebugFileLoggerService.ts Show resolved Hide resolved

subagent fix

0d66122

vijayupadya requested a review from Copilot April 5, 2026 05:41

Copilot started reviewing on behalf of vijayupadya April 5, 2026 05:42 View session

Copilot AI reviewed Apr 5, 2026

View reviewed changes

Few fixes and test

37305e1

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Agent Debug: Converge debug data with JSONL as single source of truth#4954

Agent Debug: Converge debug data with JSONL as single source of truth#4954
vijayupadya wants to merge 6 commits intomainfrom
vijayu/dgbEventsConvergence

vijayupadya commented Apr 3, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

OrenMe commented Apr 3, 2026

Uh oh!

vijayupadya commented Apr 3, 2026

Uh oh!

OrenMe commented Apr 3, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

vijayupadya commented Apr 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Chat Debug Event System: Convergence

Problem (Before)

Architecture (Before)

Architecture (After)

Extensibility

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

OrenMe commented Apr 3, 2026

Uh oh!

vijayupadya commented Apr 3, 2026

Uh oh!

OrenMe commented Apr 3, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Copilot's findings

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Copilot's findings

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

vijayupadya commented Apr 3, 2026 •

edited

Loading