Update Chat Agent Documentation to Include Tracing Feature

[dan-niles]

Purpose

Resolves wso2/product-ballerina-integrator#2136

It updates the chat agent documentation to add steps for enabling and using the tracing feature and align existing documentation with the latest UI changes introduced in BI 1.5.

Summary by CodeRabbit

• Documentation
  • Updated AI agent guide to a provider-agnostic onboarding, with WSO2 Integrator: BI as the default and optional external providers; default model set to gpt-4.1.
  • Moved API-key handling to configurable values with a first-run prompt workflow instead of direct key entry.
  • Reworded memory and tool/function flows (e.g., “Add Memory”, “Use Function”), refreshed visuals/GIFs, and reordered steps.
  • Added built-in tracing walkthrough and external tracing guidance (including Jaeger).

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[coderabbitai]

Walkthrough

Rewrites the chat agents guide to a BI-first, provider-agnostic flow (default WSO2 Integrator: BI, optional external providers), moves API-key handling to VS Code configurables and runtime prompts, updates model/memory/tools flows, introduces built-in tracing and Jaeger guidance, and replaces assets with BI-branded visuals.

Changes

Cohort / File(s)

Summary

Chat Agent Guide (single file) en/docs/integration-guides/ai/agents/introduction-to-chat-agents.md

Converted OpenAI-centric guide to BI-first/provider-agnostic onboarding (default "WSO2 Integrator: BI", optional OpenAI path); removed Config.toml API-key steps and added VS Code configurable + Expression Helper + runtime prompt; updated UI text/branding, model type (ai:GPT_40 → gpt-4.1), memory wording to “Add Memory”, unified tools into a single Functions → "Use Function" flow; added built-in tracing walkthrough and Jaeger external observability guidance; replaced and reorganized GIFs/images and adjusted step ordering and phrasing.

Sequence Diagram(s)

sequenceDiagram
    participant Dev as VS Code (Developer)
    participant Agent as Chat Agent (BI runtime)
    participant Model as Model Provider (WSO2 / External)
    participant Tracing as Tracing UI / Jaeger

    Dev->>Agent: Create/configure agent (provider, model, memory, functions)
    Dev->>Model: Optionally add credentials via VS Code configurable / Expression Helper
    Dev->>Agent: Enable Tracing (VS Code command)
    Dev->>Agent: Send chat message
    Agent->>Model: LLM request
    Model-->>Agent: LLM response
    Agent->>Agent: Invoke Function/Tool (if applicable)
    Agent->>Tracing: Emit built-in traces (tool calls, LLM calls, logs)
    Tracing->>Dev: Display logs / visualization (Show logs)
    alt Forward to Jaeger
      Agent->>Tracing: Forward traces to Jaeger endpoint
      Tracing->>Dev: External visualization in Jaeger
    end

Loading

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10–15 minutes

• Verify step numbering, headings, and internal cross-references.
• Confirm VS Code configurable / Expression Helper instructions and runtime prompt wording.
• Check that renamed GIF/image filenames and BI-branded assets referenced exist and render correctly.
• Validate tracing command names (e.g., "Enable Tracing", "Show logs") and Jaeger configuration instructions.

Poem

🐰 I swapped keys for prompts beneath a BI sky,

Functions now hop in, GIFs winking an eye,
Traces trace footsteps of calls big and small,
The agent replies — I cheer, nibble, and call,
A rabbit applauds: docs polished, workflows spry.

Pre-merge checks and finishing touches

❌ Failed checks (2 warnings)

Check name	Status	Explanation	Resolution
Description check	⚠️ Warning	The PR description includes Purpose with issue reference and goals, but omits required template sections like Approach, User stories, Release note, Documentation, Training, Certification, Marketing, Testing, Security, Samples, Related PRs, and Migrations.	Complete the PR description by filling in missing template sections: Approach (with UI screenshots), User stories, Release note, Documentation links, Training/Certification/Marketing status, and Testing/Security details.
Out of Scope Changes check	⚠️ Warning	While the PR primarily focuses on tracing documentation (issue #2136), it extensively updates documentation to align with BI 1.5 UI changes including WSO2 branding, model provider workflows, and function creation flows, which extend beyond the tracing feature objective.	Clarify whether all BI 1.5 alignment changes are within scope for this PR or if a separate PR should address UI/branding updates independently from tracing feature documentation.

✅ Passed checks (3 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title directly addresses the main change: updating chat agent documentation to include tracing feature, which aligns with the PR's primary objective.
Linked Issues check	✅ Passed	The PR comprehensively addresses issue #2136 requirements: documentation includes steps to create chat agent, enable tracing via VS Code command, use Agent Chat window, and view traces via 'Show logs'.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✨ Finishing touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment

---

📜 Recent review details

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 9f0f82b and a8c9619.

📒 Files selected for processing (1)

• en/docs/integration-guides/ai/agents/introduction-to-chat-agents.md

🚧 Files skipped from review as they are similar to previous changes (1)

• en/docs/integration-guides/ai/agents/introduction-to-chat-agents.md

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 0

🧹 Nitpick comments (1)

en/docs/integration-guides/ai/agents/introduction-to-chat-agents.md (1)

122-126: Clarify Step 8's flow relationship to Step 7.

Step 7 instructs users to click Chat and run the integration, but Step 8 repeats these actions (lines 122-123). While this is necessary because tracing must be enabled before execution, the documentation should explicitly clarify that Step 8 requires a fresh run with tracing enabled, rather than continuing the prior interaction. Consider rewording to emphasize this is a new execution with tracing active.

📜 Review details

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between a2f4ca5 and 8b5b95d.

⛔ Files ignored due to path filters (7)

• en/docs/assets/img/integration-guides/ai/agents/introduction-to-chat-agents/2.create-an-agent.gif is excluded by !**/*.gif
• en/docs/assets/img/integration-guides/ai/agents/introduction-to-chat-agents/3.configure-behaviour.gif is excluded by !**/*.gif
• en/docs/assets/img/integration-guides/ai/agents/introduction-to-chat-agents/4.configure-model.gif is excluded by !**/*.gif
• en/docs/assets/img/integration-guides/ai/agents/introduction-to-chat-agents/5.create-function.gif is excluded by !**/*.gif
• en/docs/assets/img/integration-guides/ai/agents/introduction-to-chat-agents/6.create-agent-tool.gif is excluded by !**/*.gif
• en/docs/assets/img/integration-guides/ai/agents/introduction-to-chat-agents/7.interact-with-agent.gif is excluded by !**/*.gif
• en/docs/assets/img/integration-guides/ai/agents/introduction-to-chat-agents/8.tracing.gif is excluded by !**/*.gif

📒 Files selected for processing (1)

• en/docs/integration-guides/ai/agents/introduction-to-chat-agents.md

🔇 Additional comments (2)

en/docs/integration-guides/ai/agents/introduction-to-chat-agents.md (2)

85-99: Both referenced GIF files exist and are correctly specified. No action required.

---

39-40: Verify VS Code command names are accurate.

The documentation references a command to "Configure default WSO2 Model Provider", but the exact command names in the VS Code extension cannot be definitively confirmed from available public documentation:

• Line 40: Ballerina: Configure default WSO2 model provider
• Line 121: Ballerina: Enable Tracing

These commands must match precisely with what's available in the Ballerina/BI extension, as incorrect names will prevent users from following the tutorial. Confirm these command names are correct in the current version of the extension.

[coderabbitai]

Actionable comments posted: 1

🧹 Nitpick comments (1)

en/docs/integration-guides/ai/agents/introduction-to-chat-agents.md (1)

50-58: Consider clarifying the placement of the configurable steps within Step 4.

The API key configurable setup is currently nested within a note block in the middle of the numbered steps (lines 50-58). While the steps are clear, readers might benefit from slightly clearer demarcation that this is still part of Step 4's model configuration flow, not a separate step. The current note placement could be misinterpreted as optional guidance rather than a recommended best practice for externalizing secrets.

Consider adding a brief introductory phrase like "To securely configure the API key, follow these steps:" immediately before the numbered list, or restructure the note to make the conditional flow more explicit (e.g., "If using OpenAI or another external provider, configure the API key as follows:").

📜 Review details

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between b7321bc and d302b9b.

⛔ Files ignored due to path filters (7)

• en/docs/assets/img/integration-guides/ai/agents/introduction-to-chat-agents/2.create-an-agent.gif is excluded by !**/*.gif
• en/docs/assets/img/integration-guides/ai/agents/introduction-to-chat-agents/3.configure-behaviour.gif is excluded by !**/*.gif
• en/docs/assets/img/integration-guides/ai/agents/introduction-to-chat-agents/4.configure-model.gif is excluded by !**/*.gif
• en/docs/assets/img/integration-guides/ai/agents/introduction-to-chat-agents/5.create-function.gif is excluded by !**/*.gif
• en/docs/assets/img/integration-guides/ai/agents/introduction-to-chat-agents/6.create-agent-tool.gif is excluded by !**/*.gif
• en/docs/assets/img/integration-guides/ai/agents/introduction-to-chat-agents/7.interact-with-agent.gif is excluded by !**/*.gif
• en/docs/assets/img/integration-guides/ai/agents/introduction-to-chat-agents/8.tracing.gif is excluded by !**/*.gif

📒 Files selected for processing (1)

• en/docs/integration-guides/ai/agents/introduction-to-chat-agents.md

🔇 Additional comments (2)

en/docs/integration-guides/ai/agents/introduction-to-chat-agents.md (2)

120-121: Verify the exact VS Code command name for enabling tracing.

The PR objectives reference the command as Ballerina: Enable Tracing (Experimental), but line 121 shows Ballerina: Enable Tracing (without the "(Experimental)" suffix). Confirm which command name is correct in the current version of BI/Ballerina.

---

37-41: Verify the exact VS Code command for configuring the default WSO2 model provider.

Confirm that Ballerina: Configure default WSO2 model provider is the correct command name in the current BI version, or if it has a different name or suffix (e.g., "(Experimental)").

[coderabbitai]

Actionable comments posted: 1

📜 Review details

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between d302b9b and 441c8b9.

⛔ Files ignored due to path filters (7)

• en/docs/assets/img/integration-guides/ai/agents/introduction-to-chat-agents/2.create-an-agent.gif is excluded by !**/*.gif
• en/docs/assets/img/integration-guides/ai/agents/introduction-to-chat-agents/3.configure-behaviour.gif is excluded by !**/*.gif
• en/docs/assets/img/integration-guides/ai/agents/introduction-to-chat-agents/4.configure-model.gif is excluded by !**/*.gif
• en/docs/assets/img/integration-guides/ai/agents/introduction-to-chat-agents/5.create-function.gif is excluded by !**/*.gif
• en/docs/assets/img/integration-guides/ai/agents/introduction-to-chat-agents/6.create-agent-tool.gif is excluded by !**/*.gif
• en/docs/assets/img/integration-guides/ai/agents/introduction-to-chat-agents/7.interact-with-agent.gif is excluded by !**/*.gif
• en/docs/assets/img/integration-guides/ai/agents/introduction-to-chat-agents/8.tracing.gif is excluded by !**/*.gif

📒 Files selected for processing (1)

• en/docs/integration-guides/ai/agents/introduction-to-chat-agents.md

🔇 Additional comments (6)

en/docs/integration-guides/ai/agents/introduction-to-chat-agents.md (6)

10-34: Minor UI naming updates look good.

References to "WSO2 Integrator: BI" are consistent and asset references are intact.

---

39-40: Verify VS Code command names against product implementation.

Line 39-40 references Ballerina: Configure default WSO2 model provider, and line 121 references Ballerina: Enable Tracing. However, the PR objectives mention the tracing command should be "Ballerina: Enable Tracing (Experimental)". Confirm these command names match the actual VS Code extension implementation, particularly whether the tracing command includes the "(Experimental)" suffix.

Also applies to: 121-121

---

60-60: Verify model type identifier.

Line 60 specifies gpt-4.1 as the model type. Confirm this is a valid OpenAI model identifier for the current API version. If this value comes from a dropdown or configuration schema, verify it matches the expected format (e.g., whether it should be gpt-4-turbo, gpt-4-1106-preview, or similar).

---

76-99: Verify UI terminology and asset availability.

Line 92 references "Use Function" option for adding tools. Confirm this matches the actual BI UI wording. Additionally, verify that both asset files referenced exist:

• Line 86: 6.add-functions.gif
• Line 99: 6.create-agent-tool.gif

The asset mismatch issue from the previous review appears resolved, but confirm the filenames exist in the asset directory.

---

112-126: Tracing section is well-structured and comprehensive.

The new Step 8 provides clear instructions for enabling built-in tracing and viewing detailed execution logs. The workflow is logical: enable via VS Code command → run integration → interact with agent → view traces. This directly addresses the PR objectives for documenting the tracing feature.

---

129-135: Verify external documentation links are current.

Lines 133-135 reference external documentation for Jaeger integration and Ballerina observability tools. Confirm these links are current and resolve correctly, particularly the Jaeger documentation URL on line 133 and the Ballerina observability tools page on line 135.

[coderabbitai]

Actionable comments posted: 0

🧹 Nitpick comments (2)

en/docs/integration-guides/ai/agents/introduction-to-chat-agents.md (2)

39-39: Inconsistent keyboard shortcut formatting.

Line 39 formats the keyboard shortcut as a single backtick block (`Ctrl/Cmd + Shift + P`), while line 123 breaks it into separate backticks (`Ctrl/Cmd` + `Shift` + `P`). Adopt a consistent style throughout the document.

🔎 Proposed fix to standardize on line 123's approach

- Press `Ctrl/Cmd + Shift + P` to open the VS Code Command Palette.
+ Press `Ctrl/Cmd` + `Shift` + `P` to open the VS Code Command Palette.

Also applies to: 123-123

---

110-113: Consider explicitly documenting the configurables prompt in Step 7.

Line 69 correctly references Step 7 as the point where users are prompted for configurable values. However, Step 7 currently only mentions being prompted to "run the integration" (line 111). For clarity, consider explicitly noting that users will also be prompted to provide values for any configurables they defined (e.g., openAiApiKey) when starting the integration. This would reinforce the earlier guidance in Step 4 about configurable setup.

📜 Review details

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 07e79c8 and 2212982.

📒 Files selected for processing (1)

• en/docs/integration-guides/ai/agents/introduction-to-chat-agents.md

🧰 Additional context used

🧠 Learnings (2)

📚 Learning: 2025-11-12T05:34:19.589Z

Learnt from: dulajdilshan
Repo: wso2/docs-bi PR: 169
File: en/docs/deploy/deployment-best-practices/network-level-security.md:1-20
Timestamp: 2025-11-12T05:34:19.589Z
Learning: In the wso2/docs-bi repository documentation files, using h3 (###) subsection headings directly under h1 (#) titles is an intentional style choice, even though it violates standard markdown heading increment rules.

Applied to files:

• en/docs/integration-guides/ai/agents/introduction-to-chat-agents.md

📚 Learning: 2025-11-12T05:34:18.459Z

Learnt from: dulajdilshan
Repo: wso2/docs-bi PR: 169
File: en/docs/deploy/deployment-best-practices/os-level-security.md:1-25
Timestamp: 2025-11-12T05:34:18.459Z
Learning: In the WSO2 docs-bi repository, specifically in deployment best practices documentation files under en/docs/deploy/deployment-best-practices/, using h3 (###) headings directly after h1 titles (instead of h2) is intentional and should not be flagged.

Applied to files:

• en/docs/integration-guides/ai/agents/introduction-to-chat-agents.md

🔇 Additional comments (3)

en/docs/integration-guides/ai/agents/introduction-to-chat-agents.md (3)

37-69: LGTM on configurable API key best practice.

The new configurables workflow for handling the OpenAI API key (lines 56–61) is well-documented and follows security best practices by externalizing sensitive values. The reference to the Configurations guide is helpful, and the note about prompting at runtime is valuable context for users.

---

115-138: Comprehensive tracing documentation with dual-platform guidance.

The new Step 8 effectively documents both the built-in tracing feature (with clear VS Code command and UI steps) and external observability integration via Jaeger. The structured flow from enabling tracing → running the integration → viewing logs is intuitive, and the references to Ballerina's observability documentation provide good extensibility guidance.

---

124-124: Verify the VS Code command name for the tracing feature.

The documented command name at line 124 is Ballerina: Enable Tracing, but this command does not appear in publicly available Ballerina VS Code extension documentation. If this is a new feature being introduced, verify whether the command should include the (Experimental) suffix as referenced in the PR objectives, or confirm the final command name with the extension team.

[coderabbitai]

Actionable comments posted: 1

📜 Review details

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 2212982 and 9f0f82b.

📒 Files selected for processing (1)

• en/docs/integration-guides/ai/agents/introduction-to-chat-agents.md

🧰 Additional context used

🧠 Learnings (2)

📚 Learning: 2025-11-12T05:34:19.589Z

Learnt from: dulajdilshan
Repo: wso2/docs-bi PR: 169
File: en/docs/deploy/deployment-best-practices/network-level-security.md:1-20
Timestamp: 2025-11-12T05:34:19.589Z
Learning: In the wso2/docs-bi repository documentation files, using h3 (###) subsection headings directly under h1 (#) titles is an intentional style choice, even though it violates standard markdown heading increment rules.

Applied to files:

• en/docs/integration-guides/ai/agents/introduction-to-chat-agents.md

📚 Learning: 2025-11-12T05:34:18.459Z

Learnt from: dulajdilshan
Repo: wso2/docs-bi PR: 169
File: en/docs/deploy/deployment-best-practices/os-level-security.md:1-25
Timestamp: 2025-11-12T05:34:18.459Z
Learning: In the WSO2 docs-bi repository, specifically in deployment best practices documentation files under en/docs/deploy/deployment-best-practices/, using h3 (###) headings directly after h1 titles (instead of h2) is intentional and should not be flagged.

Applied to files:

• en/docs/integration-guides/ai/agents/introduction-to-chat-agents.md

🔇 Additional comments (1)

en/docs/integration-guides/ai/agents/introduction-to-chat-agents.md (1)

124-124: Verify the VS Code command name against the PR objectives and issue #2136.

Line 124 references Ballerina: Enable Tracing, but this documentation should be verified to confirm whether the command includes the (Experimental) suffix as specified in the PR objectives and issue #2136. Check the actual VS Code command palette or the PR details to determine the exact command name that should be documented.