Skip to content

fix(sdk): coerce non-str tool descriptions in system prompts#2569

Open
Bartok9 wants to merge 5 commits into
Canner:mainfrom
Bartok9:fix/prompt-coerce-tool-description
Open

fix(sdk): coerce non-str tool descriptions in system prompts#2569
Bartok9 wants to merge 5 commits into
Canner:mainfrom
Bartok9:fix/prompt-coerce-tool-description

Conversation

@Bartok9

@Bartok9 Bartok9 commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Shared _build_tools_section in langchain + pydantic assumed tool.description is 'str|None'.
  • Non-str descriptions raised AttributeError on .strip().
  • Coerce description (and missing name) before rendering markdown bullets.

License

sdk/** Apache-2.0.

Test plan

  • langchain: test_prompt_tool_description.py (1 passed)
  • pydantic: test_instructions_tool_description.py (1 passed)

Summary by CodeRabbit

  • Bug Fixes

    • Improved “Available tools” rendering to tolerate missing/invalid tool names and non-text descriptions, ensuring consistent bullet labels and safer description formatting (including first-line truncation).
  • Tests

    • Expanded prompt/instruction unit test coverage for tool-name fallback behavior (missing/empty/invalid name, using __name__) and for non-string/multiline description handling.

_build_tools_section used (tool.description or "").strip(), which raises
when description is a non-empty non-str (e.g. int mock). Coerce safely
in both langchain and pydantic SDKs.
@coderabbitai

coderabbitai Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 1d99f105-50b5-4c8b-a7c7-1bee00a2bc18

📥 Commits

Reviewing files that changed from the base of the PR and between c6ba7cf and eb22875.

📒 Files selected for processing (3)
  • sdk/wren-langchain/src/wren_langchain/_prompt.py
  • sdk/wren-langchain/tests/unit/test_prompt_tool_description.py
  • sdk/wren-pydantic/src/wren_pydantic/_instructions.py
🚧 Files skipped from review as they are similar to previous changes (3)
  • sdk/wren-langchain/src/wren_langchain/_prompt.py
  • sdk/wren-langchain/tests/unit/test_prompt_tool_description.py
  • sdk/wren-pydantic/src/wren_pydantic/_instructions.py

Walkthrough

Both SDKs now defensively normalize tool names and descriptions when generating prompts and instructions. Tests cover missing or invalid names, non-string descriptions, and multiline descriptions.

Changes

Tool metadata rendering

Layer / File(s) Summary
Normalize tool metadata
sdk/wren-langchain/src/wren_langchain/_prompt.py, sdk/wren-pydantic/src/wren_pydantic/_instructions.py
Tool names now fall back to __name__ or "tool"; descriptions are safely coerced and truncated to the first line. Normalized names also drive tool-name conditionals.
Validate rendered tool entries
sdk/wren-langchain/tests/unit/test_prompt_tool_description.py, sdk/wren-pydantic/tests/unit/test_instructions_tool_description.py
Tests verify fallback naming, null and numeric descriptions, and multiline description rendering.

Estimated code review effort: 2 (Simple) | ~10 minutes

Poem

A rabbit hops through tool-filled hay,
Making names and notes behave today.
Nulls turn quiet, numbers shine,
Long notes stop at their first line.
“Binky!” says the bunny—“The prompts are fine!”

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 41.67% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly matches the main change: making system prompt tool descriptions safe for non-string values.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
sdk/wren-langchain/tests/unit/test_prompt_tool_description.py (1)

27-38: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Add regression coverage for all promised metadata fallbacks.

  • sdk/wren-langchain/tests/unit/test_prompt_tool_description.py#L27-L38: test absent/invalid names, missing descriptions, and the public prompt-building path.
  • sdk/wren-pydantic/tests/unit/test_instructions_tool_description.py#L30-L38: also add multiline-description coverage and test the public instructions-building path.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@sdk/wren-langchain/tests/unit/test_prompt_tool_description.py` around lines
27 - 38, Add regression tests for all metadata fallbacks in
sdk/wren-langchain/tests/unit/test_prompt_tool_description.py lines 27-38,
covering absent or invalid names, missing descriptions, and the public
prompt-building path around the existing
test_build_tools_section_non_str_description test. Also update
sdk/wren-pydantic/tests/unit/test_instructions_tool_description.py lines 30-38
to cover multiline descriptions and the public instructions-building path, while
retaining the existing fallback assertions.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@sdk/wren-langchain/src/wren_langchain/_prompt.py`:
- Around line 212-219: Centralize validated tool-name extraction in the
LangChain prompt builder and reuse that normalized name both for workflow
detection and bullet rendering, rather than accessing tool.name directly. Apply
the same change in sdk/wren-langchain/src/wren_langchain/_prompt.py lines
212-219 and sdk/wren-pydantic/src/wren_pydantic/_instructions.py lines 231-238,
ensuring each build_system_prompt/build_instructions path constructs tool_names
from the normalized fallback-aware name.

---

Nitpick comments:
In `@sdk/wren-langchain/tests/unit/test_prompt_tool_description.py`:
- Around line 27-38: Add regression tests for all metadata fallbacks in
sdk/wren-langchain/tests/unit/test_prompt_tool_description.py lines 27-38,
covering absent or invalid names, missing descriptions, and the public
prompt-building path around the existing
test_build_tools_section_non_str_description test. Also update
sdk/wren-pydantic/tests/unit/test_instructions_tool_description.py lines 30-38
to cover multiline descriptions and the public instructions-building path, while
retaining the existing fallback assertions.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 915419b1-3e9f-47f0-bf48-31c36904a303

📥 Commits

Reviewing files that changed from the base of the PR and between 507bda6 and 12073a0.

📒 Files selected for processing (4)
  • sdk/wren-langchain/src/wren_langchain/_prompt.py
  • sdk/wren-langchain/tests/unit/test_prompt_tool_description.py
  • sdk/wren-pydantic/src/wren_pydantic/_instructions.py
  • sdk/wren-pydantic/tests/unit/test_instructions_tool_description.py

Comment thread sdk/wren-langchain/src/wren_langchain/_prompt.py Outdated
Bartok9 added 2 commits July 23, 2026 04:06
…ack tests

Extract _tool_name() helper in both SDK prompt builders and reuse it for
workflow detection (tool_names set) and bullet rendering, so a missing or
non-str tool.name is handled consistently everywhere. Add regression tests
covering None/empty/numeric names and __name__ fallback.
@Bartok9

Bartok9 commented Jul 23, 2026

Copy link
Copy Markdown
Contributor Author

Thanks @coderabbitai — good call. Addressed in the latest push (76bb050):

  • Centralized validated tool-name extraction into a _tool_name() helper in both sdk/wren-langchain/src/wren_langchain/_prompt.py and sdk/wren-pydantic/src/wren_pydantic/_instructions.py. It now feeds both the workflow-detection tool_names set and the bullet rendering, so a missing/non-str name falls back consistently (__name__"tool") everywhere.
  • Added regression tests covering None/empty/numeric names and the __name__ fallback in both SDK test modules, alongside the existing description-coercion cases.

Ruff clean and unit tests green locally.

@coderabbitai

coderabbitai Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Tip

For best results, initiate chat on the files or code changes.

@Bartok9 Thanks for addressing this comprehensively. Centralizing _tool_name() so workflow detection and rendered bullets share the same fallback behavior is the right fix, and the expanded regression coverage covers the important edge cases.

🐇✨


✏️ Learnings added
Learnt from: Bartok9
URL: https://github.com/Canner/WrenAI/pull/2569

Timestamp: 2026-07-23T08:52:23.847Z
Learning: In `sdk/wren-langchain/src/wren_langchain/_prompt.py` and `sdk/wren-pydantic/src/wren_pydantic/_instructions.py`, tool names must be normalized through the shared local `_tool_name()` helper. It accepts only non-empty strings from `tool.name` and otherwise falls back to `tool.__name__`, then `"tool"`; use this result consistently for workflow detection and Markdown tool rendering.

Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.


If you found this review helpful, would you consider giving us a shout-out on X?

Thank you for using CodeRabbit!

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
sdk/wren-langchain/tests/unit/test_prompt_tool_description.py (1)

26-37: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

Assert that None descriptions render as empty.

The current assertion also passes if the output contains `wren_query`: None. Assert the exact rendered line so the coercion behavior is actually covered.

🧪 Proposed test adjustment
-    assert "`wren_query`:" in out
+    assert "- `wren_query`: " in out.splitlines()
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@sdk/wren-langchain/tests/unit/test_prompt_tool_description.py` around lines
26 - 37, Update test_build_tools_section_non_str_description to assert the exact
rendered wren_query line, requiring an empty description after the colon rather
than merely checking the tool name. Keep the existing assertions for numeric
coercion and newline trimming unchanged.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@sdk/wren-langchain/src/wren_langchain/_prompt.py`:
- Around line 207-213: Update _tool_name so the __name__ fallback is validated
with the same non-empty-string requirement as name; return "tool" whenever
__name__ is missing, empty, or not a string, ensuring the function always
returns a valid tool name.

---

Nitpick comments:
In `@sdk/wren-langchain/tests/unit/test_prompt_tool_description.py`:
- Around line 26-37: Update test_build_tools_section_non_str_description to
assert the exact rendered wren_query line, requiring an empty description after
the colon rather than merely checking the tool name. Keep the existing
assertions for numeric coercion and newline trimming unchanged.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: b5b65888-2c8e-44fa-b3a0-69a9d2c6cd2f

📥 Commits

Reviewing files that changed from the base of the PR and between 9ecd4d8 and 76bb050.

📒 Files selected for processing (4)
  • sdk/wren-langchain/src/wren_langchain/_prompt.py
  • sdk/wren-langchain/tests/unit/test_prompt_tool_description.py
  • sdk/wren-pydantic/src/wren_pydantic/_instructions.py
  • sdk/wren-pydantic/tests/unit/test_instructions_tool_description.py
🚧 Files skipped from review as they are similar to previous changes (2)
  • sdk/wren-pydantic/tests/unit/test_instructions_tool_description.py
  • sdk/wren-pydantic/src/wren_pydantic/_instructions.py

Comment thread sdk/wren-langchain/src/wren_langchain/_prompt.py
Bartok9 added 2 commits July 23, 2026 08:14
Address CodeRabbit review on Canner#2569:
- _tool_name now applies the same non-empty-str validation to the
  __name__ fallback in both langchain and pydantic SDKs, returning
  "tool" whenever __name__ is missing, empty, or non-str.
- Tighten test_build_tools_section_non_str_description to assert the
  exact rendered line for a None description (empty after colon).
@Bartok9

Bartok9 commented Jul 23, 2026

Copy link
Copy Markdown
Contributor Author

Thanks @coderabbitai — both addressed in eb22875:

  • _tool_name() now validates the __name__ fallback with the same non-empty-string check as name, so a missing/empty/non-str __name__ cleanly falls through to "tool" in both the langchain and pydantic SDKs.
  • Tightened test_build_tools_section_non_str_description to assert the exact rendered line (- \wren_query`: viaout.splitlines()), so the None`-description coercion to empty is actually covered rather than a substring match.

Ruff clean, unit tests green in both SDKs.

@coderabbitai

coderabbitai Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Tip

For best results, initiate chat on the files or code changes.

@Bartok9 Thanks for tightening both cases. Validating __name__ with the same non-empty-string contract and asserting the exact empty-description line closes the remaining gaps nicely.

🐇✨

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant