A schema response says when a datasource has stored examples (#301)

[ashwin-agami]

Refs #301. Leaves the issue open: its "done when" needs a scripted run with a live client, which is described below.

What changes

• get_datasource_schema says when a datasource has stored examples. When there are any, the response carries prompt_examples: {"stored": <count>, "next": <one-line reminder>}. It never carries the examples. Ranking and returning them stays get_prompt_examples' job.
• The count covers the whole datasource, even on an area-scoped call. A count narrowed the same way would hide exactly the examples an area filter drops, which is the issue's third chat.
• Served path: a new model_store.count_examples runs on the connection _context_sources already opens, so there is still one database connection per schema call. The count is covered by the primary key.
• Local path: examples are counted through the loader's own list_prompt_examples, so both file shapes it accepts are counted.
• Wording: the server instructions and both tool descriptions now say to pass the user's question as query and to leave area out unless sure. The reminder text is defined once, as _EXAMPLES_REMINDER.

Decisions

• Examples stay out of the schema response. It remains the model's description, and get_prompt_examples remains the ranked, budgeted source of examples.
• The parallel rule is kept. Clients are still told to call get_datasource_schema and get_prompt_examples in the same turn (test_the_two_grounding_calls_are_declared_independent). The pointer is a fallback for a client that skipped the second call.
• area stays a real filter. Scoring every example whenever area is set would make the filter pointless. Clients are told to leave it out unless sure, and leaving it out already ranks the whole datasource. A search index for large libraries is separate work.
• No spec. The change is small, and Clients skip get_prompt_examples despite the examples-first instruction #301 carries the problem and the done-when.

Tests

• New: tests/test_schema_names_stored_examples.py, 6 tests:
  • the count and the reminder;
  • an area-scoped call still counts every area;
  • no field when there are no examples;
  • a file in the loader's {examples: [...]} shape;
  • the served count only covers this datasource and this org;
  • the descriptions and instructions name the field and keep the calls independent.
• Mutation: with the field removed, the three tests that depend on it fail.
• Neighbouring suites pass: the instruction and description checks, hosted instruction wording, domain context, the tool extension seam, and filter wording.
• Already failing on main: test_prompt_examples_serving (2), test_admin_activity (6) and test_model_store_roundtrip::test_file_model_seeds_to_db_and_tools_serve_from_it fail identically on untouched main (99e44a8). They are not caused by this change.

Review

The standard agami review ran with a correctness pass and a security/runtime pass.

• Must-fix (fixed in c33a56f): a local file in the {examples: [...]} shape counted zero.
• Nits taken: an unreadable local file now counts zero instead of failing the call.
• Nits left:
  • The field is added after the size check, but it is about 250 characters.
  • The golden eval's fixed context now includes the reminder line. Its key comparison is unaffected.
• Security: nothing blocking. The count is scoped by org and datasource, the SQL is parameterized, and neither execute_sql nor the safety wording changes.
• Already present, not addressed here: on the local path a datasource name is joined into a file path unchecked. The existing file reads in the same function already allow that.

Still to verify (the issue's done-when)

Deploy locally and ask, from a claude.ai client without naming any tool:

• questions about the signed-in user, like "tickets assigned to me";
• questions about a named person.

Each conversation's tool calls should show get_prompt_examples called, with the question as query and no guessed area.

🤖 Generated with Claude Code

[Copilot]

🟡 Changes recommended

Unresolved moderate findings remain in the schema size-budget handling and shared typed contract.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

This PR makes get_datasource_schema report stored prompt-example counts while keeping retrieval in get_prompt_examples.

Changes:

• Adds local and database-backed example counting.
• Updates instructions, descriptions, tests, and changelog.
• Preserves datasource-wide counting and ranked example retrieval.

File summaries

File	Summary
tests/test_schema_names_stored_examples.py	Covers local, served, scoped, and wording behavior.
tests/test_model_store_roundtrip.py	Updates context tuple handling.
packages/agami-core/src/tools.py	Adds schema metadata and guidance; unresolved moderate findings remain around budget enforcement and typed contract coverage.
packages/agami-core/src/model_store.py	Adds datasource- and organization-scoped example counting.
CHANGELOG.md	Documents the feature; area-filter wording needs correction.

Review details

Suppressed comments (3)

CHANGELOG.md:25

• This changelog entry repeats the inaccurate claim that an area drops every example outside it; served selection still includes cross-area (area IS NULL) examples. Keep the release note consistent with the tool contract by saying it drops other named areas but retains cross-area examples.

  question as `query` and leave `area` out unless sure, because an `area` drops every example
  outside it.

packages/agami-core/src/tools.py:1609

• The hosted-path test only checks _context_sources(...)[4]; it never parses tool_get_datasource_schema with a database configured. A regression in this new insertion or its truthiness branch could therefore leave all tests green while served clients receive no prompt_examples pointer. Add a DB-backed schema-output assertion.

    org_md_raw, user_md_raw, record, company_md, example_count = _context_sources(
        profile, _current_org_id()
    )
    if example_count:
        # A pointer, never the examples (#301). Counted datasource-wide whatever `area` scoped this
        # call: an area-scoped count would hide exactly the examples an `area` filter drops.
        result["prompt_examples"] = {"stored": example_count, "next": _EXAMPLES_REMINDER}

packages/agami-core/src/tools.py:129

• The served selector deliberately retains cross-area rows (area IS NULL) when an area is supplied, so saying it drops every example outside that area is inaccurate and conflicts with the input schema's cross-area behavior. Say that it drops examples from other named areas instead.

    "question as `query` and mirror the closest match; leave `area` out unless you are sure of "
    "it, because it drops every example outside that area. Use "

• Files reviewed: 5/5 changed files
• Comments generated: 3
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[ashwin-agami]

Fixed in 620821f. The count is read before the response is built, and prompt_examples is added inside the size loop before the size check (and on the table-scoped branch), so the downgrade decision includes it.

[ashwin-agami]

Fixed in 620821f. DatasourceSchemaResult now declares prompt_examples: dict[str, Any] | None. test_the_shared_contract_declares_the_pointer checks it's a declared field, not just tolerated, and that it round-trips.

[ashwin-agami]

Leaving this as-is. get_prompt_examples' own description, read before a client calls it, already says to pass the question as query and leave area out unless sure, and so does the reminder inside prompt_examples. Repeating it on get_datasource_schema's description would be a third copy of the same advice.

[ashwin-agami]

Also fixed the three suppressed comments in 620821f:

• Area wording (instructions, get_prompt_examples description, changelog): now says an area drops other areas' examples; cross-area examples stay.
• Hosted test: test_a_served_schema_call_carries_the_pointer stores a model and examples in a SQLite database, calls get_datasource_schema with that database configured and area set, and asserts prompt_examples is in the response the tool returns.