Skip to content

Fix full-text-search recipe: correct /v1/search response shape - #571

Merged
lukekim merged 1 commit into
trunkfrom
fix/full-text-search-api-response
Aug 12, 2026
Merged

Fix full-text-search recipe: correct /v1/search response shape#571
lukekim merged 1 commit into
trunkfrom
fix/full-text-search-api-response

Conversation

@claudespice

Copy link
Copy Markdown
Contributor

Summary

Ran the full-text-search recipe end to end on current stable. The configuration and the
text_search() SQL path are correct, but the documented /v1/search response does not match
what the endpoint returns.

What the recipe said — a data object holding the requested path, and matches as a string:

{
  "matches": { "content": "... Follow these steps to get started with ..." },
  "data": { "path": "postgres/rds/README.md" },
  "primary_key": { "path": "postgres/rds/README.md" },
  "_score": 1.41,
  "dataset": "cookbook_files"
}

What the endpoint returns — no data key at all for this request, and matches maps each
indexed column to an array of matching text:

{
  "matches": { "content": ["# AWS RDS for PostgreSQL\n\nWorks with `v1.0+`\n\nFollow these steps to ge..."] },
  "primary_key": { "path": "postgres/rds/README.md" },
  "_score": 1.12,
  "dataset": "cookbook_files"
}

The reason is worth documenting rather than just patching the JSON: data carries only those
additional_columns that are not part of the primary key. Because path is this dataset's
full_text_search.row_id, requesting it returns the value under primary_key and emits no
data object. Requesting non-key columns does produce one — confirmed with
"additional_columns": ["size", "name"], which returns
"data": { "name": "README.md", "size": 2437 }. Added a short field-by-field list covering this.

Also corrected in the same pass:

  • The first SQL example claimed LIMIT 5 but showed only 3 result rows, and its top hit was
    vectors/README.md — a path that does not exist in the repo (the recipe is vectors/s3).
    Replaced with the real five-row result and relabelled it "paths and scores vary", since the
    indexed source is the live cookbook repo.
  • show tables; and the search result table now include the data-type row that spice sql prints.

Verified against

spiceai/spiceai at v2.1.4 (current stable)

Evidence

Ran with GITHUB_TOKEN set, per the recipe's prerequisites:

2026-08-06T12:18:14.265810Z  INFO runtime::init::dataset: Dataset cookbook_files registered (github:github.com/spiceai/cookbook/files/trunk), acceleration (arrow), results cache enabled.
2026-08-06T12:18:18.614284Z  INFO runtime::accelerated_table::refresh_task: Loaded 124 rows (1.33 MiB) for dataset cookbook_files in 4s 347ms.
$ curl -sS -X POST http://localhost:8090/v1/search -H 'Content-Type: application/json' \
    -d '{"datasets":["cookbook_files"],"text":"getting started","additional_columns":["path"],"limit":5}'
# result keys: ['matches', 'primary_key', '_score', 'dataset']      <- no "data"

$ # same call with "additional_columns": ["size","name"]
# result keys: ['matches', 'data', 'primary_key', '_score', 'dataset']
# data: {'name': 'README.md', 'size': 2437}

The text_search() SQL examples all ran successfully and _score is correct throughout.

The documented search response showed a 'data' object holding the requested
'path' column and 'matches' as a plain string. The runtime returns 'matches' as
an array per column, and 'data' carries only additional_columns that are not
part of the primary key — so requesting 'path' (the row_id) produces no 'data'
object at all. Documented what 'data' does contain.

Also refreshed the SQL example output, which showed 3 rows for a LIMIT 5 query
and cited vectors/README.md, a path that no longer exists.

Verified by running the recipe end to end on Spice v2.1.4.
@claudespice claudespice added bug Something isn't working documentation Improvements or additions to documentation recipe labels Aug 6, 2026
@claudespice claudespice self-assigned this Aug 6, 2026
@claudespice
claudespice requested a review from lukekim August 6, 2026 13:11
@grokspice

Copy link
Copy Markdown

@copilot review

@lukekim
lukekim merged commit 4a25350 into trunk Aug 12, 2026
8 checks passed
@lukekim
lukekim deleted the fix/full-text-search-api-response branch August 12, 2026 00:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working documentation Improvements or additions to documentation recipe

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants