fix(lightspeed): stop hardcoding vector_store_ids to product docs only#3626
fix(lightspeed): stop hardcoding vector_store_ids to product docs only#3626rajin-kichannagari wants to merge 2 commits into
Conversation
The backend plugin was explicitly setting vector_store_ids to only the rhdh-product-docs vector store, which prevented BYOK (Bring Your Own Knowledge) RAG sources from being queried via inline RAG. By removing this override, the lightspeed-stack uses its own rag.inline config to determine which stores to query, enabling multi-source RAG.
rajin-kichannagari
requested review from
a team,
karthikjeeyar,
rohitkrai03 and
yangcao77
as code owners
Changed Packages
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #3626 +/- ##
==========================================
+ Coverage 52.89% 52.93% +0.04%
==========================================
Files 2324 2324
Lines 89087 89268 +181
Branches 24900 24999 +99
==========================================
+ Hits 47122 47257 +135
- Misses 41740 41787 +47
+ Partials 225 224 -1
*This pull request uses carry forward flags. Click here to find out more. Continue to review full report in Codecov by Harness.
🚀 New features to boost your workflow:
|
| // get the vector store id for the rhdh-product-docs vector store | ||
| if (lightspeed_vector_store_id === '') { | ||
| const vectorStores = await vectorStoresOperator.vectorStores.list(); | ||
| lightspeed_vector_store_id = | ||
| vectorStores.data.find((v: any) => | ||
| v.name.startsWith('rhdh-product-docs'), | ||
| )?.id || ''; | ||
| } | ||
|
|
||
| if (lightspeed_vector_store_id !== '') { | ||
| request.body.vector_store_ids = [lightspeed_vector_store_id]; | ||
| } | ||
|
|
There was a problem hiding this comment.
@maysunfaisal @rajin-kichannagari Do we still need to get the vector store ID from the vector stores this way if the platform engineer does not use BYOK? If so, we might want to add some additional logic here to determine if this is a BYOK situation or not.
There was a problem hiding this comment.
I'm pretty sure we still need this explicit vector_store setup or otherwise Developer LIghtspeed tool will also query from Notebooks vector_store.
There was a problem hiding this comment.
@JslYoon is that why it was introduced in the first place? We also need to remove this hardcoded vector store dependency because OKP also cannot function if we are calling LCORE with hardcoded vector store names 🤔
JslYoon
left a comment
JslYoon
left a comment
There was a problem hiding this comment.
I believe we deliberately disabled rag.inline and only enabled tool.rag because the two conflicted and caused an infinite loop.
|
|
||
| logger.info(`/v1/query receives call from user: ${userEntityRef}`); | ||
|
|
||
| // get the vector store id for the rhdh-product-docs vector store |
There was a problem hiding this comment.
According to Claude,
The test fixture at fixtures/lcsHandlers.ts:281-282 mocks the vector stores endpoint returning rhdh-product-docs. Since the PR removes the code that calls that endpoint to look up the vector store
ID, this mock handler will just go unused — it won't break any tests, but it's now dead fixture code.
|
@rajin-kichannagari This PR is still valid and addresses a concern for both BYOK and OKP. I shared some insights on the Slack thread. Maybe we can chat more about this on the Parking Lot or we can discuss on Slack. |
|
4 participants
Summary
The backend plugin was explicitly setting
vector_store_idsto only therhdh-product-docsvector store, which prevented BYOK RAG sources configured inrag.inlinefrom being queried via inline RAG.By removing this override, the lightspeed-stack uses its own
rag.inlineconfig to determine which stores to query, enabling multi-source RAG.Context
When BYOK knowledge sources are configured in
lightspeed-stack.yaml, the_fetch_byok_rag()function in lightspeed-stack filters the request'svector_store_idsagainstrag.inline. Since the plugin was only sending the product docs store ID, BYOK stores were always filtered out and inline RAG was silently skipped.✔️ Checklist