Skip to content

fix(vikingdb): trust OpenViking schema for volcengine api_key mode - #2981

Merged
qin-ctx merged 1 commit into
volcengine:mainfrom
ByteDanceLiuYang:vikingdb_schema_compatibility
Jul 3, 2026
Merged

fix(vikingdb): trust OpenViking schema for volcengine api_key mode#2981
qin-ctx merged 1 commit into
volcengine:mainfrom
ByteDanceLiuYang:vikingdb_schema_compatibility

Conversation

@ByteDanceLiuYang

Copy link
Copy Markdown
Contributor

Description

The remote VikingDB backend has three configuration modes (Volcengine AK/SK, private VikingDB, and Volcengine API Key). Their handling of the content field, the search_tags field, and the grep path was previously inconsistent. In particular, the API Key mode cannot fetch the remote collection schema (and get_meta_data() had a bug that used context_collection as if it were a dict), so that mode could never write content / search_tags, and grep engine=auto would never use the remote search_by_keywords path.

This PR makes the API Key mode fully trust the standard OpenViking context schema (treating it as having all fields), aligning its write and retrieval behavior with the AK/SK and private modes. It also adds ignore_unknown_fields: true to all data-plane upsert/update/fetch/search requests across the three remote modes, so that writes and reads remain safe and do not fail on unknown fields when the remote schema differs from the local assumption.

Related Issue

Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update
  • Refactoring (no functional changes)
  • Performance improvement
  • Test update

Changes Made

  • VolcengineApiKeyCollection.get_meta_data() now returns a trusted standard schema built from CollectionSchemas.context_collection(...) (including content, search_tags, ScalarIndex, and FullText(content)), fixing the previous bug that called the method as a dict, so that write-field filtering and grep fulltext detection work correctly.
  • Added ignore_unknown_fields: true to the upsert_data and update_data request bodies of all three remote collections (VolcengineApiKeyCollection / VolcengineCollection / VikingDBCollection).
  • Added ignore_unknown_fields: true as a top-level request parameter to fetch_data (fetch_in_collection) and to search_by_vector / search_by_id / search_by_multimodal / search_by_random / search_by_keywords / search_by_scalar for all three collections.
  • Added unit tests covering: the API Key mode returning a trusted schema, and all three modes carrying ignore_unknown_fields on write and fetch/search requests; also updated the update_search_tags tests to reflect the "fetch the full record by id, then upsert" behavior.

Testing

  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • I have tested this on the following platforms:
    • Linux
    • macOS
    • Windows

Local verification:

python -m pytest -o addopts= tests/storage/test_collection_schemas.py -q
# 59 passed

python -m pytest -o addopts= tests/storage/test_viking_fs_grep.py -q
# 15 passed

python -m py_compile \
  openviking/storage/vectordb/collection/volcengine_api_key_collection.py \
  openviking/storage/vectordb/collection/volcengine_collection.py \
  openviking/storage/vectordb/collection/vikingdb_collection.py

Checklist

  • My code follows the project's coding style
  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • Any dependent changes have been merged and published

Screenshots (if applicable)

Additional Notes

  • The server-side /api/vikingdb/data/* endpoints must support the ignore_unknown_fields parameter. For older servers that do not recognize it, it is an extra field and should be confirmed to be ignored rather than rejected.
  • The delete / agg endpoints are intentionally left unchanged in this PR (their inputs do not involve record fields); they can be unified later if needed.

@qin-ctx
qin-ctx merged commit 34c191a into volcengine:main Jul 3, 2026
3 checks passed
@github-project-automation github-project-automation Bot moved this from Backlog to Done in OpenViking project Jul 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants