Skip to content

[Router] Add Valkey memory backend with TLS support#1739

Draft
daric93 wants to merge 5 commits intovllm-project:mainfrom
daric93:feature/valkey-memory-backend
Draft

[Router] Add Valkey memory backend with TLS support#1739
daric93 wants to merge 5 commits intovllm-project:mainfrom
daric93:feature/valkey-memory-backend

Conversation

@daric93
Copy link
Copy Markdown
Contributor

@daric93 daric93 commented Apr 9, 2026

Summary

  • Add Valkey (with Search module) as an alternative memory backend alongside Milvus, selectable via backend: valkey in config
  • Implement full Store interface: Store, Retrieve (vector similarity via FT.SEARCH), Get, Update, List, Forget, ForgetByScope with HNSW indexing, hybrid reranking, adaptive threshold, and access tracking
  • Refactor ConsolidateUser from *MilvusStore receiver to standalone function accepting Store interface for backend-agnostic consolidation
  • Add TLS support (tls_enabled, tls_ca_path, tls_insecure_skip_verify) using valkey-glide native TLS API
  • Use authoritative HINCRBY return value in recordRetrieval to keep metadata JSON access_count consistent under concurrent retrievals
  • Add comprehensive unit tests (707 lines), integration tests (838 lines), example configs, E2E config, CLI model, and documentation

Test plan

  • make test-semantic-router — all packages pass (0 failures)
  • make go-lint — 0 issues
  • make check-go-mod-tidy — clean
  • Integration tests against live Valkey instance — 24/24 pass
  • E2E test with config.memory-user-valkey.yaml profile against Valkey + Search module

daric93 added 3 commits April 9, 2026 10:51
Signed-off-by: Daria Korenieva <daric2612@gmail.com>
Signed-off-by: Daria Korenieva <daric2612@gmail.com>

# Conflicts:
#	src/semantic-router/pkg/extproc/router_memory.go
Signed-off-by: Daria Korenieva <daric2612@gmail.com>
@netlify
Copy link
Copy Markdown

netlify bot commented Apr 9, 2026

Deploy Preview for vllm-semantic-router failed.

Name Link
🔨 Latest commit b786bb3
🔍 Latest deploy log https://app.netlify.com/projects/vllm-semantic-router/deploys/69d9633b3ac0160007dc50e3

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 9, 2026

✅ Supply Chain Security Report — All Clear

Scanner Status Findings
AST Codebase Scan (Py, Go, JS/TS, Rust) 27 finding(s) — MEDIUM: 21 · LOW: 6
AST PR Diff Scan No issues detected
Regex Fallback Scan No issues detected

Scanned at 2026-04-10T20:54:56.580Z · View full workflow logs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant