Skip to content

Conversation

@yehudit1987
Copy link
Collaborator

Fix domain classifier returning empty or incorrect categories.

Changes:

  • Add fallback to "other" when confidence < threshold (classifier.go)
  • Add ModernBERT fallback when CandleBERT fails
  • Add PII type translation: class_X → DATE_TIME (mapping.go)
  • Include category header in PII/cache responses (response.go)
  • Auto-load supplement dataset from HuggingFace (ft_linear.py)
  • Update Helm defaults to use trained model from LLM-Semantic-Router
  • Move test cases to JSON files with go:embed
  • Remove CRD-incompatible tests from dynamic-config profile

HuggingFace resources (LLM-Semantic-Router org):

  • New trained model: category_classifier_modernbert-base_trained_model
  • New supplement dataset: category-classifier-supplement (653 examples)

Resolves #714

@netlify
Copy link

netlify bot commented Dec 14, 2025

Deploy Preview for vllm-semantic-router ready!

Name Link
🔨 Latest commit 77588d8
🔍 Latest deploy log https://app.netlify.com/projects/vllm-semantic-router/deploys/693fdd535fe71300084fe300
😎 Deploy Preview https://deploy-preview-827--vllm-semantic-router.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@yehudit1987 yehudit1987 force-pushed the fix_domain_classification branch 2 times, most recently from db737d4 to 2711b04 Compare December 14, 2025 17:49
@yehudit1987 yehudit1987 marked this pull request as ready for review December 14, 2025 18:06
@github-actions
Copy link

github-actions bot commented Dec 14, 2025

👥 vLLM Semantic Team Notification

The following members have been identified for the changed files in this PR and have been automatically assigned:

📁 deploy

Owners: @rootfs, @Xunzhuo
Files changed:

  • deploy/helm/semantic-router/values.yaml
  • deploy/kubernetes/aibrix/semantic-router-values/values.yaml

📁 e2e

Owners: @Xunzhuo
Files changed:

  • e2e/profiles/ai-gateway/values.yaml
  • e2e/profiles/dynamic-config/profile.go
  • e2e/profiles/dynamic-config/values.yaml
  • e2e/profiles/llm-d/values.yaml
  • e2e/profiles/production-stack/values.yaml
  • e2e/profiles/routing-strategies/values.yaml
  • e2e/testcases/decision_fallback.go
  • e2e/testcases/plugin_config_variations.go
  • e2e/testcases/testdata/decision_fallback_cases.json
  • e2e/testcases/testdata/domain_classify_cases.json
  • e2e/testcases/testdata/plugin_config_cases.json

📁 Root Directory

Owners: @rootfs, @Xunzhuo
Files changed:

  • perf/testdata/baselines/cache.json
  • perf/testdata/baselines/classification.json
  • perf/testdata/baselines/decision.json
  • perf/testdata/baselines/extproc.json

📁 src

Owners: @rootfs, @Xunzhuo, @wangchen615
Files changed:

  • src/semantic-router/pkg/classification/classifier.go
  • src/semantic-router/pkg/classification/mapping.go
  • src/semantic-router/pkg/classification/mcp_classifier.go
  • src/semantic-router/pkg/config/config.go
  • src/semantic-router/pkg/extproc/req_filter_cache.go
  • src/semantic-router/pkg/extproc/req_filter_pii.go
  • src/semantic-router/pkg/utils/http/response.go
  • src/semantic-router/pkg/utils/http/response_test.go
  • src/training/classifier_model_fine_tuning/ft_linear.py

vLLM

🎉 Thanks for your contributions!

This comment was automatically generated based on the OWNER files in the repository.

@yehudit1987 yehudit1987 force-pushed the fix_domain_classification branch from 2711b04 to 21efd86 Compare December 15, 2025 06:36
@yehudit1987 yehudit1987 force-pushed the fix_domain_classification branch from 21efd86 to 77588d8 Compare December 15, 2025 10:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Test] Domain Classifier Returns Empty or Wrong Classifications

5 participants