Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Support conditional LLM model mapping #1952

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

CH3CHO
Copy link
Collaborator

@CH3CHO CH3CHO commented Mar 26, 2025

Ⅰ. Describe what this PR did

Support conditional LLM model mapping, so user can configure difference mapping rules for different consumers, a.k.a. consumer-level model aliases.

Config sample:

{
 "conditionalModelMappings": [
  {
   "consumers": [
    "test-1"
   ],
   "modelMapping": {
    "*": "my-model-1",
    "deepseek-*": "my-deepseek-1",
    "test": "my-test-1"
   }
  },
  {
   "consumers": [
    "test-3"
   ],
   "modelMapping": {
    "*": "my-model-2",
    "deepseek-*": "my-deepseek-2",
    "test": "my-test-2"
   }
  }
 ],
 "modelMapping": {
  "*": "my-model",
  "deepseek-*": "my-deepseek",
  "test": "my-test"
 }
}

Ⅱ. Does this pull request fix one issue?

Ⅲ. Why don't you add test cases (unit test/integration test)?

Ⅳ. Describe how to verify it

Ⅴ. Special notes for reviews

@CH3CHO CH3CHO requested review from johnlanni and rinfx as code owners March 26, 2025 03:16
@codecov-commenter
Copy link

codecov-commenter commented Mar 26, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 43.61%. Comparing base (ef31e09) to head (ae67361).
Report is 389 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1952      +/-   ##
==========================================
+ Coverage   35.91%   43.61%   +7.70%     
==========================================
  Files          69       79      +10     
  Lines       11576    12685    +1109     
==========================================
+ Hits         4157     5532    +1375     
+ Misses       7104     6807     -297     
- Partials      315      346      +31     

see 75 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

}

if (!JsonArrayIterate(
configuration, "conditionalModelMappings", [&](const json& conditional_item) -> bool {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to add unit tests, and also need to provide documentation

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@CH3CHO CH3CHO force-pushed the feat/model-mapping-on-consumer branch from 633f87b to ce628d4 Compare March 27, 2025 01:31
@CH3CHO CH3CHO force-pushed the feat/model-mapping-on-consumer branch from 4decf8e to cbef9a5 Compare March 27, 2025 04:48
@CH3CHO CH3CHO force-pushed the feat/model-mapping-on-consumer branch from cbef9a5 to ae67361 Compare March 27, 2025 05:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants