Skip to content

feat: Support conditional LLM model mapping #1952

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 2 commits into from

Conversation

CH3CHO
Copy link
Collaborator

@CH3CHO CH3CHO commented Mar 26, 2025

Ⅰ. Describe what this PR did

Support conditional LLM model mapping, so user can configure difference mapping rules for different consumers, a.k.a. consumer-level model aliases.

Config sample:

{
 "conditionalModelMappings": [
  {
   "consumers": [
    "test-1"
   ],
   "modelMapping": {
    "*": "my-model-1",
    "deepseek-*": "my-deepseek-1",
    "test": "my-test-1"
   }
  },
  {
   "consumers": [
    "test-3"
   ],
   "modelMapping": {
    "*": "my-model-2",
    "deepseek-*": "my-deepseek-2",
    "test": "my-test-2"
   }
  }
 ],
 "modelMapping": {
  "*": "my-model",
  "deepseek-*": "my-deepseek",
  "test": "my-test"
 }
}

Ⅱ. Does this pull request fix one issue?

Ⅲ. Why don't you add test cases (unit test/integration test)?

Ⅳ. Describe how to verify it

Ⅴ. Special notes for reviews

@CH3CHO CH3CHO requested review from johnlanni and rinfx as code owners March 26, 2025 03:16
@codecov-commenter
Copy link

codecov-commenter commented Mar 26, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 45.81%. Comparing base (ef31e09) to head (5196ceb).
Report is 516 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1952      +/-   ##
==========================================
+ Coverage   35.91%   45.81%   +9.89%     
==========================================
  Files          69       79      +10     
  Lines       11576    12912    +1336     
==========================================
+ Hits         4157     5915    +1758     
+ Misses       7104     6650     -454     
- Partials      315      347      +32     

see 77 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

}

if (!JsonArrayIterate(
configuration, "conditionalModelMappings", [&](const json& conditional_item) -> bool {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to add unit tests, and also need to provide documentation

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@CH3CHO CH3CHO force-pushed the feat/model-mapping-on-consumer branch 3 times, most recently from cbef9a5 to ae67361 Compare March 27, 2025 05:45
@CH3CHO CH3CHO force-pushed the feat/model-mapping-on-consumer branch from ae67361 to 5196ceb Compare May 26, 2025 06:46
@CH3CHO CH3CHO closed this May 26, 2025
@CH3CHO CH3CHO deleted the feat/model-mapping-on-consumer branch May 26, 2025 06:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants