Skip to content

Add a function automatically search optimal config for a new model #915

@wedu-nvidia

Description

@wedu-nvidia

Currently, we need to manually search for optimal config, tp, cp, pp, etc given a new model.
We should support the automatic search for and report the final optimal one.

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions