You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Important sub-task: Support (1) different cache lengths per layer and (2) different cache types per layer. The latter would require some extra refactoring, since right now, cache types must be the same
Is your feature request related to a problem? Please describe.
Currently, all configurations must be provided via CL or hard-coded.
Describe the solution you'd like
I'd like to have a config option by a YAML file:
DenseKVCache#10