Skip to content

Commit bf0865c

Browse files
committed
Update comments
Signed-off-by: Azam Ikram <azamikram@google.com>
1 parent e1758ca commit bf0865c

2 files changed

Lines changed: 5 additions & 3 deletions

File tree

pkg/epp/framework/plugins/scheduling/scorer/prefix/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
**Type:** `prefix-cache-scorer`
44

5-
Scores candidate endpoints based on how much of the request's prefix is already cached. It can consider both the *ratio* of the prefix matched and the *absolute length* of the match.
5+
Scores candidate endpoints based on how much of the request's prefix is already cached. It can consider both the *ratio* of the prefix matched and the *absolute length* of the match. By default it only considers the ratio of prefix matched.
66

77
## What it does
88

pkg/epp/framework/plugins/scheduling/scorer/prefix/plugin.go

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -34,9 +34,10 @@ import (
3434
type Config struct {
3535
// The name of the data producer that produces PrefixCacheMatchInfo.
3636
PrefixMatchInfoProducerName string `json:"prefixMatchInfoProducerName,omitempty"`
37-
// The weight of the absolute prefix length in the score, between 0.0 and 1.0.
37+
// The weight assigned to match length, between 0.0 and 1.0.
3838
MatchLengthWeight float64 `json:"matchLengthWeight,omitempty"`
39-
// The number of tokens at which prefill performance saturates.
39+
// Normalization factor for match length in terms of tokens.
40+
// Used only when MatchLengthWeight > 0.
4041
MatchLengthScaleTokens int `json:"matchLengthScaleTokens,omitempty"`
4142
}
4243

@@ -57,6 +58,7 @@ const (
5758
// Type is the unique identifier for the prefix cache scorer plugin.
5859
PrefixCacheScorerPluginType = "prefix-cache-scorer"
5960
// The default weight of the absolute match length in the score.
61+
// Set to 0 so by default only the match ratio is considered.
6062
defaultMatchLengthWeight = 0.0
6163
// Default number of tokens used as a scaling factor.
6264
defaultMatchLengthScaleTokens = 8192

0 commit comments

Comments
 (0)