feat(inflightloadproducer): allow configuring output ratio

[asharkhan3101]

What type of PR is this?

/kind feature

What this PR does / why we need it:

This PR add a new configurable outputRatio parameter to inflightloaderproducer defaulting to same 1.5 if unspecified. This allows user to better choose a ratio that represents their traffic pattern for input to output ratio. 1.5x default doesn't work very well for high ISL workload with lot of contexts (200K or above). the estimated output token of 300K is very high in those cases.

Allowing it to be configured allows user to choose a value that best matches their workload.

Which issue(s) this PR fixes:

Fixes #1730

Release note (write NONE if no user-facing change):

Add parameter `outputRatio` to inflight load producer for user configurable output token estimation

[Copilot]

Pull request overview

Adds a user-configurable outputRatio to the in-flight load producer so output token estimation can be tuned to workload characteristics (while preserving the current 1.5 default when unset), addressing issue #1730.

Changes:

• Introduces DefaultOutputRatio and a NewSimpleTokenEstimatorWithRatio(...) constructor to parameterize output estimation.
• Extends inflight-load-producer config with optional outputRatio (validated as non-negative) and wires it into the producer’s token estimator.
• Documents the new parameter and adds factory-level tests covering default/custom/zero/negative values.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

File	Description
pkg/epp/framework/plugins/requestcontrol/dataproducer/inflightload/token_estimator.go	Adds a default ratio constant and a ratio-configurable estimator constructor.
pkg/epp/framework/plugins/requestcontrol/dataproducer/inflightload/README.md	Documents the new outputRatio parameter and its behavior.
pkg/epp/framework/plugins/requestcontrol/dataproducer/inflightload/producer.go	Adds outputRatio to config, validates it, and injects it into the token estimator.
pkg/epp/framework/plugins/requestcontrol/dataproducer/inflightload/producer_test.go	Adds tests ensuring outputRatio defaults/overrides/validation behave as intended.

[asharkhan3101]

Presence of max_tokens (if present) should be a better indicator for output token estimation and there are some other approaches that @LukeAVanDrie suggested in the thread, those can be a follow up. For now this offers some flexibility in output estimation.

[LukeAVanDrie]

Presence of max_tokens (if present) should be a better indicator for output token estimation and there are some other approaches that @LukeAVanDrie suggested in the thread, those can be a follow up. For now this offers some flexibility in output estimation.

Thanks, I agree; this is still valuable to ship, but it is papering over a structural flaw that the tunable slope doesn't cure: output is not proportional to input.

Two cheap changes can land in a follow-up:

1. Honor the request's own max_tokens / max_completion_tokens / max_output_tokens as you suggested. These already exist and are handled elsewhere in the repo (pkg/sidecar/proxy/proxy.go, the connectors), but the estimator ignores them. When a client supplies one, it's a real, free upper bound. Best estimate is roughly min(input × ratio, requestMaxTokens) when the cap is present.
2. Cap the estimate (min(input × ratio, maxOutputTokens)).

/approve

/lgtm