Skip to content

feat: added video support for approximate prefix match#1911

Merged
liu-cong merged 20 commits into
llm-d:mainfrom
capri-xiyue:capri-xiyue/issue-1746
Jul 22, 2026
Merged

feat: added video support for approximate prefix match#1911
liu-cong merged 20 commits into
llm-d:mainfrom
capri-xiyue:capri-xiyue/issue-1746

Conversation

@capri-xiyue

@capri-xiyue capri-xiyue commented Jul 7, 2026 •
edited by liu-cong
Loading

Copy link
Copy Markdown
Contributor

What type of PR is this?

/kind feature
What this PR does / why we need it:
dded video support for approximate prefix match.

Which issue(s) this PR fixes:

Part of #1746

Release note (write NONE if no user-facing change):

Support video input in approximate prefix match via user provided video metadata http headers.

@capri-xiyue
draft
6d2914d 
Signed-off-by: Xiyue Yu <xiyue@google.com>
@capri-xiyue
added metadata
e749367 
Signed-off-by: Xiyue Yu <xiyue@google.com>
@capri-xiyue
added video estimation support
3d31a1d 
Signed-off-by: Xiyue Yu <xiyue@google.com>
@capri-xiyue
capri-xiyue requested review from a team, liu-cong, sagearc and vMaroon as code owners July 7, 2026 02:50
@github-actions github-actions Bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. kind/bug Categorizes issue or PR as related to a bug. labels Jul 7, 2026
@capri-xiyue capri-xiyue changed the title feat: added video support for approximate prefix match WIP feat: added video support for approximate prefix match Jul 7, 2026
@github-actions github-actions Bot added kind/cleanup kind/documentation Categorizes issue or PR as related to documentation. kind/feature Categorizes issue or PR as related to a new feature. kind/bug Categorizes issue or PR as related to a bug. and removed kind/feature Categorizes issue or PR as related to a new feature. kind/bug Categorizes issue or PR as related to a bug. kind/documentation Categorizes issue or PR as related to documentation. kind/cleanup labels Jul 7, 2026
@capri-xiyue

capri-xiyue commented Jul 7, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

@namgyu-youn
I drafted this to set up a framework for video estimation. A few thoughts based on experimenting your PR:

  • Extracting metadata directly from the video looks complex, so let's start by grabbing it from the request headers for now.
  • I simplified the token estimation logic in this PR to prioritize a clean user interface.

Let's keep iterating in your PR to figure out the best balance between accurate token estimation and a simple user experience. And extracting metadata from video directly could be a follow up PR if we think it is feasible.

And let me know your thoughts about this PR

capri-xiyue and others added 2 commits July 6, 2026 20:05
@github-actions github-actions Bot added kind/bug Categorizes issue or PR as related to a bug. kind/cleanup and removed kind/feature Categorizes issue or PR as related to a new feature. kind/bug Categorizes issue or PR as related to a bug. kind/documentation Categorizes issue or PR as related to documentation. kind/cleanup labels Jul 7, 2026
@github-actions github-actions Bot added the kind/feature Categorizes issue or PR as related to a new feature. label Jul 14, 2026
liu-cong
liu-cong reviewed Jul 21, 2026
View reviewed changes
Comment thread pkg/epp/framework/plugins/requestcontrol/dataproducer/tokenizer/tokenizer.go
@capri-xiyue capri-xiyue mentioned this pull request Jul 21, 2026
Closed
capri-xiyue and others added 2 commits July 21, 2026 09:47
@capri-xiyue
capri-xiyue requested a review from liu-cong July 21, 2026 19:01
@github-actions github-actions Bot added kind/bug Categorizes issue or PR as related to a bug. kind/cleanup kind/documentation Categorizes issue or PR as related to documentation. kind/feature Categorizes issue or PR as related to a new feature. and removed kind/feature Categorizes issue or PR as related to a new feature. kind/bug Categorizes issue or PR as related to a bug. kind/documentation Categorizes issue or PR as related to documentation. kind/cleanup labels Jul 21, 2026
liu-cong
liu-cong reviewed Jul 22, 2026
View reviewed changes
Comment thread pkg/epp/framework/plugins/requestcontrol/dataproducer/tokenizer/tokenizer.go Outdated
@capri-xiyue
renamed variable
28b8146 
Signed-off-by: Xiyue Yu <xiyue@google.com>
@capri-xiyue
capri-xiyue requested a review from liu-cong July 22, 2026 01:32
capri-xiyue and others added 5 commits July 21, 2026 18:32
@capri-xiyue
Merge branch 'main' into capri-xiyue/issue-1746
32b565a
@capri-xiyue
updated readme
b4e0120 
Signed-off-by: Xiyue Yu <xiyue@google.com>
@capri-xiyue
deleted debug logging
8709ecd 
Signed-off-by: Xiyue Yu <xiyue@google.com>
@capri-xiyue
fixed linter
ebefc64 
Signed-off-by: Xiyue Yu <xiyue@google.com>
@capri-xiyue
fixed linter
544f2ff 
Signed-off-by: Xiyue Yu <xiyue@google.com>
@liu-cong

liu-cong commented Jul 22, 2026

Copy link
Copy Markdown
Member

lint failed due to govul checks in other packages, not related to this pr, so merging

@liu-cong
liu-cong merged commit 4c6c93c into llm-d:main Jul 22, 2026
28 of 29 checks passed
@llm-d-router-release-notes llm-d-router-release-notes Bot mentioned this pull request Jul 22, 2026
Merged
elevran pushed a commit that referenced this pull request Jul 23, 2026
@llm-d-router-release-notes
docs: add release-note fragment for PR #1911 (#2144)
3a8286e 
Signed-off-by: llm-d-router-release-notes[bot] <287676111+llm-d-router-release-notes[bot]@users.noreply.github.com>
Co-authored-by: llm-d-router-release-notes[bot] <287676111+llm-d-router-release-notes[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@liu-cong liu-cong liu-cong approved these changes
@vMaroon vMaroon Awaiting requested review from vMaroon vMaroon is a code owner
@sagearc sagearc Awaiting requested review from sagearc sagearc is a code owner

Assignees

@liu-cong liu-cong

Labels

kind/bug Categorizes issue or PR as related to a bug. kind/cleanup kind/documentation Categorizes issue or PR as related to documentation. kind/feature Categorizes issue or PR as related to a new feature. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@capri-xiyue @namgyu-youn @liu-cong