Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Adding metrics for request total, latency and size #177

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Adding metrics for request total, latency and size #177

wants to merge 1 commit into from

Conversation

courageJ
Copy link

@courageJ courageJ commented Jan 9, 2025 •
edited
Loading

The PR adds the metrics handler and basic metrics for requests.

The request latency metrics now will only work if the response mode is buffered.

The request counter only covers the success case now. A follow-up PR will be added to record request errors and update to record request counter for failure cases.

@linux-foundation-easycla Linux Foundation: EasyCLA
Copy link

linux-foundation-easycla bot commented Jan 9, 2025 •
edited
Loading

CLA Signed

The committers listed above are authorized under a signed CLA.

@k8s-ci-robot k8s-ci-robot added cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Jan 9, 2025
@k8s-ci-robot
Copy link
Contributor

Hi @courageJ. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@k8s-ci-robot k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. and removed cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. labels Jan 9, 2025
@courageJ
Copy link
Author

courageJ commented Jan 9, 2025

cc @liu-cong @ahg-g

@k8s-ci-robot k8s-ci-robot added needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jan 10, 2025
@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jan 10, 2025
@courageJ
Copy link
Author

cc @JeffLuoo

@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jan 11, 2025
@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jan 13, 2025
@liu-cong
Copy link
Contributor

/assign

liu-cong
liu-cong reviewed Jan 13, 2025
View reviewed changes
pkg/ext-proc/handlers/server.go Show resolved Hide resolved
pkg/manifests/ext_proc.yaml Outdated Show resolved Hide resolved
pkg/ext-proc/metrics/metrics.go Show resolved Hide resolved
liu-cong
liu-cong reviewed Jan 13, 2025
View reviewed changes
Copy link
Contributor

@liu-cong liu-cong left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A small comment, otherwise lgtm

pkg/ext-proc/handlers/server.go Outdated Show resolved Hide resolved
@liu-cong
Copy link
Contributor

/lgtm
/approve

@k8s-ci-robot k8s-ci-robot added lgtm "Looks good to me", indicates that a PR is ready to be merged. and removed lgtm "Looks good to me", indicates that a PR is ready to be merged. labels Jan 13, 2025
@liu-cong
Copy link
Contributor

/lgtm
/approve

@k8s-ci-robot k8s-ci-robot added lgtm "Looks good to me", indicates that a PR is ready to be merged. and removed lgtm "Looks good to me", indicates that a PR is ready to be merged. labels Jan 13, 2025
@courageJ
Copy link
Author

/retest

@k8s-ci-robot
Copy link
Contributor

@courageJ: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/retest

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@liu-cong
Copy link
Contributor

/lgtm
/approve

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jan 13, 2025
@courageJ
Copy link
Author

/assign @terrytangyuan

@ahg-g
Copy link
Contributor

ahg-g commented Jan 14, 2025

/hold

I would like to take a look, I will do so tomorrow if you don't mind

@k8s-ci-robot k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jan 14, 2025
@ahg-g
Copy link
Contributor

ahg-g commented Jan 15, 2025

/ok-to-test

@k8s-ci-robot k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Jan 15, 2025
ahg-g
ahg-g reviewed Jan 15, 2025
View reviewed changes
pkg/ext-proc/handlers/response.go Show resolved Hide resolved
pkg/ext-proc/main.go Outdated Show resolved Hide resolved
pkg/ext-proc/metrics/metrics_handler.go Outdated Show resolved Hide resolved
@courageJ
add request metrics
242a73d 
Signed-off-by: Jie WU <[email protected]>

add request metrics

Signed-off-by: Jie WU <[email protected]>

rename api and metrics

fix go mod

Adding metrics handler

Signed-off-by: Jie WU <[email protected]>

Adding metrics handler

Signed-off-by: Jie WU <[email protected]>

add request metrics

rename api and metrics

fix mod

Updated request metrics to be handled in server processing loop

Signed-off-by: Jie WU <[email protected]>

Updated request metrics to be handled in server processing loop

Signed-off-by: Jie WU <[email protected]>

fix go mod

Signed-off-by: Jie WU <[email protected]>

fix go mod

Signed-off-by: Jie WU <[email protected]>

remove preconfigured buffered response

Signed-off-by: Jie WU <[email protected]>

Add streamed response

Signed-off-by: Jie WU <[email protected]>

Handle latency with response

Signed-off-by: Jie WU <[email protected]>

refactor

Signed-off-by: Jie WU <[email protected]>

fmt

Signed-off-by: Jie WU <[email protected]>

fmt

Signed-off-by: Jie WU <[email protected]>

fmt

Signed-off-by: Jie WU <[email protected]>

refactor server

Signed-off-by: Jie WU <[email protected]>
@k8s-ci-robot
Copy link
Contributor

New changes are detected. LGTM label has been removed.

@k8s-ci-robot k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jan 16, 2025
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: courageJ, liu-cong
Once this PR has been reviewed and has the lgtm label, please ask for approval from terrytangyuan. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@netlify Netlify
Copy link

netlify bot commented Jan 16, 2025 •
edited
Loading

✅ Deploy Preview for gateway-api-inference-extension ready!

Name Link
🔨 Latest commit 242a73d
🔍 Latest deploy log https://app.netlify.com/sites/gateway-api-inference-extension/deploys/67891eb46a8c4d00087d5905
😎 Deploy Preview https://deploy-preview-177--gateway-api-inference-extension.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

ahg-g
ahg-g reviewed Jan 16, 2025
View reviewed changes
pkg/ext-proc/main.go
}
if err := svr.ListenAndServe(); err != http.ErrServerClosed {
klog.Fatalf("failed to start metrics HTTP handler: %v", err)
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we need to have this rbac protected so that only authorized users on the cluster can scrape the metrics, right?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See this example: https://github.com/kubernetes-sigs/lws/blob/b23f068de44d81a06f43a5a4fbe32993a19294ae/cmd/main.go#L123

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ahg-g ahg-g ahg-g left review comments

@liu-cong liu-cong liu-cong left review comments

@kfswain kfswain Awaiting requested review from kfswain

Assignees

@terrytangyuan terrytangyuan

@liu-cong liu-cong

Labels
cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@courageJ @k8s-ci-robot @liu-cong @ahg-g @terrytangyuan