[GSoC] Summary for Project 6: Push-based Metrics Collection #155
Conversation
Signed-off-by: Electronic-Waste <[email protected]>
Signed-off-by: Electronic-Waste <[email protected]>
andreyvelich
left a comment
andreyvelich
left a comment
There was a problem hiding this comment.
Thank you for working on this @Electronic-Waste, and sorry for the late reply!
/assign @varodrig @hbelmiro @franciscojavierarceo @kubeflow/wg-training-leads @akgraner
Please can you help us with the review, so we can merge this great blog post ?
_posts/2024-09-28-gsoc-2024-summary-push-basd-metrics-collection.md
Outdated
Show resolved
Hide resolved
_posts/2024-09-28-gsoc-2024-summary-push-basd-metrics-collection.md
Outdated
Show resolved
Hide resolved
_posts/2024-09-28-gsoc-2024-summary-push-basd-metrics-collection.md
Outdated
Show resolved
Hide resolved
_posts/2024-09-28-gsoc-2024-summary-push-basd-metrics-collection.md
Outdated
Show resolved
Hide resolved
_posts/2024-09-28-gsoc-2024-summary-push-basd-metrics-collection.md
Outdated
Show resolved
Hide resolved
_posts/2024-09-28-gsoc-2024-summary-push-basd-metrics-collection.md
Outdated
Show resolved
Hide resolved
_posts/2024-09-28-gsoc-2024-summary-push-basd-metrics-collection.md
Outdated
Show resolved
Hide resolved
_posts/2024-09-28-gsoc-2024-summary-push-basd-metrics-collection.md
Outdated
Show resolved
Hide resolved
_posts/2024-09-28-gsoc-2024-summary-push-basd-metrics-collection.md
Outdated
Show resolved
Hide resolved
franciscojavierarceo
left a comment
franciscojavierarceo
left a comment
There was a problem hiding this comment.
This is awesome! Thank you @Electronic-Waste!! I left some small nits but otherwise lgtm.
varodrig
left a comment
varodrig
left a comment
There was a problem hiding this comment.
Great post! thank you so much for working on this. It seems you had a fantastic experience and you are sharing your thoughts with the community. Great story.
I left a couple of suggestions.
_posts/2024-09-28-gsoc-2024-summary-push-basd-metrics-collection.md
Outdated
Show resolved
Hide resolved
_posts/2024-09-28-gsoc-2024-summary-push-basd-metrics-collection.md
Outdated
Show resolved
Hide resolved
_posts/2024-09-28-gsoc-2024-summary-push-basd-metrics-collection.md
Outdated
Show resolved
Hide resolved
_posts/2024-09-28-gsoc-2024-summary-push-basd-metrics-collection.md
Outdated
Show resolved
Hide resolved
…on.md Co-authored-by: Francisco Arceo <[email protected]> Signed-off-by: Shao Wang <[email protected]>
…on.md Co-authored-by: Francisco Arceo <[email protected]> Signed-off-by: Shao Wang <[email protected]>
…on.md Co-authored-by: Francisco Arceo <[email protected]> Signed-off-by: Shao Wang <[email protected]>
…on.md Co-authored-by: Francisco Arceo <[email protected]> Signed-off-by: Shao Wang <[email protected]>
…on.md Co-authored-by: Francisco Arceo <[email protected]> Signed-off-by: Shao Wang <[email protected]>
…on.md Co-authored-by: Francisco Arceo <[email protected]> Signed-off-by: Shao Wang <[email protected]>
…on.md Co-authored-by: Francisco Arceo <[email protected]> Signed-off-by: Shao Wang <[email protected]>
…on.md Co-authored-by: Francisco Arceo <[email protected]> Signed-off-by: Shao Wang <[email protected]>
Signed-off-by: Electronic-Waste <[email protected]>
Signed-off-by: Electronic-Waste <[email protected]>
|
@franciscojavierarceo @franciscojavierarceo Thanks so much for your detailed reviews! Appreciate your patience and kindness. I've addressed all of your comments. PTAL if you have time:) |
|
/lgtm |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: franciscojavierarceo The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
|
||
| The current implementation of Metrics Collector is pull-based, raising design problems such as determining the frequency at which we scrape the metrics, performance issues like the overhead caused by too many sidecar containers, and restrictions on developing environments that must support sidecar containers and admission webhooks. And also, for data scientists, they need to pay attention to the format of metrics printed in the training scripts, which is error prone and may be hard to recognize. | ||
|
|
||
| ## Solution |
There was a problem hiding this comment.
@Electronic-Waste It would be also nice if you could update this information with these:
- Briefly explain how the new push-based API works
- Link to the Kubeflow docs for the mode information
- Add you Notebook example: https://github.com/kubeflow/katib/blob/master/examples/v1beta1/sdk/mnist-with-push-metrics-collection.ipynb
There was a problem hiding this comment.
Sure. I'll add these.
4 participants
I converted the issue kubeflow/katib#2340 into this blog.
PTAL when you have time @kubeflow/wg-automl-leads @terrytangyuan