Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve logging when the cluster reaches max nodes total. #7648

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Improve logging when the cluster reaches max nodes total. #7648

wants to merge 1 commit into from
+13 −6

Conversation

jbtk
Copy link

@jbtk jbtk commented Dec 30, 2024 •
edited
Loading

  • add autoscaling status to reflect that
  • change the log severity to warning as this means that autoscaler will not be fully functional (in praticular scaling up will not work)

Note: the newly introduced status will be set only if there are no prov requests (and therefore there is no forced scale up). Otherwise the forced scaleup will override it.

What type of PR is this?

/kind cleanup

What this PR does / why we need it:

Expose more descriptive scale up status and log with higher severity when cluster reaches max total nodes.

Does this PR introduce a user-facing change?

Introduce new scale up status when cluster reaches max nodes total limit.

@k8s-ci-robot k8s-ci-robot added kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. area/cluster-autoscaler labels Dec 30, 2024
@k8s-ci-robot k8s-ci-robot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Dec 30, 2024
@k8s-ci-robot
Copy link
Contributor

Hi @jbtk. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@k8s-ci-robot k8s-ci-robot added size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Dec 30, 2024
@jbtk jbtk force-pushed the logging-max-nodes branch from 7e39e01 to c54c61e Compare January 7, 2025 10:55
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: jbtk
Once this PR has been reviewed and has the lgtm label, please assign bigdarkclown for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jan 7, 2025
@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jan 23, 2025
@jbtk
Improve logging when the cluster reaches max nodes total.
86ee2b7 
- add autoscaling status to reflect that
- change the log severity to warning as this means that autoscaler will
  not be fully functional (in praticular scaling up will not work)
- fix the scale up enforcer logic not to skip the max nodes reached
  logging point
@jbtk jbtk force-pushed the logging-max-nodes branch from c54c61e to 86ee2b7 Compare January 29, 2025 13:00
@k8s-ci-robot k8s-ci-robot added size/S Denotes a PR that changes 10-29 lines, ignoring generated files. and removed needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Jan 29, 2025
@jbtk
Copy link
Author

jbtk commented Jan 29, 2025

Note: the newly introduced status will be set only if there is no forced scale up. Otherwise the forced scaleup will override it.

This PR will revert to at least log in this case info that max nodes were reached.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@BigDarkClown BigDarkClown Awaiting requested review from BigDarkClown

@vadasambar vadasambar Awaiting requested review from vadasambar

Assignees
No one assigned
Labels
area/cluster-autoscaler cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. size/S Denotes a PR that changes 10-29 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@jbtk @k8s-ci-robot