Skip to content

Deprecate descheduler.alpha.kubernetes.io/evict in favor of descheduler.beta.kubernetes.io/evict with Boolean logic #1673

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

tiraboschi
Copy link

@tiraboschi tiraboschi commented Apr 24, 2025

In the past the descheduler was only checking the presence of descheduler.alpha.kubernetes.io/evict annotation to force a pod to be an eviction candidate.
descheduler.alpha.kubernetes.io/evict=false was producing the same effect of descheduler.alpha.kubernetes.io/evict=true.
If the user wants to exclude a specific pod from the descheduler actions, the suggested solution was to re-create it
setting an higher priority.
This could potentially work for best effort pods that can be easily recreated but it's not a viable solution for other use
cases like VMs (executed in pod) that can be temporarily excluded from the descheduler action without the need to reboot them.
Let's deprecate descheduler.alpha.kubernetes.io/evict introducing descheduler.beta.kubernetes.io/evict with proper Boolean logic.

Something like this has been already discussed in the past, see for instance:
#422
and the accepted conclusion was to set a custom (possibly non-preempting) priority classes with an higher priority for the pods that we want to exclude form the descheduling.
But I think that we still have a valid use case that is not covered by that alternative.
A priority class can only be set ad pod creation time and never amend while an annotation can be added and removed at any point in time during the lifecycle of a pod.
Now let's image that we have a long living pod (such for instance a VM running inside a pod) and the owner wants to dynamically exclude it from eviction risk only during specific period of time or he simply wants to amed his choice without the need to reboot the VM. Currently we cannot achieve that without recreating the pod while it would be really simple and intuitive with an annotation.

And the Node-pressure Eviction mechanism can still act as a safety net for corner cases where descheduler.alpha.kubernetes.io/evict=false is abused.

Fixes: #1659

@k8s-ci-robot k8s-ci-robot requested a review from damemi April 24, 2025 22:06
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign damemi for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot requested a review from JaneLiuL April 24, 2025 22:06
@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Apr 24, 2025
@k8s-ci-robot
Copy link
Contributor

Hi @tiraboschi. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@googs1025
Copy link
Member

/ok-to-test

@k8s-ci-robot k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Apr 24, 2025
@tiraboschi tiraboschi force-pushed the evict_bool branch 3 times, most recently from efaeeaa to ad6b457 Compare April 28, 2025 12:28
return found
// HaveEvictAnnotation checks if the pod have evict annotation and its value
func HaveEvictAnnotation(pod *v1.Pod) (bool, bool) {
svalue, found := pod.ObjectMeta.Annotations[evictPodAnnotationKey]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thoughts on using strconv.ParseBool?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it depends on how much we want to push in terms of backward compatibility: the current code is checking only the presence of the annotation.
strconv.ParseBool will return a syntaxError for values (like an empty string) that fails to be cleanly converted to a Bool. Using strconv.ParseBool and catching the error to consider it as a True to mimic backward compatibility is also an option.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't we consider "graduating" this feature to be descheduler.beta.kubernetes.io/evict and leave the alpha as is ?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this is also a great suggestion.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@tiraboschi
Copy link
Author

/retest

2 similar comments
@tiraboschi
Copy link
Author

/retest

@ricardomaraschini
Copy link
Contributor

/retest

@tiraboschi tiraboschi force-pushed the evict_bool branch 2 times, most recently from 7cb9d67 to 674df88 Compare May 14, 2025 11:52
@tiraboschi tiraboschi changed the title Use Boolean logic for descheduler.alpha.kubernetes.io/evict annotation Deprecate descheduler.alpha.kubernetes.io/evict in favor of descheduler.beta.kubernetes.io/evict with Boolean logic May 14, 2025
@tiraboschi tiraboschi force-pushed the evict_bool branch 2 times, most recently from 66531b7 to e1155ac Compare May 14, 2025 12:13
@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 20, 2025
…er.beta.kubernetes.io/evict with Boolean logic

In the past the descheduler was only checking the presence
of descheduler.alpha.kubernetes.io/evict annotation to force
a pod to be an eviction candidate.
descheduler.alpha.kubernetes.io/evict=false was producing the
same effect of descheduler.alpha.kubernetes.io/evict=true.
If the user wants to exclude a specific pod from the descheduler
actions, the suggested solution was to re-create it
setting an higher priority.
This could potentially work for best effort pods that can
be easily recreated but it's not a viable solution for other use
cases like VMs (executed in pod) that can be temporarily
excluded from the descheduler action without the need to reboot
them.
Let's deprecate descheduler.alpha.kubernetes.io/evict introducing
descheduler.beta.kubernetes.io/evict with proper Boolean logic.

Signed-off-by: Simone Tiraboschi <[email protected]>
@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 27, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Can't use descheduler.alpha.kubernetes.io/evict=false to disable evicton
5 participants