Add more context to "removing consolidatable status condition" message

[grosser]

Description

What problem are you trying to solve?

we want to track down why a certain node was not considered for consolidation,
for example "pod xyz was not disruptable"

for example we have a 16xl node that is running almost empty but karpenter only says

"marking consolidatable"
"removing consolidatable status condition"
... loop forever

and no indication of what is going on
... because there is a pod with a bad pdb sitting on it

How important is this feature to you?

Medium: would allow us to track down why nodes are wasting so much resources

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

[jonathan-innis]

We remove this status condition from the NodeClaim when our lastPodEventTime stored in the status is within the consolidateAfter time window. This time is updated when there is a pod add or removal event for this node. I think it might be tough for us to directly map that pod event time back to the pod that actually caused us to remove Consolidatable.

Maybe we just store this data in memory and make a best-effort on it? That might work

[jonathan-innis]

/triage accepted

[jonathan-innis]

/priority important-longterm

[jonathan-innis]

/help

[k8s-ci-robot]

@jonathan-innis:
This request has been marked as needing help from a contributor.

Guidelines

Please ensure that the issue body includes answers to the following questions:

• Why are we solving this issue?
• To address this issue, are there any code changes? If there are code changes, what needs to be done in the code and what places can the assignee treat as reference points?
• Does this issue have zero to low barrier of entry?
• How can the assignee reach out to you for help?

For more details on the requirements of such an issue, please see here and ensure that they are met.

If this request no longer meets these requirements, the label can be removed
by commenting with the /remove-help command.

In response to this:

/help

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[jonathan-innis]

/size s