Could This Cause Already Finished Workloads to Be Re-enqueued?

[chen-zyc]

// pkg/controller/core/workload_controller.go
func (h *resourceUpdatesHandler) queueReconcileForPending(ctx context.Context, _ workqueue.TypedRateLimitingInterface[reconcile.Request], opts ...client.ListOption) {
	log := ctrl.LoggerFrom(ctx)
	lst := kueue.WorkloadList{}
	opts = append(opts, client.MatchingFields{indexer.WorkloadQuotaReservedKey: string(metav1.ConditionFalse)})
	err := h.r.client.List(ctx, &lst, opts...)
	if err != nil {
		log.Error(err, "Could not list pending workloads")
	}
	log.V(4).Info("Updating pending workload requests", "count", len(lst.Items))
	for _, w := range lst.Items {
		wlCopy := w.DeepCopy()
		log := log.WithValues("workload", klog.KObj(wlCopy))
		log.V(5).Info("Queue reconcile for")
		workload.AdjustResources(ctrl.LoggerInto(ctx, log), h.r.client, wlCopy)
		if err = h.r.queues.AddOrUpdateWorkload(wlCopy); err != nil {
			log.V(2).Info("ignored an error for now", "error", err)
		}
	}
}

When a workload is marked as "finished," there is no QuotaReserved condition in its conditions field. As a result, the query here might re-add already finished workloads back into the queue. Is my understanding correct?

---

Sorry everyone, I believe this was my mistake. I was trying to run the Kueue component on a lower version of the Kubernetes cluster and made some modifications, which caused the Admitted condition to be lost when the workload reached the Finished state—leaving only the Finished condition. This led to the error mentioned above. However, this issue does not occur on a higher version of the Kubernetes cluster.

It’s my fault and I’ve wasted your time—apologies.

[mimowo]

That is a good question, definitely worth checking, as it could mean unnecessary work, but maybe we have some code around that which would protect this scenario, I'm not sure. If this is confirmed I would consider a bug.

Maybe @mbobrovskyi or @PBundyra know?

[PBundyra]

Have you observed such a behavior, that a finished workload is requeued after updates to e.g. LimitRange/RuntimeClass @chen-zyc?

[chen-zyc]

I noticed this when I restarted the kueue component. Upon restarting it, all my workloads were requeued here—including those that had already finished.

[chen-zyc]

Sorry everyone, I believe this was my mistake. I was trying to run the Kueue component on a lower version of the Kubernetes cluster and made some modifications, which caused the Admitted condition to be lost when the workload reached the Finished state—leaving only the Finished condition. This led to the error mentioned above. However, this issue does not occur on a higher version of the Kubernetes cluster.

It’s my fault and I’ve wasted your time—apologies.