Workloads requesting TAS cannot run via MultiKueue

[mimowo]

What happened:

We cannot run workloads requesting TAS via MultiKueue. This is problematic, because TAS workloads have dedicated annotations like "preferred: rack". However workloads with such annotations only match TAS ResourceFlavors.

What you expected to happen:

We can configure Kueue to schedule workloads requesting TAS via MultiKueue. They should execute on the workers, but we should have a way to run them.

The natural approach is to configure TAS RF on the management cluster (both on manager and worker).

How to reproduce it (as minimally and precisely as possible):

1. Setup MultiKueue and configure CQ using TAS ResourceFlavor. The CQ will get deactivated immediately because of the soft validation we have:

   kueue/pkg/cache/clusterqueue.go

   Lines 243 to 246 in 4bfc9a0

   	if len(c.multiKueueAdmissionChecks) > 1 {
   	reasons = append(reasons, kueue.ClusterQueueActiveReasonMultipleMultiKueueAdmissionChecks)
   	messages = append(messages, fmt.Sprintf("Cannot use multiple MultiKueue AdmissionChecks on the same ClusterQueue, found: %v", stringsutils.Join(c.multiKueueAdmissionChecks, ",")))
   	}

[mimowo]

/assign @IrvingMg
who already started to look into running TAS workloads over MultiKueue and hit the issue.

cc @mwysokin @mwielgus @tenzen-y @mszadkow

[kerthcet]

Hey, what's the current status with multiKueue + TAS? No updates on website I believe.

[tenzen-y]

Hey, what's the current status with multiKueue + TAS? No updates on website I believe.

As described in this issue, TAS+MultiKueue does not work well even though you use Kueue v0.12.0.

[tenzen-y]

As a feature stage, TAS is alpha, and MultiKueue is Beta. However, we do not have any feature stage against that combination.

[kerthcet]

Thanks.