The Future of placement.spec.locationResource & How Placements Are Extended

[stevekuznetsov]

In the first iterations of the Placement API, we had considered that there would be more than one type of Location, that would sync to different end states. However, we did not think about how a singular API surface in the Placement could provide the semantics necessary for all Locations.

Concretely, today we have two multi-cluster approaches: edge, and transparent. The transparent Placement provides an any-cast (or, sometimes, a spread/distributed multi-cast) semantic to transparent Locations, which refer to SyncTargets. The edge Placement provides a multi-cast (not spread, but replicated) semantic to edge Locations, which provide more meaning that transparent Locations in that they also encode physical proximity.

This does not match the hypothetical API that had been proposed at the start, since there is no distinct edge SyncTarget, but there are distinct edge Placements and Locations, with distinct semantics.

In this world, a transparent Placement cannot and should not ever lead deployments to land on an edge Location, and both edge and transparent Locations close over SyncTargets.

This, along with a slew of user feedback and confusion around the (transparent) placement.spec.locationResource field, lead me to think we should simply remove the feature after considering carefully what the best integration points really are here. Do we need a centralized set of objects, or is the "burden" of importing the right Placement object type to your workspace not that bad?

/cc @ncdc @sttts @davidfestal @jmprusi @MikeSpreitzer

[sttts]

However, we did not think about how a singular API surface in the Placement could provide the semantics necessary for all Locations.

I don't agree here. We did think about it. What changed is that we eventually made SyncTargets invisible to the users by introducing location workspaces and the compute admin persona.

[sttts]

Location plus Placement was intended to be universal for a class of scheduling problems. I am still convinced this is the case.

The only question is whether there are any real-world scheduling use-cases where having this dedicated API group with the inconvenience of specifying the locationResource is worth it. Plus that hiding SyncTargets from the users makes the API little awkward, which might be "just" a modelling problem we could solve differently.

With edge we have one data point where another domain needs placement, but the semantics are different, i.e. these bring their own Placement and Location kinds.

We might want to look more scheduling/placement problems, thinking of cloud service fleet-manager-like use-cases. Does the tmc inspired placement make sense in those contexts? I don't have the answer, but before we move forward we should do that exercise. My weak gut feeling is that actually the placement problems there are different, actually much simpler than in tmc.