Add metalctl command line interface

[SzymonSAP]

Proposed Changes

This PR adds metaclt move command with it's flags described in a issue #146.

To discuss

• How to ensure the CRDs and CRs already existing in the target cluster are the same as in the source targets?
  done
• How to ensure the CRDs and CRs have been successfully created in the target cluster?
  done

Fixes #146

[afritzler]

Thanks @SzymonSAP. Can you open the PR directly against the main branch?

[SzymonSAP]

Thanks @SzymonSAP. Can you open the PR directly against the main branch?

Sure, done it. Please let me know if the changes are on the right track and share any ideas what edge cases should also be covered.
I wonder about scenario in which there are already some CRDs or CRs in target cluster that are identical to source cluster. In such scenario metalctl move could be run successfully but it requires some code changes. Is it a real world use case?

[Nuckal777]

Thanks, for looking into this. 👍 Besides some small stuff, I think to larger topics are missing.

Firstly, the status of custom resources need to be copied over. This is especially important for Servers, where the status contains the lifecycle phase. Not copying the status here, would re-provision running servers (due to the reset into the discovery phase). One needs to use cl.Status().Update() to set the status after creation of a resource.

Secondly, the metal-operator uses owner references on many resources. They cannot be moved directly as they contain the UID of the owner, which is assigned on creation at random. To move these the owner needs to be looked up in the target cluster by name to patch the UID. E.g.

var endpoint metalv1alphav1.Endpoint
if err := clients.target.Get(ctx, types.NamespacedName{Name: owner.Name}, &endpoint); err != nil {
	return err
}
cr.OwnerReferences[0].UID = endpoint.UID

As a side effect this requires ordering the creation of CRs properly. E.g. Endpoints need to be handled first, then BMC and BMCSecrets and so on. In that regard it may be worth thinking about an "integration" test, which creates an Endpoint against the redfish simulator in source cluster with the metal-operator attached and then try moving.

[Nuckal777]

Thanks for implementing the handling of owner references. The implementation seems fine on a quick glance. 👍 I've added a few comments on stuff I'm unsure about.

[afritzler]

Can you please rebase this PR branch and use #179 to add the license headers to all net-new go files?

[SzymonSAP]

Can you please rebase this PR branch and use #179 to add the license headers to all net-new go files?

I have pushed requested changes and rebased the PR. Please have a look ;)

[Nuckal777]

I had a short alignment with @afritzler today. Here are the 2 main outcomes:

• We don't like to move CRDs with the move command. It is expected, that they are present in the target cluster beforehand, either via make install or helm install. You can remove the related code (Creation of CRs without the repsective CRD fails automatically). Already done. 👍
• We keep the unstructured.Unstructured approach for moving CRs, because we would like to re-use it for the boot-operator. Could you encapsulate a Move(ctx context.Context, clients Clients, crsToBeMoved apimachinery.GroupVersionKind) (or similar) function into a dedicated package/directory, please? After merging here, we can move that package to ironcore-dev/controllerutils.

[Nuckal777]

I think the status subresource creation is missing.

[SzymonSAP]

I have added status update after getting CR however in tests cl.Create creates object with status so I'm not sure if it will work on real environment. It would be good to test using clusters.

[afritzler]

Also it would make sense to add new documentation section Usage/metalctl and describe the use of the new CLI.

[afritzler]

I would leave out the registry server out of this test suite.

[SzymonSAP]

done ;)

[afritzler]

@SzymonSAP can you add above this line https://github.com/ironcore-dev/metal-operator/blob/main/internal/controller/bmc_controller_test.go#L14

- Usage:
    - metalctl: usage/metalctl.md

And describe in the dosc/usage/metalctl.md how to install and use the metalctl command line tool?

[SzymonSAP]

Also it would make sense to add new documentation section Usage/metalctl and describe the use of the new CLI.

I added documentation strongly inspired by https://cluster-api.sigs.k8s.io/clusterctl/commands/move.html