If a user modifies .spec field of Application on Managed Agent (or any non-source of truth peer), it is not reconciled back

[jgwest]

If a user modifies the .spec field of an Application on Managed Agent (or any non-source of truth peer), it is not reconciled back to the canonical value from principal.

Arguable whether this is an enhancement or a bug, it's right on the line between the two 😄. Feel free to re-classify.

Consider the following scenario:

1. Principal and managed agent are connected. They share one Application appA.
2. A user on managed agent cluster modifies the .spec field of Application.

• (Same scenario but for principal and autonomous agent)

Expected Behaviour:

• Managed-agent detects that the .spec field has been modified, and reconciles the Application's .spec field back to the canonical value from principal.

Actual Behaviour:

• Managed-agent does not reconcile .spec field back (in fact, I think it treats it as a .status update instead?) My expectation is that it will only be reconciled back when the principal is next reconciled.

At least two possible solutions:

• A) When managed agent detects a .spec change, it should send request-update message with checksum. Principal will reply with canonical version of the .spec if the checksum differs. Agent will update back to that.
• B) Agent maintains an in-memory copy of all applications, and reconciles them back to that value on detected spec change.

My personal preference is A, since it is simpler (given that it fits with other logic we already implementing). However A) would not cover all cases: for example, what if managed agent is temporarily disconnected from principal (e.g. the managed agent on a cruise ship scenario).

I think B) is a better solution, as it will better handle all scenarios, for example, what if managed agent is temporarily disconnected from principal (e.g. the managed agent on a cruise ship scenario). This solution will be more complex, as there are two hard problems in computer science, cache invalidation and naming things.

Issue Criteria:

• Peer (e.g. managed agent) detects and revert changes to the .spec field of resource (e.g. Application) back to canonical value.
• E2E test which verifies the issue is resolved.