[Umbrella] Controller Expectation

[kevin85421]

Search before asking

• I had searched in the issues and found no similar feature requirement.

Description

• [RayCluster][Fix] Add expectations of RayCluster #2150
• TODO: [RayCluster][Fix] Add expectations of RayCluster #2150 (review)

1. We store rayPod in the indexer, whereas ReplicaSet only stores an integer, which is more memory-efficient. We can run some benchmarks after this PR is merged to check for any memory issues. If so, we can switch to ReplicaSet's implementation.
2. There might be some corner cases for suspend. Maybe we can only call deleteAllPods when there are no in-flight requests (i.e. expectation is satisfied).
3. Possible memory leak if KubeRay misses the resource event.

• TODO: Run chaos tests with chaos mesh or krkn.

Use case

No response

Related issues

No response

Are you willing to submit a PR?

• Yes I am willing to submit a PR!