Refactor and implement shared integration test host #8112

rynowak · 2024-12-09T22:57:49Z

Description

This change updates implements a shared (reusable) integration test host for the Radius control-plane services. The new integration test host enables us to do in-memory testing of UCP and dynamic-rp using a "full stack" approach.

This change is a significant refactor because the "glue" code in UCP had many points of divergence with the rest of our codebase. The following major changes are the bulk of the work:

Defining new types for configuration + options in UCP
Updating the UCP configuration file to match the format of other components

Type of change

This pull request is a minor refactor, code cleanup, test improvement, or other maintenance task and doesn't change the functionality of Radius (issue link optional).

Note: this is a pretty significant refactor, not a minor one 😆

Part of: #6688

Contributor checklist

Please verify that the PR meets the following requirements, where applicable:

An overview of proposed schema changes is included in a linked GitHub issue.
A design document PR is created in the design-notes repository, if new APIs are being introduced.
If applicable, design document has been reviewed and approved by Radius maintainers/approvers.
A PR for the samples repository is created, if existing samples are affected by the changes in this PR.
A PR for the documentation repository is created, if the changes in this PR affect the documentation or any user facing updates are made.
A PR for the recipes repository is created, if existing recipes are affected by the changes in this PR.

rynowak · 2024-12-09T23:06:03Z

Since this is a really big change, here's a suggested path for reviewers:

Read through the UCP startup code

This has been significantly simplified. There were a few goals for this, which support integration-testing broadly.

Make the configuration-file the source of truth. Remove env-vars and other ways of doing things. Make the configuration-file consistent with our other components.
Make sure all of the state is constructed only once. This is the UCP options. This is also sometimes called a composition root.

Read through the hello, world test for dynamic-rp

This uses the new integration-test hosting infrastructure. You'll notice that this code heavily uses the changes from step 1.

Now look at the rest of the code. The previous bullet-points motivate the vast majority of the changes.

radius-functional-tests · 2024-12-11T20:09:51Z

Radius functional test overview

🔍 Go to test action run

Name	Value
Repository	rynowak/radius
Commit ref	`b026be8`
Unique ID	func4c3a4a67ac
Image tag	pr-func4c3a4a67ac

Click here to see the list of tools in the current test run

gotestsum 1.12.0
KinD: v0.20.0
Dapr:
Azure KeyVault CSI driver: 1.4.2
Azure Workload identity webhook: 1.3.0
Bicep recipe location ghcr.io/radius-project/dev/test/testrecipes/test-bicep-recipes/<name>:pr-func4c3a4a67ac
Terraform recipe location http://tf-module-server.radius-test-tf-module-server.svc.cluster.local/<name>.zip (in cluster)
applications-rp test image location: ghcr.io/radius-project/dev/applications-rp:pr-func4c3a4a67ac
dynamic-rp test image location: ghcr.io/radius-project/dev/dynamic-rp:pr-func4c3a4a67ac
controller test image location: ghcr.io/radius-project/dev/controller:pr-func4c3a4a67ac
ucp test image location: ghcr.io/radius-project/dev/ucpd:pr-func4c3a4a67ac
deployment-engine test image location: ghcr.io/radius-project/deployment-engine:latest

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting corerp-cloud functional tests...
⌛ Starting ucp-cloud functional tests...
✅ ucp-cloud functional tests succeeded
✅ corerp-cloud functional tests succeeded

pkg/armrpc/asyncoperation/worker/service.go

pkg/dynamicrp/integrationtest/dynamic/providers_test.go

pkg/dynamicrp/testhost/host.go

pkg/recipes/controllerconfig/config.go

pkg/ucp/backend/service.go

ytimocin · 2024-12-11T20:33:45Z

pkg/ucp/frontend/controller/resourcegroups/util.go

 	typeSegments := id.TypeSegments()
 	if len(typeSegments) >= 2 && (strings.EqualFold(typeSegments[len(typeSegments)-1].Type, "operationstatuses") ||
-		strings.EqualFold(typeSegments[len(typeSegments)-1].Type, "operationresults")) {
+		strings.EqualFold(typeSegments[len(typeSegments)-1].Type, "operationresults") ||
+		strings.EqualFold(typeSegments[len(typeSegments)-1].Type, "operations")) {


Do we handle calls to /operations endpoint?

I don't understand the question, can you be more specific? I think the comments explain this pretty well already.

pkg/ucp/queue/inmemory/client.go

pkg/ucp/testhost/host.go

test/integrationtest/testhost/clients.go

radius-functional-tests · 2024-12-11T20:45:02Z

Radius functional test overview

🔍 Go to test action run

Name	Value
Repository	rynowak/radius
Commit ref	`8caadfd`
Unique ID	func6807a4696e
Image tag	pr-func6807a4696e

Click here to see the list of tools in the current test run

gotestsum 1.12.0
KinD: v0.20.0
Dapr:
Azure KeyVault CSI driver: 1.4.2
Azure Workload identity webhook: 1.3.0
Bicep recipe location ghcr.io/radius-project/dev/test/testrecipes/test-bicep-recipes/<name>:pr-func6807a4696e
Terraform recipe location http://tf-module-server.radius-test-tf-module-server.svc.cluster.local/<name>.zip (in cluster)
applications-rp test image location: ghcr.io/radius-project/dev/applications-rp:pr-func6807a4696e
dynamic-rp test image location: ghcr.io/radius-project/dev/dynamic-rp:pr-func6807a4696e
controller test image location: ghcr.io/radius-project/dev/controller:pr-func6807a4696e
ucp test image location: ghcr.io/radius-project/dev/ucpd:pr-func6807a4696e
deployment-engine test image location: ghcr.io/radius-project/deployment-engine:latest

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting corerp-cloud functional tests...
⌛ Starting ucp-cloud functional tests...
✅ ucp-cloud functional tests succeeded
✅ corerp-cloud functional tests succeeded

rynowak · 2024-12-13T18:00:42Z

Note: we're pausing this work to clean up the database interface separately. I'll update this PR when it's ready for reviews.

rynowak · 2024-12-18T17:11:23Z

FYI folks, this is going to get it a beeeeg rebase now 👍. Stay tuned ⌚⌚⌚⌚⌚⌚

Thanks for help reviewing the database and other layering changes.

/cc @ytimocin @kachawla @lakshmimsft

cmd/applications-rp/cmd/root.go

cmd/applications-rp/radius-dev.yaml

cmd/ucpd/ucp-dev.yaml

pkg/armrpc/asyncoperation/worker/service.go

pkg/components/testhost/clients.go

codecov · 2024-12-18T22:36:23Z

Codecov Report

Attention: Patch coverage is 70.29096% with 194 lines in your changes missing coverage. Please review.

Project coverage is 59.88%. Comparing base (d87e854) to head (4f160db).
Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
pkg/server/asyncworker.go	0.00%	41 Missing ⚠️
pkg/ucp/testhost/host.go	74.28%	27 Missing ⚠️
pkg/ucp/options.go	47.61%	17 Missing and 5 partials ⚠️
pkg/ucp/frontend/api/server.go	0.00%	17 Missing ⚠️
pkg/components/testhost/clients.go	94.11%	8 Missing and 2 partials ⚠️
pkg/dynamicrp/backend/service.go	58.33%	6 Missing and 4 partials ⚠️
pkg/dynamicrp/options.go	0.00%	9 Missing and 1 partial ⚠️
pkg/ucp/config.go	0.00%	10 Missing ⚠️
pkg/components/testhost/host.go	84.48%	6 Missing and 3 partials ⚠️
pkg/armrpc/asyncoperation/worker/service.go	0.00%	8 Missing ⚠️
... and 7 more

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #8112      +/-   ##
==========================================
+ Coverage   59.51%   59.88%   +0.36%     
==========================================
  Files         580      584       +4     
  Lines       39008    38939      -69     
==========================================
+ Hits        23216    23319     +103     
+ Misses      14079    13878     -201     
- Partials     1713     1742      +29

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

pkg/components/testhost/host.go

pkg/dynamicrp/backend/service.go

pkg/dynamicrp/options.go

pkg/ucp/frontend/controller/resourcegroups/util_test.go

pkg/ucp/frontend/modules/types.go

pkg/ucp/hostoptions/hostoptions.go

pkg/ucp/hostoptions/providerconfig.go

pkg/ucp/integrationtests/aws/awstest.go

pkg/ucp/integrationtests/handler_test.go

pkg/ucp/integrationtests/planes/aws_test.go

pkg/ucp/integrationtests/resourcegroups/resourcegroups_test.go

pkg/ucp/integrationtests/testserver/testserver.go

pkg/ucp/rest/objects.go

pkg/ucp/server/server.go

pkg/ucp/testhost/host.go

rynowak · 2024-12-18T22:59:41Z

I'm looking into the functional test failures. Likely I got something wrong in the configuration change.

pkg/ucp/config.go

lakshmimsft · 2024-12-18T23:45:43Z

/LGTM

radius-functional-tests · 2024-12-19T04:36:17Z

Radius functional test overview

🔍 Go to test action run

Name	Value
Repository	rynowak/radius
Commit ref	`30e655e`
Unique ID	func132b411980
Image tag	pr-func132b411980

Click here to see the list of tools in the current test run

gotestsum 1.12.0
KinD: v0.20.0
Dapr:
Azure KeyVault CSI driver: 1.4.2
Azure Workload identity webhook: 1.3.0
Bicep recipe location ghcr.io/radius-project/dev/test/testrecipes/test-bicep-recipes/<name>:pr-func132b411980
Terraform recipe location http://tf-module-server.radius-test-tf-module-server.svc.cluster.local/<name>.zip (in cluster)
applications-rp test image location: ghcr.io/radius-project/dev/applications-rp:pr-func132b411980
dynamic-rp test image location: ghcr.io/radius-project/dev/dynamic-rp:pr-func132b411980
controller test image location: ghcr.io/radius-project/dev/controller:pr-func132b411980
ucp test image location: ghcr.io/radius-project/dev/ucpd:pr-func132b411980
deployment-engine test image location: ghcr.io/radius-project/deployment-engine:latest

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting ucp-cloud functional tests...
⌛ Starting corerp-cloud functional tests...
✅ ucp-cloud functional tests succeeded
✅ corerp-cloud functional tests succeeded

cmd/applications-rp/radius-self-hosted.yaml

pkg/armrpc/asyncoperation/worker/service.go

pkg/dynamicrp/backend/service.go

pkg/dynamicrp/options.go

pkg/server/asyncworker.go

pkg/ucp/backend/service.go

pkg/ucp/frontend/radius/routes.go

This change updates implements a shared (reusable) integration test host for the Radius control-plane services. The new integration test host enables us to do in-memory testing of UCP and dynamic-rp using a "full stack" approach. This change is a significant refactor because the "glue" code in UCP had many points of divergence with the rest of our codebase. The following major changes are the bulk of the work: - Defining new types for configuration + options in UCP - Updating the UCP configuration file to match the format of other components Signed-off-by: Ryan Nowak <[email protected]>

pkg/ucp/config.go

radius-functional-tests · 2024-12-19T17:33:53Z

Radius functional test overview

🔍 Go to test action run

Name	Value
Repository	rynowak/radius
Commit ref	`4f160db`
Unique ID	func4f053b73c8
Image tag	pr-func4f053b73c8

Click here to see the list of tools in the current test run

gotestsum 1.12.0
KinD: v0.20.0
Dapr:
Azure KeyVault CSI driver: 1.4.2
Azure Workload identity webhook: 1.3.0
Bicep recipe location ghcr.io/radius-project/dev/test/testrecipes/test-bicep-recipes/<name>:pr-func4f053b73c8
Terraform recipe location http://tf-module-server.radius-test-tf-module-server.svc.cluster.local/<name>.zip (in cluster)
applications-rp test image location: ghcr.io/radius-project/dev/applications-rp:pr-func4f053b73c8
dynamic-rp test image location: ghcr.io/radius-project/dev/dynamic-rp:pr-func4f053b73c8
controller test image location: ghcr.io/radius-project/dev/controller:pr-func4f053b73c8
ucp test image location: ghcr.io/radius-project/dev/ucpd:pr-func4f053b73c8
deployment-engine test image location: ghcr.io/radius-project/deployment-engine:latest

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting corerp-cloud functional tests...
⌛ Starting ucp-cloud functional tests...
✅ ucp-cloud functional tests succeeded
✅ corerp-cloud functional tests succeeded

rynowak requested review from a team as code owners December 9, 2024 22:57

rynowak temporarily deployed to publish-bicep December 9, 2024 22:58 — with GitHub Actions Inactive

rynowak force-pushed the udt-integration-testing branch from 3e7fd28 to 3a24e2e Compare December 10, 2024 00:04

rynowak temporarily deployed to publish-bicep December 10, 2024 00:04 — with GitHub Actions Inactive

rynowak force-pushed the udt-integration-testing branch from 3a24e2e to a1b2e17 Compare December 10, 2024 00:26

rynowak temporarily deployed to publish-bicep December 10, 2024 00:26 — with GitHub Actions Inactive

rynowak force-pushed the udt-integration-testing branch from a1b2e17 to b026be8 Compare December 10, 2024 19:37

rynowak temporarily deployed to publish-bicep December 10, 2024 19:37 — with GitHub Actions Inactive

rynowak temporarily deployed to functional-tests December 11, 2024 20:08 — with GitHub Actions Inactive

ytimocin reviewed Dec 11, 2024

View reviewed changes

ytimocin temporarily deployed to publish-bicep December 11, 2024 20:42 — with GitHub Actions Inactive

ytimocin temporarily deployed to functional-tests December 11, 2024 20:43 — with GitHub Actions Inactive

rynowak added the do-not-merge Don't merge this PR until open discussion items or isssues are complete label Dec 13, 2024

rynowak force-pushed the udt-integration-testing branch from 8caadfd to a15b36c Compare December 18, 2024 22:23

rynowak temporarily deployed to publish-bicep December 18, 2024 22:24 — with GitHub Actions Inactive