Unskipping tea tests and adding protection around possible nil models #8053

ytimocin · 2024-11-12T23:34:01Z

Description

Un-skipping tea tests and adding protection around possible nil models.

Type of change

This pull request is a minor refactor, code cleanup, test improvement, or other maintenance task and doesn't change the functionality of Radius (issue link optional).
Fixes: #issue_number

Contributor checklist

Please verify that the PR meets the following requirements, where applicable:

An overview of proposed schema changes is included in a linked GitHub issue.
A design document PR is created in the design-notes repository, if new APIs are being introduced.
If applicable, design document has been reviewed and approved by Radius maintainers/approvers.
A PR for the samples repository is created, if existing samples are affected by the changes in this PR.
A PR for the documentation repository is created, if the changes in this PR affect the documentation or any user facing updates are made.
A PR for the recipes repository is created, if existing recipes are affected by the changes in this PR.

github-actions · 2024-11-12T23:45:12Z

Unit Tests

3 548 tests +5 3 546 ✅ +7 4m 59s ⏱️ +4s
285 suites ±0 2 💤 - 2
1 files ±0 0 ❌ ±0

Results for commit 046f346. ± Comparison against base commit 3383692.

♻️ This comment has been updated with latest results.

codecov · 2024-11-12T23:45:12Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 59.80%. Comparing base (3383692) to head (046f346).

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #8053      +/-   ##
==========================================
+ Coverage   59.65%   59.80%   +0.14%     
==========================================
  Files         577      577              
  Lines       38573    38573              
==========================================
+ Hits        23010    23067      +57     
+ Misses      13916    13856      -60     
- Partials     1647     1650       +3

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

radius-functional-tests · 2024-11-13T02:11:33Z

Radius functional test overview

🔍 Go to test action run

Name	Value
Repository	radius-project/radius
Commit ref	`8636b76`
Unique ID	funcf18396758b
Image tag	pr-funcf18396758b

Click here to see the list of tools in the current test run

gotestsum 1.12.0
KinD: v0.20.0
Dapr: 1.12.0
Azure KeyVault CSI driver: 1.4.2
Azure Workload identity webhook: 1.3.0
Bicep recipe location ghcr.io/radius-project/dev/test/testrecipes/test-bicep-recipes/<name>:pr-funcf18396758b
Terraform recipe location http://tf-module-server.radius-test-tf-module-server.svc.cluster.local/<name>.zip (in cluster)
applications-rp test image location: ghcr.io/radius-project/dev/applications-rp:pr-funcf18396758b
dynamic-rp test image location: ghcr.io/radius-project/dev/dynamic-rp:pr-funcf18396758b
controller test image location: ghcr.io/radius-project/dev/controller:pr-funcf18396758b
ucp test image location: ghcr.io/radius-project/dev/ucpd:pr-funcf18396758b
deployment-engine test image location: ghcr.io/radius-project/deployment-engine:latest

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting corerp-cloud functional tests...
⌛ Starting ucp-cloud functional tests...
✅ ucp-cloud functional tests succeeded
✅ corerp-cloud functional tests succeeded

pkg/cli/cmd/deploy/deploy_test.go

rynowak · 2024-11-13T06:15:27Z

pkg/cli/cmd/radinit/display_test.go

@@ -50,6 +48,11 @@ func Test_summaryModel(t *testing.T) {
 	waitForEmpty := func(t *testing.T, reader io.Reader) string {
 		normalized := ""
 		teatest.WaitFor(t, reader, func(bts []byte) bool {
+			if bts == nil {


Do we have a theory about why this would happen? Why is this the right fix?

rynowak · 2024-11-13T06:16:31Z

pkg/cli/prompt/text/text_test.go

+		tm.WaitFinished(t, teatest.WithFinalTimeout(waitTimeout))
+
+		finalModel := tm.FinalModel(t)
+		require.NotNil(t, finalModel, "Final model should not be nil")


Can you please explain why this is a good fix given the failure pattern from the original issue? #7670

I believe that there is sometimes a very short delay of model being updated. I am still in the process of proving this. The reason for my thinking is that we get a nil pointer exception, and we don't get it all the time. I think, but am not sure yet, that waiting for the model to be completely finished may fix the issue.

As mentioned above, I am working on learning more about the tea library and proving my point. :)

I believe that there is sometimes a very short delay of model being updated

That's my theory as well. The test is flaky, which means the root cause is probably a data-race.

These are example tests from the original repo: https://github.com/charmbracelet/x/blob/20117e9c8cd5ad229645f1bca3422b7e4110c96c/exp/teatest/app_test.go#L34.

radius-functional-tests · 2024-11-15T23:43:20Z

Radius functional test overview

🔍 Go to test action run

Name	Value
Repository	radius-project/radius
Commit ref	`5194103`
Unique ID	func24303fddb5
Image tag	pr-func24303fddb5

Click here to see the list of tools in the current test run

gotestsum 1.12.0
KinD: v0.20.0
Dapr: 1.12.0
Azure KeyVault CSI driver: 1.4.2
Azure Workload identity webhook: 1.3.0
Bicep recipe location ghcr.io/radius-project/dev/test/testrecipes/test-bicep-recipes/<name>:pr-func24303fddb5
Terraform recipe location http://tf-module-server.radius-test-tf-module-server.svc.cluster.local/<name>.zip (in cluster)
applications-rp test image location: ghcr.io/radius-project/dev/applications-rp:pr-func24303fddb5
dynamic-rp test image location: ghcr.io/radius-project/dev/dynamic-rp:pr-func24303fddb5
controller test image location: ghcr.io/radius-project/dev/controller:pr-func24303fddb5
ucp test image location: ghcr.io/radius-project/dev/ucpd:pr-func24303fddb5
deployment-engine test image location: ghcr.io/radius-project/deployment-engine:latest

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting corerp-cloud functional tests...
⌛ Starting ucp-cloud functional tests...
✅ ucp-cloud functional tests succeeded
✅ corerp-cloud functional tests succeeded

radius-functional-tests · 2024-12-15T05:21:51Z

Radius functional test overview

🔍 Go to test action run

Name	Value
Repository	radius-project/radius
Commit ref	`39f59c8`
Unique ID	func16427ecbb5
Image tag	pr-func16427ecbb5

Click here to see the list of tools in the current test run

gotestsum 1.12.0
KinD: v0.20.0
Dapr:
Azure KeyVault CSI driver: 1.4.2
Azure Workload identity webhook: 1.3.0
Bicep recipe location ghcr.io/radius-project/dev/test/testrecipes/test-bicep-recipes/<name>:pr-func16427ecbb5
Terraform recipe location http://tf-module-server.radius-test-tf-module-server.svc.cluster.local/<name>.zip (in cluster)
applications-rp test image location: ghcr.io/radius-project/dev/applications-rp:pr-func16427ecbb5
dynamic-rp test image location: ghcr.io/radius-project/dev/dynamic-rp:pr-func16427ecbb5
controller test image location: ghcr.io/radius-project/dev/controller:pr-func16427ecbb5
ucp test image location: ghcr.io/radius-project/dev/ucpd:pr-func16427ecbb5
deployment-engine test image location: ghcr.io/radius-project/deployment-engine:latest

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting ucp-cloud functional tests...
⌛ Starting corerp-cloud functional tests...
✅ ucp-cloud functional tests succeeded
✅ corerp-cloud functional tests succeeded

Signed-off-by: ytimocin <[email protected]>

ytimocin · 2024-12-15T06:00:02Z

pkg/cli/cmd/radinit/display_test.go

-		// Wait for final render and exit.
-		tm.WaitFinished(t, teatest.WithFinalTimeout(waitTimeout))
-		waitForEmpty(t, tm.FinalOutput(t))
+		if err := tm.Quit(); err != nil {


Please see example tests from the original repo: https://github.com/charmbracelet/x/blob/20117e9c8cd5ad229645f1bca3422b7e4110c96c/exp/teatest/app_test.go#L34. We can discuss about them.

This is a different test though right? We're now explicitly telling the model to quit.

radius-functional-tests · 2024-12-15T06:16:49Z

Radius functional test overview

🔍 Go to test action run

Name	Value
Repository	radius-project/radius
Commit ref	`046f346`
Unique ID	func67ec607427
Image tag	pr-func67ec607427

Click here to see the list of tools in the current test run

gotestsum 1.12.0
KinD: v0.20.0
Dapr:
Azure KeyVault CSI driver: 1.4.2
Azure Workload identity webhook: 1.3.0
Bicep recipe location ghcr.io/radius-project/dev/test/testrecipes/test-bicep-recipes/<name>:pr-func67ec607427
Terraform recipe location http://tf-module-server.radius-test-tf-module-server.svc.cluster.local/<name>.zip (in cluster)
applications-rp test image location: ghcr.io/radius-project/dev/applications-rp:pr-func67ec607427
dynamic-rp test image location: ghcr.io/radius-project/dev/dynamic-rp:pr-func67ec607427
controller test image location: ghcr.io/radius-project/dev/controller:pr-func67ec607427
ucp test image location: ghcr.io/radius-project/dev/ucpd:pr-func67ec607427
deployment-engine test image location: ghcr.io/radius-project/deployment-engine:latest

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting ucp-cloud functional tests...
⌛ Starting corerp-cloud functional tests...
✅ ucp-cloud functional tests succeeded
✅ corerp-cloud functional tests succeeded

ytimocin requested review from a team as code owners November 12, 2024 23:34

ytimocin had a problem deploying to functional-tests November 12, 2024 23:34 — with GitHub Actions Failure

ytimocin temporarily deployed to publish-bicep November 12, 2024 23:34 — with GitHub Actions Inactive

ytimocin force-pushed the ytimocin/tea-tes/nil-issue branch from 665240c to 2eb7adc Compare November 13, 2024 01:45

ytimocin had a problem deploying to functional-tests November 13, 2024 01:45 — with GitHub Actions Failure

ytimocin temporarily deployed to publish-bicep November 13, 2024 01:45 — with GitHub Actions Inactive

ytimocin force-pushed the ytimocin/tea-tes/nil-issue branch from 2eb7adc to 8636b76 Compare November 13, 2024 02:01

ytimocin temporarily deployed to functional-tests November 13, 2024 02:02 — with GitHub Actions Inactive

ytimocin temporarily deployed to publish-bicep November 13, 2024 02:02 — with GitHub Actions Inactive

rynowak reviewed Nov 13, 2024

View reviewed changes

pkg/cli/cmd/deploy/deploy_test.go Show resolved Hide resolved

rynowak reviewed Nov 13, 2024

View reviewed changes

ytimocin force-pushed the ytimocin/tea-tes/nil-issue branch from 8636b76 to 8a5c371 Compare November 15, 2024 21:35

ytimocin had a problem deploying to functional-tests November 15, 2024 21:36 — with GitHub Actions Failure

ytimocin temporarily deployed to publish-bicep November 15, 2024 21:36 — with GitHub Actions Inactive

ytimocin force-pushed the ytimocin/tea-tes/nil-issue branch from 8a5c371 to 5194103 Compare November 15, 2024 23:42

ytimocin temporarily deployed to functional-tests November 15, 2024 23:42 — with GitHub Actions Inactive

ytimocin temporarily deployed to publish-bicep November 15, 2024 23:42 — with GitHub Actions Inactive

ytimocin force-pushed the ytimocin/tea-tes/nil-issue branch from 5194103 to 2173639 Compare November 25, 2024 06:47

ytimocin requested a deployment to functional-tests November 25, 2024 06:47 — with GitHub Actions Waiting

ytimocin temporarily deployed to publish-bicep November 25, 2024 06:47 — with GitHub Actions Inactive

ytimocin force-pushed the ytimocin/tea-tes/nil-issue branch from 2173639 to 39f59c8 Compare December 15, 2024 05:20

ytimocin temporarily deployed to functional-tests December 15, 2024 05:20 — with GitHub Actions Inactive

ytimocin temporarily deployed to publish-bicep December 15, 2024 05:20 — with GitHub Actions Inactive

Unskipping tea tests and adding protection around possible nil models

046f346

Signed-off-by: ytimocin <[email protected]>

ytimocin force-pushed the ytimocin/tea-tes/nil-issue branch from 39f59c8 to 046f346 Compare December 15, 2024 05:59

ytimocin temporarily deployed to publish-bicep December 15, 2024 05:59 — with GitHub Actions Inactive

ytimocin temporarily deployed to functional-tests December 15, 2024 05:59 — with GitHub Actions Inactive

ytimocin commented Dec 15, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Unskipping tea tests and adding protection around possible nil models #8053

Unskipping tea tests and adding protection around possible nil models #8053

ytimocin commented Nov 12, 2024

github-actions bot commented Nov 12, 2024 •

edited

Loading

codecov bot commented Nov 12, 2024 •

edited

Loading

radius-functional-tests bot commented Nov 13, 2024 •

edited

Loading

rynowak Nov 13, 2024

rynowak Nov 13, 2024

ytimocin Nov 15, 2024

rynowak Nov 16, 2024

ytimocin Dec 15, 2024

radius-functional-tests bot commented Nov 15, 2024 •

edited

Loading

radius-functional-tests bot commented Dec 15, 2024 •

edited

Loading

ytimocin Dec 15, 2024

rynowak Dec 16, 2024

radius-functional-tests bot commented Dec 15, 2024 •

edited

Loading

Unskipping tea tests and adding protection around possible nil models #8053

Are you sure you want to change the base?

Unskipping tea tests and adding protection around possible nil models #8053

Conversation

ytimocin commented Nov 12, 2024

Description

Type of change

Contributor checklist

github-actions bot commented Nov 12, 2024 • edited Loading

Unit Tests

codecov bot commented Nov 12, 2024 • edited Loading

Codecov Report

radius-functional-tests bot commented Nov 13, 2024 • edited Loading

Radius functional test overview

Test Status

rynowak Nov 13, 2024

Choose a reason for hiding this comment

rynowak Nov 13, 2024

Choose a reason for hiding this comment

ytimocin Nov 15, 2024

Choose a reason for hiding this comment

rynowak Nov 16, 2024

Choose a reason for hiding this comment

ytimocin Dec 15, 2024

Choose a reason for hiding this comment

radius-functional-tests bot commented Nov 15, 2024 • edited Loading

Radius functional test overview

Test Status

radius-functional-tests bot commented Dec 15, 2024 • edited Loading

Radius functional test overview

Test Status

ytimocin Dec 15, 2024

Choose a reason for hiding this comment

rynowak Dec 16, 2024

Choose a reason for hiding this comment

radius-functional-tests bot commented Dec 15, 2024 • edited Loading

Radius functional test overview

Test Status

github-actions bot commented Nov 12, 2024 •

edited

Loading

codecov bot commented Nov 12, 2024 •

edited

Loading

radius-functional-tests bot commented Nov 13, 2024 •

edited

Loading

radius-functional-tests bot commented Nov 15, 2024 •

edited

Loading

radius-functional-tests bot commented Dec 15, 2024 •

edited

Loading

radius-functional-tests bot commented Dec 15, 2024 •

edited

Loading