test: Resolve Cypress Flaky fixes #3034
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Workday/canvas-kit
|
||||||||||||||||||||||||||||||||||||||||
| Project |
Workday/canvas-kit
|
| Branch Review |
fix-flaky-24-11
|
| Run status |
|
| Run duration | 02m 41s |
| Commit |
|
| Committer | Josh |
| View all properties for this run ↗︎ | |
| Test results | |
|---|---|
Failures
|
0
|
Flaky
|
0
|
Pending
|
21
|
Skipped
|
0
|
Passing
|
923
|
| View all changes introduced in this branch ↗︎ | |
UI Coverage
21.26%
|
|
|---|---|
|
Untested elements
|
1520
|
|
Tested elements
|
408
|
Accessibility
99.14%
|
|
|---|---|
|
Failed rules
|
6 critical
5 serious
0 moderate
2 minor
|
|
Failed elements
|
162
|
…ix-flaky-24-11
josh-bagwell
added
ready for review
mannycarrera4
approved these changes
Nov 25, 2024
mannycarrera4
changed the title
|
|
||
| context('when the dropdown menu is toggled with a keypress', () => { | ||
| it('should set focus to the first menu item', () => { | ||
| // cy.findByRole('menuitem', {name: 'Second Link'}).focus(); |
There was a problem hiding this comment.
Suggested change
| // cy.findByRole('menuitem', {name: 'Second Link'}).focus(); |
| }); | ||
| it('should toggle focus to the second menu item on down keypress', () => { | ||
| cy.findByRole('menuitem', {name: 'Third Link'}).should('exist'); | ||
| // cy.findByRole('menuitem', {name: 'Second Link'}).should('have.focus'); |
There was a problem hiding this comment.
Suggested change
| // cy.findByRole('menuitem', {name: 'Second Link'}).should('have.focus'); |
| cy.focused().realType('{downarrow}'); | ||
| cy.focused().realType('{downarrow}'); | ||
| cy.focused().realType('{downarrow}'); | ||
| cy.get('[role="menuitem"]').first().should('contain', 'Second Link'); |
There was a problem hiding this comment.
Suggested change
| cy.get('[role="menuitem"]').first().should('contain', 'Second Link'); | |
| cy.findByRole('menuitem').first().should('contain', 'Second Link'); |
| @@ -170,40 +167,44 @@ describe('Menu', () => { | |||
| }); | |||
|
|
|||
| it.skip('should focus on the third item', () => { | |||
| cy.findByRole('menuitem', {name: 'Third Item'}).should('have.focus'); | |||
| cy.get('[role="menuitem"]').should('contains', 'Third Item'); | |||
There was a problem hiding this comment.
Suggested change
| cy.get('[role="menuitem"]').should('contains', 'Third Item'); | |
| cy.findByRole('menuitem').should('contains', 'Third Item'); |
| 'have.attr', | ||
| 'aria-labelledby' | ||
| ); | ||
| cy.get('[role="dialog"]').should('exist'); |
There was a problem hiding this comment.
I don't think get makes a difference here, can we change these back to findByRole?
| @@ -702,7 +702,7 @@ context(`given the 'Iframe Test' story is rendered`, () => { | |||
| }); | |||
|
|
|||
| it('should focus on the close button', () => { | |||
| cy.findByRole('button', {name: 'Close'}).should('have.focus'); | |||
| cy.get('button[aria-label="Close"]').should('have.focus'); | |||
josh-bagwell
added
automerge
and removed
ready for review
|
Merging this PR now and will update with Manny suggestions at a later date |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Fixes: #2900
This is adding fixes for the Cypress flaky specs that were caused from the Webpack Upgrade.
This is addressing Flaky results for Cypress. A test is considered to be flaky when it can pass and fail across multiple retry attempts without any code changes. For example, a test is executed and fails, then the test is executed again, without any change to the code, but this time it passes. See more about Flaky here.
A Flaky result only happens in the CI and not in local since running Cypress locally only gives
passorfailresults.We had recently switched from E2E tests (end-to-end) to Component tests when upgrading Storybook and this had created flaky results on some of our components. We think this may be due to the nature of the Component runner as it is so much quicker than E2E. Due to it's nature of being much quicker, it resulted in flaky results due to our components not being to move as quick as the component runner. There may be times where the Component runner would move quicker than an item being focused in a menu or an item switching focus from one item to the next.
To address the Flaky issues, there have been updates made to some Component Specs:
findByRole()instead of usingget(). This allows us to be more specific about what element it is grabbingbeforeEachthan is needed and improves legibility.Release Category
Test
Where Should the Reviewer Start?
/cypress/component