Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flaky test: test_tracked_slugs #4924

Closed
greg-hellings opened this issue Aug 22, 2024 · 5 comments
Closed

Flaky test: test_tracked_slugs #4924

greg-hellings opened this issue Aug 22, 2024 · 5 comments

Comments

@greg-hellings
Copy link

When running the test suite, I get intermittent failures from the test_tracked_slugs test. Running the suite back to back in the same environment I got one failure and one success. Failure as follows:

python3.12-textual> =================================== FAILURES ===================================                                                                                                                                          
python3.12-textual> ______________________ test_tracked_slugs[ test -test-5] _______________________                                                                                                                                          
python3.12-textual> [gw13] linux -- Python 3.12.4 /nix/store/l014xp1qxdl6gim3zc0jv3mpxhbp346s-python3-3.12.4/bin/python3.12                                                                                                                   
python3.12-textual> tracker = <textual._slug.TrackedSlugs object at 0x7fffee7e4e30>, text = ' test '                                                                                                                                          
python3.12-textual> expected = 'test-5'                                                                                                                                                                                                       
python3.12-textual>     @pytest.mark.xdist_group("group2")                                                                                                                                                                                    
python3.12-textual>     @pytest.mark.parametrize(                                                                                                                                                                                             
python3.12-textual>         "text, expected",                                                                                                                                                                                                 
python3.12-textual>         [                                                                                                                                                                                                                 
python3.12-textual>             ("test", "test"),                                                                                                                                                                                             
python3.12-textual>             ("test", "test-1"),                                                                                                                                                                                           
python3.12-textual>             ("test", "test-2"),                                                                                                                                                                                           
python3.12-textual>             ("-test-", "-test-"),                                                                                                                                                                                         
python3.12-textual>             ("-test-", "-test--1"),                                                                                                                                                                                       
python3.12-textual>             ("test!\"#$%&'()*+,-./:;<=>?@[]^_`{|}~test", "test-_test"),                                                                                                                                                   
python3.12-textual>             ("test!\"#$%&'()*+,-./:;<=>?@[]^_`{|}~test", "test-_test-1"),                                                                                                                                                 
python3.12-textual>             ("tëst", "t%C3%ABst"),                                                                                                                                                                                        
python3.12-textual>             ("tëst", "t%C3%ABst-1"),                                                                                                                                                                                      
python3.12-textual>             ("tëst", "t%C3%ABst-2"),                                                                                                                                                                                      
python3.12-textual>             ("test🙂test", "testtest"),                                                                                                                                                                                   
python3.12-textual>             ("test🤷test", "testtest-1"),                                                                                                                                                                                 
python3.12-textual>             ("test🤷🏻‍♀️test", "testtest-2"),                                                                                                                                                                              python3.12-textual>             ("test", "test-3"),                                                                                                                                                                                           
python3.12-textual>             ("test", "test-4"),                                                                                                                                                                                           
python3.12-textual>             (" test ", "test-5"),                                                                                                                                                                                         
python3.12-textual>         ],                                                                                                                                                                                                                
python3.12-textual>     )                                                                                                                                                                                                                     
python3.12-textual>     def test_tracked_slugs(tracker: TrackedSlugs, text: str, expected: str) -> None:                                                                                                                                      
python3.12-textual>         """The tracked slugging class should produce the expected slugs."""                                                                                                                                               
python3.12-textual> >       assert tracker.slug(text) == expected                                                                                                                                                                             
python3.12-textual> E       AssertionError: assert 'test' == 'test-5'                                                                                                                                                                         
python3.12-textual> E                                                                                                                                                                                                                         
python3.12-textual> E         - test-5                                                                                                                                                                                                        
python3.12-textual> E         ?     --                                                                                                                                                                                                        
python3.12-textual> E         + test
python3.12-textual> tests/test_slug.py:64: AssertionError
python3.12-textual> =========================== short test summary info ============================
python3.12-textual> FAILED tests/test_slug.py::test_tracked_slugs[ test -test-5] - AssertionError: assert 'test' == 'test-5'
python3.12-textual> ============ 1 failed, 2572 passed, 1 skipped, 3 xfailed in 13.93s =============
python3.12-textual> /nix/store/1r32fki5z5ivgaikgh8pq0r6vz3b0jpj-stdenv-linux/setup: line 1655: pop_var_context: head of shell_variables not a function context
error: builder for '/nix/store/89vr75dzk1mlpvmmmj2krpr6iafycc7x-python3.12-textual-0.76.0.drv' failed with exit code 1;
       last 10 log lines:
       > E         
       > E         - test-5
       > E         ?     --
       > E         + test
       >
       > tests/test_slug.py:64: AssertionError
       > =========================== short test summary info ============================
       > FAILED tests/test_slug.py::test_tracked_slugs[ test -test-5] - AssertionError: assert 'test' == 'test-5'
       > ============ 1 failed, 2572 passed, 1 skipped, 3 xfailed in 13.93s =============
       > /nix/store/1r32fki5z5ivgaikgh8pq0r6vz3b0jpj-stdenv-linux/setup: line 1655: pop_var_context: head of shell_variables not a function context
       For full logs, run 'nix log /nix/store/89vr75dzk1mlpvmmmj2krpr6iafycc7x-python3.12-textual-0.76.0.drv'.

Running the test suite again, immediately afterwards, resulted in a success. I can only assume there is some sort of race condition or non-deterministic behavior in ordering of the tests.

Copy link

Thank you for your issue. Give us a little time to review it.

PS. You might want to check the FAQ if you haven't done so already.

This is an automated reply, generated by FAQtory

@darrenburns
Copy link
Member

You'll need to pass the --dist=loadgroup option (from pytest-xdist) when running the tests in parallel. These tests have some shared state (which is probably something that should change in itself).

@greg-hellings
Copy link
Author

Ah, nice. I suppose there's not a way to force pytest to group them in the code? A quick search didn't turn anything up, for me.

@darrenburns
Copy link
Member

Not that I'm aware of, sorry.

@darrenburns darrenburns closed this as not planned Won't fix, can't repro, duplicate, stale Sep 12, 2024
Copy link

Don't forget to star the repository!

Follow @textualizeio for Textual updates.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants