Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Should parallel execution of tests work? #31

Open
esemusa opened this issue Jun 17, 2021 · 3 comments
Open

Should parallel execution of tests work? #31

esemusa opened this issue Jun 17, 2021 · 3 comments
Labels
enhancement New feature or request

Comments

@esemusa
Copy link

esemusa commented Jun 17, 2021

TLDR: Should parallel execution of tests work (by choosing Xcode parallelization)?

I can provide a minimum working example but I guess it's not needed as this does not seem to be a setup-dependent issue. But here is how to reproduce, trying to achieve test execution:

  • In Xcode, choose „Edit Scheme…“ (be sure to select the right one)
  • select the „Test“ tab
  • Select „Options“
  • select „Execute in parallel (if possible)“
  • Try to run tests as usual

My observation:

  • Xcode will start a new emulator „Cone 1 […]“.
  • The app is built
  • Tests won't run
  • In the logs, you'll find this and only this:
    • DATE CucumberSwiftParallelUITests-Runner[7950:157096] Running tests...
  • XCode will show the following execution feedback:
    • PROJECT | Build Succeeded | Today at TIME

My expectation:
If you define two .feature Files and run tests in parallel as stated above, the two .feature files should be executed in parallel

@Tyler-Keith-Thompson
Copy link
Owner

Short answer, yeah it should work. Can I get more information from you about what version of Xcode, CucumberSwift, and Swift you are using? Unfortunately Apple doesn't provide very good tie-ins to XCTest for this so, like many other libraries, I've got some swizzling happening to try and support parallel test execution and inject the Cucumber tests.

This is a bit of a tricky feature since that's prone to break with new releases, and because it's so darn hard to write a test for.

@esemusa
Copy link
Author

esemusa commented Jun 18, 2021

Xcode 12.5
Swift: 5.4
CucumberSwift 3.3.4

I downloaded your sample app project https://github.com/Tyler-Keith-Thompson/CucumberSwiftSample and also tried it there. Same result.

@Tyler-Keith-Thompson
Copy link
Owner

Alright after further investigation parallel test execution is working for unit testing targets, not working for UI testing targets. I suspect there's another method I need to swizzle in order to support it, I'll leave this as an enhancement request.

@Tyler-Keith-Thompson Tyler-Keith-Thompson added the enhancement New feature or request label Jun 26, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants