Unhelpful stack traces for async functions with multiple async calls

[benlind]

What is your Test Scenario?

I have a test block that calls an async helper function. That helper function makes more than one async assertion. If any of those sub-assertions fail except for the first assertion, the error stack trace unhelpfully only lists the helper function and doesn't mention the line in the original test block where the helper function was called.

Since we use helper functions in multiple tests, this leaves us with no easy way to determine which helper invocation led to a logged error.

Here is a simple reproduction:

fixture`Example`

test
  .page('https://google.com')('loads Google', async t => {
    await myHelper(t)
  })

async function myHelper (t) {
  await t.expect(null).eql(null)     // assertion 1
  await t.expect(null).notEql(null)  // assertion 2
}

What is the Current behavior?

When myHelper fails at assertion 2, the stack trace looks like this:

   5 |    await myHelper(t)
   6 |  })
   7 |
   8 |async function myHelper (t) {
   9 |  await t.expect(null).eql(null)     // assertion 1
> 10 |  await t.expect(null).notEql(null)  // assertion 2
  11 |}

at <anonymous> (foo.spec.js:10:24)
at asyncGeneratorStep (foo.spec.js:1:114)
at _next (foo.spec.js:1:452)

This lists the correct line in the helper function where the assertion failed, but it then just lists the first line in the file instead of indicating where in the original test block the helper was called.

However, if I flip assertion 1 and 2, then the stack trace is more helpful:

   4 |  .page('https://google.com')('loads Google', async t => {
   5 |    await myHelper(t)
   6 |  })
   7 |
   8 |async function myHelper (t) {
>  9 |  await t.expect(null).notEql(null)  // assertion 2
  10 |  await t.expect(null).eql(null)     // assertion 1
  11 |}
  12 |

 at <anonymous> (foo.spec.js:9:24)
 at asyncGeneratorStep (foo.spec.js:1:114)
 at _next (foo.spec.js:1:452)
 at <anonymous> (foo.spec.js:1:617)
 at <anonymous> (foo.spec.js:1:358)
 at _myHelper (foo.spec.js:46:153)
 at myHelper (foo.spec.js:8:33)
 at <anonymous> (foo.spec.js:5:11)
 at asyncGeneratorStep (foo.spec.js:1:114)
 at _next (foo.spec.js:1:452)

The at <anonymous> (foo.spec.js:5:11) line correctly points to the line in the test block where myHelper is called.

What is the Expected behavior?

TestCafe should output the full, helpful stack trace from the second example output above even when it is not the first assertion that fails in the helper.

• testcafe version: 1.12.0
• node.js version: 14.15.5
• command-line arguments: testcafe chrome -L
• browser name and version: Chrome 89.0.4389.90
• platform and version: macOS 10.14.6

[SergeyShurygin]

Such a stack trace is caused by Babel preprocessing. We'll look into it.

[Asuza]

I have this same issue. Trying to divide common test code into utility classes results in a stack trace that doesn't tell you anything about where the error started, only where it happened in the utility class.

[DanKaplanSES]

Is it possible to raise the priority on this? At the very least, if it's possible to identify this problem and print a link back to this issue, that would've saved me hours of research. I thought it was a node flag I needed to set.

[DanKaplanSES]

I think I found a workaround to this issue.

1. Install https://www.npmjs.com/package/trace
2. Run testcafe like this in your package.json scripts: npx --node-options='--stack_trace_limit=100 -r trace' testcafe ...

Note: I got errors on different lines when I included clarify, so I recommend not using it.

---

Update: Actually this is a workaround that sometimes works, but I recommend only using it temporarily. While this did give me a stack trace I could use to troubleshoot, it caused other tests to fail until I removed npx --node-options='--stack_trace_limit=100 -r trace' In other words, this tool can introduce breaking changes, so this is a poor workaround.

[Aleksey28]

Hi @DanKaplanSES,

Thank you for the workaround.

[DanKaplanSES]

@Aleksey28 No problem. Are there any updates about this issue? It can make life very difficult when you attempt to reuse test code.

[Aleksey28]

We don't have any updates yet. We'll update this thread once we have any results.

[DanKaplanSES]

Bumping. Is this still an issue?

[github-actions]

No updates yet. Once we get any results, we will post them in this thread.

[aleks-pro]

Hello @DanKaplanSES ,

We understand the difficulties users may have with an async stack, but this is a low priority issue at the moment. We may reprioritize it in the future or if the number of requests for this feature grows

[Dvoreth]

We also urgently need this fixed.

[AlexKamaev]

@Dvoreth
This issue has low priority so we do not have plans to start working on this in near future.

[github-actions]

This issue has been automatically marked as stale because it has not had any activity for a long period. It will be closed and archived if no further activity occurs. However, we may return to this issue in the future. If it still affects you or you have any additional information regarding it, please leave a comment and we will keep it open.

[github-actions]

We're closing this issue after a prolonged period of inactivity. If it still affects you, please add a comment to this issue with up-to-date information. Thank you.

[DanKaplanSES]

This still affects me and I would like it to be reopened.

[Bayheck]

Hello,

Unfortunately, it is currently difficult to determine the cause of this issue.

In addition, it has a low priority, so we do not have plans to start working on this issue in the near future.

We can reopen it if things change and the issue becomes a higher priority.

[AriAyvazyanDedalus]

Can you please reevaluate the priotity?

It makes debugging issues on CI extremely hard and even locally very difficult.

When it comes to flaky tests, debugging becomes nearly impossible if the origin of a function call stack is not shown.

[Bayheck]

Hello! Thank you for reaching out about this issue.

At the moment, we’re unable to prioritize reevaluation of this matter.

We understand that it can complicate debugging, especially with CI and local testing. For now, you may find the workaround provided by DanKaplanSES helpful.

Currently, there are no plans to allocate resources to address this directly, but we appreciate your understanding.

[DanKaplanSES]

Currently, there are no plans to allocate resources to address this directly, but we appreciate your understanding.

Would you accept a PR?

[PavelMor25]

Hello @DanKaplanSES,

We always appreciate and encourage your contributions. You can review our contribution guidelines. Once you've submitted your pull request, our team will review it. If everything looks good, we'll merge it into the project.