Add the ability to mark a test as failed, and let it finish its execution with a specific non-zero exit code

[oleiade]

Feature Description

Background

We have reached a consensus on a new approach for an assertions framework to support functional testing in k6. This framework requires a mechanism that can mark a test as failed, while allowing the execution to continue to its end. The test should exit with a specific non-zero exit code to indicate a failure status.

Currently, k6 lacks a way to programmatically signal a test failure that persists to the end of execution and results in a non-zero exit code. This limitation impacts functional testing, as it is critical for users to flag tests as failed without abruptly stopping execution. Additionally, functional tests may require post-test analysis even if a failure occurs, making a full run-through essential.

Objective

Provide a mechanism within k6 to mark a test as failed during execution, leading to an eventual non-zero exit code. The test should continue executing until the end unless other stopping conditions occur.

This feature would allow features/code/users to:
• Indicate a failure that will be recognized at the end of execution.
• Exit with a specific non-zero exit code to clearly signal a test failure.
• Track test results programmatically, making the framework more functional test-friendly.

Desired End State

As a user, I should be able to mark a test as failed, see my test continue executing until the end, and receive a clear indication of the failure state through a specific non-zero exit code upon process termination.

For example:
• I am running a functional test in k6, and at a certain point, an assertion fails.
• I want to mark the test as failed while allowing it to finish the remaining steps.
• When k6 completes, it exits with a non-zero code (indicating failure) to make it easier for automated systems or users to detect the failure state, even if the test finished all steps.

This feature should be flexible in its implementation—whether added to an existing API or introduced as a new API function.

Questions to answer ❗️

• Does the exit code need to be customizable by the user, or is it good enough to "just" define one and roll with it?
• What exit code should be used for the default/predefined behavior (regardless of the question above)?

Suggested Solution (optional)

Proposed Outline

1. Mark Test as Failed: Introduce a primitive to k6 that allows users to set the test as failed from their script without stopping execution.
2. Exit Code on Completion: Ensure that when a test is marked as failed, the k6 process ends with a (designated?) non-zero exit code.

Solution(s)

TODO

Already existing or connected issues / PRs (optional)

No response