Skip to content

Latest commit

 

History

History
75 lines (61 loc) · 4.25 KB

EiffelTestSuiteFinishedEvent.md

File metadata and controls

75 lines (61 loc) · 4.25 KB
Raw

EiffelTestSuiteFinishedEvent (TSF)

The EiffelTestSuiteFinishedEvent declares that a previously started test suite (declared by EiffelTestSuiteStartedEvent) has finished and reports the outcome.

Note that while similar, the data.outcome object is different from that of EiffelActivityFinishedEvent. The outcome of the test suite reports not only the conclusion of the test suite execution - whether the tests were successfully executed - but also passes a verdict on the item or items under test. To highlight this conceptual difference, both data.outcome.verdict and data.outcome.conclusion are included.

Data Members

data.outcome

Type: Object
Required: No
Description: The outcome of the test suite.

data.outcome.verdict

Type: String
Required: No
Legal values: PASSED, FAILED, INCONCLUSIVE
Description: A terse standardized verdict on the item or items under test. Unlike in EiffelTestCaseFinishedEvent, this property is optional. It offers a method of summarizing the verdict of the test suite as a whole, but may be skipped.
PASSED signifies that the item or items under test successfully passed the test suite.
FAILED signifies that the item or items under test failed to pass the test suite.
INCONCLUSIVE signifies that the verdict of the test suite was inconclusive. This SHOULD be the case if data.outcome.conclusion is not SUCCESSFUL, but may in combination with a SUCCESSFUL conclusion be used to represent unreliability or flakiness.

data.outcome.conclusion

Type: String
Required: No
Legal values: SUCCESSFUL, FAILED, ABORTED, TIMED_OUT, INCONCLUSIVE
Description: A terse standardized conclusion of the test suite, designed to be machine readable. Unlike in EiffelTestCaseFinishedEvent, this property is optional. It offers a method of summarizing the conclusion of the test suite as a whole, but may be skipped.
SUCCESSFUL signifies that the test suite was successfully concluded. Note that this does not imply that the item under test passed the tests.
FAILED signifies that the test suite could not be successfully executed. To exemplify, one or more tests failed to run due to required environments being unavailable.
ABORTED signifies that the test suite was aborted before it could be concluded.
TIMED_OUT signifies that the test suite did not conclude within the allowed time frame.
INCONCLUSIVE signifies that the outcome of the test suite could not be determined.

data.outcome.description

Type: String
Required: No
Description: A verbose description of the test suite outcome, designed to provide human readers with further information.

data.persistentLogs

Type: Object[]
Required: No
Description: An array of persistent log files generated during execution.

data.persistentLogs.name

Type: String
Required: Yes
Description: The name of the log file.

data.persistentLogs.uri

Type: String
Required: Yes
Description: The URI at which the log can be retrieved.

Version History

Version Introduced in Changes
1.0.0 edition-bordeaux Initial version.

Examples