Skip to content

Latest commit

 

History

History
94 lines (75 loc) · 4.81 KB

EiffelTestCaseFinishedEvent.md

File metadata and controls

94 lines (75 loc) · 4.81 KB
Raw

EiffelTestCaseFinishedEvent (TCF)

The EiffelTestCaseFinishedEvent declares that a previously started test case (declared by EiffelTestCaseStartedEvent) has finished and reports the outcome.

Note that while similar, the data.outcome object is different from that of EiffelActivityFinishedEvent. The outcome of the test case reports not only the conclusion of the test case execution - whether the test case was successfully executed - but also passes a verdict on the item under test. To highlight this conceptual difference, both data.outcome.verdict and data.outcome.conclusion are included.

Also note that unlike EiffelTestSuiteFinishedEvent, EiffelTestCaseFinishedEvent must report both data.outcome.verdict and data.outcome.conclusion.

Data Members

data.outcome

Type: Object
Required: Yes
Description: The outcome of the test case.

data.outcome.verdict

Type: String
Required: Yes
Legal values: PASSED, FAILED, INCONCLUSIVE
Description: A terse standardized verdict on the item or items under test.
PASSED signifies that the item or items under test successfully passed the test case.
FAILED signifies that the item or items under test failed to pass the test case.
INCONCLUSIVE signifies that the verdict of the test case was inconclusive. This SHOULD be the case if data.outcome.conclusion is not SUCCESSFUL, but may in combination with a SUCCESSFUL conclusion be used to represent unreliability or flakiness.

data.outcome.conclusion

Type: String
Required: Yes
Legal values: SUCCESSFUL, FAILED, ABORTED, TIMED_OUT, INCONCLUSIVE
Description: A terse standardized conclusion of the test case, designed to be machine readable.
SUCCESSFUL signifies that the test case was successfully concluded. Note that this does not imply that the item under test passed the tests.
FAILED signifies that the test case could not be successfully executed. To exemplify, one or more tests failed to run due to required environments being unavailable.
ABORTED signifies that the test case was aborted before it could be concluded.
TIMED_OUT signifies that the test case did not conclude within the allowed time frame.
INCONCLUSIVE signifies that the outcome of the test case could not be determined.

data.outcome.description

Type: String
Required: No
Description: A verbose description of the test case outcome, designed to provide human readers with further information.

data.outcome.metrics

Type: Object[]
Required: No
Description: A list of metrics collected during the test case execution. Note that while complete freedom is allowed in metrics names and value types, it is highly recommended to keep reported metrics concise and consistent. In other words, do not include excessive amounts of data (use data.persistentLogs for that), and avoid unnecessary variations in value names or types over time.

data.outcome.metrics.name

Type: String
Required: Yes
Description: The metrics name.

data.outcome.metrics.value

Type: Any
Required: Yes
Description: The metrics value.

data.persistentLogs

Type: Object[]
Required: No
Description: An array of persistent log files generated during execution.

data.persistentLogs.name

Type: String
Required: Yes
Description: The name of the log file.

data.persistentLogs.uri

Type: String
Required: Yes
Description: The URI at which the log can be retrieved.

Version History

Version Introduced in Changes
1.0.1 Current version data.outcome.metrics.value and data.outcome.metrics.name made mandatory.
1.0.0 edition-bordeaux Initial version.

Examples