Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Subsuite status FAILED based on verdict/outcome #61

Closed
wants to merge 1 commit into from
Closed

Subsuite status FAILED based on verdict/outcome #61

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 17 additions & 1 deletion projects/etos_suite_runner/src/etos_suite_runner/lib/suite.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,8 @@ class SubSuite(OpenTelemetryBase): # pylint:disable=too-many-instance-attribute
"""Handle test results and tracking of a single sub suite."""

released = False
failed = False
_failed = False
_failed_flag_set = False

def __init__(self, etos: ETOS, environment: dict, main_suite_id: str) -> None:
"""Initialize a sub suite."""
Expand All @@ -60,6 +61,21 @@ def __init__(self, etos: ETOS, environment: dict, main_suite_id: str) -> None:
self.test_suite_started = {}
self.test_suite_finished = {}

@property
def failed(self) -> bool:
"""Property indicating whether the sub-suite has had failures."""
if self._failed_flag_set:
return self._failed
verdict = self.outcome().get("verdict")
conclusion = self.outcome().get("conclusion")
return "FAILED" in (verdict, conclusion)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we should set failed if the verdict is failed. Verdict does not mean that the sub suite failed. It could be a genuine verdict. The verdict of sub suites should be evaluated in the results method in the Suite class. The result will be exactly the same, but it will allow us to do more in the future without editing up here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change won't be needed if we proceed with the current proposal of exit code handling/verdict matching in the Test Runner:
https://github.com/eiffel-community/etos-test-runner/pull/49/files


@failed.setter
def failed(self, value: bool) -> None:
"""Setter for the 'failed' property."""
self._failed = value
self._failed_flag_set = True

@property
def finished(self) -> bool:
"""Whether or not this sub suite has finished."""
Expand Down
Loading