Test results statuses
We're using Industry standard statuses for the test results
- Passed,
- Failed,
- Broken,
- Skipped,
- Unknown.
In progress status is for the test resukts, when we initiate the tests execution from Allure Testops, and we did not receive any results from a CI system yet.
For the statuses that differ from Passed, there is additional Attribute called Resolved (True|False), please check here: https://www.youtube.com/watch?v=OvBBBlmNhrI&list=PLINOSjry419nVTsmiQ4ACukWDioU4QK2R&index=7&t=2s)
Failed means unresolved
Failed test is always Unresolved by default
Unresolved test results are the ones you need to triage/analyse.
In the test cases list these can be found by applying a filter Resolution > Without resolution.
Resolving the failures
When the triage/analysis is complete, you have the following options to resolve the failed test results
- Rerun
- Mute
- Assign a defect
Rerun
Rerun gives you either green or red result, if the result is green, test failure is resolved, if the result is red, then you need to go for either option 2 (Mute) or option 3 (defect)
Mute
Mute allows you ignoring the failure (kind of quarantine) in the consecutive launches until you removed Mute from the linked test case. Mute hides the failure from the overview. Mute marks the failure as Resolved. Mute prevents the test result status from the usage in the statistics.
Mute resolves a test result, hides from the overview, and marks the result as muted. Mute basically means that action is being taken somewhere, and you don't care at the moment.
Defect
Defect is the acknowledgement you have analysed the cause of a failure, you understood what to blame and you are actually taking the action. Defect is the link between Allure Testops and external system where the fix for the Failure is done (some manage defects without linking them to issues in an issue tracker, this is an option as well).
When you link a defect to a failure, the result becomes Resolved.
Failed Result is removed from the unresolved test results (no need to triage/analyse anything)
Failed Result linked to a defect is moved to Defects section.
Was this article helpful?
That’s Great!
Thank you for your feedback
Sorry! We couldn't be helpful
Thank you for your feedback
Feedback sent
We appreciate your effort and will try to fix the article