How do I work with tests failures to show colleagues the failure is analysed

Modified on Fri, 11 Aug, 2023 at 9:23 AM

Test results statuses

We're using Industry standard statuses for the test results
  • Passed, 
  • Failed, 
  • Broken,
  • Skipped,
  • Unknown.
In progress status is for the test resukts, when we initiate the tests execution from Allure Testops, and we did not receive any results from a CI system yet.

For the statuses that differ from Passed, there is additional Attribute called Resolved (True|False), please check here: https://www.youtube.com/watch?v=OvBBBlmNhrI&list=PLINOSjry419nVTsmiQ4ACukWDioU4QK2R&index=7&t=2s)

Failed means unresolved

Failed test is always Unresolved by default
Unresolved test results are the ones you need to triage/analyse.
In the test cases list these can be found by applying a filter Resolution > Without resolution.

Resolving the failures

When the triage/analysis is complete, you have the following options to resolve the failed test results 
  1. Rerun 
  2. Mute 
  3. Assign a defect

Rerun

Rerun gives you either green or red result, if the result is green, test failure is resolved, if the result is red, then you need to go for either option 2 (Mute) or option 3 (defect)

Mute

Mute allows you ignoring the failure (kind of quarantine) in the consecutive launches until you removed Mute from the linked test case. Mute hides the failure from the overview. Mute marks the failure as Resolved. Mute prevents the test result status from the usage in the statistics.

Mute resolves a test result, hides from the overview, and marks the result as muted. Mute basically means that action is being taken somewhere, and you don't care at the moment.

Defect

Defect is the acknowledgement you have analysed the cause of a failure, you understood what to blame and you are actually taking the action. Defect is the link between Allure Testops and external system where the fix for the Failure is done (some manage defects without linking them to issues in an issue tracker, this is an option as well).
When you link a defect to a failure, the result becomes Resolved.
Failed Result is removed from the unresolved test results (no need to triage/analyse anything)
Failed Result linked to a defect is moved to Defects section.

Was this article helpful?

That’s Great!

Thank you for your feedback

Sorry! We couldn't be helpful

Thank you for your feedback

Let us know how can we improve this article!

Select at least one of the reasons
CAPTCHA verification is required.

Feedback sent

We appreciate your effort and will try to fix the article