No binding to AllureID in case of changes in project structure and parameter set.

If you do not have a hard binding to AllureID (see more details in this article), but there have been changes in the method signature (package, class, method name) or parameter set such that the full test name has changed (e.g. io. testops.IssuesRestTest.useCaseParameterisedTest, and became io.testops.IssuesRestTests.useCaseParameterisedTest), then this result cannot be attributed to the previous TC, because the signature does not match, in this case a new TC will be created.

If the full signature of the method is changed, the system considers such a result as a result of a new test, as the result binding inside TestOps is either by the full signature with a set of parameters or by the AllureID attribute explicitly specified in the code (included in the test result).

When changing the set of parameters: (i) number of parameters, (ii) names of existing parameters, checksums described here are changed and binding to the existing AllureID is broken.

How to avoid it?

When planning the following code changes:

• package name (or some analogue for package for your programming language)
• class name (or some analogue for a class for your programming language or test framework)
• method name (function, etc.)
• increasing or decreasing the number of parameters (was 0 became 1, was 1 became 0, etc.)
• change of parameter names

before making these changes, it is required to enter information about the AllureID of your test into the test code. The AllureID in the test results has a higher priority than the full name with the parameter set and the result of the changed test will be associated with the correct TC.

A test is failing at the setUp block

If a test crashes at the setUp stage of a block (e.g. BeforeAll, BeforeEach, before hooks, or any analogues of parts of code that perform preparatory actions before the test body is executed) and it has parameters or/and an explicitly defined AllureID, the test is usually then is marked as BROKEN, and the logic of the test itself is not executed, as a result the information about parameters and AllureID does not get into the result of such a test and the process of comparing the test result to the existing TC is broken and a new TC with a new AllureID and without any parameters is created.

How to avoid it?

There are several solutions, both organisational and technical, to avoid creating duplicate tests when tests on setUp blocks crash:

Organisational measures of their own

Dropped tests in BROKEN state need to be restarted. If the tests generate PASSED results, they will be new in the created launch, BROKEN results can then simply be removed (hidden) from the launch.

Technical measures at the integration level

Request to add functionality to the corresponding repository of the Allure Framework project to handle BROKEN tests.

Technical measures

Self-implementation of the project functionality with tests, so that ALLURE_ID=-1 (AS_ID=-1) would be added to the result when a test is dropped on a setUp block.  In this case, the result will not generate a new TC, but it will be present in the results and information about the crash will be available to the team.