Launch variables and pipeline context

Allure Testops collects the information on the environment where your tests have been executed, this means allurectl or a CI plug-in send the pipeline's (workflow's) context to Allure Testops server.

What is pipeline's context? 

Pipeline's context is the set of CI pipeline system variables (e.g. Gitlab's job name  will be CI_JOB_NAME) and user defined variables, which you use in your tests, like BROWSER_NAME – the name of browser to be used during the tests execution. All these (CI predefined variables and user-defined variables) are sent to Allure Testops alongside with the test results. Each test result then gets all the pipeline's context where it has been executed.

If in scope of single pipeline you executed the same test twice – one time with BROWSER_NAME=chrome and then BROWSER_NAME=firefox, then in the Allure Testops launch you'll have the same test result twice as the test has been executed with different environment and for Allure Testops these are two different test results.

How to use the pipeline's context?

To see what was the context of a pipeline which uploaded the text results, you need to set-up the processing of variables in Allure Testops project for it is not done automatically to prevent the database storage waste with redundant and unused data.

Global environment variables

If you need Allure Testops to start processing some variable you want to see in the launches meta-data, you need to have this variable registered as a global Environment variable.

Go to User menu > Administration > Environment

Allure Testops has 4 system variables which you cannot delete and rename and allows the end users creating their own variables.

As an example we'll create a new variable to see what was GitLab's Job name for the test results we're uploading.

So, click create and name the variable as CIJobName. And then Submit.

Now, Allure Testops can start processing this variable in your Launches.

Why do we need a global variable?

If you have several teams using different variables names for the same  (i.e. to set a Browser in pipeline some use TESTS_BROWSER, some will use JENKINS_BROWSER and so on).

Project environment variables

In Gitlab pipelines, there is a predefined Gitlab variable containing the Job's name CI_PIPELINE_NAME. We want this to be displayed in our launches meta-data, this means we need to instruct Allure Testops to start processing this environment variable from Gitlab CI pipeline context we receive.

Go to project settings and then to Environment section and create the following mapping.

Then Submit.

From the moment you added the mapping, Allure Testops will start processing of the specified Pipeline's context and will show the environment variable's value in the Launches data.

The values of CI_JOB_NAME will appear in a launch's meta-data and will be collected under the global environment variable we created.

In a launch each test result now gets additional attribute – CIJobName.

You can see it in the test result's Environment attributes and can filter the test results using this attribute.

Job parameters

Thing is, Allure Testops allows you using the collected Environment variables information to request the execution of a pipeline on CI side with desired context, i.e. with specific values of the environment variables.

In vast majority of the cases, predefined CI variables cannot be altered from external system and being set by the CI itself, but again, for the other vast majority of the cases, end users require setting of user-defined variables.

User-defined variables in pipeline

In a pipeline we usually set the variables which define some tests logic like the usage of a specific browser, specific application under test etc.

In the Gitlab example below we use TESTS_ENDPOINT to set the application under test, TESTS_BROWSER to instruct tests using specific browser, TEST_ENV to select the environment we want (prod, pre-prod, qa etc).

We can set these manually when triggering the pipelines from CI UI, and we also can to the same from Allure Testops side.

Job

When pipeline first time uploads the test results, Allure Testops creates a link between your project and CI called Job. All the Jobs can be found in Jobs section of each project.

Job contains the information about pipeline, build server (see Integrations articles in the documentation) and parameters of the job, which we are going to configure to trigger a pipeline with desired parameters.

Configuring the Job - allow trigger the pipeline

When a job is created, it usually contains the following data (Go to the Jobs, select a job and use Job's menu item Configure).

You need to select configured Integration with CI server (in our example that would be Girlab) and allow using the Job to execute tests (trigger pipelines).

Now we can trigger a pipeline.

Branch is a bit special

The pipeline created this job can be triggered on different branches. The branch to be triggered is defined by the line you see above. 

Thing is, we collect examples of branches in Branch global variable (right side) and use these to request CI to use a specific branch. This is done via API and left side of the setting is not important, we just use it to highlight this is Branch we need to use.

Job parameters

Now, back to the pipeline. Let's see what we have in our pipeline once again

TESTS_ENDPOINT: "https://pytest.never.never.never.let.you.down"
TESTS_BROWSER: "firefox"
TEST_ENV: "preprod"

4. Complete the request by pressing Submit button

On CI side the tests will be executed with the requested set of parameters