Launch variables and pipeline context
Allure Testops collects the information on the environment where your tests have been executed, this means allurectl or a CI plug-in send the pipeline's (workflow's) context to Allure Testops server.
What is pipeline's context?
Pipeline's context is the set of CI pipeline system variables (e.g. Gitlab's job name will be CI_JOB_NAME) and user defined variables, which you use in your tests, like BROWSER_NAME – the name of browser to be used during the tests execution. All these (CI predefined variables and user-defined variables) are sent to Allure Testops alongside with the test results. Each test result then gets all the pipeline's context where it has been executed.
If in scope of single pipeline you executed the same test twice – one time with BROWSER_NAME=chrome and then BROWSER_NAME=firefox, then in the Allure Testops launch you'll have the same test result twice as the test has been executed with different environment and for Allure Testops these are two different test results.
How to use the pipeline's context?
To see what was the context of a pipeline which uploaded the text results, you need to set-up the processing of variables in Allure Testops project for it is not done automatically to prevent the database storage waste with redundant and unused data.
Global environment variables
If you need Allure Testops to start processing some variable you want to see in the launches meta-data, you need to have this variable registered as a global Environment variable.
Go to User menu > Administration > Environment
Allure Testops has 4 system variables which you cannot delete and rename and allows the end users creating their own variables.
As an example we'll create a new variable to see what was GitLab's Job name for the test results we're uploading.
So, click create and name the variable as CIJobName. And then Submit.
Now, Allure Testops can start processing this variable in your Launches.
Why do we need a global variable?
If you have several teams using different variables names for the same (i.e. to set a Browser in pipeline some use TESTS_BROWSER, some will use JENKINS_BROWSER and so on).
Project environment variables
In Gitlab pipelines, there is a predefined Gitlab variable containing the Job's name CI_PIPELINE_NAME. We want this to be displayed in our launches meta-data, this means we need to instruct Allure Testops to start processing this environment variable from Gitlab CI pipeline context we receive.
Go to project settings and then to Environment section and create the following mapping.
From the moment you added the mapping, Allure Testops will start processing of the specified Pipeline's context and will show the environment variable's value in the Launches data.
The values of CI_JOB_NAME will appear in a launch's meta-data and will be collected under the global environment variable we created.
In a launch each test result now gets additional attribute – CIJobName.
You can see it in the test result's Environment attributes and can filter the test results using this attribute.
Thing is, Allure Testops allows you using the collected Environment variables information to request the execution of a pipeline on CI side with desired context, i.e. with specific values of the environment variables.
In vast majority of the cases, predefined CI variables cannot be altered from external system and being set by the CI itself, but again, for the other vast majority of the cases, end users require setting of user-defined variables.
User-defined variables in pipeline
In a pipeline we usually set the variables which define some tests logic like the usage of a specific browser, specific application under test etc.
In the Gitlab example below we use TESTS_ENDPOINT to set the application under test, TESTS_BROWSER to instruct tests using specific browser, TEST_ENV to select the environment we want (prod, pre-prod, qa etc).
We can set these manually when triggering the pipelines from CI UI, and we also can to the same from Allure Testops side.
When pipeline first time uploads the test results, Allure Testops creates a link between your project and CI called Job. All the Jobs can be found in Jobs section of each project.
Job contains the information about pipeline, build server (see Integrations articles in the documentation) and parameters of the job, which we are going to configure to trigger a pipeline with desired parameters.
Configuring the Job - allow trigger the pipeline
When a job is created, it usually contains the following data (Go to the Jobs, select a job and use Job's menu item Configure).
You need to select configured Integration with CI server (in our example that would be Girlab) and allow using the Job to execute tests (trigger pipelines).
Now we can trigger a pipeline.
Branch is a bit special
The pipeline created this job can be triggered on different branches. The branch to be triggered is defined by the line you see above.
Thing is, we collect examples of branches in Branch global variable (right side) and use these to request CI to use a specific branch. This is done via API and left side of the setting is not important, we just use it to highlight this is Branch we need to use.
Now, back to the pipeline. Let's see what we have in our pipeline once again
TESTS_ENDPOINT: "https://pytest.never.never.never.let.you.down" TESTS_BROWSER: "firefox" TEST_ENV: "preprod"
Say, we want all three variables to be used when triggering a pipeline.
To do so, we need to have all three in the Job's configuration as follows. To do so, we need add 3 variables as Job parameters
- #1 defines the name of Environment variable in CI pipeline we want to set
- #2 defines the default value to be set (if we omit this one when triggering a job, then the default value will be used)
- #3 defines the source of the suggested vales, i.e. a global environment variable we defined in the very beginning of this article.
Next thing you do is submitting the changes.
From now on, you can request pipeline execution with specific parameters.
Pipeline execution with specific environment
Now, let's use all we have configured.
Say we want trigger our tests to be executed in QA environemnt (TEST_ENV=qa), with browser Opera (TEST_BROWSER=opera) and against the application located at https://qa.test-bed.xyz (TESTS_ENDPOINT=https://qa.test-bed.xyz).
There are several ways to trigger a pipeline:
- Execute a Job
- Execute tests from the list of test cases
- Execute a test plan.
All options have the same approach in terms of setting the tests environment for the execution.
Here is an example of Job execution.
1. Cilck on Run button for the chosen Job.
2. Scroll to the Environment section and click add to create the Environment variables set
3. Add the values for the environment variables you want to use (see the item #1)
4. Complete the request by pressing Submit button
On CI side the tests will be executed with the requested set of parameters
Was this article helpful?
Thank you for your feedback
Sorry! We couldn't be helpful
Thank you for your feedback
We appreciate your effort and will try to fix the article