Storage and database cleanup

Created by George I., Modified on Wed, 05 Apr 2023 at 09:19 AM by George I.

Historical Launches data

What do we store in the database:

metadata for each test result
metadata for each launch
scenario data
fixtures data
links to the artefacts on S3

Built-in Cleanup procedures deletes artefacts and related records in the database

Built-in cleanup (see details here) is intended to delete the following data by defined rules.

- fixtures artefacts from S3 and their file references from the database

- test results attachments from S3 and their file references from the database

- tests results scenarios from the database and all their references from the database

What historical data is not deleted

Launches data

- launches metadata from the database

- each test result metadata

If you want to delete historical launches data, then by means of Allure TestOps it is not possible right now. There is standalone tool allowing the deletion of launched by AQL: https://github.com/eroshenkoam/allure-testops-utils#delete-launches-in-allure-testops

The tool needs to be used with caution as the deletion of a launch is a heavy operation against the database which during the high load times could dramatically degrade the performance even to buzz state.

Deleted data

The data you store in the DB contains records with the historical data, data related to the paths to the files etc.

This all data is being deleted by Allure TestOps from time to time.

The PostgreSQL DBE works in a way which marks the data requested for deletion as kind of garbage, and this data is in fact not deleted. The only way to delete the data marked for deletion, and therefore reclaim the wasted space is vacuum procedure which needs to be executed on a regular basis as Allure TestOps deletes the data (again) on a regular basis.

Options to be used for VACUUM and frequency of the procedure need to be decided by your personnel responsible for the databases maintenance considering the peculiarities of VACUUM procedure described in the article mentioned above (exclusive locks etc.).

After the historical data or the artefacts are deleted, you need to run VACUUM to reclaim the wasted space on the database storage.