Set up build and release pipeline storage policies（Alpha）

In the development process, pipeline execution records and test reports are continuously generated and stored in the platform component (etcd), which may impact its performance if left unprocessed for an extended period.

We provide the retention policy and an archive policy to help with this issue. The retention policy can remove execution records from etcd on demand, while the archive policy can store execution records and test reports in other storage to meet your data storage needs and alleviate storage pressure.

Clean up execution records

To reduce the impact on storage performance, uniformly configure retention policies for each build or release pipeline in a cluster to clean up execution records from etcd.

Procedure

1. In the left navigation bar, click Pipelines > Storage Settings.

2. Click CI/CD

3. In the Cleanup Policies tab, click .

4. Refer to the following instructions to configure the relevant parameters.

   • Maximum number of records to be kept per pipeline: To improve etcd storage performance, set the global maximum number of execution records to keep per pipeline.

   • Default number of records to be kept per pipeline: Although clearing execution records optimises storage resources, it is recommended not to over-clear execution records. Instead, it is recommended to set a certain number of pipeline execution records to be kept by default to allow for traceability and analysis when needed to track issues.

   Note: Once the default value is set, all newly created pipelines in DevOps by default clear execution records according to this rule. Pipeline users can also change the maximum number of execution records to be kept in DevOps, which cannot exceed the global maximum number. To set a clear execution records for an existing pipelines, refer to the following Batch setting cleanup policies for pipelines.

5. Click Update.

Batch setting cleanup policies for pipelines

In an environment where numerous pipelines require the application of cleanup policies, manually configuring each one can be a cumbersome task. It is more efficient to employ a script to batch configure these cleanup policies. Below, you will find script examples specifically tailored for builds and release pipelines.

Note: In newly activated environments with archiving policies, this operation might inadvertently cleanse records that have not yet been archived into the database. It is advisable to ensure that the new build and release pipeline execution records (BuildRun, DeliveryRun) contain the archive.katanomi.dev/resource finalizer before proceeding with this operation. You can refer to the following commands for specifics:

• BuildRun: kubectl get buildruns.builds.katanomi.dev {name} -n {namespace} -o jsonpath='{.metadata.finalizers}' | grep "archive.katanomi.dev/resource"
• DeliveryRun: kubectl get deliveryruns.deliveries.katanomi.dev {name} -n {namespace} -o jsonpath='{.metadata.finalizers}' | grep "archive.katanomi.dev/resource"

Build

The script needs to be executed in the cluster where the Build pipeline resides.

Establish a cleanup policy for builds across all namespaces within the current cluster, retaining only the most recent three execution records. An example is shown below:

#!/bin/bash

items=$(kubectl get ns --no-headers | awk '{print $1}')
for ns in ${items};
do
  builds=$(kubectl get builds.builds.katanomi.dev --no-headers -n ${ns} | awk '{print $1}')
  for build in ${builds};
  do
     kubectl patch --type='merge' builds.builds.katanomi.dev ${build} -p '{"spec":{"historyLimits": {"count": 3} } }' -n ${ns}
  done
done

Delivery

The script must be executed in the global cluster.

Establish a cleanup policy for all project pipelines, retaining only the most recent three execution records. Below is an example:

#!/bin/bash

items=$(kubectl get ns --no-headers | awk '{print $1}')
for ns in ${items};
do
  deliveries=$(kubectl get deliveries.deliveries.katanomi.dev --no-headers -n ${ns} | awk '{print $1}')
  for delivery in ${deliveries};
  do
     kubectl patch --type='merge' deliveries.deliveries.katanomi.dev ${delivery} -p '{"spec":{"historyLimits": {"count": 10} } }' -n ${ns}
  done
done

Archive execution records or test reports

To better track pipeline build and release history, you can also configure an archive policy for each build pipeline or release pipeline in your cluster to store pipeline execution records or test reports to a specified storage. If a retention policy has been configured, the archive policy ensures that you can still query the archived execution records and test reports even if the platform has purged them from etcd.

Note:

• For pipelines, after the platform has cleaned up the execution record from etcd, you can query the record normally through the archiving policy, but you cannot re-execute a phase in that record.

• The types of test reports supported for archiving are: Pytest API test reports and Robot Framework API test reports.

Configure archive storage

1. Prepare the available storage based on the resources to be archived and ensure that the storage is properly accessible by global clusters, and clusters running pipelines.

   • To archive execution records: Use relational database MySQL 8.0 or ProgreSQL 12. e.g. MySQL instances deployed via Data Services.

   • To archive test reports: Use object storage MinIO .

   Note: One of each storage is ennough. The StorageClass used by the storage instances can be the same as the default StorageClass for Continuous Integration, or can be specified separately.

2. At least one database (or bucket) needs to be created in the storage.

   It is supported to use different databases (or buckets) to store archived data separately. For example, create db-devops-archive-builds and devops-archive-deliveries databases to store build records and release records respectively.

3. Switch to the platform. In the left navigation bar, click Pipelines > Storage Settings.

4. In the Archive Storages tab, click Create.

5. Refer to the following instructions to configure the relevant parameters.

   Parameter	Description
   Name	The name of the storage instance.
   Storage Type	To archive execution records: Database To archive test reports: MinIO
   Tool	To archive execution records: MySQL, PostgreSQL To archive test reports: MinIO
   Version	(Required for Database) Database version.
   Access URL	Access URL of the storage, e.g. URL for access from ouside the cluster 192.168.100.100:30000 .
   Secret	Secret for accessing the database or bucket used for archive.
   Database name or Bucket name	The database or bucket on the storage.

Configure an archive policy

After configuration, the Archived logo is displayed on the archived execution records. The archived execution records remain unchanged, but the archived test report shows the test case execution details on the execution record.

Procedure

1. In the left navigation, click Pipelines > Storage Settings.

2. Click CI/CD.

3. In the Archive Policies tab, click .

4. Refer to the following instructions to configure the relevant parameters.

   CI Archive Policies/CD Archive Policies

   • The platform automatically selects the required storage (Database).

   • Click Add to set the retention rules for the pipeline execution records. Example: Keeps at least 3 builds on the main branch per pipeline indicates that when the main branch is built, at least 3 recent execution records are kept even if a retention policy is set.

   • With the archive policy enabled, it is recommended to periodically purge execution records from the archive storage in order to reduce storage space and minimize storage costs.

   Report Archive

   • The platform automatically selects the required storage (MinIO).

   • With the archive policy enabled, it is recommended to periodically purge the test reports from the archived storage in order to reduce storage space and minimize storage costs.

5. Click Update.