Vulnerability Scanning and Verification
In ACP (Alauda Container Platform), you can use Tekton Pipeline to build and scan the image for vulnerabilities.
Specifically, use the trivy
task to generate vulnerability scan results, then use cosign
to upload the attestation of the vulnerability scan results, and finally use kyverno
to validate the attestation of the vulnerability scan results.
TOC
Feature Overview
This method uses tools similar to trivy to scan the image for vulnerabilities and then use Kyverno to verify the vulnerability scan results:
- Use
trivy
Tekton Task to scan the image for vulnerabilities.
- Use
cosign
Tekton Task to upload the vulnerability scan results to the image.
- Configure Kyverno rules to verify the vulnerability scan results.
- Use the image to create a Pod to verify the vulnerability scan results.
Use Cases
The following scenarios require referring to the guidance in this document:
- Implementing vulnerability scanning and verification in Kubernetes clusters using Kyverno
- Enforcing security policies to only allow images with acceptable vulnerability levels to be deployed
- Setting up automated vulnerability scanning and verification in CI/CD pipelines
- Ensuring image security and vulnerability compliance in production environments
- Implementing supply chain security controls for container images by verifying their vulnerability status
Prerequisites
- A Kubernetes cluster with Tekton Pipelines, Tekton Chains and Kyverno installed
- A registry with image pushing enabled
kubectl
CLI installed and configured to access your cluster
cosign
CLI tool installed
jq
CLI tool installed
Process Overview
Step | Operation | Description |
---|
1 | Generate signing keys | Create a key pair for signing artifacts using cosign |
2 | Set up authentication | Configure registry credentials for image pushing |
3 | Configure Tekton Chains | Set up Chains to use OCI storage and configure signing, disable TaskRun SLSA Provenance |
4 | Create a sample pipeline | Create a pipeline definition with trivy scanner and cosign upload tasks |
5 | Run a sample pipeline | Create and run a PipelineRun with proper configuration |
6 | Wait for signing | Wait for the PipelineRun to be signed by Chains |
7 | Get image information | Extract image URI and digest from the PipelineRun |
8 | (Optional) Get vulnerability attestation | Get and verify the vulnerability scan attestation |
9 | Verify with Kyverno | Create and apply Kyverno policy to verify vulnerability scan results |
10 | (Optional) Require recent scan | Add condition to require vulnerability scan results within 168 hours |
11 | Clean up | Delete test resources and policies |
Step-by-Step Instructions
Steps 1-3: Basic Setup
These steps are identical to the Quick Start: Signed Provenance guide. Please follow the instructions in that guide for:
Step 4: Create a Sample Pipeline
This is a Pipeline resource, which is used to build the image and generate the cosign vulnerability attestation.
apiVersion: tekton.dev/v1
kind: Pipeline
metadata:
name: chains-demo-4
spec:
params:
- default: |-
echo "Generate a Dockerfile for building an image."
cat << 'EOF' > Dockerfile
FROM ubuntu:latest
ENV TIME=1
EOF
echo -e "\nDockerfile contents:"
echo "-------------------"
cat Dockerfile
echo "-------------------"
echo -e "\nDockerfile generated successfully!"
description: A script to generate a Dockerfile for building an image.
name: generate-dockerfile
type: string
- default: <registry>/test/chains/demo-4:latest
description: The target image address built
name: image
type: string
results:
- description: first image artifact output
name: first_image_ARTIFACT_OUTPUTS
type: object
value:
digest: $(tasks.build-image.results.IMAGE_DIGEST)
uri: $(tasks.build-image.results.IMAGE_URL)
tasks:
- name: generate-dockerfile
params:
- name: script
value: $(params.generate-dockerfile)
taskRef:
params:
- name: kind
value: task
- name: catalog
value: catalog
- name: name
value: run-script
- name: version
value: "0.1"
resolver: hub
timeout: 30m0s
workspaces:
- name: source
workspace: source
- name: build-image
params:
- name: IMAGES
value:
- $(params.image)
- name: TLS_VERIFY
value: "false"
runAfter:
- generate-dockerfile
taskRef:
params:
- name: kind
value: task
- name: catalog
value: catalog
- name: name
value: buildah
- name: version
value: "0.9"
resolver: hub
timeout: 30m0s
workspaces:
- name: source
workspace: source
- name: dockerconfig
workspace: dockerconfig
- name: trivy-scanner
params:
- name: COMMAND
value: |-
set -x
mkdir -p .git
# support for insecure registry
export TRIVY_INSECURE=true
echo "generate cyclonedx sbom"
trivy image --skip-db-update --skip-java-db-update --scanners vuln --format cyclonedx --output .git/sbom-cyclonedx.json $(tasks.build-image.results.IMAGE_URL)@$(tasks.build-image.results.IMAGE_DIGEST)
cat .git/sbom-cyclonedx.json
echo "trivy scan vulnerabilities based on cyclonedx sbom"
trivy sbom --skip-db-update --skip-java-db-update --format cosign-vuln --output .git/trivy-scan-result.json .git/sbom-cyclonedx.json
cat .git/trivy-scan-result.json
echo "trivy scan vulnerabilities based on cyclonedx sbom and output in table format"
trivy sbom --skip-db-update --skip-java-db-update --format table .git/sbom-cyclonedx.json
runAfter:
- build-image
taskRef:
params:
- name: kind
value: task
- name: catalog
value: catalog
- name: name
value: trivy-scanner
- name: version
value: "0.4"
resolver: hub
timeout: 30m0s
workspaces:
- name: source
workspace: source
- name: dockerconfig
workspace: dockerconfig
- name: cosign-uploads
params:
- name: COMMAND
value: |-
set -x
export COSIGN_ALLOW_INSECURE_REGISTRY=true
export COSIGN_TLOG_UPLOAD=false
export COSIGN_KEY=$(workspaces.signkey.path)/cosign.key
echo "Signing image vuln"
cosign attest --type vuln --predicate .git/trivy-scan-result.json $(tasks.build-image.results.IMAGE_URL)@$(tasks.build-image.results.IMAGE_DIGEST)
echo "Signing image sbom"
cosign attest --type cyclonedx --predicate .git/sbom-cyclonedx.json $(tasks.build-image.results.IMAGE_URL)@$(tasks.build-image.results.IMAGE_DIGEST)
runAfter:
- trivy-scanner
taskRef:
params:
- name: kind
value: task
- name: catalog
value: catalog
- name: name
value: cosign
- name: version
value: "0.1"
resolver: hub
timeout: 30m0s
workspaces:
- name: source
workspace: source
- name: dockerconfig
workspace: dockerconfig
- name: signkey
workspace: signkey
workspaces:
- name: source
description: The workspace for source code.
- name: dockerconfig
description: The workspace for Docker configuration.
- name: signkey
description: The workspace for private keys and passwords used for image signatures.
TIP
This tutorial demonstrates a simplified workflow by generating the Dockerfile
and git-clone
task output inline within the pipeline.
In production environments, you would typically:
- Use the
git-clone
task to fetch source code from your repository
- Build the image using the Dockerfile that exists in your source code
- This approach ensures proper version control and maintains the separation between code and pipeline configuration
Explanation of YAML fields
- The same as in Step 4: Create a Sample Pipeline, but adds the following content:
workspaces
:
signkey
: The workspace for private keys and passwords used for image signatures.
tasks
:
trivy-scanner
: The task to scan the image for vulnerabilities.
cosign-uploads
: The task to upload the attestation of the vulnerability scan results.
Save into a yaml file named chains-demo-4.yaml
and apply it with:
$ export NAMESPACE=<default>
# create the pipeline in the namespace
$ kubectl create -n $NAMESPACE -f chains-demo-4.yaml
pipeline.tekton.dev/chains-demo-4 created
Step 5: Run a Sample Pipeline
This is a PipelineRun resource, which is used to run the pipeline.
apiVersion: tekton.dev/v1
kind: PipelineRun
metadata:
generateName: chains-demo-4-
spec:
pipelineRef:
name: chains-demo-4
taskRunTemplate:
serviceAccountName: <default>
workspaces:
- name: dockerconfig
secret:
secretName: <registry-credentials>
- name: signkey
secret:
secretName: <signing-secrets>
- name: source
volumeClaimTemplate:
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
storageClassName: <nfs>
Explanation of YAML fields
- The same as in Step 5: Run a Sample Pipeline. Below only introduces the differences.
workspaces
signkey
: the secret name of the signing key.
secret.secretName
: The signing secret prepared in the previous step Get the signing secret. But you need to create a new secret with the same namespace as the pipeline run.
Save into a yaml file named chains-demo-4.pipelinerun.yaml
and apply it with:
$ export NAMESPACE=<default>
# create the pipeline run in the namespace
$ kubectl create -n $NAMESPACE -f chains-demo-4.pipelinerun.yaml
Wait for the PipelineRun to be completed.
$ kubectl get pipelinerun -n $NAMESPACE -w
chains-demo-4-<xxxxx> True Succeeded 2m 2m
Step 6: Wait for the PipelineRun to be signed
Wait for the PipelineRun has chains.tekton.dev/signed: "true"
annotation.
$ export NAMESPACE=<default>
$ export PIPELINERUN_NAME=<chains-demo-4-xxxxx>
$ kubectl get pipelinerun -n $NAMESPACE $PIPELINERUN_NAME -o yaml | grep "chains.tekton.dev/signed"
chains.tekton.dev/signed: "true"
Once the PipelineRun has chains.tekton.dev/signed: "true"
annotation, means the image is signed.
Step 7: Get the image from the PipelineRun
# Get the image URI
$ export IMAGE_URI=$(kubectl get pipelinerun -n $NAMESPACE $PIPELINERUN_NAME -o jsonpath='{.status.results[?(@.name=="first_image_ARTIFACT_OUTPUTS")].value.uri}')
# Get the image digest
$ export IMAGE_DIGEST=$(kubectl get pipelinerun -n $NAMESPACE $PIPELINERUN_NAME -o jsonpath='{.status.results[?(@.name=="first_image_ARTIFACT_OUTPUTS")].value.digest}')
# Combine the image URI and digest to form the full image reference
$ export IMAGE=$IMAGE_URI@$IMAGE_DIGEST
# Print the image reference
$ echo $IMAGE
<registry>/test/chains/demo-4:latest@sha256:93635f39cb31de5c6988cdf1f10435c41b3fb85570c930d51d41bbadc1a90046
This image will be used to verify the vulnerability scan results.
Step 8: (Optional) Get the cosign vuln attestation
TIP
If you interested about the cosign vuln attestation content, you can continue to read the following content.
More details about the cosign vuln attestation, please refer to cosign vuln attestation
Get the signing public key according to the Get the signing public key section.
# Disable tlog upload and enable private infrastructure
$ export COSIGN_TLOG_UPLOAD=false
$ export COSIGN_PRIVATE_INFRASTRUCTURE=true
$ export IMAGE=<<registry>/test/chains/demo-4:latest@sha256:5e7b466e266633464741b61b9746acd7d02c682d2e976b1674f924aa0dfa2047>
$ cosign verify-attestation --key cosign.pub --type vuln $IMAGE | jq -r '.payload | @base64d' | jq -s
The output will be similar to the following, which contains the vulnerability scan results.
cosign vuln attestation
{
"_type": "https://in-toto.io/Statement/v0.1",
"predicateType": "https://cosign.sigstore.dev/attestation/vuln/v1",
"predicate": {
"scanner": {
"uri": "pkg:github/aquasecurity/trivy@dev",
"version": "dev",
"result": {
"CreatedAt": "2025-06-07T07:05:30.098889688Z",
"Metadata": {
"OS": {
"Family": "ubuntu",
"Name": "24.04"
}
},
"Results": [
{
"Class": "os-pkgs",
"Packages": [
{
"Arch": "amd64",
"ID": "coreutils@9.4-3ubuntu6",
"Identifier": {
"BOMRef": "pkg:deb/ubuntu/coreutils@9.4-3ubuntu6?arch=amd64&distro=ubuntu-24.04",
"PURL": "pkg:deb/ubuntu/coreutils@9.4-3ubuntu6?arch=amd64&distro=ubuntu-24.04",
"UID": "82bb3c93286700bc"
},
"Licenses": [
"GPL-3.0-or-later",
"BSD-4-Clause-UC",
"GPL-3.0-only",
"ISC",
"FSFULLR",
"GFDL-1.3-no-invariants-only",
"GFDL-1.3-only"
],
"Name": "coreutils"
}
],
"Vulnerabilities": [
{
"CVSS": {
"nvd": {
"V2Score": 2.1,
"V2Vector": "AV:L/AC:L/Au:N/C:N/I:P/A:N",
"V3Score": 6.5,
"V3Vector": "CVSS:3.0/AV:L/AC:L/PR:L/UI:N/S:C/C:N/I:H/A:N"
},
"redhat": {
"V2Score": 6.2,
"V2Vector": "AV:L/AC:H/Au:N/C:C/I:C/A:C",
"V3Score": 8.6,
"V3Vector": "CVSS:3.0/AV:L/AC:L/PR:N/UI:R/S:C/C:H/I:H/A:H"
}
},
"InstalledVersion": "9.4-3ubuntu6",
"LastModifiedDate": "2025-04-20T01:37:25.86Z",
"PkgID": "coreutils@9.4-3ubuntu6",
"PkgName": "coreutils",
"PublishedDate": "2017-02-07T15:59:00.333Z",
"References": [
"http://seclists.org/oss-sec/2016/q1/452",
"http://www.openwall.com/lists/oss-security/2016/02/28/2",
"http://www.openwall.com/lists/oss-security/2016/02/28/3",
"https://access.redhat.com/security/cve/CVE-2016-2781",
"https://lists.apache.org/thread.html/rf9fa47ab66495c78bb4120b0754dd9531ca2ff0430f6685ac9b07772%40%3Cdev.mina.apache.org%3E",
"https://lore.kernel.org/patchwork/patch/793178/",
"https://mirrors.edge.kernel.org/pub/linux/utils/util-linux/v2.28/v2.28-ReleaseNotes",
"https://nvd.nist.gov/vuln/detail/CVE-2016-2781",
"https://www.cve.org/CVERecord?id=CVE-2016-2781"
],
"Severity": "LOW",
"SeveritySource": "ubuntu",
"Status": "affected",
"VendorSeverity": {
"azure": 2,
"cbl-mariner": 2,
"nvd": 2,
"redhat": 2,
"ubuntu": 1
},
"VulnerabilityID": "CVE-2016-2781"
}
]
}
],
"SchemaVersion": 2
}
},
"metadata": {
"scanStartedOn": "2025-06-07T07:05:30.104726629Z",
"scanFinishedOn": "2025-06-07T07:05:30.104726629Z"
}
}
}
Description of the fields
predicateType
: The type of the predicate.
predicate.scanner
:
uri
: The URI of the scanner.
version
: The version of the scanner.
result
: The result of the vulnerability scan.
CreatedAt
: The time when the vulnerability scan finished.
Metadata
:
OS
:
Family
: The family of the OS.
Name
: The name of the OS.
Results
: The results of the vulnerability scan.
Class
:
os-pkgs
: The OS packages.
lang-pkgs
: The language packages.
Packages
: The packages of the image.
Vulnerabilities
: The vulnerabilities of the image.
Severity
: The severity of the vulnerability.
PkgID
: The package id of the vulnerability.
PkgName
: The package name of the vulnerability.
CVSS
: The CVSS of the vulnerability.
nvd
: The NVD of the vulnerability.
redhat
: The Red Hat of the vulnerability.
Step 9: Verify the vulnerability scan results with Kyverno
Step 9.1: Create a Kyverno policy to reject images with high-risk vulnerabilities
TIP
This step requires cluster administrator privileges.
More details about Kyverno ClusterPolicy, please refer to Kyverno ClusterPolicy
The policy is as follows:
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: reject-high-risk-image
spec:
webhookConfiguration:
failurePolicy: Fail
timeoutSeconds: 30
background: false
rules:
- name: check-image
match:
any:
- resources:
kinds:
- Pod
namespaces:
- policy
verifyImages:
- imageReferences:
- "*"
# - "<registry>/test/*"
skipImageReferences:
- "ghcr.io/trusted/*"
failureAction: Enforce
verifyDigest: false
required: false
useCache: false
imageRegistryCredentials:
allowInsecureRegistry: true
secrets:
# The credential needs to exist in the namespace where kyverno is deployed
- registry-credentials
attestations:
- type: https://cosign.sigstore.dev/attestation/vuln/v1
attestors:
- entries:
- attestor:
keys:
publicKeys: |- # <- The public key of the signer
-----BEGIN PUBLIC KEY-----
MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEFZNGfYwn7+b4uSdEYLKjxWi3xtP3
UkR8hQvGrG25r0Ikoq0hI3/tr0m7ecvfM75TKh5jGAlLKSZUJpmCGaTToQ==
-----END PUBLIC KEY-----
ctlog:
ignoreSCT: true
rekor:
ignoreTlog: true
conditions:
- all:
- key: "{{ scanner.result.Results[].Vulnerabilities[].Severity }}"
operator: AllNotIn
# supported values: UNKNOWN, LOW, MEDIUM, HIGH, CRITICAL
value: ["HIGH", "CRITICAL"]
message: |
The image contains high-risk vulnerabilities, please fix them before proceeding.
Severity levels: {{ scanner.result.Results[].Vulnerabilities[].Severity }}
- key: "{{ scanner.result.Results[].Vulnerabilities[?CVSS.redhat.V3Score > `1.0`][] | length(@) }}"
operator: Equals
value: 0
message: |
The image contains high-risk vulnerabilities, please fix them before proceeding.
High-risk vulnerabilities (CVSS > 1.0): {{ scanner.result.Results[].Vulnerabilities[?CVSS.redhat.V3Score > `1.0`].CVSS.redhat.V3Score[] }}.
Severity levels: {{ scanner.result.Results[].Vulnerabilities[?CVSS.redhat.V3Score > `1.0`].Severity[] }}.
PkgIDs: {{ scanner.result.Results[].Vulnerabilities[?CVSS.redhat.V3Score > `1.0`].PkgID[] }}.
Explanation of YAML fields
- The policy is largely consistent with the one in Image Signature Verification
spec.rules[0].verifyImages[].attestations[0].conditions
type
: The cosign vuln attestation type is https://cosign.sigstore.dev/attestation/vuln/v1
attestors
: the same as above.
conditions
: The conditions to be verified.
all
: All conditions must be met.
key: "{{ scanner.result.Results[].Vulnerabilities[].Severity }}"
: The severity of the vulnerabilities must not be HIGH
or CRITICAL
.
key: "{{ scanner.result.Results[].Vulnerabilities[?CVSS.redhat.V3Score >
1.0][] | length(@) }}"
: The number of vulnerabilities with CVSS score greater than 1.0 must be 0.
Save the policy to a yaml file named kyverno.reject-high-risk-image.yaml
and apply it with:
$ kubectl apply -f kyverno.reject-high-risk-image.yaml
clusterpolicy.kyverno.io/reject-high-risk-image configured
Step 9.2: Verify the policy
In the policy
namespace where the policy is defined, create a Pod to verify the policy.
Use the built image to create a Pod.
$ export NAMESPACE=<policy>
$ export IMAGE=<<registry>/test/chains/demo-4:latest@sha256:0f123204c44969876ed12f40066ccccbfd68361f68c91eb313ac764d59428bef>
$ kubectl run -n $NAMESPACE vuln-image --image=${IMAGE} -- sleep 3600
If your image has high-risk vulnerabilities, the Pod will be blocked by the policy.
Receive the output like this:
Error from server: admission webhook "mutate.kyverno.svc-fail" denied the request:
resource Pod/policy/high-risk was blocked due to the following policies
reject-high-risk-image:
check-image: |
image attestations verification failed, verifiedCount: 0, requiredCount: 1, error: .attestations[0].attestors[0].entries[0].keys: attestation checks failed for <registry>/test/chains/demo-4:latest and predicate https://cosign.sigstore.dev/attestation/vuln/v1: The image contains high-risk vulnerabilities, please fix them before proceeding.
High-risk vulnerabilities (CVSS > 1.0): [8.6,2.7,6.2,5.9,7.5,4.7,7.4,4.7,7.4,4.7,7.4,4.7,7.4,5.9,3.6,3.6,7.3,4.4,6.5,5.4].
Severity levels: ["LOW","MEDIUM","LOW","LOW","MEDIUM","MEDIUM","MEDIUM","MEDIUM","MEDIUM","MEDIUM","MEDIUM","MEDIUM","MEDIUM","LOW","LOW","LOW","MEDIUM","MEDIUM","MEDIUM","MEDIUM"].
PkgIDs: ["coreutils@9.4-3ubuntu6","gpgv@2.4.4-2ubuntu17","gpgv@2.4.4-2ubuntu17","libgcrypt20@1.10.3-2build1","liblzma5@5.6.1+really5.4.5-1build0.1","libpam-modules@1.5.3-5ubuntu5.1","libpam-modules@1.5.3-5ubuntu5.1","libpam-modules-bin@1.5.3-5ubuntu5.1","libpam-modules-bin@1.5.3-5ubuntu5.1","libpam-runtime@1.5.3-5ubuntu5.1","libpam-runtime@1.5.3-5ubuntu5.1","libpam0g@1.5.3-5ubuntu5.1","libpam0g@1.5.3-5ubuntu5.1","libssl3t64@3.0.13-0ubuntu3.5","login@1:4.13+dfsg1-4ubuntu3.2","passwd@1:4.13+dfsg1-4ubuntu3.2","perl-base@5.38.2-3.2build2.1","golang.org/x/net@v0.23.0","golang.org/x/net@v0.23.0","stdlib@v1.22.12"].
Change the conditions in the ClusterPolicy
to allow images with high-risk vulnerabilities, but the CVSS score is less than 10.0.
conditions:
- all:
- key: "{{ scanner.result.Results[].Vulnerabilities[].Severity }}"
operator: AllNotIn
value: ["CRITICAL"]
message: |
The image contains high-risk vulnerabilities, please fix them before proceeding.
Severity levels: {{ scanner.result.Results[].Vulnerabilities[].Severity }}
- key: "{{ scanner.result.Results[].Vulnerabilities[?CVSS.redhat.V3Score > `10.0`][] | length(@) }}"
operator: Equals
value: 0
message: |
The image contains high-risk vulnerabilities, please fix them before proceeding.
High-risk vulnerabilities (CVSS > 10.0): {{ scanner.result.Results[].Vulnerabilities[?CVSS.redhat.V3Score > `10.0`].CVSS.redhat.V3Score[] }}.
Severity levels: {{ scanner.result.Results[].Vulnerabilities[?CVSS.redhat.V3Score > `10.0`].Severity[] }}.
PkgIDs: {{ scanner.result.Results[].Vulnerabilities[?CVSS.redhat.V3Score > `10.0`].PkgID[] }}.
Then create a Pod again to verify the policy.
$ kubectl run -n $NAMESPACE vuln-image --image=${IMAGE} -- sleep 3600
The Pod will be created successfully.
Step 10: (Optional) Require the vulnerability scan results to be within 168 hours
TIP
If you interested to add more conditions to the policy, you can continue to read the following content.
Since the Cosign Vulnerability Scan Record Attestation includes a scanFinishedOn
field,
and the trivy
meets the specifications, we can use this field to determine if the vulnerability scan results are within 168 hours.
We only need to add a condition to the ClusterPolicy
to check if the scanFinishedOn
field is within 168 hours.
conditions:
- all:
- key: "{{ time_since('','{{metadata.scanFinishedOn}}','') }}"
operator: LessThanOrEquals
value: "168h"
message: "The vulnerability scan results must be within 168 hours, not {{ metadata.scanFinishedOn }}"
This is not demonstrated here, interested readers can try it themselves.
Step 11: Clean up the resources
Delete the Pods created in the previous steps.
$ export NAMESPACE=<policy>
$ kubectl delete pod -n $NAMESPACE vuln-image
Delete the policy.
$ kubectl delete clusterpolicy reject-high-risk-image
Expected Results
After completing this guide:
- You have a working setup with Tekton Chains for vulnerability scanning and Kyverno for vulnerability verification
- Your container images automatically include vulnerability scan results in their attestations
- Only images with acceptable vulnerability levels can be deployed in the specified namespace
- Images with high-risk vulnerabilities are automatically blocked by Kyverno policies
- You have implemented a basic supply chain security control by verifying the vulnerability status of your container images
This guide provides a foundation for implementing supply chain security in your CI/CD pipelines. In a production environment, you should:
- Configure proper namespace isolation and access controls
- Implement secure key management for signing keys
- Set up monitoring and alerting for policy violations
- Regularly rotate signing keys and update security policies
References