Skip to main content
Version: Next 🚧

Verifying Kubewarden

The Kubewarden stack provides different attestations and assurances:

  • Provenance attestations: Inform of the build process, build dependencies, and aids in replicating the builds. Implement SLSA Standard (level 3 in our case).
  • SBOMs attestations: Contain the software dependencies. Help downstream consumers ascertain vulnerabilities of Kubewarden and its dependencies.
  • Signed artifacts: Indicate if an artifact is authentic or not, providing supply chain security. This includes the deliverables, but also the provenance and SBOM attestations.

Kubewarden artifacts, provenance attestations and SBOMs are signed using Sigstore, with the keyless workflow. This means that the signing certificate contains the following info, where * matches any following characters:

  • issuer: https://token.actions.githubusercontent.com
  • subject: https://github.com/kubewarden/*
  • x509 certificate extension for GHA, "github_workflow_repository": kubewarden/*
important

The subject used in the --certificate-identity-regexp cosign CLI flag in this tutorial utilizes the https://github.com/kubewarden/* values to simplify the explanation. However, this allows artifacts from repositories with the same prefix to bypass validation. For example: github.com/kubewarden/policy-server1.

If you want a more secure check, you need to use a full URL:

https://github.com/kubewarden/policy-server/.github/workflows/container-image.yml@refs/tags/v1.18.0

Note that the URL includes the full repository path, the workflow file path, and the version tag. If you follow this best practice, you can use the cosign CLI flag --certificate-identity with the full URL.

Container images​

To verify the keyless-signed container images produced by the Kubewarden team, you can use the cosign CLI tool. For example, to verify the kubewarden/policy-server image, you can execute the following command:

cosign verify ghcr.io/kubewarden/policy-server:v1.18.0 \
  --certificate-identity-regexp 'https://github.com/kubewarden/*' \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com

Verification for ghcr.io/kubewarden/policy-server:v1.18.0 --
The following checks were performed on each of these signatures:
  - The cosign claims were validated
  - Existence of the claims in the transparency log was verified offline
  - The code-signing certificate was verified using trusted certificate authority certificates

<snipped json>

You can then verify that the certificate in the returned JSON contains the correct issuer, subject, and github_workflow_repository extensions.

The same applies to all other images produced by the Kubewarden team, such as kubewarden/kubewarden-controller and kubewarden/audit-scanner.

All container images also have SBOM and provenance files that can be used to ensure the secure supply chain of the images. You can find attestation files on the release page of the component or attached to the container image in the OCI registry.

We use Docker to build the images and their attestations. However, the cosign command does not yet support the verification of the attestations generated by Docker from the OCI registry. For this reason, you need to download the files from the release page or the registry and verify them locally. If you want to download the attestation files from the OCI registry, you can follow the Docker documentation and use tools like crane or docker itself to download the files from the registry.

When downloading the tarball with attestation files from the release page of the Kubewarden components, extract them, verify the signature for the checksum file, and then check the attestation files:

$ tar -xvf attestation-amd64.tar.gz

$ cosign verify-blob --bundle audit-scanner-attestation-amd64-checksum-cosign.bundle \
    --certificate-oidc-issuer=https://token.actions.githubusercontent.com \
    --certificate-identity="https://github.com/kubewarden/audit-scanner/.github/workflows/attestation.yml@refs/tags/v1.18.0" \
    audit-scanner-attestation-amd64-checksum.txt
Verified OK

$ sha256sum -c audit-scanner-attestation-amd64-checksum.txt
audit-scanner-attestation-amd64-provenance.json: OK
audit-scanner-attestation-amd64-sbom-451fac2e52226302ff449bfe053b3831fd93409b4dad24581b6121cc24daa2c2.json: OK

Now that the files integrity is verified, you can inspect the SBOM and Provenance files.

Helm charts​

You can find our Helm charts in our https:// traditional Helm repository under https://charts.kubewarden.io.

The same Helm charts are signed via Sigstore's keyless signing, and pushed to an OCI repository that holds both the Helm chart, its signature, and its provenance attestation as OCI artifacts.

Since Helm 3.8.0, Helm has support for OCI registries, but because of constraints in them, they can't be searched via helm. You can find the list of charts in GitHub Container Registry.

To verify a Helm chart, you need cosign installed. Then execute the following command, for example:

cosign verify ghrc.io/kubewarden/charts/kubewarden-defaults:1.5.4 \
  --certificate-identity-regexp 'https://github.com/kubewarden/*' \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com

Verification for ghcr.io/kubewarden/charts/kubewarden-defaults:1.5.4 --
The following checks were performed on each of these signatures:
  - The cosign claims were validated
  - Existence of the claims in the transparency log was verified offline
  - The code-signing certificate was verified using trusted certificate authority certificates

<snipped json>

You can then verify that the cert in the returned json contains the correct issuer, subject, and github_workflow_repository extensions.

The chart attestations are pushed to the OCI registry as an artifact layer. See the container images section on how to verify them.

Kubewarden charts ship imagelist.txt and (policylist.txt when relevant) inside of the chart. Hence, if you already verified the chart, you can use those lists to verify the consumed container images and policies.

Kubewarden also follows the usual practices with regards to Helm charts. Hence, one can also find all the images in the Helm charts with a plugin such as helm-image, or with the following script:

#!/usr/bin/env bash
helm pull --untar kubewarden/kubewarden-controller && \
helm pull --untar kubewarden/kubewarden-defaults && \
{ helm template ./kubewarden-controller; helm template ./kubewarden-defaults } \
    | yq '..|.image? | select(.)' \
    | sort -u | sed 's/"//g'

which gives us:

ghcr.io/kubewarden/kubewarden-controller:v0.5.5
ghcr.io/kubewarden/policy-server:v0.3.1
ghcr.io/kubewarden/kubectl:v1.21.4

Now, for each image in that list you can verify their Sigstore signatures following the instructions from the previous section.

kwctl​

kwctl binaries are signed using Sigstore's blog signing.

When you download a kwctl release each zip file contains two files that can be used for verification: kwctl.sig and kwctl.pem.

In order to verify kwctl you need cosign installed, and then execute the following command:

cosign verify-blob \
  --signature kwctl-linux-x86_64.sig \
  --cert kwctl-linux-x86_64.pem kwctl-linux-x86_64
  --certificate-identity-regexp 'https://github.com/kubewarden/*' \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com

Verified OK

You can then verify that the cert in the returned json contains the correct issuer, subject, and github_workflow_repository extensions.

The SBOMs are signed and published in the GitHub Release of the project.

The provenance attestation for kwctl is verified by using gh attestation verify. For example with gh attestation verify kwctl-linux-x86_64 --repo kubewarden/kwctl.

Policies​

Policies maintained by the Kubewarden team are also signed using the Sigstore project. Similar to usual container images, one can verify them using cosign:

cosign verify ghcr.io/kubewarden/policies/verify-image-signatures:v0.2.5 \
  --certificate-identity-regexp 'https://github.com/kubewarden/*' \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com

Verification for ghcr.io/kubewarden/policies/verify-image-signatures:v0.2.5 --
The following checks were performed on each of these signatures:
  - The cosign claims were validated
  - Existence of the claims in the transparency log was verified offline
  - The code-signing certificate was verified using trusted certificate authority certificates

  <snipped json>

You can then verify that the cert in the returned json contains the correct issuer, subject, and github_workflow_repository extensions.