Managing Kubewarden with Rancher Fleet

The Kubewarden Helm charts, like other Helm charts, can be managed via Rancher Fleet. Rancher Fleet uses Kubernetes CRDs to define a GitOps approach for managing Kubernetes clusters. It does this by defining Fleet Bundles.

Installing

The Kubewarden charts are normal charts, they have dependencies (such as cert-manager), and depend transitively on each other (kubewarden-crds ← kubewarden-controller ← kubewarden-defaults, see the Quickstart docs).

On Rancher Fleet, one can codify the chart dependencies using dependsOn in the fleet.yaml file.

At the time of writing and by how Rancher Fleet works, one may see transient errors until the charts are ready, such as:

ErrApplied(1) [Cluster fleet-local/local: dependent bundle(s) are not ready:
[kubewarden-example-helm-kubewarden-controller]]

These errors don't signify a problem, and once each chart has finished deploying, they will be gone.

Removing

caution

When blindly removing the GitRepo, all 3 Kubewarden charts get removed at once. This means the kubewarden-crds chart gets removed.

Kubewarden uses a pre-delete helm hook job in kubewarden-controller chart that deletes the default policy-server. This pre-delete hook is needed because we need to vacate the webhooks of the policies (this is true any Policy Engine) before deleting the PolicyServer. If not, the cluster will have webhooks for policies that don't exist anymore, rejecting everything and being in a failed state.

Removing the GitRepo and hence the kubewarden-crds chart at the same time as the kubewarden-controller chart will make the pre-delete hook job to fail, and the removal to be incomplete, leaving leftovers in the cluster.

Uninstalling CRDs automatically is normally not supported in any tooling, and Rancher Fleet is no exception.

If you want to perform a correct removal, make sure to remove first the Bundle for kubewarden-defaults from the cluster by commiting those changes to the repo holding the Fleet configuration and waiting for it being applied. Then kubewarden-controller in the same way, and last, kubewarden-crds.

Another option is to add 2 GitRepos, one for the CRDs only, and another for the rest of the Kubewarden charts. This way you can remove the Kubewarden charts first and the Kubewarden CRDs last.

Example

Have a look at github.com/kubewarden/fleet-example for an example of Fleet Bundle definitions.