Context aware policies
The policy-server
has the capability to expose cluster information to
policies,
so that they can take decisions based on other existing resources,
and not only based on the details provided by the admission request.
The retrieval of Kubernetes resources is performed by the Policy Server hosting the policy. Access to Kubernetes is regulated by RBAC rules applied to the Service Account used by the Policy Server.
The default
Policy Server deployed by Kubewarden helm charts has access to the following Kubernetes resources:
- Namespaces
- Services
- Ingresses
The policy server performs caching of the results obtained from the Kubernetes API server to reduce the amount of load on this core part of Kubernetes. That means some information might be stale or missing.
Support matrix​
Policy type | Support | Notes |
---|---|---|
Traditional programming languages | ✅ | - |
Rego | ✅ | Since Kubewarden 1.9 release |
WASI | ✅ | Since Kubewarden 1.10.0 release, only for Go SDK |
Constraints​
Kubewarden's priority is to reduce the number of queries done against the Kubernetes API server. Because of that two constraints have to be considered:
- Memory usage: data fetched from Kubernetes is cached in memory by the Policy Server process. The more data is fetched, the more memory is going to be consumed by the Policy Server Pods.
- Consistency: the cache kept by Policy Server could contain stale data. New resources might be missing, deleted resources might still be available and changed ones could have old data. This could affect policy evaluation.
Listing multiple resources​
Kubewarden policies can fetch multiple resources at the same time. For example, they can make a query like
"get all the Pods defined inside of the default
namespace that have the label color
set to green
".
When such a query is done, the Policy Server fetches all the resources matching the user criteria. Resources are fetched in batches to reduce the
load on the Kubernetes API server.
Before storing the resources in memory, the managedFields
attribute of each resource is dropped to reduce the amount of memory consumed.
This attribute is not useful for policies and takes a significant amount of memory.
The Policy Server then creates a Kubernetes watch to keep the cached list of objects updated. The Policy Server doesn't control the speed at which the Kubernetes API Server sends notifications about resource changes. This depends on different external factors, like the number of watches created against the Kubernetes API server and its load.
Finally, the current code suffers from the following limitation. Given these two queries:
kubectl get pods -n default
kubectl get pods -n default -l color=green
Policy Server will create two watches and will duplicate all the Pods of the second query. We will work to remove this limitation in future releases of Kubewarden.
Fetching a specific resource​
Kubewarden policies can get a specific resource defined inside of the cluster. For example, they can make a query like
"get the Pod named psql-0
defined inside of the db
namespace".
By default this query fetches the object and stores it inside of an in-memory cache. The cache entry is wiped after 5 seconds. During this time window, the cached data is served to policies.
The policy author can also decide to make a direct query, one that skips the cache entirely. In this way, fresh data is always served. This however can cause more load on the Kubernetes API server (depending on how frequently the policy is triggered) and introduces more latency when evaluating an admission request.
The direct/cached query behavior can be configured on a per-query level by the policy author using the Kubewarden SDKs.
ClusterAdmissionPolicies​
ClusterAdmissionPolicies have the field
spec.contextAwareResources.
This field provides a list a GroupVersionKind
resources that the policy needs to access.
This allows policy writers to ship the "permissions" that the policy needs together with the policy.
Moreover, this allows policy operators to review the "permissions" needed by the policy at deployment time.
Testing context aware policies locally​
As well as running policies in a cluster for end-to-end tests,
you can use the kwctl
CLI utility to run policies and mock requests against the cluster.
For this, kwctl run
can first record all the interactions with the cluster into a file:
kwctl run \
--allow-context-aware \
-r request.json \
--record-host-capabilities-interactions replay-session.yml \
annotated-policy.wasm
which creates the following replay-session.yml
file:
# replay-session.yml
---
- type: Exchange
request: |
!KubernetesGetResource
api_version: /v1
kind: Pod
name: p-testing
namespace: local
disable_cache: true
response:
type: Success
payload: '{"apiVersion":"","kind":"Pod", <snipped> }'
With the replay session, you can now simulate the cluster interactions without the need of a cluster, which is ideal for CI and end-to-end tests:
kwctl run \
--allow-context-aware \
-r request.json \
--replay-host-capabilities-interactions replay-session.yml \
annotated-policy.wasm
Language SDKs​
Language SDK's that support cluster context at the moment expose functions that allow policies to retrieve the current state of the cluster.
If you want more information about the waPC function used by the SDKs, check the Kubernetes capabilities reference documentation.
Rust​
See the functions exposing this functionality at the Rust SDK reference docs.
Go​
See the functions exposing this functionality at the Go SDK reference docs.
Rego policies​
Gatekeeper​
The context aware information is exposed under the data.inventory
key, like Gatekeeper does.
The inventory is populated with the resources the policy has been granted access to via the spec.contextAwareResources
field.
Open Policy Agent​
The context aware information is exposed under the data.kubernetes
key,
like
kube-mgmt
does by default.
The inventory is populated with the resources the policy has been granted access to via the spec.contextAwareResources
field.