Skip to main content
Version: 1.8

End-to-end testing

So far we have tested the policy using a set of Go unit tests. This section shows how we can write end-to-end tests that run against the actual WebAssembly binary produced by TinyGo.

Prerequisites​

These tools need to be installed on your development machine:

  • docker or another container engine: used to build the WebAssembly policy. We will rely on the compiler shipped within the official TinyGo container image.
  • bats: used to write the tests and automate their execution.
  • kwctl: CLI tool provided by Kubewarden to run its policies outside of Kubernetes, among other actions. This is covered in depth inside of this section](/testing-policies/01-intro.md) of the documentation.

Writing tests​

We are going to use bats to write and automate our tests. Each test will be composed by the following steps:

  1. Run the policy using kwctl.
  2. Perform some assertions against the output produced by the kwctl.

All the end-to-end tests are located inside a file called e2e.bats. The scaffolded project already includes such a file. We will just change its contents to reflect how our policy behaves.

As a final note, for the end-to-end tests, we will use the same test fixtures files we previously used inside of the Go unit tests.

The first test ensures a request is approved when no settings are provided:

@test "accept when no settings are provided" {
run kwctl run -r test_data/pod.json policy.wasm

# this prints the output when one the checks below fails
echo "output = ${output}"

# request is accepted
[ $(expr "$output" : '.*"allowed":true.*') -ne 0 ]
}

We can execute the end-to-end tests by using this command:

make e2e-tests

This will produce the following output:

bats e2e.bats
✓ accept when no settings are provided

1 test, 0 failures

Let's write a test to ensure a request is approved when a user-defined constraint is respected:

@test "accept because label is satisfying a constraint" {
run kwctl run annotated-policy.wasm \
-r test_data/pod.json \
--settings-json '{"constrained_labels": {"cc-center": "\\d+"}}'

# this prints the output when one the checks below fails
echo "output = ${output}"

[ "$status" -eq 0 ]
[ $(expr "$output" : '.*allowed.*true') -ne 0 ]
}

Next, we can write a test to ensure a request is accepted when none of the labels is on the deny list:

@test "accept labels are not on deny list" {
run kwctl run \
-r test_data/pod.json \
--settings-json '{"denied_labels": ["foo", "bar"]}' \
policy.wasm

# this prints the output when one the checks below fails
echo "output = ${output}"

[ $(expr "$output" : '.*"allowed":true.*') -ne 0 ]
}

Let's improve the test coverage by adding a test that rejects a request because one of the labels is on the deny list:

@test "reject because label is on deny list" {
run kwctl run annotated-policy.wasm \
-r test_data/pod.json \
--settings-json '{"denied_labels": ["foo", "owner"]}'

# this prints the output when one the checks below fails
echo "output = ${output}"

[ "$status" -eq 0 ]
[ $(expr "$output" : '.*allowed.*false') -ne 0 ]
[ $(expr "$output" : ".*Label owner is on the deny list.*") -ne 0 ]
}

The following test ensures a request is rejected when one of its labels doesn't satisfy the constraint provided by the user.

@test "reject because label is not satisfying a constraint" {
run kwctl run annotated-policy.wasm \
-r test_data/pod.json \
--settings-json '{"constrained_labels": {"cc-center": "team-\\d+"}}'

# this prints the output when one the checks below fails
echo "output = ${output}"

[ "$status" -eq 0 ]
[ $(expr "$output" : '.*allowed.*false') -ne 0 ]
[ $(expr "$output" : ".*The value of cc-center doesn't pass user-defined constraint.*") -ne 0 ]
}

Now let's make sure the validation fails if one of the constrained labels is not found:

@test "reject because constrained label is missing" {
run kwctl run annotated-policy.wasm \
-r test_data/pod.json \
--settings-json '{"constrained_labels": {"organization": "\\d+"}}'

# this prints the output when one the checks below fails
echo "output = ${output}"

[ "$status" -eq 0 ]
[ $(expr "$output" : '.*allowed.*false') -ne 0 ]
[ $(expr "$output" : ".*Constrained label organization not found inside of Pod.*") -ne 0 ]
}

We want to ensure settings' validation is working properly. This can be done with the following tests:

@test "fail settings validation because of conflicting labels" {
run kwctl run \
-r test_data/pod.json \
--settings-json '{"denied_labels": ["foo", "cc-center"], "constrained_labels": {"cc-center": "^cc-\\d+$"}}' \
policy.wasm

# this prints the output when one the checks below fails
echo "output = ${output}"

# settings validation failed
[ $(expr "$output" : '.*"valid":false.*') -ne 0 ]
[ $(expr "$output" : ".*Provided settings are not valid: These labels cannot be constrained and denied at the same time: Set{cc-center}.*") -ne 0 ]
}

@test "fail settings validation because of invalid constraint" {
run kwctl run \
-r test_data/pod.json \
--settings-json '{"constrained_labels": {"cc-center": "^cc-[12$"}}' \
policy.wasm

# this prints the output when one the checks below fails
echo "output = ${output}"

[ $(expr "$output" : '.*"valid":false.*') -ne 0 ]
[ $(expr "$output" : ".*Provided settings are not valid: error parsing regexp.*") -ne 0 ]
}

Conclusion​

We have reached a pretty good level of coverage, let's run all the end-to-end tests:

$ make e2e-tests
bats e2e.bats
e2e.bats
✓ accept when no settings are provided
✓ accept because label is satisfying a constraint
✓ accept labels are not on deny list
✓ reject because label is on deny list
✓ reject because label is not satisfying a constraint
✓ reject because constrained label is missing
✓ fail settings validation because of conflicting labels
✓ fail settings validation because of invalid constraint

8 tests, 0 failures