Image Policy Webhooks on Kubernetes (image scanner admission controller)
Adding Trivy Scanner as custom Admission Controller
We will include an Image Policy Webhook on our kubeadm Kubernetes cluster in order to enhance its security, not allowing containers with more than 3 CRITICAL vulnerabilities from getting scheduled on our cluster.
To accomplish this, the first step involves deploying a Scanner. In this instance, I have utilized a custom trivy scanner that I developed in Go, which utilizes the Trivy scanner in its operation. You can review the project here: go-trivy-scanner
Changes required to kube-api
Add the option --admission-control-config-file=/etc/kubernetes/admission-control/image-policy-webhook-conf.yaml
Append the plugin ImagePolicyWebhook
to the option --enable-admission-plugins
Add the volume
volumes:
- hostPath:
path: /etc/kubernetes/admission-control
type: DirectoryOrCreate
name: etc-kubernetes-admission-control
And the volume-mounts to kube-api
volumeMounts:
- mountPath: /etc/kubernetes/admission-control
name: etc-kubernetes-admission-control
readOnly: true
Configuration files
Proceeding with our custom Image Policy Webhook.
file /etc/kubernetes/admission-control/image-policy-webhook-conf.yaml
apiVersion: apiserver.config.k8s.io/v1
kind: AdmissionConfiguration
plugins:
- name: ImagePolicyWebhook
path: /etc/kubernetes/admission-control/imagepolicyconfig.yaml
We define the Image Policy Config here:
file /etc/kubernetes/admission-control/imagepolicyconfig.yaml
imagePolicy:
kubeConfigFile: /etc/kubernetes/admission-control/trivy-scanner.kubeconfig
allowTTL: 50
denyTTL: 50
retryBackoff: 500
defaultAllow: true
And we define the kubeconfig file, this is the minimal supported configuration to make it work:
file /etc/kubernetes/admission-control/trivy-scanner.kubeconfig
apiVersion: v1
kind: Config
clusters:
- cluster:
server: https://trivy-scanner<my-domain>/scan
name: okd
users:
- name: admin
user: {}
preferences: {}
contexts:
- context:
cluster: okd
user: admin
name: admin
current-context: admin
Reviewing our kube-api static pod
After that, on our master node, we will configure the static Pod kube-api, located on /etc/kubernetes/manifests/kube-apiserver.yaml
mounting an admission-controller directory, where we will place our config files.
apiVersion: v1
kind: Pod
metadata:
annotations:
kubeadm.kubernetes.io/kube-apiserver.advertise-address.endpoint: 192.168.124.20:6443
creationTimestamp: null
labels:
component: kube-apiserver
tier: control-plane
name: kube-apiserver
namespace: kube-system
spec:
containers:
- command:
- kube-apiserver
- --advertise-address=192.168.124.20
- --allow-privileged=true
- --authorization-mode=Node,RBAC
- --client-ca-file=/etc/kubernetes/pki/ca.crt
- --enable-admission-plugins=NodeRestriction,ImagePolicyWebhook
- --admission-control-config-file=/etc/kubernetes/admission-control/image-policy-webhook-conf.yaml
[...]
volumeMounts:
[...]
- mountPath: /etc/kubernetes/admission-control
name: etc-kubernetes-admission-control
readOnly: true
[...]
volumes:
[...]
- hostPath:
path: /etc/kubernetes/admission-control
type: DirectoryOrCreate
name: etc-kubernetes-admission-control
This will restart our kube-api container
we can validate with
crictl ps -a
crictl logs <container>
Test the Image Policy Webhook
Once kube-api is back online, we can try to deploy a faulty pod with lot of vulnerabilities, this should fail:
file faulty-pod.yaml
apiVersion: v1
kind: Pod
metadata:
name: imagepolicy-nginx-pod
spec:
containers:
- name: nginx
image: nginx:1.14.2
kubectl create -f faulty-pod.yaml
Error from server (Forbidden): error when creating "faulty-pod.yaml": pods "imagepolicy-nginx-pod" is forbidden: image policy webhook backend denied one or more images: More than 3 CRITICAL vulnerabilities, rejected: [nginx:1.14.2]