Image Policy Webhooks on Kubernetes (image scanner admission controller)

Adding Trivy Scanner as custom Admission Controller

We will include an Image Policy Webhook on our kubeadm Kubernetes cluster in order to enhance its security, now allowing containers with more than 3 CRITICAL vulnerabilities from getting scheduled on our cluster.

To accomplish this, the first step involves deploying a Scanner. In this instance, I have utilized a custom trivy scanner that I developed in Go, which utilizes the Trivy scanner in its operation. You can review the project here: go-trivy-scanner

Changes required to kube-api

Add the option --admission-control-config-file=/etc/kubernetes/admission-control/image-policy-webhook-conf.yaml

Append the plugin ImagePolicyWebhook to the option --enable-admission-plugins

Add the volume

  volumes:
  - hostPath:
      path: /etc/kubernetes/admission-control
      type: DirectoryOrCreate
    name: etc-kubernetes-admission-control

And the volume-mounts to kube-api

    volumeMounts:
    - mountPath: /etc/kubernetes/admission-control
      name: etc-kubernetes-admission-control
      readOnly: true

Configuration files

Proceeding with our custom Image Policy Webhook.

file /etc/kubernetes/admission-control/image-policy-webhook-conf.yaml

apiVersion: apiserver.config.k8s.io/v1
kind: AdmissionConfiguration
plugins:
  - name: ImagePolicyWebhook
    path: /etc/kubernetes/admission-control/imagepolicyconfig.yaml

We define the Image Policy Config here:

file /etc/kubernetes/admission-control/imagepolicyconfig.yaml

imagePolicy:
  kubeConfigFile: /etc/kubernetes/admission-control/trivy-scanner.kubeconfig
  allowTTL: 50
  denyTTL: 50
  retryBackoff: 500
  defaultAllow: true

And we define the kubeconfig file, this is the minimal supported configuration to make it work:

file /etc/kubernetes/admission-control/trivy-scanner.kubeconfig

apiVersion: v1
kind: Config
clusters:
- cluster:
    server: https://trivy-scanner<my-domain>/scan
  name: okd
users:
- name: admin
  user: {}
preferences: {}
contexts:
- context:
    cluster: okd
    user: admin
  name: admin
current-context: admin

Reviewing our kube-api static pod

After that, on our master node, we will configure the static Pod kube-api, located on /etc/kubernetes/manifests/kube-apiserver.yaml mounting an admission-controller directory, where we will place our config files.

apiVersion: v1
kind: Pod
metadata:
  annotations:
    kubeadm.kubernetes.io/kube-apiserver.advertise-address.endpoint: 192.168.124.20:6443
  creationTimestamp: null
  labels:
    component: kube-apiserver
    tier: control-plane
  name: kube-apiserver
  namespace: kube-system
spec:
  containers:
  - command:
    - kube-apiserver
    - --advertise-address=192.168.124.20
    - --allow-privileged=true
    - --authorization-mode=Node,RBAC
    - --client-ca-file=/etc/kubernetes/pki/ca.crt
    - --enable-admission-plugins=NodeRestriction,ImagePolicyWebhook
    - --admission-control-config-file=/etc/kubernetes/admission-control/image-policy-webhook-conf.yaml
    [...]
    volumeMounts:
    [...]
    - mountPath: /etc/kubernetes/admission-control
      name: etc-kubernetes-admission-control
      readOnly: true
    [...]
  volumes:
  [...]
  - hostPath:
      path: /etc/kubernetes/admission-control
      type: DirectoryOrCreate
    name: etc-kubernetes-admission-control

This will restart our kube-api container

we can validate with

crictl ps -a
crictl logs <container>

Test the Image Policy Webhook

Once kube-api is back online, we can try to deploy a faulty pod with lot of vulnerabilities, this should fail:

file faulty-pod.yaml

apiVersion: v1
kind: Pod
metadata:
  name: imagepolicy-nginx-pod
spec:
  containers:
  - name: nginx
    image: nginx:1.14.2

kubectl create -f faulty-pod.yaml
Error from server (Forbidden): error when creating "faulty-pod.yaml": pods "imagepolicy-nginx-pod" is forbidden: image policy webhook backend denied one or more images: More than 3 CRITICAL vulnerabilities, rejected: [nginx:1.14.2]