How I added initContainers and env vars to every deployment without Kyverno

TL;DR

We needed to modify deployments in the background and decided against Kyverno (too big). Instead a leightweight mutating Webhook called "kTailor" was built and published under Apache 2.0. Its main purpose is to insert env vars, initContainers and sidecars.

The Starting Point

Moving containers through time in a customer project, we're adding libfaketime to each deployment via an initContainer - banks sometimes need to do that to check if the software works fine on new years day. This method can be turned on and off easily by a switch in a central helm chart which is used for all internal apps. So far, so good.

Then came the first external app, and we didn't want to mess with its helm chart every time we wanted it to travel through time. So what to do if you've need to go back to 1985 but don't have 1.21 GW?

The Motivation

Well, there's Kyverno. Besides enforcing rules and policies it's capable of modifying deployments in the background without having to touch the helm chart - exactly what we needed. But I found that Kyverno is also a heavy weight and a bit oversized for our needs. Besides, there's all that paperwork to get allowance to run another external app deep inside your clusters ...

So I decided to write a small, generic Mutating Webhook that fits to our needs.

What kTailor does

kTailor is written in Go and thus practically Kubernetes native. Its goals on mutating a deployment (and no other k8s ressources) are the following:

• Modify the main container and
  
  
  
  
  • add, modify or remove env vars
  
  
  • add or remove volumeMounts
  
  
  
  


• Add initContainers


• Add sidecar containers


• Add volumes

That's more than enough in many cases. It enables us to inject the libfaketime library in the background and thus move the containers "through time". The technique is the same as used by many products like Dynatrace, Istio and others, every case when one needs to add a sidecar or initcontainer in the background.

With kTailor, now you can do it on your own. The software is Open Source and free to use. It's syntax is that of a simple k8s deployment and thus easy to handle - no need to learn to write Kyverno policies.

How it works

All rules for modification are stored in a simple configMap of which you can have as many as you need. You can store these templates aside your app in its namespace ("local") or in kTailors namespace ("central"). Important: The configMap needs a special to motivate kTailor to load it in its cache, like:



ktailor.dev/template: "true"

The trigger point is a label, too. Add a label like the following to a deployment and kTailor will add/modify the objects that are mentioned in your template:



ktailor.dev/fit: "central.my-template"

Online ressources

You'll find the repo at github and the documentation at ktailor.dev. And there's an online demo at killercoda.com that you can try for free: The scenario spawns a small one-node kubernetes cluster and installs the webhook along with two small demo applications.

Have fun

Maybe it's useful for some other k8s users, e.g. to

• insert a http proxy sidecar to see where your app sends requests to (without you knowing?), or


• insert a proxysql sidecar to track and measure the apps database calls, maybe to find and optimate the slower ones.

Have fun with kTailor and I'm looking forward to your feedback.