Understanding Kubernetes' new sidecar container feature

Last week, the Kubernetes project merged a new alpha feature enabling users to define “sidecar containers” in specifications. The new feature is intended to help define behavior for helper containers in multi-container pods that might assist in configuration, networking, log and metrics collection, and so on.

In theory, Kubernetes expects you to run one container per pod. In practice, many use cases call for multi-container pods—when you’re using certain service meshes, for example, you might have sidecars in almost all of your pods. 

Sometimes, helper containers are only used for initialization: configuring and managing secrets for the primary container, for example. Kubernetes has given users a way to define an initContainer for some time. This new feature ultimately provides more granularity to initContainer to reflect the particular requirements of sidecars, simplifying a common usage pattern and opening up some interesting design space for the future.

In this new feature gate, sidecar containers are defined to…

• Start up before other containers in a pod, since they probably need to initialize first. This is important for things like service meshes, where you want the sidecar to be ready to make networking connections for main container processes, as well as in logging, where you want the collector sidecar to be able to grab startup logs from the main container.

• Keep running for the full lifespan of the pod, since they’re likely needed on an ongoing basis. In the cases of networking and metrics/logging, for example, you need the sidecar to run for as long as the main process does.

• Never block the pod from being terminated, since these are only supporting the pod’s core functionality – without the new feature, a running sidecar container can stop a job from being completed even if a pod’s core task is done

On the arguably uglier side, in this alpha implementation you define a sidecar container by adding a restartPolicy field with the value Always to your initContainer spec. For example:

kind: Pod
metadata:
  name: myapp-pod
spec:
  initContainers:
  - name: init-myservice
    image: busybox:1.28
  - name: init-mydb
    image: busybox:1.28
  - name: istio-proxy
    image: istio/proxyv2:1.16.0
    args: ["proxy", "sidecar"]
    restartPolicy: Always
  containers:
  - name: myapp-container
    image: busybox:1.28

In the spec above, init-myservice and init-mydb are standard issue initContainers, while the restartPolicy field set to Always makes istio-proxy a sidecar container. 

The Kubernetes Enhancement Proposal (KEP) for the new feature acknowledges this ostensible inelegance, noting that initContainer “is not a good fit for sidecar containers as they typically do more than initialization” and suggesting that “infrastructure containers” would be a better name that might be adopted in the future. The KEP explains the thinking behind the chosen structure by saying:

A senior contributor added a little more flavor on Hacker News, noting:

In another comment, they added that the team wanted to...

The KEP offers detailed insight on both the problem cases that prompted the feature as well as the interesting wider context that some organizations were running forks of Kubernetes in order to implement features similar to this. 

If you’re impatient to try out the new feature on a fresh test cluster, you’ll need to enable the SidecarContainers feature gate for the kubelet, kube-apiserver, kube-controller-manager, and kube-scheduler. The KEP offers useful details on default policies and implementation, and you can expect to see more discussion of the feature with the release of Kubernetes 1.28 in August.