You've successfully subscribed to Nuvotex Blog
Great! Next, complete checkout for full access to Nuvotex Blog
Welcome back! You've successfully signed in.
Success! Your account is fully activated, you now have access to all content.
Success! Your billing info is updated.
Billing info update failed.

Ubuntu 22.04 (or 21.10), kubernetes & cgroups v2

Daniel Nachtrub
Daniel Nachtrub

On some of our systems we've recently upgraded Ubuntu to 22.04 LTS (from 20.04 LTS). And one thing that has been problematic afterwards is that containers kept crashing - or to be more exact: the containers have been killed because the probes failed.

This affected cilium as well as the kube-apiserver or kube-controller. The most common denominator has been that all affected containers have been executed in hostNetwork mode.

The error message has been like:

Startup probe failed: Get "": dial tcp connect: connection refused
example error message

Validating the connection from the host itself showed it is working successfully.

The reason for this is that Ubuntu enabled cgroups v2 with 21.10 (or 22.04 LTS) by default and that this causes issues.

Workaround: disable cgroups v2

Until now we only have a workaround for this, by enabling cgroups v1 using a boot parameter. This can be done setting it like this:


Having applied this, update grub and reboot afterwards.

apply settings

Now you have a similar behaviour like on 20.04 LTS. Having said that, it's just a workaround, we're still checking out a solution that makes use uf cgroups v2.

Daniel Nachtrub

Kind of likes computers. Linux foundation certified: LFCS / CKA / CKAD / CKS. Microsoft certified: Cybersecurity Architect Expert & Azure Solutions Architect Expert.