Manually remove a member from etcd

Recently, I needed to rollback a kubernetes control-plane node to an older snapshot. This cause (obviously) that etcd could not operate anymore in the etcd cluster.

The approach in this scenario is to remove the etcd node from the cluster and add it again. The removal of it is rather easy.

• Take one of the remaining nodes and exec into the etcd container
• List all members
• Drop the affected member

To list all members and remove the desired one, just two commands are required.

# list members
etcdctl member list --endpoints=https://127.0.0.1:2379 --cacert=/etc/kubernetes/pki/etcd/ca.crt --cert=/etc/kubernetes/pki/etcd/peer.crt --key=/etc/kubernetes/pki/etcd/peer.key

# remove
etcdctl member remove <MEMBERID> --endpoints=https://127.0.0.1:2379 --cacert=/etc/kubernetes/pki/etcd/ca.crt --cert=/etc/kubernetes/pki/etcd/peer.crt --key=/etc/kubernetes/pki/etcd/peer.key

That's it - now you can join back the member to the cluster and it will replicate all data.