microk8s - change location of hostpath storage
Adjusting the hostpath storage location on microk8s requires you to adjust parameters in the associated deployment. This easy change is shown in this post.
From time to time we need small container hosts - very small, single hosts. To work with these and still get benefits of gitops deployments, we use microk8s which has reasonable footprint.
On our installations we're strictly moving persistent data to dedicated disks and do not store these on the operating system disks (for various reasons). So, our layout looks for like this:
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 64G 0 disk
├─sda1 8:1 0 512M 0 part /boot/efi
└─sda2 8:2 0 63.5G 0 part /
sdb 8:16 0 128G 0 disk
└─sdb1 8:17 0 128G 0 part /mnt/data
As these hosts are by design not meant to be scaled out, we want to avoid using a replicated storage like ceph or mayastor as this adds overhead in terms of memory and reduces throughput. We want a very simple system, hostPath storage.
Long story short: We enable hostpath storage according to the documentation with microk8s enable hostpath-storage.
But - then all PVs will default to /var/snap/microk8s/common/default-storage/ and we don't want this. We want the location be /mnt/data/microk8s.
To adjust this, the documentation states to create a new storageClass - which didn't work in our case. We needed to adjust the deployment (namespace: kube-system) of the hostpath-provisioner as the hostPath root for the provisioning is mapped there. Additional (to make things more clear) we provided a custom, default storageClass without a pvDir parameter.
apiVersion: apps/v1
kind: Deployment
metadata:
annotations:
labels:
k8s-app: hostpath-provisioner
name: hostpath-provisioner
namespace: kube-system
spec:
progressDeadlineSeconds: 600
replicas: 1
revisionHistoryLimit: 0
selector:
matchLabels:
k8s-app: hostpath-provisioner
strategy:
rollingUpdate:
maxSurge: 25%
maxUnavailable: 25%
type: RollingUpdate
template:
metadata:
creationTimestamp: null
labels:
k8s-app: hostpath-provisioner
spec:
containers:
- env:
- name: NAMESPACE
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.namespace
- name: NODE_NAME
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: spec.nodeName
- name: PV_DIR
value: /mnt/data/microk8s
- name: BUSYBOX_IMAGE
value: busybox:1.28.4
image: cdkbot/hostpath-provisioner:1.2.0
imagePullPolicy: IfNotPresent
name: hostpath-provisioner
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /var/snap/microk8s/common/default-storage
name: pv-volume
dnsPolicy: ClusterFirst
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
serviceAccount: microk8s-hostpath
serviceAccountName: microk8s-hostpath
terminationGracePeriodSeconds: 30
volumes:
- hostPath:
path: /mnt/data/microk8s
type: ""
name: pv-volume
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
annotations:
storageclass.kubernetes.io/is-default-class: "true"
name: hostpath-data
provisioner: microk8s.io/hostpath
reclaimPolicy: Retain
volumeBindingMode: WaitForFirstConsumer
allowVolumeExpansion: trueAs you can see, the only thing we adjusted is the mountPath and PV_DIR within the deployment.
Newly created PVs will now be placed on the new location - would be nice if the manual would be updated to reflect this requirement. Enjoy!
