I believe the Kubelet static pod deserves its headlines, considering it plays a significant role in our proof-of-concept. Kubelet is the component managing our KeepAlived, HAProxy, and CoreDNS, making sure these applications are healthy and up and running all the time. Therefore, before we start using Kubelet in our POC, let's start this blog by defining Kubelet, how it works, and Kubelet Static Pods?
The Kubelet is a critical component of the Kubernetes system, responsible for managing and monitoring each node in the cluster. As each node in a Kubernetes cluster can be thought of as a small Kubernetes deployment in and of itself, the Kubelet plays a vital role in ensuring that each node is healthy and running its workloads as expected.
The Kubelet receives information about the state of the cluster from the Kubernetes API server and responds to any changes in that state by managing the node accordingly. For example, API server detects that a pod is deleted from the cluster, it will notify all of the nodes in the cluster. The Kubelet on each node will then remove any running containers associated with that Pod.
Kubelet is also responsible for maintaining a local cache of all of the pods in the cluster. This cache makes Kubelet quickly respond to any changes without querying the API server for each shift.
The static Pod is a manifest that describes how to run the application using Kubelet daemon sets without the Kubernetes API server observing them. The manifest includes settings for liveness and readiness probes, resource limits, logging, and other options. The Kubelet uses this manifest to create containers on the node that it is running on.
The Kubelet is the primary component of any Kubernetes node and is responsible for maintaining a list of pods running on the node. Pods can either be static or dynamic. Static pods are defined in a JSON or YAML file, and Kubelet watches for changes to these files. Kubelet creates dynamic pods based on instructions from the Kubernetes API server. Kubelet manages static pods by creating and deleting them as needed, based on the contents of the pod manifest files. It also monitors the health of static pods, and if a pod dies, Kubelet will restart it. Kubelet ensures that the pods running on a node are always in the desired state.
Static pods are commonly utilized by software that bootstraps Kubernetes. For example, static pods are used by Kubeadm to bring up Kubernetes control plane components such as API-Server, Controller-Manager, and Kube-Scheduler. One of the reasons it chose static Pods is that it can not generate these as standard pods since the Kube API Server is not yet ready.
When Kubeadm starts the Kubernetes control plane, it creates Pod manifests for the API-Server, Controller-Manager and Kube-Scheduler in a directory that Kubelet watches. The control plane components are then brought up as static pods by Kubelet. Kubelet may regularly sync Pod manifests; you can variously configure this setting. You can use,
- A web URL configured using
--manifest-urlargument to the Kubelet process. -or watch a directory on the host file system specified with the
--pod-manifest-pathparameter to Kubelet Process
- You can use the Kubelet config file to configure this setting using the
Take a look at the following output of the Kubelet running process,
root 2508 9.1 0.5 4897580 221580 ? Ssl 14:44 3:28 kubelet --config=/etc/kubernetes/kubelet.conf --bootstrap-kubeconfig=/etc/kubernetes/kubeconfig --kubeconfig=/var/lib/kubelet/kubeconfig --container-runtime=remote --container-runtime-endpoint=/var/run/crio/crio.sock --runtime-cgroups=/system.slice/crio.service --node-labels=node-role.kubernetes.io/master,node.openshift.io/os_id=rhcos --node-ip=192.168.7.143 --address=192.168.7.143 --minimum-container-ttl-duration=6m0s --cloud-provider= --volume-plugin-dir=/etc/kubernetes/kubelet-plugins/volume/exec --hostname-override= --register-with-taints=node-role.kubernetes.io/master=:NoSchedule --pod-infra-container-image=quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4b6376a0a30fd543e43ed46f53722300e58271fe9d615329334c703cd46dbf94 --system-reserved=cpu=500m,memory=1Gi --v=2
The output showing kubelet is using config
--config=/etc/kubernetes/kubelet.conf, and if you cat this file, you will see the following,
kind: KubeletConfiguration apiVersion: kubelet.config.k8s.io/v1beta1 authentication: x509: clientCAFile: /etc/kubernetes/kubelet-ca.crt anonymous: enabled: false cgroupDriver: systemd cgroupRoot: / clusterDNS: - 172.30.0.10 clusterDomain: cluster.local containerLogMaxSize: 50Mi maxPods: 250 kubeAPIQPS: 50 kubeAPIBurst: 100 rotateCertificates: true serializeImagePulls: false staticPodPath: /etc/kubernetes/manifests systemCgroups: /system.slice systemReserved: ephemeral-storage: 1Gi featureGates: APIPriorityAndFairness: true LegacyNodeRoleBehavior: false NodeDisruptionExclusion: true RotateKubeletServerCertificate: true ServiceNodeExclusion: true SupportPodPidsLimit: true DownwardAPIHugePages: true PodSecurity: true CSIMigrationAWS: false CSIMigrationAzureDisk: false CSIMigrationAzureFile: false CSIMigrationGCE: false CSIMigrationOpenStack: false CSIMigrationvSphere: false serverTLSBootstrap: true tlsMinVersion: VersionTLS12 tlsCipherSuites: - TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 - TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 - TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 - TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256 - TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256
staticPodPath is pointed to
/etc/kubernetes/manifests directory. So this means any static pods manifest in this directory kubelet will keep watching for the changes and adjust accordingly.
We'll handle our Hyperconverged OCP cluster components using the same method.
Now we understand Kubelet and how it works, and we have gathered all static pods manifest files for KeepAlived, HAProxy, and CoreDNS components in our previous blog post. We can start creating custom ignition config files. So the idea is to generate the default ignition config using the
openshift-install tool and merge this default config using the Filetranspiler tool to create the custom-ignition config. Finally, we can upload this ignition config to some webserver and fetch that config during the installation process.
Let's see how we can achieve that in the upcoming blog.