Balance innovation and agility with security and compliance
risks using a 3-step process across all cloud infrastructure.
Step up business agility without compromising
security or compliance
Everything you need to become a Kubernetes expert.
Always for free!
Everything you need to know about Magalix
culture and much more
A daemon is a process that runs in the background. Typically, a daemon does not produce visible output to the user; it does not accept inputs. Daemons exist to perform background jobs. For example, in Linux, we have the httpd to respond to HTTP requests, sshd to grant remote users secure remote-shell access. We also have several kernel daemons that do not accept users input; they exist to perform housekeeping and other essential tasks that the kernel needs to function correctly. Sometimes, users may create or install their demons. For example, logrotated is a popular Linux daemon that routinely archives old log files in configurable paths according to user-defined settings. Another example is log shippers (filebeat, fluentd,etc.) that periodically send logs to a log aggregation service like ELK stack for analysis and correlation.
Kubernetes is often referred to as the “data center operating system.” As we just discussed, an operating system needs daemons to perform background jobs that users do not (and should not) interact with. So, in Kubernetes, higher-level controllers like Deployments need to continually monitor the number of running Pods so that it spawns or kills Pods as required. Such a task needs to run through a daemon: a background process that needs no user interaction, it’s always running and is chiefly managed by the Kubernetes engine itself. Kubernetes administrators may also need daemons to execute tasks on the running nodes. For that purpose, Kubernetes offers the DaemonSet resource. Like a Deployment, a ReplicaSet, or a StatefulSet, a DaemonSet creates and manages Pods. However, those Pods are configured so that they run on all the cluster nodes.
Because that’s not how the cluster works, a daemon that’s directly installed on a node is not managed by Kubernetes, it is, instead, controlled by and reports to the node’s operating system. Any changes that you need to make to the daemon configuration need to be performed on every node. If the daemon stops working or reports errors, Kubernetes does nothing for you. You need to configure the OS or some third-party tool to restart the daemon if it fails. But, wasn’t Kubernetes designed for that sort of tasks? That’s why you’d be much better off using a DaemonSet.
The following is a stripped-down version of the official fluentd daemonset definition file:
apiVersion: extensions/v1beta1 kind: DaemonSet metadata: name: fluentd namespace: kube-system labels: k8s-app: fluentd-logging version: v1 spec: template: metadata: labels: k8s-app: fluentd-logging version: v1 spec: nodeSelector: env: prod containers: - name: fluentd image: fluent/fluentd-kubernetes-daemonset:elasticsearch volumeMounts: - name: varlog mountPath: /var/log - name: varlibdockercontainers mountPath: /var/lib/docker/containers readOnly: true terminationGracePeriodSeconds: 30 volumes: - name: varlog hostPath: path: /var/log - name: varlibdockercontainers hostPath: path: /var/lib/docker/containers
Let’s have a closer look at the key components of this definition:
DaemonSets and ReplicaSets may look very similar. However, they have several significant differences:
Most of the time, you don’t need to communicate with Pods spawned by DaemonSets. Those Pods are mainly used for background tasks, housekeeping, log aggregation, and so on. But what if you do want to send an HTTP request to the Pod and examine the response? For example, a custom log-collection Pod may have a health or status endpoint that displays information about the number of logs it already processed, whether there were errors...etc. Let’s explore the different options that you have:
Prevent Kubernetes NetworkPolicy misconfigurations by enforcing policy as code