What Is Monitoring?
Today, the massive use of technology in large-scale ventures is very popular for businesses. Monitoring systems are responsible for monitoring a company's equipment (hardware, networks, communications, operating systems, or applications), to monitor its operation and performance, and identify potential errors. A successful monitoring system is capable of monitoring devices, infrastructure, software, facilities, and even processes in industry. We often use control systems to track device resources, such as usage and frequency of CPUs, or the amount of free RAMs. These are often used to display free space on one or more hard drives, the temperature of the CPU and other critical parts, and networking details. It’s very useful to have a good monitoring device to identify and avoid failures.
Types Of Monitoring Tools With Kubernetes
Prometheus is one of the most common surveillance tools used by Kubernetes. It’s a part of the Native Computing Base Cloud. This project, initially created by SoundCloud and subsequently donated to CNCF, is inspired by Google Borg Monitor.
As a time series Prometheus stores all of its info. This data can be queried through the language of the PromQL database and visualized through an integrated browser of text. Since Prometheus is not a dashboard, Grafana relies on it for data visualization.
Version 1.0 of this tool was published in 2016, and is now one of Kubernetes’ most commonly used monitoring devices. Other Kubernetes ecosystem resources, including Istio, include an embedded Prometheus adapter that exposes metrics produced.
Prometheus can be built as a single binary that can be run on your host or as a Docker server directly. With the Prometheus Controller, running Prometheus on Kubernetes is easily accomplished.
2. Container Advisor (CAdvisor)
CAdvisor is an agent for the use of container tools and output analysis - it’s built into the Kubelet binary. CAdvistor auto-discovers all computer containers and collects memory, network use, file system and CPU statistics. CAdvisor supports native Docker containers. It does not function on the pod level but on each node. Nonetheless, be advised: CAdvisor is a simple yet restricted device, so if you're looking to store metrics for long-term usage or conduct complex monitoring behavior, cAdvisor won't suit your requirements:
3. Kubernetes Dashboard
Kubernetes Dashboard is a web-based, Kubernetes cluster UI add-on. It has several features allowing users to build and manage workloads, as well as discovery, load balancing, setup, storage, and monitoring. It's helpful to small clusters, and people who are just starting to learn Kubernetes.
This tool offers different views aggregated over all nodes for CPU and memory usage metrics. It can also be used to track workload health status (Pods, Deployments, Replica Sets, Cron jobs, etc.). It’s very quick and easy to install the Kubernetes Dashboard, which can be done using ready-to-use YAML files.
Kubewatch is a Kubernetes watchman that publishes updates of events in a Slack channel and lets you determine the resources to be monitored. It is written in Golang and uses a client library from Kubernetes to communicate with a server from the Kubernetes API.
You can pick the resources you need to monitor: daemon sets, servers, pods, replica sets, replication controllers, services, secrets, and configuration maps using a simple YAML file.
LAB: Creating Prometheus On Kubernetes Cluster To Monitor Your Nodes
There are many ways to install Prometheus in your Kubernetes cluster:
- Directly as a single binary running on your hosts, which is fine for learning, testing and developing purposes but not appropriate for a containerized deployment.
- As a Docker container which has several orchestration options: Raw Docker containers, Kubernetes Deployments / StatefulSets, the Helm Kubernetes package manager, Kubernetes operators, etc.
Prometheus Web Interface (Port 9090 By Default).
The better option to deploy the Prometheus server inside a container:
docker run -p 9090:9090 -v /tmp/prometheus.yml:/etc/prometheus/prometheus.yml prom/prometheus
Note that you can easily adapt this Docker container into a proper Kubernetes Deployment object that will mount the configuration from a ConfigMap, expose a service, and deploy multiple replicas, etc.
And then you can apply this yaml file below:
apiVersion: apps/v1 kind: Deployment metadata: name: prometheus-deployment labels: app: prometheus purpose: example spec: replicas: 2 selector: matchLabels: app: prometheus purpose: example template: metadata: labels: app: prometheus purpose: example spec: containers: - name: prometheus-example image: prom/prometheus volumeMounts: - name: config-volume mountPath: /etc/prometheus/prometheus.yml subPath: prometheus.yml ports: - containerPort: 9090 volumes: - name: config-volume configMap: name: prometheus-example-cm
Next, add this service file below (to the above yaml file):
--- kind: Service apiVersion: v1 metadata: name: prometheus-example-service spec: selector: app: prometheus purpose: example ports: - name: promui protocol: TCP port: 9090 targetPort: 9090
And we can run the below command :
kubectl create configmap prometheus-example-cm --from-fileprometheus.yml
If you don’t want to configure a LoadBalancer, then you can specify the type NodePort for your service.
After a few seconds, you should see the pods of Prometheus running in your cluster as below:
$ kubectl get pods NAME READY STATUS RESTARTS AGE prometheus-deployment-68c5f4d474-cn5cb 1/1 Running 0 3h prometheus-deployment-68c5f4d474-ldk9p 1/1 Running 0 3h
Monitoring The Kubernetes Nodes With Prometheus:
The Kubernetes nodes or hosts will need to be monitored and we have plenty of tools to monitor a Linux host. In this guide, we’re going to use the Prometheus Node-exporter:
- It’s hosted by the Prometheus project itself
- It will be automatically deployed when we use the Prometheus operator in the next chapters
- Can be deployed as a DaemonSet, so it will automatically scale if you add or remove nodes from your cluster.
You have several options to deploy this service, for example, using the DaemonSet in this repo for minikube:
kubectl create ns monitoring kubectl create -f https://raw.githubusercontent.com/bakins/minikube-prometheus-demo/master/node-exporter-daemonset.yml
If you want to use Helm 3, remember to create the RBAC roles and service accounts for the tiller component before proceeding.
helm init --service-account tiller helm install --name node-exporter stable/prometheus-node-exporter
Once the chart is installed and running, you can display the service that you need:
kubectl get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE node-exporter-prometheus-node-exporter ClusterIP 10.101.57.207 9100/TCP 17m
Once you add the config, you can start collecting and displaying the node metrics as seen in the below graph:
- To sum up the monitoring process, all of your Kubernetes data can easily be put together into one view. Available in cluster explorer Kubernetes, logs provide a near-instant search with complete contextual log information.
- Additionally, you can connect the log messages with program, network, Kubernetes, and event data while configuring logs in context.