Balance innovation and agility with security and compliance
risks using a 3-step process across all cloud infrastructure.
Step up business agility without compromising
security or compliance
Everything you need to become a Kubernetes expert.
Always for free!
Everything you need to know about Magalix
culture and much more
The core function of Kubernetes is resources scheduling. Kubernetes scheduler makes sure that containers get enough resources to execute properly. This process is governed by the scheduling policies. Before digging into how the scheduler works, let’s make sure we understand the basic constructs of resources definitions, allocation, and restrictions inside the Kubernetes cluster.
Kubernetes has only two built-in manageable resources: CPU and memory. CPU base units are cores, and memory is specified in bytes. These two resources play a critical role in how the scheduler allocates pods to nodes. Memory and CPU can be requested, allocated, and consumed. You should always set the right CPU memory values. You will be in control of your cluster and make sure that a misbehaving application does not impact the capacity available for other pods in your cluster.
Kubernetes uses the requests & limits structure to control resources such as CPU and memory.
CPU is a compressible resource, which means that once your container reaches the limit, it will keep running but the operating system will throttle it and keep de-scheduling from using the CPU. Memory, on the other hand, is none compressible resource. Once your container reaches the memory limit, it will be terminated, aka OOM (Out of Memory) killed. If your container keeps OOM killed, Kubernetes will report that it is in a crash loop.
The limit can never be lower than the request. Kubernetes will throw an error and won’t let you run the container if your limit is higher than the request.
TIP: Set requests and limits at the container level. It is a good practice to set it at the container level for more control and a more efficient distribution of containers. If your main container consumes gigabytes of memory and a sidecar container that need a few megabytes, setting the request and limits at the pod level will give them the same amount of memory. Use Magalix Autopilot to automatically set and update the values of limits and requests of your containers.
To specify a CPU request for a Container, include the resources:requests field in the Container’s resource manifest. Similarly, to specify a CPU limit, include resources:limits.
CPU unit inside Kubernetes is originally equivalent to one hyper thread if you are running on a bare-metal Intel process with Hyperthreading. It is important to understand how this mapped to the different CPU capacities of major cloud providers out there. Misinterpreting these may cause bad performance results inside your Kubernetes cluster. The table below is a quick mapping of different cloud providers' CPU units.
|Infrastructure||1 CPU Equivalent|
|bare-metal Intel processor with Hyperthreading||1 Hyperthread|
To specify a memory request for a Container, include the resources requests field in the Container’s resource manifest. To specify a memory limit, include resources:limits. The Container has a memory request of 100 MiB and a memory limit of 200 MiB. If you want to learn more about Memory requests & limits see this reference from the Kubernetes documentation.
Limits and requests for memory are measured in bytes. You can express memory as a plain integer or as a fixed-point integer using one of these suffixes: E, P, T, G, M, K.
Take a look at the below example. It is a Pod with two containers. Each Container has a request of 0.5 CPU and 300MiB of memory. Each Container has a limit of 1 CPU and 500MiB of memory.
Ideally, you want your team members to always set limits and resources. But in the real world, your team will forget to so :) Engineers can easily forget to set the resources, or someone can just get to the old habits of over-provisioning to be at the safe side.
You can prevent these scenarios. You can set up ResourceQuotas and LimitRanges at the Namespace level.
You can lock namespaces using ResourceQuotas. ResourceQuotas let’s you just look at how you can restrict CPU and Memory resource usage for containers inside that namespace. A Quota for resources might look something like this:
You can see there are four sections. Configuring each of these sections is optional.
TIP: If you are using a production and development Namespace, it is recommended to avoid defining quota on the production Namespace and define strict quotas on the development Namespace. You don’t want your production containers to be throttled or evicted because the dev env need more resources!
the LimitRange applies to an individual container. This can help prevent your team members from creating super tiny or super large containers inside the Namespace. A LimitRange might look like this:
You can see there are four optional sections.
It is important to understand how pods get granted resources and what happens when they exceed what is allocated or the overall capacity of your cluster so you can tune your containers and cluster capacity correctly. Below is the typical pod scheduling workflow:
You allocate resources to your pod through the spec and run kubectl apply command
Kubernetes scheduler will use round-robin scheduling policy to pick a Node to run your pod
For each node Kubernetes checks if the node has enough resources to fulfill the resources requests.
The pod is scheduled for the first node has enough resources.
The pod goes into the pending state if none of the available nodes has enough resources.
If you are using CA autoscaler, your cluster will scale the number of nodes to allocate more capacity.
If a node is running pods where the sum of their limits is more than its available capacity, Kubernetes goes into the Overcommitted State.
Because CPU can be compressed, Kubernetes will make sure your containers get the CPU they requested and will throttle the rest.
Memory cannot be compressed, so Kubernetes needs to start making decisions on what containers to terminate if the Node runs out of memory.
Keep in mind that Kubernetes optimizes for the whole system’s health and availability. When it goes into the overcommitted state, the Kubernetes scheduler may make decisions to kill pods. Generally, if a pod is using resources more than requested, that pod becomes a candidate for termination. Here you go the conditions that you should keep in mind:
Self-service developer platform is all about creating a frictionless development process, boosting developer velocity, and increasing developer autonomy. Learn more about self-service platforms and why it’s important.
More and more businesses are adopting GitOps. Learn about the 5 reasons why GitOps is important for businesses.