Weaveworks 2022.03 release featuring Magalix PaC | Learn more
Balance innovation and agility with security and compliance
risks using a 3-step process across all cloud infrastructure.
Step up business agility without compromising
security or compliance
Everything you need to become a Kubernetes expert.
Always for free!
Everything you need to know about Magalix
culture and much more
When it comes to cloud-native technologies, a lot of people fall into the trap of overlooking security. They mistakenly think that, since their application is hosted on somebody else’s datacenter, that somebody shall take all the required precautions to protect their business against digital attacks. While this assumption holds some truth, we have to make an important distinction between what lies on you (the client) and what is handled by the cloud provider in terms of security. For example, let’s assume that you host your application on a number of EC2 instances on AWS (Amazon Web Service). The following are some of what AWS is responsible for:
On the other hand, AWS cannot be held liable for the following (for example):
The above is just an over-simplified example of what can go wrong if you don’t take security as seriously as it should be taken. Also, having a third-party to host your application is - by no means - a guarantee that you’re 100% protected. And that leads to an important question:
If you research the web for security-related material, read books, attend training, you’ll never find a source claiming that you can achieve a 100% security level. The reason is that there’s, simply, nothing as a 100%-secured resource. What all information security professionals are working hard to achieve is to make the job harder for the attacker. Let’s take passwords for example. A little more than a decade ago, eight characters were sufficient to regard a password as strong, and MD5 was a powerful hashing algorithm. But now, you can easily figure out what an md5-hashed password was originally by using rainbow tables. Decrypting MD5 became very easy to the point that you can do it online (you can try https://www.md5online.org/md5-decrypt.html for an example). So, the point that I’m trying to drive home is that, no matter how hard you try, you’re only making the job harder for the attacker for the time being. You should always be updated to date with the latest security practices, technologies, and recommendations. In this series of articles, we are helping you achieve that with respect to Kubernetes. So, we discuss topics like configuration of Kubernetes components, controlling and limiting access to them. Also, we explain how to ensure that the code you’re hosting on the cluster is not vulnerable to attacks, how to restrict containers’ behavior so that they cannot be manipulated by attackers. Finally, we discuss the recommended ways of storing and retrieving credentials in your Kubernetes cluster.
In this article, we start the series by discussing two important defense principles: the multi-layered architecture and the least-privileged approach.
What measures would you follow when you need to secure your house?
Naturally, you’d lock the front door at all times, perhaps install an alarm system so that even if a burglar was able to unlock the door the alarm will set off. You might also take additional measures like not leaving large amounts of cash in the house and putting them in a bank instead. Alternatively, you could buy a safe and use it to store your valuables. Why are you doing this? In fact, you’re following the multi-layered security architecture when deciding how to protect your house and belongings even if you don’t know it. Let’s have a look at the layers that you have in place:
The lock at the door.
The alarm system.
The safe.
Now, if the attacker was able to break the door, disable the alarm, he/she would have a hard time trying to open the safe. Certainly, it might happen that someone could unlock the door, bypass the alarm, and manage to open the safe. But the question is how likely is this to happen?
Similarly, you shouldn’t rely on a single practice/security layer when designing your Kubernetes cluster security.
Let’s return to the house-security example. Would you design your door lock to use the same key that’s used to open your safe? Of course not. The reason is that if that key gets lost or stolen, it’ll be used to bypass only one security layer not all. So, if the attacker has the door key, he still needs to figure out how to unlock the safe. If he’s got the safe key, he still needs to make his way through the front door.
If we apply this analogy to software and Kubernetes in particular, you shouldn’t grant a component more privileges than it actually needs. For example, an Ingress controller should be granted read access to the services and namespaces but shouldn’t be able to modify them. If the controller was compromised, the attacker will have read-only access to other components.
In information security, the attack surface refers to all the possible ways through which an attacker can get into a system. So, which do you think is more vulnerable to burglars: a house with only one door or one with three or four gates?
The more the entrance points to a resource, the more “exposed” it is to attacks. In the software world, the same principle still holds. Assume that you are coding a Python application that needs to connect to an external API. While you were testing different HTTP access methods, you came across the requests library. And then, after using it for some time, you decided that it’s not the best approach and you settled on another module. The problem is that you forgot to remove the import requests line from your code, which means that the library is still available to the code although it is not used. Now, an attacker can use any known vulnerabilities in requests to compromise your application. Such an attack could’ve been easily mitigated by simply not importing modules that you’re not using.
Let’s start applying those principles on Kubernetes, starting with the components. The core of Kubernetes is the API server so let’s start with it.
The API server offers a REST interface for Kubernetes controllers (and users) to interact with different cluster components. It is the first and the most important core component to consider when securing your cluster.
By default, the API server listens on port 8080 for HTTP requests. If you have this default configuration then all the communication between you and the API server is sent in cleartext. Any network sniffing program (for example, Wireshark) can reveal sensitive information about your cluster including application settings (in configMaps), Secrets and so on. If you are running your cluster as a service from a cloud provider, chances are the API server is already using the secure port 443 over TLS to encrypt communications. However, you can always check whether or not this is the case by trying to send an HTTP GET request to the API server and make sure you don’t receive a valid response. For example:
curl -X GET apiserver:8080
If you have a valid output, then you need to close that port by using the --insecure-port=0 when starting the API server. Note that Kubernetes versions that are later than 1.10 may already have this option deprecated or totally removed so that the system starts at the secure port 443 by default. In all cases, you should do this check.
For your reference, securing the API server falls under the multi-layered security principle. The next layer that we should protect is the kubelet component.
The kubelet is an agent that sits at the top of every node in the cluster. It is responsible for receiving and executing commands against the containers running on the nodes. To do this, the kubelet has its own API interface through which it receives the instructions through HTTP requests. Securing the kubelet component involves multiple layers of its own:
The etcd is the distributed database that contains all the information necessary for the cluster to function correctly. If an attacker could gain write access to this database, they are effectively controlling the cluster. The following startup flags will help mitigate several risks related to running etcd:
Since the API server also communicates with etcd, a similar configuration must be applied to it so it can communicate with etcd through HTTPS.
Self-service developer platform is all about creating a frictionless development process, boosting developer velocity, and increasing developer autonomy. Learn more about self-service platforms and why it’s important.
Explore how you can get started with GitOps using Weave GitOps products: Weave GitOps Core and Weave GitOps Enterprise. Read more.
More and more businesses are adopting GitOps. Learn about the 5 reasons why GitOps is important for businesses.
Implement the proper governance and operational excellence in your Kubernetes clusters.
Comments and Responses