Author CKA: Vincenzo Tagliavia (CKA, CKAD, CKS)
Last modified: 19 Nov 2024
Our checklist of Kubernetes security best practices help simplify and improve your security posture with Kubernetes on-premise deployments. We categorize Kubernetes security best practices into 3 main architectural components: 1. system, 2. orchestrator and 3. application.
Yes. Kubernetes provides authentication, authorization, admission controls and auditing mechanisms (AAAA) to secure your cluster. Like with any other technology, knowledge of the platform and its default configurations will help prevent mis-configurations. The better your team’s knowledge around Kubernetes, the better your security posture will be.
Security breaches come mainly from misconfigurations. These misconfigurations are the direct result of lack of knowledge or insufficient security policies. The impact security breaches have on organizations is enormous but, it isn’t Kubernetes the cause.
1. Misconfiguration
Lack of specialized knowledge and human errors are very common. For example, if your team does not know that Kubernetes default values are laxed by design, they won’t know how to configure them.
2. Secrets Management
Kubernetes does’t use encryption by default, only base64 encoding. Third-party secret providers are the de-facto standard to decouple secrets management and risks from the cluster. ETCD's encryption at rest isn't enabled by default either.
3. Weak Role Base Access Control
If Admission controllers and Policies are not properly set up, you will miss that granularity that is important to control request-response cycles.
4. Supply Chain Vulnerabilities
Containers run software inside nodes. Containers’ images are pulled from registries that are either hosted inside or outside the cluster. So these registries are an input to nodes and must be secured. If we don’t, we leave the cluster vulnerable to supply chain attacks.
1. Install open-source security forensic tools
If you have limited resources to support security processes in your organization, install a security forensic tool, such as kubesec or Falco. The Center for Internet Security (CIS) and Kubernetes Benchmarks provide a rich set of security datasets and best practices for Kubernetes environments.
Alternatively, if your security posture is more proactive and you design policies that support your security processes, the Open Policy Agent (OPA) is an additional open-source tool to consider. OPA has a learning curve but its advantages stack up in your favor.
2. Apply redundant and defense-in-depth features
Each layer in your Kubernetes architecture should include both redundant security measures and defense-in-depth strategies. When you combine and apply security frameworks like AAA, the Principle of Least Privilege, or Zero-Trust architecture for example, you strengthen your security posture exponentially. Redundancy and defense-in-depth together have an even more compounding effect.
Redundancy requires duplication of security resources to support failover and high availability. Redundancy however, increases your resource consumption and may impact performance and throughput in some cases. This trade-off is based on design decisions and your specific requirements.
Defense-in-Depth involves different security measures to enhance system robustness and resilience. Like with redundancy measures, you need to consider trade-offs at the design stage.
3. Focus on Supply Chain Vulnerabilities, not just Kubernetes
The CNCF Security Model is an extension of the CISA Security Whitepaper and represents a DevOps pipeline model with four interrelated phases:
Each phase requires its own security attestations and controls. But what happens if you have no direct control over these phases? Security integrations and decentralized processes become major pain points. These pain points will determine your security implementations at runtime.
The high-level view offers security best practices for each Kubernetes architecture component: system, cluster and application layers.
System security includes hardware components and the OS Host.
1. Use hardened OS images
Hardened OS images are security-enhanced OS that reduce the attack surface. For example, such operating systems can restrict WRITE operations or only ship the necessary binaries to perform essential functionalities. Talos Linux and Bottlerocket from AWS are two examples of hardened OS(s).
2. Employ Kernel security-enhanced modules
Enable SELinux, AppArmor or Seccomp to restrict system calls to the kernel. These Kernel modules enforce Mandatory Access Control (MAC) in addition to the default OS’s Discretionary Access Control (DAC), which is more permissive. MAC modules are more difficult to operationalize at scale so specific expertise is key to implement them.
3. Incorporate hardware security modules
Hardware Security Module (HSM) and Trusted Platform Module (TPM) are examples of hardware modules. Thes modules are tough to compromise because they operate at the hardware level and provide cryptographic protection and isolation for your workloads. Hardware modules require cost/benefit analysis and specific knowledge to implement. Other factors to consider are potential compatibility issues with some OS.
Security at this layer involves control plane configuration and hardening. The main components that require interventions are the Kubernetes API Server, ETCD and the Kubelet. The following list is not exhaustive.
1. Implement Native Admission Controllers
You implement security policies (Admission Controllers) to validate or modify access to the cluster. These plugins allow more fine-grained authentication and authorization mechanisms. Kubernetes provides Admission Controllers natively but you can also use external policy frameworks (e.g. OPA).
2. Enable Kubernetes API Server Audit logging.
This requires configuration of the Kubernetes API Server and it is compatible with some open-source static and dynamic scanners. Audit Logging only needs a reference to a config file and a backend volume for storage. With knowledge of Linux OS Systems, logs management is vastly simplified.
3. Use a Kubernetes Administrator (CKA or CKS)
We recommend you to consult a Kubernetes specialist to configure and harden your clusters. Kubernetes on-premise or bare metal is a completely different beast than managing Kubernetes in a cloud environment.
You secure your applications by managing supply chain and application lifecycle stages: development, distribution, deployment and runtime. Multi-tenancy setups and distributed teams are common challenges.
1. Adopt the 12-Factor-App development framework
The 12-Factor-App development framework is a set of best practices and guidelines to ship high-quality code. If you have control over development (first phase of CNCF's Security Model), this framework supports portability, maintainability, testability.It can slow down developments and has a learning curve.
2. Adopt a “shift-left” security approach
Integrate security measures early in the development process. For example, use container scanning tools to detect vulnerabilities in images before they are deployed. Or enforce open-source dependencies checks to ensure only high-quality inputs enter your code.
3. Implement Network Policies
Think of Network Policies as native lightweight firewall rules. They allow you to control how traffic flows inside and outside the cluster. Network Policies operate at the OSI Layer 3 and 4. You can specify rules by namespace, pod labels and CIDR block ranges.
Use Service Meshes for more complex requirements. For example, if you require more control over TLS or if logging network security events is crucial to your organization, Services Meshes support either and much more.
Security implementations don’t act in a vacuum. If your strategy doesn’t prioritize and support security, for example with the right policies, processes and people, implementations will be affected.
The majority of security breaches are caused by misconfigurations. Therefore, if we invest in knowledge and talent we prevent these breaches.
DevSecOps supports cross-team collaboration and security integrations. If you support DevSecOps at each stage of your application lifecycle, you improve your security posture.
AI tools predict, analyze and identify patterns across millions of data points. This can be a source of competitive advantage for years to come.
AI/ML models can learn from extensive datasets of known vulnerabilities to predict and identify potential security issues in code or configuration files. If we feed these models with new knowledge and new security breaches as they happen, we could build the next-generation of security tools with more precision and power than ever before.
Kubernetes has security mechanisms baked into the platform but we have to configure and implement them. Lack of know-how – resulting in misconfigurations – and poor allocation of resources to security are common causes of security breaches.
Why are security concerns growing?
Because we lack high-quality specialists and meticulous design in infrastructure deployments. We suggest kubernetes security best practices with a simplified framework that helps secure your infrastructure. But business policies and processes must support implementations and stopping or slowing down projects will not make your cluster more secure. Quite the contrary.