CKS: Cilium Network Policies

Introduction

In the previous blog of this series we discussed the basics of runtime security on Kubernetes with Falco as well as some tips and tricks to efficiently tackle the question(s) on the CKS exam. This is the third part where we will be covering the basics of Cilium network policies. Like always, you won’t find direct answers to any of the exam questions here but I will try to point you in the right direction.

Cilium

I’ve learned to appreciate Cilium more and more. It is a container networking tool that uses the Linux kernel technology called eBPF to dynamically insert code into the kernel which allows it to efficiently and flexibly implement security, networking and observability logic. If you remember the previous post about Falco, it was also heavily using eBPF. It enabled so many great tools already that I will definitely take a deep-dive into eBPF at some point. Powerful stuff!

Cilium is a lot of things, but at its heart it is a Container Networking Interface plugin, often referred to as CNI-plugin or Network-plugin. Such a plugin is an implementation of the Container Networking Interface which was created to standardise networking across as many tools as possible in the container and container-orchestration ecosystem. A network-plugin is only concerned about network connectivity between containers, yet different vendors can add their own flavor on top of it to do a lot more.

And a lot more is what Cilium has done. Not only is it a CNI enabling networking connectivity in you cluster(s!), it also acts as an observability tool and service mesh. There is so much to cover, so this post will only handle the CKS exam requirements but do watch out for a deep-dive on the awesome capabilities of Cilium.

Network Policies in Kubernetes

Before we dive into Cilium and its network policies, lets discuss the prerequisites. As you have achieved your Certified Kubernetes Administrator (CKA) certification you are already familiar with configuring a CNI, container- and Linux networking as well as network policies in Kubernetes. You can skip ahead. For those who are not yet familiar with them, lets go over the basics quickly.

By default, any pod can access any other pod over the network in you cluster. Yes, you are working on your own little island, know all other developers by name and trusting them not to make unauthorized calls to other services.. I think we can agree that a potential hacker will be filled with joy when they find out. Last time I checked the Zero Trust principle mentioned something like “Never trust, always verify”.

Network policies are meant to restrict network traffic flows at the IP-address or port level (OSI Layer 3 & 4) and to only allow those flows that are necessary. Network policies are implemented by the network plugin, this will bring us to Cilium in a minute. With the default network policies available in Kubernetes it is possible to allow or deny communication between a pod and other network ‘entities’. These entities are:

Other pods
Namespaces
IP ranges

For pod and namespace based network policies you can use selectors to choose one or more entities based on their labels specified in their metadata. For IP-based network policies you should define the CIDR-ranges to allow or deny.

By default, all ingress and egress traffic is allowed. A pod is isolated for ingress or egress if there is any network policy that both selects the pod and has a policy type defined for ingress and/or egress respectively. When a pod is isolated from ingress or egress, the only allowed connections are specified in the ingress or egress list properties. You can have multiple different policies that target the same pods as they will be added together.

To start from a clean slate for our new application we create a new namespace and immediately define a network policy that blocks all incoming traffic like the one below. As you can see the pod selector is an empty object which means it will select all pods in our namespace. The policy type is ingress so only ingress traffic is restricted, and no allowed sources are present in the ingress list so all traffic will be blocked.

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-ingress
spec:
  podSelector: {}
  policyTypes:
  - Ingress

We can compare that to the following ingress policy which at first sight looks almost identical yet has very different results.

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-all-ingress
spec:
  podSelector: {}
  ingress:
  - {}
  policyTypes:
  - Ingress

Because the ingress list is present in the policy above and it specifies an empty selector object, it means that it selects all ‘sources’ and thus allows all traffic to flow towards your pods in this namespace. Very simple yet dangerous mistake to make!

Once we have a default deny policy in place we can start to allow certain traffic. The following example shows how we can allow flows from the pods labeled app: backend in a namespace labeled team: one to our workload labeled app: database in the current namespace.

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-backend-team-one-to-database
spec:
  podSelector:
    matchLabels:
      app: database
  ingress:
  - from:
    - namespaceSelector:
        matchLabels:
          team: one
      podSelector:
        matchLabels:
          app: backend
  policyTypes:
  - Ingress

Note that the policy above has a single entry in the from section to combine the namespace and pod selectors. If we separated these it would mean that all pods in the namespace labeled team: one and all pods labeled app: backend in the current namespace would be allowed to communicate with our database workload. A very different result.

Outside of managing access from namespace and pods we can also work with IP ranges. This can be done with the ipBlock property as shown in the example below.

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-ip-range-to-database
spec:
  podSelector:
    matchLabels:
      app: database
  ingress:
  - from:
    - ipBlock:
        cidr: 10.0.0.0/24
policyTypes:
  - Ingress

Something especially useful about the ip block is that you can disallow ingress or egress from and to a single ip address. During the exam you might be asked to use a default Kubernetes network policy to block egress to a metadata endpoint in your cluster. This can be achieved with the except property of ip block. The following policy allows egress to any ip address except a single one.

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-ip-range-except
spec:
  podSelector: {}
  egress:
  - to:
    - ipBlock:
        cidr: 0.0.0.0/0
        except: 123.45.67.89/32
policyTypes:
  - Egress

That was a quick overview of the default network policies available on Kubernetes.

Cilium Network Policies

While the overview of default network policies might seem like a lot it’s functionality quite limited compared to Cilium. If we list the different source and destination selectors you’ll immediately start to see the differences. Here is a small summary:

Kubernetes (Default)

Pods: with the podSelector property
Namespaces: with the namespaceSelector property
IP ranges: with the ipBlock property

Cilium

Endpoints: with the fromEndpoints and toEndpoints properties
Services: with the fromServices and toServices properties
Entities: with the fromEntities and toEntities properties
Nodes: with the fromNodes and toNodes properties
IP ranges: with the fromCIDR and toCIDR properties
DNS: with the toFQDN property

Note: these are only the OSI Layer 3 policies Cilium offers. There are even more possibilities using Layer 4 and Layer 7.

There are a few things you can notice here. First of all there are a lot more options. Secondly, there is no mention of pods. Third, there is no mention of anything namespace related. Let’s go over it quickly.

The basics

In Cilium, we talk about endpoints instead of pods. The term endpoint refers to a collection of containers that share the same ip address. In the code snippet below you can see that this also implies that we replace the pod selector in the default Kubernetes policies with a endpoint selector.

This is a default deny policy in Cilium.

apiVersion: "cilium.io/v2"
kind: CiliumNetworkPolicy
metadata:
  name: "deny-all-ingress-to-my-app"
spec:
  endpointSelector:
    matchLabels:
      app: my-app
  ingress:
  - {}

As you can see the endpoint selector works with the same matchLabels property as you would define in Kubernetes policies. Something that is very different however is the ingress list property which has an empty object. In the Kubernetes policies it would allow all sources as ingress but here it indicates that no ingress is allowed. That’s a big difference.

To allow an endpoint add the fromEndpoints property to the ingress list like the following example. This policy will allow traffic to flow from the frontend to the backend application.

apiVersion: "cilium.io/v2"
kind: CiliumNetworkPolicy
metadata:
  name: "allow-from-frontend-to-backend"
spec:
  endpointSelector:
    matchLabels:
      app: backend
  ingress:
  - fromEndpoints:
    - matchLabels:
        app: frontend

Namespaces in Cilium

As we said before, there was not mention of namespaces in the different allow or deny capabilities for ingress and egress so far. Cilium does it a little differently. If we take the previous example and adjust it so that our frontend is located in a different namespace we get the following policy.

apiVersion: "cilium.io/v2"
kind: CiliumNetworkPolicy
metadata:
  name: "allow-from-frontend-namespace-to-backend"
spec:
  endpointSelector:
    matchLabels:
      app: backend
  ingress:
  - fromEndpoints:
    - matchLabels:
        k8s:io.kubernetes.pod.namespace: my-frontend-namespace
        app: frontend

In the matchLabels section we can use k8s:io.kubernetes.pod.namespace followed by a namespace you want to target. This will make sure that the parent matchLabels property will only match endpoints in the namespace you specified. You can add addition labels to the matchLabels to further narrow the selector. For instance, the above code will only look for endpoints labeled app: frontend in the my-frontend-namespace namespace.

Other capabilities on OSI Layer 3

In the list of L3 policies you can also find some great quality of life improvements over the Kubernetes network policies. It’s possible to target services and nodes directly. Technically also possible with Kubernetes default policies, however it required you to hard-code ip ranges in the ipBlock section. The below policy includes two ways to target a service, either by name or by label selector. Under the k8sService and k8sServiceSelector we can also specify a namespace property.

apiVersion: "cilium.io/v2"
kind: CiliumNetworkPolicy
metadata:
  name: "allow-frontend-egress-to-service"
spec:
  endpointSelector:
    matchLabels:
      app: frontend
  egress:
  - toServices:
    # Services may be referenced by namespace + name
    - k8sService:
        serviceName: backend
        namespace: another-namespace
    # Services may be referenced by namespace + label selector
    - k8sServiceSelector:
        selector:
          matchLabels:
            app: backend
        namespace: another-namespace

Outside of endpoints (+ namespaces), services and nodes you should also be familiar with Entities which are an easier way of targeting commonly used endpoints via their categorical name so you don’t have to hard-code their IP ranges. For instance using Entities you can allow or deny traffic from and to the local host, kube-apiserver, ingress, cluster or even the public network. A comprehensive list can be found in the documentation.

apiVersion: "cilium.io/v2"
kind: CiliumNetworkPolicy
metadata:
  name: "allow-entities"
spec:
  endpointSelector:
    matchLabels:
      app: backend
  egress:
  - toEntities:
    - kube-apiserver
  ingress:
  - fromEntities:
    - host
    - ingress

Note: this was only the basics L3 capabilities, there is much more to explore on L4 and L7. On the exam you might also be asked to allow some specific ports so do check out the L4 documentation. I’ve not heard of L7 being a requirement.

Mutual Authentication

As part of the policies you will be writing on your CKS exam you might be asked to ensure that a specific rule requires mutual authentication. Unlike traditional TLS which is one-way, mutual authentication in Cilium uses mTLS where both the client and server have to authenticate each other. It is just a simple property to be added to any rule you configured like the example below, yet looking for the right documentation page can lose you a lot of precious time during the exam.

apiVersion: "cilium.io/v2"
kind: CiliumNetworkPolicy
metadata:
  name: "allow-entities"
spec:
  endpointSelector:
    matchLabels:
      app: backend
  ingress:
  - fromEndpoints:
    - matchLabels:
        app: frontend
    authentication:
      mode: "required"

Outro

This was a quick overview of what will be expected of you in terms of Network Policies and Cilium Network Policies on the Certified Kubernetes Security Specialist (CKS) exam. Hope this helps you on your learning journey! The most important thing to do now is practice. Do remember to be very familiar with the documentation so that you can find the necessary details efficiently. Good luck on the exam!

Definitely be on the lookout for the next part in the CKS series. As I primarily work in an Azure context I also want to cover some AKS related topics in the near future like running Cilium on AKS with the Azure CNI powered by Cilium as well as setting up the Container Networking Observability features that were added to AKS recently allowing you to use Cilium Hubble to inspect your network.

So much more to cover! See you in the next one.