---
title: "Enable eBPF on an existing cluster"
description: "Switch a running Calico Enterprise cluster to the eBPF data plane on an existing installation as an alternative to the iptables data plane."
product: "Calico Enterprise"
version: "3.23 (latest)"
section: "Operations"
canonical_url: "https://docs.tigera.io/calico-enterprise/latest/operations/ebpf/enabling-ebpf"
---

# Enable eBPF on an existing cluster

## Big picture

Enable the eBPF data plane on an existing cluster.

## Value

The eBPF data plane mode has several advantages over standard Linux networking pipeline mode:

- It scales to higher throughput.

- It uses less CPU per GBit.

- It has native support for Kubernetes services (without needing kube-proxy) that:

  - Reduces first packet latency for packets to services.
  - Preserves external client source IP addresses all the way to the pod.
  - Supports DSR (Direct Server Return) for more efficient service routing.
  - Uses less CPU than kube-proxy to keep the data plane in sync.

To learn more and see performance metrics from our test environment, see the blog, [Introducing the Calico eBPF data plane](https://www.projectcalico.org/introducing-the-calico-ebpf-dataplane/).

## Concepts

### eBPF

eBPF (or "extended Berkeley Packet Filter"), is a technology that allows safe mini programs to be attached to various low-level hooks in the Linux kernel. eBPF has a wide variety of uses, including networking, security, and tracing. You’ll see a lot of non-networking projects leveraging eBPF, but for Calico Enterprise our focus is on networking, and in particular, pushing the networking capabilities of the latest Linux kernels to the limit.

## Before you begin

**Required**

- Review [Requirements, limitations, and performance](https://docs.tigera.io/calico-enterprise/latest/operations/ebpf/install.md#before-you-begin)

## How to

- [Verify that your cluster is ready for eBPF mode](#verify-that-your-cluster-is-ready-for-ebpf-mode)
- [Configure Calico Enterprise to talk directly to the API server](#configure-calico-enterprise-to-talk-directly-to-the-api-server)
- [Configure kube-proxy](#configure-kube-proxy)
- [Enable eBPF mode](#enable-ebpf-mode)
- [Try out DSR mode](#try-out-direct-server-return-mode)
- [Reversing the process](#reversing-the-process)

### Verify that your cluster is ready for eBPF mode

This section explains how to make sure your cluster is suitable for eBPF mode.

To check that the kernel on a node is suitable, you can run

```bash
uname -rv
```

The output should look like this:

```text
5.10.0-26-generic #28~20.04.1-Ubuntu SMP Fri Jan 27 14:30:10 UTC 2023
```

In this case the kernel version is v5.10, which is suitable.

On Red Hat-derived distributions, you may see something like this:

```text
4.18.0-305.el8.x86_64 (mockbuild@x86-vm-08.build.eng.bos.redhat.com)
```

Since the Red Hat kernel is v4.18 with at least build number 305 (RHEL 8.4), this kernel is suitable.

### Configure Calico Enterprise to talk directly to the API server

In eBPF mode, Calico Enterprise implements Kubernetes service networking directly (rather than relying on `kube-proxy`). Of course, this makes it highly desirable to disable `kube-proxy` when running in eBPF mode to save resources and avoid confusion over which component is handling services.

To be able to disable `kube-proxy`, Calico Enterprise needs to communicate to the API server *directly* rather than going through `kube-proxy`. To make *that* possible, we need to find a persistent, static way to reach the API server. The best way to do that varies by Kubernetes distribution:

- If you created a cluster manually (for example by using `kubeadm`) then the right address to use depends on whether you opted for a high-availability cluster with multiple API servers or a simple one-node API server.

  - If you opted to set up a high availability cluster then you should use the address of the load balancer that you used in front of your API servers. As noted in the Kubernetes documentation, a load balancer is required for a HA set-up but the precise type of load balancer is not specified.
  - If you opted for a single control plane node then you can use the address of the control plane node itself. However, it's important that you use a *stable* address for that node such as a dedicated DNS record, or a static IP address. If you use a dynamic IP address (such as an EC2 private IP) then the address may change when the node is restarted causing Calico Enterprise to lose connectivity to the API server.

- `kops` typically sets up a load balancer of some sort in front of the API server. You should use the FQDN and port of the API load balancer, for example `api.internal.<clustername>` as the `KUBERNETES_SERVICE_HOST` below and 443 as the `KUBERNETES_SERVICE_PORT`.

- OpenShift requires various DNS records to be created for the cluster; one of these is exactly what we need: `api-int.<cluster_name>.<base_domain>` should point to the API server or to the load balancer in front of the API server. Use that (filling in the `<cluster_name>` and `<base_domain>` as appropriate for your cluster) for the `KUBERNETES_SERVICE_HOST` below. OpenShift uses 6443 for the `KUBERNETES_SERVICE_PORT`.

- MKE runs a reverse proxy in each node that can be used to reach the API server. You should use `proxy.local` as the `KUBERNETES_SERVICE_HOST` and `6444` as the `KUBERNETES_SERVICE_PORT`.

- For AKS and EKS clusters you should use the FQDN of the API server's load balancer. This can be found with

  ```text
  kubectl cluster-info
  ```

  which gives output like the following:

  ```text
  Kubernetes master is running at https://60F939227672BC3D5A1B3EC9744B2B21.gr7.us-west-2.eks.amazonaws.com
  ...
  ```

  In this example, you would use `60F939227672BC3D5A1B3EC9744B2B21.gr7.us-west-2.eks.amazonaws.com` for `KUBERNETES_SERVICE_HOST` and `443` for `KUBERNETES_SERVICE_PORT` when creating the config map.

Once you've found the correct address for your API server, create the following config map in the `tigera-operator` namespace using the host and port that you found above:

```yaml
kind: ConfigMap
apiVersion: v1
metadata:
  name: kubernetes-services-endpoint
  namespace: tigera-operator
data:
  KUBERNETES_SERVICE_HOST: '<API server host>'
  KUBERNETES_SERVICE_PORT: '<API server port>'
```

The operator will pick up the change to the config map automatically and do a rolling update of Calico Enterprise to pass on the change. Confirm that pods restart and then reach the `Running` state with the following command:

```bash
watch kubectl get pods -n calico-system
```

If you do not see the pods restart then it's possible that the `ConfigMap` wasn't picked up (sometimes Kubernetes is slow to propagate `ConfigMap`s (see Kubernetes [issue #30189](https://github.com/kubernetes/kubernetes/issues/30189))). You can try restarting the operator.

### Configure kube-proxy

In eBPF mode Calico Enterprise replaces `kube-proxy` so it wastes resources (and reduces performance) to run both.\
This section explains how to disable `kube-proxy` in some common environments.

> **WARNING:** If you are running `kube-proxy` in IPVS mode, you must switch it to iptables mode before disabling `kube-proxy` or enabling eBPF mode. This applies to all platforms and is required for a successful migration. After switching to iptables mode, restart your nodes before proceeding.

#### Clusters that run `kube-proxy` with a `DaemonSet` (such as `kubeadm`)

For a cluster that runs `kube-proxy` in a `DaemonSet` (such as a `kubeadm`-created cluster), you can disable `kube-proxy` reversibly by adding a node selector to `kube-proxy`'s `DaemonSet` that matches no nodes, for example:

```bash
kubectl patch ds -n kube-system kube-proxy -p '{"spec":{"template":{"spec":{"nodeSelector":{"non-calico": "true"}}}}}'
```

Then, should you want to start `kube-proxy` again, you can simply remove the node selector.

> **SECONDARY:** This approach is not suitable for AKS with Azure CNI since that platform makes use of the Kubernetes add-on manager. the change will be reverted by the system. For AKS, you should follow [Avoiding conflicts with kube-proxy](#avoiding-conflicts-with-kube-proxy) below.

#### OpenShift

If you are running OpenShift, you can disable `kube-proxy` as follows:

```bash
kubectl patch networks.operator.openshift.io cluster --type merge -p '{"spec":{"deployKubeProxy": false}}'
```

To re-enable it:

```bash
kubectl patch networks.operator.openshift.io cluster --type merge -p '{"spec":{"deployKubeProxy": true}}'
```

#### MKE

If you are running MKE, you can disable `kube-proxy` as follows:

Follow the step procedure in [Modify an existing MKE configuration](https://docs.mirantis.com/mke/current/ops/administer-cluster/configure-an-mke-cluster/use-an-mke-configuration-file.html#modify-an-existing-mke-configuration) to download, edit, and upload your MKE configuration. During the editing step, add the following configuration: `kube_proxy_mode=disabled` and `kube_default_drop_masq_bits=true`.

### Avoiding conflicts with kube-proxy

If you cannot disable `kube-proxy` (for example, because it is managed by your Kubernetes distribution), then you *must* change Felix configuration parameter `BPFKubeProxyIptablesCleanupEnabled` to `false`. This can be done with `kubectl` as follows:

```bash
kubectl patch felixconfiguration default --patch='{"spec": {"bpfKubeProxyIptablesCleanupEnabled": false}}'
```

If both `kube-proxy` and `BPFKubeProxyIptablesCleanupEnabled` is enabled then `kube-proxy` will write its iptables rules and Felix will try to clean them up resulting in iptables flapping between the two.

You should also set `bpfKubeProxyHealthzPort` to `0` to disable the health check server in Calico Enterprise's BPF kube-proxy replacement, which by default binds to port 10256 and would conflict with the Kubernetes `kube-proxy` already running on the node. The Kubernetes `kube-proxy` can serve the health check equally well, so there is no degradation.

```bash
kubectl patch felixconfiguration default --patch='{"spec": {"bpfKubeProxyHealthzPort": 0}}'
```

### MKE: Change the VXLAN port before enabling eBPF

> **WARNING:** MKE uses Docker Swarm overlay networking, which creates VXLAN devices on UDP port 4789 inside Docker network namespaces. When Calico Enterprise switches to eBPF mode on kernels where BTF is available (typically v5.8+), Felix creates the `vxlan.calico` device in flow mode, which acts as a catch-all on its UDP port. The kernel rejects this when another VXLAN device (Docker Swarm's) already holds the same port, causing `vxlan.calico` to stay DOWN with `address already in use` errors.
>
> **You must change the VXLAN port before enabling eBPF on MKE clusters:**
>
> ```bash
> kubectl patch felixconfiguration default --type merge -p '{"spec":{"vxlanPort":4790}}'
> ```
>
> Wait for all calico-node pods to recreate the VXLAN device on the new port, then verify on each node:
>
> ```bash
> kubectl exec -n calico-system <calico-node-pod> -- ip -d link show vxlan.calico
> ```
>
> Confirm the device shows `dstport 4790` (or your chosen port) and is UP before proceeding. Ensure that the chosen UDP port is allowed by your underlying network between all nodes.

### Enable eBPF mode

To enable eBPF mode, change the `spec.calicoNetwork.linuxDataplane` parameter in the operator's `Installation` resource to `"BPF"`.

```bash
kubectl patch installation.operator.tigera.io default --type merge -p '{"spec":{"calicoNetwork":{"linuxDataplane":"BPF"}}}'
```

When enabling eBPF mode, preexisting connections continue to use the non-BPF datapath; such connections should not be disrupted, but they do not benefit from eBPF mode’s advantages.

> **SECONDARY:** The operator rolls out the change with a rolling update (non-disruptive) and then swiftly transitions all nodes to eBPF mode. However, it's inevitable that some nodes will enter eBPF mode before others. This can disrupt the flow of traffic through node ports.

### Try out direct server return mode

Direct server return (DSR) mode skips a hop through the network for traffic to services (such as node ports) from outside the cluster. This reduces latency and CPU overhead but it requires the underlying network to allow nodes to send traffic with each other's IPs.

In AWS, this requires all your nodes to be in the same subnet and for the source/dest check to be disabled. In GCP, the source/dest check should also be disabled, which can be done by enabling IP forwarding.

DSR mode is disabled by default; to enable it, set the `BPFExternalServiceMode` Felix configuration parameter to `"DSR"`. This can be done with `kubectl`:

```bash
kubectl patch felixconfiguration default --patch='{"spec": {"bpfExternalServiceMode": "DSR"}}'
```

To switch back to tunneled mode, set the configuration parameter to `"Tunnel"`:

```bash
kubectl patch felixconfiguration default --patch='{"spec": {"bpfExternalServiceMode": "Tunnel"}}'
```

Switching external traffic mode can disrupt in-progress connections.

### Reversing the process

To revert to standard Linux networking:

1. Reverse the changes to the operator's `Installation`:

   ```bash
   kubectl patch installation.operator.tigera.io default --type merge -p '{"spec":{"calicoNetwork":{"linuxDataplane":"Iptables"}}}'
   ```

2. If you disabled `kube-proxy`, re-enable it (for example, by removing the node selector added above).

   ```bash
   kubectl patch ds -n kube-system kube-proxy --type merge -p '{"spec":{"template":{"spec":{"nodeSelector":{"non-calico": null}}}}}'
   ```

3. If you are running MKE, follow the step procedure in [Modify an existing MKE configuration](https://docs.mirantis.com/mke/current/ops/administer-cluster/configure-an-mke-cluster/use-an-mke-configuration-file.html#modify-an-existing-mke-configuration) to download, edit, and upload your MKE configuration. During the editing step, add the following configuration: `kube_proxy_mode` to `iptables`.

4. Since disabling eBPF mode is disruptive to existing connections, monitor existing workloads to make sure they re-establish any connections that were disrupted by the switch.