-
Notifications
You must be signed in to change notification settings - Fork 110
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add documentation for Wireguard Threaded NAPI feature
Docs for feature PR: projectcalico/calico#9260
- Loading branch information
Showing
3 changed files
with
74 additions
and
3 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -54,6 +54,7 @@ enables encrypting IPv4 traffic over an IPv4 underlay network and `wireguardEnab | |
| - [Install WireGuard](#install-wireguard) | ||
| - [Enable WireGuard for a cluster](#enable-wireguard-for-a-cluster) | ||
| - [Verify encryption is enabled](#verify-encryption-is-enabled) | ||
| - [Enable WireGuard Threaded NAPI for a cluster](#enable-wireguard-threaded-napi-for-a-cluster) | ||
| - [Disable WireGuard for an individual node](#disable-wireguard-for-an-individual-node) | ||
| - [Disable WireGuard for a cluster](#disable-wireguard-for-a-cluster) | ||
|
|
||
|
|
@@ -199,6 +200,28 @@ To view WireGuard statistics in Manager UI, you must enable them. From the left | |
|
|
||
|  | ||
|
|
||
| ### Enable WireGuard Threaded NAPI for a cluster | ||
|
|
||
| :::warning | ||
|
|
||
| There is a [known issue](https://lore.kernel.org/netdev/CALrw=nEoT2emQ0OAYCjM1d_6Xe_kNLSZ6dhjb5FxrLFYh4kozA@mail.gmail.com/T/) with WireGuard threaded NAPI that may cause NAPI to get stuck holding the global `rtnl_mutex` when a peer is removed (most likely during a node reboot when peers are removed). | ||
| Kernels which include [this patch](https://lore.kernel.org/netdev/[email protected]/) are able to recover after a node drain. | ||
| This feature should only be considered if you have high packets per second workloads that are causing dropping packets due to a saturated `softirq` CPU core. | ||
|
|
||
| ::: | ||
|
|
||
| WireGuard Threaded NAPI increases the maximum packets per second that a WireGuard interface can process by enabling [Threaded NAPI](https://docs.kernel.org/networking/napi.html#threaded-napi) on the WireGuard interfaces. | ||
|
|
||
| You can read more about its performance improvements [here](https://netdevconf.info/0x18/docs/netdev-0x18-paper23-talk-paper.pdf). | ||
|
|
||
| Enable Threaded NAPI WireGuard across all the nodes using the following command. | ||
|
|
||
| ```bash | ||
| kubectl patch felixconfiguration default --type='merge' -p '{"spec":{"wireguardThreadingEnabled":true}}' | ||
| ``` | ||
|
|
||
| The above command enables Threaded NAPI for both the IPv4 and IPv6 WireGuard interfaces. | ||
|
|
||
| ### Disable WireGuard for an individual node | ||
|
|
||
| To disable WireGuard on a specific node with WireGuard installed, modify the node-specific Felix configuration. e.g., to turn off encryption for pod traffic on node `my-node`, use the following command. This command disables WireGuard for both IPv4 and IPv6, modify it accordingly if disabling only either IP version: | ||
|
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -25,14 +25,15 @@ When this feature is enabled, {{prodname}} automatically creates and manages Wir | |
|
|
||
| - `wireguardEnabled` - enables encrypting IPv4 traffic over an IPv4 underlay network | ||
| - `wireguardEnabledV6` - enables encrypting IPv6 traffic over an IPv6 underlay network | ||
| - `wireguardThreadingEnabled` - enables [threaded NAPI](https://docs.kernel.org/networking/napi.html#threaded-napi) on the WireGuard interfaces for IPv4 and IPv6 | ||
|
|
||
| ## Before you begin | ||
|
|
||
| **Terminology** | ||
|
|
||
| - Inter-node pod traffic: Traffic leaving a pod from one node destined to a pod on another node | ||
| - Inter-node, host-network traffic: traffic generated by the node itself or a host-networked-pod destined to another node or host-networked-pod | ||
| - Same-node pod traffic: Traffic between pods on the same node | ||
| - Inter-node pod traffic: Traffic leaving a pod from one node destined to a pod on another node | ||
| - Inter-node, host-network traffic: traffic generated by the node itself or a host-networked-pod destined to another node or host-networked-pod | ||
| - Same-node pod traffic: Traffic between pods on the same node | ||
|
|
||
| **Supported encryption** | ||
|
|
||
|
|
@@ -64,6 +65,7 @@ When this feature is enabled, {{prodname}} automatically creates and manages Wir | |
| - [Verify encryption is enabled](#verify-encryption-is-enabled) | ||
| - [Enable WireGuard statistics](#enable-wireguard-statistics) | ||
| - [View WireGuard statistics](#view-wireguard-statistics) | ||
| - [Enable WireGuard Threaded NAPI for a cluster](#enable-wireguard-threaded-napi-for-a-cluster) | ||
| - [Disable WireGuard for an individual node](#disable-wireguard-for-an-individual-node) | ||
| - [Disable WireGuard for a cluster](#disable-wireguard-for-a-cluster) | ||
|
|
||
|
|
@@ -211,6 +213,28 @@ When viewing WireGuard statistics, you might wonder why the charts in the Manage | |
| - Packet loss. If a node resends a lost packet, then the node counts the packet twice where the receiver counts it only once. | ||
| - Averaging/smoothing. The statistics are smoothed out over a few seconds. | ||
|
|
||
| ### Enable WireGuard Threaded NAPI for a cluster | ||
|
|
||
| :::warning | ||
|
|
||
| There is a [known issue](https://lore.kernel.org/netdev/CALrw=nEoT2emQ0OAYCjM1d_6Xe_kNLSZ6dhjb5FxrLFYh4kozA@mail.gmail.com/T/) with WireGuard threaded NAPI that may cause NAPI to get stuck holding the global `rtnl_mutex` when a peer is removed (most likely during a node reboot when peers are removed). | ||
| Kernels which include [this patch](https://lore.kernel.org/netdev/[email protected]/) are able to recover after a node drain. | ||
| This feature should only be considered if you have high packets per second workloads that are causing dropping packets due to a saturated `softirq` CPU core. | ||
|
|
||
| ::: | ||
|
|
||
| WireGuard Threaded NAPI increases the maximum packets per second that a WireGuard interface can process by enabling [Threaded NAPI](https://docs.kernel.org/networking/napi.html#threaded-napi) on the WireGuard interfaces. | ||
|
|
||
| You can read more about its performance improvements [here](https://netdevconf.info/0x18/docs/netdev-0x18-paper23-talk-paper.pdf). | ||
|
|
||
| Enable Threaded NAPI WireGuard across all the nodes using the following command. | ||
|
|
||
| ```bash | ||
| kubectl patch felixconfiguration default --type='merge' -p '{"spec":{"wireguardThreadingEnabled":true}}' | ||
| ``` | ||
|
|
||
| The above command enables Threaded NAPI for both the IPv4 and IPv6 WireGuard interfaces. | ||
|
|
||
| ### Disable WireGuard for an individual node | ||
|
|
||
| To disable WireGuard on a specific node with WireGuard installed, modify the node-specific Felix configuration. e.g., to turn off encryption for pod traffic on node `my-node`, use the following command. This command disables WireGuard for both IPv4 and IPv6, modify it accordingly if disabling only either IP version: | ||
|
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -25,6 +25,7 @@ When this feature is enabled, {{prodname}} automatically creates and manages Wir | |
|
|
||
| - `wireguardEnabled` - enables encrypting IPv4 traffic over an IPv4 underlay network | ||
| - `wireguardEnabledV6` - enables encrypting IPv6 traffic over an IPv6 underlay network | ||
| - `wireguardThreadingEnabled` - enables [threaded NAPI](https://docs.kernel.org/networking/napi.html#threaded-napi) on the WireGuard interfaces for IPv4 and IPv6 | ||
|
|
||
| ## Before you begin... | ||
|
|
||
|
|
@@ -63,6 +64,7 @@ When this feature is enabled, {{prodname}} automatically creates and manages Wir | |
| - [Enable WireGuard for a cluster](#enable-wireguard-for-a-cluster) | ||
| - [Disable WireGuard for an individual node](#disable-wireguard-for-an-individual-node) | ||
| - [Verify configuration](#verify-configuration) | ||
| - [Enable WireGuard Threaded NAPI for a cluster](#enable-wireguard-threaded-napi-for-a-cluster) | ||
| - [Disable WireGuard for a cluster](#disable-wireguard-for-a-cluster) | ||
|
|
||
| ### Install WireGuard | ||
|
|
@@ -216,6 +218,28 @@ The above command can be used to change other WireGuard attributes. For a list o | |
|
|
||
| We recommend that you review and modify the MTU used by {{prodname}} networking when WireGuard is enabled to increase network performance. Follow the instructions in the [Configure MTU to maximize network performance](../networking/configuring/mtu.mdx) guide to set the MTU to a value appropriate for your network. | ||
|
|
||
| ### Enable WireGuard Threaded NAPI for a cluster | ||
|
|
||
| :::warning | ||
|
|
||
| There is a [known issue](https://lore.kernel.org/netdev/CALrw=nEoT2emQ0OAYCjM1d_6Xe_kNLSZ6dhjb5FxrLFYh4kozA@mail.gmail.com/T/) with WireGuard threaded NAPI that may cause NAPI to get stuck holding the global `rtnl_mutex` when a peer is removed (most likely during a node reboot when peers are removed). | ||
| Kernels which include [this patch](https://lore.kernel.org/netdev/[email protected]/) are able to recover after a node drain. | ||
| This feature should only be considered if you have high packets per second workloads that are causing dropping packets due to a saturated `softirq` CPU core. | ||
|
|
||
| ::: | ||
|
|
||
| WireGuard Threaded NAPI increases the maximum packets per second that a WireGuard interface can process by enabling [Threaded NAPI](https://docs.kernel.org/networking/napi.html#threaded-napi) on the WireGuard interfaces. | ||
|
|
||
| You can read more about its performance improvements [here](https://netdevconf.info/0x18/docs/netdev-0x18-paper23-talk-paper.pdf). | ||
|
|
||
| Enable Threaded NAPI WireGuard across all the nodes using the following command. | ||
|
|
||
| ```bash | ||
| kubectl patch felixconfiguration default --type='merge' -p '{"spec":{"wireguardThreadingEnabled":true}}' | ||
| ``` | ||
|
|
||
| The above command enables Threaded NAPI for both the IPv4 and IPv6 WireGuard interfaces. | ||
|
|
||
| ### Disable WireGuard for an individual node | ||
|
|
||
| To disable WireGuard on a specific node with WireGuard installed, modify the node-specific Felix configuration. e.g., to turn off encryption for traffic on node `my-node`, use the following command. This command disables WireGuard for both IPv4 and IPv6, modify it accordingly if disabling only either IP version: | ||
|
|
||