Skip to content

Functional Block: CPS

Qiaobin Fu edited this page Jan 24, 2020 · 22 revisions

The Control Plane Services (CPS) block has the responsibility of handling all control plane operations for Gatekeeper and Grantor servers. This includes passing any control plane protocol messages to routing daemons running on the host, and accepting routing tables updates from the routing daemons.

Table of Contents

Description

Although Gatekeeper servers' main role is only in the data plane, they still need to peer in vantage points and with internal routers of the AS in order to become a node in the ingoing and outgoing network paths.

Instead of adding support for individual control plane protocols like BGP, OSPF, and IS-IS in Gatekeeper, we enable network operators to run control plane daemons on the host. This is a win-win scenario, as Gatekeeper avoids the trouble of supporting these protocols and network operators do not need to learn another tool to manage their routes.

The control plane component accomplishes its goal by steering packets from control plane protocols received on the front and back interfaces and sending them to Kernel NIC Interfaces (KNIs). Routing daemons can receive the control plane packets on KNIs, update and compute new routes, and send them back to Gatekeeper through the KNI as well.

The CPS block is allocated one lcore in Gatekeeper.

Static Configuration

All static configuration variables can be configured in lua/cps.lua.

Variables to Change for Basic Operation

These variables are likely to change from deployment-to-deployment based on the operator's preferences.

Log Level

log_level

The log level for the CPS block. Can be set to any one of the following values: RTE_LOG_EMERG, RTE_LOG_ALERT, RTE_LOG_CRIT, RTE_LOG_ERR, RTE_LOG_WARNING, RTE_LOG_NOTICE, RTE_LOG_INFO, RTE_LOG_DEBUG.

Since we typically use RTE_LOG_ERROR as the most severe log condition, we recommend not to set this value below RTE_LOG_ERROR.

Variables to Change for Performance Reasons

It is not crucial to change these variables, and they only need to be changed to fine tune the performance of Gatekeeper. Otherwise, the default values are likely fine.

Mailbox Maximum Entries (Exponential)

mailbox_max_entries_exp

The log (base 2) of the maximum size of the CPS mailbox. For example, if the variable is set to 7, then room for 2^7 = 128 entries will be made in the mailbox.

Also used to determine how many entries will actually be available for use in the mailbox, which for efficiency reasons is one less than the maximum size of the mailbox (127 in the example above).

Mailbox Cache Size

mailbox_mem_cache_size

Number of mailbox entries to keep in the cache for more efficient use of the mailbox. Set to 0 to disable the cache of the memory pool for the mailbox.

Mailbox Burst Size

mailbox_burst_size

Maximum number of entries to receive in a burst every time the mailbox is checked.

Log Rate Limit Interval

log_ratelimit_interval_ms

The interval at which logs are rate limited (in milliseconds). For a given interval, only log_ratelimit_burst log entries are permitted. The count of entries is reset for each new interval.

Log Rate Limit Burst

log_ratelimit_burst

The number of entries per interval allowed to be logged. When the number of log entries exceeds this limit in a given interval, the entries will be dropped.

Interface Maximum Packet Burst (Front and Back)

front_max_pkt_burst & back_max_pkt_burst

Maximum number of packets received in each burst on the front and back interfaces, respectively.

The CPS block only receives bursts of packets directly from the interfaces when ntuple filters are available in hardware. Otherwise, packets relevant to the CPS block are received through the CPS mailbox and these variables are not used.

ARP Maximum Entries (Exponential)

arp_max_entries_exp

The log (base 2) of the maximum size of the ARP memory pool. For example, if the variable is set to 10, then room for 2^10 = 1024 entries will be made in the memory pool.

Also used to determine how many entries will actually be available for use in the memory pool, which for efficiency reasons is one less than the maximum size of the memory pool (1023 in the example above).

ND Maximum Entries (Exponential)

nd_max_entries_exp

The log (base 2) of the maximum size of the ND memory pool. For example, if the variable is set to 10, then room for 2^10 = 1024 entries will be made in the memory pool.

Also used to determine how many entries will actually be available for use in the memory pool, which for efficiency reasons is one less than the maximum size of the memory pool (1023 in the example above).

Variables Unlikely to Change

These variables likely only need to be changed under extreme circumstances or for deployment-specific reasons.

TCP Port for BGP

tcp_bgp_port

The port number where a Border Gateway Protocol (BGP) speaker is listening on a TCP connection. Packets received by Gatekeeper destined for this TCP port (either via IPv4 or IPv6) are directed through the CPS block to the BGP speaker.

The standard value for the BGP port is 179, but in theory by proper configuration of the participating BGP speakers it could be different.

Number of Attempts to Get KNI Link

num_attempts_kni_link_set

The number of attempts to try to set the KNIs up or down. Because doing so requires a callback through the kernel, and because bringing these interfaces up and down may take time, this can sometimes take multiple attempts. A reasonable value is 5.

Maximum Route Updates

max_rt_update_pkts

Maximum number of Netlink route update packets from the routing daemon to serve in a batch. In normal operation there won't often be many route updates at once. A reasonable value is 8.

IP Address Resolution Scan Interval

scan_interval_sec

The time in seconds between scans of outstanding IP address resolution requests. We keep track of which IP address resolution requests the KNI has made, and occasionally scan this list to remove outstanding requests that were not answered. A reasonable value is 5.

Path of rte_kni Kernel Module

kni_kmod_path

The KNI mechanism in DPDK requires the support of a kernel module. If not specified (default), the system automatically searches for the kernel module in /lib/modules/.

If specified, this variable defines where in the filesystem the rte_kni.ko kernel module file is found. It should be an absolute (not relative) path, such as "/home/user/gatekeeper/dependencies/dpdk/build/kmod/rte_kni.ko". Specifying a particular "rte_kni.ko" file in this way could be useful while working to port Gatekeeper to a newer version of DPDK or patching the file rte_kni.ko.

Clone this wiki locally