LIBXDP

libxdp is a light eBPF library who add 2 features for XDP programs.

• Load multiple programs on single network device using a "dispatcher program" thanks to freplace
• Configuring AF_XDP and functions to read and write on theses sockets

You can check more information on the libxdp readme.

Manage multiple XDP programs

Load and unload

Libxdp can help you to load, attach, unload and manage your XDP program.

• xdp_program__from_bpf_obj
• xdp_program__find_file
• xdp_program__open_file
• xdp_program__from_fd
• xdp_program__from_id
• xdp_program__from_pin

Note

xdp_program__find_file, will search the bpf_object to the path set by LIBXDP_OBJECT_PATH. By default it will be /usr/lib/bpf.

Metadata

XDP program contains metadata to manage programs in the dispatcher.

To modify metadata you can use :

#include <bpf/bpf_helpers.h>
#include <xdp/xdp_helpers.h>

struct {
    __uint(priority, 10);  // priority
    __uint(XDP_PASS, 1);   // chain call action
    __uint(XDP_DROP, 1);   // chain call action
} XDP_RUN_CONFIG(my_xdp_func);

Or by using xdp functions:
This won't modify the BTF file, but these metadata will be stored with the attachment of the program.

Warning

This work only before the attachment of the program to the dispatcher.

• xdp_program__run_prio
• xdp_program__set_run_prio
• xdp_program__chain_call_enabled
• xdp_program__set_chain_call_enabled
• xdp_program__print_chain_call_actions

Priority

The priority of a program is an integer used to determine the order program execution on the interface. Programs are ordered in increasing priority from low to high. For passing packets to next program in the priority, the program should return a one of chain call actions.

Note

The default priority value is 50 if not specified.

Chain call action

Chain call actions are the return codes a program uses to indicate that a packet should continue in the dispatcher program. If a program returns one of these actions, subsequent programs in the call chain will execute. If a different action is returned the processing stop.

Note

By default, this is set to XDP_PASS

The dispatcher program

To support multiple non-offloaded programs on the same network interface, libxdp uses a dispatcher program, a small wrapper that sequentially calls each component program. The dispatcher expects return codes and proceeds to the next program based on the chain call actions of the previous program.

• xdp_multiprog__get_from_ifindex
• xdp_multiprog__next_prog
• xdp_multiprog__close
• xdp_multiprog__detach
• xdp_multiprog__attach_mode
• xdp_multiprog__main_prog
• xdp_multiprog__hw_prog
• xdp_multiprog__is_legacy

XDP dispatcher pinning

The kernel will automatically detach component programs from the dispatcher once their last reference disappears. To prevent this, libxdp pins the component program references in bpffs (BPF file system) before attaching the dispatcher to the network interface. The generated path names for pinning are:

• /sys/fs/bpf/xdp/dispatch-IFINDEX-DID : Dispatcher program for IFINDEX with BPF program ID DID.
• /sys/fs/bpf/xdp/dispatch-IFINDEX-DID/prog0-prog : Component program 0, program reference.
• /sys/fs/bpf/xdp/dispatch-IFINDEX-DID/prog0-link : Component program 0, bpf_link reference.
• /sys/fs/bpf/xdp/dispatch-IFINDEX-DID/prog1-prog : Component program 1, program reference.
• /sys/fs/bpf/xdp/dispatch-IFINDEX-DID/prog1-link : Component program 1, bpf_link reference.

The dispatcher can up max to ten programs If set, the LIBXDP_BPFFS environment variable will override the default location of bpffs, though the xdp subdirectory is always used. If no bpffs is mounted, libxdp will check the LIBXDP_BPFFS_AUTOMOUNT environment variable. If this is set to 1, libxdp will attempt to automount a bpffs. If not set, libxdp will revert to loading a single program without a dispatcher, as if the kernel did not support the features required for multiprog attachment.

AF_XDP sockets

You can find an explanation of the AF_XDP concept here

AF_XDP sockets provide a high-performance mechanism for redirecting network packets to user space. The libxdp library implements helper functions for configuring these sockets and managing packet I/O.

Note

Previously, this functionality was part of libbpf, but it has been moved to libxdp.
As of libbpf 1.0, AF_XDP socket support has been fully transitioned to libxdp.

Control path

Libxdp provides utility functions to help create and manage umems and AF_XDP sockets. You need to create a umem area and then link an AF_XDP socket to it.

Umem Area

it's a memory region designated to store packets. It holds the packets that are received and those that need to be sent. xsk_umem__get_data is used to access the packet data in the umem area. But in unaligned mode, you need to use the three last function

• xsk_umem__create
• xsk_umem__create_with_fd
• xsk_umem__delete
• xsk_umem__fd
• xsk_umem__get_data
• xsk_umem__extract_addr
• xsk_umem__extract_offset
• xsk_umem__add_offset_to_addr

Sockets

Once the umem is created, you can create AF_XDP sockets that are linked to this umem. These sockets can either: * Exclusively own the umem: This is done using the function xsk_socket__create(). * Share the umem with other sockets: This is done using the function xsk_socket__create_shared().

• xsk_socket__create
• xsk_socket__create_shared
• xsk_socket__delete
• xsk_socket__fd
• xsk_setup_xdp_prog
• xsk_socket__update_xskmap

The XSK map is used by the XDP program to manage the mapping between the network interface and the user-space sockets.

Data Path

There are four FIFO rings, categorized into two main types :

• Producer Rings: These include the fill and TX rings, using xsk_ring_prod* functions :

  • Fill ring : Provide buffers to the kernel.
  • TX ring : Send packets.

• Consumer Rings: These include the Rx and completion rings, using xsk_ring_cons* functions :

  • Rx ring : Receive packets from the kernel.
  • Completion ring : Acknowledge completion of transmitted packets.

The producer rings manage the supply of buffers for sending and receiving packets, while the consumer rings manage the recovery and reuse of these buffers after the packets have been processed. You can read more information about the concept in the detailed section AF_XDP in this wiki.

Note

All the data path functions are static inline functions.

Note

To advance to the next entry, simply do idx++.

Producer rings

For producer rings, you start with reserving one or more slots in a producer ring and then when they have been filled out, you submit them so that the kernel will act on them. After this you release them back to the kernel so it can use them for new packets.
Others functions writes entries in the fill and TX rings.

• xsk_ring_prod__reserve
• xsk_ring_prod__submit
• xsk_ring_prod__fill_addr
• xsk_ring_prod__tx_desc
• xsk_ring_prod__needs_wakeup

Consumer rings

For a consumer ring, you peek if there are any new packets in the ring and if so you can read them from the ring. After this you release them back to the kernel so it can use them for new packets There is also a cancel operation for consumer rings if the application does not want to consume all packets received with the peek operation.
Others functions reads entries from the completion and Rx rings.

• xsk_ring_cons__peek
• xsk_ring_cons__cancel
• xsk_ring_cons__release
• xsk_ring_cons__comp_addr
• xsk_ring_cons__rx_desc

Examples

You can find example in the bpf examples repository

Tools

You can use xdp-tools (and bpftool) to help you to build XDP programs.

This doc is written base on the doc provided on libxdp readme