Loxilb is a networking solution that utilizes eBPF, leverage eBPF to modify the packet processing rules as they traverse the network stack in the Linux kernel. This can be particularly useful for network behavior without altering existing applications or the operating system itself.
loxilb generates two primary object files during build:
/opt/loxilb/llb_ebpf_main.o # TC layer processing (305KB)
/opt/loxilb/llb_xdp_main.o # XDP layer processing (95KB)
Feature | TC eBPF | XDP |
---|---|---|
Packet Format | Socket Buffer (skb) | XDP Frame Format |
Processing Level | L2-L7 | Primarily L2 |
eBPF (extended Berkeley Packet Filter) maps are essential data structures within the Linux kernel that facilitate efficient storage and sharing of data between eBPF programs and user-space applications.
They function as key-value stores, enabling eBPF programs to maintain state across multiple invocations and allowing communication between the kernel and user space.
LoxiLB utilizes eBPF maps to store and manage various data structures essential for packet processing. These maps are pinned to the filesystem, allowing for persistent storage and easy access. Commonly used maps include:
Interface Maps: Store information about network interfaces.
Connection Tracking Maps: Maintain state information for active connections.
NAT Maps: Handle Network Address Translation entries.
Policy Maps: Store security and routing policies.
Then I will detail analysis of loxilb's end-to-end packet processing pipeline and Load Balancer based on this architecture diagram.
An eBPF tail call is a mechanism that allows one eBPF program to call another eBPF program without returning to the original program. It's like a "jump" instruction that transfers control completely to another program and have benefit in:
Fast Path (pipe1)and Slow Path (pipe2), these two pipelines address the need for optimizing different types of packet processing tasks based on their complexity and requirements.
[Incoming Packet]
↓
[Parse Packet Headers]
↓
[Connection Lookup]─────Yes──┐
↓ ↓
[Is Established?] [Process & Forward (Fast Path)]
↓ │
No │
↓ │
[Tail Call to Slow Path] │
│ │
└──────────────────────┘
### TCP Example:
CLOSED → SYN_SENT → SYN_RECEIVED → ESTABLISHED
│ │
│ ▼
[Slow Path Processing] [Move to Fast Path]
### UDP Example:
NEW → SEEN → ESTABLISHED
│ │
│ ▼
[Slow Path] [Move to Fast Path]
kernel/llb_kern_entry.c
[Incoming Packet] → [packet parsing]
↓
[intf map] - Interface configuration lookup
- Interface index
- VLAN information
- Zone information
- Policy parameters
dp_parse_d0
), the packet parsing module extracts key metadata such as:
xfi
structure, which guides further processing.intf_map
.kernel/llb_kern_policer.c
[QoS Stage]
↓
[pol map] - Policy lookup
- Traffic policing rules
- Rate limiting
- Traffic shaping
↓
[qos map] - QoS parameters
- Priority queues
- Bandwidth allocation
pol_map
: Defines access control and prioritization policies.qos_map
: Ensures compliance with Quality of Service requirements (e.g., rate shaping).kernel/llb_kern_l2fwd.c
l2fwd
(Layer 2 forwarding)l2tunfwd
(Layer 2 tunneling forwarding)smac_map
: Source MAC addressdmac_map
: Destination MAC addressrmac_map
: Router MAC addressl2fwd
(MAC-based forwarding)[l2fwd Stage]
↓
[smac map] - Source MAC processing
- MAC learning
- Source validation
↓
[dmac map] - Destination MAC processing
- MAC lookup
- L2 forwarding decision
l2tunfwd
(tunneling at Layer 2)[l2tunfwd Stage]
- VXLAN processing
- NVGRE processing
- Other L2 tunnel protocols
kernel/llb_kern_l3fwd.c
dp_ing_l3()
: Entry point for L3 ingress processing.dp_l3_fwd()
: Main function for Layer 3 forwarding.dp_do_rtv4()
and dp_do_rtv6()
: Handle IPv4 and IPv6 route lookups and apply forwarding actions.dp_do_ctops()
: Handles connection tracking and NAT for packets.ct_map
: Connection tracking table for stateful flow handling.rt_v4_map
: IPv4 routing table.rt_v6_map
: IPv6 routing table.LL_DP_RTV4_STATS_MAP
and LL_DP_RTV6_STATS_MAP
: Statistics for routes.kernel/llb_kern_ct.c
Provides connection tracking infrastructure
Connection State Tracking:
NAT Support Infrastructure:
kernel/llb_kern_natlbfwd.c
:Network Address Translation (NAT) and Load Balancer forwarding implementation
if (act->sel_type == NAT_LB_SEL_RR) {
bpf_spin_lock(&act->lock);
i = act->sel_hint;
// Iterate through endpoints
while (n < LLB_MAX_NXFRMS) {
if (nxfrm_act->inactive == 0) {
// Select next backend in rotation
act->sel_hint = (i + 1) % LLB_MAX_NXFRMS;
sel = i;
break;
}
}
}
if (act->sel_type == NAT_LB_SEL_HASH) {
// Hash packet for backend selection
sel = dp_get_pkt_hash(ctx) % act->nxfrm;
// Fallback if selected backend is inactive
if (act->nxfrms[sel].inactive) {
for (i = 0; i < LLB_MAX_NXFRMS; i++) {
if (act->nxfrms[i].inactive == 0) {
sel = i;
break;
}
}
}
}
if (act->sel_type == NAT_LB_SEL_LC) {
struct dp_nat_epacts *epa;
__u32 lc = 0;
// Find backend with least active connections
for (i = 0; i < LLB_MAX_NXFRMS; i++) {
if (nxfrm_act->inactive == 0) {
__u32 as = epa->active_sess[i];
if (lc > as || sel < 0) {
sel = i;
lc = as;
}
}
}
}
Hello, I'm William Lin. I'd like to share my excitement about being a member of the free5gc project, which is a part of the Linux Foundation. I'm always eager to discuss any aspects of core network development or related technologies.