Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement XDP map types #527

Merged
merged 16 commits into from
Sep 25, 2023
Merged

Implement XDP map types #527

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
ec8293a
aya: Implement XDP Map Types
dave-tucker May 30, 2022
e90d521
bpf: Implement XDP maps
dave-tucker May 30, 2022
ede3e91
aya: Update XDP maps implementations
Tuetuopay Aug 4, 2023
ad3087d
bpf: Update XDP maps implementation
Tuetuopay Aug 4, 2023
4452364
macros: add 'map' option to xdp macro
Tuetuopay Aug 4, 2023
139f382
aya: add support for map-bound XDP programs
Tuetuopay Aug 4, 2023
0647927
xdp: add support for chained xdp programs in {cpu,dev}map
Tuetuopay Aug 4, 2023
f7fbbcd
aya: fix docstring missing trailing period
Tuetuopay Aug 4, 2023
db49633
bpf: make xdp maps functions safe
Tuetuopay Aug 4, 2023
9ed1d3d
bpf: add documentation for XDP maps
Tuetuopay Aug 4, 2023
c6754c6
maps/xdp: use ProgramFd instead of impl AsRawFd
Tuetuopay Aug 4, 2023
63ce2f0
bpf/devmap: don't expose `bpf_devmap_value`
Tuetuopay Aug 4, 2023
00dc7a5
maps/xdp: make maps work on kernels not supporting ProgIds
Tuetuopay Aug 4, 2023
46551de
xtask: bless public-api
Tuetuopay Aug 17, 2023
579e3ce
aya, bpf: misc fixes following review comments
Tuetuopay Sep 18, 2023
0edc13b
bpf: add a shared try_redirect_map function for XDP maps
Tuetuopay Sep 24, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions aya-bpf-macros/src/lib.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -128,6 +128,28 @@ pub fn sk_msg(attrs: TokenStream, item: TokenStream) -> TokenStream {
}
}

/// Marks a function as an eBPF XDP program that can be attached to a network interface.
///
/// On some NIC drivers, XDP probes are compatible with jumbo frames through the use of
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment is confusing. You get jumbo frames even without frags, they're just
always linearised

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

uuuh no, unless I'm mistaken, you cannot attach an xdp program not marked frags on an interface where the mtu is greater than page size - headroom. I've only managed to do it on mellanox connectx-6 hardware with a vendor-specific flag turned on with ethtool and the frags option. afaik only mlx5 and i40e have support for frags (and i40e was non-functional last time I tried)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think so? Maybe it depends on the attach mode? On aws I've definitely
worked with 9001 bytes large frames with regular (no frags) xdp programs

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AFAIK you can attach a non-frags program to a jumbo frame-enabled interface just fine - where jumbo frames mean MTU set to anything with MTU > 1500. Without frags support, you only get up to a page of data.
I'll double-check this next week though.

Copy link
Contributor Author

@Tuetuopay Tuetuopay Sep 22, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can as long as the jumbo frame + xdp headroom fits in a page (4096 bytes):

ebpf:

#![no_std]
#![no_main]

use aya_bpf::{bindings::xdp_action::XDP_PASS, macros::xdp, programs::XdpContext};

#[xdp] fn xdp_no_frags(_ctx: XdpContext) -> u32 { XDP_PASS }
#[xdp(frags)] fn xdp_with_frags(_ctx: XdpContext) -> u32 { XDP_PASS }

then the userspace either one or the other depending on the --frags flag:

use std::convert::TryInto;

use aya::{Bpf, include_bytes_aligned, programs::{Xdp, XdpFlags}};
use anyhow::Context;
use structopt::StructOpt;

fn main() {
    if let Err(e) = try_main() { eprintln!("error: {:#}", e); }
}

#[derive(Debug, StructOpt)]
struct Opt {
    #[structopt(short, long, default_value = "eth0")] iface: String,
    #[structopt(short, long)] frags: bool,
}

fn try_main() -> Result<(), anyhow::Error> {
    let opt = Opt::from_args();
    let mut bpf = Bpf::load(include_bytes_aligned!(
        "../../target/bpfel-unknown-none/release/aya-test"
    ))?;

    let name = if opt.frags { "xdp_with_frags" } else { "xdp_no_frags" };
    let program: &mut Xdp = bpf.program_mut(name).unwrap().try_into()?;
    program.load()?;
    program.attach(&opt.iface, XdpFlags::DRV_MODE).context("failed to attach XDP program in driver mode")?;

    println!("loaded xdp program {name}");

    Ok(())
}

testing on a connectx-6 dx nic:

root@cx6:~# ip l show dev enp23s0np0
4: enp23s0np0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000
    link/ether b8:ce:f6:bc:cb:4c brd ff:ff:ff:ff:ff:ff
root@cx6:~# ./aya-test -i enp23s0np0
loaded xdp program xdp_no_frags
root@cx6:~# ip l set dev enp23s0np0 mtu 3000 # still fits in a page
root@cx6:~# ./aya-test -i enp23s0np0
loaded xdp program xdp_no_frags
root@cx6:~# ip l set dev enp23s0np0 mtu 4000
root@cx6:~# ./aya-test -i enp23s0np0
error: failed to attach XDP program in driver mode: `bpf_link_create` failed: Invalid argument (os error 22)
root@cx6:~# ip l set dev enp23s0np0 mtu 9000
root@cx6:~# ./aya-test -i enp23s0np0
error: failed to attach XDP program in driver mode: `bpf_link_create` failed: Invalid argument (os error 22)
root@cx6:~# ./aya-test -i enp23s0np0 --frags
loaded xdp program xdp_with_frags

so yes, with jumbo being "anything above > 1500 bytes", frags are not always required. however, this is the strict definition of jumbo, while the usual definition is mtu 9k, in which case frags are definitely always required. this comes from the fact that, for performance reasons, xdp contexts including payload and head/tailroom is required to fit in a single page. packets thus get fragmented with pointers to other frags if it does not fit in a single page. the kernel requires this opt-in as the packet is not linear anymore, and the program needs some bpf helpers to access the payload further.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think so? Maybe it depends on the attach mode? On aws I've definitely worked with 9001 bytes large frames with regular (no frags) xdp programs

ha missed this part. yes skb mode does not have this restriction, and will linearize the data. driver mode however is another beast, and to get the most of your hardware is required. on virtualized stuff it does not really matter, because unless the host does nic vf + pcie passthrough, it's quite likely it already generated a sk_buff for the packet, and passed it directly to the gest because virtio. (dunno about ena tho)

/// multi-buffer packets. Programs can opt-in this support by passing the `frags` argument.
///
/// XDP programs can also be chained through the use of CPU maps and dev maps, but must opt-in
/// with the `map = "cpumap"` or `map = "devmap"` arguments.
///
/// # Minimum kernel version
///
/// The minimum kernel version required to use this feature is 4.8.
///
/// # Examples
///
/// ```no_run
/// use aya_bpf::{bindings::xdp_action::XDP_PASS, macros::xdp, programs::XdpContext};
///
/// #[xdp(frags)]
/// pub fn xdp(ctx: XdpContext) -> u32 {
/// XDP_PASS
/// }
/// ```
#[proc_macro_error]
#[proc_macro_attribute]
pub fn xdp(attrs: TokenStream, item: TokenStream) -> TokenStream {
Expand Down
157 changes: 148 additions & 9 deletions aya-bpf-macros/src/xdp.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,31 +1,52 @@
use std::borrow::Cow;

use proc_macro2::TokenStream;
use quote::quote;
use syn::{ItemFn, Result};
use syn::{Error, ItemFn, Result};

use crate::args::{err_on_unknown_args, pop_bool_arg, Args};
use crate::args::{err_on_unknown_args, pop_bool_arg, pop_string_arg, Args};

pub(crate) struct Xdp {
item: ItemFn,
frags: bool,
map: Option<XdpMap>,
}

#[derive(Clone, Copy)]
pub(crate) enum XdpMap {
CpuMap,
DevMap,
}

impl Xdp {
pub(crate) fn parse(attrs: TokenStream, item: TokenStream) -> Result<Xdp> {
let item = syn::parse2(item)?;
let mut args: Args = syn::parse2(attrs)?;

let frags = pop_bool_arg(&mut args, "frags");
let map = match pop_string_arg(&mut args, "map").as_deref() {
Some("cpumap") => Some(XdpMap::CpuMap),
Some("devmap") => Some(XdpMap::DevMap),
Some(name) => {
return Err(Error::new_spanned(
"map",
format!("Invalid value. Expected 'cpumap' or 'devmap', found '{name}'"),
))
}
None => None,
};

err_on_unknown_args(&args)?;
Ok(Xdp { item, frags })
Ok(Xdp { item, frags, map })
}

pub(crate) fn expand(&self) -> Result<TokenStream> {
let section_name: Cow<'_, _> = if self.frags {
"xdp.frags".into()
} else {
"xdp".into()
let mut section_name = vec![if self.frags { "xdp.frags" } else { "xdp" }];
match self.map {
Some(XdpMap::CpuMap) => section_name.push("cpumap"),
Some(XdpMap::DevMap) => section_name.push("devmap"),
None => (),
};
let section_name = section_name.join("/");

let fn_vis = &self.item.vis;
let fn_name = self.item.sig.ident.clone();
let item = &self.item;
Expand Down Expand Up @@ -97,4 +118,122 @@ mod tests {
};
assert_eq!(expected.to_string(), expanded.to_string());
}

#[test]
fn test_xdp_cpumap() {
let prog = Xdp::parse(
parse_quote! { map = "cpumap" },
parse_quote! {
fn prog(ctx: &mut ::aya_bpf::programs::XdpContext) -> i32 {
0
}
},
)
.unwrap();
let expanded = prog.expand().unwrap();
let expected = quote! {
#[no_mangle]
#[link_section = "xdp/cpumap"]
fn prog(ctx: *mut ::aya_bpf::bindings::xdp_md) -> u32 {
return prog(::aya_bpf::programs::XdpContext::new(ctx));

fn prog(ctx: &mut ::aya_bpf::programs::XdpContext) -> i32 {
0
}
}
};
assert_eq!(expected.to_string(), expanded.to_string());
}

#[test]
fn test_xdp_devmap() {
let prog = Xdp::parse(
parse_quote! { map = "devmap" },
parse_quote! {
fn prog(ctx: &mut ::aya_bpf::programs::XdpContext) -> i32 {
0
}
},
)
.unwrap();
let expanded = prog.expand().unwrap();
let expected = quote! {
#[no_mangle]
#[link_section = "xdp/devmap"]
fn prog(ctx: *mut ::aya_bpf::bindings::xdp_md) -> u32 {
return prog(::aya_bpf::programs::XdpContext::new(ctx));

fn prog(ctx: &mut ::aya_bpf::programs::XdpContext) -> i32 {
0
}
}
};
assert_eq!(expected.to_string(), expanded.to_string());
}

#[test]
#[should_panic(expected = "Invalid value. Expected 'cpumap' or 'devmap', found 'badmap'")]
fn test_xdp_bad_map() {
Xdp::parse(
parse_quote! { map = "badmap" },
parse_quote! {
fn prog(ctx: &mut ::aya_bpf::programs::XdpContext) -> i32 {
0
}
},
)
.unwrap();
}

#[test]
fn test_xdp_frags_cpumap() {
let prog = Xdp::parse(
parse_quote! { frags, map = "cpumap" },
parse_quote! {
fn prog(ctx: &mut ::aya_bpf::programs::XdpContext) -> i32 {
0
}
},
)
.unwrap();
let expanded = prog.expand().unwrap();
let expected = quote! {
#[no_mangle]
#[link_section = "xdp.frags/cpumap"]
fn prog(ctx: *mut ::aya_bpf::bindings::xdp_md) -> u32 {
return prog(::aya_bpf::programs::XdpContext::new(ctx));

fn prog(ctx: &mut ::aya_bpf::programs::XdpContext) -> i32 {
0
}
}
};
assert_eq!(expected.to_string(), expanded.to_string());
}

#[test]
fn test_xdp_frags_devmap() {
let prog = Xdp::parse(
parse_quote! { frags, map = "devmap" },
parse_quote! {
fn prog(ctx: &mut ::aya_bpf::programs::XdpContext) -> i32 {
0
}
},
)
.unwrap();
let expanded = prog.expand().unwrap();
let expected = quote! {
#[no_mangle]
#[link_section = "xdp.frags/devmap"]
fn prog(ctx: *mut ::aya_bpf::bindings::xdp_md) -> u32 {
return prog(::aya_bpf::programs::XdpContext::new(ctx));

fn prog(ctx: &mut ::aya_bpf::programs::XdpContext) -> i32 {
0
}
}
};
assert_eq!(expected.to_string(), expanded.to_string());
}
}
8 changes: 8 additions & 0 deletions aya-obj/src/maps.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -176,6 +176,14 @@ impl Map {
}
}

/// Set the value size in bytes
pub fn set_value_size(&mut self, size: u32) {
match self {
Map::Legacy(m) => m.def.value_size = size,
Map::Btf(m) => m.def.value_size = size,
}
}

/// Returns the max entry number
pub fn max_entries(&self) -> u32 {
match self {
Expand Down
40 changes: 34 additions & 6 deletions aya-obj/src/obj.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ use crate::{
btf::BtfFeatures,
generated::{BPF_CALL, BPF_JMP, BPF_K},
maps::{BtfMap, LegacyMap, Map, MINIMUM_MAP_SIZE},
programs::XdpAttachType,
relocation::*,
util::HashMap,
};
Expand Down Expand Up @@ -47,17 +48,22 @@ pub struct Features {
bpf_perf_link: bool,
bpf_global_data: bool,
bpf_cookie: bool,
cpumap_prog_id: bool,
alessandrod marked this conversation as resolved.
Show resolved Hide resolved
devmap_prog_id: bool,
btf: Option<BtfFeatures>,
}

impl Features {
#[doc(hidden)]
#[allow(clippy::too_many_arguments)]
pub fn new(
bpf_name: bool,
bpf_probe_read_kernel: bool,
bpf_perf_link: bool,
bpf_global_data: bool,
bpf_cookie: bool,
cpumap_prog_id: bool,
devmap_prog_id: bool,
btf: Option<BtfFeatures>,
) -> Self {
Self {
Expand All @@ -66,6 +72,8 @@ impl Features {
bpf_perf_link,
bpf_global_data,
bpf_cookie,
cpumap_prog_id,
devmap_prog_id,
btf,
}
}
Expand Down Expand Up @@ -95,6 +103,16 @@ impl Features {
self.bpf_cookie
}

/// Returns whether XDP CPU Maps support chained program IDs.
pub fn cpumap_prog_id(&self) -> bool {
self.cpumap_prog_id
}

/// Returns whether XDP Device Maps support chained program IDs.
pub fn devmap_prog_id(&self) -> bool {
self.devmap_prog_id
}

/// If BTF is supported, returns which BTF features are supported.
pub fn btf(&self) -> Option<&BtfFeatures> {
self.btf.as_ref()
Expand Down Expand Up @@ -204,8 +222,6 @@ pub struct Function {
/// - `struct_ops+`
/// - `fmod_ret+`, `fmod_ret.s+`
/// - `iter+`, `iter.s+`
/// - `xdp.frags/cpumap`, `xdp/cpumap`
/// - `xdp.frags/devmap`, `xdp/devmap`
#[derive(Debug, Clone)]
#[allow(missing_docs)]
pub enum ProgramSection {
Expand All @@ -221,6 +237,7 @@ pub enum ProgramSection {
SocketFilter,
Xdp {
frags: bool,
attach_type: XdpAttachType,
},
SkMsg,
SkSkbStreamParser,
Expand Down Expand Up @@ -283,8 +300,19 @@ impl FromStr for ProgramSection {
"uprobe.s" => UProbe { sleepable: true },
"uretprobe" => URetProbe { sleepable: false },
"uretprobe.s" => URetProbe { sleepable: true },
"xdp" => Xdp { frags: false },
"xdp.frags" => Xdp { frags: true },
"xdp" | "xdp.frags" => Xdp {
frags: kind == "xdp.frags",
attach_type: match pieces.next() {
None => XdpAttachType::Interface,
Some("cpumap") => XdpAttachType::CpuMap,
Some("devmap") => XdpAttachType::DevMap,
Some(_) => {
return Err(ParseError::InvalidProgramSection {
section: section.to_owned(),
})
}
},
},
"tp_btf" => BtfTracePoint,
"tracepoint" | "tp" => TracePoint,
"socket" => SocketFilter,
Expand Down Expand Up @@ -2012,7 +2040,7 @@ mod tests {
assert_matches!(
obj.parse_section(fake_section(
BpfSectionKind::Program,
"xdp/foo",
"xdp",
bytes_of(&fake_ins()),
None
)),
Expand All @@ -2035,7 +2063,7 @@ mod tests {
assert_matches!(
obj.parse_section(fake_section(
BpfSectionKind::Program,
"xdp.frags/foo",
"xdp.frags",
bytes_of(&fake_ins()),
None
)),
Expand Down
2 changes: 2 additions & 0 deletions aya-obj/src/programs/mod.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,9 @@
pub mod cgroup_sock;
pub mod cgroup_sock_addr;
pub mod cgroup_sockopt;
pub mod xdp;

pub use cgroup_sock::CgroupSockAttachType;
pub use cgroup_sock_addr::CgroupSockAddrAttachType;
pub use cgroup_sockopt::CgroupSockoptAttachType;
pub use xdp::XdpAttachType;
24 changes: 24 additions & 0 deletions aya-obj/src/programs/xdp.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
//! XDP programs.

use crate::generated::bpf_attach_type;

/// Defines where to attach an `XDP` program.
#[derive(Copy, Clone, Debug)]
pub enum XdpAttachType {
/// Attach to a network interface.
Interface,
/// Attach to a cpumap. Requires kernel 5.9 or later.
CpuMap,
/// Attach to a devmap. Requires kernel 5.8 or later.
DevMap,
}

impl From<XdpAttachType> for bpf_attach_type {
fn from(value: XdpAttachType) -> Self {
match value {
XdpAttachType::Interface => bpf_attach_type::BPF_XDP,
XdpAttachType::CpuMap => bpf_attach_type::BPF_XDP_CPUMAP,
XdpAttachType::DevMap => bpf_attach_type::BPF_XDP_DEVMAP,
}
}
}
2 changes: 1 addition & 1 deletion aya/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ edition = "2021"
rust-version = "1.66"

[dependencies]
assert_matches = { workspace = true }
async-io = { workspace = true, optional = true }
aya-obj = { workspace = true, features = ["std"] }
bitflags = { workspace = true }
Expand All @@ -28,7 +29,6 @@ thiserror = { workspace = true }
tokio = { workspace = true, features = ["rt"], optional = true }

[dev-dependencies]
assert_matches = { workspace = true }
tempfile = { workspace = true }

[features]
Expand Down
Loading