Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Datapath overhaul: zero-copy metadata with ingot, 'Compiled' UFTs #585

Merged
merged 117 commits into from
Nov 19, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
117 commits
Select commit Hold shift + click to select a range
8fa90c8
Bump zerocopy to 0.8 prerelease, begin the Great Work
FelixMcFelix Aug 19, 2024
3d9eb6f
Iterating...
FelixMcFelix Aug 19, 2024
cdea9f4
The most hacked-together 'fast-path' imaginable.
FelixMcFelix Aug 21, 2024
ff0fa3e
Merge branch 'master' into ingot
FelixMcFelix Aug 21, 2024
7ba068b
Merge conflict errors.
FelixMcFelix Aug 21, 2024
759b337
Attempt to minimise fastpath lock contention.
FelixMcFelix Aug 21, 2024
7b5100a
Nudge along CI a little bit
FelixMcFelix Aug 21, 2024
17b5c02
Wow, that was wasteful.
FelixMcFelix Aug 21, 2024
b905c8c
...CI?
FelixMcFelix Aug 21, 2024
470f2bc
...CI??
FelixMcFelix Aug 21, 2024
6345ba6
Don't recompute flowhash for UFTs
FelixMcFelix Aug 23, 2024
fb4f9c1
Reintroduce headroom for ETH alignment.
FelixMcFelix Aug 23, 2024
6200a29
Test fixups, post-V6EH world.
FelixMcFelix Aug 29, 2024
f1346b6
Try out emit_uninit.
FelixMcFelix Aug 30, 2024
c4be1cd
Long march to complete integration... in progress.
FelixMcFelix Sep 4, 2024
9a359cc
Good spot to call it for the day.
FelixMcFelix Sep 4, 2024
d417e69
Iterating.
FelixMcFelix Sep 5, 2024
4aabcb0
We're now past OPTE::engine, at least.
FelixMcFelix Sep 5, 2024
750c8ac
Lazy workarounds to get back to later.
FelixMcFelix Sep 5, 2024
c1a1658
Against the odds, XDE compiles.
FelixMcFelix Sep 6, 2024
f73b532
Unbreak DHCPv4 responses.
FelixMcFelix Sep 6, 2024
640b962
Works, but I am dropping perf on the floor somewhere now.
FelixMcFelix Sep 9, 2024
ea26bbd
Actually remember to use cached l4 hash.
FelixMcFelix Sep 9, 2024
3e6794e
Temporarily re-add thin process.
FelixMcFelix Sep 10, 2024
f88fe1b
Re-enable cksum update when needed.
FelixMcFelix Sep 10, 2024
ac79b20
Start work on removing all the insane `into`s, everywhere.
FelixMcFelix Sep 12, 2024
89d6845
Cleanup the last tranche of needless intos
FelixMcFelix Sep 12, 2024
942cdc1
Take v6 next layer fixes, attempt to chase down bad csum updates.
FelixMcFelix Sep 12, 2024
d1dd000
Some extra constraints, in case we can make Ptr storage work.
FelixMcFelix Sep 13, 2024
c6ab7f2
Pull in latest ingot -- 2.8Gbps w/o faster path
FelixMcFelix Sep 13, 2024
7123158
Bump up to latest.
FelixMcFelix Sep 25, 2024
42e1604
Early plumbing for stage1/light meta
FelixMcFelix Sep 25, 2024
88f2377
Sketching out the fasterpath. Not yet working.
FelixMcFelix Sep 26, 2024
afbc587
Speed loss due to copy-in, let's rethink.
FelixMcFelix Sep 27, 2024
7164eaa
Whoops.
FelixMcFelix Sep 27, 2024
2ebc377
3Gbps intra-VPC, over wire.
FelixMcFelix Sep 27, 2024
a6da8e7
Pre-save pushed headers in Compile -- 3.1Gbps.
FelixMcFelix Sep 27, 2024
9035440
Remove thin process.
FelixMcFelix Sep 27, 2024
70dab2a
Tricksy ~~hobbits~~ SDT probes.
FelixMcFelix Sep 27, 2024
80d8a72
With new error handling!
FelixMcFelix Oct 8, 2024
110d22e
As I rewrite a test suite...
FelixMcFelix Oct 9, 2024
0ea48db
Post--review-feedback on ingot.
FelixMcFelix Oct 10, 2024
3d34ae3
Progress.
FelixMcFelix Oct 11, 2024
572c169
More progress...
FelixMcFelix Oct 11, 2024
3ee9f18
More progress on the 'it compiles, at least'.
FelixMcFelix Oct 11, 2024
2811546
Need a new nightly + some tweaks.
FelixMcFelix Oct 13, 2024
05b2e5b
One or two checksum bugs still to unearth, then TCP state.
FelixMcFelix Oct 14, 2024
82bc861
One more.
FelixMcFelix Oct 14, 2024
e752e3c
Iterating.
FelixMcFelix Oct 15, 2024
9cb1006
Merge branch 'master' into ingot
FelixMcFelix Oct 15, 2024
cd7d1a5
Arc<> up FlowEntries.
FelixMcFelix Oct 16, 2024
1a9212a
Reworked TCP flow tracking is GO.
FelixMcFelix Oct 16, 2024
a3af0f5
Tests are green (!!!!!!!!!!!!!!!!!!)
FelixMcFelix Oct 16, 2024
10c3bb4
??
FelixMcFelix Oct 16, 2024
103d2b9
Auto-fix warnings.
FelixMcFelix Oct 17, 2024
b174395
Tweaks.
FelixMcFelix Oct 17, 2024
49680e1
Strip dead code, fill out the `todo!`s
FelixMcFelix Oct 18, 2024
7ced0c6
The Axe Shall Fall (pt.1)
FelixMcFelix Oct 19, 2024
496a25c
The Axe Shall Fall (pt.2)
FelixMcFelix Oct 19, 2024
c7be20c
The Axe Shall Fall (pt.3)
FelixMcFelix Oct 21, 2024
9615362
Some warnings. More test migration to come.
FelixMcFelix Oct 21, 2024
48f19f2
Cleaning up names
FelixMcFelix Oct 22, 2024
445d8c0
Fixup ubench.
FelixMcFelix Oct 22, 2024
d27c77c
Wherein I push code around.
FelixMcFelix Oct 23, 2024
06584e3
More movement, reclaim some validation we didn't get via ingot.
FelixMcFelix Oct 23, 2024
90c79b5
ExternalOpt critical checking, as req'd by RFC 8926
FelixMcFelix Oct 24, 2024
bf667f4
Maybe actually do the Geneve validation.
FelixMcFelix Oct 24, 2024
6e9ed60
Wherein I Moev Things.
FelixMcFelix Oct 25, 2024
d187bbe
A test suite.
FelixMcFelix Oct 28, 2024
5294b99
One more test to reintroduce.
FelixMcFelix Oct 28, 2024
cec6ebe
More Movement of Stuff.
FelixMcFelix Oct 28, 2024
af6bbe8
More tweaks.
FelixMcFelix Oct 28, 2024
e14bf22
Fix ubench.
FelixMcFelix Oct 28, 2024
d5fa92e
Fixup port entry/return probes
FelixMcFelix Oct 28, 2024
fbc661d
Cleanup, find meaningful TODOs.
FelixMcFelix Oct 29, 2024
7698a31
Finally do away with ingot_packet.rs
FelixMcFelix Oct 29, 2024
852cfd4
Licensing, docs fixes.
FelixMcFelix Oct 29, 2024
a60f598
An import.
FelixMcFelix Oct 29, 2024
de099e8
Again.
FelixMcFelix Oct 29, 2024
3fcedc9
Thanks, clippy. Thippy.
FelixMcFelix Oct 29, 2024
54219da
Clippy appeasement at last.
FelixMcFelix Oct 29, 2024
f7e64e5
Fixup fuzzers.
FelixMcFelix Oct 30, 2024
22a0ede
Self-review pt.1
FelixMcFelix Oct 30, 2024
40c8840
Self review pt.2 -- less copies on gateway ping
FelixMcFelix Oct 30, 2024
f0070f9
Self review pt.4
FelixMcFelix Oct 30, 2024
603f916
SR Pt.5
FelixMcFelix Oct 30, 2024
3dc3a59
SR Pt.6
FelixMcFelix Oct 30, 2024
cda3564
SR pt.7
FelixMcFelix Oct 30, 2024
956779d
..
FelixMcFelix Oct 30, 2024
84d8651
Bump ingot.
FelixMcFelix Oct 31, 2024
aac32dc
SR pt.8
FelixMcFelix Oct 31, 2024
bd7641a
Self-review pt.9
FelixMcFelix Oct 31, 2024
c3ebd68
SR Pt.9
FelixMcFelix Oct 31, 2024
5328994
Indicate fast/slowpath in port-process-return
FelixMcFelix Oct 31, 2024
9fa4856
Inline to prevent port-process-return from dinging us.
FelixMcFelix Oct 31, 2024
d682ea0
SR Pt.9
FelixMcFelix Nov 1, 2024
cdf1d59
Remove vestigial Packet<Initialized>.
FelixMcFelix Nov 1, 2024
5e290a3
Whoops.
FelixMcFelix Nov 1, 2024
9393ad6
Update position on UFT compilation in the architecture doc
FelixMcFelix Nov 1, 2024
a4648b4
Mark parse errors in mod-level kstat.
FelixMcFelix Nov 4, 2024
230e3c7
Fuzzers, accidentally a file.
FelixMcFelix Nov 4, 2024
5afd63a
Missed doctests.
FelixMcFelix Nov 4, 2024
d8ab666
Review feedback -- de-`pub` inner, de-`inner`.
FelixMcFelix Nov 7, 2024
470ed8a
Document the UDP zero-checksum rationale.
FelixMcFelix Nov 7, 2024
5f53d2c
Review feedback: document mblk refcnt assumptions.
FelixMcFelix Nov 7, 2024
2e800b3
Ingot repo is opened now 🎉
FelixMcFelix Nov 7, 2024
72a116a
Review feedback: pullup packet bodies beyond header MsgBlks
FelixMcFelix Nov 8, 2024
69e844c
Fixup body transform + pullup interaction.
FelixMcFelix Nov 8, 2024
ac44642
Fix chain truncation.
FelixMcFelix Nov 8, 2024
b386de8
Inline fastpath/lightweight parsing paths
FelixMcFelix Nov 8, 2024
27ecc8d
Review feedback.
FelixMcFelix Nov 11, 2024
d7c8b21
Review feedback: copy `db_struioun` on pullup and chain trim
FelixMcFelix Nov 11, 2024
33137dd
All ptrs.
FelixMcFelix Nov 11, 2024
569db38
Fmt.
FelixMcFelix Nov 11, 2024
6ac41b8
Bump to merged ingot, one nit.
FelixMcFelix Nov 19, 2024
540e396
Bump nightly compiler, fix clippy.
FelixMcFelix Nov 19, 2024
9801755
Pick up open Renovate PRs.
FelixMcFelix Nov 19, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/buildomat/jobs/opte-api.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
#: name = "opte-api"
#: variety = "basic"
#: target = "helios-2.0"
#: rust_toolchain = "nightly-2024-05-12"
#: rust_toolchain = "nightly-2024-11-18"
#: output_rules = []
#:

Expand All @@ -24,7 +24,7 @@ header "check API_VERSION"
./check-api-version.sh

header "check style"
ptime -m cargo +nightly-2024-05-12 fmt -- --check
ptime -m cargo +nightly-2024-11-18 fmt -- --check

header "analyze std"
ptime -m cargo clippy --all-targets
Expand Down
4 changes: 2 additions & 2 deletions .github/buildomat/jobs/opte-ioctl.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
#: name = "opte-ioctl"
#: variety = "basic"
#: target = "helios-2.0"
#: rust_toolchain = "nightly-2024-05-12"
#: rust_toolchain = "nightly-2024-11-18"
#: output_rules = []
#:

Expand All @@ -21,7 +21,7 @@ rustc --version
cd lib/opte-ioctl

header "check style"
ptime -m cargo +nightly-2024-05-12 fmt -- --check
ptime -m cargo +nightly-2024-11-18 fmt -- --check

header "analyze"
ptime -m cargo clippy --all-targets
8 changes: 4 additions & 4 deletions .github/buildomat/jobs/opte.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
#: name = "opte"
#: variety = "basic"
#: target = "helios-2.0"
#: rust_toolchain = "nightly-2024-05-12"
#: rust_toolchain = "nightly-2024-11-18"
#: output_rules = []
#:

Expand All @@ -21,7 +21,7 @@ rustc --version
cd lib/opte

header "check style"
ptime -m cargo +nightly-2024-05-12 fmt -- --check
ptime -m cargo +nightly-2024-11-18 fmt -- --check

header "check docs"
#
Expand All @@ -30,13 +30,13 @@ header "check docs"
#
# Use nightly which is needed for the `kernel` feature.
RUSTDOCFLAGS="-D warnings" ptime -m \
cargo +nightly-2024-05-12 doc --no-default-features --features=api,std,engine,kernel
cargo +nightly-2024-11-18 doc --no-default-features --features=api,std,engine,kernel

header "analyze std + api"
ptime -m cargo clippy --all-targets

header "analyze no_std + engine + kernel"
ptime -m cargo +nightly-2024-05-12 clippy --no-default-features --features engine,kernel
ptime -m cargo +nightly-2024-11-18 clippy --no-default-features --features engine,kernel

header "test"
ptime -m cargo test
4 changes: 2 additions & 2 deletions .github/buildomat/jobs/opteadm.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
#: name = "opteadm"
#: variety = "basic"
#: target = "helios-2.0"
#: rust_toolchain = "nightly-2024-05-12"
#: rust_toolchain = "nightly-2024-11-18"
#: output_rules = [
#: "=/work/debug/opteadm",
#: "=/work/debug/opteadm.debug.sha256",
Expand All @@ -30,7 +30,7 @@ rustc --version
pushd bin/opteadm

header "check style"
ptime -m cargo +nightly-2024-05-12 fmt -- --check
ptime -m cargo +nightly-2024-11-18 fmt -- --check

header "analyze"
ptime -m cargo clippy --all-targets
Expand Down
8 changes: 4 additions & 4 deletions .github/buildomat/jobs/oxide-vpc.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
#: name = "oxide-vpc"
#: variety = "basic"
#: target = "helios-2.0"
#: rust_toolchain = "nightly-2024-05-12"
#: rust_toolchain = "nightly-2024-11-18"
#: output_rules = []
#:

Expand All @@ -21,7 +21,7 @@ rustc --version
cd lib/oxide-vpc

header "check style"
ptime -m cargo +nightly-2024-05-12 fmt -- --check
ptime -m cargo +nightly-2024-11-18 fmt -- --check

header "check docs"
#
Expand All @@ -30,13 +30,13 @@ header "check docs"
#
# Use nightly which is needed for the `kernel` feature.
RUSTDOCFLAGS="-D warnings" ptime -m \
cargo +nightly-2024-05-12 doc --no-default-features --features=api,std,engine,kernel
cargo +nightly-2024-11-18 doc --no-default-features --features=api,std,engine,kernel

header "analyze std + api + usdt"
ptime -m cargo clippy --features usdt --all-targets

header "analyze no_std + engine + kernel"
ptime -m cargo +nightly-2024-05-12 clippy --no-default-features --features engine,kernel
ptime -m cargo +nightly-2024-11-18 clippy --no-default-features --features engine,kernel

header "test"
ptime -m cargo test
2 changes: 1 addition & 1 deletion .github/buildomat/jobs/p5p.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
#: name = "opte-p5p"
#: variety = "basic"
#: target = "helios-2.0"
#: rust_toolchain = "nightly-2024-05-12"
#: rust_toolchain = "nightly-2024-11-18"
#: output_rules = [
#: "=/out/opte.p5p",
#: "=/out/opte.p5p.sha256",
Expand Down
6 changes: 3 additions & 3 deletions .github/buildomat/jobs/xde.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
#: name = "opte-xde"
#: variety = "basic"
#: target = "helios-2.0"
#: rust_toolchain = "nightly-2024-05-12"
#: rust_toolchain = "nightly-2024-11-18"
#: output_rules = [
#: "=/work/debug/xde.dbg",
#: "=/work/debug/xde.dbg.sha256",
Expand Down Expand Up @@ -75,7 +75,7 @@ pushd xde
cp xde.conf /work/xde.conf

header "check style"
ptime -m cargo +nightly-2024-05-12 fmt -p xde -p xde-link -- --check
ptime -m cargo +nightly-2024-11-18 fmt -p xde -p xde-link -- --check

header "analyze"
ptime -m cargo clippy -- \
Expand Down Expand Up @@ -123,7 +123,7 @@ sha256sum $REL_TGT/xde_link.so > $REL_TGT/xde_link.so.sha256

header "build xde integration tests"
pushd xde-tests
cargo +nightly-2024-05-12 fmt -- --check
cargo +nightly-2024-11-18 fmt -- --check
cargo clippy --all-targets
cargo build --test loopback
loopback_test=$(
Expand Down
Loading