-
Notifications
You must be signed in to change notification settings - Fork 9
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Datapath overhaul: zero-copy metadata with ingot, 'Compiled' UFTs #585
Merged
Merged
Changes from 109 commits
Commits
Show all changes
117 commits
Select commit
Hold shift + click to select a range
8fa90c8
Bump zerocopy to 0.8 prerelease, begin the Great Work
FelixMcFelix 3d9eb6f
Iterating...
FelixMcFelix cdea9f4
The most hacked-together 'fast-path' imaginable.
FelixMcFelix ff0fa3e
Merge branch 'master' into ingot
FelixMcFelix 7ba068b
Merge conflict errors.
FelixMcFelix 759b337
Attempt to minimise fastpath lock contention.
FelixMcFelix 7b5100a
Nudge along CI a little bit
FelixMcFelix 17b5c02
Wow, that was wasteful.
FelixMcFelix b905c8c
...CI?
FelixMcFelix 470f2bc
...CI??
FelixMcFelix 6345ba6
Don't recompute flowhash for UFTs
FelixMcFelix fb4f9c1
Reintroduce headroom for ETH alignment.
FelixMcFelix 6200a29
Test fixups, post-V6EH world.
FelixMcFelix f1346b6
Try out emit_uninit.
FelixMcFelix c4be1cd
Long march to complete integration... in progress.
FelixMcFelix 9a359cc
Good spot to call it for the day.
FelixMcFelix d417e69
Iterating.
FelixMcFelix 4aabcb0
We're now past OPTE::engine, at least.
FelixMcFelix 750c8ac
Lazy workarounds to get back to later.
FelixMcFelix c1a1658
Against the odds, XDE compiles.
FelixMcFelix f73b532
Unbreak DHCPv4 responses.
FelixMcFelix 640b962
Works, but I am dropping perf on the floor somewhere now.
FelixMcFelix ea26bbd
Actually remember to use cached l4 hash.
FelixMcFelix 3e6794e
Temporarily re-add thin process.
FelixMcFelix f88fe1b
Re-enable cksum update when needed.
FelixMcFelix ac79b20
Start work on removing all the insane `into`s, everywhere.
FelixMcFelix 89d6845
Cleanup the last tranche of needless intos
FelixMcFelix 942cdc1
Take v6 next layer fixes, attempt to chase down bad csum updates.
FelixMcFelix d1dd000
Some extra constraints, in case we can make Ptr storage work.
FelixMcFelix c6ab7f2
Pull in latest ingot -- 2.8Gbps w/o faster path
FelixMcFelix 7123158
Bump up to latest.
FelixMcFelix 42e1604
Early plumbing for stage1/light meta
FelixMcFelix 88f2377
Sketching out the fasterpath. Not yet working.
FelixMcFelix afbc587
Speed loss due to copy-in, let's rethink.
FelixMcFelix 7164eaa
Whoops.
FelixMcFelix 2ebc377
3Gbps intra-VPC, over wire.
FelixMcFelix a6da8e7
Pre-save pushed headers in Compile -- 3.1Gbps.
FelixMcFelix 9035440
Remove thin process.
FelixMcFelix 70dab2a
Tricksy ~~hobbits~~ SDT probes.
FelixMcFelix 80d8a72
With new error handling!
FelixMcFelix 110d22e
As I rewrite a test suite...
FelixMcFelix 0ea48db
Post--review-feedback on ingot.
FelixMcFelix 3d34ae3
Progress.
FelixMcFelix 572c169
More progress...
FelixMcFelix 3ee9f18
More progress on the 'it compiles, at least'.
FelixMcFelix 2811546
Need a new nightly + some tweaks.
FelixMcFelix 05b2e5b
One or two checksum bugs still to unearth, then TCP state.
FelixMcFelix 82bc861
One more.
FelixMcFelix e752e3c
Iterating.
FelixMcFelix 9cb1006
Merge branch 'master' into ingot
FelixMcFelix cd7d1a5
Arc<> up FlowEntries.
FelixMcFelix 1a9212a
Reworked TCP flow tracking is GO.
FelixMcFelix a3af0f5
Tests are green (!!!!!!!!!!!!!!!!!!)
FelixMcFelix 10c3bb4
??
FelixMcFelix 103d2b9
Auto-fix warnings.
FelixMcFelix b174395
Tweaks.
FelixMcFelix 49680e1
Strip dead code, fill out the `todo!`s
FelixMcFelix 7ced0c6
The Axe Shall Fall (pt.1)
FelixMcFelix 496a25c
The Axe Shall Fall (pt.2)
FelixMcFelix c7be20c
The Axe Shall Fall (pt.3)
FelixMcFelix 9615362
Some warnings. More test migration to come.
FelixMcFelix 48f19f2
Cleaning up names
FelixMcFelix 445d8c0
Fixup ubench.
FelixMcFelix d27c77c
Wherein I push code around.
FelixMcFelix 06584e3
More movement, reclaim some validation we didn't get via ingot.
FelixMcFelix 90c79b5
ExternalOpt critical checking, as req'd by RFC 8926
FelixMcFelix bf667f4
Maybe actually do the Geneve validation.
FelixMcFelix 6e9ed60
Wherein I Moev Things.
FelixMcFelix d187bbe
A test suite.
FelixMcFelix 5294b99
One more test to reintroduce.
FelixMcFelix cec6ebe
More Movement of Stuff.
FelixMcFelix af6bbe8
More tweaks.
FelixMcFelix e14bf22
Fix ubench.
FelixMcFelix d5fa92e
Fixup port entry/return probes
FelixMcFelix fbc661d
Cleanup, find meaningful TODOs.
FelixMcFelix 7698a31
Finally do away with ingot_packet.rs
FelixMcFelix 852cfd4
Licensing, docs fixes.
FelixMcFelix a60f598
An import.
FelixMcFelix de099e8
Again.
FelixMcFelix 3fcedc9
Thanks, clippy. Thippy.
FelixMcFelix 54219da
Clippy appeasement at last.
FelixMcFelix f7e64e5
Fixup fuzzers.
FelixMcFelix 22a0ede
Self-review pt.1
FelixMcFelix 40c8840
Self review pt.2 -- less copies on gateway ping
FelixMcFelix f0070f9
Self review pt.4
FelixMcFelix 603f916
SR Pt.5
FelixMcFelix 3dc3a59
SR Pt.6
FelixMcFelix cda3564
SR pt.7
FelixMcFelix 956779d
..
FelixMcFelix 84d8651
Bump ingot.
FelixMcFelix aac32dc
SR pt.8
FelixMcFelix bd7641a
Self-review pt.9
FelixMcFelix c3ebd68
SR Pt.9
FelixMcFelix 5328994
Indicate fast/slowpath in port-process-return
FelixMcFelix 9fa4856
Inline to prevent port-process-return from dinging us.
FelixMcFelix d682ea0
SR Pt.9
FelixMcFelix cdf1d59
Remove vestigial Packet<Initialized>.
FelixMcFelix 5e290a3
Whoops.
FelixMcFelix 9393ad6
Update position on UFT compilation in the architecture doc
FelixMcFelix a4648b4
Mark parse errors in mod-level kstat.
FelixMcFelix 230e3c7
Fuzzers, accidentally a file.
FelixMcFelix 5afd63a
Missed doctests.
FelixMcFelix d8ab666
Review feedback -- de-`pub` inner, de-`inner`.
FelixMcFelix 470ed8a
Document the UDP zero-checksum rationale.
FelixMcFelix 5f53d2c
Review feedback: document mblk refcnt assumptions.
FelixMcFelix 2e800b3
Ingot repo is opened now 🎉
FelixMcFelix 72a116a
Review feedback: pullup packet bodies beyond header MsgBlks
FelixMcFelix 69e844c
Fixup body transform + pullup interaction.
FelixMcFelix ac44642
Fix chain truncation.
FelixMcFelix b386de8
Inline fastpath/lightweight parsing paths
FelixMcFelix 27ecc8d
Review feedback.
FelixMcFelix d7c8b21
Review feedback: copy `db_struioun` on pullup and chain trim
FelixMcFelix 33137dd
All ptrs.
FelixMcFelix 569db38
Fmt.
FelixMcFelix 6ac41b8
Bump to merged ingot, one nit.
FelixMcFelix 540e396
Bump nightly compiler, fix clippy.
FelixMcFelix 9801755
Pick up open Renovate PRs.
FelixMcFelix File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Worth noting that lto has historically had no effect on my benchmarking, but in ingot's microbenchmarks it was fairly crucial.
I don't know if we want to define an alternate profile so that it is scoped to only be used on
ubench
and the kmodule --opteadm
takes quite a bit longer to build now.