UdpSocket::recv_from occupies majority of total cpu time

[franklee26]

Hi all,

I'm playing around with building my own network protocol over UDP. As part of my POC I had an idea for how to handle socket reads: spawn n tasks where each task would recv_from().await over a shared udp socket in a loop. Each task is Arc::clone() from the same udp socket (see here).

I briefly profiled the performance of this via flamegraphs & tokio console (I'm running on an m4 macbook pro w/ 10 performance cores. ). The flamegraph screenshot is attached below for a run where I was reading a total of ~500mb where each udp datagram is ~1kb. This was run via the loopback interface. 82% of the usage came from recv_from and I'm inclined to believe I am doing something incorrectly/invoking an anti-pattern. I tinkered with changing the number of these reader tasks and I see similar distributions.

1. Is this to be expected? I half expected the .recv_from to be fast enough where my own session logic would dominate usage. That doesn't seem to be the case; the stacks on the far left account for my processing.
2. How can I use the udp socket more effectively? I've read about SO_REUSEPORT but afaik tokio's udp socket doesn't directly support this and I would need to do more tinkering. Would this even be relevant here?