Comparison benchmarks and comparison in Readme #113
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,129 @@ | ||
| #![cfg(not(tarpaulin))] | ||
|
|
||
| use criterion::{black_box, criterion_group, Bencher, Criterion}; | ||
| use ringbuffer::{AllocRingBuffer, ConstGenericRingBuffer, RingBuffer}; | ||
| use std::collections::{LinkedList, VecDeque}; | ||
| use std::sync::mpsc::channel; | ||
|
|
||
| const ITER: usize = 1024 * 16; | ||
| const CAP: usize = 1024; | ||
|
|
||
| fn std_chan(b: &mut Bencher) { | ||
| let (tx, rx) = channel(); | ||
|
|
||
| b.iter(|| { | ||
| for i in 0..ITER { | ||
| let _ = tx.send(i); | ||
| black_box(()); | ||
| } | ||
|
|
||
| for i in 0..ITER { | ||
| let res = rx.recv(); | ||
| let _ = black_box(res); | ||
| } | ||
| }); | ||
| } | ||
|
|
||
| fn vec(b: &mut Bencher) { | ||
| let mut vd = Vec::with_capacity(CAP); | ||
|
|
||
| b.iter(|| { | ||
| for i in 0..ITER { | ||
| let _ = vd.push(i); | ||
| black_box(()); | ||
| } | ||
|
|
||
| for i in 0..ITER { | ||
| let res = vd.remove(0); | ||
| let _ = black_box(res); | ||
| } | ||
| }); | ||
| } | ||
|
|
||
| fn vecdeque(b: &mut Bencher) { | ||
| let mut vd = VecDeque::with_capacity(CAP); | ||
|
|
||
| b.iter(|| { | ||
| for i in 0..ITER { | ||
| let _ = vd.push_back(i); | ||
| black_box(()); | ||
| } | ||
| for i in 0..ITER { | ||
| let res = vd.pop_front(); | ||
| let _ = black_box(res); | ||
| } | ||
| }); | ||
| } | ||
|
|
||
| fn linked_list(b: &mut Bencher) { | ||
| let mut ll = LinkedList::new(); | ||
|
|
||
| b.iter(|| { | ||
| for i in 0..ITER { | ||
| let _ = ll.push_back(i); | ||
| black_box(()); | ||
| } | ||
|
|
||
| for i in 0..ITER { | ||
| let res = ll.pop_front(); | ||
| let _ = black_box(res); | ||
| } | ||
| }); | ||
| } | ||
|
|
||
| fn cg_rb(b: &mut Bencher) { | ||
| let mut rb = ConstGenericRingBuffer::<_, CAP>::new(); | ||
|
|
||
| b.iter(|| { | ||
| for i in 0..ITER { | ||
| let _ = rb.push(i); | ||
| black_box(()); | ||
| } | ||
| for i in 0..ITER { | ||
|
There was a problem hiding this comment. Doing this more than CAP amount of time is just a no-op right? There was a problem hiding this comment. sort of, the buffer has to reject the items all the rest of the times which might take time just as our dropping of old items does. There was a problem hiding this comment. sorry that's for the heapless version. Here it will drop old items ITER-CAP times There was a problem hiding this comment. I'm just kinda confused why we are testing different things for different structures, shouldn't we test all under the use case of: A buffer which can contain at max CAP items, removing old entries as new ones get pushed. Clearing out the buffer at the end. There was a problem hiding this comment. no other datastructure has that behavior. They're different datastructures, we cannot test them under the same circumstances There was a problem hiding this comment. Then the comparison is meaningless imo, it should be about using similar datastructures for the same purpose. I think we can emulate ringbuffer behaviour on most if not all of these datastructures There was a problem hiding this comment. I disagree. No benchmark can completely fairly compare these datastructures, but it does give a good overview of differences in order of magnitude. Of course a channel is not a ringbuffer, but pushing 2^14 items to a channel takes about 14 times longer than to a ringbuffer which deals with removing old items. Simply pushing 2^14 items to a vecdeque that has to deal with growing is about 3x slower than pushing to a ringbuffer. Pushing to a heapless deque which drops items on insert is a few percent slower than pushing to a ringbuffer which deletes old items. None of them are the same datastructure, but it does show the order of magnitude of differences you can expect when using these datastructures. Each datastructure is good at different things, but this is their normal performance when you use them as a queue. In fact, I don't really want to modify these datastructures to function identically to ringbuffer. Then we don't measure how these datastructures normally perform as a queue for the purpose they were designed for. Instead we measure how well we can turn the datastructure into a ringbuffer. There was a problem hiding this comment. As long as we add a disclaimer/description of the benchmarks in the readme this is fine then I think |
||
| let res = rb.dequeue(); | ||
| let _ = black_box(res); | ||
| } | ||
| }); | ||
| } | ||
|
|
||
| fn heapless_deque(b: &mut Bencher) { | ||
| let mut rb = heapless::Deque::<_, CAP>::new(); | ||
|
|
||
| b.iter(|| { | ||
| for i in 0..ITER { | ||
| let _ = rb.push_back(i); | ||
| black_box(()); | ||
| } | ||
| for i in 0..ITER { | ||
| let res = rb.pop_front(); | ||
| let _ = black_box(res); | ||
| } | ||
| }); | ||
| } | ||
|
|
||
| fn al_rb(b: &mut Bencher) { | ||
| let mut rb = AllocRingBuffer::with_capacity_non_power_of_two(CAP); | ||
|
|
||
| b.iter(|| { | ||
| for i in 0..ITER { | ||
| let _ = rb.push(i); | ||
| black_box(()); | ||
| } | ||
| for i in 0..ITER { | ||
| let res = rb.dequeue(); | ||
| let _ = black_box(res); | ||
| } | ||
| }); | ||
| } | ||
|
|
||
| fn criterion_benchmark(c: &mut Criterion) { | ||
| c.bench_function("comparison std channel", std_chan); | ||
| c.bench_function("comparison std vec", vec); | ||
| c.bench_function("comparison std linked list", linked_list); | ||
| c.bench_function("comparison std vecdeque (growable ringbuffer)", vecdeque); | ||
| c.bench_function("comparison const generic ringbuffer", cg_rb); | ||
| c.bench_function("comparison alloc ringbuffer", al_rb); | ||
| c.bench_function("comparison heapless deque", heapless_deque); | ||
| } | ||
|
|
||
| criterion_group!(comparison_benches, criterion_benchmark); | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shouldn't we push until
CAPand then start doing remove & push, to have the number of elements stay onCAPThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
remove from front push on back specifically as well
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
hmm, I guess that makes sure that the first iteration is not an outlier