Initial rayon support: ParallelExtend + FromParallelIterator

[Others]

This was a straightforward step towards #14.

I'm interested in trying to get some parallel iterator work going, but I thought I'd start with some small stuff :)

Points that need addressing:

• par_extend_sync is a crap name, so we likely need to think of something better!
• we create a guard for every insert operation which sucks, and I wonder if we can do better
  (this is especially relevant since I believe there's going to be a similar problem with parallel iterator support)

---

This change is 

[codecov]

Codecov Report

Merging #89 into master will decrease coverage by 1.59%.
The diff coverage is 0.00%.

Impacted Files	Coverage Δ
src/lib.rs	100.00% <ø> (ø)
src/map_ref.rs	87.03% <ø> (+9.67%)	⬆️
src/rayon_impls.rs	0.00% <0.00%> (ø)
src/set.rs	94.02% <ø> (+35.29%)	⬆️
src/set_ref.rs	45.23% <ø> (+45.23%)	⬆️
src/node.rs	73.08% <0.00%> (-4.89%)	⬇️
src/raw/mod.rs	85.43% <0.00%> (-3.89%)	⬇️
src/map.rs	81.48% <0.00%> (-2.62%)	⬇️
src/iter/traverser.rs	91.20% <0.00%> (-0.80%)	⬇️
... and 3 more

[jonhoo]

Thank you!

@cuviper Since you have the most rayon experience (I believe), would you mind taking a look at this?

[Others]

@jonhoo No problem. Do you have any suggestions for getting my tests running on CI? They’re behind a feature flag so I don’t think they’re being picked up.

[jonhoo]

By default the normal test suite should be run with --all-features I believe. If you want them to also run with miri/ASAN/LSAN, you'll want to modify these lines:

flurry/azure-pipelines.yml

Line 60 in 448d9f6

- script: cargo miri test -- -Zmiri-ignore-leaks

flurry/azure-pipelines.yml

Line 76 in 448d9f6

env ASAN_OPTIONS="detect_odr_violation=0" RUSTFLAGS="-Z sanitizer=address" cargo test --lib --tests --features sanitize --target x86_64-unknown-linux-gnu

flurry/azure-pipelines.yml

Line 94 in 448d9f6

env RUSTFLAGS="-Z sanitizer=leak" cargo test --features sanitize --target x86_64-unknown-linux-gnu

To also include your feature (or just make them --all-features — that seems reasonable too).

[Others]

Ah, I guess they're running but not picked up by codecov?
I'm assuming MIRI isn't going to work, as the tests use threading. I'll add --all-features to the sanitizer runs.

I've been thinking a bit about having to create a guard for every element we want to add. Creating a guard is a bit expensive (it definitely has a performance impact on my project using flurry) and might be dragging down performance. We could consider using thread-local to optimize into just creating one guard per worker thread. Whoever reviews this might want to consider whether that optimization is worth it.

[jonhoo]

Ahhh, yeah, that's awkward. Looks like the template we're using doesn't pass --all-features to tarpaulin. Easiest way to fix that is probably to just inline the code to run it. We won't need the templating stuff or any of the token detection stuff, so only the last job should be relevant.

So, the guard stuff is actually pretty thorny 😅 We've discussed it a lot over in jonhoo/bustle#2 (comment) (and the following comments) which you may want to take a look at. I think we should probably not make a change to how guards work in this PR. Definitely agree that we should be able to do better than creating a guard for every access in rayon though. @cuviper does rayon have a way to do some "prep" on each thread that is running the parallel iterator?

[Others]

Yeah my initial idea to use a thread local on the stack doesn't work, as Guard is not Send. (Which makes sense, since it pins a thread.)

[cuviper]

The code looks good to me.

I'll let @jonhoo decide if CI/coverage is good enough...

[Others]

Just pinging that this is still on my radar, will hopefully get to this in the later half of the week :)

[Others]

Okay I think this hits a decent subset of the impls we want, plus mostly okay tests. Might be suffering a bit of PR creep, so I'd rather not make it any bigger.

[jonhoo]

This looks good to me — thanks!

[cheako]

One of Rust's features is the naming convention, it seems clumsy and random(like a blaster). That has both pro(s) and con(s), like not knowing what something is. I think the OP(@Others @jonhoo) here should be extended with a short description of what is the goal this PR is trying to accomplish... With an intention on explaining it to someone who will then follow newly added links to rayon site(s) for deeper explanation and further reading I.E. someone who thinks rayon has more todo with light than as a Rust crate.

Walk entries on multiple threads using convenience [rayon](https://docs.rs/rayon) wrapper [trait](https://docs.rs/rayon/latest/rayon/iter/trait.ParallelIterator.html).  Also consume same trait for [insertion](https://docs.rs/rayon/latest/rayon/iter/trait.ParallelExtend.html) on multiple threads.

[jonhoo]

I'm not exactly sure what you're asking for here? If you'd like to see documentation on a particular method be improved, then I'd be happy to take a PR with that improvement!

[cheako]

The goal was unclear without prior knowledge of rayon or whatever. Without links to rayon that leads to googling or looking to see if there is a docs page.

[jonhoo]

The goal of what exactly? Of the PR? Of the commit? Or of the interface in the code? I don't generally expect contributors to document their PRs/commits directly, since it's assumed that those will be seen by developers of the library, not of its users.