Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[nextest-filtering] Replace nom parser with winnow #1214

Merged
merged 49 commits into from
Jan 20, 2024

Conversation

epage
Copy link
Contributor

@epage epage commented Jan 10, 2024

@sunshowers expressed interest in winnow and I figured porting an existing project they were familiar with would make it easier when they try it out.

When i first switched from noms pure Fn(I) -> (I, O) parser model to Fn(&mut I) -> O, I was concerned about it making things worse. It does have the downside of requiring more lifetime annotations to disambiguate things but I feel like it has been proven time and again to simplify code. In this case, it removed a lot of bookkeeping from the functions, making the intentional bookkeeping more obvious, and it allowed the removal of RefCell for error recovery.

I broke this up into a lot of commits

  • Smaller nuggets to review
  • Easier to call out the differences
  • Easier for me to deal with merge conflicts.

I did put a little too much in the unpeek-ectomy commit towards the end. The main downside is that we can't git bisect to one specific bookkeeping change.

I did this as 1 PR though because the decision to commit to this is all-or-nothing.

Copy link

codecov bot commented Jan 10, 2024

Codecov Report

Attention: 11 lines in your changes are missing coverage. Please review.

Comparison is base (983446d) 77.40% compared to head (72d2fe6) 77.49%.

Files Patch % Lines
nextest-filtering/src/parsing.rs 97.70% 9 Missing ⚠️
nextest-filtering/src/errors.rs 80.00% 1 Missing ⚠️
nextest-filtering/src/parsing/glob.rs 94.73% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1214      +/-   ##
==========================================
+ Coverage   77.40%   77.49%   +0.09%     
==========================================
  Files          72       72              
  Lines       17859    17884      +25     
==========================================
+ Hits        13823    13859      +36     
+ Misses       4036     4025      -11     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

epage added 28 commits January 10, 2024 12:54
The explicit `complete` version is deprecated, so switching straight to
`impl Parser for &str`, a shortcut for the auto-`complete` version.
`char` was consolidated into `one_of` but this jumped straight to
the `impl Parser for char` which is easier to read.
`pair` is deprecated in favor of `impl Parser for (...)`
In winnow 0.5, we can replace this with `tag` and speed it up
Free-standing `recognize` is deprecated in favor of trait methods but it
wasn't even needed here.
This added back in a `tag` that a later release (0.4?) can remove due to
better type inference.
This added a lot of `tag` / `one_of`s that can be removed with a later
release (0.4?)
epage added 19 commits January 10, 2024 12:54
This was possible due to improvements in type inference (when you have a
generics problem, throw more `PhantomData` at it...).
In 0.5, `parse_next` modifies input so the old behavior is under the
name `parse_peek` (as it doesn't commit to the parse but returns what it
would commit).
So `unpeek` takes a combinator that doesn't modify input ("peek parses")
and makes it commit to the result modifying the input.

This is added only to ease the transition.  Future commits will work to
remove `unpeek` calls.
Unsure why it was originally needed.  One difference with `nom` is that
we return `Stream::Slice` from `token` parsers, rather than `Stream`,
simplifying the more common cases.
Because parsers now take `&mut I` and all cloning is gone,
we don't need `RefCell` anymore.
@sunshowers
Copy link
Member

Wow, thanks! Will look very soon. (you want to run cargo xfmt to fix the lint issue)

@sunshowers
Copy link
Member

This is awesome! Thank you. I'll go ahead and land this.

@sunshowers sunshowers merged commit dde8680 into nextest-rs:main Jan 20, 2024
13 checks passed
@epage epage deleted the winnow branch January 21, 2024 00:28
@epage
Copy link
Contributor Author

epage commented Jan 21, 2024

As you use it, I welcome any feedback to help make it better!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants