Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

lib/markdown2: introduce a new Markdown engine #2720

Merged
merged 17 commits into from
Jun 27, 2018

Conversation

Morriar
Copy link
Member

@Morriar Morriar commented Jun 20, 2018

This PR introduces a new Markdown engine called markdown2.

The current Markdown implementation is functional but does not follows the CommonMark specification as it passes only 273 tests of the 627 from the CommonMark test suite.

Also there is a few issues with it as stated in #791, #1394, #1525 and #2507.

This new implementation aims at resolving these issues and follow the CommonMark specification.

Features:

  • Markdown AST creation and rendering
  • Rendering to HTML
  • Rendering to Markdown
  • Rendering to Man format
  • Rendering to LaTeX
  • Parsing of Github extended mode
  • Parsing of wikilinks
  • Parsing of LaTeX/Maths expression (upcoming in next PR)
  • Respect of the CommonMark specification with only 10 tests failed (UTF-8 related)
  • Extensive testing with a total of 980 test units...

I didn't remove the old markdown implementation since the benchmarks are not really good for now. Since markdown2 uses a lot of regular expressions, performances can be an issue compared to markdown:

screenshot from 2018-06-20 19 21 08

In the following PR some Nit tools will be migrated to the new implementation:

  • nitunit
  • nitiwiki
  • nitdoc
  • nitweb

@Morriar
Copy link
Member Author

Morriar commented Jun 21, 2018

test this please

@Morriar
Copy link
Member Author

Morriar commented Jun 21, 2018

Tests are ok, the "location marker" found by check whitespace is a Markdown setext header:

My Heading
=========

privat added a commit that referenced this pull request Jun 26, 2018
This PR introduces a new Markdown engine called `markdown2`.

The current Markdown implementation is functional but does not follows the [CommonMark](http://commonmark.org/) specification as it passes only 273 tests of the 627 from the [CommonMark test suite](https://github.com/CommonMark/CommonMark).

Also there is a few issues with it as stated in #791, #1394, #1525 and #2507.

This new implementation aims at resolving these issues and follow the CommonMark specification.

Features:
* Markdown AST creation and rendering
* Rendering to HTML
* Rendering to Markdown
* Rendering to Man format
* Rendering to LaTeX
* Parsing of Github extended mode
* Parsing of wikilinks
* Parsing of LaTeX/Maths expression (upcoming in next PR)
* Respect of the CommonMark specification with only 10 tests failed (UTF-8 related)
* Extensive testing with a total of 980 test units...

I didn't remove the old markdown implementation since the benchmarks are not really good for now. Since `markdown2` uses a lot of regular expressions, performances can be an issue compared to `markdown`:

![screenshot from 2018-06-20 19 21 08](https://user-images.githubusercontent.com/583144/41689548-1a8bc5d0-74bf-11e8-899e-52b0eb093d57.png)

In the following PR some Nit tools will be migrated to the new implementation:
* nitunit
* nitiwiki
* nitdoc
* nitweb

Pull-Request: #2720
privat added a commit that referenced this pull request Jun 26, 2018
This PR does some cleaning around commands hierarchy to prepare for the merge with new-markdown (#2720).

Pull-Request: #2723
Copy link
Member

@privat privat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wow

redef fun parent do return super

# Can this block contain other blocks?
var is_container = false
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't is_container be a function?

@privat privat merged commit 278b289 into nitlang:master Jun 27, 2018
@Morriar Morriar deleted the new-markdown branch June 27, 2018 18:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants