Footnotes not working anymore

[nathanlesage]

Heya!

I noticed a bug recently in the Blog plugin: I don't exactly know when or why it happened, but as of recently, footnotes are no longer properly parsed. I did not change the syntax and double-checked for eventual syntax errors, but it appears the Markdown parser doesn't like footnotes anymore. This behavior is visible both when outputting post.content_html | raw on the page as well as in the preview field in the backend.

Interestingly, this did work in the past. And even more so, since the blog plugin "caches" the rendered HTML in the content_html variable, the old posts with footnotes still work. To give you an example:

• A recent post I published a few days ago. You can search for [^1] to discover that the markup is not rendered. This post lets me assume this bug has been there at least since July.
• This is the last post which properly parsed footnotes, from June 30th.

Current versions:

• WinterCMS v1.2.3
• Blog Plugin v2.0.2

[LukeTowers]

This is likely due to the recent change in the library used to parse the markdown, @bennothommo do you have any thoughts?

[nathanlesage]

Hey, is there maybe any update on this issue?

[bennothommo]

Sorry @nathanlesage for missing this one.

There is a Footnote Extension for Commonmark, the library we now use for Markdown parsing. I'll have to double-check if that's already being included or not - we may have to get the Blog plugin to automatically include it.

[nathanlesage]

That would be awesome, thank you! I would jump in if I had some free time, so I'm sorry about just nagging you... I hope it works well, and happy holidays to you, of course!

[bennothommo]

@wintercms/maintainers I had an idea about perhaps making the following features of Commonmark controllable, perhaps as properties for the Post component, or as additional settings in the BlogSettings model:

• Table of contents
• Footnotes
• Block and inline attributes

These features are available in Winter\Storm\Parse\Markdown, but by default are disabled. The intention was to allow developers to enable them when they want them.

However, if we were to make these available to be toggled in the Post component, we would need to parse the Markdown at the component level, instead of relying on the content_html attribute which is generated on save. This would effectively make that attribute redundant. In order to save on performance if we go down this path, we'd probably need to cache the rendered content - perhaps with a language prefix if the Translate plugin is installed.

I'm not sure if content_html is used anywhere else.

Do you guys have any thoughts on this?

[mjauvin]

I think the idea of extra markdown features is interesting.

[nathanlesage]

I personally found the route via content_html always a bit weird. I understood it always as a cache mechanism, but it is true that this could also be handled on a per-page basis. Right now it saves me because it allows me to monkey patch footnote support, but I would happily live without it.

In general, it may make sense to do the following:

• People should always use the md twig filter instead of using content_html (which is essentially the same but cached in the database)
• Allow developers to hook into the Markdown parser (I'm not sure if the PHPLeague one makes that easy, I only know that JS based parsers make extensions very simple)

Then you would also have the benefit that the Markdown parser natively only needs to support whatever PHPLeague offers, and divert the efforts to support additional syntaxes to plugin developers. This might be great, also for supporting non-standard syntax (think citations and other goodies that Pandoc offers).

[bennothommo]

@nathanlesage great thoughts.

You are correct on content_html being mainly a caching mechanism, although it does serve well as a "normalised" value for when people change the Blog plugin to use the WYSIWYG editor over the Markdown editor - it can be safely assumed that its value will always be the resultant HTML.

Nevertheless, if we want the Markdown to be customisable, it cannot be used.

Using the md filter will introduce a penalty when reading the blog post. I don't think it's too much, but it is still a penalty. It could be mitigated by caching within the component itself, if we wanted to go down that route, although we'll need to allow for the Translate plugin to be able to do its thing if needed. We'd also need the Blog plugin to be able to invalidate the cache when a change is made to the blog post (or the page it is on).

As for customising the Markdown parser - we have a wrapper around the PHP League's Commonmark library which can enable or disable features at will. If someone wanted to customise it even further, they can extend our parser. So the capability is already there.