When exporting all my notes to Markdown, denote: links point to .org files

[unindented]

The ox-md export engine, whenever it finds a link to a .org file, it replaces its extension with .md:

https://github.com/emacsmirror/org/blob/master/lisp/ox-md.el#L545-L548

The ox-html export engine does a similar thing (if org-html-link-org-files-as-html is non-nil):

https://github.com/emacsmirror/org/blob/master/lisp/ox-html.el#L3266-L3274

It seems like denote translates to .html when the backend is 'html:

https://github.com/protesilaos/denote/blob/main/denote.el#L4792-L4795

But it doesn't do the same for 'md:

https://github.com/protesilaos/denote/blob/main/denote.el#L4799

Is there a way to get denote: links to behave like normal links to .org files when in 'md?

[protesilaos]

Hello @unindented! Can you please show me some sample files that I can use? I want to check what is happening here. We do not have any code for the various Org export backends though.

[unindented]

Can you please show me some sample files that I can use? I want to check what is happening here.

Thanks for your help @protesilaos. I've created a minimal repro at https://github.com/unindented/denote-417-repro.

We do not have any code for the various Org export backends though.

I think org-export-* relies on org-link-*, so this piece of code is what's causing HTML exports to link to .html files correctly, but MD exports to link to .org files:

denote/denote.el

Lines 4822 to 4831 in 3ae863f

	(cond
	((eq format 'html)
	(if query
	(format "<a href=\"%s.html%s\">%s</a>" anchor query desc)
	(format "<a href=\"%s.html\">%s</a>" anchor desc)))
	((eq format 'latex) (format "\\href{%s}{%s}" (replace-regexp-in-string "[\\{}$%&_#~^]" "\\\\\\&" path) desc))
	((eq format 'texinfo) (format "@uref{%s,%s}" path desc))
	((eq format 'ascii) (format "[%s] <denote:%s>" desc path))
	((eq format 'md) (format "[%s](%s)" desc path))
	(t path))))

[protesilaos]

Thank you for this! I now see that this is specific to the org-publish case which goes through all the files. I do not know if there is something we can/should do on our end though. The part you point me to in denote.el does not have any logic to modify the file extension. Is adding such conditionality the right thing? And if so, what should we be checking for?

Note that here we are not testing just for the md export, because exporting an individual file that links to other Org files should retain the .org extension. So just changing the following is not enough:

((eq format 'ascii) (format "[%s] <denote:%s>" desc path))

Plus, with the org-publish case we may have the same issue for the other export backends, but let's see what we can find.

[unindented]

I now see that this is specific to the org-publish case which goes through all the files

You'll see the same behavior if you run org-md-export-to-markdown or org-md-export-as-markdown on an individual file with denote: links. That is, those links will maintain the .org extension, while file: links will not. I believe org-publish is built on top of org-export, so they should behave the same in this regard.

I've updated the repro repo to show the difference in behavior between denote: and file: links: unindented/denote-417-repro@1fc3d62

The part you point me to in denote.el does not have any logic to modify the file extension.

For 'html you do (unless I'm misunderstanding the code):

denote/denote.el

Lines 4792 to 4795 in d3e3762

	((eq format 'html)
	(if query
	(format "<a href=\"%s.html%s\">%s</a>" anchor query desc)
	(format "<a href=\"%s.html\">%s</a>" anchor desc)))

Is adding such conditionality the right thing?

I don't know... ox-md.el does change the extension of all links ending in .org: https://github.com/emacsmirror/org/blob/7a3fc3d78bad1fb4d7afaccdaca8fc1dbffe4b5f/lisp/ox-md.el#L545-L548

And ox-html.el does too (when org-html-link-org-files-as-html is non-nil): https://github.com/emacsmirror/org/blob/7a3fc3d78bad1fb4d7afaccdaca8fc1dbffe4b5f/lisp/ox-html.el#L3266-L3274

exporting an individual file that links to other Org files should retain the .org extension

Yup, I think that's the mismatch between the built-in org-export backends and denote. In org-export backends like 'md, it seems the assumption is that, if you're exporting one file to a certain format, you're exporting its linked files to that format too. That's why they replace the extension of links, from .org to .md.

In denote, you don't want to make that assumption. (Unless you're exporting 'html, because in that case we are assuming that all denote: links should be to .html files, per the code above.)

Not sure what the right path is here. Maybe denote could expose an option equivalent to org-html-link-org-files-as-html, which would cause it to change the extension of links to match the exporting backend? So if you're exporting with 'md backend all links to .org would get translated to .md, et cetera?

[protesilaos]

From: Daniel Perez Alvarez ***@***.***> Date: Sun, 1 Sep 2024 00:43:52 -0700 > I now see that this is specific to the `org-publish` case which goes through all the files You'll see the same behavior if you run `org-md-export-to-markdown` or `org-md-export-as-markdown` on an individual file with `denote:` links. That is, those links will maintain the `.org` extension, while `file:` links will not. I believe `org-publish` is built on top of `org-export`, so they should show behave the same in this regard. [... 19 lines elided] Not sure what the right path is here. Maybe `denote` could expose an option equivalent to `org-html-link-org-files-as-html`, which would cause it to change the extension of links to match the exporting backend? So if you're exporting with `'md` backend all links to `.org` would get translated to `.md`, et cetera?

The introduction of a variable is a reasonable approach. Though I am curious to understand why 'file:' links behave this way, as you note above. In principle, 'denote:' should be the same as 'file:' in this regard. I know there are places where Org hardcodes the assumption that a link is 'file:' or 'id:', such as with image previews. This is not good, because it defeats the purpose of custom hyperlink types (I have been meaning to file a bug for image previews, but have not found time to do it). So we can add the variable to fix the immediate problem, but it is best we also fix Org in the process to help us make 'denote:' links first class citizens.

…

-- Protesilaos Stavrou https://protesilaos.com

[unindented]

In principle, 'denote:' should be the same as 'file:' in this regard.

I think when you call org-link-set-parameters with an :export option, you're overriding the behavior of the Org export backend in use:

denote/denote.el

Line 4819 in d3e3762

:export #'denote-link-ol-export)))))

I don't know if there's a way to call the "parent" export function, as maybe that'd be a more elegant approach. In that hypothetical, denote converts the denote: link into a file: link with the right path, and then calls the parent export function with it. Does that make any sense?

[protesilaos]

From: Daniel Perez Alvarez ***@***.***> Date: Sun, 1 Sep 2024 08:51:46 -0700 > In principle, 'denote:' should be the same as 'file:' in this regard. I think when you call `org-link-set-parameters` with an `:export` option, you're overriding the behavior of the Org export backend in use: https://github.com/protesilaos/denote/blob/d3e3762fe4ac5e3ac664bdeb49ea811d11d345ac/denote.el#L4819

Yes, that makes sense. We need to do this to guarantee the desired behaviour of converting the link to a file path. But then we have the problem you have identified.

I don't know if there's a way to call the "parent" export function, as maybe that'd be a more elegant approach. In that hypothetical, `denote` converts the `denote:` link into a `file:` link with the right path, and then calls the parent export function with it. Does that make any sense?

Sounds good, though I am not aware of a way to do this. If you know, then I am happy to go ahead with it. Though, again, I suspect part of the issue here is with Org hardcoding some logic for 'file:' links. I have not checked for this case, though I have seen it before. If there was some passlist that packages could register themselves to and then, perhaps, define all the extra functions they need, it would make all custom link types equally good.

…

-- Protesilaos Stavrou https://protesilaos.com