In text citation formats and auto-replacement

[hdashnow]

Hi all,

Here's a proposed format for in-text citation within the json text fields:

Drawing from: https://greenelab.github.io/meta-review/#citation-by-identifier

String in STRchive data json:
"details":"This locus was pathogenic at size 10 [doi:10.1098/rsif.2017.0387]. There is some debate about the meaning of interruptions [pmid:25851694]"

Viewed on gene page:

This locus was pathogenic at size 10 [5]. There is some debate about the meaning of interruptions [16]"

Where the link jumps down to the corresponding reference at the end of that page (or does a tool tip if wanting to get fancy, not required).

These unique paper identifiers will correspond to a json or yaml for all references, hopefully generated by one of the manubot tools. For example like in lab-template.

@vincerubinetti do you think it makes sense to limit this to a set of standard identifiers? For example, just use doi: url: and maybe pmid:? Use all in that table? Some other combination? I'm guessing you need to know what the possible characters are before the : for correct parsing. Any other thoughts on making this work?

[vincerubinetti]

So Manubot, just FYI, supports hundreds of identifiers. For this case, even if we only have a few types we'll likely see, I wouldn't want to hard code it. That is, I'd probably just match it with a simple regex like \[[a-z]+:.+\] (some letters, a colon, then any chars, surrounded in brackets). I don't know where these description fields are coming from or how you're sanitizing them. Would that run the risk of any false positives?

[hdashnow]

Okay, so just to clarify, you just feed anything before the colon to manubot to find the citation? So as long as it's one they support it should work?

We write this text, so we should get any weird input that isn't our own fault. There are a few text fields that could have citations:
age_onset
mechanism_detail
prevalence_details
disease_description
details

The only instances of \[[a-z]+:.+\] that I can think of that might cause problems are URLs, for example [http:] or [https:]. Would adding something extra out the front like [@doi:] help to avoid that sort of thing?

[vincerubinetti]

Yes, you just call manubot with e.g. doi:123456. This is the command you'd be using, and the same one that lab-website-template calls directly:

https://github.com/manubot/manubot?tab=readme-ov-file#cite

manubot cite doi:10.1098/rsif.2017.0387 pubmed:29424689 pmc:PMC5640425 arxiv:1806.05726

Regarding http:, if you wanted to be most consistent with Manubot, you can prefix those with url:, so it'd be [url:https://google.com]. Manubot can cite those too (though less reliably and with less details I think). If you don't want to do that, I think we could also just add an http/https/mailto/whatever exception to the regex so it doesn't conflict. Probably no need for the @.

[hdashnow]

Thanks. Yes we would use url: for URL citations. I'm thinking more for regular links in markdown format we might not want them to be converted to citations. Sounds like a good plan to have exceptions for http, https, mailto and ftp

• Note to self: will need to add some documentation for this for future us

[laurelhiatt]

word. I think this is one of the final steps probably, since I will comb through the final json for formatting, so we can check in before that to create documentation before/during that process.

[vincerubinetti]

Maybe we should just follow the Manubot convention:
https://raw.githubusercontent.com/manubot/rootstock/refs/heads/main/content/02.delete-me.md

Lorem ipsum [@doi:12345; @pmid:12345]

Seems like this would avoid all ambiguity. Though they do one weird thing that I don't like which is citation by URL is in the form @{https://greenelab.github.io/meta-review/}, whereas we could just be consistent and do @url:https://greenelab.github.io/meta-review/.

I don't know.

[laurelhiatt]

adding the @ signs. thank you!

[vincerubinetti]

Seems like we've landed on a free-text citation format. How about the top-level citations data format? I was thinking something like this:

STRchive-database.json (note the literature field):

STRchive-citations.json:

[laurelhiatt]

Sure, works for me!

…

On Nov 18, 2024, at 8:51 AM, Vincent Rubinetti ***@***.***> wrote: Seems like we've landed on a free-text citation format. How about the top-level citations data format? I was thinking something like this: STRchive-database.json: Screenshot.2024-11-18.at.10.49.49.AM.png (view on web) <https://github.com/user-attachments/assets/d921dc6f-d199-4223-9f60-6fdfe7b069b1> STRchive-citations.json: Screenshot.2024-11-18.at.10.49.57.AM.png (view on web) <https://github.com/user-attachments/assets/335d89fe-4755-43d4-a53a-c45e043c39da> — Reply to this email directly, view it on GitHub <#56 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AULUPMFCSKOCX5LYYWOY6ST2BIEHTAVCNFSM6AAAAABQ4NHEHWVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTCMRZGM3TOOI>. You are receiving this because you commented.

[vincerubinetti]

Note that I was assuming/thinking that the literature field would also include all of the free text citations as well. I could extract the free text citations and combine them myself, but I think anyone using the json directly might appreciate having them all in one place?

[hdashnow]

Just to make sure I understand, I was assuming that we would provide the IDs (e.g. PMID) and then use manubot to pull the full citation. Is that STRchive-citations.json for the manubot output? Or something we are populating ourselves?

[vincerubinetti]

Yes my idea was that Manubot would eventually generate strchive-citations.json, with some minimal "processing" by us, e.g. Manubot returns date in a weird format like date_parts: [ [2000, 12, 5] ], and e.g. it returns authors as something like authors: [ {given: "Steve", family: "Smith" } ].

And we'd need to pass Manubot a combined list of per-locus citations (the new literature field), and citations extracted from the free-text fields like mechanism_details (which presumably you already have some kind of script to do?).

But for the immediate-term, I thought we'd just keep the existing PMID json, even though it doesn't have DOIs.

[hdashnow]

Great. I'm sure you've already discussed this with Laurel, but just adding that we don't already have scripts for some of this. We only have the script to pull all PMIDs + title and year for all citations matching a given search pattern. Not for processing the ones in the json free text fields.