Hmm, but it probably makes sense for sequences to be keyed by accession in LAPIS.

What would Nextclade do with:

For me that's probably what we want to aim towards - the accession as the real ID and then the display name as a description.

Indeed, Nextclade would just show the whole thing that follows >

We should add a fasta description, that's what I meant.

I made a LAPIS issue

It's worth checking that this won't break some commonly used (but stupid) programs... a lot of them cannot handle anything with spaces or even with slightly weird character. Like IQTree possibly? While that's technically their problem it's a big pain for users to have to strip down fasta names before running such stuff (as we do in Nextstrain...)

Also, some programs will assume the fasta header (whole thing) matches metadata column X (to match between metadata & fasta files), so users may also run into trouble there.

Ideally this could be something configurable by the user - to pick how they want their download.

I was just testing out iqtree with NCBI Virus (haven't finished that yet) and they indeed have nice customisation:

IQtree processed the sequences properly (i.e. discarding the descriptions). Given this is the default GenBank format I would expect most well-engineered tools to be able to cope with it but I agree there will be some exceptions (including lots of my janky scripts). The LAPIS implementation we've been discussing most recently would allow customisation.

Thanks for the research @theosanderson

As an aside, the NCBI virus default is not great, the title is usually very uninformative - one reason I'm excited for Loculus to do this better 😀

Just so we're on the same page about the implementation of this, my understanding at the moment is: SILO/LAPIS doesn't support this; if we want to have it anyways, we'll have to not rely on the direct download from there anymore, but instead build our own download endpoint in the backend, which either proxies the LAPIS one (downloading from there, modifying the files, zip again and send to user) or we build a new download endpoint from scratch.

A potential (temporary) solution could also be what @theosanderson did here where he added a new (external) service to join the data: #3439

I've implemented something here, 80/20 solution, not streaming as perfect is the enemy of the good. Some thoughts here: #3448 (comment)

Would still be nice if LAPIS implemented that feature. It's the obvious place to do this. It's a bit silly that we have to download metadata and sequences from LAPIS separately, then join them on the website server before sending them on.

This is @theosanderson's single file service (the rest is just boilerplate): https://github.com/theosanderson/metadata-sequence-combiner/blob/main/pages/api/combine.ts

Interesting how easy it is to do that when you don't have zodios hooks and types to jump fight 😄 Might not work for segment selection though.