Publish Rust crates

[felixhekhorn]

Continuation of the discussion in #372

We should start publish our rust crates to crates.io. For that I can see some questions:

• naming: as pointed out by @alecandido on crates.io we can be called eko, but this would not work inside python/maturin as eko is as a name already taken of course - can we disambiguate the two? or we just keep ekors? The name eko is reserved
• ownership: this is in analogy to PyPI - I guess someone needs to own the crate on crates.io. Can we make it the NNPDF organization? i.e. who can push to this registry and how can we do it in practice (e.g. eko is currently owned by @alecandido).
• automatization: eventually we want to automatize the publication in the CI - I guess we should take some inspiration from PineAPPL. (Actually that file looks scarily complex, @cschwan we may need you help 😇 ). This concerns both the Rust package and it's associated python wheel. This is directly correlated with the ownership.

@alecandido your help would be greatly appreciated 🙏 .

[cschwan]

I suggest to use eko-rs, that's a naming convention that many crates use. The suffix -rs is often used to denote an 'oxidized' version of a library (originally written in different language, but now a rewrite/bindings to Rust).

The owner of the crate on https://crates.io/ is the one linked that has their Github account linked. I believe multiple owners are possible, so the practical approach would be that one of you or even both of you own it, and then we ask @scarrazza if we could link the NNPDF organization account to crates.io. I'd also like to give the ownership of PineAPPL to the NNPDF organization.

In any case, the practical significance is that we need to store a key in the CI in to order publish the crate. Publishing is actually quite easy, see the corresponding action for PineAPPL (all the other stuff is more complicated indeed):

https://github.com/NNPDF/pineappl/blob/05a115f2bd446aeeb5f01cbd5cfb3c4bb16213dc/.github/workflows/release.yml#L174-L198

This automatically publishes all crates if a commit has been tagged with a tag name v[0-9]+*:

https://github.com/NNPDF/pineappl/blob/05a115f2bd446aeeb5f01cbd5cfb3c4bb16213dc/.github/workflows/release.yml#L6

[alecandido]

naming: as pointed out by @alecandido on crates.io we can be called eko, but this would not work inside python/maturin as eko is as a name already taken of course - can we disambiguate the two? or we just keep ekors? The name eko is reserved

You can disambiguate the two, I'm already doing that in other cases, e.g. in PineAPPL:
https://github.com/NNPDF/pineappl/blob/05a115f2bd446aeeb5f01cbd5cfb3c4bb16213dc/pineappl_py/pyproject.toml#L6
https://github.com/NNPDF/pineappl/blob/05a115f2bd446aeeb5f01cbd5cfb3c4bb16213dc/pineappl_py/Cargo.toml#L8
(here the problem is the opposite, since the original is Rust, so the package/crate is named pineappl_py, while the python package is just pineappl - but, of course, being strings you can choose whatever, the info is just that they don't have to be the same).

ownership: this is in analogy to PyPI - I guess someone needs to own the crate on crates.io. Can we make it the NNPDF organization? i.e. who can push to this registry and how can we do it in practice (e.g. eko is currently owned by @alecandido).

I can pass ownership, no problem. To NNPDF would be ideal, but I'm not sure that is possible with crates.io.

EDIT: it is, it could even be owned by the @NNPDF/theory team
https://doc.rust-lang.org/cargo/reference/publishing.html#cargo-owner

(all the other stuff is more complicated indeed)

All the complexity comes for the CLI, the C API, supporting py3.6 (which EKO does not)

NNPDF/pineappl@05a115f/.github/workflows/release.yml#L174-L198

This automatically publishes all crates

Crates

In principle, you do not even need to cd, you could do something like find ... | xargs ... 'cargo publish -p {}', and publish all crates with a one-liner.

Wheels

If you do not have any special need, you should be able to just use https://github.com/PyO3/maturin-action and you're done.

There is even a maturin subcommand to just generate it:
https://www.maturin.rs/distribution.html?highlight=ci#github-actions

You might have to change working directory, because the workspace is not top-level - but I could also make it top-level, just moving crates/Cargo.toml to Cargo.toml (and adding the crates/ path in the members declaration, that could also be declared as crates/* at this point).

---

All in all everything should be pretty simple and smooth, but... do we have to apply every time the patches? Or are you going all-in to Rust and dropping them?

[felixhekhorn]

do we have to apply every time the patches? Or are you going all-in to Rust and dropping them?

eventually, we will go all-in - but for the moment we go incrementally (e.g. currently only LO QCD is supported), so it will still take a bit (although we should push for it #369 )

in any case also a manual solution would be fine at the moment, just to get started (i.e. sort out the problems) and to reserve the respective names ...

[alecandido]

Actually, the patches are just for running with eko in Python, crates and wheels will be separately distributed, so no worries.

Whenever you need the workflow, just let me know, and I will try to make them and make reusable, to put them in https://github.com/NNPDF/workflows.
Once done for one repo, they could be used for all of them (some differences in layout could be accounted for - eventually even PineAPPLpy and EKO could share the same workflows).

[felixhekhorn]

When ever you have time please go ahead ...