Interest in supporting more backends

[Sudneo]

Hello, and thanks for this project! I am following it for a while (especially the split of the library/CLI!).

I am working on a Kubernetes backup solution for my own lab, and I would like to ship backups to B2. I am aware that right now the support for B2 is through rclone, but I would like to remove another binary dependency to keep my tool purely in rust and to be more container-friendly.

I am by no means a good software developer, and I am very new at rust specifically, but I wanted to ask whether you were interested in contributions to add more backends (well, specifically B2), or whether you intended to keep the amount of backends supported minimal and rely on rclone/REST (and default) only.

If you are not against the idea in principle, I can start putting something together (basically, a rust(ic) port of https://github.com/restic/restic/blob/master/internal/backend/b2/b2.go) and open a PR in rustic_core repo.

P.s.

I was questioning whether it would make more sense to open an issue in rustic_core or to ask the question here, hopefully I ended up making the right choice!

[simonsan]

Hey there,
rustic-rs/rustic_core#10 (comment)

I think it would be useful to implement an OpendalBackend with https://github.com/apache/incubator-opendal so we get a lot of storages for free:

From this POV I'm not sure if it makes sense to implement a backend specifically for rustic or if it wouldn't be better to implement a B2 storage for OpenDal and integrate that.

[Sudneo]

Thanks for chiming in! That seems like a good plan/idea (although I am a little bit skeptical about many abstraction layers), and it would be great to spend time integrating one library that offers N storage backends (basically, like rclone but with library bindings, not binary execution, if I understand well?).

This said, I am afraid my rust knowledge (and frankly, probably even my programming experience) is way too weak for that, especially since B2 does not seem to be among the s3-compatible services and contributing to such a complex project seems out of reach for me.

Happy to put this on hold in any case!

[simonsan]

Maybe @aawsome can give their opinion as well, if it's not in the list of already supported services with OpenDAL, it could make sense to implement it directly as a backend in rustic - I don't really have an opinion on that, though.

[aawsome]

My personal opinion is, if there is a PR which adds value for some users (and doesn't break anything or introduce disadvantages for other users), it should be merged.

About some strategic / design decisions:

• I also prefer adding many backends though a powerful framework. But it could be also two or more frameworks and cherry-picking backends or even individual implementations - whatever gives the best result. I am also very fine with adding support for some storage provider one way and changing it later when other alternatives give more advantages.
• Currently, all backends are implemented in the rustic_core crate, so a PR there is the right place
• I would like to see the backends in a separate crate in future and rustic_core more generic about backends to support. This would also allow developers to self-implement and use their special-backend.
• Moreover I think we should have a way to allow developers to cherry-pick implemented backends. So maybe, we even use multiple crates for all backends or use feature flags to enable/disable backends.

[aawsome]

Note that we

• have included opendal
• have already separated the crates, there is the new crate rustic_backend

Release 0.7.0 is coming soon...

[Sudneo]

I understand the general long-term vision of integrating one backend which abstracts many, in this case, the "optimal" solution would be to contribute to opendal, adding support for B2 and later integrated opendal with rustic_core, but this is definitely above my skills.

I will try to work (slowly :) ) on a PR to add B2 support directly into rustic_core. If I even manage to do that, I guess we can discuss further! Thanks

[aawsome]

Note that we have included opendal in rustic, which will be released soon. S3 support seems to be very mature and is also directly added, all opendal services are accessible via the opendal:<SERVICE> syntax already.
Have a look at latest nightly build

Feedback is very much appreciated!

[Sudneo]

Hey @aawsome that's...awesome!

I tried using b2 backend and it worked like a charm so far. I will do some more experiments, but it seems that opendal was indeed the answer :)

Thanks a lot!