Add virtualised filesystem example

[dreamliner787-9]

This PR adds virtualised filesystem example into LionsOS. Currently, this example only supports qemu_virt_aarch64.

See the README for more details: https://github.com/au-ts/lionsos/tree/fs_driver_vm/examples/vfs

[dreamliner787-9]

Switching to draft for now due to closure of au-ts/libvmm#142

Although 3d16d0f should still compiles and works on QEMU.

Things to fix:

• Make copy of liburing, similar to how lwip was handled.
• Copy FS UIO driver from libvmm into components/fs/linux (or similar).
• Fix vfs example build script to reflect the above.
• Make a physical board target for the vfs example.

[JE-Archer]

What is the rationale for vendoring liburing instead of adding it as a submodule? (I'm not sure we have a rationale for why we do this for lwip.)

[JE-Archer]

Should be "client" not specifically MicroPython.

[Ivan-Velickovic]

What is the rationale for vendoring liburing instead of adding it as a submodule? (I'm not sure we have a rationale for why we do this for lwip.)

It's really only our dependencies that should be submodules, since they update so often. Anything that in the long-term doesn't change much such as lwIP, MicroPython or liburing can just be copy-and-pasted.

Dealing with submodules really doesn't scale well. In general we'll likely have to find another method of getting dependency source code but for now I think we should minimise the amount of submodules.

[JE-Archer]

It's really only our dependencies that should be submodules, since they update so often. Anything that in the long-term doesn't change much such as lwIP, MicroPython or liburing can just be copy-and-pasted.

Our dependencies change frequently. We just don’t update them. 😉

Dealing with submodules really doesn't scale well. In general we'll likely have to find another method of getting dependency source code but for now I think we should minimise the amount of submodules.

I get that they are a pain, but in what sense don’t they scale? And if we are going to keep using them for our own repositories then we’re stuck with them anyway.

Ultimately git-submodule is doing some useful things which make it easy to update dependencies or maintain a fork (like storing the commit hash of the submodule in the repo). If we don’t use submodules then we had better be disciplined about doing those things manually instead of what we have done with lwip. With lwip, we have some unknown version (commit hash not stored) that is at least 2 years old (who knows exactly?), which is actually a fork of upstream with our own changes, but those changes aren’t visible in our commit history because the au-ts sDDF repo doesn’t have the commit history from the original repo.

Suggestion for how to vendor dependencies properly:

1. include both the origin repo URL and the commit hash of the vendored copy;
2. never modify the vendored code directly—it should always just be a cache of the info in repo/commit identified by (1), enforced by CI.

This means that if e.g. we find a bug in liburing, we don't fix the bug with a commit in our own repo. Instead, we would fork liburing at au-ts/liburing, fix the bug there (and open a pull request to upstream) and update au-ts/lionsos to vendor au-ts/liburing with the bugfix commit. When the bugfix is merged upstream, go back to vendoring upstream. Similarly, if we have to maintain a fork, maintain it at au-ts/liburing so that we can easily rebase onto upstream.

This is basically what git-submodule is doing except changing where the work is done. Maybe there is tooling for it.

[lsf37]

Just pointing out here that this discussion has happened before in almost the same words and ended in using Google repo.

[Ivan-Velickovic]

Just pointing out here that this discussion has happened before in almost the same words and ended in using Google repo.

Google repo would be going backwards from where we are now the way I see it.

[lsf37]

You're free to decide whatever you like, I'm just pointing out that I see history repeating. Both options suck. Monorepos also suck. All solutions to this problem that I have ever heard of suck, because ultimately what actually sucks are frequently changing dependencies. You can choose how much flexibility vs overhead you want, but that's pretty much it.

[Ivan-Velickovic]

You're free to decide whatever you like, I'm just pointing out that I see history repeating. Both options suck. Monorepos also suck. All solutions to this problem that I have ever heard of suck, because ultimately what actually sucks are frequently changing dependencies. You can choose how much flexibility vs overhead you want, but that's pretty much it.

You're probably right but the thing that I really don't like about both these solutions is that as a user of LionsOS I have somehow become responsible for managing the project's dependencies. This is true for git submodules as well as the google repo tool.

[Ivan-Velickovic]

Just putting this as a note to my self so I don't merge: we can't store any large artifacts in the repositories.

[Ivan-Velickovic]

Same here.