Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flesh out multithreading story #5

Open
h33p opened this issue Nov 30, 2023 · 0 comments
Open

Flesh out multithreading story #5

h33p opened this issue Nov 30, 2023 · 0 comments

Comments

@h33p
Copy link
Member

h33p commented Nov 30, 2023

Currently mfio has only been properly used on a single threaded environment. There is no reason why sharing an arc to a backend wouldn't work across multiple threads (so long as one thread is responsible for driving the backend), however, this would likely be incredibly inefficient design.

In addition, we are probably losing performance through locks and alike by ensuring our objects are Send everywhere - maybe that's not really needed?

What I'm currently leaning towards is encouraging thread-per-core design, or in work-stealing async runtime PoV - backend-per-task. Meaning, there's one backend per top level future.

However, if we stick with backend-per-thread, we can spawn the backend on a localset, and use it from a global set task, which would lead to rather efficient design. The difficulty arises in moving file handles and alike from one backend to the next, after thread transition. This would be a no-op on thread backend, but would become incredibly complicated to synchronize on others, and in some cases, perhaps impossible to do. Maybe we should just do thread-per-core (make mfio things !Send) and call it a day.

Tl;dr, there are many options, but no clear position was decided upon, and to what extent all of the usecases should be supported.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant