You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently mfio has only been properly used on a single threaded environment. There is no reason why sharing an arc to a backend wouldn't work across multiple threads (so long as one thread is responsible for driving the backend), however, this would likely be incredibly inefficient design.
In addition, we are probably losing performance through locks and alike by ensuring our objects are Send everywhere - maybe that's not really needed?
What I'm currently leaning towards is encouraging thread-per-core design, or in work-stealing async runtime PoV - backend-per-task. Meaning, there's one backend per top level future.
However, if we stick with backend-per-thread, we can spawn the backend on a localset, and use it from a global set task, which would lead to rather efficient design. The difficulty arises in moving file handles and alike from one backend to the next, after thread transition. This would be a no-op on thread backend, but would become incredibly complicated to synchronize on others, and in some cases, perhaps impossible to do. Maybe we should just do thread-per-core (make mfio things !Send) and call it a day.
Tl;dr, there are many options, but no clear position was decided upon, and to what extent all of the usecases should be supported.
The text was updated successfully, but these errors were encountered:
Currently
mfio
has only been properly used on a single threaded environment. There is no reason why sharing an arc to a backend wouldn't work across multiple threads (so long as one thread is responsible for driving the backend), however, this would likely be incredibly inefficient design.In addition, we are probably losing performance through locks and alike by ensuring our objects are
Send
everywhere - maybe that's not really needed?What I'm currently leaning towards is encouraging thread-per-core design, or in work-stealing async runtime PoV - backend-per-task. Meaning, there's one backend per top level future.
However, if we stick with backend-per-thread, we can spawn the backend on a localset, and use it from a global set task, which would lead to rather efficient design. The difficulty arises in moving file handles and alike from one backend to the next, after thread transition. This would be a no-op on thread backend, but would become incredibly complicated to synchronize on others, and in some cases, perhaps impossible to do. Maybe we should just do thread-per-core (make mfio things
!Send
) and call it a day.Tl;dr, there are many options, but no clear position was decided upon, and to what extent all of the usecases should be supported.
The text was updated successfully, but these errors were encountered: