Currently mfio has only been properly used on a single threaded environment. There is no reason why sharing an arc to a backend wouldn't work across multiple threads (so long as one thread is responsible for driving the backend), however, this would likely be incredibly inefficient design.
In addition, we are probably losing performance through locks and alike by ensuring our objects are Send everywhere - maybe that's not really needed?
What I'm currently leaning towards is encouraging thread-per-core design, or in work-stealing async runtime PoV - backend-per-task. Meaning, there's one backend per top level future.
However, if we stick with backend-per-thread, we can spawn the backend on a localset, and use it from a global set task, which would lead to rather efficient design. The difficulty arises in moving file handles and alike from one backend to the next, after thread transition. This would be a no-op on thread backend, but would become incredibly complicated to synchronize on others, and in some cases, perhaps impossible to do. Maybe we should just do thread-per-core (make mfio things !Send) and call it a day.
Tl;dr, there are many options, but no clear position was decided upon, and to what extent all of the usecases should be supported.
Currently
mfiohas only been properly used on a single threaded environment. There is no reason why sharing an arc to a backend wouldn't work across multiple threads (so long as one thread is responsible for driving the backend), however, this would likely be incredibly inefficient design.In addition, we are probably losing performance through locks and alike by ensuring our objects are
Sendeverywhere - maybe that's not really needed?What I'm currently leaning towards is encouraging thread-per-core design, or in work-stealing async runtime PoV - backend-per-task. Meaning, there's one backend per top level future.
However, if we stick with backend-per-thread, we can spawn the backend on a localset, and use it from a global set task, which would lead to rather efficient design. The difficulty arises in moving file handles and alike from one backend to the next, after thread transition. This would be a no-op on thread backend, but would become incredibly complicated to synchronize on others, and in some cases, perhaps impossible to do. Maybe we should just do thread-per-core (make mfio things
!Send) and call it a day.Tl;dr, there are many options, but no clear position was decided upon, and to what extent all of the usecases should be supported.