feat(services/opfs): OPFS backend for OpenDAL#7199
Open
jccampagne wants to merge 41 commits intoapache:mainfrom
Open
feat(services/opfs): OPFS backend for OpenDAL#7199jccampagne wants to merge 41 commits intoapache:mainfrom
jccampagne wants to merge 41 commits intoapache:mainfrom
Conversation
65dc6f2 to
225ae90
Compare
Author
|
|
Author
|
It turns out I will probably need the Synch version as well (using FileSystemSyncAccessHandle) for my project. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
This is for OPFS backend: see #5799
Added write, read, list, delete, stat; tests.
I will extend this PR for the missing features (read, stat, delete), if that's ok.
This is still a draft, so there are debug logs, and comments in the code to help with devs; most of them will be removed later.
Rationale for this change
I needed OPFS for my Rust project targetting WASM32 (in the browser).
I discovered OpenDAL but OPFS support was limited, so I decided to implement it.
I managed to make it work for my project (using OpenDAL as a dependency), I had implemented (read, write, stat, delete...). But the code was more like a prototype quality as I was learning OpenDA architecture and API.
This is my 3rd iteration. Still early, and incomplete, but I would like some early feedback if possible to make sure I am on the right track to follow the spirit of OpenDAL. Do not hesitate to comment.
As I understand it, OPFS as 2 kinds of APIs:
synchronous interface: using
FileSystemSyncAccessHandleto read and write, can only be used in workers;asynchronous interface:
GetFile+FileSystemWritableFileStream, can be using in main JS thread and works, but less efficient.I decided to go for the async version as it suits my use case (for now), and it it the most generic use case and requires less setup to use (no workers).
I saw the
TwoWaystrait - that could be used later to implement the alternative using the synch version of OPFS.Using SendWrapper to make some struct Send and Sync.
What changes are included in this PR?
Added write, read, list, delete, stat.
Added edge tests in
core/edge/opfs_wasm32.I ran the tests on 3 different engine:
Are there any user-facing changes?
Addition of write to the OPFS operator.
AI Usage Statement
Yes, Gemini and Claude to navigate the codebase, documentation; and help with some code.