Web-capable Box API; based on IndexedDB

[greenrobot]

To support the Flutter Web platform, we need an alternative to native libraries, as wasm does not seem to cover everything the native libs do; e.g. "proper" file APIs.

To provide a minimal valuable implementation, implement the Box API using IndexedDB (and FlatBuffers similar to how we use it in the native side). No relations, indexes etc.

Once that works, open a ticket to do simple queries next.

[vaind]

Yes, that shouldn't be too hard to do. I've already thought about this a little a while back and how I've seen this done in Dart is:

• extract interfaces of the public APIs (Store, Box for now) instead of concrete classes
• have an implementation of those interfaces depending on the platform using conditional imports

[vaind]

Also a thing to consider: not all browsers support IndexedDB, so if we wanted to have a complete solution, we'll need (to do) something like https://pub.dev/packages/idb_shim

[greenrobot]

With like 97% of browsers supporting it we might be fine.

Another thing to consider: if we'd have the lower-level implemented in Type-/JavaScript, we could use that for plain web apps too. That probably makes sense when we start adding more features like queries.

[vaind]

Since this would require API changes (switch to interfaces) we should try to prepare those prior to 1.0, with the actual implementation coming later as a minor release.

[vaind]

if we'd have the lower-level implemented in Type-/JavaScript, we could use that for plain web apps too.

notes:

• to implement interaction with JavaScript (i.e. call a objectbox-js impelmentation from Dart) there's dart:js and package:js, looks like the latter supersedes the former. Regarding the performance of such an interop - couldn't find anything useful yet. From my understanding, there shouldn't be much cost to calling javascript, considering the rest of the dart code is compiled to javascript as well, maybe just some checks/data validations before crossing the boundary to the code generated by dart2js.
• there's no support to integrate dart with TypeScript (maybe except if you compiled your TS to JS but I don't know how that works), see this ancient Dart SDK issue.
• code written in Dart can be compiled to JS (dart2js), albeit at some size costs. From what I've read, Dart code compiled to JS should be faster than the "same" code in JS due to the optimizations Dart is able to do, e.g. because of its type system.

[greenrobot]

1. interfaces, yes. separate issue?

2. According to https://stackoverflow.com/a/26428441/551269, you may be able to call TypeScript just like JS. Would suggest a prototype.

[vaind]

you may be able to call TypeScript just like JS

As long as you write TypeScript just like JS, e.g. no generics (Box)

[vaind]

Turns out we don't actually need interfaces, just two variants of the concrete classes - the conditional import ensures the right implementation is imported. See #189

[greenrobot]

While we might not need interfaces (at least for now), it might be that we want them for the flexibility. Just tossing in that thought; more a discussion for later, I guess.

[Buggaboo]

I think that conditional is a compile-time thingy, it's not polymorphic dynamic binding.

[Coinners]

Any news?

[greenrobot]

While we'll keep the first steps quite simple, we should also discuss the target architecture we'll want in the end. To start this, I could imagine the following layers from top to bottom (let me know you see are technical show stoppers):

• Very thin Dart wrapper API (e.g. Box)
• TypeScript API (exposing a Box API, doing the interaction with the "native" WebAssembly layer)
• ObjectBox WebAssembly (the core native parts and ObjectBox Sync)
• JS/TypeScript HAL to provide the features we cannot use in WebAssembly
  • Key/Value storage based on IndexDB
  • Networking if required (not sure what wasm offers here)

While this is a more "heavy" approach, it would allow us to use the entire existing ObjectBox functionality (including Sync).

[zs-dima]

SQLite solution looks more reliable than IndexDB.
Moor support Web with SQLite already.

[vaind]

SQLite solution looks more reliable than IndexDB.
Moor support Web with SQLite already.

Do you have a reference for that? I don't know moor deeply but from a quick look at its web-only code it looks like it supports local storage and indexeddb as a backend: https://github.com/simolus3/moor/blob/develop/moor/lib/src/web/storage.dart

[greenrobot]

Regarding moor: following their docs I landed here: https://github.com/sql-js/sql.js/ which "doesn't persist the changes made to the database", which would not exactly make a good K/V storage for us…

[zs-dima]

@greenrobot

1. SQLite is ACID and IndexedDB is NOT ACID
2. https://github.com/sql-js/sql.js/ text follows: "allows you to import any existing sqlite file, and to export the created database"
   So SQLite persistence depends on implementation

[vaind]

1. SQLite is ACID

Correct, but sql-js isn't, being in-memory it's not durable.

[zs-dima]

@vaind
https://github.com/sql-js/sql.js/ text follows: "allows you to import any existing sqlite file, and to export the created database"
So SQLite DB could be saved to disk as well

[greenrobot]

@zs-dima I'm sorry, that does not help. I think you lack some background on the proposed stack and the purpose of the lowest (HAL/storage) layer. sql.js does not look like a fit becuse it offers SQL without storage, which is the exact opposite of what we are looking for. We rather need storage without SQL. 🙂

[greenrobot]

We rather need storage without SQL.

Or, no storage and NoSQL 🙃 Seriously, the idea of being in-memory only might become interesting, at least as an intermediate solution: we could use a native in-memory K/V. But not sure if that is really less work.

[vaind]

@richard457 commented in #190:
Found this This if it can be of help. !

(Responding here where the architecture discussion takes place.)

That looks like a good approach if we could get the ObjectBox core compiled with Emscripten. In that case, no JS/TS wrapper would be necessary at all. @greenrobot do you think it makes sense to try this approach (compile the core with Emscripten)?

[richard457]

@greenrobot did you check on the #190 (comment) and #185 (comment) ? what do you think?

[greenrobot]

My understanding a few months back was that ObjectBox cannot run in WebAssembly unmodified due to missing mmap functionality in wasm. If people are willing to investigate more, I'd suggest to create another issue that's just about wasm.

[gustavorozolin]

I need migrate my App's local db and look like ObjectBox fit perfectly but my app require web support :(

Any previews for this feature?

[Macacoazul01]

@greenrobot isar/isar#380

[ThaDaVos]

Curious as to what's the state of this? Or should one create an interface/adapter themselves and use something else for storage in the web?

[greenrobot]

A quick update on this: we are currently working on an alternative lower layer for the database. While this is not directly related solution this issue, it would fit the use case and together with wasm, we might make it work for Dart/Flutter too. This won't use IndexedDB but plain files. Don't expect anything soon yet, however...

[Paroca72]

No have a way to use the objectbox library as a memory DB?
On web I understand the problem to save the data persistently but maybe make it work in memory is something feasible.
In this case I could just continue to use the Objectbox library structure and sync the data with a remote server.

If I recall good Isar v4 already do something like that.
Cannot store data permanently on Web but you can still use it as in-memory db.