CRDTs

[smebberson]

I was reading https://news.ycombinator.com/item?id=32873172 and wanted to start a discussion here.

CRDT on TinyBase would be an absolutely killer feature. I have a relatively new app that is client side only and implemented in Dexie. It's in private beta. I decided on Dexie just before TinyBase was released but I've been following along with TinyBase and watching progress.

You mentioned v3 might have CRDTs. Is this anything you're working on at the moment. Is there anything I could do to help out with this effort?

[jamesgpearce]

Yep, I am game for this, though I've had to do a bunch of background reading. Still not sure of quite the right strategy yet... but in our favor we have the fact that TinyBase can listen and update on a cellular granularity, reducing the likely conflicts per row or table. eg https://archive.jlongster.com/using-crdts-in-the-wild seems like a worthy avenue.

(In the meantime I want to get some codegen ORM functionality out in a v2.2-ish release, so we have time to discuss)

[tantaman]

We're close to releasing crdt support in SQLite itself -- https://github.com/aphrodite-sh/cf-sqlite if that'd be useful.

If you just want some background, the original design is described in the readme. we've landed on something that doesn't require vector clocks recently, however -- https://tantaman.com/2022-10-18-lamport-sufficient-for-lww.html

[jamesgpearce]

OK, now that v2.2 is out, I want to start thinking about this.

In our favor:

• we have cell-level set/get/listen, so each update is a primitive atom
• del is just set(null) which might mean we can avoid special cases for tomb-stoning
• ...

Against us:

• gotta stay tiny! - so dependent techniques will need to be peer-dependencies or home-written
• lack of clarity (to me) about how we make this architecturally flexible (eg peer-peer or via server)

Premise

My starting point is @jlongster's Actual approach (since I am a mere mortal): a no dependency HLC and a growing log of last-write-wins. https://www.dotconferences.com/2019/12/james-long-crdts-for-mortals

The current checkpoints module is an example of keeping a sequence of cell-level changes like that (and it handles deletes a charm). But I need to think about what a flexible API to get or replay such a log in a CRDT context would look like.

Plan:

• Start hacking on tiny clock and hash code. We're going to need that whatever.
• Start a demo app that syncs peer-to-peer (eg over websockets? idk) just to get a feel of how to get a tiny crdt format going
• Gather as many ideas as I can

I've started a dedicated disposable repo for these half-baked ideas. https://github.com/tinyplex/tinysync

[jamesgpearce]

v4.0 supports Yjs, Automerge, and cr-sqlite!