Client Side Caching with Otter

[rueian]

Otter is a high performance lockless cache which uses proactive ttl and s3-fifo eviction policies.

Lockless is an important property that redis server-assisted client-side caching needs because the invalidation messages from redis should be applied as fast as possible. Applying invalidation messages should be the first priority if possible.

But in the current implementation, the pipelining goroutine handing invalidation messages competes with other reader goroutines to acquire the LRU lock. This will delay the invalidations and further block the pipeline. Using otter can solve this.

[ouvaa]

@rueian provide an option to use https://github.com/phuslu/lru?
otter is slow on set but fast on get, phuslu lru is more balanced and uses lesser mem

meaning for cache write intensive workload, better to use phuslu, for read intensive, better to use otter

[1a1a11a]

@ouvaa Do you have insights on why phuslu is better at write?

[rueian]

Hi @ouvaa, rueidis is read intensive but writes should be prioritized.

[sshankar]

@rueian Do you have a work in progress branch with otter we can test with?

[rueian]

Hi @sshankar, sorry for my late reply.

Unfortunately, my progress on this is currently paused. I haven't finished adding singleflight load
mechanism to otter. That is a key feature that otter currently doesn't have. If we build the mechanism outside of otter, we will need to pay high overhead for it.

What makes you interested in otter?