Simplify the abstraction of evict policy

[xiaguan]

Regarding the abstraction of eviction in foryer-memory, i.e., the Eviction trait, I feel it is somewhat over-abstracted. We should simplify some of its interfaces and the use of generics to reduce the development and maintenance costs of the eviction algorithms.

Here are two references:

• https://github.com/cloudflare/pingora/blob/main/pingora-cache/src/eviction/mod.rs
• https://github.com/xiaguan/page_cache/blob/master/src/policy.rs

Looking forward to everyone's opinions."

[xiaguan]

I'm wondering why LRU evction in foyer-memory doesn't impl access().
And I can't understand the handle cleary, can you simplely explain it for me? Thanks.

[xiaguan]

I think we should consider refactoring foryer-memory 🤔 .
One reason is that our abstractions are somewhat complex, and the lack of documentation makes development and maintenance quite challenging 😭 .
Another reason is that the use of generics and unsafe might not be user-friendly for future users 🤕 .
Do you have any thoughts on how to address these issues step by step? I am also very willing to make my contribution 🥰

[MrCroxx]

I'm wondering why LRU evction in foyer-memory doesn't impl access(). And I can't understand the handle cleary, can you simplely explain it for me? Thanks.

The latest used LRU entry is moved when releasing, instead of accessing. When serving as an LSM-tree block cache, a block iterator can hold a block for a much longer time than other operations. If the entry is moved when accessing, a block that is held by a slow iterator will be evicted too soon after it is unheld.

[MrCroxx]

One reason is that our abstractions are somewhat complex.

I agree. But some of the complexity comes from the use cases. We can discuss about where you want to refactor before you do it to avoid misunderstandings. 🥰

lack of documentation makes development and maintenance quite challenging 😭

Sorry about that. I will write some example and docs for it. For now, you can refer to how foyer-memory is used in RisingWave. block cache and meta cache

Another reason is that the use of generics and unsafe might not be user-friendly for future users 🤕 .

The unsafe is used for implementing intrusive data structures. To achieve better performance (e.g. remove a entry from a linked list with a entry pointer) and solve the multi-container problem (one entry is held by both the eviction container and the indexer), the intrusive data structure is a good choice to avoid redundant memory usage, memory copy and unnecessary lock usage to make the ownership checker happy.

The complexity is hidden behind the safe abstractions. It will not bother the user (and as you see, a naive user can focus on 2 generic types only). I think it is worth bothering the developer to provide better performance for users.

Do you have any thoughts on how to address these issues step by step?

As the saying goes, the onlooker sees clearly, but those involved are often confused. Ideas are always welcomed.