[Feature requirement] Support extra read on core to warm cache

[beef9999]

Hello maintainers, we are using OCF to develop a read-only remote cache, therefore, the core is accessed by remote network and the cache is in local.

OCF's minimum read size is the sector size (512B), and that leads to low efficiency because we have to visit remote core too frequently. A desired way to fix this problem is to read extra data from core (sometimes called prefetch) while cache misses.

Our current implementation (or walk-around) is purely at the upper side. For example, starting a background read task every time the core is read, within a designated nearby range. But it may still cause duplicate read, because user doesn't know if any part of range is already filled, based on the existing OCF APIs. Maybe it should be done in the lower side (OCF itself), with a large and continuous read, and skip the filled range automatically?

So do you have plan to add this support? Or at least provide some APIs to tell if the specified cache lines will hit.

[beef9999]

@michalwy

[robertbaldyga]

Hi @beef9999 . Currently we do not have any plans to implement prefetch. However we surely can consider this.

From technical perspective implementing prefetch in OCF comes with some trade-offs. Normally OCF uses user provided buffers to read data from core, while for prefetch it would need to allocate bigger buffer and then perform memcpy to user buffer. That would increase both latency and CPU utilization.

• Is that additional latency / CPU utilization cost is acceptable in your scenario?

• Is your application able to provide any hint about when prefetch would be needed? (That would potentially decrease amount of memcpy operations.)

• How big prefetches would be needed? Close to cache line size of much bigger?

• What is expected timeframe? When this feature would be needed?

[lihuiba]

• Is that additional latency / CPU utilization cost is acceptable in your scenario?

Yes. As the core is remote, even possibly geographically distributed, extra resource is definitely acceptable.

• Is your application able to provide any hint about when prefetch would be needed? (That would potentially decrease amount of memcpy operations.)

A simplest approach is a configurable larger refill unit, e.g. 64KB, so that cache will read data from core in unit of 64KB (also aligned to that).
Higher level approaches are also desirable, such as intelligent prefetching based-on realtime access pattern, or trace-based prefetching.

• How big prefetches would be needed? Close to cache line size of much bigger?

It's better configurable. WAN needs bigger prefetches, while LAN needs smaller ones.

[beef9999]

@robertbaldyga

The containerd/overlaybd project is focused on next generation container remote image, and cache has been playing an important role in the whole architecture. We have just released a new file cache that was built on top of OCF:

Overview
Document
Code

What is expected timeframe? When this feature would be needed?

The soon the better... For now the performance of the new ocf cache is not satisfying as I mentioned above, because of the lack of a proper prefetch. So we didn't make it the default cache choice.

I think the priorities to solve this issues would be:

1. OCF provides an API to query metadata hit
2. OCF supports larger metadata, like 512KB
3. OCF supports prefetch internally

3 is optional. I believe with 1 and 2, I myself can then implement an efficient prefetch.

[beef9999]

https://github.com/containerd/overlaybd/blob/ecd15832005c0243bf678146a1d7323d83409113/src/overlaybd/fs/cache/ocf_cache/ease_bindings/volume.cpp#L106-L111

This code shows how we do prefetch now in the app side, i.e. , issue a new read in the nearby range every time the core is visited.

[gaowayne]

@beef9999 I am the owner the PRC for WSR and OCF, could you please send me one email? [email protected], we can chat a little bit about your requirement and opty size for Xeon CPU and Optane SSD.

[beef9999]

Any update on this issue? Will OCF provide an API to query metadata hit?

[robertbaldyga]

@beef9999 I think we should create three separate GitHub issues for each of those features (with "enhancement" label). It will make it easier to discuss about more specific details of each of them. Currently we do not have any definite date when those features would be implemented. We will revisit them planning future releases.

[robertbaldyga]

This enhancement has been split into three separate entries: #674, #675 and #676. I'm closing the original one.