-
Notifications
You must be signed in to change notification settings - Fork 16
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Reuse the same reader across ReadChunk calls so that prefetched block…
…s are not discared. In the sequential non-parallel path (ReadRecord->ReadAheadFromBuffer->ReadChunk) ReadChunk reads the whole chunk into memory. Most readers will round up the read to the blocksize and thereby prefetch some data from the next chunk. Currently ReadChunk recreates the reader each time and the previously prefetched data is lost and must be read again. Instead use get_backing_reader() directly each time. We can do this since ReadChunk is only called in single threaded context (Initialize, ReadRecord (parallelism disabled)) so there cannot be any concurrent access to the underlying reader. ReadTrace (before): ``` offset=64, size=1M <-- 1st chunk offset=1M, size=1M ... offset=8M, size=1M <-- block aligned read crosses chunk boundary offset=8.3M, size=0.7M <-- redundant read for 2nd chunk offset=9M, size=1M ... ``` ReadTrace (after): ``` offset=64, size=1M <-- 1st chunk offset=1M, size=1M ... offset=8M, size=1M <-- block aligned read crosses chunk boundary and is reused for the 2nd chunk offset=9M, size=1M ... ``` PiperOrigin-RevId: 577312179
- Loading branch information
1 parent
b1403b5
commit fa16989
Showing
2 changed files
with
15 additions
and
15 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters