You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The DataFusion compactor includes a LoggingObjectStore wrapper around the ObjectStore trait. This is responsible for logging requests made to the underlying object store (S3 in our case).
Description
As a base for improving the use of ObjectStore implementations, we want to re-factor the existing implementation of LoggingObjectStore. This is to remove some pointer indirection, add unit tests, and reduce noise in the code. This will form the basis of future work to reduce the number of GET requests to S3 that the DataFusion compactor makes.
Analysis
Resolving this issue in itself should make the compaction code that deals with remote object stores cleaner.
The text was updated successfully, but these errors were encountered:
Background
The DataFusion compactor includes a LoggingObjectStore wrapper around the ObjectStore trait. This is responsible for logging requests made to the underlying object store (S3 in our case).
Description
As a base for improving the use of ObjectStore implementations, we want to re-factor the existing implementation of LoggingObjectStore. This is to remove some pointer indirection, add unit tests, and reduce noise in the code. This will form the basis of future work to reduce the number of GET requests to S3 that the DataFusion compactor makes.
Analysis
Resolving this issue in itself should make the compaction code that deals with remote object stores cleaner.
The text was updated successfully, but these errors were encountered: