feat: support gzip & zstd compression #599
Conversation
kemingy
commented
Nov 17, 2024
kemingy
commented
- close feat: Add compression encoding support #597
Signed-off-by: Keming <[email protected]>
|
This passed the test, but I found that there is no "Content-Encoding" in the response headers during the test, while manually running a script can get the full headers as expected: |
mosec/args.py
Outdated
|
|
||
| parser.add_argument( | ||
| "--compression", | ||
| help="Enable Zstd & Gzip compression for the request body", |
There was a problem hiding this comment.
Zstd & Gzip are enabled together? They are two algorithms as I understand?
There was a problem hiding this comment.
Yes. The compression layer will choose the algorithm according to the request headers.
I enabled both since gzip is widely used (included in Python std) and zstd is currently the best. Users can choose the one that better suits their use cases.
There was a problem hiding this comment.
Cool! Can we add an example (similar to the test script) and a pointer in README to make this feature more visible?
There was a problem hiding this comment.
I'm still unsure what the best use case for compression is in model serving. Numpy vectors to bytes can benefit a little. Images that are using compression (JPEG) should not be applied again. @aseaday do you have suggestions?
There was a problem hiding this comment.
Numpy vectors can benefit a lot in our cases.
We had a feature image translator in our product immersivetranslate.
To fullfill image translator, there is a ML model pipeline: detection, recgonize and inpaint. It is also a common desgin for image OCR.
The variables between step in this pipeline are a msgpack format numpy data such as a image mask. A image mask is [3,1024,1024] array. Before compression, the file size will be 6MB, after compression, the filee size will be 2MB because of Sparsity
Signed-off-by: Keming <[email protected]>
Signed-off-by: Keming <[email protected]>
