[Keep] Support transparent compression of blocks in Keep
Support automatic compression of blocks in the Keep server. Proposed design:
- Keep server can accept PUT blocks compressed with gzip, or use gzip to compress uncompressed blocks before saving to disk. Compare compressed/uncompressed sizes to ensure that compression isn't adding unnecessary overhead.
- On GET, Keep clients provide the "Accept-Encoding: gzip" header, and the server responds with "Content-Encoding: gzip" and spools the compressed data directly off disk.
- Keep client decompresses the data before delivering it to the application.
- Support random access into large files without needing special file formats or explicit application support.
- Reduce disk and network usage across the board.
- Transparent to user
- Adds a "decompress and then re-compress on Keep block boundaries" step when working with a collection that's already compressed at the file level.
- May increase latency and client overhead because each block needs to be decompressed in order to use it