Feature #16519
Updated by Ward Vandewege almost 4 years ago
There is now a Go package to speed up md5sum calculations when the hardware supports it (AVX/AVX2 extensions, which are common):
https://github.com/minio/md5-simd
which is described here:
https://blog.min.io/accelerating-aggregate-md5-hashing-up-to-800-with-avx512-2/
Keepstore should leverage this library to speed up its hashing, if the hardware it runs on supports the necessary extensions.
Ideally, this goes into our codebase in a such a way that all our Go code that calculates md5sums leverages it automatically.