Idea #2853
Updated by Tom Clegg over 9 years ago
See: There is a growing consensus that rendezvous hashing is a more appropriate solution for this problem than consistent hashing: http://en.wikipedia.org/wiki/Rendezvous_hashing Some thoughts here from earlier technical discussions: * Rendezvous hashing makes it harder to perform weighted block distribution (we may be able to get around this by adding virtual storage nodes). Currently we do not expect to want weighted block distribution. Rather, we want distribution to be equal in steady state, and we want to minimize excess probes during rebalancing. nodes) * Computing N hashes per block transaction (one for each keep node or virtual node) seems acceptable as long as N isn't huge; the more precise you want your weighting to be, the more hashes you have to compute. Using a fast hash function would be helpful here. * If the map of storage nodes is tree structured, you can get O(log n) performance, e.g. compare group1 to group2, then follow the branch and compare group3 to group4, etc. However, this makes it impossible to add an arbitrary number of storage nodes with equal weight. etc