Story #9446

Updated by Peter Amstutz almost 5 years ago

Currently @arvados.keep.KeepClient.put@ creates one KeepWriterThread per keep server and then uses a complex (and error prone) locking strategy implemented in ThreadLimiter to ensure that only certain threads perform uploads in a certain order.

Refactor this code to use the following alternate strategy:

* For N wanted copies create N upload threads
* Start a new upload to the next server in sorted_roots when an upload fails

You may want to use a Queue (instead of explicit locks) to communicate between the main thread and the upload threads.

Consider setting up a thread pool attached to the keep client object and dispatching work instead of spawning new threads.

Back