Bug #5745
Updated by Tom Clegg almost 9 years ago
h3. background
The most obvious benefit of this change is that, even when keepstore is busy, the process of probing N volumes to find an existing block can proceed quickly, using the OS's filesystem cache, instead of waiting in every volume's serialize queue along the way. With the current implementation, a keepstore can easily accumulate a queue of PUT requests (each consuming a decent chunk of RAM) that are just waiting to find out whether the blocks already exist -- and of course they'll wait in the queue an N+1st time when it's time to actually write them to disk.
h3. solution / implementation
Suggest refactoring the "serialize" implementation to use a sync.Mutex instead of passing channels to a goroutine over a "queue" channel.
In Read, before calling @ioutil.ReadFile@ or doing any locking, do a stat() to see whether the file exists. If not, return an error without acquiring the volume lock.
In Read and Write, just before reading or writing data:
<pre><code class="go">
if v.mutex {
v.mutex.Lock()
defer v.mutex.Unlock()
}
</code></pre>
Then we can rename Write to Put, rename Read to Get, and delete the old Get and Put wrappers, IOHandler, IORequest, IOResponse, and KeepGet/KeepPut constants.