Idea #7477
Updated by Tom Clegg about 9 years ago
The gist of this story is "Make the Go Keep client have the same functionality the Python Keep client gets from #7235." Don't set any hard time limit for a block to be transferred to or from a Keep proxy. Instead, simply ensure that the connection keeps making progress, and fails if it stalls.
h3. Implementation
Make a wrapper for an io.Reader that returns an error if traffic rate falls below a given rate.
The returned reader should
* Be a private type (@type minProgressReader struct {...}@)
* Proxy each @Read()@ call to the underlying reader
* Return an error if the underlying reader goes more than @period@ seconds without returning at least @minBytes@ bytes
* in its @Close()@ method, call @Close()@ on the underlying reader if it implements @io.ReadCloser@, otherwise do nothing
* if given a threshold of 0 bytes or interval of 0 seconds, do not enforce any limit at all, just pass through Read() to the underlying reader.
Should be possible to implement the timer using https://golang.org/pkg/time/#NewTimer
Perhaps something like this
<pre><code class="go">
func NewMinProgressReader(r io.Reader, minBytes int, period time.Duration) io.ReadCloser {
return &minProgressReader{
reader: r,
minBytes: minBytes,
timer: time.NewTimer(period),
}
}
func (r *minProgressReader) Read(buf []byte) (int, error) {
if r.err != nil {
return 0, r.err
}
var n int
var err error
ok := make(chan struct{})
go func() {
n, err = r.reader.Read(buf)
ok <- struct{}{}
}()
select {
case <-ok:
r.haveBytes += n
if r.haveBytes >= r.minBytes {
r.timer.Reset()
r.haveBytes = 0
}
return n, err
case <-timer.C:
r.Close()
r.err = ErrTransferTooSlow
return n, r.err
}
}
</code></pre>
Wrap the relevant readers in @sdk/go/keepclient@ with @NewMinProgressReader()@.
The caller should be able to adjust the #bytes and time interval by setting public fields on a KeepClient. @MakeKeepClient()@ should initialize with reasonable defaults. The existing network timeout settings should be disabled: this will make them redundant.
The current (non-proxy) timeout default is 20 seconds for an entire block transfer, which would be expressed as @minBytes==2<<26 && period==20*time.Second@. Something more like @minBytes==2<<20 && period==20*time.Second@ is probably reasonable for both proxy and non-proxy scenarios.