Project

General

Profile

Idea #7477

Updated by Tom Clegg over 8 years ago

The gist of this story is "Make the Go Keep client have the same functionality the Python Keep client gets from #7235."    Don't set any hard time limit for a block to be transferred to or from a Keep proxy.    Instead, simply ensure that the connection keeps making progress, and fails if it stalls. 

 h3. Implementation 

 Make a wrapper for an io.Reader that returns an error if traffic rate falls below a given rate. 

 The returned reader should 
 * Be a private type (@type minProgressReader struct {...}@) 
 * Proxy each @Read()@ call to the underlying reader 
 * Return an error if the underlying reader goes more than @period@ seconds without returning at least @minBytes@ bytes 
 * in its @Close()@ method, call @Close()@ on the underlying reader if it implements @io.ReadCloser@, otherwise do nothing 
 * if given a threshold of 0 bytes or interval of 0 seconds, do not enforce any limit at all, just pass through Read() to the underlying reader. 

 Should be possible to implement the timer using https://golang.org/pkg/time/#NewTimer 

 Perhaps something like this 

 <pre><code class="go"> 
 func NewMinProgressReader(r io.Reader, minBytes int, period time.Duration) io.ReadCloser { 
	 return &minProgressReader{ 
		 reader:     r, 
		 minBytes: minBytes, 
		 timer:      time.NewTimer(period), 
	 } 
 } 

 func (r *minProgressReader) Read(buf []byte) (int, error) { 
	 if r.err != nil { 
		 return 0, r.err 
	 } 
	 var n int 
	 var err error 
	 ok := make(chan struct{}) 
	 go func() { 
		 n, err = r.reader.Read(buf) 
		 ok <- struct{}{} 
	 }() 
	 select { 
	 case <-ok: 
		 r.haveBytes += n 
		 if r.haveBytes >= r.minBytes { 
			 r.timer.Reset() 
			 r.haveBytes = 0 
		 } 
		 return n, err 
	 case <-timer.C: 
		 r.Close() 
		 r.err = ErrTransferTooSlow 
		 return n, r.err 
	 } 
 } 
 </code></pre> 

 Wrap the relevant readers in @sdk/go/keepclient@ with @NewMinProgressReader()@. 

 The caller should be able to adjust the #bytes and time interval by setting public fields on a KeepClient. @MakeKeepClient()@ should initialize with reasonable defaults. The existing network timeout settings should be disabled: this will make them redundant. 

 The current (non-proxy) timeout default is 20 seconds for an entire block transfer, which would be expressed as @minBytes==2<<26 && period==20*time.Second@. Something more like @minBytes==2<<20 && period==20*time.Second@ is probably reasonable for both proxy and non-proxy scenarios. 

Back