Idea #10996
Updated by Tom Clegg about 7 years ago
h3. Background In sdk/go/keepclient, @(*keepclient.KeepClient)CollectionFileReader(collection map[string]interface{}, filename string)@ returns Go SDK should provide an Open method/func that, for a keepclient.Reader, which implements io.Reader, io.Seeker, and io.Closer but not io.Writer. In services/crunch-run, @(*CollectionWriter).Open(path string)@ given collection, returns an io.WriteCloser, i.e., it can write but not read or seek. h3. Improvement In sdk/go/arvados, a Collection should have Open, Create, and OpenFile methods that correspond to os.Open, os.Create, and os.OpenFile. Open returns a read-only file, so it can be implemented by calling CollectionFileReader. As a starting point, Create and OpenFile can be implemented by refactoring the CollectionWriter bits into the SDK, but the returned object would not be readable or seekable. Then the reading+seeking and writing code need with an interface similar to be reconciled so OpenFile and Create return an object that can read, write, and seek. *os.File (Reader, Writer, Seeker, Closer). h3. Optimization ideas Presumably not implemented at the same time, but can be kept in mind so we don't burn too many bridges: improvements * BlockCache could use AsyncStream, e.g. 10990-keep-web-ranges-pa * Could cache smaller portions of blocks