https://dev.arvados.org/https://dev.arvados.org/favicon.ico?15576888422014-02-28T13:23:48ZArvadosArvados - Idea #1885: Keep proxy service: forward external requests to the appropriate keep servershttps://dev.arvados.org/issues/1885?journal_id=81812014-02-28T13:23:48ZTom Cleggtom@curii.com
<ul><li><strong>Subject</strong> changed from <i>Keep can be accessed from outside the cluster, e.g., using some sort of gateway or proxy</i> to <i>Keep federation: proxy service forwards external requests to the appropriate keep server(s)</i></li></ul> Arvados - Idea #1885: Keep proxy service: forward external requests to the appropriate keep servershttps://dev.arvados.org/issues/1885?journal_id=81822014-02-28T13:26:13ZTom Cleggtom@curii.com
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/8182/diff?detail_id=5993">diff</a>)</li></ul> Arvados - Idea #1885: Keep proxy service: forward external requests to the appropriate keep servershttps://dev.arvados.org/issues/1885?journal_id=99982014-04-29T09:24:34ZTom Cleggtom@curii.com
<ul><li><strong>Subject</strong> changed from <i>Keep federation: proxy service forwards external requests to the appropriate keep server(s)</i> to <i>Keep proxy service forwards external requests to the appropriate keep servers</i></li><li><strong>Description</strong> updated (<a title="View differences" href="/journals/9998/diff?detail_id=8080">diff</a>)</li><li><strong>Story points</strong> set to <i>3.0</i></li></ul> Arvados - Idea #1885: Keep proxy service: forward external requests to the appropriate keep servershttps://dev.arvados.org/issues/1885?journal_id=100022014-04-29T09:26:47ZTom Cleggtom@curii.com
<ul><li><strong>Project</strong> changed from <i>35</i> to <i>Arvados</i></li><li><strong>Subject</strong> changed from <i>Keep proxy service forwards external requests to the appropriate keep servers</i> to <i>Keep proxy service: forward external requests to the appropriate keep servers</i></li></ul> Arvados - Idea #1885: Keep proxy service: forward external requests to the appropriate keep servershttps://dev.arvados.org/issues/1885?journal_id=100052014-04-29T09:47:37ZTom Cleggtom@curii.com
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/10005/diff?detail_id=8089">diff</a>)</li></ul> Arvados - Idea #1885: Keep proxy service: forward external requests to the appropriate keep servershttps://dev.arvados.org/issues/1885?journal_id=102322014-05-07T11:17:35ZTom Cleggtom@curii.com
<ul><li><strong>Target version</strong> set to <i>2014-05-28 Pipeline Factory</i></li></ul> Arvados - Idea #1885: Keep proxy service: forward external requests to the appropriate keep servershttps://dev.arvados.org/issues/1885?journal_id=102482014-05-07T11:38:22ZTom Cleggtom@curii.com
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/10248/diff?detail_id=8380">diff</a>)</li><li><strong>Story points</strong> changed from <i>3.0</i> to <i>5.0</i></li></ul> Arvados - Idea #1885: Keep proxy service: forward external requests to the appropriate keep servershttps://dev.arvados.org/issues/1885?journal_id=103502014-05-07T15:42:47ZPeter Amstutzpeter.amstutz@curii.com
<ul><li><strong>Assigned To</strong> set to <i>Peter Amstutz</i></li></ul> Arvados - Idea #1885: Keep proxy service: forward external requests to the appropriate keep servershttps://dev.arvados.org/issues/1885?journal_id=103902014-05-08T09:39:02ZPeter Amstutzpeter.amstutz@curii.com
<ul><li><strong>Story points</strong> changed from <i>5.0</i> to <i>3.0</i></li></ul> Arvados - Idea #1885: Keep proxy service: forward external requests to the appropriate keep servershttps://dev.arvados.org/issues/1885?journal_id=107092014-05-22T16:42:10ZWard Vandewegeward@curii.com
<ul></ul><p>go.sh: you need to force GOPATH to be absolute, for example like this:</p>
<pre><code>rootdir=$(dirname $(readlink -f $0))</code></pre>
<p>otherwise, go blows up like this if you run go.sh from a symlinked path:</p>
<pre><code>go: GOPATH entry is relative; must be absolute path: ".".<br /> Run 'go help gopath' for usage.</code></pre>
<p>go.sh: needs documentation for arguments required. Is go.sh build keep what you are intending it to be run as?</p>
<p>go.sh: does not seem to replicate build.sh's functionality - it does not put the keep binary in bin/. Or maybe I'm not calling it the right way, cf. the previous point.</p>
<p>services/keep/src/arvados.org/keepproxy/keepproxy.go: has some commented out code that should be removed</p>
<p>services/keep/src/arvados.org/keepproxy/keepproxy.go: don't hardcode https on line 165, that should be configurable (but default to https) or ideally auto-discovered</p>
<p>services/keep/src/arvados.org/keepproxy/keepproxy_test.go: same, don't hardcode https on line 74</p>
<p>services/keep/src/arvados.org/keepproxy/keepproxy_test.go: these two commands need to check for failure and print the command output if run_test_server.py fails:</p>
<pre><code>exec.Command("python", "run_test_server.py", "start").Run()<br /> exec.Command("python", "run_test_server.py", "start_keep").Run()</code></pre>
<p>FYI - I'm planning to restrict the number of concurrent connections to the Keep Proxy at the nginx level.</p>
<p>Any thoughts on expected memory use patterns of keep proxy? I assume the worst case scenario (in put mode) would be many clients all writing full 64MiB blocks, slowly. That scenario is what the limit on concurrent connections will protect against.</p>
<p>Please write a log line for every GET/HEAD/PUT request to STDOUT. Something simple will do for now. In Keep we use timestamp method /hash at the moment, and that's plenty fine.</p>
<p>Other than that, I'd say this looks pretty good! Tests pass.</p> Arvados - Idea #1885: Keep proxy service: forward external requests to the appropriate keep servershttps://dev.arvados.org/issues/1885?journal_id=107212014-05-23T10:23:39ZPeter Amstutzpeter.amstutz@curii.com
<ul></ul><ol>
<li>Added comment to go.sh. Go.sh is just a wrapper around the go toolchain so regular go commands apply.</li>
<li>"go install arvados.org/keepproxy" should build keepproxy and copy it to bin/</li>
<li>The commented out code was to force the user to disable GET, which in Tom's original writeup was not supposed to be implemented (but I did it anyway). I'm not clear on the status of keep permissions, in theory the proxy supports it but I haven't tested it.</li>
<li>Discussed https when calling API server on IRC</li>
<li>Checks error return on .Run() now in the tests</li>
<li>Regarding memory usage:
<ul>
<li>GET requests should stream through with minimal buffering. It effectively goes directly from the http.Response reader from the Keep server directly into the http.ResponseWriter that sends to the client, with only a small stub that incrementally tallys the md5 checksum. The buffer used by the Go http library is something like 32 KiB</li>
<li>PUT requests pre-allocate a buffer based on the Content-Length in the PUT header, and that buffer is held until the block has been sufficiently replicated or it gives up after contacting all Keep servers. That means we probably do want a relatively conservative cap on the number of concurrent PUT requests allowed, e.g. 16 * 64 MiB will require 1 GiB of RAM.</li>
</ul>
</li>
<li>Added a bunch of logging, and uncovered a rather serious and stupid bug in not correctly failing on authorization errors :-)</li>
</ol> Arvados - Idea #1885: Keep proxy service: forward external requests to the appropriate keep servershttps://dev.arvados.org/issues/1885?journal_id=107262014-05-23T11:38:07ZWard Vandewegeward@curii.com
<ul></ul><p>This looks good to merge to me; just one change please: also log requests that are 404 or fail in other ways. It can just be the same log line as for a successful request for now, unless you want to upgrade it a little bit and add the HTTP response code in the log line, that would be nice.</p> Arvados - Idea #1885: Keep proxy service: forward external requests to the appropriate keep servershttps://dev.arvados.org/issues/1885?journal_id=107302014-05-23T15:55:10ZAnonymous
<ul><li><strong>Status</strong> changed from <i>New</i> to <i>Resolved</i></li></ul><p>Applied in changeset arvados|commit:ca77ae5cb94c31232abc8923174280fd848ef408.</p>