https://dev.arvados.org/https://dev.arvados.org/favicon.ico?15576888422015-10-19T19:46:14ZArvadosArvados - Bug #7573: Keepstore: very uneven distribution of blob between 2 Keepstore servershttps://dev.arvados.org/issues/7573?journal_id=315302015-10-19T19:46:14ZTom Cleggtom@curii.com
<ul><li><strong>Category</strong> set to <i>SDKs</i></li><li><strong>Status</strong> changed from <i>New</i> to <i>Feedback</i></li></ul><p><a class="issue tracker-1 status-3 priority-4 priority-default closed parent" title="Bug: [SDKs] Python Keep client uses wrong probe order for put() (Resolved)" href="https://dev.arvados.org/issues/6358">#6358</a> contained fixes for two different Python SDK bugs affecting block distribution. One of them is almost certainly the biggest contributor to this problem.</p>
<p>You should get even distribution when the writer (arv-put) is from arvados-python-client-0.1.20151019192928 or newer.</p>
<p>Uploads through keepproxy (including browser uploads) would not have been affected by either of those bugs, so this explanation assumes your arv-put process had direct access to keepstore servers (e.g., it was running on a shell node).</p> Arvados - Bug #7573: Keepstore: very uneven distribution of blob between 2 Keepstore servershttps://dev.arvados.org/issues/7573?journal_id=316022015-10-21T10:20:45ZPeter Grandipg@arvados.for.sabi.co.uk
<ul></ul><blockquote>
<p>our arv-put process had direct access to keepstore servers (e.g., it was running on a shell node).</p>
</blockquote>
<p>Indeed; and with the update mentioned above new uploads are now more evently distributed between the <code>keep0</code> and <code>keep1</code> servers, and are still evenly distributed between the two filetrees 01 and 02 on the <code>keep1</code> server.</p> Arvados - Bug #7573: Keepstore: very uneven distribution of blob between 2 Keepstore servershttps://dev.arvados.org/issues/7573?journal_id=316062015-10-21T13:33:49ZBrett Smithbrett.smith@curii.com
<ul><li><strong>Status</strong> changed from <i>Feedback</i> to <i>Resolved</i></li></ul><p>Peter Grandi wrote:</p>
<blockquote>
<p>Indeed; and with the update mentioned above new uploads are now more evently distributed between the <code>keep0</code> and <code>keep1</code> servers, and are still evenly distributed between the two filetrees 01 and 02 on the <code>keep1</code> server.</p>
</blockquote>
<p>Glad to hear it. We have other feedback that the fix improved distribution as well, so I'm going to mark this as resolved. Thanks for the report, and please don't hesitate to reopen this or file a new report if you see any other funny behavior.</p>