https://dev.arvados.org/https://dev.arvados.org/favicon.ico?15576888422022-06-13T18:13:59ZArvadosArvados - Bug #19192: WebDAVCache not performing as expected for S3 requestshttps://dev.arvados.org/issues/19192?journal_id=1043042022-06-13T18:13:59ZTom Cleggtom@curii.com
<ul><li><strong>Status</strong> changed from <i>New</i> to <i>In Progress</i></li></ul> Arvados - Bug #19192: WebDAVCache not performing as expected for S3 requestshttps://dev.arvados.org/issues/19192?journal_id=1043052022-06-13T18:15:27ZTom Cleggtom@curii.com
<ul><li><strong>File</strong> <a href="/attachments/3020">keep-web</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/3020/keep-web">keep-web</a> added</li></ul> Arvados - Bug #19192: WebDAVCache not performing as expected for S3 requestshttps://dev.arvados.org/issues/19192?journal_id=1043622022-06-20T14:52:07ZTom Cleggtom@curii.com
<ul></ul><p>Debug logging indicates (*cache)pruneSessions() is not running, which causes expired sessions to get stuck in the cache forever. I'm guessing there's a situation where fs.MemorySize() deadlocks, and once that happens, pruneSessions can't continue/resume, and we effectively have no cache.</p>
<p>SIGABRT would produce a stack trace to confirm this and show where it's getting stuck.</p> Arvados - Bug #19192: WebDAVCache not performing as expected for S3 requestshttps://dev.arvados.org/issues/19192?journal_id=1043712022-06-20T16:46:07ZTom Cleggtom@curii.com
<ul><li><strong>File</strong> <a href="/attachments/3023">keep-web</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/3023/keep-web">keep-web</a> added</li></ul><p>Here's a debug version that will dump a stack trace and exit if cache pruning takes longer than 30s.</p> Arvados - Bug #19192: WebDAVCache not performing as expected for S3 requestshttps://dev.arvados.org/issues/19192?journal_id=1043792022-06-21T02:49:18ZTom Cleggtom@curii.com
<ul><li><strong>File</strong> <i>keep-web</i> added</li></ul><p>19192-fix-deadlock @ <a class="changeset" title="19192: Add a few bytes to MemorySize to account for data structures. Arvados-DCO-1.1-Signed-off-..." href="https://dev.arvados.org/projects/arvados/repository/arvados/revisions/dab0c5596e39dc455d88bba797717e829fe5caf5">dab0c5596e39dc455d88bba797717e829fe5caf5</a> -- <a class="external" href="https://ci.arvados.org/job/developer-run-tests/3177/"<a href="https://ci.arvados.org/job/developer-run-tests/3177/">developer-run-tests: #3177 <img src="https://ci.arvados.org/buildStatus/icon?job=developer-run-tests&build=3177" alt="" /></a></a></p>
<p>Fixes a lock-ordering bug in lookupnode causing deadlock on concurrent open() and readdir() calls in the same lookupnode (project) directory.</p> Arvados - Bug #19192: WebDAVCache not performing as expected for S3 requestshttps://dev.arvados.org/issues/19192?journal_id=1043802022-06-21T02:57:01ZTom Cleggtom@curii.com
<ul><li><strong>File</strong> deleted (<del><i>keep-web</i></del>)</li></ul> Arvados - Bug #19192: WebDAVCache not performing as expected for S3 requestshttps://dev.arvados.org/issues/19192?journal_id=1043812022-06-21T03:24:27ZTom Cleggtom@curii.com
<ul><li><strong>File</strong> <a href="/attachments/3025">keep-web</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/3025/keep-web">keep-web</a> added</li></ul><p>Here's a dev build with both the bugfix from note-5 and the watchdog / stack dump from note-4.</p> Arvados - Bug #19192: WebDAVCache not performing as expected for S3 requestshttps://dev.arvados.org/issues/19192?journal_id=1043832022-06-21T17:46:22ZLucas Di Pentimalucas.dipentima@curii.com
<ul></ul><p><code>19192-fix-deadlock</code> LGTM, thanks!</p> Arvados - Bug #19192: WebDAVCache not performing as expected for S3 requestshttps://dev.arvados.org/issues/19192?journal_id=1044542022-06-22T15:15:45ZPeter Amstutzpeter.amstutz@curii.com
<ul><li><strong>Target version</strong> changed from <i>2022-06-22 Sprint</i> to <i>2022-07-06</i></li></ul> Arvados - Bug #19192: WebDAVCache not performing as expected for S3 requestshttps://dev.arvados.org/issues/19192?journal_id=1044842022-06-22T17:25:26ZTom Cleggtom@curii.com
<ul><li><strong>Related to</strong> <i><a class="issue tracker-6 status-3 priority-4 priority-default closed parent" href="/issues/19205">Idea #19205</a>: In Go services, monitor request times and record when they continue processing after client disconnects, or exceed a maximum request time</i> added</li></ul> Arvados - Bug #19192: WebDAVCache not performing as expected for S3 requestshttps://dev.arvados.org/issues/19192?journal_id=1047402022-07-06T15:31:51ZPeter Amstutzpeter.amstutz@curii.com
<ul><li><strong>Target version</strong> changed from <i>2022-07-06</i> to <i>2022-07-20</i></li></ul> Arvados - Bug #19192: WebDAVCache not performing as expected for S3 requestshttps://dev.arvados.org/issues/19192?journal_id=1047412022-07-06T15:33:55ZPeter Amstutzpeter.amstutz@curii.com
<ul><li><strong>Target version</strong> changed from <i>2022-07-20</i> to <i>2022-07-06</i></li></ul> Arvados - Bug #19192: WebDAVCache not performing as expected for S3 requestshttps://dev.arvados.org/issues/19192?journal_id=1047422022-07-06T15:35:35ZPeter Amstutzpeter.amstutz@curii.com
<ul><li><strong>Status</strong> changed from <i>In Progress</i> to <i>Resolved</i></li><li><strong>Category</strong> set to <i>Keep</i></li></ul> Arvados - Bug #19192: WebDAVCache not performing as expected for S3 requestshttps://dev.arvados.org/issues/19192?journal_id=1052632022-08-03T20:10:31ZPeter Amstutzpeter.amstutz@curii.com
<ul><li><strong>Release</strong> set to <i>52</i></li></ul> Arvados - Bug #19192: WebDAVCache not performing as expected for S3 requestshttps://dev.arvados.org/issues/19192?journal_id=1054522022-08-09T16:06:47ZTom Cleggtom@curii.com
<ul><li><strong>Related to</strong> <i><a class="issue tracker-1 status-3 priority-4 priority-default closed parent" href="/issues/19368">Bug #19368</a>: [keep-web] [S3] slow requests caused by logUploadOrDownload</i> added</li></ul>