Bug #15946

[crunch-run] [collectionfs] Deadlock while writing output collection

Added by Tom Clegg about 2 years ago. Updated about 2 years ago.

Assigned To:
Target version:
Start date:
Due date:
% Done:


Estimated time:
(Total: 0.00 h)
Story points:
Release relationship:


Running several similar containers, some succeeded but some got stuck while writing the final output collection. The log indicates all of the expected files were written to collectionfs ("copying ... (... bytes)") but the finished collection was never saved.


Task #15947: Review 15946-collectionfs-deadlockResolvedTom Clegg

Related issues

Related to Arvados - Bug #15928: Deadlock in crunch-run testsResolved12/11/2019

Related to Arvados - Bug #15910: [crunch-run] Crash while writing output filesResolved12/04/2019

Associated revisions

Revision 949abe8b
Added by Tom Clegg about 2 years ago

Merge branch '15946-collectionfs-deadlock'

fixes #15946

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <>


#1 Updated by Tom Clegg about 2 years ago

Found a case where commitBlock would set a "flushing in progress" flag on a segment, but then return early (and never indicate
it was done) because a different segment had another flush in progress. Once that happened, sync operations like MarshalManifest would block forever.

15946-collectionfs-deadlock @ c05caa378debd04205690c6cb96508e4e7fb6c8b -- https://ci.arvados.org/view/Developer/job/developer-run-tests/1700/

#2 Updated by Tom Clegg about 2 years ago

  • Related to Bug #15928: Deadlock in crunch-run tests added

#3 Updated by Tom Clegg about 2 years ago

  • Related to Bug #15910: [crunch-run] Crash while writing output files added

#4 Updated by Lucas Di Pentima about 2 years ago

This LGTM, thanks!

#5 Updated by Anonymous about 2 years ago

  • % Done changed from 0 to 100
  • Status changed from In Progress to Resolved

#6 Updated by Peter Amstutz about 2 years ago

  • Target version changed from 2020-01-15 Sprint to 2020-01-02 Sprint

Also available in: Atom PDF