Bug #13455

[keepproxy] Test suite sometimes hangs in go 1.10.1

Added by Tom Clegg over 2 years ago. Updated over 2 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
-
Target version:
Start date:
05/15/2018
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)
Story points:
-
Release:
Release relationship:
Auto

Subtasks

Task #13460: ReviewResolvedLucas Di Pentima

Associated revisions

Revision bce4ee3a (diff)
Added by Tom Clegg over 2 years ago

Avoid copying mutex after first use.

refs #13455

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <>

History

#1 Updated by Tom Clegg over 2 years ago

  • Subject changed from [keepproxy] Test suite hangs in go 1.10.1 to [keepproxy] Test suite sometimes hangs in go 1.10.1

#2 Updated by Tom Clegg over 2 years ago

  • Status changed from New to In Progress

I haven't been able to reproduce a hung keepproxy test myself since a221be676f0650993e76a9e402b93fa4ef7b5cf6 but jenkins has done it twice, in "remainder" builds 1933 and 1949.

While investigating other "remainder" failures I fixed a keepstore test bug that broke builds 1932 and 1945: 6c56f80642aae0ec0ff0bfc939ae33dd49e0ca6b

While attempting to reproduce the keepproxy hang locally, I did hit occasional "concurrent map writes" panics, which seem to have been caused by inadvertently copying a sync.Mutex after first use. After fixing that in bce4ee3aabf1dabc41ea5b231f0407cad3347f11, keepproxy tests seem reliable here (tried 64x, no failures).

It's possible that same mutex bug could cause tests to hang -- e.g., if the mutex was copied while locked -- and that the 10 minute test timeout isn't effective when it happens because https://github.com/golang/go/issues/24050 .

#3 Updated by Tom Clegg over 2 years ago

  • Status changed from In Progress to Resolved

14 "run-tests-remainder" runs since merging this, with no timeouts or failures.

#4 Updated by Tom Morris over 2 years ago

  • Release set to 13

Also available in: Atom PDF