Bug #17853: [keep-web] concurrent map iteration and map write - Arvados

Actions

Copy link

Bug #17853

closed

[keep-web] concurrent map iteration and map write

Added by Tom Clegg over 3 years ago. Updated about 3 years ago.

Status:

Resolved

Priority:

Normal

Assigned To:

Tom Clegg

Category:

Target version:

2021-07-07 sprint

Story points:

Release:

Arvados 2.3.0

Release relationship:

Auto

Description

developer-run-tests-remainder: #2663 /consoleFull

We've fixed this before (in #17464 when it started happening) but, evidently, the fix was not complete.

fatal error: concurrent map iteration and map write

goroutine 11372 [running]:
runtime.throw(0xe70ca4, 0x26)
    /usr/src/go/src/runtime/panic.go:1117 +0x72 fp=0xc0042e7598 sp=0xc0042e7568 pc=0x43a5f2
runtime.mapiternext(0xc0042e7658)
    /usr/src/go/src/runtime/map.go:858 +0x54c fp=0xc0042e7618 sp=0xc0042e7598 pc=0x412d8c
runtime.mapiterinit(0xd68960, 0xc005dbced0, 0xc0042e7658)
    /usr/src/go/src/runtime/map.go:848 +0x1c5 fp=0xc0042e7638 sp=0xc0042e7618 pc=0x412745
git.arvados.org/arvados.git/sdk/go/arvados.(*treenode).MemorySize(0xc00558c630, 0x0)
    /tmp/workspace/developer-run-tests-remainder/sdk/go/arvados/fs_base.go:355 +0xf2 fp=0xc0042e76d8 sp=0xc0042e7638 pc=0x8cfdf2
git.arvados.org/arvados.git/sdk/go/arvados.(*fileSystem).MemorySize(0xc0052fd730, 0x0)
    /tmp/workspace/developer-run-tests-remainder/sdk/go/arvados/fs_base.go:650 +0x33 fp=0xc0042e76f8 sp=0xc0042e76d8 pc=0x8d2313
git.arvados.org/arvados.git/services/keep-web.(*cache).pruneSessions(0xc003517d08)
    /tmp/workspace/developer-run-tests-remainder/services/keep-web/cache.go:277 +0x415 fp=0xc0042e77d8 sp=0xc0042e76f8 pc=0xc82475
runtime.goexit()
    /usr/src/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc0042e77e0 sp=0xc0042e77d8 pc=0x474241
created by git.arvados.org/arvados.git/services/keep-web.(*cache).GetSession
    /tmp/workspace/developer-run-tests-remainder/services/keep-web/cache.go:249 +0x1b0

Subtasks 1 (0 open — 1 closed)

Actions

Copy link

Updated by Tom Clegg over 3 years ago

Status changed from New to In Progress

Actions

Copy link

Updated by Tom Clegg over 3 years ago

Description updated (diff)

17853-fix-write-with-rlock @ dd8ae798774d29b35532b99b78cae67a151654be -- developer-run-tests: #2565

-       if createMode {
-               parent.Lock()
-               defer parent.Unlock()
-       } else {
-               parent.RLock()
-               defer parent.RUnlock()
-       }
+       // We always need to take Lock() here, not just RLock(). Even
+       // if we know we won't be creating a file, parent might be a
+       // lookupnode, which sometimes populates its inodes map during
+       // a Child() call.
+       parent.Lock()
+       defer parent.Unlock()

Actions

Copy link