Bug #15007

[dispatch-cloud] Azure driver panic in Address() (missing nic?)

Added by Tom Clegg 3 months ago. Updated 25 days ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
-
Target version:
Start date:
Due date:
% Done:

100%

Estimated time:
Story points:
-
Release relationship:
Auto

Description

Version 1.3.1.20190318151156-1 running on c97qk:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x10 pc=0x858bd9]
goroutine 68 [running]:
git.curoverse.com/arvados.git/lib/cloud/azure.(*azureInstance).Address(0xc420413d60, 0xc420296d10, 0xad)
        /tmp/tmp.AW7MSQbSFH/src/git.curoverse.com/arvados.git/lib/cloud/azure/azure.go:648 +0x9
git.curoverse.com/arvados.git/lib/dispatchcloud/worker.(*Pool).updateWorker(0xc42019d340, 0xb5e2e0, 0xc4204fb600, 0xc4202b70b0, 0xe, 0xc4202b70b0, 0xe, 0x1, 0xd59f8000, 0xba43b7400, ...)
        /tmp/tmp.AW7MSQbSFH/src/git.curoverse.com/arvados.git/lib/dispatchcloud/worker/pool.go:366 +0x3a4
git.curoverse.com/arvados.git/lib/dispatchcloud/worker.(*Pool).sync(0xc42019d340, 0xbf1c1ec2f96cd79c, 0x1266d687d61, 0xe36da0, 0xc42006b0c0, 0x4, 0x4)
        /tmp/tmp.AW7MSQbSFH/src/git.curoverse.com/arvados.git/lib/dispatchcloud/worker/pool.go:770 +0x452
git.curoverse.com/arvados.git/lib/dispatchcloud/worker.(*Pool).getInstancesAndSync(0xc42019d340, 0x0, 0x0)
        /tmp/tmp.AW7MSQbSFH/src/git.curoverse.com/arvados.git/lib/dispatchcloud/worker/pool.go:749 +0x27e
git.curoverse.com/arvados.git/lib/dispatchcloud/worker.(*Pool).runSync(0xc42019d340)
        /tmp/tmp.AW7MSQbSFH/src/git.curoverse.com/arvados.git/lib/dispatchcloud/worker/pool.go:675 +0x12f
created by git.curoverse.com/arvados.git/lib/dispatchcloud/worker.NewPool.func1
        /tmp/tmp.AW7MSQbSFH/src/git.curoverse.com/arvados.git/lib/dispatchcloud/worker/pool.go:116 +0xb9

Suspect it's just that #14844 didn't account for ai.nic being nil on line 648 here:

func (ai *azureInstance) Address() string {
        if ai.nic.IPConfigurations != nil &&

It seems like ai.nic becomes nil in Instances() if az.vmClient.listComplete() refers to a nic that didn't show up in az.manageNics() (interfaces[*(*result.Value().NetworkProfile.NetworkInterfaces)[0].ID] would be nil). Either Azure sometimes uses references to nics before they start showing up in az.netClient.listComplete(), or it's just that we do our "list nics" and "list instances" calls in the wrong order in Instances().

Either way, if ai.nic != nil && ... should fix it.

Associated revisions

Revision 7ef48a59
Added by Tom Clegg 3 months ago

Merge branch '15007-azure-panic'

fixes #15007

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <>

Revision c681e0b8 (diff)
Added by Tom Clegg 3 months ago

Add another nil check to Azure driver.

refs #15007

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <>

History

#1 Updated by Tom Clegg 3 months ago

  • Status changed from New to Resolved
  • % Done changed from 0 to 100

#2 Updated by Tom Morris 3 months ago

  • Target version changed from Arvados Future Sprints to 2019-03-27 Sprint

#3 Updated by Tom Morris 25 days ago

  • Assigned To set to Tom Clegg

#4 Updated by Tom Morris 25 days ago

  • Release set to 15

Also available in: Atom PDF