Bug #12467: crunch-run not waiting for Docker image to finish loading. - Arvados

Actions

Copy link

Bug #12467

closed

crunch-run not waiting for Docker image to finish loading.

Added by Peter Amstutz about 7 years ago. Updated about 7 years ago.

Status:

Resolved

Priority:

Normal

Assigned To:

Peter Amstutz

Category:

Target version:

2017-10-25 Sprint

Story points:

Description

It is failing to load Docker image, even though the image load seemingly succeeds:

2017-10-19T16:41:46.528938874Z Fetching Docker image from collection '997e3bc9579153e723b599b300d9123d+342'
2017-10-19T16:41:46.549138991Z Using Docker image id 'sha256:20a23446d37fac4cb2ec6462310aaee7119f4594948765a3397b13f9baec4812'
2017-10-19T16:41:46.549599576Z Loading Docker image from keep
2017-10-19T16:41:49.814403711Z Creating Docker container
2017-10-19T16:41:49.816658048Z While creating container: Error: No such image: sha256:20a23446d37fac4cb2ec6462310aaee7119f4594948765a3397b13f9baec4812

It seems that loading is asynchronous, the ImageLoad method returns after the data has been transmitted but before the layers have been decompressed. We need to wait for a response indicating that loading is finished.

Subtasks 1 (0 open — 1 closed)

Actions

Copy link

Updated by Peter Amstutz about 7 years ago

Description updated (diff)

Actions

Copy link

Updated by Peter Amstutz about 7 years ago

Status changed from New to In Progress
Assigned To set to Peter Amstutz

12467-read-imgload-response @ d64215e8f0057cc7b4c6295932bc7f44d27a1eb5

Actions

Copy link

Updated by Tom Clegg about 7 years ago

Better form to do "defer response.Body.Close()" before ioutil.ReadAll(), so it gets closed even if read fails.

I think the error message has copy-paste-itis: "While reading response to container image into Docker: %v"

I'm guessing it means "error reading response from docker.ImageLoad: %v"

You don't need to say string(rbody) -- %s prints a []byte as a string.

Actions

Copy link

Updated by Peter Amstutz about 7 years ago

Tom Clegg wrote:

Better form to do "defer response.Body.Close()" before ioutil.ReadAll(), so it gets closed even if read fails.

I think the error message has copy-paste-itis: "While reading response to container image into Docker: %v"

I'm guessing it means "error reading response from docker.ImageLoad: %v"

You don't need to say string(rbody) -- %s prints a []byte as a string.

All fixed, plus a bonus fix log.Printf -> runner.CrunchLog.Printf

12467-read-imgload-response @ a976b02dc7ecba18277aceeaf086ab76fe4bf3f0

Actions

Copy link