Story #9132

[Crunch2] crunch-run uses official docker client library

Added by Peter Amstutz 11 months ago. Updated about 12 hours ago.

Status:In ProgressStart date:03/08/2017
Priority:NormalDue date:
Assignee:Radhika Chippada% Done:

0%

Category:-
Target version:2017-03-29 sprint
Story points1.0Remaining (hours)0.00 hour
Velocity based estimate-
ReleaseCrunch v2

Description

Current crunch-run is using curoverse/dockerclient, but this is not actively maintained and lacks support for attaching stdin. The supported Go API for Docker is Docker's own client code https://github.com/docker/docker/tree/master/client

This should be a straight-across port to equivalent functionality.


Subtasks

Task #11192: Review 9132-dockerclientIn ProgressRadhika Chippada

Task #11224: Update API usageIn ProgressPeter Amstutz


Related issues

Blocks Arvados - Feature #8465: [Crunch2] Support stdin redirection New

History

#1 Updated by Peter Amstutz 11 months ago

  • Description updated (diff)

#2 Updated by Tom Morris 6 months ago

  • Target version set to Arvados Future Sprints

#3 Updated by Peter Amstutz about 1 month ago

  • Description updated (diff)

#4 Updated by Peter Amstutz about 1 month ago

  • Subject changed from [Crunch2] crunch-run uses docker/engine-api instead of dockerclient to [Crunch2] crunch-run uses official docker client library

#5 Updated by Peter Amstutz about 1 month ago

  • Description updated (diff)

#6 Updated by Tom Morris about 1 month ago

  • Target version changed from Arvados Future Sprints to 2017-03-15 sprint

#7 Updated by Peter Amstutz 28 days ago

  • Assignee set to Peter Amstutz

#8 Updated by Peter Amstutz 22 days ago

  • Status changed from New to In Progress

#9 Updated by Peter Amstutz 14 days ago

  • Target version changed from 2017-03-15 sprint to 2017-03-29 sprint

#10 Updated by Peter Amstutz 14 days ago

  • Assignee changed from Peter Amstutz to Radhika Chippada

#11 Updated by Peter Amstutz 14 days ago

Please take over 9132-dockerclient

#12 Updated by Radhika Chippada 6 days ago

Branch 9132-dockerclient @ 374dff075f0d2961b3843d952ccca9f55d972e9a

  • Uses new docker client API.
  • What ContainerStartOptions should we use?
  • The ContainerWait method returns an int64 and hence I replaced the waitDocker chan as such.
  • The test "TestStopOnSignal" is failing sometimes when the whole crunch-run test set is executed with the error “cannot allocate memory” locally, but it appears that it is an issue with development env and passing consistently in jenkins

Thanks.

#13 Updated by Tom Clegg 6 days ago

Radhika Chippada wrote:

  • What ContainerStartOptions should we use?

Empty LGTM. The options seem to be for checkpoint suspend/resume -- sounds interesting, but we're not using it.

  • The ContainerWait method returns an int64 and hence I replaced the waitDocker chan as such.

nit: could change the "finish" field and the "exitCode" arguments too, then the int64() cast wouldn't be needed.

  • The test "TestStopOnSignal" is failing sometimes when the whole crunch-run test set is executed with the error “cannot allocate memory” locally, but it appears that it is an issue with development env and passing consistently in jenkins

Tried run-tests.sh --repeat 20, no problems.

LGTM, thanks!

#14 Updated by Peter Amstutz about 17 hours ago

I'm running stuff and getting an exit code of "2" when it should be "0". I think maybe the return code from ContainerWait is not being interpreted properly?

#15 Updated by Radhika Chippada about 12 hours ago

I think maybe the return code from ContainerWait is not being interpreted properly

We are just returning what is returned by ContainerWait and so we don't have any interpretation happening.

However, according to https://docs.docker.com/engine/reference/run/#exit-status , which takes me to http://tldp.org/LDP/abs/html/exitcodes.html , it could be that we are not configuring the client correctly.

This stackflow page led me to the above links: http://stackoverflow.com/questions/31297616/what-is-the-authoritative-list-of-docker-run-exit-codes

Also available in: Atom PDF