Story #13501

[arvados-cwl-runner] [arv-keepdocker] Use subprocess32 instead of subprocess

Added by Thomas Hickman over 2 years ago. Updated over 2 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Lucas Di Pentima
Category:
-
Target version:
2018-07-18 Sprint
Start date:
07/09/2018
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)
Story points:
0.5
Release:
Arvados 1.2
Release relationship:
Auto

Description

Google has created a backport of python3.3's subprocess module to python2, which includes includes "many important reliability bug fixes relevant on POSIX platforms" (see https://github.com/google/python-subprocess32) -- including making it safer to use with multi-threaded programs. It would be good to use this instead of the native python2 subprocess module.
  • cwltool already uses subprocess32
  • arvados-cwl-runner uses native subprocess
  • arv-keepdocker uses native subprocess
  • nodemanager uses native subprocess

Subtasks

Task #13725: Review 13501-subprocess32ResolvedPeter Amstutz

Associated revisions

Revision 688319dc
Added by Lucas Di Pentima over 2 years ago

Merge branch '13501-subprocess32'
Closes #13501

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <>

History

#1 Updated by Tom Morris over 2 years ago

  • Tracker changed from Bug to Story
  • Target version set to To Be Groomed

#2 Updated by Tom Clegg over 2 years ago

  • Description updated (diff)

#3 Updated by Tom Clegg over 2 years ago

  • Story points set to 0.5

#4 Updated by Tom Clegg over 2 years ago

  • Subject changed from Use subprocess32 instead of subprocess to [arvados-cwl-runner] [arv-keepdocker] Use subprocess32 instead of subprocess

#5 Updated by Tom Morris over 2 years ago

  • Target version changed from To Be Groomed to 2018-07-18 Sprint

#6 Updated by Tom Clegg over 2 years ago

  • Description updated (diff)

#7 Updated by Lucas Di Pentima over 2 years ago

  • Assigned To set to Lucas Di Pentima

#8 Updated by Peter Amstutz over 2 years ago

Notes:

  • We are already building a backport of subprocess32 3.5
  • Node manager also uses threads and subprocess, it should be included in this list of modules to be updated

#9 Updated by Lucas Di Pentima over 2 years ago

  • Status changed from New to In Progress

#10 Updated by Lucas Di Pentima over 2 years ago

Updates at 81fa59a1e - branch 13501-subprocess32
Test run: https://ci.curoverse.com/job/developer-run-tests/791/

  • Replaced subprocess with subprocess32 on nodemanager, arvados-cwl-runner & arv-keepdocker.
  • Updated tests and package dependencies where needed.
  • Tested debian8 package building locally.

#11 Updated by Peter Amstutz over 2 years ago

Lucas Di Pentima wrote:

Updates at 81fa59a1e - branch 13501-subprocess32
Test run: https://ci.curoverse.com/job/developer-run-tests/791/

  • Replaced subprocess with subprocess32 on nodemanager, arvados-cwl-runner & arv-keepdocker.
  • Updated tests and package dependencies where needed.
  • Tested debian8 package building locally.

This LGTM.

#12 Updated by Lucas Di Pentima over 2 years ago

  • Status changed from In Progress to Resolved

#13 Updated by Tom Morris over 2 years ago

  • Release set to 13

Also available in: Atom PDF