Project

General

Profile

Actions
Copy link

Idea #13501

closed

[arvados-cwl-runner] [arv-keepdocker] Use subprocess32 instead of subprocess

Added by Thomas Hickman almost 6 years ago. Updated almost 6 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Lucas Di Pentima
Category:
-
Target version:
2018-07-18 Sprint
Start date:
07/09/2018
Due date:
Story points:
0.5
Release:
Arvados 1.2
Release relationship:
Auto

Description

Google has created a backport of python3.3's subprocess module to python2, which includes includes "many important reliability bug fixes relevant on POSIX platforms" (see https://github.com/google/python-subprocess32) -- including making it safer to use with multi-threaded programs. It would be good to use this instead of the native python2 subprocess module.
  • cwltool already uses subprocess32
  • arvados-cwl-runner uses native subprocess
  • arv-keepdocker uses native subprocess
  • nodemanager uses native subprocess

Subtasks 1 (0 open — 1 closed)

Task #13725: Review 13501-subprocess32ResolvedPeter Amstutz07/09/2018Actions
Actions
Copy link
 #1

Updated by Tom Morris almost 6 years ago

  • Tracker changed from Bug to Idea
  • Target version set to To Be Groomed
Actions
Copy link
 #2

Updated by Tom Clegg almost 6 years ago

  • Description updated (diff)
Actions
Copy link
 #3

Updated by Tom Clegg almost 6 years ago

  • Story points set to 0.5
Actions
Copy link
 #4

Updated by Tom Clegg almost 6 years ago

  • Subject changed from Use subprocess32 instead of subprocess to [arvados-cwl-runner] [arv-keepdocker] Use subprocess32 instead of subprocess
Actions
Copy link
 #5

Updated by Tom Morris almost 6 years ago

  • Target version changed from To Be Groomed to 2018-07-18 Sprint
Actions
Copy link
 #6

Updated by Tom Clegg almost 6 years ago

  • Description updated (diff)
Actions
Copy link
 #7

Updated by Lucas Di Pentima almost 6 years ago

  • Assigned To set to Lucas Di Pentima
Actions
Copy link
 #8

Updated by Peter Amstutz almost 6 years ago

Notes:

  • We are already building a backport of subprocess32 3.5
  • Node manager also uses threads and subprocess, it should be included in this list of modules to be updated
Actions
Copy link
 #9

Updated by Lucas Di Pentima almost 6 years ago

  • Status changed from New to In Progress
Actions
Copy link
 #10

Updated by Lucas Di Pentima almost 6 years ago

Updates at 81fa59a1e - branch 13501-subprocess32
Test run: https://ci.curoverse.com/job/developer-run-tests/791/

  • Replaced subprocess with subprocess32 on nodemanager, arvados-cwl-runner & arv-keepdocker.
  • Updated tests and package dependencies where needed.
  • Tested debian8 package building locally.
Actions
Copy link
 #11

Updated by Peter Amstutz almost 6 years ago

Lucas Di Pentima wrote:

Updates at 81fa59a1e - branch 13501-subprocess32
Test run: https://ci.curoverse.com/job/developer-run-tests/791/

  • Replaced subprocess with subprocess32 on nodemanager, arvados-cwl-runner & arv-keepdocker.
  • Updated tests and package dependencies where needed.
  • Tested debian8 package building locally.

This LGTM.

Actions
Copy link
 #12

Updated by Lucas Di Pentima almost 6 years ago

  • Status changed from In Progress to Resolved
Actions
Copy link
 #13

Updated by Tom Morris almost 6 years ago

  • Release set to 13
Actions
Copy link

Also available in: Atom PDF