Project

General

Profile

Actions

Idea #13501

closed

[arvados-cwl-runner] [arv-keepdocker] Use subprocess32 instead of subprocess

Added by Thomas Hickman about 6 years ago. Updated about 6 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
-
Target version:
Start date:
07/09/2018
Due date:
Story points:
0.5
Release:
Release relationship:
Auto

Description

Google has created a backport of python3.3's subprocess module to python2, which includes includes "many important reliability bug fixes relevant on POSIX platforms" (see https://github.com/google/python-subprocess32) -- including making it safer to use with multi-threaded programs. It would be good to use this instead of the native python2 subprocess module.
  • cwltool already uses subprocess32
  • arvados-cwl-runner uses native subprocess
  • arv-keepdocker uses native subprocess
  • nodemanager uses native subprocess

Subtasks 1 (0 open1 closed)

Task #13725: Review 13501-subprocess32ResolvedPeter Amstutz07/09/2018Actions
Actions #1

Updated by Tom Morris about 6 years ago

  • Tracker changed from Bug to Idea
  • Target version set to To Be Groomed
Actions #2

Updated by Tom Clegg about 6 years ago

  • Description updated (diff)
Actions #3

Updated by Tom Clegg about 6 years ago

  • Story points set to 0.5
Actions #4

Updated by Tom Clegg about 6 years ago

  • Subject changed from Use subprocess32 instead of subprocess to [arvados-cwl-runner] [arv-keepdocker] Use subprocess32 instead of subprocess
Actions #5

Updated by Tom Morris about 6 years ago

  • Target version changed from To Be Groomed to 2018-07-18 Sprint
Actions #6

Updated by Tom Clegg about 6 years ago

  • Description updated (diff)
Actions #7

Updated by Lucas Di Pentima about 6 years ago

  • Assigned To set to Lucas Di Pentima
Actions #8

Updated by Peter Amstutz about 6 years ago

Notes:

  • We are already building a backport of subprocess32 3.5
  • Node manager also uses threads and subprocess, it should be included in this list of modules to be updated
Actions #9

Updated by Lucas Di Pentima about 6 years ago

  • Status changed from New to In Progress
Actions #10

Updated by Lucas Di Pentima about 6 years ago

Updates at 81fa59a1e - branch 13501-subprocess32
Test run: https://ci.curoverse.com/job/developer-run-tests/791/

  • Replaced subprocess with subprocess32 on nodemanager, arvados-cwl-runner & arv-keepdocker.
  • Updated tests and package dependencies where needed.
  • Tested debian8 package building locally.
Actions #11

Updated by Peter Amstutz about 6 years ago

Lucas Di Pentima wrote:

Updates at 81fa59a1e - branch 13501-subprocess32
Test run: https://ci.curoverse.com/job/developer-run-tests/791/

  • Replaced subprocess with subprocess32 on nodemanager, arvados-cwl-runner & arv-keepdocker.
  • Updated tests and package dependencies where needed.
  • Tested debian8 package building locally.

This LGTM.

Actions #12

Updated by Lucas Di Pentima about 6 years ago

  • Status changed from In Progress to Resolved
Actions #13

Updated by Tom Morris about 6 years ago

  • Release set to 13
Actions

Also available in: Atom PDF