Story #13501
[arvados-cwl-runner] [arv-keepdocker] Use subprocess32 instead of subprocess
Status:
Resolved
Priority:
Normal
Assigned To:
Category:
-
Target version:
Start date:
07/09/2018
Due date:
% Done:
100%
Estimated time:
(Total: 0.00 h)
Story points:
0.5
Release:
Release relationship:
Auto
Description
Google has created a backport of python3.3's subprocess module to python2, which includes includes "many important reliability bug fixes relevant on POSIX platforms" (see https://github.com/google/python-subprocess32) -- including making it safer to use with multi-threaded programs. It would be good to use this instead of the native python2 subprocess module.
- cwltool already uses subprocess32
- arvados-cwl-runner uses native subprocess
- arv-keepdocker uses native subprocess
- nodemanager uses native subprocess
Subtasks
Associated revisions
History
#1
Updated by Tom Morris over 2 years ago
- Tracker changed from Bug to Story
- Target version set to To Be Groomed
#2
Updated by Tom Clegg over 2 years ago
- Description updated (diff)
#3
Updated by Tom Clegg over 2 years ago
- Story points set to 0.5
#4
Updated by Tom Clegg over 2 years ago
- Subject changed from Use subprocess32 instead of subprocess to [arvados-cwl-runner] [arv-keepdocker] Use subprocess32 instead of subprocess
#5
Updated by Tom Morris over 2 years ago
- Target version changed from To Be Groomed to 2018-07-18 Sprint
#6
Updated by Tom Clegg over 2 years ago
- Description updated (diff)
#7
Updated by Lucas Di Pentima over 2 years ago
- Assigned To set to Lucas Di Pentima
#8
Updated by Peter Amstutz over 2 years ago
Notes:
- We are already building a backport of subprocess32 3.5
- Node manager also uses threads and subprocess, it should be included in this list of modules to be updated
#9
Updated by Lucas Di Pentima over 2 years ago
- Status changed from New to In Progress
#10
Updated by Lucas Di Pentima over 2 years ago
Updates at 81fa59a1e - branch 13501-subprocess32
Test run: https://ci.curoverse.com/job/developer-run-tests/791/
- Replaced
subprocess
withsubprocess32
on nodemanager, arvados-cwl-runner & arv-keepdocker. - Updated tests and package dependencies where needed.
- Tested debian8 package building locally.
#11
Updated by Peter Amstutz over 2 years ago
Lucas Di Pentima wrote:
Updates at 81fa59a1e - branch
13501-subprocess32
Test run: https://ci.curoverse.com/job/developer-run-tests/791/
- Replaced
subprocess
withsubprocess32
on nodemanager, arvados-cwl-runner & arv-keepdocker.- Updated tests and package dependencies where needed.
- Tested debian8 package building locally.
This LGTM.
#12
Updated by Lucas Di Pentima over 2 years ago
- Status changed from In Progress to Resolved
Applied in changeset arvados|688319dca9cbbb27452a0062943bb38b6935254e.
#13
Updated by Tom Morris over 2 years ago
- Release set to 13
Merge branch '13501-subprocess32'
Closes #13501
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>