Story #3651

Running programs from Keep

Added by Abram Connelly almost 6 years ago. Updated over 5 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
-
Category:
-
Target version:
-
Start date:
08/21/2014
Due date:
% Done:

0%

Estimated time:
Story points:
-

Description

Currently there are three basic ways to run programs:

- Create a Docker image with your program installed and run it normally
- Dump the binary into your Git repository and run it from your source directory
- Store it in Keep and either copy it to the local filesystem or run it from Keep

From what I understand, if there is a 'program suite', then the Docker solution is preferred. If the programs are small or are unique to the crunch job the Git solution is preferred. Using a program stored in Keep is a method that can be used for when either of the previous instances don't make sense.

In terms of using a program from Keep, two common options to actually execute the program are:

- Copy the program to a local directory, perhaps /tmp, and execute
- Execute directly from the Keep mount

Either because programs can get quite large or just because of added complexity, it would be nice to avoid copying over files to the local filesystem if it can be helped. As it stands now, you cannot execute programs from a keep mount (the executable bit is not set). To get around this, I have the following kludge:

    /lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 $TASK_KEEPMOUNT/$MYLOCATORSTRING/myprograminkeep

This works fine, but it would be nice to be able to do this in a more consistent, less kludgy way.


Related issues

Has duplicate Arvados - Feature #4838: [Keep] FUSE driver should expose files with permissions mode 0555Resolved12/18/2014

History

#1 Updated by Peter Amstutz over 5 years ago

  • Status changed from New to Resolved

Fixed in #4838

Also available in: Atom PDF