Story #8563

Updated by Sarah Guthrie over 3 years ago

Write a new wiki page describing: e.g.,

* Basic introduction to writing a Dockerfile (with links to more resources/references), resources/references?), using a small but real analysis tool
* How to build your Docker image
* How to upload your Docker image to Arvados
* How to call your tool from a Crunch script, including best practices (using subprocess.Popen, capturing stdout, uploading results, setting success based on Popen's returncode)
** How to upload Ideally, include a few common output from the tool using arvados.crunch.TaskOutputDir()
** Explain when TaskOutputDir does not work:
*** The tool writes things that fuse does not support (symbollic links and named pipes)
*** The I/O access patterns are not performant with fuse (ex: 20 file handles on one file - tophat)
** For when it doesn't work, explain
scenarios: how to use a tempdir and capture stdout; how to save one file from that directory or the as output; how to save an entire directory tree to Keep
as output

Some of this might be covered in the "Port a Pipeline" wiki page. If so, we might adapt some of its content.