Arvados

This goes in the developer docs, logically between "introductory materials" and "API reference".

Acceptance criteria

• Describe how Arvados uses git repositories, and the layout of those repositories
  • it expects crunch scripts to be in the crunch_scripts directory, and to be executable
• Describe generally how a Crunch script is invoked and the environment it runs in
  • API server and token
  • Availability of SDK (depends on docker image)
  • Current job and task
  • Keep mount point
  • Other important environment variables
• Describe what a Crunch script must do to be considered successful:
  • update task record with success=true and output
  • exit 0
  • exit 111 → temporary failure, try again <= N times
• Introduce the concept of tasks and sequencing, and illustrate how they're useful for separating work.
  • sequence vs. qsequence
• Explain the output collation process.
  • task can provide either a manifest [fragment], or a Keep locator of one