Arvados

The most efficient development cycle for crunch scripts involves running jobs straight from a checked-out source tree in an Arvados VM. Your have direct access to Keep, you can run in a screen session, Arvados SDKs are installed, etc.

However:

• Editing files in your VM is annoying because SSH is too laggy for terminal-based editors to be fun, let alone Xorg-based editors which probably aren't installed in the VM anyway.
• Editing files on your workstation and using "rsync" to update your VM copy is annoying. Besides, you'll eventually end up editing your VM copy and then overwriting your edits with a subsequent rsync.
• Editing files on your workstation and using "git commit; git push" is even more annoying and leaves you with a much more detailed git history than anyone wants.

Emacs users have a way to edit files on remote systems that works pretty well:

• Enable SSH ControlMaster, ControlPath, ControlPersist options in your .ssh/config file. (And know how to kill the control file manually when SSH is too stubborn about using a hung network socket)
• Open an ssh session to your shell VM (say, shell.qr1hi)
• Open Emacs on your workstation.
• Open ///shell.qr1hi:yourrepo/crunch_scripts/foo.