Progress, progress, progress
An overview of recent development progress
It has been a while since we've blogged about our development progress. Since the previous Arvados Update, we've completed 5 development sprints. We resolved 678 issues and tasks: 73 feature stories, 148 bugs and 12 support issues. These issues and stories were further broken down into 445 tasks.
Here are some of the new feature highlights:
- New cli commands:
- arv edit can be used to edit Arvados objects from the command line. Arv edit opens up the editor of your choice (set the EDITOR environment variable) with the json description of the object. Saving the json will update the Arvados object on the API server.
- arv create can be used to create Arvados objects from the command line. Arv create opens up the editor of your choice (set the EDITOR environment variable) and allows you to type or paste a json description. When saved the object will be created on the API server if it passes validation.
- arv copy can be used to copy a pipeline instance, template or collection from one Arvados instance to another. It takes care of copying the object and all its dependencies.
- Arvados Node Manager: this component manages Arvados compute nodes in a cloud computing environment, automatically spinning up new nodes and shutting down excess ones as needed. It supports Amazon Web Services and Google Cloud support is in the works.
- Crunch: we added run-command - a generic "command line wrapper" crunch script.
- Workbench: workbench remains a significant focus area. Here are a couple of the more important new workbench features:
- switched to a project-based interface
- added a 'Home' project
- upgraded the dashboard to show the current state of the cluster - how many (idle) compute nodes are available, what jobs and pipelines are running, etc. The jobs and pipeline information shown to the user is subject to the permission model.
- added a search feature
- added a 'manage account' page where users can view the virtual machines and repositories they have access to, as well as manage tokens and ssh keys.
- overhauled the 'show pipeline instance' page
- added a persistent top nav bar
- added a diagnostic suite feature, which allows for automated running of diagnostic pipelines
- added a project sharing feature
- Installation: we added a 'binary' installation method for local evaluation and testing. This installation method downloads prebuilt docker images from the Docker Registry and spins them up locally.
- Testing: we have added a large number of tests all over the codebase, which are run automatically by our Jenkins server as part of our build pipeline.
In addition to those new features, the last three sprints have been centered around improving the user experience. We continue to put a lot of effort into finding and fixing bugs, and we are actively focusing on making Arvados easier to use.
If you want to try out Arvados, head over to curoverse.com and hit the "Log In" button. After you log in with a Google account, you can try out a cloud-hosted copy of Arvados - we're in open beta, so it's free!
Alternatively, have a look at the installation instructions to install Arvados yourself.
Let us know how it goes on the Arvados mailing list or the IRC channel!