Project

General

Profile

CLI client » History » Version 4

Tom Clegg, 12/19/2017 05:21 AM

1 1 Tom Clegg
h1. CLI client
2
3
Arvados exposes lots of functionality via REST/HTTP. Most of them deserve to be exposed through a command line tool that is easy to use interactively and in shell scripts.
4
5
h2. Background / current status
6
7
Currently there is an "arv" command line tool in source:sdk/cli (packaged as a Ruby gem and deb/rpm). It exposes REST APIs (as found in the discovery document) generically by mapping API names and parameters to command line arguments. It also knows how to invoke some Python programs (e.g., "arv put" passes through to "arv-put") which have their own calling syntax.
8
9
Problems:
10
* Awkward interface: <code>arv job list --filters '[["script","in",["foo","bar"]]]'</code>
11
* Lots of conventions: <code>arv collection get --uuid FOO</code> vs. <code>arv get FOO</code> vs. <code>arv-get FOO</code> (download files), <code>arv edit FOO</code> vs. <code>arv collection update --uuid FOO --collection '{...}'</code>
12
* Annoying to install (especially on older systems) because of Ruby requirements, and because both Ruby and Python pieces are needed to get a full set
13
* Slow to start up (partly because of Bundler?)
14
15
h2. Desired scope
16
17
Command line tool should be able to
18
* Display/list Arvados objects in human- and machine-readable form
19
* Create/edit all Arvados objects in a generic way (current @"arv edit"@ is pretty good)
20
* Perform common actions in a natural way (@"arv cancel JOB_UUID"@)
21
* Watch job logs in real time (@"arv tail -f JOB_OR_PIPELINE_UUID"@?)
22
* Transfer files/dirs between the local filesystem and Arvados collections (Keep)
23
* Expose collection data (and other Arvados objects) as a local filesystem through a FUSE mount
24 2 Tom Clegg
* Search/get objects across multiple sites, similar to workbench multi-site search
25 1 Tom Clegg
* ...more features go here
26
27
h2. Desired interface (patterns)
28
29
As with git, everything should be accessible through an "arv" front door. Some commands might result in forking other binaries like "arv-get" but this is an implementation detail that can change over time, not part of the public API.
30
31
Calling syntax should look familiar/normal to people who use related tools like Docker. In many cases one should be able to take a non-Arvados command, prefix it with the word "arv", and have it do the analogous thing with Arvados: e.g., @"arv less PDH/file.txt"@ or @"arv rm UUID"@.
32
33 3 Tom Clegg
Server components should also be accessible the same way. This could be implemented by having "arv" invoke a separately packaged "arvados-server" binary at runtime, or by shipping an alternate "arv" binary that also has the server components embedded. The server-included binary, in addition to being larger, will be less distributable because AGPL.
34
35 1 Tom Clegg
(More rules/patterns go here)
36
37
h2. Desired interface (specifics)
38
39
(Specific commands go here)
40
41
h2. Implementation
42
43 4 Tom Clegg
This should be done in Go. Considering portability, performance, stability, and development speed, Python and Ruby just don't stand a chance.