Project

General

Profile

CLI client » History » Version 2

Tom Clegg, 09/01/2017 02:05 PM

1 1 Tom Clegg
h1. CLI client
2
3
Arvados exposes lots of functionality via REST/HTTP. Most of them deserve to be exposed through a command line tool that is easy to use interactively and in shell scripts.
4
5
h2. Background / current status
6
7
Currently there is an "arv" command line tool in source:sdk/cli (packaged as a Ruby gem and deb/rpm). It exposes REST APIs (as found in the discovery document) generically by mapping API names and parameters to command line arguments. It also knows how to invoke some Python programs (e.g., "arv put" passes through to "arv-put") which have their own calling syntax.
8
9
Problems:
10
* Awkward interface: <code>arv job list --filters '[["script","in",["foo","bar"]]]'</code>
11
* Lots of conventions: <code>arv collection get --uuid FOO</code> vs. <code>arv get FOO</code> vs. <code>arv-get FOO</code> (download files), <code>arv edit FOO</code> vs. <code>arv collection update --uuid FOO --collection '{...}'</code>
12
* Annoying to install (especially on older systems) because of Ruby requirements, and because both Ruby and Python pieces are needed to get a full set
13
* Slow to start up (partly because of Bundler?)
14
15
h2. Desired scope
16
17
Command line tool should be able to
18
* Display/list Arvados objects in human- and machine-readable form
19
* Create/edit all Arvados objects in a generic way (current @"arv edit"@ is pretty good)
20
* Perform common actions in a natural way (@"arv cancel JOB_UUID"@)
21
* Watch job logs in real time (@"arv tail -f JOB_OR_PIPELINE_UUID"@?)
22
* Transfer files/dirs between the local filesystem and Arvados collections (Keep)
23
* Expose collection data (and other Arvados objects) as a local filesystem through a FUSE mount
24 2 Tom Clegg
* Search/get objects across multiple sites, similar to workbench multi-site search
25 1 Tom Clegg
* ...more features go here
26
27
h2. Desired interface (patterns)
28
29
As with git, everything should be accessible through an "arv" front door. Some commands might result in forking other binaries like "arv-get" but this is an implementation detail that can change over time, not part of the public API.
30
31
Calling syntax should look familiar/normal to people who use related tools like Docker. In many cases one should be able to take a non-Arvados command, prefix it with the word "arv", and have it do the analogous thing with Arvados: e.g., @"arv less PDH/file.txt"@ or @"arv rm UUID"@.
32
33
(More rules/patterns go here)
34
35
h2. Desired interface (specifics)
36
37
(Specific commands go here)
38
39
h2. Implementation
40
41
For portability and ease of installation, 100% Python would be better than a Ruby/Python mix. 100% Go would be the best.
42
43
For speed (startup delay and data transfer are the main concerns) Go would be the best.
44
45
For implementation speed Python might be better than Go. (Currently the Python SDK is more complete.)