Arvados

Regarding branch 4156-cli-tests:

• Initially there were 17 failures and I fixed 15 of them. Please let me know if any of these fixes are incorrect.
  • About 10 of them were failing because they were expecting pdh instead of uuid. Updated these tests to look for a collection uuid and are now passing.
  • A couple of them were failing as they were expecting /^ERROR/ and updating them to /Error/ made them pass
  • A couple of them (test_file_to_file_no_overwrite_file and test_file_to_file_no_overwrite_file_in_dir in test_arv-get) were expecting "already exists" error. However, no error is being returned. Commented out the error expectation and the test is now passing.

• There are 2 tests that are still failing in test_arv-tag. I never used this before and hence could not understand how to address these. Please give me a hint as to what needs to be done. It appears that the first one "test_single_tag_single_obj" should not use --short, but I could not get it to work even after removing this.
  • Based on our conversations, I gather that arv-tag is not yet working or something along those lines. Skipping these two tests for now.

I tested my work as follows:

• Set ARVADOS_API_HOST and ARVADOS_API_TOKEN to point to my dev api server

• Run my api server

• Use the following commands to test:

cd ~/arvados/sdk/python/tests
python run_test_server.py start
python run_test_server.py start_keep

cd ~/arvados/sdk/cli
bundle exec rake test

Skipped all tests that require apiserver / keep. There are 32 such tests out of 48 tests. 2 arv-tag tests are also skipped. Thus, only 14 out of the 48 tests are executed currently.

Should these change to "Error" like test_nonexistent_block instead of being commented out?

@@ -58,7 +64,7 @@ class TestArvGet < Minitest::Test
     out, err = capture_subprocess_io do
       assert_arv_get false, @@foo_manifest_locator + '/foo', 'tmp/foo'
     end
-    assert_match /^ERROR:/, err
+#    assert_match /^ERROR:/, err
     assert_equal '', out
     assert_equal 'baz', IO.read('tmp/foo')
   end
@@ -70,12 +76,14 @@ class TestArvGet < Minitest::Test
     out, err = capture_subprocess_io do
       assert_arv_get false, @@foo_manifest_locator + '/', 'tmp/'
     end
-    assert_match /^ERROR:/, err
+#    assert_match /^ERROR:/, err
     assert_equal '', out
     assert_equal 'baz', IO.read('tmp/foo')
   end

The rest LGTM, thanks

I might have seen it incorrectly during my initial testing and thought that this error is not resulting. Now, I see it work correctly. Updated accordingly. Thanks.

Reviewing arvados-dev branch at commit:12a59fa

I'm afraid this doesn't work for me with RVM. It reports:

Failures (4):
Fail: cli tests (18s)
Fail: apiserver tests (84s)
Fail: sdk/python tests (254s)
Fail: workbench tests (19s)

The API server, Python SDK, and Workbench test failures all look like this:

 --> Skipping compiling of passenger_native_support.so
 --> Downloading precompiled passenger_native_support.so for the current Ruby interpreter...
     (set PASSENGER_DOWNLOAD_NATIVE_SUPPORT_BINARY=0 to disable)
     Could not download https://oss-binaries.phusionpassenger.com/binaries/passenger/by_release/4.0.41/rubyext-ruby-2.1.1-x86_64-linux.tar.gz: The requested URL returned error: 404
     Trying next mirror...
     Could not download https://s3.amazonaws.com/phusion-passenger/binaries/passenger/by_release/4.0.41/rubyext-ruby-2.1.1-x86_64-linux.tar.gz: The requested URL returned error: 403
 --> Continuing without passenger_native_support.so.
Unable to autodetect the currently active RVM gem set name. This could happen if you ran this program using 'sudo' instead of 'rvmsudo'. When using RVM, you're always supposed to use 'rvmsudo' instead of 'sudo!'.

Please try rerunning this program using 'rvmsudo'. If that doesn't help, please contact this program's author for support.
Stopping web server... done
/home/brett/repos/arvados/services/api/test/websocket_runner.rb:10:in `block in _system': passenger returned exit code 1 (RuntimeError)

Here's run-tests' diagnostic output:

WORKSPACE=/home/brett/repos/arvados
PATH=/tmp/tmp.3R54wd3YHD/.gem/ruby/2.1.0/bin:/home/brett/.gem/ruby/2.1.0/bin:/home/brett/firefox31:/home/brett/.rvm/gems/ruby-2.1.1/bin:/home/brett/.rvm/gems/ruby-2.1.1@global/bin:/home/brett/.rvm/rubies/ruby-2.1.1/bin:/bin:/usr/bin:/sbin:/usr/sbin:/usr/games:/home/brett/.local/bin:/usr/local/go/bin:/usr/local/bin:/usr/local/sbin:/usr/local/games:/home/brett/bin:/home/brett/.rvm/bin:/home/brett/.gem/ruby/2.1.0/bin
Will install dependencies to /home/brett/.gem/ruby/2.1.0
Will install arvados gems to /tmp/tmp.3R54wd3YHD/.gem/ruby/2.1.0
Gem search path is GEM_PATH=/tmp/tmp.3R54wd3YHD/.gem/ruby/2.1.0:/home/brett/.gem/ruby/2.1.0:/home/brett/.rvm/rubies/ruby-2.1.1/lib/ruby/gems/2.1.0

Here's the output of my rvm info:

ruby-2.1.1:

  system:  
    uname:       "Linux brinstar 3.12-0.bpo.1-amd64 #1 SMP Debian 3.12.9-1~bpo70+1 (2014-02-07) x86_64 GNU/Linux" 
    system:      "debian/7/x86_64" 
    bash:        "/bin/bash => GNU bash, version 4.2.37(1)-release (x86_64-pc-linux-gnu)" 
    zsh:         "/bin/zsh => zsh 4.3.17 (x86_64-unknown-linux-gnu)" 

  rvm:
    version:      "rvm 1.25.20 (stable) by Wayne E. Seguin <wayneeseguin@gmail.com>, Michal Papis <mpapis@gmail.com> [https://rvm.io/]" 
    updated:      "8 months 25 days 56 minutes 3 seconds ago" 
    path:         "/home/brett/.rvm" 

  ruby:
    interpreter:  "ruby" 
    version:      "2.1.1p76" 
    date:         "2014-02-24" 
    platform:     "x86_64-linux" 
    patchlevel:   "2014-02-24 revision 45161" 
    full_version: "ruby 2.1.1p76 (2014-02-24 revision 45161) [x86_64-linux]" 

  homes:
    gem:          "/home/brett/.rvm/gems/ruby-2.1.1" 
    ruby:         "/home/brett/.rvm/rubies/ruby-2.1.1" 

  binaries:
    ruby:         "/home/brett/.rvm/rubies/ruby-2.1.1/bin/ruby" 
    irb:          "/home/brett/.rvm/rubies/ruby-2.1.1/bin/irb" 
    gem:          "/home/brett/.rvm/rubies/ruby-2.1.1/bin/gem" 
    rake:         "/home/brett/.rvm/gems/ruby-2.1.1/bin/rake" 

  environment:
    PATH:         "/home/brett/.rvm/gems/ruby-2.1.1/bin:/home/brett/.rvm/gems/ruby-2.1.1@global/bin:/home/brett/.rvm/rubies/ruby-2.1.1/bin:/bin:/usr/bin:/sbin:/usr/sbin:/usr/games:/home/brett/.local/bin:/usr/local/go/bin:/usr/local/bin:/usr/local/sbin:/usr/local/games:/home/brett/bin:/home/brett/.rvm/bin:/home/brett/.gem/ruby/2.1.0/bin" 
    GEM_HOME:     "/home/brett/.rvm/gems/ruby-2.1.1" 
    GEM_PATH:     "/home/brett/.rvm/gems/ruby-2.1.1:/home/brett/.rvm/gems/ruby-2.1.1@global" 
    MY_RUBY_HOME: "/home/brett/.rvm/rubies/ruby-2.1.1" 
    IRBRC:        "/home/brett/.rvm/rubies/ruby-2.1.1/.irbrc" 
    RUBYOPT:      "" 
    gemset:       "" 

ruby-2.1.5:

  system:
    uname:       "Linux tom.shell.9tee4.arvadosapi.com 3.14-0.bpo.2-amd64 #1 SMP Debian 3.14.15-2~bpo70+1 (2014-08-21) x86_64 GNU/Linux" 
    system:      "debian/7/x86_64" 
    bash:        "/bin/bash => GNU bash, version 4.2.37(1)-release (x86_64-pc-linux-gnu)" 
    zsh:         " => not installed" 

  rvm:
    version:      "rvm 1.26.4 (latest) by Wayne E. Seguin <wayneeseguin@gmail.com>, Michal Papis <mpapis@gmail.com> [https://rvm.io/]" 
    updated:      "1 hour 54 minutes 34 seconds ago" 
    path:         "/usr/local/rvm" 

  ruby:
    interpreter:  "ruby" 
    version:      "2.1.5p273" 
    date:         "2014-11-13" 
    platform:     "x86_64-linux" 
    patchlevel:   "2014-11-13 revision 48405" 
    full_version: "ruby 2.1.5p273 (2014-11-13 revision 48405) [x86_64-linux]" 

  homes:
    gem:          "/usr/local/rvm/gems/ruby-2.1.5" 
    ruby:         "/usr/local/rvm/rubies/ruby-2.1.5" 

  binaries:
    ruby:         "/usr/local/rvm/rubies/ruby-2.1.5/bin/ruby" 
    irb:          "/usr/local/rvm/rubies/ruby-2.1.5/bin/irb" 
    gem:          "/usr/local/rvm/rubies/ruby-2.1.5/bin/gem" 
    rake:         "/usr/local/rvm/rubies/ruby-2.1.5/bin/rake" 

  environment:
    PATH:         "/usr/local/rvm/gems/ruby-2.1.5/bin:/usr/local/rvm/gems/ruby-2.1.5@global/bin:/usr/local/rvm/rubies/ruby-2.1.5/bin:/usr/local/rvm/bin:/home/tomclegg/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/usr/local/go/bin:/usr/local/go/bin" 
    GEM_HOME:     "/usr/local/rvm/gems/ruby-2.1.5" 
    GEM_PATH:     "/usr/local/rvm/gems/ruby-2.1.5:/usr/local/rvm/gems/ruby-2.1.5@global" 
    MY_RUBY_HOME: "/usr/local/rvm/rubies/ruby-2.1.5" 
    IRBRC:        "/usr/local/rvm/rubies/ruby-2.1.5/.irbrc" 
    RUBYOPT:      "" 
    gemset:       "" 

WORKSPACE=/home/tomclegg/src/arvados
PATH=/tmp/tmp.7gulvkJKHj/.gem/ruby/2.1.0/bin:/home/tomclegg/.gem/ruby/2.1.0/bin:/usr/local/rvm/gems/ruby-2.1.5/bin:/usr/local/rvm/gems/ruby-2.1.5@global/bin:/usr/local/rvm/rubies/ruby-2.1.5/bin:/usr/local/rvm/bin:/home/tomclegg/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/usr/local/go/bin:/usr/local/go/bin
Will install dependencies to /home/tomclegg/.gem/ruby/2.1.0
Will install arvados gems to /tmp/tmp.7gulvkJKHj/.gem/ruby/2.1.0
Gem search path is GEM_PATH=/tmp/tmp.7gulvkJKHj/.gem/ruby/2.1.0:/home/tomclegg/.gem/ruby/2.1.0:/usr/local/rvm/rubies/ruby-2.1.5/lib/ruby/gems/2.1.0
New python executable in /tmp/tmp.WgsCC2pNJn/bin/python
Installing setuptools, pip...done.