Actions
Bug #4785
closed
[Tests] Test current version of arvados and arvados-cli gems, not other installed versions.
Added by Tom Clegg over 9 years ago. Updated over 9 years ago.
Story points:
0.5
Updated by Tom Clegg over 9 years ago
- Category set to Tests
- Assigned To set to Tom Clegg
Updated by Tom Clegg over 9 years ago
[Copied from #4156 note-18]
Reviewing arvados-dev branch at commit:12a59fa
I'm afraid this doesn't work for me with RVM. It reports:
Failures (4): Fail: cli tests (18s) Fail: apiserver tests (84s) Fail: sdk/python tests (254s) Fail: workbench tests (19s)
The API server, Python SDK, and Workbench test failures all look like this:
--> Skipping compiling of passenger_native_support.so
--> Downloading precompiled passenger_native_support.so for the current Ruby interpreter...
(set PASSENGER_DOWNLOAD_NATIVE_SUPPORT_BINARY=0 to disable)
Could not download https://oss-binaries.phusionpassenger.com/binaries/passenger/by_release/4.0.41/rubyext-ruby-2.1.1-x86_64-linux.tar.gz: The requested URL returned error: 404
Trying next mirror...
Could not download https://s3.amazonaws.com/phusion-passenger/binaries/passenger/by_release/4.0.41/rubyext-ruby-2.1.1-x86_64-linux.tar.gz: The requested URL returned error: 403
--> Continuing without passenger_native_support.so.
Unable to autodetect the currently active RVM gem set name. This could happen if you ran this program using 'sudo' instead of 'rvmsudo'. When using RVM, you're always supposed to use 'rvmsudo' instead of 'sudo!'.
Please try rerunning this program using 'rvmsudo'. If that doesn't help, please contact this program's author for support.
Stopping web server... done
/home/brett/repos/arvados/services/api/test/websocket_runner.rb:10:in `block in _system': passenger returned exit code 1 (RuntimeError)
Here's run-tests' diagnostic output:
WORKSPACE=/home/brett/repos/arvados PATH=/tmp/tmp.3R54wd3YHD/.gem/ruby/2.1.0/bin:/home/brett/.gem/ruby/2.1.0/bin:/home/brett/firefox31:/home/brett/.rvm/gems/ruby-2.1.1/bin:/home/brett/.rvm/gems/ruby-2.1.1@global/bin:/home/brett/.rvm/rubies/ruby-2.1.1/bin:/bin:/usr/bin:/sbin:/usr/sbin:/usr/games:/home/brett/.local/bin:/usr/local/go/bin:/usr/local/bin:/usr/local/sbin:/usr/local/games:/home/brett/bin:/home/brett/.rvm/bin:/home/brett/.gem/ruby/2.1.0/bin Will install dependencies to /home/brett/.gem/ruby/2.1.0 Will install arvados gems to /tmp/tmp.3R54wd3YHD/.gem/ruby/2.1.0 Gem search path is GEM_PATH=/tmp/tmp.3R54wd3YHD/.gem/ruby/2.1.0:/home/brett/.gem/ruby/2.1.0:/home/brett/.rvm/rubies/ruby-2.1.1/lib/ruby/gems/2.1.0
Here's the output of my rvm info:
ruby-2.1.1:
system:
uname: "Linux brinstar 3.12-0.bpo.1-amd64 #1 SMP Debian 3.12.9-1~bpo70+1 (2014-02-07) x86_64 GNU/Linux"
system: "debian/7/x86_64"
bash: "/bin/bash => GNU bash, version 4.2.37(1)-release (x86_64-pc-linux-gnu)"
zsh: "/bin/zsh => zsh 4.3.17 (x86_64-unknown-linux-gnu)"
rvm:
version: "rvm 1.25.20 (stable) by Wayne E. Seguin <wayneeseguin@gmail.com>, Michal Papis <mpapis@gmail.com> [https://rvm.io/]"
updated: "8 months 25 days 56 minutes 3 seconds ago"
path: "/home/brett/.rvm"
ruby:
interpreter: "ruby"
version: "2.1.1p76"
date: "2014-02-24"
platform: "x86_64-linux"
patchlevel: "2014-02-24 revision 45161"
full_version: "ruby 2.1.1p76 (2014-02-24 revision 45161) [x86_64-linux]"
homes:
gem: "/home/brett/.rvm/gems/ruby-2.1.1"
ruby: "/home/brett/.rvm/rubies/ruby-2.1.1"
binaries:
ruby: "/home/brett/.rvm/rubies/ruby-2.1.1/bin/ruby"
irb: "/home/brett/.rvm/rubies/ruby-2.1.1/bin/irb"
gem: "/home/brett/.rvm/rubies/ruby-2.1.1/bin/gem"
rake: "/home/brett/.rvm/gems/ruby-2.1.1/bin/rake"
environment:
PATH: "/home/brett/.rvm/gems/ruby-2.1.1/bin:/home/brett/.rvm/gems/ruby-2.1.1@global/bin:/home/brett/.rvm/rubies/ruby-2.1.1/bin:/bin:/usr/bin:/sbin:/usr/sbin:/usr/games:/home/brett/.local/bin:/usr/local/go/bin:/usr/local/bin:/usr/local/sbin:/usr/local/games:/home/brett/bin:/home/brett/.rvm/bin:/home/brett/.gem/ruby/2.1.0/bin"
GEM_HOME: "/home/brett/.rvm/gems/ruby-2.1.1"
GEM_PATH: "/home/brett/.rvm/gems/ruby-2.1.1:/home/brett/.rvm/gems/ruby-2.1.1@global"
MY_RUBY_HOME: "/home/brett/.rvm/rubies/ruby-2.1.1"
IRBRC: "/home/brett/.rvm/rubies/ruby-2.1.1/.irbrc"
RUBYOPT: ""
gemset: ""
Updated by Tom Clegg over 9 years ago
Meanwhile...
tomclegg@tom.9tee4:~/src/arvados (master)$ rvm info; time ~/src/arvados-dev/jenkins/run-tests.sh WORKSPACE=/home/tomclegg/src/arvados apiserver_test="TESTOPTS=-v"
ruby-2.1.5:
system:
uname: "Linux tom.shell.9tee4.arvadosapi.com 3.14-0.bpo.2-amd64 #1 SMP Debian 3.14.15-2~bpo70+1 (2014-08-21) x86_64 GNU/Linux"
system: "debian/7/x86_64"
bash: "/bin/bash => GNU bash, version 4.2.37(1)-release (x86_64-pc-linux-gnu)"
zsh: " => not installed"
rvm:
version: "rvm 1.26.4 (latest) by Wayne E. Seguin <wayneeseguin@gmail.com>, Michal Papis <mpapis@gmail.com> [https://rvm.io/]"
updated: "1 hour 54 minutes 34 seconds ago"
path: "/usr/local/rvm"
ruby:
interpreter: "ruby"
version: "2.1.5p273"
date: "2014-11-13"
platform: "x86_64-linux"
patchlevel: "2014-11-13 revision 48405"
full_version: "ruby 2.1.5p273 (2014-11-13 revision 48405) [x86_64-linux]"
homes:
gem: "/usr/local/rvm/gems/ruby-2.1.5"
ruby: "/usr/local/rvm/rubies/ruby-2.1.5"
binaries:
ruby: "/usr/local/rvm/rubies/ruby-2.1.5/bin/ruby"
irb: "/usr/local/rvm/rubies/ruby-2.1.5/bin/irb"
gem: "/usr/local/rvm/rubies/ruby-2.1.5/bin/gem"
rake: "/usr/local/rvm/rubies/ruby-2.1.5/bin/rake"
environment:
PATH: "/usr/local/rvm/gems/ruby-2.1.5/bin:/usr/local/rvm/gems/ruby-2.1.5@global/bin:/usr/local/rvm/rubies/ruby-2.1.5/bin:/usr/local/rvm/bin:/home/tomclegg/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/usr/local/go/bin:/usr/local/go/bin"
GEM_HOME: "/usr/local/rvm/gems/ruby-2.1.5"
GEM_PATH: "/usr/local/rvm/gems/ruby-2.1.5:/usr/local/rvm/gems/ruby-2.1.5@global"
MY_RUBY_HOME: "/usr/local/rvm/rubies/ruby-2.1.5"
IRBRC: "/usr/local/rvm/rubies/ruby-2.1.5/.irbrc"
RUBYOPT: ""
gemset: ""
WORKSPACE=/home/tomclegg/src/arvados
PATH=/tmp/tmp.7gulvkJKHj/.gem/ruby/2.1.0/bin:/home/tomclegg/.gem/ruby/2.1.0/bin:/usr/local/rvm/gems/ruby-2.1.5/bin:/usr/local/rvm/gems/ruby-2.1.5@global/bin:/usr/local/rvm/rubies/ruby-2.1.5/bin:/usr/local/rvm/bin:/home/tomclegg/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/usr/local/go/bin:/usr/local/go/bin
Will install dependencies to /home/tomclegg/.gem/ruby/2.1.0
Will install arvados gems to /tmp/tmp.7gulvkJKHj/.gem/ruby/2.1.0
Gem search path is GEM_PATH=/tmp/tmp.7gulvkJKHj/.gem/ruby/2.1.0:/home/tomclegg/.gem/ruby/2.1.0:/usr/local/rvm/rubies/ruby-2.1.5/lib/ruby/gems/2.1.0
New python executable in /tmp/tmp.WgsCC2pNJn/bin/python
Installing setuptools, pip...done.
api server tests pass → 336 tests, 1475 assertions, 0 failures, 0 errors, 0 skips
Updated by Tom Clegg over 9 years ago
In passenger-4.0.41/lib/phusion_passenger/platform_info/ruby.rb there's this:
# Detect usage of gem-wrappers: https://github.com/rvm/gem-wrappers
# This is currently used by RVM >= 1.25, although it's not exclusive to RVM.
if GEM_HOME && File.exist?("#{GEM_HOME}/wrappers/ruby")
return "#{GEM_HOME}/wrappers/ruby"
end
(Brett doesn't have GEM_HOME/wrappers/ruby. I do.)
If GEM_HOME/wrappers/ruby doesn't exist, passenger tries to find the right ruby path by examining various rvm artifacts, a process which can end badly at "Unable to autodetect the currently active RVM gem".
Updated by Tom Clegg over 9 years ago
4156-bundle-install-user @ 7221608
- Leave GEM_HOME alone if it's set. (If rvm has chosen it, it should be writable, and a suitable place to install deps.)
- Don't use
--user-install; that overrides GEM_HOME.
Updated by Brett Smith over 9 years ago
Tom Clegg wrote:
In
passenger-4.0.41/lib/phusion_passenger/platform_info/ruby.rbthere's this:[...]
(Brett doesn't have GEM_HOME/wrappers/ruby. I do.)
It's a little more complicated than that. You're saying that based on the GEM_HOME that the previous version of run-tests generated for me:
(04:20:19 PM) tomclegg: brett does /home/brett/.gem/ruby/2.1.0/wrappers/ruby exist? (04:20:38 PM) Me: Nope.
But that's not my real $GEM_HOME, which does have this wrapper:
brinstar % echo $GEM_HOME /home/brett/.rvm/gems/ruby-2.1.1 brinstar % ls -l "$GEM_HOME/wrappers/ruby" -rwxrwxr-x 1 brett brett 274 Dec 9 19:15 /home/brett/.rvm/gems/ruby-2.1.1/wrappers/ruby
4156-bundle-install-user @ 7221608
This makes the Passenger error go away, but the same set of tests is failing. API server tests all look the same:
1) Failure: WebsocketTest#test_connect,_missing_method [/home/brett/repos/arvados/services/api/test/integration/websocket_test.rb:49]: Should have opened web socket
Python SDK:
======================================================================
FAIL: runTest (tests.test_websockets.WebsocketTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/brett/repos/arvados/sdk/python/tests/test_websockets.py", line 29, in runTest
self.assertIsInstance(self.ws, self.WS_TYPE)
AssertionError: <PollClient(Thread-56, started 140374460786432)> is not an instance of <class 'arvados.events.EventClient'>
Workbench:
EEEEEEEEEEESEEEEFEEEEEEEEEEEEEFEEEEEFEFFFFFEFFFFFFFFFFEFFFFFFEFFFFFF...........Saved ./tmp/workbench-fail-1.png
F...Saved ./tmp/workbench-fail-2.png
FSaved ./tmp/workbench-fail-3.png
FSaved ./tmp/workbench-fail-4.png
FSaved ./tmp/workbench-fail-5.png
FSaved ./tmp/workbench-fail-6.png
FSaved ./tmp/workbench-fail-7.png
F/home/brett/repos/arvados/apps/workbench/app/models/arvados_api_client.rb:144:in `api': Unparseable response from API server (ArvadosApiClient::InvalidApiResponseException)
from /home/brett/repos/arvados/apps/workbench/test/test_helper.rb:272:in `reset_api_fixtures_now'
from /home/brett/repos/arvados/apps/workbench/test/test_helper.rb:257:in `run'
from /home/brett/.rvm/gems/ruby-2.1.1/gems/minitest-5.4.3/lib/minitest.rb:150:in `block in __run'
from /home/brett/.rvm/gems/ruby-2.1.1/gems/minitest-5.4.3/lib/minitest.rb:150:in `map'
from /home/brett/.rvm/gems/ruby-2.1.1/gems/minitest-5.4.3/lib/minitest.rb:150:in `__run'
from /home/brett/.rvm/gems/ruby-2.1.1/gems/minitest-5.4.3/lib/minitest.rb:127:in `run'
from /home/brett/.rvm/gems/ruby-2.1.1/gems/minitest-5.4.3/lib/minitest.rb:56:in `block in autorun'
API server is running, pid 10072
Sending TERM to API server, pid 10072
API server is running, pid 10035
Sending TERM to API server, pid 10035
SimpleCov failed to recognize the test framework and/or suite used. Please specify manually using SimpleCov.command_name 'Unit Tests'.
Coverage report generated for Unknown Test Framework to /home/brett/repos/arvados/apps/workbench/coverage. 750 / 2550 LOC (29.41%) covered.
Coverage report Rcov style generated for Unknown Test Framework to /home/brett/repos/arvados/apps/workbench/coverage/rcov
These are the same sorts of failures I saw in my own branch, if that's helpful at all.
Updated by Tom Clegg over 9 years ago
OK, let's try 8bfa3e7...
Passenger invokes special behavior when rvm is in use, and expects everything to look like nice clean rvm. Therefore we cannot alter our GEM_* vars.
New approach when rvm is in use:- use the current ruby.
- create/use a single rvm gemset "@arvados-tests" for dependencies and built gems.
- remove any existing arvados and arvados-cli gems from this gemset.
- no need to alter GEM_HOME when installing gems.
The "detect whether rvm is in use" block is kind of ugly, but (somewhat surprisingly) there seems to be no command for "where is the setup script". This really is the documented way to use rvm in a script.
Updated by Brett Smith over 9 years ago
8bfa3e7 is functional, thanks. A few small comments on the code itself:
- Would it make more sense to uninstall the arvados and arvados-cli gems at the top of their respective install functions? I worry that, with the current scheme, we might forget to update the uninstall list if we add a gem in the future.
- In setup_ruby_environment's non-RVM branch:
- The first comment still talks about "If rvm is in use," which no longer makes sense in this context.
- Should the first echo happen after we set $GEM_HOME? It seems like a more useful diagonstic that way.
- I think we're cutting tmpdir_gem_home out of gempath twice, once with cut and once with the bash string manipulation syntax.
- Should there be a blank line after the setup_ruby_environment definition? It seems like we use that style pretty consistently across all our code.
While I was reviewing this, I found and filed #4821, but I don't believe that's related to anything you've done here. Thanks.
Updated by Tom Clegg over 9 years ago
Brett Smith wrote:
8bfa3e7 is functional, thanks. A few small comments on the code itself:
- Would it make more sense to uninstall the arvados and arvados-cli gems at the top of their respective install functions? I worry that, with the current scheme, we might forget to update the uninstall list if we add a gem in the future.
Good plan, fixed.
- In setup_ruby_environment's non-RVM branch:
- The first comment still talks about "If rvm is in use," which no longer makes sense in this context.
Fixed.
- Should the first echo happen after we set $GEM_HOME? It seems like a more useful diagonstic that way.
Removed. (I think I meant to say GEMHOME there. Anyway, the real information is provided by a later echo, which I updated to report $(gem env gemdir) instead of $GEM_HOME. We assume they're the same thing, but gem env seems like a better authority on the subject of what's really going to happen.)
- I think we're cutting tmpdir_gem_home out of gempath twice, once with cut and once with the bash string manipulation syntax.
Fixed (now just the cut).
- Should there be a blank line after the setup_ruby_environment definition? It seems like we use that style pretty consistently across all our code.
Fixed.
Now at commit:3d3cee4
Updated by Brett Smith over 9 years ago
Tom Clegg wrote:
- Should the first echo happen after we set $GEM_HOME? It seems like a more useful diagonstic that way.
Removed.
I don't see this as of commit:3d3cee41 (although I do see the change you discussed to the later echo). But whether you patch it up or not, I think this is good to merge. Thanks.
Actions