Arvados

workbench.c97qk:/home/nico# dpkg -i arvados-workbench_0.1.20160104175817.7fa91fc-3_amd64.deb 
(Reading database ... 64673 files and directories currently installed.)
Preparing to unpack arvados-workbench_0.1.20160104175817.7fa91fc-3_amd64.deb ...
Unpacking arvados-workbench (0.1.20160104175817.7fa91fc-3) over (0.1.20151218204040.2b699de-1) ...
dpkg: warning: unable to delete old directory '/var/www/arvados-workbench/current/tmp/cache': Directory not empty
dpkg: warning: unable to delete old directory '/var/www/arvados-workbench/current/tmp': Directory not empty
Setting up arvados-workbench (0.1.20160104175817.7fa91fc-3) ...

Assumption: nginx is configured to serve Rails from
            /var/www/arvados-workbench/current
Assumption: nginx and passenger run as www-data:www-data

Creating symlinks to configuration in /etc/arvados/workbench ...... done.
Running bundle install... done.
Ensuring directory and file permissions ...... done.
Checking application.yml for completeness...action_controller.perform_caching true
activation_contact_link          mailto:beta@curoverse.com
/.../
shell_in_a_box_url               https://webshell.c97qk.arvadosapi.com/%{hostname}
show_user_agreement_inline       false
site_name                        Curoverse
source_version                   2b699de-1 
support_email_address            support@curoverse.com
user_profile_form_fields         [{"key"=>"organization", "type"=>"text", "form_field_title"=>"Organization", "form_field_description"=>"Your company or institution", "required"=>true}, {"key"=>"organization_email", "type"=>"text", "form_field_title"=>"E-mail at Organization", "form_field_description"=>"Your corporate or institutional e-mail address", "required"=>true}, {"key"=>"lab", "type"=>"text", "form_field_title"=>"Department or Lab", "form_field_description"=>"Your lab or organizational unit"}, {"key"=>"website_url", "type"=>"text", "form_field_title"=>"Website", "form_field_description"=>"Your website url"}, {"key"=>"role", "type"=>"select", "form_field_title"=>"Role", "form_field_description"=>"Choose the category that best describes your role in your organization.", "options"=>["Bio-informatician", "Data Scientist", "Analyst", "Researcher", "Software Developer", "System Administrator", "Other"]}]
user_profile_form_message        Welcome to Curoverse. All <span style="color:red">required fields</span> must be completed before you can proceed.
 done.
Precompiling assets... done.
 * Restarting nginx nginx
   ...done.

but something is broken after that .

investigating

this is odd! I get the following

Refusing to start in production mode with missing configuration.

The following configuration settings must be specified in
config/application.yml:
* arvados_login_base
* arvados_v1_base
* arvados_insecure_https
* secret_token

 (RuntimeError)
  /var/www/arvados-workbench/current/config/load_config.rb:46:in `block in <top (required)>'
  /var/www/arvados-workbench/shared/vendor_bundle/ruby/2.1.0/gems/railties-4.1.12/lib/rails/railtie.rb:210:in `instance_eval'
  /var/www/arvados-workbench/shared/vendor_bundle/ruby/2.1.0/gems/railties-4.1.12/lib/rails/railtie.rb:210:in `configure'
  /var/www/arvados-workbench/shared/vendor_bundle/ruby/2.1.0/gems/railties-4.1.12/lib/rails/railtie.rb:182:in `configure'
  /var/www/arvados-workbench/current/config/load_config.rb:16:in `<top (required)>'
  /var/www/arvados-workbench/shared/vendor_bundle/ruby/2.1.0/gems/activesupport-4.1.12/lib/active_support/dependencies.rb:247:in `require'
  /var/www/arvados-workbench/shared/vendor_bundle/ruby/2.1.0/gems/activesupport-4.1.12/lib/active_support/dependencies.rb:247:in `block in require'
  /var/www/arvados-workbench/shared/vendor_bundle/ruby/2.1.0/gems/activesupport-4.1.12/lib/active_support/dependencies.rb:232:in `load_dependency'
  /var/www/arvados-workbench/shared/vendor_bundle/ruby/2.1.0/gems/activesupport-4.1.12/lib/active_support/dependencies.rb:247:in `require'
  /var/www/arvados-workbench/current/config/application.rb:54:in `<top (required)>'
  /var/www/arvados-workbench/current/config/environment.rb:2:in `require'
  /var/www/arvados-workbench/current/config/environment.rb:2:in `<top (required)>'
  /var/www/arvados-workbench/current/config.ru:3:in `require'
  /var/www/arvados-workbench/current/config.ru:3:in `block in <main>'
  /var/www/arvados-workbench/shared/vendor_bundle/ruby/2.1.0/gems/rack-1.5.5/lib/rack/builder.rb:55:in `instance_eval'
  /var/www/arvados-workbench/shared/vendor_bundle/ruby/2.1.0/gems/rack-1.5.5/lib/rack/builder.rb:55:in `initialize'
  /var/www/arvados-workbench/current/config.ru:1:in `new'
  /var/www/arvados-workbench/current/config.ru:1:in `<main>'
  /usr/share/passenger/helper-scripts/rack-preloader.rb:107:in `eval'
  /usr/share/passenger/helper-scripts/rack-preloader.rb:107:in `preload_app'
  /usr/share/passenger/helper-scripts/rack-preloader.rb:153:in `<module:App>'
  /usr/share/passenger/helper-scripts/rack-preloader.rb:29:in `<module:PhusionPassenger>'
  /usr/share/passenger/helper-scripts/rack-preloader.rb:28:in `<main>'

workbench.c97qk:/home/nico# ls -l /var/www/arvados-workbench/current/config/application.yml
lrwxrwxrwx 1 root root 38 Jan  4 20:27 /var/www/arvados-workbench/current/config/application.yml -> /etc/arvados/workbench/application.yml
workbench.c97qk:/home/nico# ls -l /etc/arvados/workbench/application.yml
-rw-r--r-- 1 root root 3340 Nov 11 19:46 /etc/arvados/workbench/application.yml
workbench.c97qk:/home/nico# grep  arvados_login_base /etc/arvados/workbench/application.yml                                                                                 
  arvados_login_base: https://c97qk.arvadosapi.com/login
workbench.c97qk:/home/nico# find / -name application.yml
/var/www/arvados-workbench/current/config/application.yml
/etc/arvados/workbench/application.yml

ok ... the bug(?) is the following

we currently have a COPY of application.yml in /etc/arvados/package and the new package installs a symlink but our current instalations have a restricted permissions there for root only

I fixed this by doing:

chown -R www-data.www-data /etc/arvados

which in my opinion should be reflected the package.

brett please review puppet branch 8014-rails-postinst-scripts-wip

otherwise puppet will undo any changes we propose

Nico Cesar wrote:

ok ... the bug(?) is the following

we currently have a COPY of application.yml in /etc/arvados/package and the new package installs a symlink but our current instalations have a restricted permissions there for root only

I fixed this by doing:

chown -R www-data.www-data /etc/arvados

which in my opinion should be reflected the package.

It would be semi-evil to overwrite the access bits of stuff in /etc willy-nilly. I think that could potentially surprise a lot of users. In commit:15d90bd I have updated the scripts to set better defaults, including in the situation where we're migrating from a previous configuration (e.g., an old package using arvados-*-upgrade.sh). You won't see the changes take effect on c97qk since that migration has already happened, but you should be able to see it on other test clusters.

Nico Cesar wrote:

brett please review puppet branch 8014-rails-postinst-scripts-wip

otherwise puppet will undo any changes we propose

In the last change to the Workbench module, the last switch to www-data isn't quoted. Would that be a syntax issue?

Besides that, this is fine to merge. As discussed, what would be even better is to have this whole tree owned by root:www-data, with 0750 permissions for directories and 0640 permissions for files.

merged 8014-rails-postinst-scripts-wip on puppet ...

now I'm building new packages to review 15d90bd57ef3f2e284579bafd8815ceaf33484ed to see how it behaves in c97qk

Nico Cesar wrote:

merged 8014-rails-postinst-scripts-wip on puppet ...

now I'm building new packages to review 15d90bd57ef3f2e284579bafd8815ceaf33484ed to see how it behaves in c97qk

I found this after installing:

c97qk:~# ls -l /etc/arvados/api/ -d
drwxr-x--- 2 root root 4096 Jan  4 20:22 /etc/arvados/api/
c97qk:~# ls -l /etc/arvados/ -d
drwxr-x--- 3 root root 4096 Oct 16 15:13 /etc/arvados/

Which as far as I can tell was how it was supposed to work... since puppet cant run on those servers to repair it (because now is a DOWNGRADED status and puppet cant see #6623 ) I did it manually

But for the record this merge LGTM ... after merging we need to enable https://ci.curoverse.com/view/Arvados%20build%20pipeline/job/deploy-to-c97qk/ again.