Actions
Bug #22349
closed
RHEL8 Appstream Ruby not useable on 3.0
Added by Ivan Diaz Alvarez 4 months ago. Updated 4 months ago.
Status:
Resolved
Priority:
Normal
Assigned To:
-
Category:
-
Target version:
-
Story points:
-
Release:
Release relationship:
Auto
Description
We did an ARVQA update with 3.0 and the 3.1 Ruby Appstream, as recommended in the upgrade notes, this couldn't progress because of unhandled passenger/nginx dependencies, RHEL8 doesn't handle those gracefully, so in the end we are upgrading with RVM/Ruby 2.7.2 which caused no problems (we did a RVM/3.1 install but required many manual steps).
How can the upgrade notes be followed on a RHEL8 installation (with EPEL)? What versions can be used?
Updated by Brett Smith 4 months ago
Confirmed that the very latest Passenger package for RHEL8 is still dutifully built against the original system Ruby, 2.5, and depends on that specific version. Therefore it is not possible to install that package alongside our new API server package.
Our API server package already installs the Passenger gem for the same version of Ruby it's using. I propose we stop running the RailsAPI server through nginx and instead ship a systemd service to run Passenger standalone. That should be safe since controller is the only client with direct access, and this would simplify installation on all platforms: users would no longer need to (explicitly) install Passenger or configure it inside nginx.
Updated by Brett Smith 4 months ago
- Status changed from New to In Progress
Now that we expect to use distro Ruby arvados-api-server needs to declare its dependencies on: ruby, ruby-devel, zlib-devel (we do this for Debian but not Red Hat).
If we do go ahead with the systemd plan, the postinst script needs to be updated to remove detection of a web server. We'll need to figure out another ownership plan.
Updated by Brett Smith 4 months ago
102 gems installed
done.
Running bundle install...Don't run Bundler as root. Installing your bundle as root will break this application for all non-root users on this machine.
done.
Verifying bundle is complete... done.
Assumption: nginx is configured to serve Rails from
/var/www/arvados-api/current
Assumption: nginx and passenger run as nginx
Creating symlinks to configuration in /etc/arvados/api .../var/tmp/rpm-tmp.wAs9IP: line 204: cmp: command not found
... done.
Ensuring directory and file permissions ...chown: cannot access '/var/www/arvados-api/current/db/schema.rb': No such file or directory
... done.
Checking configuration for completeness...time="2024-11-27T10:56:55.100467296-05:00" level=warning msg="deprecated or unknown config entry: Clusters.z2a09.Containers.CloudVMs.DeployRunnerDirectory"
time="2024-11-27T10:56:55.103973167-05:00" level=warning msg="Clusters.z2a09.Collections.BlobSigningKey: secret token is not set (use 32+ random characters from a-z, A-Z, 0-9)"
time="2024-11-27T10:56:55.103991844-05:00" level=warning msg="LocalKeepBlobBuffersPerVCPU is 1 but will not be used because at least one volume (SAMPLE) uses AccessViaHosts -- suggest changing to 0"
Defaulting to memory cache, because /var/www/arvados-api/current/tmp/cache does not exist
done.
NOTE: The arvados-api-server package was not configured completely because
database setup could not be completed.
There are a few minor issues here that should be addressed but I don't understand why it says database setup wasn't completed, there's nothing in here about that.
Updated by Peter Amstutz 4 months ago
Brett Smith wrote in #note-3:
[...]
There are a few minor issues here that should be addressed but I don't understand why it says database setup wasn't completed, there's nothing in here about that.
It may just be that the script says that if there's any error because the job of the postinst script is mainly to set up the database.
Updated by Brett Smith 4 months ago
+ run_and_report 'Setting up database' bin/rake db:schema:load db:seed + local 'action_message=Setting up database' + shift + local retcode=0 + echo -n 'Setting up database...' Setting up database...+ bin/rake db:schema:load db:seed WARN[2024-11-27T11:49:11.771649185-05:00] deprecated or unknown config entry: Clusters.z2a09.Containers.CloudVMs.DeployRunnerDirectory WARN[2024-11-27T11:49:11.774778259-05:00] LocalKeepBlobBuffersPerVCPU is 1 but will not be used because at least one volume (SAMPLE) uses AccessViaHosts -- suggest changing to 0 Defaulting to memory cache, because /var/www/arvados-api/current/tmp/cache does not exist rake aborted! ActionView::Template::Error: both URI are relative /var/www/arvados-api/current/app/views/admin_notifier/new_user.text.erb:14:in `_app_views_admin_notifier_new_user_text_erb__2952295690788535752_10640' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionview-7.0.8.4/lib/action_view/base.rb:244:in `public_send' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionview-7.0.8.4/lib/action_view/base.rb:244:in `_run' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionview-7.0.8.4/lib/action_view/template.rb:157:in `block in render' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activesupport-7.0.8.4/lib/active_support/notifications.rb:208:in `instrument' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionview-7.0.8.4/lib/action_view/template.rb:361:in `instrument_render_template' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionview-7.0.8.4/lib/action_view/template.rb:155:in `render' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionview-7.0.8.4/lib/action_view/renderer/template_renderer.rb:65:in `block (2 levels) in render_template' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activesupport-7.0.8.4/lib/active_support/notifications.rb:208:in `instrument' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionview-7.0.8.4/lib/action_view/renderer/template_renderer.rb:60:in `block in render_template' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionview-7.0.8.4/lib/action_view/renderer/template_renderer.rb:79:in `render_with_layout' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionview-7.0.8.4/lib/action_view/renderer/template_renderer.rb:59:in `render_template' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionview-7.0.8.4/lib/action_view/renderer/template_renderer.rb:11:in `render' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionview-7.0.8.4/lib/action_view/renderer/renderer.rb:61:in `render_template_to_object' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionview-7.0.8.4/lib/action_view/renderer/renderer.rb:29:in `render_to_object' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionview-7.0.8.4/lib/action_view/rendering.rb:117:in `block in _render_template' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionview-7.0.8.4/lib/action_view/base.rb:270:in `in_rendering_context' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionview-7.0.8.4/lib/action_view/rendering.rb:116:in `_render_template' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionview-7.0.8.4/lib/action_view/rendering.rb:103:in `render_to_body' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionpack-7.0.8.4/lib/abstract_controller/rendering.rb:27:in `render' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionmailer-7.0.8.4/lib/action_mailer/base.rb:992:in `block in collect_responses_from_templates' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionmailer-7.0.8.4/lib/action_mailer/base.rb:989:in `each' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionmailer-7.0.8.4/lib/action_mailer/base.rb:989:in `each' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionmailer-7.0.8.4/lib/action_mailer/base.rb:989:in `map' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionmailer-7.0.8.4/lib/action_mailer/base.rb:989:in `collect_responses_from_templates' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionmailer-7.0.8.4/lib/action_mailer/base.rb:967:in `collect_responses' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionmailer-7.0.8.4/lib/action_mailer/base.rb:875:in `mail' /var/www/arvados-api/current/app/mailers/admin_notifier.rb:21:in `new_user' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionpack-7.0.8.4/lib/abstract_controller/base.rb:215:in `process_action' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionpack-7.0.8.4/lib/abstract_controller/callbacks.rb:234:in `block in process_action' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activesupport-7.0.8.4/lib/active_support/callbacks.rb:99:in `run_callbacks' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionpack-7.0.8.4/lib/abstract_controller/callbacks.rb:233:in `process_action' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionpack-7.0.8.4/lib/abstract_controller/base.rb:151:in `process' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionmailer-7.0.8.4/lib/action_mailer/rescuable.rb:27:in `block in process' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionmailer-7.0.8.4/lib/action_mailer/rescuable.rb:19:in `handle_exceptions' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionmailer-7.0.8.4/lib/action_mailer/rescuable.rb:26:in `process' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionview-7.0.8.4/lib/action_view/rendering.rb:39:in `process' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionmailer-7.0.8.4/lib/action_mailer/base.rb:646:in `block in process' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activesupport-7.0.8.4/lib/active_support/notifications.rb:206:in `block in instrument' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activesupport-7.0.8.4/lib/active_support/notifications/instrumenter.rb:24:in `instrument' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activesupport-7.0.8.4/lib/active_support/notifications.rb:206:in `instrument' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionmailer-7.0.8.4/lib/action_mailer/base.rb:645:in `process' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionmailer-7.0.8.4/lib/action_mailer/message_delivery.rb:128:in `block in processed_mailer' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/andand-1.3.3/lib/andand.rb:60:in `me' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionmailer-7.0.8.4/lib/action_mailer/message_delivery.rb:127:in `processed_mailer' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionmailer-7.0.8.4/lib/action_mailer/message_delivery.rb:118:in `deliver_now' /var/www/arvados-api/current/app/models/user.rb:926:in `send_admin_notifications' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activesupport-7.0.8.4/lib/active_support/callbacks.rb:400:in `block in make_lambda' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activesupport-7.0.8.4/lib/active_support/callbacks.rb:236:in `block in halting_and_conditional' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activesupport-7.0.8.4/lib/active_support/callbacks.rb:599:in `block in invoke_after' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activesupport-7.0.8.4/lib/active_support/callbacks.rb:599:in `each' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activesupport-7.0.8.4/lib/active_support/callbacks.rb:599:in `invoke_after' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activesupport-7.0.8.4/lib/active_support/callbacks.rb:108:in `run_callbacks' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activesupport-7.0.8.4/lib/active_support/callbacks.rb:929:in `_run_create_callbacks' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activerecord-7.0.8.4/lib/active_record/callbacks.rb:459:in `_create_record' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activerecord-7.0.8.4/lib/active_record/timestamp.rb:108:in `_create_record' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activerecord-7.0.8.4/lib/active_record/persistence.rb:1069:in `create_or_update' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activerecord-7.0.8.4/lib/active_record/callbacks.rb:455:in `block in create_or_update' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activesupport-7.0.8.4/lib/active_support/callbacks.rb:118:in `block in run_callbacks' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activerecord-7.0.8.4/lib/active_record/autosave_association.rb:370:in `around_save_collection_association' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activesupport-7.0.8.4/lib/active_support/callbacks.rb:127:in `block in run_callbacks' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activesupport-7.0.8.4/lib/active_support/callbacks.rb:138:in `run_callbacks' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activesupport-7.0.8.4/lib/active_support/callbacks.rb:929:in `_run_save_callbacks' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activerecord-7.0.8.4/lib/active_record/callbacks.rb:455:in `create_or_update' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activerecord-7.0.8.4/lib/active_record/timestamp.rb:126:in `create_or_update' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activerecord-7.0.8.4/lib/active_record/persistence.rb:648:in `save!' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activerecord-7.0.8.4/lib/active_record/validations.rb:53:in `save!' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activerecord-7.0.8.4/lib/active_record/transactions.rb:302:in `block in save!' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activerecord-7.0.8.4/lib/active_record/transactions.rb:354:in `block in with_transaction_returning_status' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activerecord-7.0.8.4/lib/active_record/connection_adapters/abstract/transaction.rb:319:in `block in within_new_transaction' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activesupport-7.0.8.4/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `handle_interrupt' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activesupport-7.0.8.4/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `block in synchronize' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activesupport-7.0.8.4/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `handle_interrupt' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activesupport-7.0.8.4/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `synchronize' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activerecord-7.0.8.4/lib/active_record/connection_adapters/abstract/transaction.rb:317:in `within_new_transaction' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activerecord-7.0.8.4/lib/active_record/connection_adapters/abstract/database_statements.rb:316:in `transaction' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activerecord-7.0.8.4/lib/active_record/transactions.rb:350:in `with_transaction_returning_status' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activerecord-7.0.8.4/lib/active_record/transactions.rb:302:in `save!' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activerecord-7.0.8.4/lib/active_record/suppressor.rb:54:in `save!' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activerecord-7.0.8.4/lib/active_record/persistence.rb:55:in `create!' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activerecord-7.0.8.4/lib/active_record/relation.rb:870:in `_create!' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activerecord-7.0.8.4/lib/active_record/relation.rb:115:in `block in create!' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activerecord-7.0.8.4/lib/active_record/relation.rb:881:in `_scoping' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activerecord-7.0.8.4/lib/active_record/relation.rb:428:in `scoping' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activerecord-7.0.8.4/lib/active_record/relation.rb:115:in `create!' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activerecord-7.0.8.4/lib/active_record/relation.rb:124:in `first_or_create!' /var/www/arvados-api/current/lib/current_api_client.rb:61:in `block in system_user' /var/www/arvados-api/current/lib/current_api_client.rb:234:in `check_cache' /var/www/arvados-api/current/lib/current_api_client.rb:59:in `system_user' /var/www/arvados-api/current/app/models/database_seeds.rb:11:in `block in install' /var/www/arvados-api/current/lib/update_permissions.rb:280:in `batch_update_permissions' /var/www/arvados-api/current/app/models/database_seeds.rb:10:in `install' /var/www/arvados-api/current/db/seeds.rb:9:in `<top (required)>' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/railties-7.0.8.4/lib/rails/engine.rb:557:in `load' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/railties-7.0.8.4/lib/rails/engine.rb:557:in `block in load_seed' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activesupport-7.0.8.4/lib/active_support/callbacks.rb:118:in `block in run_callbacks' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activesupport-7.0.8.4/lib/active_support/execution_wrapper.rb:92:in `wrap' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/railties-7.0.8.4/lib/rails/engine.rb:626:in `block (2 levels) in <class:Engine>' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activesupport-7.0.8.4/lib/active_support/callbacks.rb:127:in `instance_exec' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activesupport-7.0.8.4/lib/active_support/callbacks.rb:127:in `block in run_callbacks' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activesupport-7.0.8.4/lib/active_support/callbacks.rb:138:in `run_callbacks' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/railties-7.0.8.4/lib/rails/engine.rb:557:in `load_seed' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activerecord-7.0.8.4/lib/active_record/tasks/database_tasks.rb:497:in `load_seed' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activerecord-7.0.8.4/lib/active_record/railties/databases.rake:397:in `block (2 levels) in <top (required)>' Caused by: URI::BadURIError: both URI are relative /var/www/arvados-api/current/app/views/admin_notifier/new_user.text.erb:14:in `_app_views_admin_notifier_new_user_text_erb__2952295690788535752_10640' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionview-7.0.8.4/lib/action_view/base.rb:244:in `public_send' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionview-7.0.8.4/lib/action_view/base.rb:244:in `_run' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionview-7.0.8.4/lib/action_view/template.rb:157:in `block in render' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activesupport-7.0.8.4/lib/active_support/notifications.rb:208:in `instrument' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionview-7.0.8.4/lib/action_view/template.rb:361:in `instrument_render_template' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionview-7.0.8.4/lib/action_view/template.rb:155:in `render' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionview-7.0.8.4/lib/action_view/renderer/template_renderer.rb:65:in `block (2 levels) in render_template' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activesupport-7.0.8.4/lib/active_support/notifications.rb:208:in `instrument' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionview-7.0.8.4/lib/action_view/renderer/template_renderer.rb:60:in `block in render_template' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionview-7.0.8.4/lib/action_view/renderer/template_renderer.rb:79:in `render_with_layout' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionview-7.0.8.4/lib/action_view/renderer/template_renderer.rb:59:in `render_template' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionview-7.0.8.4/lib/action_view/renderer/template_renderer.rb:11:in `render' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionview-7.0.8.4/lib/action_view/renderer/renderer.rb:61:in `render_template_to_object' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionview-7.0.8.4/lib/action_view/renderer/renderer.rb:29:in `render_to_object' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionview-7.0.8.4/lib/action_view/rendering.rb:117:in `block in _render_template' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionview-7.0.8.4/lib/action_view/base.rb:270:in `in_rendering_context' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionview-7.0.8.4/lib/action_view/rendering.rb:116:in `_render_template' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionview-7.0.8.4/lib/action_view/rendering.rb:103:in `render_to_body' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionpack-7.0.8.4/lib/abstract_controller/rendering.rb:27:in `render' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionmailer-7.0.8.4/lib/action_mailer/base.rb:992:in `block in collect_responses_from_templates' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionmailer-7.0.8.4/lib/action_mailer/base.rb:989:in `each' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionmailer-7.0.8.4/lib/action_mailer/base.rb:989:in `each' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionmailer-7.0.8.4/lib/action_mailer/base.rb:989:in `map' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionmailer-7.0.8.4/lib/action_mailer/base.rb:989:in `collect_responses_from_templates' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionmailer-7.0.8.4/lib/action_mailer/base.rb:967:in `collect_responses' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionmailer-7.0.8.4/lib/action_mailer/base.rb:875:in `mail' /var/www/arvados-api/current/app/mailers/admin_notifier.rb:21:in `new_user' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionpack-7.0.8.4/lib/abstract_controller/base.rb:215:in `process_action' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionpack-7.0.8.4/lib/abstract_controller/callbacks.rb:234:in `block in process_action' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activesupport-7.0.8.4/lib/active_support/callbacks.rb:99:in `run_callbacks' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionpack-7.0.8.4/lib/abstract_controller/callbacks.rb:233:in `process_action' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionpack-7.0.8.4/lib/abstract_controller/base.rb:151:in `process' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionmailer-7.0.8.4/lib/action_mailer/rescuable.rb:27:in `block in process' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionmailer-7.0.8.4/lib/action_mailer/rescuable.rb:19:in `handle_exceptions' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionmailer-7.0.8.4/lib/action_mailer/rescuable.rb:26:in `process' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionview-7.0.8.4/lib/action_view/rendering.rb:39:in `process' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionmailer-7.0.8.4/lib/action_mailer/base.rb:646:in `block in process' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activesupport-7.0.8.4/lib/active_support/notifications.rb:206:in `block in instrument' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activesupport-7.0.8.4/lib/active_support/notifications/instrumenter.rb:24:in `instrument' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activesupport-7.0.8.4/lib/active_support/notifications.rb:206:in `instrument' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionmailer-7.0.8.4/lib/action_mailer/base.rb:645:in `process' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionmailer-7.0.8.4/lib/action_mailer/message_delivery.rb:128:in `block in processed_mailer' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/andand-1.3.3/lib/andand.rb:60:in `me' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionmailer-7.0.8.4/lib/action_mailer/message_delivery.rb:127:in `processed_mailer' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/actionmailer-7.0.8.4/lib/action_mailer/message_delivery.rb:118:in `deliver_now' /var/www/arvados-api/current/app/models/user.rb:926:in `send_admin_notifications' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activesupport-7.0.8.4/lib/active_support/callbacks.rb:400:in `block in make_lambda' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activesupport-7.0.8.4/lib/active_support/callbacks.rb:236:in `block in halting_and_conditional' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activesupport-7.0.8.4/lib/active_support/callbacks.rb:599:in `block in invoke_after' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activesupport-7.0.8.4/lib/active_support/callbacks.rb:599:in `each' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activesupport-7.0.8.4/lib/active_support/callbacks.rb:599:in `invoke_after' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activesupport-7.0.8.4/lib/active_support/callbacks.rb:108:in `run_callbacks' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activesupport-7.0.8.4/lib/active_support/callbacks.rb:929:in `_run_create_callbacks' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activerecord-7.0.8.4/lib/active_record/callbacks.rb:459:in `_create_record' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activerecord-7.0.8.4/lib/active_record/timestamp.rb:108:in `_create_record' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activerecord-7.0.8.4/lib/active_record/persistence.rb:1069:in `create_or_update' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activerecord-7.0.8.4/lib/active_record/callbacks.rb:455:in `block in create_or_update' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activesupport-7.0.8.4/lib/active_support/callbacks.rb:118:in `block in run_callbacks' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activerecord-7.0.8.4/lib/active_record/autosave_association.rb:370:in `around_save_collection_association' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activesupport-7.0.8.4/lib/active_support/callbacks.rb:127:in `block in run_callbacks' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activesupport-7.0.8.4/lib/active_support/callbacks.rb:138:in `run_callbacks' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activesupport-7.0.8.4/lib/active_support/callbacks.rb:929:in `_run_save_callbacks' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activerecord-7.0.8.4/lib/active_record/callbacks.rb:455:in `create_or_update' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activerecord-7.0.8.4/lib/active_record/timestamp.rb:126:in `create_or_update' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activerecord-7.0.8.4/lib/active_record/persistence.rb:648:in `save!' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activerecord-7.0.8.4/lib/active_record/validations.rb:53:in `save!' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activerecord-7.0.8.4/lib/active_record/transactions.rb:302:in `block in save!' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activerecord-7.0.8.4/lib/active_record/transactions.rb:354:in `block in with_transaction_returning_status' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activerecord-7.0.8.4/lib/active_record/connection_adapters/abstract/transaction.rb:319:in `block in within_new_transaction' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activesupport-7.0.8.4/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `handle_interrupt' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activesupport-7.0.8.4/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `block in synchronize' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activesupport-7.0.8.4/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `handle_interrupt' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activesupport-7.0.8.4/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `synchronize' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activerecord-7.0.8.4/lib/active_record/connection_adapters/abstract/transaction.rb:317:in `within_new_transaction' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activerecord-7.0.8.4/lib/active_record/connection_adapters/abstract/database_statements.rb:316:in `transaction' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activerecord-7.0.8.4/lib/active_record/transactions.rb:350:in `with_transaction_returning_status' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activerecord-7.0.8.4/lib/active_record/transactions.rb:302:in `save!' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activerecord-7.0.8.4/lib/active_record/suppressor.rb:54:in `save!' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activerecord-7.0.8.4/lib/active_record/persistence.rb:55:in `create!' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activerecord-7.0.8.4/lib/active_record/relation.rb:870:in `_create!' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activerecord-7.0.8.4/lib/active_record/relation.rb:115:in `block in create!' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activerecord-7.0.8.4/lib/active_record/relation.rb:881:in `_scoping' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activerecord-7.0.8.4/lib/active_record/relation.rb:428:in `scoping' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activerecord-7.0.8.4/lib/active_record/relation.rb:115:in `create!' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activerecord-7.0.8.4/lib/active_record/relation.rb:124:in `first_or_create!' /var/www/arvados-api/current/lib/current_api_client.rb:61:in `block in system_user' /var/www/arvados-api/current/lib/current_api_client.rb:234:in `check_cache' /var/www/arvados-api/current/lib/current_api_client.rb:59:in `system_user' /var/www/arvados-api/current/app/models/database_seeds.rb:11:in `block in install' /var/www/arvados-api/current/lib/update_permissions.rb:280:in `batch_update_permissions' /var/www/arvados-api/current/app/models/database_seeds.rb:10:in `install' /var/www/arvados-api/current/db/seeds.rb:9:in `<top (required)>' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/railties-7.0.8.4/lib/rails/engine.rb:557:in `load' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/railties-7.0.8.4/lib/rails/engine.rb:557:in `block in load_seed' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activesupport-7.0.8.4/lib/active_support/callbacks.rb:118:in `block in run_callbacks' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activesupport-7.0.8.4/lib/active_support/execution_wrapper.rb:92:in `wrap' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/railties-7.0.8.4/lib/rails/engine.rb:626:in `block (2 levels) in <class:Engine>' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activesupport-7.0.8.4/lib/active_support/callbacks.rb:127:in `instance_exec' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activesupport-7.0.8.4/lib/active_support/callbacks.rb:127:in `block in run_callbacks' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activesupport-7.0.8.4/lib/active_support/callbacks.rb:138:in `run_callbacks' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/railties-7.0.8.4/lib/rails/engine.rb:557:in `load_seed' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activerecord-7.0.8.4/lib/active_record/tasks/database_tasks.rb:497:in `load_seed' /var/www/arvados-api/shared/vendor_bundle/ruby/3.1.0/gems/activerecord-7.0.8.4/lib/active_record/railties/databases.rake:397:in `block (2 levels) in <top (required)>' Tasks: TOP => db:seed (See full trace by running task with --trace) + retcode=1 + echo ' failed.' failed. + return 1 + NOT_READY_REASON='database setup could not be completed' + '[' -n nginx ']' + chown -R nginx: /var/www/arvados-api/current/tmp + chmod -R 2775 /var/www/arvados-api/current/tmp + '[' -z 'database setup could not be completed' ']' + '[' -n 'database setup could not be completed' ']' + cat NOTE: The arvados-api-server package was not configured completely because database setup could not be completed.
Updated by Brett Smith 4 months ago
So yes, the real error is:
- the default new user template refers to
<%= Rails.configuration.Services.Workbench1.ExternalURL %> - I hadn't set that URL (because I was trying to throw this together as fast as possible, so I hadn't set any external URLs, and especially why would I set a URL for an obsolete service?)
- and even though we run
rake config:checkearlier, apparently the first time that template tries to get instantiated is when we start running database tasks - so if you don't have the configuration defined, the template can't be instantiated, "database setup" fails, and you get no help from the output
Updated by Brett Smith 4 months ago
Brett Smith wrote in #note-6:
- so if you don't have the configuration defined, the template can't be instantiated, "database setup" fails, and you get no help from the output
Specifically, we went down the else branch of the prepare_database postinst function, which specifically doesn't show the output of the rake task. But even if it did, there is absolutely no way a regular user could've figured out the problem from the output. The "real" error message is ActionView::Template::Error: both URI are relative, and even if you find it amongst the gigantic stack trace, it gives you zero information about which template has the problem or where this might be under your control. The only reason I could find it is by figuring if we don't see it on "normal" clusters, then the problem is likely somewhere in cluster configuration, and then going through the templates in there and figuring it out.
Updated by Peter Amstutz 4 months ago
Brett Smith wrote in #note-6:
So yes, the real error is:
- the default new user template refers to
<%= Rails.configuration.Services.Workbench1.ExternalURL %>- I hadn't set that URL (because I was trying to throw this together as fast as possible, so I hadn't set any external URLs, and especially why would I set a URL for an obsolete service?)
FWIW, and to be fair I don't know if we wrote down the decision anywhere, but I think where we ended up was that Workbench1.ExternalURL and Workbench2.ExternalURL need to be set to the same value, indefinitely, to maintain backwards compatibility for clients that read those configuration values.
- and even though we run
rake config:checkearlier, apparently the first time that template tries to get instantiated is when we start running database tasks
Sounds like the place to catch this would be having config:check enforce setting Workbench1.ExternalURL.
Updated by Brett Smith 4 months ago
22349-deploy-bundle-passenger @ e0672ea6373de84b7b839338f6b9f1144cfbd637
- All agreed upon points are implemented / addressed.
- This adds a service definition to the Rails API package to run it using a standalone Passenger server that's already included in the bundle. This lets us run Passenger with the same version of Ruby as the Rails API server and eliminates the need for a separate Passenger install.
- Anything not implemented (discovered or discussed during work) has a follow-up story.
- Passenger complains about a security update available at startup, I need to file a separate issue about that, as well as the initial config challenges noted above.
- Code is tested and passing, both automated and manual, what manual testing was done is described
- I was able to build the package on my system, install it in a fresh Alma 8 VM following the new instructions, and
curlthe discovery document.
- I was able to build the package on my system, install it in a fresh Alma 8 VM following the new instructions, and
- Documentation has been updated.
- Yes
- Behaves appropriately at the intended scale (describe intended scale).
- There should be no change in scale, because access to the RailsAPI server is mediated through controller, and none of those settings have changed.
- Considered backwards and forwards compatibility issues between client and server.
- This change will conflict with existing setups, as documented in the upgrade note. I hope the migration is straightforward enough to be worth it.
- Follows our coding standards and GUI style guidelines.
- N/A (no style guide for any of these languages)
Updated by Lucas Di Pentima 4 months ago
I'm doing and e2e test by deploying a Debian11 Arvados 3.0 test cluster with the installer and then:
- Build
arvados-api-serverandarvados-serverdeb packages from this branch - Install them on the controller node
- Apply the installer changes and re-deploy
The issue I observed:
- The new
arvados-railsapiservice wasn't able to start- Fixed by Brett's suggestion of adding the following to the service file:
[Service] Environment=HOME=/var/www/arvados-api
- Fixed by Brett's suggestion of adding the following to the service file:
- RailsAPI errors out with the following message on
production.log:root@controller:/var/www/arvados-api# tail shared/log/production.log [ E 2024-12-06 22:06:43.5013 32308/T8p age/Cor/App/Implementation.cpp:221 ]: Could not spawn process for application /var/www/arvados-api/current: An internal error occurred while preparing to spawn an application process: Cannot get user database entry for user UID 63563; it looks like your system's user database is broken, please fix it.
- I'm not sure from where this 63563 user id comes from, yet.
Updated by Brett Smith 4 months ago
Lucas Di Pentima wrote in #note-11:
I'm not sure from where this 63563 user id comes from, yet.
It's the UID for the dynamic user created by systemd for the service. It does exist, but I'm guessing the prebuilt Passenger binary doesn't use modern enough libraries to find it. I will have to stop using DynamicUser and make the service less secure for everyone in order to support our older distributions.
Updated by Brett Smith 4 months ago
Now at b22b3ff1436294d25a278ce81e0f723632d96826. b1574b59e072a622e0f771853b3291e3405b7bcf is the fix for the various user issues, then there are some other refinements. From here I was able to build and install a bullseye package and then get the discovery document as before.
Updated by Brett Smith 4 months ago
I have added some commits to the branch to make the package testable, add a little testing of the systemd configuration, and then fix some small distro-specific bugs. Now at bab651830b8053c8da743088fb18f8c24e39687d; please review the changes after b22b3ff1436294d25a278ce81e0f723632d96826. build-packages-multijob: #4461
Updated by Brett Smith 4 months ago
- Related to Bug #22389: Single-host single-hostname installation fails: Rails API server cannot start; /etc/arvados/config.yml "permission denied added
Updated by Brett Smith 4 months ago
- Related to Bug #22391: Standardize systemd service handling added
Updated by Brett Smith 4 months ago
- Status changed from In Progress to Resolved
Applied in changeset arvados|e679e285891a30f1d45addc91746e2be87274e73.
Updated by Brett Smith 4 months ago
- Precedes Bug #22396: Standalone Passenger documentation updates added
Actions