Support #5461
closedfollowing manual install instructions on laptop
Description
Trying to install arvados locally on an Ubuntu laptop. Details below:
http://doc.arvados.org/install/install-manual-prerequisites.html (click Next bottom right)
sudo apt-get install \
bison build-essential gettext libcurl3 libcurl3-gnutls \
libcurl4-openssl-dev libpcre3-dev libpq-dev libreadline-dev \
libssl-dev libxslt1.1 postgresql sudo wget zlib1g-dev
$ cd $HOME # (or wherever you want to install)
$ git clone https://github.com/curoverse/arvados.git
$ cd arvados/services/api
/arvados/services/api$ bundle install
My application.yml file looks something like this:
```
development:
# Mandatory site secrets. See application.default.yml for more info.
secret_token: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
blob_signing_key: yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
uuid_prefix: foobar
workbench_address: https://localhost:3031
secret_token: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
blob_signing_key: yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
uuid_prefix: barfoo
workbench_address: https://localhost:9899
- workbench_address: https://workbench.foobar.arvadosapi.com
```
I created a keepstore.key file that is 128 characters long.
I followed the instructions here: http://doc.arvados.org/install/install-keepstore.html
Results:
```
[hpenvy15 ~/arvados/services/api] $ keepstore -permission-key-file="/home/avilella/arvados/services/api/keepstore.key"
2015/03/13 11:56:49 Keep started: pid 4460
2015/03/13 11:56:49 adding Keep volume: /tmp/tmp.ZeLkn1xGDR/keep
2015/03/13 11:56:49 adding Keep volume: /tmp/tmp.XJotjPWD5P/keep
2015/03/13 11:56:49 Error setting up arvados client Missing required environment variable ARVADOS_API_HOST
```
I went to https://console.developers.google.com and created a project and got the following (dummy values shown):
```
Project ID: smthg-standard-12345 Project Number: 123456789012
CLIENT SECRET 1234567890QWERTYadsfghxy
API KEY AIzaSyDCmGeVk5ZXdMbfTLNzOovnzsmSJ84RquE
```
I then continued with the database setup. This is what my database.yml file looks like:
```#development:
- adapter: postgresql
- encoding: utf8
- database: arvados_development
- username: arvados
- password: 12345678900qwertyasdfgxy
- host: localhost
#
#test: - adapter: postgresql
- encoding: utf8
- database: arvados_test
- username: arvados
- password: 12345678900qwertyasdfgxy
- host: localhost
production:
adapter: postgresql
encoding: utf8
database: arvados_production
username: arvados
password: 12345678900qwertyasdfgxy
host: localhost
```
I then tried to initialize the production database like this:
```
[hpenvy15 ~/arvados/services/api] $ RAILS_ENV=production bundle exec rake db:setup
[hpenvy15 ~/arvados/services/api] $ RAILS_ENV=production bundle exec rake db:setup
psql:/home/avilella/arvados/services/api/db/structure.sql:22: ERROR: must be owner of extension plpgsql
rake aborted!
ActiveRecord::RecordInvalid: Validation failed: Uuid not a valid Arvados uuid 'barfoo-tpzed-000000000000000'
/var/lib/gems/2.1.0/gems/activerecord-3.2.17/lib/active_record/validations.rb:56:in `save!'
/var/lib/gems/2.1.0/gems/activerecord-3.2.17/lib/active_record/attribute_methods/dirty.rb:33:in `save!'
/var/lib/gems/2.1.0/gems/activerecord-3.2.17/lib/active_record/transactions.rb:264:in `block in save!'
/var/lib/gems/2.1.0/gems/activerecord-3.2.17/lib/active_record/transactions.rb:313:in `block in with_transaction_returning_status'
/var/lib/gems/2.1.0/gems/activerecord-3.2.17/lib/active_record/connection_adapters/abstract/database_statements.rb:192:in `transaction'
/var/lib/gems/2.1.0/gems/activerecord-3.2.17/lib/active_record/transactions.rb:208:in `transaction'
/var/lib/gems/2.1.0/gems/activerecord-3.2.17/lib/active_record/transactions.rb:311:in `with_transaction_returning_status'
/var/lib/gems/2.1.0/gems/activerecord-3.2.17/lib/active_record/transactions.rb:264:in `save!'
/var/lib/gems/2.1.0/gems/activerecord-3.2.17/lib/active_record/validations.rb:41:in `create!'
/var/lib/gems/2.1.0/gems/activerecord-3.2.17/lib/active_record/relation.rb:91:in `block in create!'
/var/lib/gems/2.1.0/gems/activerecord-3.2.17/lib/active_record/relation.rb:241:in `block in scoping'
/var/lib/gems/2.1.0/gems/activerecord-3.2.17/lib/active_record/scoping.rb:98:in `with_scope'
/var/lib/gems/2.1.0/gems/activerecord-3.2.17/lib/active_record/relation.rb:241:in `scoping'
/var/lib/gems/2.1.0/gems/activerecord-3.2.17/lib/active_record/relation.rb:91:in `create!'
/var/lib/gems/2.1.0/gems/activerecord-3.2.17/lib/active_record/relation.rb:126:in `first_or_create!'
/home/avilella/arvados/services/api/lib/current_api_client.rb:64:in `block in system_user'
/home/avilella/arvados/services/api/lib/current_api_client.rb:222:in `check_cache'
/home/avilella/arvados/services/api/lib/current_api_client.rb:57:in `system_user'
/home/avilella/arvados/services/api/app/models/database_seeds.rb:4:in `install'
/home/avilella/arvados/services/api/db/seeds.rb:5:in `<top (required)>'
/var/lib/gems/2.1.0/gems/activesupport-3.2.17/lib/active_support/dependencies.rb:245:in `load'
/var/lib/gems/2.1.0/gems/activesupport-3.2.17/lib/active_support/dependencies.rb:245:in `block in load'
/var/lib/gems/2.1.0/gems/activesupport-3.2.17/lib/active_support/dependencies.rb:236:in `load_dependency'
/var/lib/gems/2.1.0/gems/activesupport-3.2.17/lib/active_support/dependencies.rb:245:in `load'
/var/lib/gems/2.1.0/gems/railties-3.2.17/lib/rails/engine.rb:525:in `load_seed'
/var/lib/gems/2.1.0/gems/activerecord-3.2.17/lib/active_record/railties/databases.rake:347:in `block (2 levels) in <top (required)>'
Tasks: TOP => db:setup => db:seed
(See full trace by running task with --trace)
```
Any ideas?
Updated by Albert Vilella almost 10 years ago
Same content as above but properly quoted with pre tag:
Trying to install arvados locally on an Ubuntu laptop. Details below:
http://doc.arvados.org/install/install-manual-prerequisites.html (click Next bottom right)
sudo apt-get install \
bison build-essential gettext libcurl3 libcurl3-gnutls \
libcurl4-openssl-dev libpcre3-dev libpq-dev libreadline-dev \
libssl-dev libxslt1.1 postgresql sudo wget zlib1g-dev
$ cd $HOME # (or wherever you want to install)
$ git clone https://github.com/curoverse/arvados.git
$ cd arvados/services/api
/arvados/services/api$ bundle install
My application.yml file looks something like this:
development: # Mandatory site secrets. See application.default.yml for more info. secret_token: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx blob_signing_key: yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy uuid_prefix: foobar workbench_address: https://localhost:3031 production: # Mandatory site secrets. See application.default.yml for more info. secret_token: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx blob_signing_key: yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy uuid_prefix: barfoo workbench_address: https://localhost:9899 # workbench_address: https://workbench.foobar.arvadosapi.com
I created a keepstore.key file that is 128 characters long.
I followed the instructions here: http://doc.arvados.org/install/install-keepstore.html
Results:
[hpenvy15 ~/arvados/services/api] $ keepstore -permission-key-file="/home/avilella/arvados/services/api/keepstore.key" 2015/03/13 11:56:49 Keep started: pid 4460 2015/03/13 11:56:49 adding Keep volume: /tmp/tmp.ZeLkn1xGDR/keep 2015/03/13 11:56:49 adding Keep volume: /tmp/tmp.XJotjPWD5P/keep 2015/03/13 11:56:49 Error setting up arvados client Missing required environment variable ARVADOS_API_HOST
I went to https://console.developers.google.com and created a project and got the following (dummy values shown):
Project ID: smthg-standard-12345 Project Number: 123456789012 CLIENT SECRET 1234567890QWERTYadsfghxy API KEY AIzaSyDCmGeVk5ZXdMbfTLNzOovnzsmSJ84RquE
I then continued with the database setup. This is what my database.yml file looks like:
#development: # adapter: postgresql # encoding: utf8 # database: arvados_development # username: arvados # password: 12345678900qwertyasdfgxy # host: localhost # #test: # adapter: postgresql # encoding: utf8 # database: arvados_test # username: arvados # password: 12345678900qwertyasdfgxy # host: localhost production: adapter: postgresql encoding: utf8 database: arvados_production username: arvados password: 12345678900qwertyasdfgxy host: localhost
I then tried to initialize the production database like this:
[hpenvy15 ~/arvados/services/api] $ RAILS_ENV=production bundle exec rake db:setup [hpenvy15 ~/arvados/services/api] $ RAILS_ENV=production bundle exec rake db:setup psql:/home/avilella/arvados/services/api/db/structure.sql:22: ERROR: must be owner of extension plpgsql rake aborted! ActiveRecord::RecordInvalid: Validation failed: Uuid not a valid Arvados uuid 'barfoo-tpzed-000000000000000' /var/lib/gems/2.1.0/gems/activerecord-3.2.17/lib/active_record/validations.rb:56:in `save!' /var/lib/gems/2.1.0/gems/activerecord-3.2.17/lib/active_record/attribute_methods/dirty.rb:33:in `save!' /var/lib/gems/2.1.0/gems/activerecord-3.2.17/lib/active_record/transactions.rb:264:in `block in save!' /var/lib/gems/2.1.0/gems/activerecord-3.2.17/lib/active_record/transactions.rb:313:in `block in with_transaction_returning_status' /var/lib/gems/2.1.0/gems/activerecord-3.2.17/lib/active_record/connection_adapters/abstract/database_statements.rb:192:in `transaction' /var/lib/gems/2.1.0/gems/activerecord-3.2.17/lib/active_record/transactions.rb:208:in `transaction' /var/lib/gems/2.1.0/gems/activerecord-3.2.17/lib/active_record/transactions.rb:311:in `with_transaction_returning_status' /var/lib/gems/2.1.0/gems/activerecord-3.2.17/lib/active_record/transactions.rb:264:in `save!' /var/lib/gems/2.1.0/gems/activerecord-3.2.17/lib/active_record/validations.rb:41:in `create!' /var/lib/gems/2.1.0/gems/activerecord-3.2.17/lib/active_record/relation.rb:91:in `block in create!' /var/lib/gems/2.1.0/gems/activerecord-3.2.17/lib/active_record/relation.rb:241:in `block in scoping' /var/lib/gems/2.1.0/gems/activerecord-3.2.17/lib/active_record/scoping.rb:98:in `with_scope' /var/lib/gems/2.1.0/gems/activerecord-3.2.17/lib/active_record/relation.rb:241:in `scoping' /var/lib/gems/2.1.0/gems/activerecord-3.2.17/lib/active_record/relation.rb:91:in `create!' /var/lib/gems/2.1.0/gems/activerecord-3.2.17/lib/active_record/relation.rb:126:in `first_or_create!' /home/avilella/arvados/services/api/lib/current_api_client.rb:64:in `block in system_user' /home/avilella/arvados/services/api/lib/current_api_client.rb:222:in `check_cache' /home/avilella/arvados/services/api/lib/current_api_client.rb:57:in `system_user' /home/avilella/arvados/services/api/app/models/database_seeds.rb:4:in `install' /home/avilella/arvados/services/api/db/seeds.rb:5:in `<top (required)>' /var/lib/gems/2.1.0/gems/activesupport-3.2.17/lib/active_support/dependencies.rb:245:in `load' /var/lib/gems/2.1.0/gems/activesupport-3.2.17/lib/active_support/dependencies.rb:245:in `block in load' /var/lib/gems/2.1.0/gems/activesupport-3.2.17/lib/active_support/dependencies.rb:236:in `load_dependency' /var/lib/gems/2.1.0/gems/activesupport-3.2.17/lib/active_support/dependencies.rb:245:in `load' /var/lib/gems/2.1.0/gems/railties-3.2.17/lib/rails/engine.rb:525:in `load_seed' /var/lib/gems/2.1.0/gems/activerecord-3.2.17/lib/active_record/railties/databases.rake:347:in `block (2 levels) in <top (required)>' Tasks: TOP => db:setup => db:seed (See full trace by running task with --trace)
Updated by Peter Amstutz almost 10 years ago
Hi avilella, your site prefix is too long. It needs to be 5 characters and "barfoo" is 6.
Updated by Albert Vilella almost 10 years ago
- See below script as it is now
- Install dependencies
sudo apt-get install \
bison build-essential gettext libcurl3 libcurl3-gnutls \
libcurl4-openssl-dev libpcre3-dev libpq-dev libreadline-dev \
libssl-dev libxslt1.1 postgresql git wget zlib1g-dev- 1
- Install Ruby and Bundler
pushd $HOME
mkdir -p src
pushd src
wget http://cache.ruby-lang.org/pub/ruby/2.1/ruby-2.1.5.tar.gz
tar xzf ruby-2.1.5.tar.gz
cd ruby-2.1.5
./configure
make
sudo make install
sudo gem install bundler
popd
rm -rf src/ruby-2.1.5
popd
git clone https://github.com/curoverse/arvados.git- 2
- Install service API
pushd arvados/services/api
bundle install
cpi config/environments/production.rb.example config/environments/production.rb-f1`
cp -i config/application.yml.example config/application.yml
sudo mkdir -p /var/lib/arvados/git
sudo git clone --bare ../../.git /var/lib/arvados/git/arvados.git
sed -i "s/secret_token.*$/secret_token: `rake secret`/g" config/application.yml
sed -i "s/blob_signing_key.*$/blob_signing_key: `rake secret`/g" config/application.yml
sed -i "s/https:\/\/workbench.*$/https:\/\/workbench.gamma.foo.com/g" config/application.yml
sed -i "s/bogus/gamma/g" config/application.yml
sed -n -l 1 's/\(^.*blob_signing_key: \)\(.*\)/\2/p' config/application.yml | uniq > keepstore.key
keepstore --permission-key-file=keepstore.key &
prefix=`arv --format=uuid user current | cut -d
"Site prefix is '$prefix'"
read -rd $'\000' keepservice <<EOF; arv keep_service create --keep-service "$keepservice" {
"service_host":"keep0.gamma.foo.com",
"service_port":25107,
"service_ssl_flag":false,
"service_type":"disk"
}
EOF
- 3
echo "deb http://apt.arvados.org/ wheezy main" | sudo tee /etc/apt/sources.list.d/apt.arvados.org.list
sudo /usr/bin/apt-key adv --keyserver pool.sks-keyservers.net --recv 1078ECD7
sudo /usr/bin/apt-get update
sudo /usr/bin/apt-get install keepstore
- TODO: hard coded here: d47b6bmmpmm5hceqwsda9at5x
sudo -u postgres createuser --createdb --encrypted --pwprompt -R -S arvados
cp -i config/database.yml.sample config/database.yml
sed -i "s/xxxxxxxx/d47b6bmmpmm5hceqwsda9at5x/g" config/database.yml
RAILS_ENV=production bundle exec rake db:setup
sudo su postgres
createdb arvados_production -E UTF8 -O arvados
createdb arvados_development -E UTF8 -O arvados
exit
RAILS_ENV=production bundle exec rake db:structure:load
RAILS_ENV=production bundle exec rake db:seed
RAILS_ENV=development bundle exec rake db:structure:load
RAILS_ENV=development bundle exec rake db:seed
- omniauth
cp -i config/initializers/omniauth.rb.example config/initializers/omniauth.rb
sed -i "s/^\(APP_SECRET = \)\(.*$\)/\1'`ruby -e 'puts rand(2**512).to_s(36)'`'/g" config/initializers/omniauth.rb
bundle exec rails server --port=3030
popd
- Install Phusion wheezy
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 561F9B9CAC40B2F7
sudo apt-get install apt-transport-https ca-certificates
echo "deb https://oss-binaries.phusionpassenger.com/apt/passenger wheezy main" | sudo tee /etc/apt/sources.list.d/passenger.list
sudo apt-get update
sudo apt-get install nginx-extras passenger
- Edit /etc/nginx/nginx.conf and uncomment passenger_root and passenger_ruby.
sudo service nginx restart
echo "server {" >> /etc/nginx/sites-available/foo
echo " listen 80;" >> /etc/nginx/sites-available/foo
echo " root `pwd`/public;" >> /etc/nginx/sites-available/foo
echo " index index.html index.htm;" >> /etc/nginx/sites-available/foo
echo "}" >> /etc/nginx/sites-available/foo
ln -s /etc/nginx/sites-available/foo /etc/nginx/sites-enabled/
popd
- Install workbench
pushd arvados/apps/workbench
bundle install
sudo apt-get install \
bison build-essential gettext libcurl3 libcurl3-gnutls \
libcurl4-openssl-dev libpcre3-dev libpq-dev libreadline-dev \
libssl-dev libxslt1.1 sudo wget zlib1g-dev graphviz
apt-get install libsqlite3-dev
cp -i config/environments/production.rb.example config/environments/production.rb
cp -i config/application.yml.example config/application.yml
pushd ~/arvados/services/api
sed -i "s/secret_token.*$/secret_token: `bundle exec rake secret`/g" ~/arvados/apps/workbench/config/application.yml
popd
arvados_login_base: https://prefix_uuid.your.domain/login
arvados_v1_base: https://prefix_uuid.your.domain/arvados/v1
popd
- Install SSO
pushd $HOME
git clone https://github.com/curoverse/sso-devise-omniauth-provider.git
pushd sso-devise-omniauth-provider
bundle install
RAILS_ENV=production bundle exec rake db:create
RAILS_ENV=production bundle exec rake db:migrate
cp -i config/initializers/secret_token.rb.example config/initializers/secret_token.rb
sed -i "s/^\(.*secret_token = \).*/\1'`bundle exec rake secret`'/g" config/initializers/secret_token.rb
- review http://doc.arvados.org/install/install-sso.html
- https://console.developers.google.com/project/foo-arvados/apiui/credential#
cp -i config/environments/production.rb.example config/environments/production.rb - cp -i config/environments/production.rb config/environments/development.rb
openssl req -nodes -newkey rsa:2048 -keyout foo.key -x509 -days 3650 -out foo.cer
RAILS_ENV=production bundle exec rails console
popd
popd
- keep
unset ARVADOS_API_HOST_INSECURE
export ARVADOS_API_HOST=localhost:3030
export ARVADOS_API_HOST_INSECURE=1
export ARVADOS_API_TOKEN=3baqay8azvan45u7gpduetzd8g1dml8n17sp7znn4ij3dblq39
prefix=`arv --format=uuid user current | cutd-f1` && echo "Site prefix is '$prefix'"
read -rd $'\000' keepservice <<EOF; arv keep_service create --keep-service "$keepservice" {
"service_host":"keep0.gamma.foo.com",
"service_port":25107,
"service_ssl_flag":false,
"service_type":"disk"
}
EOF
export ARVADOS_API_HOST=workbench.gamma.foo.com
export ARVADOS_API_TOKEN=3baqay8azvan45u7gpduetzd8g1dml8n17sp7znn4ij3dblq39
export ARVADOS_API_HOST=workbench.gamma.foo.com:3030
export ARVADOS_API_HOST_INSECURE=1
read -rd $'\000' keepservice <<EOF; arv keep_service create --keep-service "$keepservice"
{
"service_host":"workbench.gamma.foo.com",
"service_port":443,
"service_ssl_flag":true,
"service_type":"proxy"
}
EOF
echo "deb http://apt.arvados.org/ wheezy main" | sudo tee /etc/apt/sources.list.d/apt.arvados.org.list
sudo /usr/bin/apt-get install keepproxy
export ARVADOS_API_TOKEN=3baqay8azvan45u7gpduetzd8g1dml8n17sp7znn4ij3dblq39
export ARVADOS_API_HOST=workbench.gamma.foo.com:3030
export ARVADOS_API_HOST_INSECURE=1
read -rd $'\000' newjob <<EOF; arv job create --job "$newjob"
{"script_parameters":{"input":"a7f932428b30582f54ea26084813eb7b3e8f3251"},
"script_version":"master",
"script":"hash",
"repository":"arvados"}
EOF
- Install Perl SDK
sudo apt-get install libjson-perl libio-socket-ssl-perl libwww-perl libipc-system-simple-perl
cd ~/arvados/sdk/perl
perl Makefile.PL
sudo make install
sudo apt-get install libnet-ssleay-perl
sudo apt-get install libcrypt-ssleay-perl
perl -MArvados -e ''
perl <<'EOF'
use Arvados;
my $arv = Arvados->new('apiVersion' => 'v1');
my $me = $arv->{'users'}->{'current'}->execute;
print ("arvados.v1.users.current.full_name = '", $me->{'full_name'}, "'\n");
EOF
- Install Python SDK
sudo apt-get install python-pip python-dev libattr1-dev libfuse-dev pkg-config python-yaml
sudo pip install arvados-python-client
pip install --upgrade pyvcf
adduser crunch
- TODO: extras at bottom of http://doc.arvados.org/install/install-manual-prerequisites.html
Updated by Albert Vilella almost 10 years ago
- Sorry, again now with pre tags:
#/bin/bash # Install dependencies sudo apt-get install \ bison build-essential gettext libcurl3 libcurl3-gnutls \ libcurl4-openssl-dev libpcre3-dev libpq-dev libreadline-dev \ libssl-dev libxslt1.1 postgresql git wget zlib1g-dev ### 1 ## Install Ruby and Bundler pushd $HOME mkdir -p src pushd src wget http://cache.ruby-lang.org/pub/ruby/2.1/ruby-2.1.5.tar.gz tar xzf ruby-2.1.5.tar.gz cd ruby-2.1.5 ./configure make sudo make install sudo gem install bundler popd rm -rf src/ruby-2.1.5 popd git clone https://github.com/curoverse/arvados.git ### 2 # Install service API pushd arvados/services/api bundle install cp -i config/environments/production.rb.example config/environments/production.rb cp -i config/application.yml.example config/application.yml sudo mkdir -p /var/lib/arvados/git sudo git clone --bare ../../.git /var/lib/arvados/git/arvados.git sed -i "s/secret_token.*$/secret_token: `rake secret`/g" config/application.yml sed -i "s/blob_signing_key.*$/blob_signing_key: `rake secret`/g" config/application.yml sed -i "s/https:\/\/workbench.*$/https:\/\/workbench.gamma.foo.com/g" config/application.yml sed -i "s/bogus/gamma/g" config/application.yml sed -n -l 1 's/\(^.*blob_signing_key: \)\(.*\)/\2/p' config/application.yml | uniq > keepstore.key keepstore --permission-key-file=keepstore.key & prefix=`arv --format=uuid user current | cut -d- -f1` "Site prefix is '$prefix'" read -rd $'\000' keepservice <<EOF; arv keep_service create --keep-service "$keepservice" { "service_host":"keep0.gamma.foo.com", "service_port":25107, "service_ssl_flag":false, "service_type":"disk" } EOF ### 3 echo "deb http://apt.arvados.org/ wheezy main" | sudo tee /etc/apt/sources.list.d/apt.arvados.org.list sudo /usr/bin/apt-key adv --keyserver pool.sks-keyservers.net --recv 1078ECD7 sudo /usr/bin/apt-get update sudo /usr/bin/apt-get install keepstore ruby -e 'puts rand(2**128).to_s(36)' # TODO: hard coded here: d47b6bmmpmm5hceqwsda9at5x sudo -u postgres createuser --createdb --encrypted --pwprompt -R -S arvados cp -i config/database.yml.sample config/database.yml sed -i "s/xxxxxxxx/d47b6bmmpmm5hceqwsda9at5x/g" config/database.yml RAILS_ENV=production bundle exec rake db:setup sudo su postgres createdb arvados_production -E UTF8 -O arvados createdb arvados_development -E UTF8 -O arvados exit RAILS_ENV=production bundle exec rake db:structure:load RAILS_ENV=production bundle exec rake db:seed RAILS_ENV=development bundle exec rake db:structure:load RAILS_ENV=development bundle exec rake db:seed ### omniauth cp -i config/initializers/omniauth.rb.example config/initializers/omniauth.rb sed -i "s/^\(APP_SECRET = \)\(.*$\)/\1'`ruby -e 'puts rand(2**512).to_s(36)'`'/g" config/initializers/omniauth.rb bundle exec rails server --port=3030 popd ## Install Phusion wheezy sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 561F9B9CAC40B2F7 sudo apt-get install apt-transport-https ca-certificates echo "deb https://oss-binaries.phusionpassenger.com/apt/passenger wheezy main" | sudo tee /etc/apt/sources.list.d/passenger.list sudo apt-get update sudo apt-get install nginx-extras passenger # Edit /etc/nginx/nginx.conf and uncomment passenger_root and passenger_ruby. sudo service nginx restart echo "server {" >> /etc/nginx/sites-available/foo echo " listen 80;" >> /etc/nginx/sites-available/foo echo " root `pwd`/public;" >> /etc/nginx/sites-available/foo echo " index index.html index.htm;" >> /etc/nginx/sites-available/foo echo "}" >> /etc/nginx/sites-available/foo ln -s /etc/nginx/sites-available/foo /etc/nginx/sites-enabled/ popd ## Install workbench pushd arvados/apps/workbench bundle install sudo apt-get install \ bison build-essential gettext libcurl3 libcurl3-gnutls \ libcurl4-openssl-dev libpcre3-dev libpq-dev libreadline-dev \ libssl-dev libxslt1.1 sudo wget zlib1g-dev graphviz apt-get install libsqlite3-dev cp -i config/environments/production.rb.example config/environments/production.rb cp -i config/application.yml.example config/application.yml pushd ~/arvados/services/api sed -i "s/secret_token.*$/secret_token: `bundle exec rake secret`/g" ~/arvados/apps/workbench/config/application.yml popd arvados_login_base: https://prefix_uuid.your.domain/login arvados_v1_base: https://prefix_uuid.your.domain/arvados/v1 popd ## Install SSO pushd $HOME git clone https://github.com/curoverse/sso-devise-omniauth-provider.git pushd sso-devise-omniauth-provider bundle install RAILS_ENV=production bundle exec rake db:create RAILS_ENV=production bundle exec rake db:migrate cp -i config/initializers/secret_token.rb.example config/initializers/secret_token.rb sed -i "s/^\(.*secret_token = \).*/\1'`bundle exec rake secret`'/g" config/initializers/secret_token.rb # review http://doc.arvados.org/install/install-sso.html # https://console.developers.google.com/project/foo-arvados/apiui/credential# cp -i config/environments/production.rb.example config/environments/production.rb # cp -i config/environments/production.rb config/environments/development.rb openssl req -nodes -newkey rsa:2048 -keyout foo.key -x509 -days 3650 -out foo.cer RAILS_ENV=production bundle exec rails console popd popd ### keep unset ARVADOS_API_HOST_INSECURE export ARVADOS_API_HOST=localhost:3030 export ARVADOS_API_HOST_INSECURE=1 export ARVADOS_API_TOKEN=3baqay8azvan45u7gpduetzd8g1dml8n17sp7znn4ij3dblq39 prefix=`arv --format=uuid user current | cut -d- -f1` && echo "Site prefix is '$prefix'" read -rd $'\000' keepservice <<EOF; arv keep_service create --keep-service "$keepservice" { "service_host":"keep0.gamma.foo.com", "service_port":25107, "service_ssl_flag":false, "service_type":"disk" } EOF export ARVADOS_API_HOST=workbench.gamma.foo.com export ARVADOS_API_TOKEN=3baqay8azvan45u7gpduetzd8g1dml8n17sp7znn4ij3dblq39 export ARVADOS_API_HOST=workbench.gamma.foo.com:3030 export ARVADOS_API_HOST_INSECURE=1 read -rd $'\000' keepservice <<EOF; arv keep_service create --keep-service "$keepservice" { "service_host":"workbench.gamma.foo.com", "service_port":443, "service_ssl_flag":true, "service_type":"proxy" } EOF echo "deb http://apt.arvados.org/ wheezy main" | sudo tee /etc/apt/sources.list.d/apt.arvados.org.list sudo /usr/bin/apt-get install keepproxy export ARVADOS_API_TOKEN=3baqay8azvan45u7gpduetzd8g1dml8n17sp7znn4ij3dblq39 export ARVADOS_API_HOST=workbench.gamma.foo.com:3030 export ARVADOS_API_HOST_INSECURE=1 read -rd $'\000' newjob <<EOF; arv job create --job "$newjob" {"script_parameters":{"input":"a7f932428b30582f54ea26084813eb7b3e8f3251"}, "script_version":"master", "script":"hash", "repository":"arvados"} EOF # Install Perl SDK sudo apt-get install libjson-perl libio-socket-ssl-perl libwww-perl libipc-system-simple-perl cd ~/arvados/sdk/perl perl Makefile.PL sudo make install sudo apt-get install libnet-ssleay-perl sudo apt-get install libcrypt-ssleay-perl perl -MArvados -e '' perl <<'EOF' use Arvados; my $arv = Arvados->new('apiVersion' => 'v1'); my $me = $arv->{'users'}->{'current'}->execute; print ("arvados.v1.users.current.full_name = '", $me->{'full_name'}, "'\n"); EOF # Install Python SDK sudo apt-get install python-pip python-dev libattr1-dev libfuse-dev pkg-config python-yaml sudo pip install arvados-python-client pip install --upgrade pyvcf adduser crunch # TODO: extras at bottom of http://doc.arvados.org/install/install-manual-prerequisites.html
Updated by Davide Fiorentino lo Regio almost 10 years ago
#/bin/bash # Install dependencies sudo apt-get install \ bison build-essential gettext libcurl3 libcurl3-gnutls \ libcurl4-openssl-dev libpcre3-dev libpq-dev libreadline-dev \ libssl-dev libxslt1.1 postgresql git wget zlib1g-dev ### 1 ## Install Ruby and Bundler pushd $HOME mkdir -p src pushd src wget http://cache.ruby-lang.org/pub/ruby/2.1/ruby-2.1.5.tar.gz tar xzf ruby-2.1.5.tar.gz cd ruby-2.1.5 ./configure make sudo make install sudo gem install bundler popd rm -rf src/ruby-2.1.5 popd git clone https://github.com/curoverse/arvados.git ### 2 # Install service API pushd arvados/services/api bundle install cp -i config/environments/production.rb.example config/environments/production.rb cp -i config/application.yml.example config/application.yml sudo mkdir -p /var/lib/arvados/git sudo git clone --bare ../../.git /var/lib/arvados/git/arvados.git sed -i "s/secret_token.*$/secret_token: `rake secret`/g" config/application.yml sed -i "s/blob_signing_key.*$/blob_signing_key: `rake secret`/g" config/application.yml sed -i "s/https:\/\/workbench.*$/https:\/\/workbench.gamma.foo.com/g" config/application.yml sed -i "s/bogus/gamma/g" config/application.yml sed -n -l 1 's/\(^.*blob_signing_key: \)\(.*\)/\2/p' config/application.yml | uniq > keepstore.key echo "deb http://apt.arvados.org/ wheezy main" | sudo tee /etc/apt/sources.list.d/apt.arvados.org.list sudo /usr/bin/apt-key adv --keyserver pool.sks-keyservers.net --recv 1078ECD7 sudo /usr/bin/apt-get update sudo /usr/bin/apt-get install keepstore keepstore --permission-key-file=keepstore.key & prefix=`arv --format=uuid user current | cut -d- -f1` "Site prefix is '$prefix'" read -rd $'\000' keepservice <<EOF; arv keep_service create --keep-service "$keepservice" { "service_host":"keep0.gamma.foo.com", "service_port":25107, "service_ssl_flag":false, "service_type":"disk" } EOF ### 3 ruby -e 'puts rand(2**128).to_s(36)' # d47b6bmmpmm5hceqwsda9at5x sudo -u postgres createuser --createdb --encrypted --pwprompt -R -S arvados cp -i config/database.yml.sample config/database.yml sed -i "s/xxxxxxxx/d47b6bmmpmm5hceqwsda9at5x/g" config/database.yml RAILS_ENV=production bundle exec rake db:setup sudo su postgres createdb arvados_production -E UTF8 -O arvados createdb arvados_development -E UTF8 -O arvados exit RAILS_ENV=production bundle exec rake db:structure:load RAILS_ENV=production bundle exec rake db:seed RAILS_ENV=development bundle exec rake db:structure:load RAILS_ENV=development bundle exec rake db:seed cp -i config/initializers/omniauth.rb.example config/initializers/omniauth.rb sed -i "s/^\(APP_SECRET = \)\(.*$\)/\1'`ruby -e 'puts rand(2**512).to_s(36)'`'/g" config/initializers/omniauth.rb bundle exec rails server --port=3030 popd ## Install Phusion sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 561F9B9CAC40B2F7 sudo apt-get install apt-transport-https ca-certificates echo "deb https://oss-binaries.phusionpassenger.com/apt/passenger wheezy main" | sudo tee /etc/apt/sources.list.d/passenger.list sudo apt-get update sudo apt-get install nginx-extras passenger # Edit /etc/nginx/nginx.conf and uncomment passenger_root and passenger_ruby. popd ## Install workbench pushd arvados/apps/workbench bundle install sudo apt-get install \ bison build-essential gettext libcurl3 libcurl3-gnutls \ libcurl4-openssl-dev libpcre3-dev libpq-dev libreadline-dev \ libssl-dev libxslt1.1 sudo wget zlib1g-dev graphviz \ libsqlite3-dev cp -i config/environments/production.rb.example config/environments/production.rb cp -i config/application.yml.example config/application.yml pushd ~/arvados/services/api sed -i "s/secret_token.*$/secret_token: `bundle exec rake secret`/g" ~/arvados/apps/workbench/config/application.yml popd # set these arvados_login_base: https://prefix_uuid.your.domain/login arvados_v1_base: https://prefix_uuid.your.domain/arvados/v1 popd ## Install SSO pushd $HOME git clone https://github.com/curoverse/sso-devise-omniauth-provider.git pushd sso-devise-omniauth-provider bundle install RAILS_ENV=production bundle exec rake db:create RAILS_ENV=production bundle exec rake db:migrate RAILS_ENV=development bundle exec rake db:create RAILS_ENV=development bundle exec rake db:migrate cp -i config/initializers/secret_token.rb.example config/initializers/secret_token.rb sed -i "s/^\(.*secret_token = \).*/\1'`bundle exec rake secret`'/g" config/initializers/secret_token.rb # review http://doc.arvados.org/install/install-sso.html # https://console.developers.google.com/project/foo-arvados/apiui/credential# cp -i config/environments/production.rb.example config/environments/production.rb cp -i config/environments/production.rb config/environments/development.rb openssl req -nodes -newkey rsa:2048 -keyout foo.key -x509 -days 3650 -out foo.cer # RAILS_ENV=production bundle exec rails console RAILS_ENV=production bundle exec rails console popd popd unset ARVADOS_API_HOST_INSECURE export ARVADOS_API_HOST=localhost:3030 export ARVADOS_API_HOST_INSECURE=1 export ARVADOS_API_TOKEN=3baqay8azvan45u7gpduetzd8g1dml8n17sp7znn4ij3dblq39 prefix=`arv --format=uuid user current | cut -d- -f1` && echo "Site prefix is '$prefix'" read -rd $'\000' keepservice <<EOF; arv keep_service create --keep-service "$keepservice" { "service_host":"keep0.gamma.foo.com", "service_port":25107, "service_ssl_flag":false, "service_type":"disk" } EOF export ARVADOS_API_HOST=workbench.gamma.foo.com export ARVADOS_API_TOKEN=3baqay8azvan45u7gpduetzd8g1dml8n17sp7znn4ij3dblq39 export ARVADOS_API_HOST=workbench.gamma.foo.com:3030 export ARVADOS_API_HOST_INSECURE=1 read -rd $'\000' keepservice <<EOF; arv keep_service create --keep-service "$keepservice" { "service_host":"workbench.gamma.foo.com", "service_port":443, "service_ssl_flag":true, "service_type":"proxy" } EOF echo "deb http://apt.arvados.org/ wheezy main" | sudo tee /etc/apt/sources.list.d/apt.arvados.org.list sudo /usr/bin/apt-get install keepproxy export ARVADOS_API_HOST=workbench.gamma.foo.com export ARVADOS_API_TOKEN=3baqay8azvan45u7gpduetzd8g1dml8n17sp7znn4ij3dblq39 export ARVADOS_API_HOST=workbench.gamma.foo.com:3030 export ARVADOS_API_HOST_INSECURE=1 read -rd $'\000' newjob <<EOF; arv job create --job "$newjob" {"script_parameters":{"input":"a7f932428b30582f54ea26084813eb7b3e8f3251"}, "script_version":"master", "script":"hash", "repository":"arvados"} EOF sudo apt-get install libjson-perl libio-socket-ssl-perl libwww-perl libipc-system-simple-perl cd ~/arvados/sdk/perl perl Makefile.PL sudo make install sudo apt-get install libnet-ssleay-perl sudo apt-get install libcrypt-ssleay-perl perl -MArvados -e '' perl <<'EOF' use Arvados; my $arv = Arvados->new('apiVersion' => 'v1'); my $me = $arv->{'users'}->{'current'}->execute; print ("arvados.v1.users.current.full_name = '", $me->{'full_name'}, "'\n"); EOF sudo apt-get install python-pip python-dev libattr1-dev libfuse-dev pkg-config python-yaml sudo pip install arvados-python-client pip install --upgrade pyvcf adduser crunch