Arvados

Starting from https://dev.arvados.org/projects/arvados/wiki/Hacking_prerequisites/71:

root@a88a5742be7f:~# apt-get update
Get:1 http://deb.debian.org/debian buster InRelease [121 kB]
Get:2 http://security.debian.org/debian-security buster/updates InRelease [65.4 kB]
Get:3 http://deb.debian.org/debian buster-updates InRelease [51.9 kB]
Get:4 http://security.debian.org/debian-security buster/updates/main amd64 Packages [208 kB]
Get:5 http://deb.debian.org/debian buster/main amd64 Packages [7905 kB]
Get:6 http://deb.debian.org/debian buster-updates/main amd64 Packages [7868 B]
Fetched 8360 kB in 4s (2048 kB/s)                        
Reading package lists... Done

root@a88a5742be7f:~# apt-get -y install sudo
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following additional packages will be installed:
  lsb-base
The following NEW packages will be installed:
  lsb-base sudo
0 upgraded, 2 newly installed, 0 to remove and 0 not upgraded.
Need to get 1273 kB of archives.
After this operation, 3928 kB of additional disk space will be used.
Get:1 http://deb.debian.org/debian buster/main amd64 lsb-base all 10.2019051400 [28.4 kB]
Get:2 http://deb.debian.org/debian buster/main amd64 sudo amd64 1.8.27-1+deb10u2 [1245 kB]
Fetched 1273 kB in 1s (2214 kB/s)
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package lsb-base.
(Reading database ... 6677 files and directories currently installed.)
Preparing to unpack .../lsb-base_10.2019051400_all.deb ...
Unpacking lsb-base (10.2019051400) ...
Selecting previously unselected package sudo.
Preparing to unpack .../sudo_1.8.27-1+deb10u2_amd64.deb ...
Unpacking sudo (1.8.27-1+deb10u2) ...
Setting up lsb-base (10.2019051400) ...
Setting up sudo (1.8.27-1+deb10u2) ...
invoke-rc.d: could not determine current runlevel
invoke-rc.d: policy-rc.d denied execution of start.

root@a88a5742be7f:~# adduser me
Adding user `me' ...
Adding new group `me' (1000) ...
Adding new user `me' (1000) with group `me' ...
Creating home directory `/home/me' ...
Copying files from `/etc/skel' ...
New password: 
Retype new password: 
passwd: password updated successfully
Changing the user information for me
Enter the new value, or press ENTER for the default
    Full Name []: 
    Room Number []: 
    Work Phone []: 
    Home Phone []: 
    Other []: 
Is the information correct? [Y/n] 

Installing prerequisites:

root@a88a5742be7f:~# apt-get install --no-install-recommends build-essential ca-certificates git golang libpam0g-dev
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following additional packages will be installed:
  binutils binutils-common binutils-x86-64-linux-gnu bzip2 cpp cpp-8 dpkg-dev g++ g++-8 gcc gcc-8 git-man
  golang-1.11 golang-1.11-doc golang-1.11-go golang-1.11-src golang-doc golang-go golang-src libasan5
  libatomic1 libbinutils libc-dev-bin libc6-dev libcc1-0 libcurl3-gnutls libdpkg-perl liberror-perl libexpat1
  libgcc-8-dev libgdbm-compat4 libgdbm6 libgomp1 libgssapi-krb5-2 libisl19 libitm1 libk5crypto3 libkeyutils1
  libkrb5-3 libkrb5support0 libldap-2.4-2 libldap-common liblsan0 libmpc3 libmpfr6 libmpx2 libnghttp2-14
  libpcre2-8-0 libperl5.28 libpsl5 libquadmath0 librtmp1 libsasl2-2 libsasl2-modules-db libssh2-1 libssl1.1
  libstdc++-8-dev libtsan0 libubsan1 linux-libc-dev make openssl patch perl perl-modules-5.28 xz-utils
Suggested packages:
  binutils-doc bzip2-doc cpp-doc gcc-8-locales debian-keyring g++-multilib g++-8-multilib gcc-8-doc
  libstdc++6-8-dbg gcc-multilib manpages-dev autoconf automake libtool flex bison gdb gcc-doc gcc-8-multilib
  libgcc1-dbg libgomp1-dbg libitm1-dbg libatomic1-dbg libasan5-dbg liblsan0-dbg libtsan0-dbg libubsan1-dbg
  libmpx2-dbg libquadmath0-dbg gettext-base git-daemon-run | git-daemon-sysvinit git-doc git-el git-email
  git-gui gitk gitweb git-cvs git-mediawiki git-svn bzr | brz mercurial subversion glibc-doc gnupg | gnupg2
  sensible-utils bzr gdbm-l10n krb5-doc krb5-user libstdc++-8-doc make-doc ed diffutils-doc perl-doc
  libterm-readline-gnu-perl | libterm-readline-perl-perl libb-debug-perl liblocale-codes-perl
Recommended packages:
  fakeroot gnupg | gnupg2 libalgorithm-merge-perl less ssh-client pkg-config manpages manpages-dev
  libfile-fcntllock-perl liblocale-gettext-perl krb5-locales publicsuffix libsasl2-modules netbase
The following NEW packages will be installed:
  binutils binutils-common binutils-x86-64-linux-gnu build-essential bzip2 ca-certificates cpp cpp-8 dpkg-dev
  g++ g++-8 gcc gcc-8 git git-man golang golang-1.11 golang-1.11-doc golang-1.11-go golang-1.11-src golang-doc
  golang-go golang-src libasan5 libatomic1 libbinutils libc-dev-bin libc6-dev libcc1-0 libcurl3-gnutls
  libdpkg-perl liberror-perl libexpat1 libgcc-8-dev libgdbm-compat4 libgdbm6 libgomp1 libgssapi-krb5-2 libisl19
  libitm1 libk5crypto3 libkeyutils1 libkrb5-3 libkrb5support0 libldap-2.4-2 libldap-common liblsan0 libmpc3
  libmpfr6 libmpx2 libnghttp2-14 libpam0g-dev libpcre2-8-0 libperl5.28 libpsl5 libquadmath0 librtmp1 libsasl2-2
  libsasl2-modules-db libssh2-1 libssl1.1 libstdc++-8-dev libtsan0 libubsan1 linux-libc-dev make openssl patch
  perl perl-modules-5.28 xz-utils
0 upgraded, 71 newly installed, 0 to remove and 0 not upgraded.
Need to get 124 MB of archives.
After this operation, 576 MB of additional disk space will be used.
Do you want to continue? [Y/n] Y
....
Setting up perl-modules-5.28 (5.28.1-6) ...
Setting up libexpat1:amd64 (2.2.6-2+deb10u1) ...
Setting up libkeyutils1:amd64 (1.6-6) ...
Setting up libpsl5:amd64 (0.20.2-2) ...
Setting up golang-1.11-src (1.11.6-1+deb10u3) ...
Setting up libssl1.1:amd64 (1.1.1d-0+deb10u3) ...
debconf: unable to initialize frontend: Dialog
debconf: (No usable dialog-like program is installed, so the dialog based frontend cannot be used. at /usr/share/perl5/Debconf/FrontEnd/Dialog.pm line 76.)
debconf: falling back to frontend: Readline
Setting up binutils-common:amd64 (2.31.1-16) ...
Setting up libnghttp2-14:amd64 (1.36.0-2+deb10u1) ...
Setting up linux-libc-dev:amd64 (4.19.118-2+deb10u1) ...
Setting up libgomp1:amd64 (8.3.0-6) ...
Setting up bzip2 (1.0.6-9.2~deb10u1) ...
Setting up libldap-common (2.4.47+dfsg-3+deb10u2) ...
Setting up libkrb5support0:amd64 (1.17-3) ...
Setting up golang-1.11-go (1.11.6-1+deb10u3) ...
Setting up libsasl2-modules-db:amd64 (2.1.27+dfsg-1+deb10u1) ...
Setting up libasan5:amd64 (8.3.0-6) ...
Setting up make (4.2.1-1.2) ...
Setting up libmpfr6:amd64 (4.0.2-1) ...
Setting up librtmp1:amd64 (2.4+20151223.gitfa8646d.1-2) ...
Setting up xz-utils (5.2.4-1) ...
update-alternatives: using /usr/bin/xz to provide /usr/bin/lzma (lzma) in auto mode
Setting up libquadmath0:amd64 (8.3.0-6) ...
Setting up libmpc3:amd64 (1.1.0-1) ...
Setting up libatomic1:amd64 (8.3.0-6) ...
Setting up patch (2.7.6-3+deb10u1) ...
Setting up libpcre2-8-0:amd64 (10.32-5) ...
Setting up libk5crypto3:amd64 (1.17-3) ...
Setting up libsasl2-2:amd64 (2.1.27+dfsg-1+deb10u1) ...
Setting up libmpx2:amd64 (8.3.0-6) ...
Setting up libubsan1:amd64 (8.3.0-6) ...
Setting up libisl19:amd64 (0.20-2) ...
Setting up git-man (1:2.20.1-2+deb10u3) ...
Setting up libssh2-1:amd64 (1.8.0-2.1) ...
Setting up libkrb5-3:amd64 (1.17-3) ...
Setting up libbinutils:amd64 (2.31.1-16) ...
Setting up cpp-8 (8.3.0-6) ...
Setting up libc-dev-bin (2.28-10) ...
Setting up golang-src (2:1.11~1) ...
Setting up openssl (1.1.1d-0+deb10u3) ...
Setting up libcc1-0:amd64 (8.3.0-6) ...
Setting up liblsan0:amd64 (8.3.0-6) ...
Setting up libitm1:amd64 (8.3.0-6) ...
Setting up libgdbm6:amd64 (1.18.1-4) ...
Setting up binutils-x86-64-linux-gnu (2.31.1-16) ...
Setting up libtsan0:amd64 (8.3.0-6) ...
Setting up golang-1.11-doc (1.11.6-1+deb10u3) ...
Setting up golang-go (2:1.11~1) ...
Setting up libldap-2.4-2:amd64 (2.4.47+dfsg-3+deb10u2) ...
Setting up binutils (2.31.1-16) ...
Setting up ca-certificates (20200601~deb10u1) ...
debconf: unable to initialize frontend: Dialog
debconf: (No usable dialog-like program is installed, so the dialog based frontend cannot be used. at /usr/share/perl5/Debconf/FrontEnd/Dialog.pm line 76.)
debconf: falling back to frontend: Readline
Updating certificates in /etc/ssl/certs...
126 added, 0 removed; done.
Setting up libgssapi-krb5-2:amd64 (1.17-3) ...
Setting up libgdbm-compat4:amd64 (1.18.1-4) ...
Setting up golang-1.11 (1.11.6-1+deb10u3) ...
Setting up libgcc-8-dev:amd64 (8.3.0-6) ...
Setting up libperl5.28:amd64 (5.28.1-6) ...
Setting up cpp (4:8.3.0-1) ...
Setting up libc6-dev:amd64 (2.28-10) ...
Setting up golang-doc (2:1.11~1) ...
Setting up libstdc++-8-dev:amd64 (8.3.0-6) ...
Setting up golang (2:1.11~1) ...
Setting up gcc-8 (8.3.0-6) ...
Setting up libpam0g-dev:amd64 (1.3.1-5) ...
Setting up libcurl3-gnutls:amd64 (7.64.0-4+deb10u1) ...
Setting up gcc (4:8.3.0-1) ...
Setting up perl (5.28.1-6) ...
Setting up libdpkg-perl (1.19.7) ...
Setting up g++-8 (8.3.0-6) ...
Setting up dpkg-dev (1.19.7) ...
Setting up liberror-perl (0.17027-2) ...
Setting up git (1:2.20.1-2+deb10u3) ...
Setting up g++ (4:8.3.0-1) ...
update-alternatives: using /usr/bin/g++ to provide /usr/bin/c++ (c++) in auto mode
Setting up build-essential (12.6) ...
Processing triggers for libc-bin (2.28-10) ...
Processing triggers for ca-certificates (20200601~deb10u1) ...
Updating certificates in /etc/ssl/certs...
0 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.

now for the user actions::

root@a88a5742be7f:~# sudo -u me -i
me@a88a5742be7f:~$ groups
me sudo
me@a88a5742be7f:~$ cd
me@a88a5742be7f:~$ git clone https://git.arvados.org/arvados.git
Cloning into 'arvados'...
me@a88a5742be7f:~$ cd arvados
me@a88a5742be7f:~/arvados$ git checkout 16534-example-api-pkg
Branch '16534-example-api-pkg' set up to track remote branch '16534-example-api-pkg' from 'origin'.
Switched to a new branch '16534-example-api-pkg'
me@a88a5742be7f:~/arvados$ git pull
Already up to date.
me@a88a5742be7f:~/arvados$ git log  | head
commit bef1b89ee6ba0225765a5fdb7e2ddf826b30cb30
Author: Tom Clegg <tom@tomclegg.ca>
Date:   Thu Jul 9 10:28:05 2020 -0400

    16534: Add example of calling other controller APIs.

    Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>
me@a88a5742be7f:~/arvados$ go mod download
go: finding github.com/dnaeon/go-vcr v1.0.1
go: finding github.com/ghodss/yaml v1.0.0
go: finding github.com/jmoiron/sqlx v1.2.0
go: finding github.com/mitchellh/go-homedir v0.0.0-20161203194507-b8bc1bf76747
go: finding github.com/Azure/azure-sdk-for-go v19.1.0+incompatible
go: finding github.com/dimchansky/utfbom v1.0.0
go: finding github.com/aws/aws-sdk-go v1.25.30
go: finding github.com/flynn/go-shlex v0.0.0-20150515145356-3f9db97f8568
go: finding github.com/imdario/mergo v0.3.8-0.20190415133143-5ef87b449ca7
go: finding github.com/go-asn1-ber/asn1-ber v1.4.1
go: finding github.com/bgentry/speakeasy v0.1.0
go: finding github.com/pelletier/go-buffruneio v0.2.0
go: finding github.com/bradleypeabody/godap v0.0.0-20170216002349-c249933bc092
go: finding gopkg.in/src-d/go-git.v4 v4.0.0
go: finding github.com/arvados/goamz v0.0.0-20190905141525-1bba09f407ef
go: finding github.com/go-sql-driver/mysql v1.4.0
go: finding github.com/docker/distribution v2.6.0-rc.1.0.20180105232752-277ed486c948+incompatible
go: finding github.com/mattn/go-sqlite3 v1.9.0
go: finding golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550
go: finding github.com/opencontainers/image-spec v1.0.1-0.20171125024018-577479e4dc27
go: finding rsc.io/getopt v0.0.0-20170811000552-20be20937449
go: finding github.com/kevinburke/ssh_config v0.0.0-20171013211458-802051befeb5
go: finding github.com/marstr/guid v1.1.1-0.20170427235115-8bdf7d1a087c
go: finding github.com/prometheus/common v0.7.0
go: finding github.com/src-d/gcfg v1.3.0
go: finding google.golang.org/api v0.13.0
go: finding github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4
go: finding golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45
go: finding github.com/satori/go.uuid v1.2.1-0.20180103174451-36e9d2ebbde5
go: finding github.com/coreos/go-oidc v2.1.0+incompatible
go: finding github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223
go: finding gopkg.in/src-d/go-billy.v4 v4.0.1
go: finding gopkg.in/yaml.v2 v2.2.4
go: finding github.com/xanzy/ssh-agent v0.1.0
go: finding golang.org/x/net v0.0.0-20190613194153-d28f0bde5980
go: finding github.com/go-logfmt/logfmt v0.4.0
go: finding github.com/alcortesm/tgz v0.0.0-20161220082320-9c5fe88206d7
go: finding golang.org/x/sys v0.0.0-20190422165155-953cdadca894
go: finding github.com/go-ldap/ldap v3.0.3+incompatible
go: finding github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751
go: finding github.com/prometheus/client_golang v1.2.1
go: finding github.com/docker/docker v1.4.2-0.20180109013817-94b8a116fbf1
go: finding github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99
go: finding github.com/lib/pq v1.0.0
go: finding golang.org/x/net v0.0.0-20190108225652-1e06a53dbb7e
go: finding google.golang.org/appengine v1.4.0
go: finding github.com/matttproud/golang_protobuf_extensions v1.0.1
go: finding golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3
go: finding github.com/hashicorp/golang-lru v0.5.1
go: finding golang.org/x/text v0.3.0
go: finding github.com/golang/protobuf v1.2.0
go: finding gopkg.in/alecthomas/kingpin.v2 v2.2.6
go: finding github.com/gorilla/context v1.1.1
go: finding github.com/coreos/go-systemd v0.0.0-20180108085132-cc4f39464dc7
go: finding github.com/lib/pq v1.3.0
go: finding gopkg.in/asn1-ber.v1 v1.0.0-20181015200546-f715ec2f112d
go: finding github.com/Azure/go-autorest v10.15.2+incompatible
go: finding github.com/opencontainers/go-digest v1.0.0-rc1
go: finding github.com/julienschmidt/httprouter v1.2.0
go: finding github.com/fsnotify/fsnotify v1.4.9
go: finding gopkg.in/square/go-jose.v2 v2.3.1
go: finding golang.org/x/sys v0.0.0-20190412213103-97732733099d
go: finding gopkg.in/yaml.v2 v2.2.2
go: finding golang.org/x/sys v0.0.0-20191105231009-c1f44814a5cd
go: finding github.com/cespare/xxhash/v2 v2.1.0
go: finding github.com/Microsoft/go-winio v0.4.5
go: finding github.com/prometheus/client_model v0.0.0-20190129233127-fd36f4220a90
go: finding github.com/jmespath/go-jmespath v0.0.0-20180206201540-c2b33e8439af
go: finding gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405
go: finding github.com/prometheus/client_golang v1.0.0
go: finding gopkg.in/warnings.v0 v0.1.2
go: finding golang.org/x/sys v0.0.0-20191005200804-aed5e4c7ecf9
go: finding github.com/gogo/protobuf v1.1.1
go: finding cloud.google.com/go v0.38.0
go: finding github.com/googleapis/gax-go/v2 v2.0.5
go: finding github.com/docker/go-connections v0.3.0
go: finding github.com/karalabe/xgo v0.0.0-20191115072854-c5ccff8648a7
go: finding google.golang.org/appengine v1.5.0
go: finding golang.org/x/text v0.3.2
go: finding github.com/anmitsu/go-shlex v0.0.0-20161002113705-648efa622239
go: finding golang.org/x/sys v0.0.0-20181116152217-5ac8a444bdc5
go: finding golang.org/x/net v0.0.0-20180724234803-3673e40ba225
go: finding github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd
go: finding gopkg.in/src-d/go-git-fixtures.v3 v3.5.0
go: finding github.com/alecthomas/units v0.0.0-20190717042225-c3de453c63f4
go: finding golang.org/x/net v0.0.0-20190620200207-3b0461eec859
go: finding github.com/beorn7/perks v1.0.1
go: finding golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421
go: finding google.golang.org/genproto v0.0.0-20190418145605-e7d98fc518a7
go: finding golang.org/x/time v0.0.0-20181108054448-85acf8d2951c
go: finding golang.org/x/net v0.0.0-20190503192946-f4e77d36d62c
go: finding golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2
go: finding github.com/go-kit/kit v0.9.0
go: finding golang.org/x/sync v0.0.0-20181108010431-42b317875d0f
go: finding golang.org/x/sync v0.0.0-20190227155943-e225da77a7e6
go: finding golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4
go: finding github.com/googleapis/gax-go/v2 v2.0.4
go: finding github.com/arvados/cgofuse v1.2.0-arvados1
go: finding github.com/prometheus/procfs v0.0.5
go: finding github.com/sergi/go-diff v1.0.0
go: finding github.com/google/go-cmp v0.2.0
go: finding golang.org/x/sys v0.0.0-20190507160741-ecd444e8653b
go: finding github.com/golang/protobuf v1.3.2
go: finding github.com/pkg/errors v0.8.1
go: finding github.com/json-iterator/go v1.1.7
go: finding golang.org/x/lint v0.0.0-20190301231843-5614ed5bae6f
go: finding golang.org/x/exp v0.0.0-20190121172915-509febef88a4
go: finding honnef.co/go/tools v0.0.0-20190106161140-3f1c8253044a
go: finding github.com/docker/go-units v0.3.3-0.20171221200356-d59758554a3d
go: finding google.golang.org/api v0.4.0
go: finding github.com/prometheus/common v0.4.1
go: finding github.com/pkg/errors v0.8.0
go: finding github.com/google/martian v2.1.0+incompatible
go: finding github.com/davecgh/go-spew v1.1.1
go: finding golang.org/x/sync v0.0.0-20190423024810-112230192c58
go: finding golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2
go: finding go.opencensus.io v0.21.0
go: finding github.com/stretchr/testify v1.4.0
go: finding golang.org/x/tools v0.0.0-20190506145303-2d16b83fe98c
go: finding github.com/google/go-cmp v0.3.0
go: finding golang.org/x/net v0.0.0-20181114220301-adae6a3d119a
go: finding github.com/stretchr/testify v1.3.0
go: finding github.com/modern-go/reflect2 v1.0.1
go: finding github.com/go-kit/kit v0.8.0
go: finding google.golang.org/grpc v1.20.1
go: finding github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc
go: finding github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf
go: finding github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024
go: finding github.com/gliderlabs/ssh v0.2.2
go: finding cloud.google.com/go v0.34.0
go: finding golang.org/x/lint v0.0.0-20190409202823-959b441ac422
go: finding github.com/prometheus/client_golang v0.9.1
go: finding golang.org/x/sys v0.0.0-20191010194322-b09406accb47
go: finding golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e
go: finding google.golang.org/genproto v0.0.0-20190502173448-54afdca5d873
go: finding github.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515
go: finding github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c
go: finding github.com/sirupsen/logrus v1.2.0
go: finding golang.org/x/tools v0.0.0-20190312170243-e65039ee4138
go: finding github.com/golang/protobuf v1.3.1
go: finding github.com/google/pprof v0.0.0-20181206194817-3ea8567a2e57
go: finding golang.org/x/tools v0.0.0-20190311212946-11955173bddd
go: finding github.com/go-logfmt/logfmt v0.3.0
go: finding github.com/go-stack/stack v1.8.0
go: finding github.com/gorilla/mux v1.6.1-0.20180107155708-5bbbb5b2b572
go: finding github.com/golang/mock v1.2.0
go: finding github.com/BurntSushi/toml v0.3.1
go: finding golang.org/x/net v0.0.0-20190311183353-d8887717615a
go: finding github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b
go: finding github.com/hashicorp/golang-lru v0.5.0
go: finding honnef.co/go/tools v0.0.0-20190418001031-e561f6794a2a
go: finding github.com/prometheus/client_model v0.0.0-20180712105110-5c3871d89910
go: finding golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f
go: finding google.golang.org/grpc v1.19.0
go: finding github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421
go: finding github.com/prometheus/procfs v0.0.0-20181005140218-185b4288413d
go: finding golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33
go: finding golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a
go: finding github.com/json-iterator/go v1.1.6
go: finding gopkg.in/check.v1 v1.0.0-20161208181325-20d25e280405
go: finding google.golang.org/appengine v1.1.0
go: finding cloud.google.com/go v0.26.0
go: finding github.com/davecgh/go-spew v1.1.0
go: finding github.com/stretchr/objx v0.1.0
go: finding github.com/msteinert/pam v0.0.0-20190215180659-f29b9f28d6f9
go: finding golang.org/x/net v0.0.0-20180826012351-8a410e7b638d
go: finding golang.org/x/crypto v0.0.0-20180904163835-0709b304e793
go: finding github.com/jmcvetta/randutil v0.0.0-20150817122601-2bb1b664bcff
go: finding github.com/dgrijalva/jwt-go v3.1.0+incompatible
go: finding github.com/pquerna/cachecontrol v0.0.0-20180517163645-1555304b9b35
go: finding github.com/google/gofuzz v1.0.0
go: finding github.com/client9/misspell v0.3.4
go: finding github.com/pmezard/go-difflib v1.0.0
go: finding golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3
go: finding golang.org/x/tools v0.0.0-20190114222345-bf090417da8b
go: finding github.com/konsorten/go-windows-terminal-sequences v1.0.1
go: finding github.com/stretchr/objx v0.1.1
go: finding github.com/prometheus/procfs v0.0.2
go: finding honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099
go: finding github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973
go: finding github.com/golang/mock v1.1.1
go: finding google.golang.org/genproto v0.0.0-20190307195333-5fe7a883aa19
go: finding gopkg.in/yaml.v2 v2.2.1
go: finding github.com/sirupsen/logrus v1.4.2
go: finding github.com/beorn7/perks v1.0.0
go: finding google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8
go: finding golang.org/x/net v0.0.0-20190213061140-3a22650c66bd
go: finding golang.org/x/tools v0.0.0-20190226205152-f727befe758c
go: finding golang.org/x/sys v0.0.0-20180830151530-49385e6e1522
go: finding golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3
go: finding golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be
go: finding github.com/modern-go/reflect2 v0.0.0-20180701023420-4b7aa43c6742
go: finding golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961
go: finding github.com/stretchr/testify v1.2.2
go: finding github.com/AdRoll/goamz v0.0.0-20170825154802-2731d20f46f4

me@a88a5742be7f:~/arvados$ cd cmd/arvados-server
me@a88a5742be7f:~/arvados/cmd/arvados-server$ go install
# git.arvados.org/arvados.git/sdk/go/arvados
../../sdk/go/arvados/keep_service.go:155:14: undefined: http.NewRequestWithContext

arvados won't compile in buster because of golang 1.13 is required. I did this:

root@a88a5742be7f:~# echo deb http://deb.debian.org/debian buster-backports main >> /etc/apt/sources.list.d/backports.list
root@a88a5742be7f:~# apt-get update
Hit:1 http://security.debian.org/debian-security buster/updates InRelease
Hit:2 http://deb.debian.org/debian buster InRelease
Hit:3 http://deb.debian.org/debian buster-updates InRelease
Get:4 http://deb.debian.org/debian buster-backports InRelease [46.7 kB]
Get:5 http://deb.debian.org/debian buster-backports/main amd64 Packages [341 kB]
Fetched 387 kB in 1s (473 kB/s)
Reading package lists... Done
root@a88a5742be7f:~# apt-get -yq --no-install-recommends -t buster-backports install golang-go
Reading package lists...
Building dependency tree...
Reading state information...
The following additional packages will be installed:
  golang-1.14-go golang-1.14-src golang-src
Suggested packages:
  bzr | brz mercurial subversion
Recommended packages:
  pkg-config
The following NEW packages will be installed:
  golang-1.14-go golang-1.14-src
The following packages will be upgraded:
  golang-go golang-src
2 upgraded, 2 newly installed, 0 to remove and 18 not upgraded.
Need to get 63.0 MB of archives.
After this operation, 326 MB of additional disk space will be used.
Get:1 http://deb.debian.org/debian buster-backports/main amd64 golang-1.14-src amd64 1.14-2~bpo10+1 [13.3 MB]
Get:2 http://deb.debian.org/debian buster-backports/main amd64 golang-1.14-go amd64 1.14-2~bpo10+1 [49.8 MB]
Get:3 http://deb.debian.org/debian buster-backports/main amd64 golang-src amd64 2:1.14~1~bpo10+1 [4780 B]
Get:4 http://deb.debian.org/debian buster-backports/main amd64 golang-go amd64 2:1.14~1~bpo10+1 [23.9 kB]
Fetched 63.0 MB in 27s (2379 kB/s)
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package golang-1.14-src.
(Reading database ... 23385 files and directories currently installed.)
Preparing to unpack .../golang-1.14-src_1.14-2~bpo10+1_amd64.deb ...
Unpacking golang-1.14-src (1.14-2~bpo10+1) ...
Selecting previously unselected package golang-1.14-go.
Preparing to unpack .../golang-1.14-go_1.14-2~bpo10+1_amd64.deb ...
Unpacking golang-1.14-go (1.14-2~bpo10+1) ...
Preparing to unpack .../golang-src_2%3a1.14~1~bpo10+1_amd64.deb ...
Unpacking golang-src (2:1.14~1~bpo10+1) over (2:1.11~1) ...
Preparing to unpack .../golang-go_2%3a1.14~1~bpo10+1_amd64.deb ...
Unpacking golang-go (2:1.14~1~bpo10+1) over (2:1.11~1) ...
Setting up golang-1.14-src (1.14-2~bpo10+1) ...
Setting up golang-src (2:1.14~1~bpo10+1) ...
Setting up golang-1.14-go (1.14-2~bpo10+1) ...
Setting up golang-go (2:1.14~1~bpo10+1) ...
root@a88a5742be7f:~# sudo -u me -i
me@a88a5742be7f:~$ cd arvados/
me@a88a5742be7f:~/arvados$ cd cmd/arvados-server
me@a88a5742be7f:~/arvados/cmd/arvados-server$ go install

that worked.

Ahother problem with pre-requisites is that is unclear why it mentions to have a user, but then the user can do very little of the installation because of the package dependencies:

me@a88a5742be7f:~/arvados/cmd/arvados-server$ ~/go/bin/arvados-server install -type test
E: Could not open lock file /var/lib/dpkg/lock-frontend - open (13: Permission denied)
E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), are you root?
INFO[2020-07-14T17:56:36.011351803Z] exiting                                       error="exit status 100" 
me@a88a5742be7f:~/arvados/cmd/arvados-server$ sudo ~/go/bin/arvados-server install -type test

the process finished with some "hard to tell if it all went according to plan" message ...

  inflating: /var/lib/arvados/gradle-5.3.1/lib/plugins/plexus-utils-3.1.0.jar  
  inflating: /var/lib/arvados/gradle-5.3.1/lib/plugins/xbean-reflect-3.7.jar  
  inflating: /var/lib/arvados/gradle-5.3.1/lib/plugins/ion-java-1.0.2.jar  
  inflating: /var/lib/arvados/gradle-5.3.1/lib/plugins/junit-platform-engine-1.3.1.jar  
  inflating: /var/lib/arvados/gradle-5.3.1/lib/plugins/junit-platform-commons-1.3.1.jar  
  inflating: /var/lib/arvados/gradle-5.3.1/lib/plugins/apiguardian-api-1.0.0.jar  
  inflating: /var/lib/arvados/gradle-5.3.1/lib/plugins/opentest4j-1.1.1.jar  
+ ln -sf /var/lib/arvados/gradle-5.3.1/bin/gradle /usr/local/bin/
+ rm /var/lib/arvados/tmp/gradle-5.3.1-bin.zip
+ sed -i 's/^# *\(en_US.UTF-8\)/\1/' /etc/locale.gen
+ locale-gen
Generating locales (this might take a while)...
  en_US.UTF-8... done
Generation complete.
INFO[2020-07-14T18:08:50.584237848Z] postgresql cluster 11-main is down; trying to start 
2020-07-14 18:08:50.715 UTC [37582] LOG:  listening on IPv4 address "127.0.0.1", port 5432
2020-07-14 18:08:50.715 UTC [37582] LOG:  could not bind IPv6 address "::1": Cannot assign requested address
2020-07-14 18:08:50.715 UTC [37582] HINT:  Is another postmaster already running on port 5432? If not, wait a few seconds and retry.
2020-07-14 18:08:50.720 UTC [37582] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432" 
2020-07-14 18:08:50.729 UTC [37585] LOG:  database system was shut down at 2020-07-14 18:05:16 UTC
2020-07-14 18:08:50.732 UTC [37582] LOG:  database system is ready to accept connections
2020-07-14 18:08:51.638 UTC [37593] root@root FATAL:  role "root" does not exist
+ sudo systemctl restart postgresql
System has not been booted with systemd as init system (PID 1). Can't operate.
Failed to connect to bus: Host is down
WARN[2020-07-14T18:08:51.709524468Z] `systemctl restart postgresql` failed; hoping postgresql does not need to be restarted 
CREATE COLLATION
CREATE COLLATION
2020-07-14 18:08:51.885 UTC [37646] postgres@postgres ERROR:  role "arvados" does not exist
2020-07-14 18:08:51.885 UTC [37646] postgres@postgres STATEMENT:  ALTER ROLE arvados WITH LOGIN SUPERUSER ENCRYPTED PASSWORD 'insecure_arvados_test'
CREATE ROLE
INFO[2020-07-14T18:08:51.918523701Z] sent SIGTERM; waiting for postgres to shut down 
2020-07-14 18:08:51.918 UTC [37582] LOG:  received smart shutdown request
2020-07-14 18:08:51.937 UTC [37582] LOG:  background worker "logical replication launcher" (PID 37591) exited with exit code 1
2020-07-14 18:08:51.937 UTC [37586] LOG:  shutting down
2020-07-14 18:08:51.966 UTC [37582] LOG:  database system is shut down

took some time but it finished.

time ~/arvados/build/run-tests.sh WORKSPACE=~/arvados
(...)
Pass: apps/workbench_functionals tests (149s)
Pass: apps/workbench_benchmark tests (348s)
Failures (1):
Fail: apps/workbench_integration tests (1007s)
...
rm: cannot remove '/tmp/tmp.NdEARKy05e/GOPATH/pkg/mod/....
....
real    65m30.806s
user    29m15.676s
sys    4m24.804s

I'll dig down on the workbench_integration test, could be legit.

on the other hand I saw a lot of

 rm: cannot remove '/tmp/tmp.NdEARKy05e/GOPATH/pkg/mod/....

I guess that there is a missing sudo somewhere.

second run won't work. this was the previous one: "662M /tmp/tmp.NdEARKy05e",

me@a88a5742be7f:~$ time ~/arvados/build/run-tests.sh WORKSPACE=~/arvados
Checking dependencies:
locale: en_US.UTF-8
virtualenv: 15.1.0
ruby: ruby 2.5.7p206 (2019-10-01 revision 67816) [x86_64-linux]
go: go version go1.14 linux/amd64
gcc: gcc (Debian 8.3.0-6) 8.3.0
fuse.h: /usr/include/fuse/fuse.h
gnutls.h: /usr/include/gnutls/gnutls.h
Python2 pyconfig.h: /usr/include/python2.7/pyconfig.h
Python3 pyconfig.h: /usr/include/python3.7m/pyconfig.h
/bin/netstat
nginx: nginx version: nginx/1.14.2
perl: This is perl 5, version 28, subversion 1 (v5.28.1) built for x86_64-linux-gnu-thread-multi
perl ExtUtils::MakeMaker: 7.34
perl JSON: 4.02
perl LWP: 6.36
perl Net::SSL: 2.88
gitolite: /usr/bin/gitolite
npm: 6.4.1
cadaver: cadaver 0.23.3
libattr1 xattr.h: /usr/include/attr/xattr.h
libcurl curl.h: /usr/include/x86_64-linux-gnu/curl/curl.h
libpq libpq-fe.h: /usr/include/postgresql/libpq-fe.h
libpam pam_appl.h: /usr/include/security/pam_appl.h
postgresql: psql (PostgreSQL) 11.7 (Debian 11.7-0+deb10u1)
phantomjs: 1.9.8
xvfb: /usr/bin/Xvfb
graphviz: dot - graphviz version 2.40.1 (20161225.0304)
geckodriver: geckodriver 0.24.0 ( 2019-01-28)
R: /usr/bin/Rscript
testthat: /usr/bin/pandoc
procs with /dev/fuse open:
S   UID     PID    PPID  C PRI  NI   RSS    SZ WCHAN  TTY        TIME CMD
S  1000   86249       1  0  80   0 122520 649244 -    pts/0      0:00 python setup.py test
grep fuse /proc/self/mountinfo:
WORKSPACE=/home/me/arvados
Will install dependencies to /home/me/.gem/ruby/2.5.0
Will install arvados gems to /tmp/tmp.HaSn9B1QoE/GEMHOME/.gem/ruby/2.5.0
Gem search path is GEM_PATH=/tmp/tmp.HaSn9B1QoE/GEMHOME/.gem/ruby/2.5.0
/home/me/arvados/build/run-tests.sh: line 565: /tmp/tmp.HaSn9B1QoE/GEMHOME/.gem/ruby/2.5.0/bin/bundle: No such file or directory
tee: /dev/stderr: Permission denied
Fatal: install bundler (encountered in setup_ruby_environment at /home/me/arvados/build/run-tests.sh line 566)

real    0m2.262s
user    0m1.310s
sys    0m0.241s
me@a88a5742be7f:~$ 

doing a quick search for bundle, there are several of them installed after all this process...

/var/lib/arvados/bin/bundle
/var/lib/arvados/lib/ruby/gems/2.5.0/gems/bundler-2.1.4/exe/bundle
/home/me/.gem/ruby/2.5.0/bin/bundle
/home/me/.gem/ruby/2.5.0/gems/rails-controller-testing-1.0.4/test/dummy/bin/bundle
/home/me/.gem/ruby/2.5.0/gems/bundler-1.11.0/exe/bundle
/home/me/.gem/ruby/2.5.0/gems/bundler-1.11.0/lib/bundler/man/bundle
/home/me/.gem/ruby/2.5.0/gems/bundler-1.17.3/exe/bundle
/home/me/.gem/ruby/2.5.0/gems/rake-13.0.1/bin/bundle
/home/me/.gem/ruby/2.5.0/gems/railties-5.0.7.2/lib/rails/generators/rails/app/templates/bin/bundle
/home/me/.gem/ruby/2.5.0/gems/bundler-2.0.2/exe/bundle
/home/me/.gem/ruby/2.5.0/bundler/gems/arvados-c210114aa8c7/apps/workbench/bin/bundle
/home/me/.gem/ruby/2.5.0/bundler/gems/arvados-c210114aa8c7/services/api/bin/bundle
/home/me/.gem/ruby/2.5.0/bundler/gems/arvados-81725af5d5d2/apps/workbench/bin/bundle
/home/me/.gem/ruby/2.5.0/bundler/gems/arvados-81725af5d5d2/services/api/bin/bundle
/home/me/arvados/apps/workbench/bin/bundle
/home/me/arvados/apps/workbench/node_modules/.bin/bundle
/home/me/arvados/apps/workbench/node_modules/mithril/bundler/bin/bundle
/home/me/arvados/apps/workbench/node_modules/labeled-stream-splicer/test/bundle
/home/me/arvados/services/api/bin/bundle

Ward did some edits and Hakcking_prerequisites with debian:10 is working now.