Arvados

Manual testing

~/arvados $ WORKSPACE=`pwd` ./build/run-build-packages-one-target.sh --target debian10 --only-build libpam-arvados-experimental --build-version `git describe --tags`
~/arvados $ docker run --rm -it -v ~/arvados/packages/debian10:/pkg:ro debian:10 bash
root@6106d894ce52:/# apt update && apt install rsyslog ca-certificates && /etc/init.d/rsyslog start
root@6106d894ce52:/# dpkg -i /pkg/libpam-arvados-experimental_2.0.0-551-gb694b711c_amd64.deb 
root@6106d894ce52:/# perl -pi~ -e 's{pam_unix.so nullok_secure}{/usr/lib/pam_arvados.so 9tee4.arvadosapi.com tom.shell debug}' /etc/pam.d/common-auth
root@6106d894ce52:/# adduser tom
root@6106d894ce52:/# login
6106d894ce52 login: tom
Password: {asdfasdfasdfasdf}
DEBU[0001] username="tom" arvados_api_host="9tee4.arvadosapi.com" hostname="tom.shell" insecure=false 
ERRO[0001] authentication failed                         error="request failed: https://9tee4.arvadosapi.com/arvados/v1/virtual_machines?cluster_id=&count=&filters=%5B%5B%22hostname%22%2C%22%3D%22%2C%22tom.shell%22%5D%5D&limit=2&offset=0: 401 Unauthorized: Not logged in (req-1j6dbhbi5vobl1nl40md)" 

Login incorrect
6106d894ce52 login: tom
Password: {valid token}
DEBU[0012] username="tom" arvados_api_host="9tee4.arvadosapi.com" hostname="tom.shell" insecure=false 
DEBU[0012] permission granted based on link with UUID 9tee4-o0j2j-t0nayfy5our04w9 
Last login: Tue Jun 23 13:58:02 UTC 2020 on pts/0
Linux 6106d894ce52 4.9.0-12-amd64 #1 SMP Debian 4.9.210-1 (2020-01-20) x86_64
...
tom@6106d894ce52:/$ whoami
tom

• Should we list ca-certificates as a recommended/dependency pkg? Without it, even valid certs fail verification -- but the "insecure" option means verification isn't strictly necessary.
• Should the package also drop an example/template file in /usr/share/pam-configs/arvados like its python-based predecessor? This is how docker_test.go enables the module, but it doesn't work out of the box IRL because the operator needs to specify their real API host. (If there was a default/conventional API host name like "api", we could use that, but we don't have that rn.)
• Should we make the "vm name" config arg optional, and call gethostname() if it isn't provided?

15348-pam-module @ b694b711c69fdfd46b6732a6d6c67e37951dab39 -- developer-run-tests: #1927

Tom Clegg wrote:

Manual testing

[...]

WORKSPACE=`pwd` bash -x ./build/run-build-packages-one-target.sh --target debian10 --only-build lib/pam --build-version `git describe --tags`
+ read -rd '\000' helpmessage
++ basename ./build/run-build-packages-one-target.sh
++ basename ./build/run-build-packages-one-target.sh
+ set -e
+ [[ -n /home/nico/jobs/curii/arvados ]]
+ [[ -d /home/nico/jobs/curii/arvados ]]
++ getopt --name ./build/run-build-packages-one-target.sh --longoptions help,debug,test-packages,target:,command:,only-test:,force-test,only-build:,force-build,build-version: -- '' --target debian10 --only-build lib/pam --build-version 2.0.0-553-ge37223349
+ PARSEDOPTS=' --target '\''debian10'\'' --only-build '\''lib/pam'\'' --build-version '\''2.0.0-553-ge37223349'\'' --'
+ '[' 0 -ne 0 ']'
+ TARGET=debian10
+ FORCE_BUILD=0
+ COMMAND=
+ DEBUG=
+ eval set -- ' --target '\''debian10'\'' --only-build '\''lib/pam'\'' --build-version '\''2.0.0-553-ge37223349'\'' --'
++ set -- --target debian10 --only-build lib/pam --build-version 2.0.0-553-ge37223349 --
+ '[' 7 -gt 0 ']'
+ case "$1" in
+ TARGET=debian10
+ shift
+ shift
+ '[' 5 -gt 0 ']'
+ case "$1" in
+ ONLY_BUILD=lib/pam
+ shift
+ shift
+ '[' 3 -gt 0 ']'
+ case "$1" in
+ [[ -z 2.0.0-553-ge37223349 ]]
+ [[ 2.0.0-553-ge37223349 =~ (.*)-(.*) ]]
+ ARVADOS_BUILDING_VERSION=2.0.0-553
+ ARVADOS_BUILDING_ITERATION=ge37223349
+ shift
+ shift
+ '[' 1 -gt 0 ']'
+ case "$1" in
+ '[' 1 -gt 1 ']'
+ shift
+ '[' 0 -gt 0 ']'
+ set -e
+ [[ -n 2.0.0-553 ]]
+ echo 'build version='\''2.0.0-553'\'', package iteration='\''ge37223349'\'''
build version='2.0.0-553', package iteration='ge37223349'
+ [[ -n '' ]]
+ IMAGE=arvados/build:debian10
+ [[ '' != '' ]]
+++ readlink -e ./build/run-build-packages-one-target.sh
++ dirname /home/nico/jobs/curii/arvados/build/run-build-packages-one-target.sh
+ JENKINS_DIR=/home/nico/jobs/curii/arvados/build
+ [[ -n '' ]]
+ pushd /home/nico/jobs/curii/arvados/build/package-build-dockerfiles
~/jobs/curii/arvados/build/package-build-dockerfiles ~/jobs/curii/arvados
+ make debian10/generated
wget -cqO common-generated/go1.13.4.linux-amd64.tar.gz https://dl.google.com/go/go1.13.4.linux-amd64.tar.gz
wget -cqO common-generated/node-v6.11.2-linux-x64.tar.xz https://nodejs.org/dist/v6.11.2/node-v6.11.2-linux-x64.tar.xz
wget -cqO common-generated/mpapis.asc https://rvm.io/mpapis.asc
wget -cqO common-generated/pkuczynski.asc https://rvm.io/pkuczynski.asc
test -d debian10/generated || mkdir debian10/generated
cp -rlt debian10/generated common-generated/*
+ echo debian10
debian10
+ cd debian10
+ docker build --tag=arvados/build:debian10 .
Sending build context to Docker daemon  129.3MB
Step 1/15 : FROM debian:buster
 ---> 1b686a95ddbf
Step 2/15 : MAINTAINER Ward Vandewege <wvandewege@veritasgenetics.com>
 ---> Using cache
 ---> b7f49096ee66
Step 3/15 : ENV DEBIAN_FRONTEND noninteractive
 ---> Using cache
 ---> 8b308c3c0436
Step 4/15 : RUN /usr/bin/apt-get update && /usr/bin/apt-get install -q -y python2.7-dev python3 python-setuptools python3-setuptools python3-pip libcurl4-gnutls-dev curl git procps libattr1-dev libfuse-dev libgnutls28-dev libpq-dev python-pip unzip python3-venv python3-dev libpam-dev
 ---> Using cache
 ---> cf3d26f5e874
Step 5/15 : RUN /usr/bin/pip install 'virtualenv<20'
 ---> Using cache
 ---> 84a15a95a74a
Step 6/15 : ADD generated/mpapis.asc /tmp/
 ---> Using cache
 ---> ef75c80a4b11
Step 7/15 : ADD generated/pkuczynski.asc /tmp/
 ---> Using cache
 ---> 450b8f25abff
Step 8/15 : RUN gpg --import --no-tty /tmp/mpapis.asc &&     gpg --import --no-tty /tmp/pkuczynski.asc &&     curl -L https://get.rvm.io | bash -s stable &&     /usr/local/rvm/bin/rvm install 2.5 &&     /usr/local/rvm/bin/rvm alias create default ruby-2.5 &&     /usr/local/rvm/bin/rvm-exec default gem install bundler --version 2.0.2 &&     /usr/local/rvm/bin/rvm-exec default gem install fpm --version 1.10.2
 ---> Using cache
 ---> 0508a7b319b2
Step 9/15 : ADD generated/go1.13.4.linux-amd64.tar.gz /usr/local/
 ---> Using cache
 ---> 19ce1c3474c2
Step 10/15 : RUN ln -s /usr/local/go/bin/go /usr/local/bin/
 ---> Using cache
 ---> 293a6057d340
Step 11/15 : ADD generated/node-v6.11.2-linux-x64.tar.xz /usr/local/
 ---> Using cache
 ---> 254067212682
Step 12/15 : RUN ln -s /usr/local/node-v6.11.2-linux-x64/bin/* /usr/local/bin/
 ---> Using cache
 ---> 06bffd70074e
Step 13/15 : RUN git clone --depth 1 git://git.arvados.org/arvados.git /tmp/arvados && cd /tmp/arvados/services/api && /usr/local/rvm/bin/rvm-exec default bundle && cd /tmp/arvados/apps/workbench && /usr/local/rvm/bin/rvm-exec default bundle
 ---> Using cache
 ---> 4240ce16dbe8
Step 14/15 : ENV WORKSPACE /arvados
 ---> Using cache
 ---> a407a5d55df0
Step 15/15 : CMD ["/usr/local/rvm/bin/rvm-exec", "default", "bash", "/jenkins/run-build-packages.sh", "--target", "debian10"]
 ---> Using cache
 ---> 769e26445baf
Successfully built 769e26445baf
Successfully tagged arvados/build:debian10

real    0m0.483s
user    0m0.053s
sys    0m0.106s
+ popd
~/jobs/curii/arvados
+ test -z ''
+ packages='arvados-api-server
        arvados-client
        arvados-docker-cleaner
        arvados-git-httpd
        arvados-node-manager
        arvados-src
        arvados-workbench
        crunch-dispatch-local
        crunch-dispatch-slurm
        crunch-run
        crunchstat
        keep-balance
        keep-block-check
        keepproxy
        keep-rsync
        keepstore
        keep-web
        libarvados-perl
        python-arvados-fuse
        python-arvados-python-client
        python-arvados-cwl-runner'
+ FINAL_EXITCODE=0
+ package_fails=
+ mkdir -p /home/nico/jobs/curii/arvados/apps/workbench/vendor/cache-debian10
+ mkdir -p /home/nico/jobs/curii/arvados/services/api/vendor/cache-debian10
+ docker_volume_args=(-v "$JENKINS_DIR:/jenkins" -v "$WORKSPACE:/arvados" -v /arvados/services/api/vendor/bundle -v /arvados/apps/workbench/vendor/bundle -v "$WORKSPACE/services/api/vendor/cache-$TARGET:/arvados/services/api/vendor/cache" -v "$WORKSPACE/apps/workbench/vendor/cache-$TARGET:/arvados/apps/workbench/vendor/cache")
+ [[ -n '' ]]
+ echo

+ echo 'START: build packages on arvados/build:debian10'
START: build packages on arvados/build:debian10
+ [[ ! -e /home/nico/jobs/curii/arvados/packages/debian10/processed ]]
+ set +e
+ mv -f /home/nico/jobs/curii/arvados/packages/debian10/processed /home/nico/jobs/curii/arvados/packages/debian10/processed/
+ set -e
+ docker run --rm -v /home/nico/jobs/curii/arvados/build:/jenkins -v /home/nico/jobs/curii/arvados:/arvados -v /arvados/services/api/vendor/bundle -v /arvados/apps/workbench/vendor/bundle -v /home/nico/jobs/curii/arvados/services/api/vendor/cache-debian10:/arvados/services/api/vendor/cache -v /home/nico/jobs/curii/arvados/apps/workbench/vendor/cache-debian10:/arvados/apps/workbench/vendor/cache --env ARVADOS_BUILDING_VERSION=2.0.0-553 --env ARVADOS_BUILDING_ITERATION=ge37223349 --env ARVADOS_DEBUG= --env ONLY_BUILD=lib/pam --env FORCE_BUILD=0 arvados/build:debian10
/usr/local/lib/python2.7/dist-packages/setuptools/dist.py:476: UserWarning: Normalizing '2.0.0-553' to '2.0.0.post553'
  normalized_version,
/usr/local/lib/python3.7/dist-packages/setuptools/dist.py:476: UserWarning: Normalizing '2.0.0-553' to '2.0.0.post553'
  normalized_version,
Cloning into 'cwltest'...
Note: checking out '1.0.20190906212748'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b <new-branch-name>

HEAD is now at 98ca3c9 Merge pull request #95 from common-workflow-language/valid_package_data
go: finding cloud.google.com/go v0.38.0
go: finding github.com/arvados/goamz v0.0.0-20190905141525-1bba09f407ef
go: finding github.com/Azure/azure-sdk-for-go v19.1.0+incompatible
go: finding github.com/Azure/go-autorest v10.15.2+incompatible
go: finding github.com/BurntSushi/toml v0.3.1
go: finding github.com/Microsoft/go-winio v0.4.5
go: finding github.com/alcortesm/tgz v0.0.0-20161220082320-9c5fe88206d7
go: finding github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751
go: finding github.com/alecthomas/units v0.0.0-20190717042225-c3de453c63f4
go: finding github.com/anmitsu/go-shlex v0.0.0-20161002113705-648efa622239
go: finding github.com/arvados/cgofuse v1.2.0-arvados1
go: finding github.com/aws/aws-sdk-go v1.25.30
go: finding github.com/beorn7/perks v1.0.1
go: finding github.com/bgentry/speakeasy v0.1.0
go: finding github.com/bradleypeabody/godap v0.0.0-20170216002349-c249933bc092
go: finding github.com/cespare/xxhash/v2 v2.1.0
go: finding github.com/client9/misspell v0.3.4
go: finding github.com/coreos/go-oidc v2.1.0+incompatible
go: finding github.com/coreos/go-systemd v0.0.0-20180108085132-cc4f39464dc7
go: finding github.com/davecgh/go-spew v1.1.1
go: finding github.com/dgrijalva/jwt-go v3.1.0+incompatible
go: finding github.com/dimchansky/utfbom v1.0.0
go: finding github.com/dnaeon/go-vcr v1.0.1
go: finding github.com/docker/distribution v2.6.0-rc.1.0.20180105232752-277ed486c948+incompatible
go: finding github.com/docker/docker v1.4.2-0.20180109013817-94b8a116fbf1
go: finding github.com/docker/go-connections v0.3.0
go: finding github.com/docker/go-units v0.3.3-0.20171221200356-d59758554a3d
go: finding github.com/flynn/go-shlex v0.0.0-20150515145356-3f9db97f8568
go: finding github.com/fsnotify/fsnotify v1.4.9
go: finding github.com/ghodss/yaml v1.0.0
go: finding github.com/gliderlabs/ssh v0.2.2
go: finding github.com/go-asn1-ber/asn1-ber v1.4.1
go: finding github.com/go-kit/kit v0.9.0
go: finding github.com/go-ldap/ldap v3.0.3+incompatible
go: finding github.com/go-logfmt/logfmt v0.4.0
go: finding github.com/go-stack/stack v1.8.0
go: finding github.com/gogo/protobuf v1.1.1
go: finding github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b
go: finding github.com/golang/mock v1.2.0
go: finding github.com/golang/protobuf v1.3.2
go: finding github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c
go: finding github.com/google/go-cmp v0.3.0
go: finding github.com/google/gofuzz v1.0.0
go: finding github.com/google/martian v2.1.0+incompatible
go: finding github.com/google/pprof v0.0.0-20181206194817-3ea8567a2e57
go: finding github.com/googleapis/gax-go/v2 v2.0.5
go: finding github.com/gorilla/context v1.1.1
go: finding github.com/gorilla/mux v1.6.1-0.20180107155708-5bbbb5b2b572
go: finding github.com/hashicorp/golang-lru v0.5.1
go: finding github.com/imdario/mergo v0.3.8-0.20190415133143-5ef87b449ca7
go: finding github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99
go: finding github.com/jmcvetta/randutil v0.0.0-20150817122601-2bb1b664bcff
go: finding github.com/jmespath/go-jmespath v0.0.0-20180206201540-c2b33e8439af
go: finding github.com/json-iterator/go v1.1.7
go: finding github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024
go: finding github.com/julienschmidt/httprouter v1.2.0
go: finding github.com/karalabe/xgo v0.0.0-20191115072854-c5ccff8648a7
go: finding github.com/kevinburke/ssh_config v0.0.0-20171013211458-802051befeb5
go: finding github.com/konsorten/go-windows-terminal-sequences v1.0.1
go: finding github.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515
go: finding github.com/lib/pq v1.3.0
go: finding github.com/marstr/guid v1.1.1-0.20170427235115-8bdf7d1a087c
go: finding github.com/matttproud/golang_protobuf_extensions v1.0.1
go: finding github.com/mitchellh/go-homedir v0.0.0-20161203194507-b8bc1bf76747
go: finding github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd
go: finding github.com/modern-go/reflect2 v1.0.1
go: finding github.com/msteinert/pam v0.0.0-20190215180659-f29b9f28d6f9
go: finding github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223
go: finding github.com/opencontainers/go-digest v1.0.0-rc1
go: finding github.com/opencontainers/image-spec v1.0.1-0.20171125024018-577479e4dc27
go: finding github.com/pelletier/go-buffruneio v0.2.0
go: finding github.com/pkg/errors v0.8.1
go: finding github.com/pmezard/go-difflib v1.0.0
go: finding github.com/pquerna/cachecontrol v0.0.0-20180517163645-1555304b9b35
go: finding github.com/prometheus/client_golang v1.2.1
go: finding github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4
go: finding github.com/prometheus/common v0.7.0
go: finding github.com/prometheus/procfs v0.0.5
go: finding github.com/satori/go.uuid v1.2.1-0.20180103174451-36e9d2ebbde5
go: finding github.com/sergi/go-diff v1.0.0
go: finding github.com/sirupsen/logrus v1.4.2
go: finding github.com/src-d/gcfg v1.3.0
go: finding github.com/stretchr/objx v0.1.1
go: finding github.com/stretchr/testify v1.4.0
go: finding github.com/xanzy/ssh-agent v0.1.0
go: finding go.opencensus.io v0.21.0
go: finding golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550
go: finding golang.org/x/exp v0.0.0-20190121172915-509febef88a4
go: finding golang.org/x/lint v0.0.0-20190409202823-959b441ac422
go: finding golang.org/x/net v0.0.0-20190620200207-3b0461eec859
go: finding golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45
go: finding golang.org/x/sync v0.0.0-20190423024810-112230192c58
go: finding golang.org/x/sys v0.0.0-20191105231009-c1f44814a5cd
go: finding golang.org/x/text v0.3.2
go: finding golang.org/x/time v0.0.0-20181108054448-85acf8d2951c
go: finding golang.org/x/tools v0.0.0-20190506145303-2d16b83fe98c
go: finding google.golang.org/api v0.13.0
go: finding google.golang.org/appengine v1.5.0
go: finding google.golang.org/genproto v0.0.0-20190502173448-54afdca5d873
go: finding google.golang.org/grpc v1.20.1
go: finding gopkg.in/alecthomas/kingpin.v2 v2.2.6
go: finding gopkg.in/asn1-ber.v1 v1.0.0-20181015200546-f715ec2f112d
go: finding gopkg.in/check.v1 v1.0.0-20161208181325-20d25e280405
go: finding gopkg.in/square/go-jose.v2 v2.3.1
go: finding gopkg.in/src-d/go-billy.v4 v4.0.1
go: finding gopkg.in/src-d/go-git-fixtures.v3 v3.5.0
go: finding gopkg.in/src-d/go-git.v4 v4.0.0
go: finding gopkg.in/warnings.v0 v0.1.2
go: finding gopkg.in/yaml.v2 v2.2.4
go: finding honnef.co/go/tools v0.0.0-20190418001031-e561f6794a2a
go: finding rsc.io/getopt v0.0.0-20170811000552-20be20937449
+ echo

+ echo 'OK: build packages on arvados/build:debian10 succeeded'
OK: build packages on arvados/build:debian10 succeeded
+ test -n ''
+ exit 0

I get the successfull message but the deb package isn't there.

[nico:~/jobs/curii/arvados] master ± find -name *deb
[nico:~/jobs/curii/arvados] master ± cd packages/debian10
[nico:~/jobs/curii/arvados/packages/debian10] master ± find
.
./processed

I think this was a successfull test (even if error="permission denied" was emitted) I don't have shell access in that machine

root@bdd7de30d774:/# login nico
Password: 
DEBU[0002] username="nico" arvados_api_host="su92l.arvadosapi.com" hostname="shell" insecure=false 
ERRO[0002] authentication failed                         error="request failed: https://su92l.arvadosapi.com/arvados/v1/virtual_machines?cluster_id=&count=&filters=%5B%5B%22hostname%22%2C%22%3D%22%2C%22shell%22%5D%5D&limit=2&offset=0: 401 Unauthorized: Not logged in (req-8kxe4zmvi8l71kk5qman)" 

Login incorrect
bdd7de30d774 login: nico
Password: 
DEBU[0018] username="nico" arvados_api_host="su92l.arvadosapi.com" hostname="shell" insecure=false 
ERRO[0019] authentication failed                         error="permission denied" 

Login incorrect

I've been trying out different things and here are the results:

If the user has not been enabled with the "Setup Account" dialog in the cluster, there will be a somehow confusing error: no results for hostname \"shell\". I think this is because the virtual machine will look like a 404 to that token

root@bdd7de30d774:/# login nico
Password: 
DEBU[0002] username="nico" arvados_api_host="pirca.arvadosapi.com" hostname="shell" insecure=false 
ERRO[0002] authentication failed                         error="no results for hostname \"shell\"" 

Login incorrect
bdd7de30d774 login: 

After asking Javier to enable the account, the expected behaviour happened (with wrong token and with correct token below):

root@bdd7de30d774:/# login nico
Password: 
DEBU[0002] username="nico" arvados_api_host="pirca.arvadosapi.com" hostname="shell" insecure=false 
ERRO[0003] authentication failed                         error="request failed: https://pirca.arvadosapi.com/arvados/v1/virtual_machines?cluster_id=&count=&filters=%5B%5B%22hostname%22%2C%22%3D%22%2C%22shell%22%5D%5D&limit=2&offset=0: 401 Unauthorized: Not logged in (req-c46udi1gizfl146us3cn)" 

Login incorrect
bdd7de30d774 login: 

root@bdd7de30d774:/# login nico 
Password: 
DEBU[0002] username="nico" arvados_api_host="pirca.arvadosapi.com" hostname="shell" insecure=false 
DEBU[0002] permission granted based on link with UUID pirca-o0j2j-jsltqiqcm88kgvj 
Linux bdd7de30d774 5.6.0-2-amd64 #1 SMP Debian 5.6.14-1 (2020-05-23) x86_64

The programs included with the Debian GNU/Linux system are free software;

Updated error message.

-               return fmt.Errorf("no results for hostname %q", hostname)
+               // It's possible there is no VM entry for the
+               // configured hostname, but typically this just means
+               // the user does not have permission to see (let alone
+               // log in to) this VM.
+               return errors.New("permission denied")

15348-pam-module @ ce3903121eb9645e99f6f6846de421b9af1bb23f -- developer-run-tests: #1934

My personal opinion to this questions

Should we list ca-certificates as a recommended/dependency pkg? Without it, even valid certs fail verification -- but the "insecure" option means verification isn't strictly necessary.

I say yes to this, we have to look all the distros we supporta and see if the ca-certificates (or equivalent) works as we expect.

Should the package also drop an example/template file in /usr/share/pam-configs/arvados like its python-based predecessor? This is how docker_test.go enables the module, but it doesn't work out of the box IRL because the operator needs to specify their real API host. (If there was a default/conventional API host name like "api", we could use that, but we don't have that rn.)

Yes. drop the file and also have a message when you install the package that has the words "YOU NEED TO CONFIGURE THIS" large, friendly letters.

Should we make the "vm name" config arg optional, and call gethostname() if it isn't provided?

My take here: make it optional. Most of the gethostname() results could be pretty random, not every sysadmin takes the time to make it right, specially with cloud instances (and sometimes the cloud agent will sabotage you when rebooting the machine, I'm talking to you Azure).

commit ce3903121eb9645e99f6f6846de421b9af1bb23f LGTM

Tom Clegg wrote:

Questions

• Should we list ca-certificates as a recommended/dependency pkg? Without it, even valid certs fail verification -- but the "insecure" option means verification isn't strictly necessary.

I'm leaning towards making ca-certificates a dependency. It's pretty annoying to debug pam problems; removing a pitfall seems smart.

• Should the package also drop an example/template file in /usr/share/pam-configs/arvados like its python-based predecessor? This is how docker_test.go enables the module, but it doesn't work out of the box IRL because the operator needs to specify their real API host. (If there was a default/conventional API host name like "api", we could use that, but we don't have that rn.)

Probably unnecessary if we document things well. Do what Nico said above, please.

• Should we make the "vm name" config arg optional, and call gethostname() if it isn't provided?

That's a nice to have from my perspective, I wouldn't block on it or spend much time on it.

developer-run-tests: #1940

• rename pkg to libpam-arvados-go
• webshell install docs say to use libpam-arvados-go instead of the python one
• package includes /usr/share/doc/libpam-arvados-go/README with usage summary and a link to webshell doc page
• package includes an example/template file in /usr/share/pam-configs that pam-auth-update can use
• hostname is optional; if omitted (or "-" which is needed as a placeholder if you're using insecure/debug options) the current hostname is used instead
• package lists ca-certificates as a dependency (except I'm not sure this is working... suspect run-library.sh isn't doing what it looks like it's doing) (edit: see note below)

Found & fixed the packaging bug so the dependency works now.

This also means the next version of keep-web will actually have the fix we thought we added in #16100 (cf. http://apt.arvados.org/pool/buster/main/k/keep-web/keep-web_2.0.3-1_amd64.deb which does not actually list any dependencies).

15348-pam-module @ d5b5f3d7c9a218c26fcb40c5e1d79136b2d75749 -- developer-run-tests: #1941

15348-pam-module @ d5b5f3d7c9a218c26fcb40c5e1d79136b2d75749 LGTM. We will need to cherry pick this commit and do an errata for the #16100 bugfix in the release notes for the next point release, if we do another one on the 2.0 series.