https://dev.arvados.org/https://dev.arvados.org/favicon.ico?15576888422019-06-11T14:48:17ZArvadosArvados - Idea #15348: [pam] PAM module in Gohttps://dev.arvados.org/issues/15348?journal_id=753932019-06-11T14:48:17ZTom Cleggtom@curii.com
<ul><li><strong>Related to</strong> <i><a class="issue tracker-6 status-6 priority-4 priority-default closed" href="/issues/14964">Idea #14964</a>: Port arvados-pam to Python 3</i> added</li></ul> Arvados - Idea #15348: [pam] PAM module in Gohttps://dev.arvados.org/issues/15348?journal_id=754402019-06-12T14:41:06ZTom Morristfmorris@veritasgenetics.com
<ul><li><strong>Target version</strong> set to <i>Arvados Future Sprints</i></li><li><strong>Story points</strong> set to <i>3.0</i></li></ul> Arvados - Idea #15348: [pam] PAM module in Gohttps://dev.arvados.org/issues/15348?journal_id=848792020-06-17T16:30:34ZTom Cleggtom@curii.com
<ul><li><strong>Target version</strong> changed from <i>Arvados Future Sprints</i> to <i>2020-07-01 Sprint</i></li><li><strong>Assigned To</strong> set to <i>Tom Clegg</i></li></ul> Arvados - Idea #15348: [pam] PAM module in Gohttps://dev.arvados.org/issues/15348?journal_id=849452020-06-19T15:02:29ZTom Cleggtom@curii.com
<ul><li><strong>Status</strong> changed from <i>New</i> to <i>In Progress</i></li></ul> Arvados - Idea #15348: [pam] PAM module in Gohttps://dev.arvados.org/issues/15348?journal_id=849642020-06-23T13:45:33ZTom Cleggtom@curii.com
<ul></ul><p>Manual testing</p>
<pre>
~/arvados $ WORKSPACE=`pwd` ./build/run-build-packages-one-target.sh --target debian10 --only-build libpam-arvados-experimental --build-version `git describe --tags`
~/arvados $ docker run --rm -it -v ~/arvados/packages/debian10:/pkg:ro debian:10 bash
root@6106d894ce52:/# apt update && apt install rsyslog ca-certificates && /etc/init.d/rsyslog start
root@6106d894ce52:/# dpkg -i /pkg/libpam-arvados-experimental_2.0.0-551-gb694b711c_amd64.deb
root@6106d894ce52:/# perl -pi~ -e 's{pam_unix.so nullok_secure}{/usr/lib/pam_arvados.so 9tee4.arvadosapi.com tom.shell debug}' /etc/pam.d/common-auth
root@6106d894ce52:/# adduser tom
root@6106d894ce52:/# login
6106d894ce52 login: tom
Password: {asdfasdfasdfasdf}
DEBU[0001] username="tom" arvados_api_host="9tee4.arvadosapi.com" hostname="tom.shell" insecure=false
ERRO[0001] authentication failed error="request failed: https://9tee4.arvadosapi.com/arvados/v1/virtual_machines?cluster_id=&count=&filters=%5B%5B%22hostname%22%2C%22%3D%22%2C%22tom.shell%22%5D%5D&limit=2&offset=0: 401 Unauthorized: Not logged in (req-1j6dbhbi5vobl1nl40md)"
Login incorrect
6106d894ce52 login: tom
Password: {valid token}
DEBU[0012] username="tom" arvados_api_host="9tee4.arvadosapi.com" hostname="tom.shell" insecure=false
DEBU[0012] permission granted based on link with UUID <a href="https://arvadosapi.com/9tee4-o0j2j-t0nayfy5our04w9">9tee4-o0j2j-t0nayfy5our04w9</a>
Last login: Tue Jun 23 13:58:02 UTC 2020 on pts/0
Linux 6106d894ce52 4.9.0-12-amd64 #1 SMP Debian 4.9.210-1 (2020-01-20) x86_64
...
tom@6106d894ce52:/$ whoami
tom
</pre>
Questions
<ul>
<li>Should we list ca-certificates as a recommended/dependency pkg? Without it, even valid certs fail verification -- but the "insecure" option means verification isn't strictly necessary.</li>
<li>Should the package also drop an example/template file in <code>/usr/share/pam-configs/arvados</code> like its python-based predecessor? This is how docker_test.go enables the module, but it doesn't work out of the box IRL because the operator needs to specify their real API host. (If there was a default/conventional API host name like "api", we could use that, but we don't have that rn.)</li>
<li>Should we make the "vm name" config arg optional, and call <code>gethostname()</code> if it isn't provided?</li>
</ul>
<p>15348-pam-module @ <a class="changeset" title="15348: Add pam_sm_setcred stub. Otherwise, auth fails: "PAM unable to resolve symbol: pam_sm_set..." href="https://dev.arvados.org/projects/arvados/repository/arvados/revisions/b694b711c69fdfd46b6732a6d6c67e37951dab39">b694b711c69fdfd46b6732a6d6c67e37951dab39</a> -- <a class="external" href="https://ci.arvados.org/view/Developer/job/developer-run-tests/1927/"<a href="https://ci.arvados.org/view/Developer/job/developer-run-tests/1927/">developer-run-tests: #1927 <img src="https://ci.arvados.org/buildStatus/icon?job=developer-run-tests&build=1927" alt="" /></a></a></p> Arvados - Idea #15348: [pam] PAM module in Gohttps://dev.arvados.org/issues/15348?journal_id=849802020-06-23T14:22:42ZTom Cleggtom@curii.com
<ul><li><strong>File</strong> <a href="/attachments/2561">libpam-arvados-experimental_2.0.0-551-gb694b711c_amd64.deb</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/2561/libpam-arvados-experimental_2.0.0-551-gb694b711c_amd64.deb">libpam-arvados-experimental_2.0.0-551-gb694b711c_amd64.deb</a> added</li></ul> Arvados - Idea #15348: [pam] PAM module in Gohttps://dev.arvados.org/issues/15348?journal_id=850392020-06-25T14:36:24ZNico César
<ul></ul><p>Tom Clegg wrote:</p>
<blockquote>
<p>Manual testing</p>
<p>[...]</p>
</blockquote>
<pre>
WORKSPACE=`pwd` bash -x ./build/run-build-packages-one-target.sh --target debian10 --only-build lib/pam --build-version `git describe --tags`
+ read -rd '\000' helpmessage
++ basename ./build/run-build-packages-one-target.sh
++ basename ./build/run-build-packages-one-target.sh
+ set -e
+ [[ -n /home/nico/jobs/curii/arvados ]]
+ [[ -d /home/nico/jobs/curii/arvados ]]
++ getopt --name ./build/run-build-packages-one-target.sh --longoptions help,debug,test-packages,target:,command:,only-test:,force-test,only-build:,force-build,build-version: -- '' --target debian10 --only-build lib/pam --build-version 2.0.0-553-ge37223349
+ PARSEDOPTS=' --target '\''debian10'\'' --only-build '\''lib/pam'\'' --build-version '\''2.0.0-553-ge37223349'\'' --'
+ '[' 0 -ne 0 ']'
+ TARGET=debian10
+ FORCE_BUILD=0
+ COMMAND=
+ DEBUG=
+ eval set -- ' --target '\''debian10'\'' --only-build '\''lib/pam'\'' --build-version '\''2.0.0-553-ge37223349'\'' --'
++ set -- --target debian10 --only-build lib/pam --build-version 2.0.0-553-ge37223349 --
+ '[' 7 -gt 0 ']'
+ case "$1" in
+ TARGET=debian10
+ shift
+ shift
+ '[' 5 -gt 0 ']'
+ case "$1" in
+ ONLY_BUILD=lib/pam
+ shift
+ shift
+ '[' 3 -gt 0 ']'
+ case "$1" in
+ [[ -z 2.0.0-553-ge37223349 ]]
+ [[ 2.0.0-553-ge37223349 =~ (.*)-(.*) ]]
+ ARVADOS_BUILDING_VERSION=2.0.0-553
+ ARVADOS_BUILDING_ITERATION=ge37223349
+ shift
+ shift
+ '[' 1 -gt 0 ']'
+ case "$1" in
+ '[' 1 -gt 1 ']'
+ shift
+ '[' 0 -gt 0 ']'
+ set -e
+ [[ -n 2.0.0-553 ]]
+ echo 'build version='\''2.0.0-553'\'', package iteration='\''ge37223349'\'''
build version='2.0.0-553', package iteration='ge37223349'
+ [[ -n '' ]]
+ IMAGE=arvados/build:debian10
+ [[ '' != '' ]]
+++ readlink -e ./build/run-build-packages-one-target.sh
++ dirname /home/nico/jobs/curii/arvados/build/run-build-packages-one-target.sh
+ JENKINS_DIR=/home/nico/jobs/curii/arvados/build
+ [[ -n '' ]]
+ pushd /home/nico/jobs/curii/arvados/build/package-build-dockerfiles
~/jobs/curii/arvados/build/package-build-dockerfiles ~/jobs/curii/arvados
+ make debian10/generated
wget -cqO common-generated/go1.13.4.linux-amd64.tar.gz https://dl.google.com/go/go1.13.4.linux-amd64.tar.gz
wget -cqO common-generated/node-v6.11.2-linux-x64.tar.xz https://nodejs.org/dist/v6.11.2/node-v6.11.2-linux-x64.tar.xz
wget -cqO common-generated/mpapis.asc https://rvm.io/mpapis.asc
wget -cqO common-generated/pkuczynski.asc https://rvm.io/pkuczynski.asc
test -d debian10/generated || mkdir debian10/generated
cp -rlt debian10/generated common-generated/*
+ echo debian10
debian10
+ cd debian10
+ docker build --tag=arvados/build:debian10 .
Sending build context to Docker daemon 129.3MB
Step 1/15 : FROM debian:buster
---> 1b686a95ddbf
Step 2/15 : MAINTAINER Ward Vandewege <wvandewege@veritasgenetics.com>
---> Using cache
---> b7f49096ee66
Step 3/15 : ENV DEBIAN_FRONTEND noninteractive
---> Using cache
---> 8b308c3c0436
Step 4/15 : RUN /usr/bin/apt-get update && /usr/bin/apt-get install -q -y python2.7-dev python3 python-setuptools python3-setuptools python3-pip libcurl4-gnutls-dev curl git procps libattr1-dev libfuse-dev libgnutls28-dev libpq-dev python-pip unzip python3-venv python3-dev libpam-dev
---> Using cache
---> cf3d26f5e874
Step 5/15 : RUN /usr/bin/pip install 'virtualenv<20'
---> Using cache
---> 84a15a95a74a
Step 6/15 : ADD generated/mpapis.asc /tmp/
---> Using cache
---> ef75c80a4b11
Step 7/15 : ADD generated/pkuczynski.asc /tmp/
---> Using cache
---> 450b8f25abff
Step 8/15 : RUN gpg --import --no-tty /tmp/mpapis.asc && gpg --import --no-tty /tmp/pkuczynski.asc && curl -L https://get.rvm.io | bash -s stable && /usr/local/rvm/bin/rvm install 2.5 && /usr/local/rvm/bin/rvm alias create default ruby-2.5 && /usr/local/rvm/bin/rvm-exec default gem install bundler --version 2.0.2 && /usr/local/rvm/bin/rvm-exec default gem install fpm --version 1.10.2
---> Using cache
---> 0508a7b319b2
Step 9/15 : ADD generated/go1.13.4.linux-amd64.tar.gz /usr/local/
---> Using cache
---> 19ce1c3474c2
Step 10/15 : RUN ln -s /usr/local/go/bin/go /usr/local/bin/
---> Using cache
---> 293a6057d340
Step 11/15 : ADD generated/node-v6.11.2-linux-x64.tar.xz /usr/local/
---> Using cache
---> 254067212682
Step 12/15 : RUN ln -s /usr/local/node-v6.11.2-linux-x64/bin/* /usr/local/bin/
---> Using cache
---> 06bffd70074e
Step 13/15 : RUN git clone --depth 1 git://git.arvados.org/arvados.git /tmp/arvados && cd /tmp/arvados/services/api && /usr/local/rvm/bin/rvm-exec default bundle && cd /tmp/arvados/apps/workbench && /usr/local/rvm/bin/rvm-exec default bundle
---> Using cache
---> 4240ce16dbe8
Step 14/15 : ENV WORKSPACE /arvados
---> Using cache
---> a407a5d55df0
Step 15/15 : CMD ["/usr/local/rvm/bin/rvm-exec", "default", "bash", "/jenkins/run-build-packages.sh", "--target", "debian10"]
---> Using cache
---> 769e26445baf
Successfully built 769e26445baf
Successfully tagged arvados/build:debian10
real 0m0.483s
user 0m0.053s
sys 0m0.106s
+ popd
~/jobs/curii/arvados
+ test -z ''
+ packages='arvados-api-server
arvados-client
arvados-docker-cleaner
arvados-git-httpd
arvados-node-manager
arvados-src
arvados-workbench
crunch-dispatch-local
crunch-dispatch-slurm
crunch-run
crunchstat
keep-balance
keep-block-check
keepproxy
keep-rsync
keepstore
keep-web
libarvados-perl
python-arvados-fuse
python-arvados-python-client
python-arvados-cwl-runner'
+ FINAL_EXITCODE=0
+ package_fails=
+ mkdir -p /home/nico/jobs/curii/arvados/apps/workbench/vendor/cache-debian10
+ mkdir -p /home/nico/jobs/curii/arvados/services/api/vendor/cache-debian10
+ docker_volume_args=(-v "$JENKINS_DIR:/jenkins" -v "$WORKSPACE:/arvados" -v /arvados/services/api/vendor/bundle -v /arvados/apps/workbench/vendor/bundle -v "$WORKSPACE/services/api/vendor/cache-$TARGET:/arvados/services/api/vendor/cache" -v "$WORKSPACE/apps/workbench/vendor/cache-$TARGET:/arvados/apps/workbench/vendor/cache")
+ [[ -n '' ]]
+ echo
+ echo 'START: build packages on arvados/build:debian10'
START: build packages on arvados/build:debian10
+ [[ ! -e /home/nico/jobs/curii/arvados/packages/debian10/processed ]]
+ set +e
+ mv -f /home/nico/jobs/curii/arvados/packages/debian10/processed /home/nico/jobs/curii/arvados/packages/debian10/processed/
+ set -e
+ docker run --rm -v /home/nico/jobs/curii/arvados/build:/jenkins -v /home/nico/jobs/curii/arvados:/arvados -v /arvados/services/api/vendor/bundle -v /arvados/apps/workbench/vendor/bundle -v /home/nico/jobs/curii/arvados/services/api/vendor/cache-debian10:/arvados/services/api/vendor/cache -v /home/nico/jobs/curii/arvados/apps/workbench/vendor/cache-debian10:/arvados/apps/workbench/vendor/cache --env ARVADOS_BUILDING_VERSION=2.0.0-553 --env ARVADOS_BUILDING_ITERATION=ge37223349 --env ARVADOS_DEBUG= --env ONLY_BUILD=lib/pam --env FORCE_BUILD=0 arvados/build:debian10
/usr/local/lib/python2.7/dist-packages/setuptools/dist.py:476: UserWarning: Normalizing '2.0.0-553' to '2.0.0.post553'
normalized_version,
/usr/local/lib/python3.7/dist-packages/setuptools/dist.py:476: UserWarning: Normalizing '2.0.0-553' to '2.0.0.post553'
normalized_version,
Cloning into 'cwltest'...
Note: checking out '1.0.20190906212748'.
You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.
If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:
git checkout -b <new-branch-name>
HEAD is now at 98ca3c9 Merge pull request #95 from common-workflow-language/valid_package_data
go: finding cloud.google.com/go v0.38.0
go: finding github.com/arvados/goamz v0.0.0-20190905141525-1bba09f407ef
go: finding github.com/Azure/azure-sdk-for-go v19.1.0+incompatible
go: finding github.com/Azure/go-autorest v10.15.2+incompatible
go: finding github.com/BurntSushi/toml v0.3.1
go: finding github.com/Microsoft/go-winio v0.4.5
go: finding github.com/alcortesm/tgz v0.0.0-20161220082320-9c5fe88206d7
go: finding github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751
go: finding github.com/alecthomas/units v0.0.0-20190717042225-c3de453c63f4
go: finding github.com/anmitsu/go-shlex v0.0.0-20161002113705-648efa622239
go: finding github.com/arvados/cgofuse v1.2.0-arvados1
go: finding github.com/aws/aws-sdk-go v1.25.30
go: finding github.com/beorn7/perks v1.0.1
go: finding github.com/bgentry/speakeasy v0.1.0
go: finding github.com/bradleypeabody/godap v0.0.0-20170216002349-c249933bc092
go: finding github.com/cespare/xxhash/v2 v2.1.0
go: finding github.com/client9/misspell v0.3.4
go: finding github.com/coreos/go-oidc v2.1.0+incompatible
go: finding github.com/coreos/go-systemd v0.0.0-20180108085132-cc4f39464dc7
go: finding github.com/davecgh/go-spew v1.1.1
go: finding github.com/dgrijalva/jwt-go v3.1.0+incompatible
go: finding github.com/dimchansky/utfbom v1.0.0
go: finding github.com/dnaeon/go-vcr v1.0.1
go: finding github.com/docker/distribution v2.6.0-rc.1.0.20180105232752-277ed486c948+incompatible
go: finding github.com/docker/docker v1.4.2-0.20180109013817-94b8a116fbf1
go: finding github.com/docker/go-connections v0.3.0
go: finding github.com/docker/go-units v0.3.3-0.20171221200356-d59758554a3d
go: finding github.com/flynn/go-shlex v0.0.0-20150515145356-3f9db97f8568
go: finding github.com/fsnotify/fsnotify v1.4.9
go: finding github.com/ghodss/yaml v1.0.0
go: finding github.com/gliderlabs/ssh v0.2.2
go: finding github.com/go-asn1-ber/asn1-ber v1.4.1
go: finding github.com/go-kit/kit v0.9.0
go: finding github.com/go-ldap/ldap v3.0.3+incompatible
go: finding github.com/go-logfmt/logfmt v0.4.0
go: finding github.com/go-stack/stack v1.8.0
go: finding github.com/gogo/protobuf v1.1.1
go: finding github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b
go: finding github.com/golang/mock v1.2.0
go: finding github.com/golang/protobuf v1.3.2
go: finding github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c
go: finding github.com/google/go-cmp v0.3.0
go: finding github.com/google/gofuzz v1.0.0
go: finding github.com/google/martian v2.1.0+incompatible
go: finding github.com/google/pprof v0.0.0-20181206194817-3ea8567a2e57
go: finding github.com/googleapis/gax-go/v2 v2.0.5
go: finding github.com/gorilla/context v1.1.1
go: finding github.com/gorilla/mux v1.6.1-0.20180107155708-5bbbb5b2b572
go: finding github.com/hashicorp/golang-lru v0.5.1
go: finding github.com/imdario/mergo v0.3.8-0.20190415133143-5ef87b449ca7
go: finding github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99
go: finding github.com/jmcvetta/randutil v0.0.0-20150817122601-2bb1b664bcff
go: finding github.com/jmespath/go-jmespath v0.0.0-20180206201540-c2b33e8439af
go: finding github.com/json-iterator/go v1.1.7
go: finding github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024
go: finding github.com/julienschmidt/httprouter v1.2.0
go: finding github.com/karalabe/xgo v0.0.0-20191115072854-c5ccff8648a7
go: finding github.com/kevinburke/ssh_config v0.0.0-20171013211458-802051befeb5
go: finding github.com/konsorten/go-windows-terminal-sequences v1.0.1
go: finding github.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515
go: finding github.com/lib/pq v1.3.0
go: finding github.com/marstr/guid v1.1.1-0.20170427235115-8bdf7d1a087c
go: finding github.com/matttproud/golang_protobuf_extensions v1.0.1
go: finding github.com/mitchellh/go-homedir v0.0.0-20161203194507-b8bc1bf76747
go: finding github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd
go: finding github.com/modern-go/reflect2 v1.0.1
go: finding github.com/msteinert/pam v0.0.0-20190215180659-f29b9f28d6f9
go: finding github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223
go: finding github.com/opencontainers/go-digest v1.0.0-rc1
go: finding github.com/opencontainers/image-spec v1.0.1-0.20171125024018-577479e4dc27
go: finding github.com/pelletier/go-buffruneio v0.2.0
go: finding github.com/pkg/errors v0.8.1
go: finding github.com/pmezard/go-difflib v1.0.0
go: finding github.com/pquerna/cachecontrol v0.0.0-20180517163645-1555304b9b35
go: finding github.com/prometheus/client_golang v1.2.1
go: finding github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4
go: finding github.com/prometheus/common v0.7.0
go: finding github.com/prometheus/procfs v0.0.5
go: finding github.com/satori/go.uuid v1.2.1-0.20180103174451-36e9d2ebbde5
go: finding github.com/sergi/go-diff v1.0.0
go: finding github.com/sirupsen/logrus v1.4.2
go: finding github.com/src-d/gcfg v1.3.0
go: finding github.com/stretchr/objx v0.1.1
go: finding github.com/stretchr/testify v1.4.0
go: finding github.com/xanzy/ssh-agent v0.1.0
go: finding go.opencensus.io v0.21.0
go: finding golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550
go: finding golang.org/x/exp v0.0.0-20190121172915-509febef88a4
go: finding golang.org/x/lint v0.0.0-20190409202823-959b441ac422
go: finding golang.org/x/net v0.0.0-20190620200207-3b0461eec859
go: finding golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45
go: finding golang.org/x/sync v0.0.0-20190423024810-112230192c58
go: finding golang.org/x/sys v0.0.0-20191105231009-c1f44814a5cd
go: finding golang.org/x/text v0.3.2
go: finding golang.org/x/time v0.0.0-20181108054448-85acf8d2951c
go: finding golang.org/x/tools v0.0.0-20190506145303-2d16b83fe98c
go: finding google.golang.org/api v0.13.0
go: finding google.golang.org/appengine v1.5.0
go: finding google.golang.org/genproto v0.0.0-20190502173448-54afdca5d873
go: finding google.golang.org/grpc v1.20.1
go: finding gopkg.in/alecthomas/kingpin.v2 v2.2.6
go: finding gopkg.in/asn1-ber.v1 v1.0.0-20181015200546-f715ec2f112d
go: finding gopkg.in/check.v1 v1.0.0-20161208181325-20d25e280405
go: finding gopkg.in/square/go-jose.v2 v2.3.1
go: finding gopkg.in/src-d/go-billy.v4 v4.0.1
go: finding gopkg.in/src-d/go-git-fixtures.v3 v3.5.0
go: finding gopkg.in/src-d/go-git.v4 v4.0.0
go: finding gopkg.in/warnings.v0 v0.1.2
go: finding gopkg.in/yaml.v2 v2.2.4
go: finding honnef.co/go/tools v0.0.0-20190418001031-e561f6794a2a
go: finding rsc.io/getopt v0.0.0-20170811000552-20be20937449
+ echo
+ echo 'OK: build packages on arvados/build:debian10 succeeded'
OK: build packages on arvados/build:debian10 succeeded
+ test -n ''
+ exit 0
</pre>
<p>I get the successfull message but the deb package isn't there.</p>
<pre>
[nico:~/jobs/curii/arvados] master ± find -name *deb
[nico:~/jobs/curii/arvados] master ± cd packages/debian10
[nico:~/jobs/curii/arvados/packages/debian10] master ± find
.
./processed
</pre> Arvados - Idea #15348: [pam] PAM module in Gohttps://dev.arvados.org/issues/15348?journal_id=850432020-06-25T15:21:38ZNico César
<ul></ul><p>I think this was a successfull test (even if error="permission denied" was emitted) I don't have shell access in that machine</p>
<pre>
root@bdd7de30d774:/# login nico
Password:
DEBU[0002] username="nico" arvados_api_host="su92l.arvadosapi.com" hostname="shell" insecure=false
ERRO[0002] authentication failed error="request failed: https://su92l.arvadosapi.com/arvados/v1/virtual_machines?cluster_id=&count=&filters=%5B%5B%22hostname%22%2C%22%3D%22%2C%22shell%22%5D%5D&limit=2&offset=0: 401 Unauthorized: Not logged in (req-8kxe4zmvi8l71kk5qman)"
Login incorrect
bdd7de30d774 login: nico
Password:
DEBU[0018] username="nico" arvados_api_host="su92l.arvadosapi.com" hostname="shell" insecure=false
ERRO[0019] authentication failed error="permission denied"
Login incorrect
</pre> Arvados - Idea #15348: [pam] PAM module in Gohttps://dev.arvados.org/issues/15348?journal_id=850482020-06-25T16:05:14ZWard Vandewegeward@curii.com
<ul><li><strong>Related to</strong> <i><a class="issue tracker-1 status-3 priority-4 priority-default closed parent" href="/issues/6649">Bug #6649</a>: [Documentation] webshell installation</i> added</li></ul> Arvados - Idea #15348: [pam] PAM module in Gohttps://dev.arvados.org/issues/15348?journal_id=850692020-06-25T16:39:22ZNico César
<ul></ul><p>I've been trying out different things and here are the results:</p>
<p>If the user has not been enabled with the "Setup Account" dialog in the cluster, there will be a somehow confusing error: no results for hostname \"shell\". I think this is because the virtual machine will look like a 404 to that token</p>
<pre>
root@bdd7de30d774:/# login nico
Password:
DEBU[0002] username="nico" arvados_api_host="pirca.arvadosapi.com" hostname="shell" insecure=false
ERRO[0002] authentication failed error="no results for hostname \"shell\""
Login incorrect
bdd7de30d774 login:
</pre>
<p>After asking Javier to enable the account, the expected behaviour happened (with wrong token and with correct token below):</p>
<pre>
root@bdd7de30d774:/# login nico
Password:
DEBU[0002] username="nico" arvados_api_host="pirca.arvadosapi.com" hostname="shell" insecure=false
ERRO[0003] authentication failed error="request failed: https://pirca.arvadosapi.com/arvados/v1/virtual_machines?cluster_id=&count=&filters=%5B%5B%22hostname%22%2C%22%3D%22%2C%22shell%22%5D%5D&limit=2&offset=0: 401 Unauthorized: Not logged in (req-c46udi1gizfl146us3cn)"
Login incorrect
bdd7de30d774 login:
root@bdd7de30d774:/# login nico
Password:
DEBU[0002] username="nico" arvados_api_host="pirca.arvadosapi.com" hostname="shell" insecure=false
DEBU[0002] permission granted based on link with UUID <a href="https://arvadosapi.com/pirca-o0j2j-jsltqiqcm88kgvj">pirca-o0j2j-jsltqiqcm88kgvj</a>
Linux bdd7de30d774 5.6.0-2-amd64 #1 SMP Debian 5.6.14-1 (2020-05-23) x86_64
The programs included with the Debian GNU/Linux system are free software;
</pre> Arvados - Idea #15348: [pam] PAM module in Gohttps://dev.arvados.org/issues/15348?journal_id=850912020-06-26T20:29:33ZTom Cleggtom@curii.com
<ul></ul><p>Updated error message.</p>
<pre><code class="diff syntaxhl"><span class="gd">- return fmt.Errorf("no results for hostname %q", hostname)
</span><span class="gi">+ // It's possible there is no VM entry for the
+ // configured hostname, but typically this just means
+ // the user does not have permission to see (let alone
+ // log in to) this VM.
+ return errors.New("permission denied")
</span></code></pre>
<p>15348-pam-module @ <a class="changeset" title="15348: Change "no such vm" error message to "permission denied". Arvados-DCO-1.1-Signed-off-by: ..." href="https://dev.arvados.org/projects/arvados/repository/arvados/revisions/ce3903121eb9645e99f6f6846de421b9af1bb23f">ce3903121eb9645e99f6f6846de421b9af1bb23f</a> -- <a class="external" href="https://ci.arvados.org/view/Developer/job/developer-run-tests/1934/"<a href="https://ci.arvados.org/view/Developer/job/developer-run-tests/1934/">developer-run-tests: #1934 <img src="https://ci.arvados.org/buildStatus/icon?job=developer-run-tests&build=1934" alt="" /></a></a></p> Arvados - Idea #15348: [pam] PAM module in Gohttps://dev.arvados.org/issues/15348?journal_id=851192020-06-29T15:58:34ZNico César
<ul></ul><p>My personal opinion to this questions</p>
<blockquote>
<p>Should we list ca-certificates as a recommended/dependency pkg? Without it, even valid certs fail verification -- but the "insecure" option means verification isn't strictly necessary.</p>
</blockquote>
<p>I say yes to this, we have to look all the distros we supporta and see if the ca-certificates (or equivalent) works as we expect.</p>
<blockquote>
<p>Should the package also drop an example/template file in /usr/share/pam-configs/arvados like its python-based predecessor? This is how docker_test.go enables the module, but it doesn't work out of the box IRL because the operator needs to specify their real API host. (If there was a default/conventional API host name like "api", we could use that, but we don't have that rn.)</p>
</blockquote>
<p>Yes. drop the file and also have a message when you install the package that has the words "YOU NEED TO CONFIGURE THIS" large, friendly letters.</p>
<blockquote>
<p>Should we make the "vm name" config arg optional, and call gethostname() if it isn't provided?</p>
</blockquote>
<p>My take here: make it optional. Most of the gethostname() results could be pretty random, not every sysadmin takes the time to make it right, specially with cloud instances (and sometimes the cloud agent will sabotage you when rebooting the machine, I'm talking to you Azure).</p> Arvados - Idea #15348: [pam] PAM module in Gohttps://dev.arvados.org/issues/15348?journal_id=851202020-06-29T15:59:19ZNico César
<ul></ul><p>commit ce3903121eb9645e99f6f6846de421b9af1bb23f LGTM</p> Arvados - Idea #15348: [pam] PAM module in Gohttps://dev.arvados.org/issues/15348?journal_id=851212020-06-29T16:03:02ZWard Vandewegeward@curii.com
<ul></ul><p>Tom Clegg wrote:</p>
<blockquote>
Questions
<ul>
<li>Should we list ca-certificates as a recommended/dependency pkg? Without it, even valid certs fail verification -- but the "insecure" option means verification isn't strictly necessary.</li>
</ul>
</blockquote>
<p>I'm leaning towards making ca-certificates a dependency. It's pretty annoying to debug pam problems; removing a pitfall seems smart.</p>
<blockquote>
<ul>
<li>Should the package also drop an example/template file in <code>/usr/share/pam-configs/arvados</code> like its python-based predecessor? This is how docker_test.go enables the module, but it doesn't work out of the box IRL because the operator needs to specify their real API host. (If there was a default/conventional API host name like "api", we could use that, but we don't have that rn.)</li>
</ul>
</blockquote>
<p><del>Probably unnecessary if we document things well.</del> Do what Nico said above, please.</p>
<blockquote>
<ul>
<li>Should we make the "vm name" config arg optional, and call <code>gethostname()</code> if it isn't provided?</li>
</ul>
</blockquote>
<p>That's a nice to have from my perspective, I wouldn't block on it or spend much time on it.</p> Arvados - Idea #15348: [pam] PAM module in Gohttps://dev.arvados.org/issues/15348?journal_id=852182020-07-01T15:32:42ZPeter Amstutzpeter.amstutz@curii.com
<ul><li><strong>Target version</strong> changed from <i>2020-07-01 Sprint</i> to <i>2020-07-15</i></li></ul> Arvados - Idea #15348: [pam] PAM module in Gohttps://dev.arvados.org/issues/15348?journal_id=852312020-07-01T15:38:05ZTom Cleggtom@curii.com
<ul><li><strong>Target version</strong> changed from <i>2020-07-15</i> to <i>To Be Groomed</i></li></ul> Arvados - Idea #15348: [pam] PAM module in Gohttps://dev.arvados.org/issues/15348?journal_id=852602020-07-02T15:28:39ZTom Cleggtom@curii.com
<ul></ul>15348-pam-module @ <a class="changeset" title="15348: Look up hostname if not provided in config args. Arvados-DCO-1.1-Signed-off-by: Tom Clegg..." href="https://dev.arvados.org/projects/arvados/repository/arvados/revisions/183f8c6feeb8d3adbf36a1a4adf122607fb55617">183f8c6feeb8d3adbf36a1a4adf122607fb55617</a> -- <a class="external" href="https://ci.arvados.org/view/Developer/job/developer-run-tests/1940/"<a href="https://ci.arvados.org/view/Developer/job/developer-run-tests/1940/">developer-run-tests: #1940 <img src="https://ci.arvados.org/buildStatus/icon?job=developer-run-tests&build=1940" alt="" /></a></a>
<ul>
<li>rename pkg to libpam-arvados-go</li>
<li>webshell install docs say to use libpam-arvados-go instead of the python one</li>
<li>package includes /usr/share/doc/libpam-arvados-go/README with usage summary and a link to webshell doc page</li>
<li>package includes an example/template file in /usr/share/pam-configs that pam-auth-update can use</li>
<li>hostname is optional; if omitted (or "-" which is needed as a placeholder if you're using insecure/debug options) the current hostname is used instead</li>
<li>package lists ca-certificates as a dependency <del>(except I'm not sure this is working... suspect run-library.sh isn't doing what it looks like it's doing)</del> (edit: see note below)</li>
</ul> Arvados - Idea #15348: [pam] PAM module in Gohttps://dev.arvados.org/issues/15348?journal_id=852642020-07-02T21:03:36ZTom Cleggtom@curii.com
<ul></ul><p>Found & fixed the packaging bug so the dependency works now.</p>
<p>This also means the next version of keep-web will actually have the fix we thought we added in <a class="issue tracker-1 status-3 priority-4 priority-default closed parent" title="Bug: [keep-web] Avoid sniffing for content type when file extension matches a MIME type (Resolved)" href="https://dev.arvados.org/issues/16100">#16100</a> (cf. <a class="external" href="http://apt.arvados.org/pool/buster/main/k/keep-web/keep-web_2.0.3-1_amd64.deb">http://apt.arvados.org/pool/buster/main/k/keep-web/keep-web_2.0.3-1_amd64.deb</a> which does not actually list any dependencies).</p>
<p>15348-pam-module @ <a class="changeset" title="15348: Fix packaging bug that was ignoring fpm-info.sh for Go progs. Until now, the libpam-arvad..." href="https://dev.arvados.org/projects/arvados/repository/arvados/revisions/d5b5f3d7c9a218c26fcb40c5e1d79136b2d75749">d5b5f3d7c9a218c26fcb40c5e1d79136b2d75749</a> -- <a class="external" href="https://ci.arvados.org/view/Developer/job/developer-run-tests/1941/"<a href="https://ci.arvados.org/view/Developer/job/developer-run-tests/1941/">developer-run-tests: #1941 <img src="https://ci.arvados.org/buildStatus/icon?job=developer-run-tests&build=1941" alt="" /></a></a></p> Arvados - Idea #15348: [pam] PAM module in Gohttps://dev.arvados.org/issues/15348?journal_id=852652020-07-02T21:06:11ZTom Cleggtom@curii.com
<ul><li><strong>File</strong> <a href="/attachments/2565">libpam-arvados-go_2.0.0-599-gd5b5f3d7c_amd64.deb</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/2565/libpam-arvados-go_2.0.0-599-gd5b5f3d7c_amd64.deb">libpam-arvados-go_2.0.0-599-gd5b5f3d7c_amd64.deb</a> added</li></ul> Arvados - Idea #15348: [pam] PAM module in Gohttps://dev.arvados.org/issues/15348?journal_id=852732020-07-06T15:29:19ZTom Cleggtom@curii.com
<ul><li><strong>Target version</strong> changed from <i>To Be Groomed</i> to <i>2020-07-15</i></li></ul> Arvados - Idea #15348: [pam] PAM module in Gohttps://dev.arvados.org/issues/15348?journal_id=852752020-07-06T15:49:46ZWard Vandewegeward@curii.com
<ul><li><strong>Related to</strong> <i><a class="issue tracker-1 status-3 priority-4 priority-default closed parent" href="/issues/16100">Bug #16100</a>: [keep-web] Avoid sniffing for content type when file extension matches a MIME type</i> added</li></ul> Arvados - Idea #15348: [pam] PAM module in Gohttps://dev.arvados.org/issues/15348?journal_id=852772020-07-06T15:56:52ZWard Vandewegeward@curii.com
<ul></ul><p>15348-pam-module @ <a class="changeset" title="15348: Fix packaging bug that was ignoring fpm-info.sh for Go progs. Until now, the libpam-arvad..." href="https://dev.arvados.org/projects/arvados/repository/arvados/revisions/d5b5f3d7c9a218c26fcb40c5e1d79136b2d75749">d5b5f3d7c9a218c26fcb40c5e1d79136b2d75749</a> LGTM. We will need to cherry pick this commit and do an errata for the <a class="issue tracker-1 status-3 priority-4 priority-default closed parent" title="Bug: [keep-web] Avoid sniffing for content type when file extension matches a MIME type (Resolved)" href="https://dev.arvados.org/issues/16100">#16100</a> bugfix in the release notes for the next point release, if we do another one on the 2.0 series.</p> Arvados - Idea #15348: [pam] PAM module in Gohttps://dev.arvados.org/issues/15348?journal_id=852792020-07-06T18:18:19ZWard Vandewegeward@curii.com
<ul><li><strong>Release</strong> set to <i>34</i></li></ul><p>Added to release 2.0.4 because we should include <a class="changeset" title="15348: Fix packaging bug that was ignoring fpm-info.sh for Go progs. Until now, the libpam-arvad..." href="https://dev.arvados.org/projects/arvados/repository/arvados/revisions/d5b5f3d7c9a218c26fcb40c5e1d79136b2d75749">d5b5f3d7c9a218c26fcb40c5e1d79136b2d75749</a> to fix the package dependency bug that affected <a class="issue tracker-1 status-3 priority-4 priority-default closed parent" title="Bug: [keep-web] Avoid sniffing for content type when file extension matches a MIME type (Resolved)" href="https://dev.arvados.org/issues/16100">#16100</a></p> Arvados - Idea #15348: [pam] PAM module in Gohttps://dev.arvados.org/issues/15348?journal_id=852812020-07-06T19:08:55ZTom Cleggtom@curii.com
<ul><li><strong>Status</strong> changed from <i>In Progress</i> to <i>Resolved</i></li></ul> Arvados - Idea #15348: [pam] PAM module in Gohttps://dev.arvados.org/issues/15348?journal_id=857382020-08-04T20:28:22ZWard Vandewegeward@curii.com
<ul><li><strong>Release</strong> changed from <i>34</i> to <i>25</i></li></ul> Arvados - Idea #15348: [pam] PAM module in Gohttps://dev.arvados.org/issues/15348?journal_id=874312020-09-29T21:57:46ZWard Vandewegeward@curii.com
<ul><li><strong>Related to</strong> <i><a class="issue tracker-1 status-3 priority-4 priority-default closed" href="/issues/16920">Bug #16920</a>: [libpam-arvados-go] the deployed pam config breaks /usr/sbin/pam-auth-update</i> added</li></ul>