Arvados

For test -d x || mkdir x why not just mkdir -p x?

Merged your #6682 into this so I could build in docker container instead of installing fpm etc. myself. The ubuntu1404 packages built with no problems¹.

I haven't been able to install the resulting packages, though. My strategy is:

• $ docker run -it -v $HOME/arvados/packages:/pkg:ro ubuntu:trusty bash
  container# cd /pkg/ubuntu1404
  container# dpkg -i *.deb
  container# apt-get -f install
  

Normally that lets me install packages from local files, then satisfy any missing dependencies from apt sources. Instead, that gets me here:

• The following packages will be REMOVED:
    arvados-node-manager libpam-arvados python-arvados-fuse
    python-arvados-python-client
  

This seems to give me a list of packages that aren't offered either in this set of packages or in the usual trusty sources.

container# dpkg -i arvados-node-manager* libpam-arvados* python-arvados-fuse* python-arvados-python-client* 2>&1 | grep 'is not installed' | sort -u
  Package libpam-python is not installed.
  Package python-daemon is not installed.
  Package python-gflags is not installed.
  Package python-pykka is not installed.
  Package python-ws4py is not installed.
  Package python2.7 is not installed.

I'm guessing these are the things that shouldn't need to be backported/installed as debs in 1404 like they did in 1204? Anyway, it seems like either they need to be removed from the dependencies lists in the deb, or we need to generate them.

¹ I did see a "gem: command not found" fly by which I haven't tracked down yet. I wonder if an uncaught error is causing incomplete api/workbench packages.

Tom Clegg wrote:

For test -d x || mkdir x why not just mkdir -p x?

Because mkdir -p's functional purpose is to create any directories that are necessary to make the argument exist, including parents, and that's not the desired behavior here. If the parent directory is missing, something is very wrong and we should fail.

Merged your #6682 into this so I could build in docker container instead of installing fpm etc. myself. The ubuntu1404 packages built with no problems¹.

I haven't been able to install the resulting packages, though. My strategy is:

• [...]

Normally that lets me install packages from local files, then satisfy any missing dependencies from apt sources. Instead, that gets me here:

• [...]

This seems to give me a list of packages that aren't offered either in this set of packages or in the usual trusty sources.

Those packages are in trusty's universe section. The trusty Docker image has the apt sources for trusty enabled, but not in apt's database. Starting your test method with apt-get update should get you a lot farther.

I can make sure a note about this gets added to the documentation when I prepare that if you like. Realistically, the same dependency on universe is likely true for our ubuntu1204 packages as well.

I'm guessing these are the things that shouldn't need to be backported/installed as debs in 1404 like they did in 1204? Anyway, it seems like either they need to be removed from the dependencies lists in the deb, or we need to generate them.

Just as a philosophical note here: our packages should absolutely declare their dependencies correctly. Failure to do so just makes installs harder, and we already have a couple of bugs filed to that effect (#6638, #6656).

¹ I did see a "gem: command not found" fly by which I haven't tracked down yet. I wonder if an uncaught error is causing incomplete api/workbench packages.

This affects the Ruby gem packages, which we don't use in production or recommend in our install guide. This bug is #6759.

Thanks.

Brett Smith wrote:

Tom Clegg wrote:

For test -d x || mkdir x why not just mkdir -p x?

Because mkdir -p's functional purpose is to create any directories that are necessary to make the argument exist, including parents, and that's not the desired behavior here. If the parent directory is missing, something is very wrong and we should fail.

Fair enough.

Those packages are in trusty's universe section. The trusty Docker image has the apt sources for trusty enabled, but not in apt's database. Starting your test method with apt-get update should get you a lot farther.

Ah, indeed, I missed that step, and that fixed it. Sorry, apparently I'm still not accustomed to systems where [it's as if] apt-get update has never been run. But it does make sense that docker images are like that.

I'm guessing these are the things that shouldn't need to be backported/installed as debs in 1404 like they did in 1204? Anyway, it seems like either they need to be removed from the dependencies lists in the deb, or we need to generate them.

Just as a philosophical note here: our packages should absolutely declare their dependencies correctly. Failure to do so just makes installs harder, and we already have a couple of bugs filed to that effect (#6638, #6656).

Oh, of course. I just meant "either the packages don't really need them, in which case they shouldn't be listed; or the packages do really need them, in which case we should make sure apt can find them." It didn't occur to me (but I see now) that sentence would also describe the innovative approach "let's unlist dependencies to make the packages install brokenly without complaining, like the Workbench/API packages do!"

¹ I did see a "gem: command not found" fly by which I haven't tracked down yet. I wonder if an uncaught error is causing incomplete api/workbench packages.

This affects the Ruby gem packages, which we don't use in production or recommend in our install guide. This bug is #6759.

Thanks, in that case all my questions are answered and you're good to merge.