Story #16264

Handle R SDK dependencies better

Added by Peter Amstutz 14 days ago. Updated 8 days ago.

Status:
New
Priority:
Normal
Assigned To:
-
Category:
SDKs
Target version:
Start date:
Due date:
% Done:

0%

Estimated time:
Story points:
-

Description

Our build is currently broken because Rcpp (a transitive dependency of a package we depend on) had a new release that is broken on R 3.3.

We are running our tests on Debian 9, which ships with R 3.3.

A fix is already committed to upstream Rcpp, so it is likely the problem will resolve itself, but we should have a more stable strategy for R dependencies.

Recompiling R dependencies every time is also very expensive.

Options

Use Debian packages

When we did the R SDK originally (about two years ago) most of the packages we needed were not in Debian. Since then, it appears they have been added to Debian 10, and are also available in a Debian 9 backport:

https://packages.debian.org/search?keywords=r-cran-devtools&searchon=names&suite=all&section=all

Of course this requires targeting Debian for the test builds, but we do that already. To get the maximum benefit it also requires that all the R packages we need are in Debian.

Pinned install

R doesn't have anything like a Gemfile.lock or "pip freeze". However someone has created a tool that computes an install plan so each package is installed at exactly each version we want.

https://www.ccdatalab.org/blog/2019/8/9/pinning-transitive-r-dependencies-for-fun-and-reproducible-builds

This works on any R install. The drawback is that we still end up compiling the whole world every time.


Related issues

Related to Arvados - Bug #16267: arvbox and jenkins images updated to use arvados-server installNew

Related to Arvados - Story #16053: [boot] subcommand to install/update dev and runtime dependenciesIn Progress03/31/2020

History

#1 Updated by Peter Amstutz 14 days ago

  • Description updated (diff)

#2 Updated by Peter Amstutz 13 days ago

  • Related to Bug #16267: arvbox and jenkins images updated to use arvados-server install added

#3 Updated by Peter Amstutz 13 days ago

Currently I think the solution is for #16053 to include installing the appropriate R packages from Debian 10.

#4 Updated by Peter Amstutz 13 days ago

  • Related to Story #16053: [boot] subcommand to install/update dev and runtime dependencies added

#5 Updated by Peter Amstutz 8 days ago

  • Category set to SDKs

#6 Updated by Peter Amstutz 8 days ago

  • Target version changed from 2020-04-08 Sprint to 2020-04-22

Also available in: Atom PDF