[SDKs] `arv copy` does not adjust component filters, causing unpredictable runs
A user writes a pipeline template following our Git strategy for pipeline development. They set filters to determine what jobs are eligible for reuse, including a repository filter.
Another user uses
arv copy to copy this template somewhere else. The destination Git repository has a different name. However,
arv copy does not adjust the filters.
At this point, the pipeline template won't run as intended on the destination cluster. Best case, there is no Git repository on the destination cluster with the same name as the source repository, so the run fails almost immediately. Worst case, there is a Git repository with the same name on the destination cluster, and then something unpredictable happens.
#3 Updated by Brett Smith over 5 years ago
arv copy should do sanity checks on the filters early on.
We should detect the common case of "repository is the component's repository, and script_version is a commit hash," and do the translation correctly.
If it detects filters that it can't translate reliably (they refer to another repository, they have a symbolic script_version filter that isn't being copied, something else), it should immediately abort, let the user know what the problem is, and offer the option to do the copy anyway with some flag (
If the user uses the
--force flag, copy everything verbatim. The user is expected to fix them on the destination cluster.
#5 Updated by Brett Smith over 5 years ago
(10:06:47 AM) Me: tomclegg: https://dev.arvados.org/issues/9049 suggests having `arv-copy --force` mean "copy a pipeline template's filters verbatim, even if they're not meaningful on the destination." arv-copy already has --force; it means "upload a collection to destination Keep even if it apparently already exists."
(10:07:26 AM) Me: --force-filters? Maybe we can follow the pattern set by apt, and have a whole family of --force-foo options, and then eventually a --force-all.
(10:07:47 AM) tomclegg: --force-filters ... yeah.
(10:08:01 AM) tomclegg: brett: sgtm.