Bug #9049
closed
[SDKs] `arv copy` does not adjust component filters, causing unpredictable runs
Added by Brett Smith almost 8 years ago.
Updated almost 8 years ago.
Description
A user writes a pipeline template following our Git strategy for pipeline development. They set filters to determine what jobs are eligible for reuse, including a repository filter.
Another user uses arv copy to copy this template somewhere else. The destination Git repository has a different name. However, arv copy does not adjust the filters.
At this point, the pipeline template won't run as intended on the destination cluster. Best case, there is no Git repository on the destination cluster with the same name as the source repository, so the run fails almost immediately. Worst case, there is a Git repository with the same name on the destination cluster, and then something unpredictable happens.
- Subject changed from [SDKs] `arv copy` does not adjust script_filters to [SDKs] `arv copy` does not adjust component filters, causing unpredictable runs
- Description updated (diff)
arv copy should do sanity checks on the filters early on.
We should detect the common case of "repository is the component's repository, and script_version is a commit hash," and do the translation correctly.
If it detects filters that it can't translate reliably (they refer to another repository, they have a symbolic script_version filter that isn't being copied, something else), it should immediately abort, let the user know what the problem is, and offer the option to do the copy anyway with some flag (--force?).
If the user uses the --force flag, copy everything verbatim. The user is expected to fix them on the destination cluster.
- Status changed from New to In Progress
- Assigned To set to Brett Smith
(10:06:47 AM) Me: tomclegg: https://dev.arvados.org/issues/9049 suggests having `arv-copy --force` mean "copy a pipeline template's filters verbatim, even if they're not meaningful on the destination." arv-copy already has --force; it means "upload a collection to destination Keep even if it apparently already exists."
(10:07:26 AM) Me: --force-filters? Maybe we can follow the pattern set by apt, and have a whole family of --force-foo options, and then eventually a --force-all.
(10:07:47 AM) tomclegg: --force-filters ... yeah.
(10:08:01 AM) tomclegg: brett: sgtm.
- Target version set to 2016-05-25 sprint
test 227b7d4fe9e0c65d9e2148af5c10632c60271822
I read the code and learned about contextlib.contextmanager. that was new to me.
a minor change is that I think functools isn't being used
all the rest LGTM for merge into master.
- Status changed from In Progress to Resolved
- % Done changed from 0 to 100
Applied in changeset arvados|commit:2a40242e7f47841c02b4eb9f23a9fafc230b37c4.
Also available in: Atom
PDF
Updated by 
Updated by