Project

General

Profile

Actions

Feature #3775

closed

[Crunch] arv-crunch-job should work in local mode with an arvados-hosted repository

Added by Tom Clegg almost 8 years ago. Updated almost 8 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
Crunch
Target version:
Start date:
08/29/2014
Due date:
% Done:

100%

Estimated time:
(Total: 5.00 h)
Story points:
1.0

Description

Currently, in local mode, repository="arvados" only works if ./arvados/.git happens to exist when crunch-job runs.

New behavior:
  • You can use an arbitrary git URL in your job's "repository" attribute.
  • You can use the name of an Arvados-hosted repository. The named repo's fetch-url will be used. (This is likely to require SSH agent forwarding since the fetch-url is typically given as an SSH path.)
  • You can specify a local filesystem path starting with / or ./ or ../.
  • Compatibility issue: When running jobs in your terminal, "repository":"foo" in a job record used to mean "use ./foo or ./foo/.git". Now it means "use hosted repository called foo" just like it would in a job submitted to the API server. To get the old behavior, you can now use "repository":"./foo".
Unchanged (but relevant) behavior:
  • You can override the job record's "repository" attribute by passing a different URL/name to crunch-job as a --git-dir argument.
Caveats:
  • Remote git URLs still aren't supported when submitting jobs to the API server to run on worker nodes: there, you can (still) only use the name of a hosted repository. New features only work when using arv pipeline run --run-here or arv-crunch-job.

Subtasks 3 (0 open3 closed)

Task #4059: Accept arbitrary git url as repository if --git-dir not givenResolvedTom Clegg08/29/2014

Actions
Task #4060: Look up fetch-url of repo via arvados API if bare repo name given in repository attrResolvedTom Clegg08/29/2014

Actions
Task #4082: Review 3775-fetch-git-repoResolvedTom Clegg08/29/2014

Actions

Related issues

Related to Arvados - Bug #4048: [Crunch] arv-run-pipeline-instance --run-jobs-here cannot run jobs with local script path in script_version.ClosedTom Clegg

Actions
Actions

Also available in: Atom PDF