Project

General

Profile

Actions

Feature #15250

open

a-c-r set TTL on final output collection

Added by Peter Amstutz over 5 years ago. Updated 15 days ago.

Status:
New
Priority:
Normal
Assigned To:
-
Category:
CWL
Target version:
-
Story points:
-

Description

Add feature to set the TTL on the final output collection (separate from existing feature which sets TTL on intermediate collections).


Subtasks 1 (1 open0 closed)

Task #17459: ReviewNewPeter AmstutzActions

Related issues

Related to Arvados Epics - Idea #20273: More CWL runner improvementsNewActions
Actions #1

Updated by Peter Amstutz over 5 years ago

  • Description updated (diff)
Actions #2

Updated by Peter Amstutz over 3 years ago

  • Target version changed from To Be Groomed to 2021-03-17 sprint
  • Assigned To set to Jiayong Li
Actions #3

Updated by Peter Amstutz over 3 years ago

  • Target version changed from 2021-03-17 sprint to 2021-03-31 sprint
Actions #4

Updated by Peter Amstutz over 3 years ago

  • Category set to CWL
  • Subject changed from [cwl] a-c-r set TTL on final output collection to a-c-r set TTL on final output collection
Actions #5

Updated by Peter Amstutz over 3 years ago

  • Target version changed from 2021-03-31 sprint to 2021-04-14 sprint
Actions #6

Updated by Peter Amstutz over 3 years ago

  • Target version changed from 2021-04-14 sprint to 2021-04-28 bughunt sprint
Actions #7

Updated by Peter Amstutz over 3 years ago

  • Target version deleted (2021-04-28 bughunt sprint)
Actions #8

Updated by Jiayong Li over 3 years ago

  • Status changed from New to In Progress
Actions #9

Updated by Jiayong Li over 3 years ago

1. First of all, would you like to have final output ttl set by an a-c-r argument, or an extension in the workflow cwl?
2. Following your hint, I'm trying to draw the parallel between setting intermediate output ttl (arvcontainer.py) and setting final output ttl (executor.py). In arvcontainer.py, you extract output_ttl (L268), set it in the container_request dictionary (L284), then use arvrunner.api.container_requests().update() (L309). But this looks quite different for final output: in executor.py, final output collection is made using the make_output_collection method (L421), this is not tied to a container_request. Do you think I should use the get_trash_at api call for the collection class?

Actions #10

Updated by Peter Amstutz almost 3 years ago

  • Target version set to 2021-11-10 sprint
Actions #11

Updated by Peter Amstutz almost 3 years ago

  • Related to Idea #17848: CWL runner improvements added
Actions #12

Updated by Peter Amstutz almost 3 years ago

Jiayong Li wrote:

1. First of all, would you like to have final output ttl set by an a-c-r argument, or an extension in the workflow cwl?

The user should be able to provide both on the command or using a CWL hint. That is how storage classes, intermediate output TTL, etc work.

2. Following your hint, I'm trying to draw the parallel between setting intermediate output ttl (arvcontainer.py) and setting final output ttl (executor.py). In arvcontainer.py, you extract output_ttl (L268), set it in the container_request dictionary (L284), then use arvrunner.api.container_requests().update() (L309). But this looks quite different for final output: in executor.py, final output collection is made using the make_output_collection method (L421), this is not tied to a container_request. Do you think I should use the get_trash_at api call for the collection class?

You want to add "trash_at=" to final.save_new(). trash_at takes a Python datetime object in the future.

executor.py:475

final.save_new(name=name, owner_uuid=self.project_uuid, storage_classes=storage_classes, ensure_unique_name=True, trash_at=...)
Actions #13

Updated by Peter Amstutz almost 3 years ago

  • Target version changed from 2021-11-10 sprint to 2021-11-24 sprint
Actions #14

Updated by Peter Amstutz almost 3 years ago

  • Target version changed from 2021-11-24 sprint to 2021-12-08 sprint
Actions #15

Updated by Peter Amstutz almost 3 years ago

  • Target version changed from 2021-12-08 sprint to 2022-01-05 sprint
Actions #16

Updated by Peter Amstutz over 2 years ago

  • Target version changed from 2022-01-05 sprint to 2022-01-19 sprint
Actions #17

Updated by Peter Amstutz over 2 years ago

  • Target version changed from 2022-01-19 sprint to 2022-02-02 sprint
Actions #18

Updated by Peter Amstutz over 2 years ago

  • Target version changed from 2022-02-02 sprint to 2022-02-16 sprint
Actions #19

Updated by Peter Amstutz over 2 years ago

  • Target version changed from 2022-02-16 sprint to 2022-03-02 sprint
Actions #20

Updated by Peter Amstutz over 2 years ago

  • Target version deleted (2022-03-02 sprint)
Actions #21

Updated by Peter Amstutz almost 2 years ago

  • Target version set to 2022-11-09 sprint
Actions #22

Updated by Peter Amstutz almost 2 years ago

  • Target version changed from 2022-11-09 sprint to 2022-11-23 sprint
Actions #23

Updated by Peter Amstutz almost 2 years ago

  • Target version changed from 2022-11-23 sprint to 2022-12-07 Sprint
Actions #24

Updated by Peter Amstutz almost 2 years ago

  • Target version changed from 2022-12-07 Sprint to 2022-12-21 Sprint
Actions #25

Updated by Peter Amstutz almost 2 years ago

  • Target version changed from 2022-12-21 Sprint to 2023-01-18 sprint
Actions #26

Updated by Peter Amstutz almost 2 years ago

  • Target version changed from 2023-01-18 sprint to 2023-02-01 sprint
Actions #27

Updated by Peter Amstutz over 1 year ago

  • Target version changed from 2023-02-01 sprint to Future
Actions #28

Updated by Peter Amstutz over 1 year ago

  • Status changed from In Progress to New
Actions #29

Updated by Peter Amstutz over 1 year ago

  • Related to Idea #20273: More CWL runner improvements added
Actions #30

Updated by Peter Amstutz over 1 year ago

  • Related to deleted (Idea #17848: CWL runner improvements)
Actions #32

Updated by Brett Smith 15 days ago

  • Target version deleted (Future)
Actions #33

Updated by Brett Smith 15 days ago

  • Assigned To deleted (Jiayong Li)
Actions

Also available in: Atom PDF