Project

General

Profile

Actions

Feature #21889

open

Periodic cron-like scheduler to run workflows automatically

Added by Peter Amstutz 4 months ago. Updated 5 days ago.

Status:
New
Priority:
Normal
Assigned To:
-
Category:
Crunch
Target version:
Story points:
-

Description

I don't know what component should be responsible for this, but I find myself wanting a way to schedule periodic tasks.

For my use case, I want to run automatic housekeeping and reporting tasks. For example, running a daily/weekly cluster usage report which writes an HTML report to a collection (see #21888)

I could envision it used for tasks such as automating data transfer to/from the cluster and chaining tasks from that (such as running analysis).

Should discuss if it makes sense to cobble something together in the short term (a smallish task-specific python script running on the shell node) and scale it out.

Alternately, we could add a way to schedule containers so that they are not ready to run until a specific time has passed, this would allow tasks to run and then re-schedule themselves.

Actions #1

Updated by Peter Amstutz 4 months ago

  • Description updated (diff)
Actions #3

Updated by Peter Amstutz 5 days ago

  • Description updated (diff)
Actions #4

Updated by Peter Amstutz 5 days ago

  • Category changed from CWL to Crunch
Actions

Also available in: Atom PDF