Project

General

Profile

Actions

Feature #8028

closed

[Crunch2] Dispatch containers locally

Added by Peter Amstutz over 8 years ago. Updated about 8 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Radhika Chippada
Category:
Crunch
Target version:
Story points:
2.0
Release:
Release relationship:
Auto

Description

This is a stub to support development and testing of other Crunch2 components. It isn't meant to be production-ready for this story, although in future work it may become part of a production setup and serve as a template for writing new dispatchers.

It's the "standalone worker" example described at Container dispatch, without the "determine local capacity" part: it just takes all containers that appear in the queue, and starts them immediately as child processes.

The implementation should emphasize simplicity: we don't want to load this up with features. Anything non-trivial probably needs to be done in a different layer anyway (e.g., API server or crunch-run).

Implementation

"crunch-dispatch-local", a Go program in source:services/crunch-dispatch-local/
  • The main goroutine polls the queue every 10 (configurable with "-poll-interval 5s") seconds using a time.NewTicker(). When a new container appears, pass it to a "run" goroutine.
  • The "run" goroutine starts "crunch-run" (configurable with "-crunch-run-command /usr/bin/crunch-run") and waits until [a] the crunch-run command exits, or [b] the container's priority changes to zero. If the latter happens, send TERM to the crunch-run command, and keep waiting until it exits.
  • Emit the relevant logs mentioned at Container dispatch.

It's not very efficient for each "run" goroutine to poll the API server about its own container record, but it's not necessary to optimize this right now. (Ideally, when we are ready to optimize this, we'll go straight to websocket updates rather than batching our polling for all containers that are running now, etc.)


Subtasks 1 (0 open1 closed)

Task #8140: Review branches 8028-crunch-dispatch-local in arvados and arvados-dev repositoriesResolvedRadhika Chippada01/19/2016Actions

Related issues

Related to Arvados - Feature #8128: [Crunch2] API support for crunch-dispatchResolvedTom Clegg04/28/2016Actions
Actions

Also available in: Atom PDF