Bug #12465

[crunchv2] Improve crunch-run environment reporting

Added by Peter Amstutz about 1 year ago. Updated about 1 year ago.

Status:
New
Priority:
Normal
Assigned To:
-
Category:
-
Target version:
Start date:
Due date:
% Done:

0%

Estimated time:
Story points:
-

Description

Crunch-run should log its environment earlier in the startup process (before loading the Docker image or creating arv-mount)

It should also log some additional information:

  • crunch-run version
  • Docker version
  • arv-mount version
  • cloud instance id (/var/tmp/arv-node-data/meta-data/instance-id)
  • cloud instance type (/var/tmp/arv-node-data/meta-data/instance-type)

Related issues

Related to Arvados - Bug #12933: [crunch2] add equivalent of cloud_node lineResolved2018-01-11

History

#1 Updated by Peter Amstutz about 1 year ago

  • Description updated (diff)

#2 Updated by Peter Amstutz about 1 year ago

  • Description updated (diff)

#3 Updated by Tom Morris about 1 year ago

  • Target version set to To Be Groomed

Crunch v1 logs the following information:

 runtime_constraints {"min_ram_mb_per_node":10000,"max_tasks_per_node":0,"min_cores_per_node":2,"docker_image":"foo/bar","min_scratch_mb_per_node":51024}
 compute146 clustr-7ekkf-fe12d0nlvw3xa4b {"total_scratch_mb":102345,"total_ram_mb":14011,"total_cpu_cores":2,"cloud_node":{"price":0.149,"size":"Standard_D11_v2"}}

We should make sure that the Crunch v2 information is a superset of that (and Ideally in a compatible format)

#4 Updated by Tom Morris about 1 year ago

While we're at it, we should fix the formatting of the existing logging. For example, the missing spaces after the "node-info" prefixes here:

2017-10-26T15:11:40.858866900Z Docker response: {"stream":"Loaded image ID: sha256:bf3a9bd0230a9f581d5a5c8f7816a529e9b84da644766b79bd926b7594fd52e4\n"}
2017-10-26T15:12:07.853864200Z Creating Docker container
node-infoHost Information
node-info  Linux compute26.su92l.arvadosapi.com 4.10.0-37-generic #41~16.04.1-Ubuntu SMP Fri Oct 6 22:42:59 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
node-info  
node-infoCPU Information

#5 Updated by Tom Morris about 1 year ago

  • Target version changed from To Be Groomed to 2017-12-20 Sprint

#6 Updated by Peter Amstutz about 1 year ago

  • Assigned To set to Peter Amstutz

#7 Updated by Peter Amstutz about 1 year ago

  • Assigned To deleted (Peter Amstutz)
  • Target version changed from 2017-12-20 Sprint to Arvados Future Sprints

#8 Updated by Peter Amstutz 11 months ago

  • Related to Bug #12933: [crunch2] add equivalent of cloud_node line added

Also available in: Atom PDF