Bug #12748

[crunch2] crunchstat-summary doesn't report elapsed time for crunch2 jobs

Added by Tom Morris about 2 years ago. Updated about 1 year ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
-
Target version:
Start date:
10/04/2018
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)
Story points:
-
Release:
Release relationship:
Auto

Description

From examination, the elapsed time calculation is on a crunch1-only code path. This also means that average CPU utilization doesn't get reported since it depends on the elapsed time.


Subtasks

Task #12770: Review 12690-12748-crunchstat-summary ResolvedTom Clegg

Associated revisions

Revision 04b9d6e9
Added by Peter Amstutz about 1 year ago

Merge branch '12690-12748-crunchstat-summary'

refs #12690
refs #12748
refs #13759

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <>

History

#1 Updated by Tom Morris about 2 years ago

  • Status changed from New to In Progress
  • Assigned To set to Tom Morris

I've implemented a stop gap using the existing data sources in 42017ce7d, but it may be better to handle this differently for crunch2 by using the first and last timestamps of the crunch-run.txt file

#2 Updated by Tom Morris about 2 years ago

  • Target version changed from 2017-12-06 Sprint to 2017-12-20 Sprint

#3 Updated by Tom Morris almost 2 years ago

  • Target version changed from 2017-12-20 Sprint to 2018-01-17 Sprint

#4 Updated by Tom Morris almost 2 years ago

Branch containing the commit is 12690-container-request-keep-stats.

Problem confirmed in actual crunch2 outputs.

#5 Updated by Tom Morris almost 2 years ago

  • Target version changed from 2018-01-17 Sprint to 2018-01-31 Sprint

#6 Updated by Tom Morris almost 2 years ago

  • Target version changed from 2018-01-31 Sprint to 2018-02-14 Sprint

#7 Updated by Tom Morris almost 2 years ago

  • Target version changed from 2018-02-14 Sprint to 2018-02-28 Sprint

#8 Updated by Tom Morris almost 2 years ago

  • Target version changed from 2018-02-28 Sprint to 2018-03-14 Sprint

#9 Updated by Tom Morris over 1 year ago

  • Target version changed from 2018-03-14 Sprint to 2018-03-28 Sprint

#10 Updated by Tom Morris over 1 year ago

  • Target version changed from 2018-03-28 Sprint to 2018-04-11 Sprint

#11 Updated by Tom Morris over 1 year ago

  • Target version changed from 2018-04-11 Sprint to 2018-04-25 Sprint

#12 Updated by Tom Morris over 1 year ago

  • Target version changed from 2018-04-25 Sprint to 2018-05-09 Sprint

#13 Updated by Tom Morris over 1 year ago

  • Target version changed from 2018-05-09 Sprint to 2018-05-23 Sprint

#14 Updated by Tom Morris over 1 year ago

  • Target version changed from 2018-05-23 Sprint to 2018-06-06 Sprint

#15 Updated by Tom Morris over 1 year ago

  • Target version changed from 2018-06-06 Sprint to 2018-06-20 Sprint

#16 Updated by Tom Morris over 1 year ago

  • Target version changed from 2018-06-20 Sprint to 2018-07-03 Sprint

#17 Updated by Tom Morris over 1 year ago

  • Target version changed from 2018-07-03 Sprint to 2018-07-18 Sprint

#18 Updated by Tom Morris over 1 year ago

  • Target version changed from 2018-07-18 Sprint to 2018-08-01 Sprint

#19 Updated by Tom Morris over 1 year ago

  • Target version changed from 2018-08-01 Sprint to 2018-08-15 Sprint

#20 Updated by Tom Morris over 1 year ago

  • Target version changed from 2018-08-15 Sprint to 2018-09-05 Sprint

#21 Updated by Peter Amstutz over 1 year ago

The elapsed time is wildly wrong, it should probably be 0 (the actual runtime is less than 10 seconds and doesn't span a full collection interval.

### Summary for extract-metrics (e51c5-dz642-mtnlzmwjjyryis1)
category    metric    task_max    task_max_rate    job_total
blkio:0:0    read    0    0    0
blkio:0:0    write    0    0    0
cpu    cpus    2    -    -
cpu    sys    0    -    0.00
cpu    user    0    -    0.00
cpu    user+sys    0    -    0.00
fuseops    read    0    0    0
fuseops    write    0    0    0
keepcache    hit    0    0    0
keepcache    miss    0    0    0
keepcalls    get    0    0    0
keepcalls    put    0    0    0
mem    cache    0    -    -
mem    pgmajfault    0    -    0
mem    rss    585728    -    -
net:keep0    rx    0    0    0
net:keep0    tx    0    0    0
net:keep0    tx+rx    0    0    0
time    elapsed    86398    -    86398
# Number of tasks: 1
# Max CPU time spent by a single task: 0s
# Max CPU usage in a single interval: 0%
# Overall CPU usage: 0.00%
# Max memory used by a single task: 0.00GB
# Max network traffic in a single task: 0.00GB
# Max network speed in a single interval: 0.00MB/s
# Keep cache miss rate 0.00%
# Keep cache utilization 0.00%
#!! extract-metrics max RSS was 1 MiB -- try runtime_constraints "ram":1020054732

#22 Updated by Tom Morris over 1 year ago

  • Target version changed from 2018-09-05 Sprint to 2018-09-19 Sprint

#23 Updated by Tom Morris about 1 year ago

  • Target version changed from 2018-09-19 Sprint to 2018-10-03 Sprint

#24 Updated by Peter Amstutz about 1 year ago

  • Assigned To changed from Tom Morris to Peter Amstutz
  • Target version changed from 2018-10-03 Sprint to 2018-10-17 sprint

#26 Updated by Peter Amstutz about 1 year ago

This covers tickets #12690, #12748 and #13759

#27 Updated by Tom Clegg about 1 year ago

LGTM, thanks

#28 Updated by Peter Amstutz about 1 year ago

  • Status changed from In Progress to Resolved

#29 Updated by Tom Morris about 1 year ago

  • Release set to 14

Also available in: Atom PDF