Project

General

Profile

Feature #12746

Updated by Tom Morris about 6 years ago

It turns out that the missing net:eth0 stats are due to an architectural change between crunch1 This job: https://workbench.qr1hi.arvadosapi.com/containers/qr1hi-dz642-fu1ewicplzfht71#Log 
 ran 5 minutes and crunch2.  

 In crunch1 these recorded lots of instances of cpu & memory stats were useful for monitoring the I/O bandwidth during the upload phase when arv-put was run in the container to do the upload. crunchstat.txt, but no net stats. 
 In crunch2, the uploads are done outside the container by the new crunch2 crunch-run component which doesn't log any stats during the upload 

 Crunch2 needs to provide equivalent stats to Crunch1 during the upload phase. The I've seen other jobs where network bandwidth stats are the most important, but memory and CPU stats would be useful as well. reported, so they're not missing altogether -- just unreliable. 

 It would also be useful to have these stats during the Docker image download phase at the beginning of the job. 

 https://collections.qr1hi.arvadosapi.com/c=5dd066573faaf3eea0e9abf580b24c27-418/_/crunchstat.txt?disposition=inline 

Back