Feature #3137

Write bytes read and network performance stats to arv-mount logs

Added by Peter Amstutz about 5 years ago. Updated almost 4 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
FUSE
Target version:
Start date:
11/18/2015
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)
Story points:
1.0

Description

Toggle with a --crunchstat-interval argument (default 0 means don't output stats).

Use the same format as crunchstat, so the results show up in Workbench graphs. Example:

crunchstat: blkio:0:0 123456 write 1234560 read -- interval 10.0002 seconds 0 write 0 read
crunchstat: net:keep0 1546380901 tx 3166208 rx -- interval 10.0002 seconds 1546352010 tx 2066527 rx

blkio:0:0 indicates filesystem traffic between userspace programs and arv-mount.

net:keep0 indicates network traffic between arv-mount and keep servers.

Additionally, it would be helpful to know the number of IO operations on both sides, i.e., number of Keep block reads/writes and number of userspace read() and write() calls serviced. (Formatting TBD?)


Subtasks

Task #7788: Review 3137-arv-mount-statsResolvedPeter Amstutz

Task #7795: Add statsResolvedPeter Amstutz


Related issues

Related to Arvados - Bug #7747: [FUSE] arv-mount is too slow for strelkaNew11/10/2015

Associated revisions

Revision d6f163ed
Added by Peter Amstutz almost 4 years ago

Merge branch '3137-arv-mount-stats' closes #3137

History

#1 Updated by Peter Amstutz about 5 years ago

  • Subject changed from arv-mount logs bytes read for stats to Write bytes read and network performance stats to arv-mount logs

#2 Updated by Tom Clegg almost 4 years ago

  • Target version set to 2015-12-02 sprint

#3 Updated by Tom Clegg almost 4 years ago

  • Description updated (diff)
  • Category set to FUSE
  • Story points set to 1.0

#4 Updated by Tom Clegg almost 4 years ago

  • Description updated (diff)

#5 Updated by Brett Smith almost 4 years ago

  • Assigned To set to Peter Amstutz

#6 Updated by Tom Clegg almost 4 years ago

  • Description updated (diff)

#7 Updated by Brett Smith almost 4 years ago

Reviewing db55813

Heads up, arvados_fuse needs its dependency on arvados-python-client bumped to get arvados.keep.Counter.

Beyond that, I just have a couple of naming suggestions, nothing to block a merge:

  • In the Stat class, the relationship between "egress" and "egr" is not clear from the names. Ditto "ingress" and "ing." Would better names help clarify that one is a specific value in time from the other?
    Possibly related: there doesn't seem to be any need to hold onto "egr_prev" and "ing_prev" as instance variables. They only function as local variables to calculate the interval value in the update method.
  • In the Counter class, "lk" seems like a sort of gratuitous abbreviation of "lock".

Thanks.

#8 Updated by Peter Amstutz almost 4 years ago

  • Status changed from New to Resolved
  • % Done changed from 50 to 100

Applied in changeset arvados|commit:d6f163ed99c6ae234c551bb3ecd23c7af8610c8f.

Also available in: Atom PDF