Story #7832

[FUSE] [SDKs] Define __slots__ to improve memory utilization

Added by Brett Smith about 4 years ago. Updated about 4 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
SDKs
Target version:
Start date:
11/20/2015
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)
Story points:
0.5

Description

Both arv-get and FUSE use a lot of memory. Idea: define __slots__ for classes where it will noticeably improve memory utilization.

First class candidates: objects that are both instantiated commonly and long-lived. The Collection classes, File classes, Range, and LocatorAndRange in the SDK. The Directory and File classes in FUSE.

Second class candidates: objects that are instantiated commonly but with shorter lives. KeepService, ThreadLimiter, RetryLoop.


Subtasks

Task #7867: Review 7832-pysdk-use-slotsResolvedPeter Amstutz

Associated revisions

Revision f486405f
Added by Peter Amstutz about 4 years ago

Merge branch '7832-pysdk-use-slots' refs #7832

History

#1 Updated by Brett Smith about 4 years ago

  • Target version set to Arvados Future Sprints

#2 Updated by Peter Amstutz about 4 years ago

7832-pysdk-use-slots

Adds slots to Range, LocatorAndRange, and CacheSlot. These are the obvious low hanging fruit, and together they reduce memory usage by half on a very complex manifest (215dd32873bfa002aa0387c6794e4b2c+54081534)

master:

USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
peter    21943 10.4  9.3 1312088 1139036 pts/3 Sl+  16:58   0:36 python get.py

7832-pysdk-use-slots

USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
peter    26853 16.5  4.9 781648 608492 pts/3   Sl+  17:04   0:37 python get.py

#3 Updated by Peter Amstutz about 4 years ago

  • Target version changed from Arvados Future Sprints to 2015-12-02 sprint

#4 Updated by Peter Amstutz about 4 years ago

  • Status changed from New to In Progress
  • Story points set to 0.5

#5 Updated by Peter Amstutz about 4 years ago

  • Assigned To set to Peter Amstutz

#6 Updated by Tom Clegg about 4 years ago

LGTM @ 9045a08

#7 Updated by Peter Amstutz about 4 years ago

  • Status changed from In Progress to Resolved

Also available in: Atom PDF