Idea #7832: [FUSE] [SDKs] Define __slots__ to improve memory utilization - Arvados

Actions

Copy link

Idea #7832

closed

[FUSE] [SDKs] Define slots to improve memory utilization

Added by Brett Smith over 8 years ago. Updated over 8 years ago.

Status:

Resolved

Priority:

Normal

Assigned To:

Peter Amstutz

Category:

SDKs

Target version:

2015-12-02 sprint

Story points:

0.5

Description

Both arv-get and FUSE use a lot of memory. Idea: define __slots__ for classes where it will noticeably improve memory utilization.

First class candidates: objects that are both instantiated commonly and long-lived. The Collection classes, File classes, Range, and LocatorAndRange in the SDK. The Directory and File classes in FUSE.

Second class candidates: objects that are instantiated commonly but with shorter lives. KeepService, ThreadLimiter, RetryLoop.

Subtasks 1 (0 open — 1 closed)

Actions

Copy link

Updated by Brett Smith over 8 years ago

Target version set to Arvados Future Sprints

Actions

Copy link

Updated by Peter Amstutz over 8 years ago

7832-pysdk-use-slots

Adds slots to Range, LocatorAndRange, and CacheSlot. These are the obvious low hanging fruit, and together they reduce memory usage by half on a very complex manifest (215dd32873bfa002aa0387c6794e4b2c+54081534)

master:

USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
peter    21943 10.4  9.3 1312088 1139036 pts/3 Sl+  16:58   0:36 python get.py

7832-pysdk-use-slots

USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
peter    26853 16.5  4.9 781648 608492 pts/3   Sl+  17:04   0:37 python get.py

Actions

Copy link