Project

General

Profile

Actions

Bug #10629

closed

[FUSE] slow enumerating files by collection uuid

Added by Tom Morris over 7 years ago. Updated about 7 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
Performance
Target version:
Story points:
1.0

Description

It takes 38 minutes (!!) to enumerate 242K files. During this time the arv-mount process is pegged at 100% CPU and no network traffic is being done. The manifest text totals just 9MB (9243914 char), so it's taking over 4 min / MB to parse this.

$ time find keep/by_id/e51c5-4zz18-l3dq8bw20uwz0qd -print | wc -l
241751
real 38m0.969s
user 0m0.224s
sys 0m0.300s

$ wc *.manifest
2497 252893 9243914 e51c5-4zz18-l3dq8bw20uwz0qd.manifest


Subtasks 1 (0 open1 closed)

Task #11114: Review 10629-fuse-listing-perfResolvedPeter Amstutz11/24/2016Actions

Related issues

Related to Arvados - Bug #6019: [FUSE] `ls` in a large home directory (4400+ items) takes too longClosed05/13/2015Actions
Related to Arvados - Bug #9732: [FUSE] performance issues stat() in pythonNewActions
Actions

Also available in: Atom PDF