Bug #6997
Updated by Brett Smith over 8 years ago
h2. Steps to reproduce # Start keepstore[1]. # Write 3K small files[2] (so the index isn't trivially small). # Hit keepstore with 15 concurrent 40MiB writes[3] (so it allocates 15 data buffers). Repeat, just to be sure. # Hit keepstore with 20 concurrent /index requests[4]. Repeat a few times. |*Runtime*|*Build*|*Env*|*After writes*|*After many indexes*| |go1.3.3|0d9da68||1.8G|*3.7G*| |go1.3.3|0d9da68|GOGC=10|2.1G|2.3G| |go1.3.3|0d9da68|GOMAXPROCS=8|2.4G|*>4.5G*| |go1.3.3|0d9da68|GOMAXPROCS=8 GOGC=10|2.4G|2.6G| |go1.4.2|0d9da68|GOMAXPROCS=8|2.3G|*3.5G*| |go1.4.2|0d9da68|GOMAXPROCS=8 GOGC=10|2.1G|2.4G| Evidently, "index" generates a lot of garbage, and the default GOGC=100 is much too high for a program like keepstore that normally fills >50% of the system's RAM with active buffers. See http://golang.org/pkg/runtime/debug/#SetGCPercent fn1. @mkdir /tmp/zzz; echo abc | GOMAXPROCS=8 keepstore -volume=/tmp/zzz -data-manager-token-file=/dev/stdin@ fn2. @for i in `seq 0 3000`; do echo -n "$i "; curl -X PUT -H 'Authorization: OAuth2 abc' --data-binary $i localhost:25107/`echo -n $i | md5sum | head -c32`; done@ fn3. <code>dd if=/dev/zero of=/tmp/zero bs=1000000 count=40; for i in `seq 0 15`; do curl -X PUT -H 'Authorization: OAuth2 foo' --data-binary @/tmp/zero localhost:25107/48e9a108a3ec623652e7988af2f88867 & done</code> fn4. @for i in `seq 0 20`; do curl -s -H 'Authorization: OAuth2 abc' localhost:25107/index >/dev/null & done@ h2. Fix Specify GOGC=10 in the keepstore run script, both in our own deployments and the Keepstore install guide. script.