Bug #11583
Updated by Tom Clegg almost 7 years ago
The crunch-run test suite seems to have started leaking memory in #9132. RSS climbs steadily with each test case. Leaking memory on each docker invocation wouldn't be a big problem in real life, since each crunch-run process only invokes one container. However, this could be a clue about the second leak.
Even in versions before #9132, TestWriteLogsLarge consumes multiple gigabytes of memory. The test loop itself doesn't seem to allocate a significant amount of memory. If the problem is that crunch-run tries to buffer everything up to the next newline in memory regardless of length, an errant job could (e.g., with a progress bar) cause crunch-run to allocate an arbitrary amount of memory, too.
<pre>
for i := 0; i < 2000000; i++ {
cr.CrunchLog.Printf("Hello %d", i)
}
</pre>