Arvados

The crunch-run test suite seems to have started leaking memory in #9132. RSS climbs steadily with each test case. Leaking memory on each docker invocation wouldn't be a big problem in real life, since each crunch-run process only invokes one container. However, this could be a clue about the second leak.

Even in versions before #9132, TestWriteLogsLarge consumes multiple gigabytes of memory. The test loop itself doesn't seem to allocate a significant amount of memory. If the problem is that crunch-run tries to buffer everything up to the next newline in memory regardless of length, an errant job could (e.g., with a progress bar) cause crunch-run to allocate an arbitrary amount of memory, too.

    for i := 0; i < 2000000; i++ {
        cr.CrunchLog.Printf("Hello %d", i)
    }