Bug #11547
Updated by Peter Amstutz almost 7 years ago
arvados-cwl-runner created a container request that was about 75 MB of JSON. This caused huge memory bloat in the API server, paralyzed requests to both the containers and container_requests table (details on #11546), and resulted in memory bloat of other services as well, such as crunch-dispatch-slurm:
<pre>
$ ps axv | grep crunch-dispatch-slurm
PID TTY STAT TIME MAJFL TRS DRS RSS %MEM COMMAND
43366 ? Sl 34:03 2248 0 3676584 2918600 10.1 /usr/bin/crunch-dispatch-slurm
</pre>
arvados-cwl-runner shouldn't produce huge container requests, but the API server shouldn't accept them, either. Add a configurable limit on the size of serialization fields.
Suggest setting the default limit to 4 MB (with testing to ensure that performance is stil acceptable with 4 MB of JSON).