Bug #19872
Updated by Peter Amstutz about 2 years ago
Running a test case with running "zcat" on a bunch of gzipped fastq files, it is getting the following error after a while:
2022-12-12 17:12:52 arvados.api[8871] DEBUG: [req-49kou2g0vgw59vt7dbt1] Retrying API request in 4 s after socket error
Traceback (most recent call last):
File "/opt/rh/rh-python38/root/usr/local/lib/python3.8/site-packages/arvados_python_client-2.5.0.dev20221202182828-py3.8.egg/arvados/api.py", line 88, in _intercept_http_request
File "/opt/rh/rh-python38/root/usr/local/lib/python3.8/site-packages/httplib2-0.20.1-py3.8.egg/httplib2/__init__.py", line 1711, in request
File "/opt/rh/rh-python38/root/usr/local/lib/python3.8/site-packages/httplib2-0.20.1-py3.8.egg/httplib2/__init__.py", line 1427, in _request
File "/opt/rh/rh-python38/root/usr/local/lib/python3.8/site-packages/httplib2-0.20.1-py3.8.egg/httplib2/__init__.py", line 1349, in _conn_request
File "/opt/rh/rh-python38/root/usr/local/lib/python3.8/site-packages/httplib2-0.20.1-py3.8.egg/httplib2/__init__.py", line 1125, in connect
File "/opt/rh/rh-python38/root/usr/lib64/python3.8/socket.py", line 918, in getaddrinfo
OSError: [Errno 24] Too many open files
The most likely explanation is that Python isn't garbage collecting the mmap'd keep cache blocks as expected -- need to investigate.