Project

General

Profile

Actions

Bug #17765

closed

[arv-put] assumes text only input when reading from stdin

Added by Ward Vandewege almost 3 years ago. Updated almost 3 years ago.

Status:
Duplicate
Priority:
Normal
Assigned To:
-
Category:
-
Target version:
-
Story points:
-

Description

arv-put reads from standard input by default. But is seems to assume text-only input that way. Perhaps another variation of the similar bugs we saw resulting from the switch to Python 3?

$ cat some-binary-file |arv-put
2021-06-04 19:53:08 arvados.arv_put[18241] INFO: No cache usage requested for this run.
Traceback (most recent call last):
  File "/usr/bin/arv-put", line 7, in <module>
    main()
  File "/usr/share/python3/dist/python3-arvados-python-client/lib/python3.7/site-packages/arvados/commands/put.py", line 1270, in main
    trash_at=trash_at)
  File "/usr/share/python3/dist/python3-arvados-python-client/lib/python3.7/site-packages/arvados/commands/put.py", line 508, in __init__
    self._build_upload_list()
  File "/usr/share/python3/dist/python3-arvados-python-client/lib/python3.7/site-packages/arvados/commands/put.py", line 526, in _build_upload_list
    self._write_stdin(self.filename or 'stdin')
  File "/usr/share/python3/dist/python3-arvados-python-client/lib/python3.7/site-packages/arvados/commands/put.py", line 746, in _write_stdin
    self._write(sys.stdin, output)
  File "/usr/share/python3/dist/python3-arvados-python-client/lib/python3.7/site-packages/arvados/commands/put.py", line 839, in _write
    data = source_fd.read(arvados.config.KEEP_BLOCK_SIZE)
  File "/usr/share/python3/dist/python3-arvados-python-client/lib/python3.7/codecs.py", line 322, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 1: invalid start byte

Related issues

Is duplicate of Arvados - Bug #17522: [arv-put] should use binary mode when reading stdinResolvedLucas Di Pentima04/14/2021Actions
Actions

Also available in: Atom PDF