Bug #17765

[arv-put] assumes text only input when reading from stdin

Added by Ward Vandewege 12 months ago. Updated 12 months ago.

Status:
Duplicate
Priority:
Normal
Assigned To:
-
Category:
-
Target version:
-
Start date:
Due date:
% Done:

0%

Estimated time:
Story points:
-

Description

arv-put reads from standard input by default. But is seems to assume text-only input that way. Perhaps another variation of the similar bugs we saw resulting from the switch to Python 3?

$ cat some-binary-file |arv-put
2021-06-04 19:53:08 arvados.arv_put[18241] INFO: No cache usage requested for this run.
Traceback (most recent call last):
  File "/usr/bin/arv-put", line 7, in <module>
    main()
  File "/usr/share/python3/dist/python3-arvados-python-client/lib/python3.7/site-packages/arvados/commands/put.py", line 1270, in main
    trash_at=trash_at)
  File "/usr/share/python3/dist/python3-arvados-python-client/lib/python3.7/site-packages/arvados/commands/put.py", line 508, in __init__
    self._build_upload_list()
  File "/usr/share/python3/dist/python3-arvados-python-client/lib/python3.7/site-packages/arvados/commands/put.py", line 526, in _build_upload_list
    self._write_stdin(self.filename or 'stdin')
  File "/usr/share/python3/dist/python3-arvados-python-client/lib/python3.7/site-packages/arvados/commands/put.py", line 746, in _write_stdin
    self._write(sys.stdin, output)
  File "/usr/share/python3/dist/python3-arvados-python-client/lib/python3.7/site-packages/arvados/commands/put.py", line 839, in _write
    data = source_fd.read(arvados.config.KEEP_BLOCK_SIZE)
  File "/usr/share/python3/dist/python3-arvados-python-client/lib/python3.7/codecs.py", line 322, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 1: invalid start byte

Related issues

Is duplicate of Arvados - Bug #17522: [arv-put] should use binary mode when reading stdinResolved04/14/2021

History

#1 Updated by Ward Vandewege 12 months ago

  • Description updated (diff)

#2 Updated by Ward Vandewege 12 months ago

  • Description updated (diff)
  • Subject changed from [arv-put] assumes non-binary input when reading from stdin to [arv-put] assumes text only input when reading from stdin

#3 Updated by Tom Clegg 12 months ago

  • Is duplicate of Bug #17522: [arv-put] should use binary mode when reading stdin added

#4 Updated by Tom Clegg 12 months ago

  • Target version deleted (To Be Groomed)
  • Status changed from New to Duplicate

#5 Updated by Ward Vandewege 12 months ago

This was already fixed as part of #17522, the 2.2.0 release no longer has this bug.

Also available in: Atom PDF