[SDKs] support text-mode open() in Python 3
#13 Updated by Tom Clegg almost 3 years ago
This change mostly comes down to wrapping the ArvadosFileReader/Writer in an io.BufferedReader/Random and then in an io.TextIOWrapper, if text mode is requested.With the previous code, the "block boundary isn't on a character boundary" test crashed in Python 2 when using text mode, and the same wrapper fixes it, so I figured it would make sense to fix both at once. However, this does cause a slight API change that could conceivably break existing code. In text mode, the file-like object returned by Collection.open()...
- ...is not an ArvadosFileReader/Writer. Code that tests
isinstance(f, ...)will behave differently. I found no examples of this in our own repo.
- ...cannot seek to arbitrary positions (see TextIOBase seek restrictions). The only occurrence of this I found in our repo was in the "open in text mode in Python 2" test.
- ...does not accept close(flush=False). This revealed (via sdk/cwl test failures) that source:sdk/python/arvados/commands/run.py was accidentally using text mode to copy arbitrary files.
#14 Updated by Tom Clegg almost 3 years ago
This broke some things: https://ci.curoverse.com/job/developer-run-tests-sdk-python-ruby/275/console
#16 Updated by Tom Clegg almost 3 years ago
#19 Updated by Peter Amstutz almost 3 years ago
This has a regression:
2018-12-14 20:28:39 arvados.cwl-runner ERROR: [step-valuefrom5-wf.cwl] While getting final output object: 'TextIOWrapper' object has no attribute 'size' Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/arvados_cwl/runner.py", line 465, in done if f.size() > 0: AttributeError: 'TextIOWrapper' object has no attribute 'size'