Bug #4957

[SDKs] arv keep put blows up when trying to put a danging symlink

Added by Ward Vandewege over 4 years ago. Updated about 4 years ago.

Status:
New
Priority:
Normal
Assigned To:
-
Category:
SDKs
Target version:
Start date:
Due date:
% Done:

0%

Estimated time:
Story points:
0.5

Description

For example:

$ arv keep put mirobot_updater
Traceback (most recent call last):
  File "/usr/local/bin/arv-put", line 4, in <module>
    main()
  File "/usr/local/lib/python2.7/dist-packages/arvados/commands/put.py", line 411, in main
    bytes_expected = expected_bytes_for(args.paths)
  File "/usr/local/lib/python2.7/dist-packages/arvados/commands/put.py", line 333, in expected_bytes_for
    bytesum += os.path.getsize(os.path.join(path, filename))
  File "/usr/lib/python2.7/genericpath.py", line 49, in getsize
    return os.stat(filename).st_size
OSError: [Errno 2] No such file or directory: 'mirobot_updater/resources/app/node_modules/.bin/serialportlist'

and

$ ls mirobot_updater/resources/app/node_modules/.bin/serialportlist -laF
lrwxrwxrwx 1 wardtest wardtest 35 Jan 10 19:41 mirobot_updater/resources/app/node_modules/.bin/serialportlist -> ../serialport/bin/serialportList.js

arv keep put should just skip things it can not save, and ideally list at the end of the run which file(s) it skipped.


Related issues

Related to Arvados - Story #3815: [SDK] arv-put supports --ignore-symlinksNew

History

#1 Updated by Ward Vandewege over 4 years ago

  • Description updated (diff)

#2 Updated by Brett Smith over 4 years ago

  • Subject changed from [SDK] arv keep put blows up when trying to put a danging symlink to [SDKs] arv keep put blows up when trying to put a danging symlink
  • Category set to SDKs
  • Target version changed from Bug Triage to Arvados Future Sprints

It does this if you specify any kind of nonexistent file, not just a dangling symlink. I assume you want nicer error reporting? There's a lot of room for improvement in all the CLI tools there—exception tracebacks are still the way most problems are presented to users.

#3 Updated by Ward Vandewege over 4 years ago

Brett Smith wrote:

It does this if you specify any kind of nonexistent file, not just a dangling symlink. I assume you want nicer error reporting? There's a lot of room for improvement in all the CLI tools there—exception tracebacks are still the way most problems are presented to users.

Yeah, I think the desired behavior would be that arv keep put skips any files it can not upload, and prints out the list of errors at the end. It shouldn't abort the upload because there's one dangling symlink in a directory tree.

#4 Updated by Tom Clegg over 4 years ago

  • Target version changed from Arvados Future Sprints to 2015-03-11 sprint

#5 Updated by Brett Smith over 4 years ago

Ward Vandewege wrote:

Yeah, I think the desired behavior would be that arv keep put skips any files it can not upload, and prints out the list of errors at the end. It shouldn't abort the upload because there's one dangling symlink in a directory tree.

  • Do you want it to skip on any OS error when it tries to open the file?
  • Do you want to adjust the behavior if there's an error while reading the file after opening?
  • What should the exit code be in this case?
  • Should there be different exit codes for the case of "couldn't upload some files" vs. "couldn't upload any files?"

#6 Updated by Peter Amstutz over 4 years ago

See also #3815

#7 Updated by Peter Amstutz over 4 years ago

  • Assigned To set to Peter Amstutz

#8 Updated by Ward Vandewege over 4 years ago

  • Target version changed from 2015-03-11 sprint to 2015-04-01 sprint

#9 Updated by Peter Amstutz over 4 years ago

  • Assigned To deleted (Peter Amstutz)

#10 Updated by Tom Clegg over 4 years ago

Perhaps rsync's behavior in the face of IO errors is a good model to follow. "Some stuff failed along the way, so exiting non-zero. But I did everything I could."

#11 Updated by Ward Vandewege over 4 years ago

  • Description updated (diff)
  • Assigned To set to Peter Amstutz

#12 Updated by Ward Vandewege over 4 years ago

Tom Clegg wrote:

Perhaps rsync's behavior in the face of IO errors is a good model to follow. "Some stuff failed along the way, so exiting non-zero. But I did everything I could."

Yeah. That's right.

#13 Updated by Ward Vandewege over 4 years ago

  • Assigned To deleted (Peter Amstutz)

#14 Updated by Ward Vandewege about 4 years ago

  • Target version changed from 2015-04-01 sprint to Arvados Future Sprints

Also available in: Atom PDF