Bug #6149

[Crunch] Install step logs error message from `setup.py egg_info`

Added by Bryan Cosca over 6 years ago. Updated over 6 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Brett Smith
Category:
Crunch
Target version:
Start date:
05/26/2015
Due date:
% Done:

100%

Estimated time:
(Total: 1.00 h)
Story points:
0.5

Description

In https://cloud.curoverse.com/collections/fefe083539813d3a01108c90fa684f7c+85/qr1hi-8i9sb-74468yxq0cauxbw.log.txt

I came across the error message:

stderr starting: ['srun','--nodelist=compute6','-D','/tmp','--job-name=qr1hi-8i9sb-74468yxq0cauxbw','sh','-c','mkdir -p /tmp/crunch-job/opt && cd /tmp/crunch-job && perl -']
2015-05-26_19:51:23 qr1hi-8i9sb-74468yxq0cauxbw 3908 stderr fatal: Not a git repository (or any parent up to mount point /tmp)
2015-05-26_19:51:23 qr1hi-8i9sb-74468yxq0cauxbw 3908 stderr Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
2015-05-26_19:51:23 qr1hi-8i9sb-74468yxq0cauxbw 3908 stderr Traceback (most recent call last):
2015-05-26_19:51:23 qr1hi-8i9sb-74468yxq0cauxbw 3908 stderr File "/tmp/crunch-job/opt/python/setup.py", line 49, in <module>
2015-05-26_19:51:23 qr1hi-8i9sb-74468yxq0cauxbw 3908 stderr cmdclass={'egg_info': tagger},
2015-05-26_19:51:23 qr1hi-8i9sb-74468yxq0cauxbw 3908 stderr File "/usr/lib/python2.7/distutils/core.py", line 152, in setup
2015-05-26_19:51:23 qr1hi-8i9sb-74468yxq0cauxbw 3908 stderr dist.run_commands()
2015-05-26_19:51:23 qr1hi-8i9sb-74468yxq0cauxbw 3908 stderr File "/usr/lib/python2.7/distutils/dist.py", line 953, in run_commands
2015-05-26_19:51:23 qr1hi-8i9sb-74468yxq0cauxbw 3908 stderr self.run_command(cmd)
2015-05-26_19:51:23 qr1hi-8i9sb-74468yxq0cauxbw 3908 stderr File "/usr/lib/python2.7/distutils/dist.py", line 971, in run_command
2015-05-26_19:51:23 qr1hi-8i9sb-74468yxq0cauxbw 3908 stderr cmd_obj.ensure_finalized()
2015-05-26_19:51:23 qr1hi-8i9sb-74468yxq0cauxbw 3908 stderr File "/usr/lib/python2.7/distutils/cmd.py", line 109, in ensure_finalized
2015-05-26_19:51:23 qr1hi-8i9sb-74468yxq0cauxbw 3908 stderr self.finalize_options()
2015-05-26_19:51:23 qr1hi-8i9sb-74468yxq0cauxbw 3908 stderr File "/usr/lib/python2.7/dist-packages/setuptools/command/egg_info.py", line 85, in finalize_options
2015-05-26_19:51:23 qr1hi-8i9sb-74468yxq0cauxbw 3908 stderr self.vtags = self.tags()
2015-05-26_19:51:23 qr1hi-8i9sb-74468yxq0cauxbw 3908 stderr File "/tmp/crunch-job/opt/python/gittaggers.py", line 19, in tags
2015-05-26_19:51:23 qr1hi-8i9sb-74468yxq0cauxbw 3908 stderr self.tag_build = self.git_timestamp_tag()
2015-05-26_19:51:23 qr1hi-8i9sb-74468yxq0cauxbw 3908 stderr File "/tmp/crunch-job/opt/python/gittaggers.py", line 14, in git_timestamp_tag
2015-05-26_19:51:23 qr1hi-8i9sb-74468yxq0cauxbw 3908 stderr '--format=format:%ct', '.']).strip()
2015-05-26_19:51:23 qr1hi-8i9sb-74468yxq0cauxbw 3908 stderr File "/usr/lib/python2.7/subprocess.py", line 544, in check_output
2015-05-26_19:51:23 qr1hi-8i9sb-74468yxq0cauxbw 3908 stderr raise CalledProcessError(retcode, cmd, output=output)
2015-05-26_19:51:23 qr1hi-8i9sb-74468yxq0cauxbw 3908 stderr subprocess.CalledProcessError: Command '['git', 'log', '--first-parent', '--max-count=1', '--format=format:%ct', '.']' returned non-zero exit status 128

but it doesn't affect me, it just looks bad


Subtasks

Task #6284: Review 6149-quiet-egg-info-stderr-wipResolvedBrett Smith

Associated revisions

Revision f0a92e38
Added by Brett Smith over 6 years ago

Merge branch '6149-quiet-egg-info-stderr-wip'

Closes #6149, #6284.

History

#1 Updated by Bryan Cosca over 6 years ago

  • Target version set to Bug Triage

#2 Updated by Brett Smith over 6 years ago

  • Subject changed from Git log error message to [Crunch] Install step logs error message from `setup.py egg_info`
  • Category set to Crunch
  • Target version changed from Bug Triage to 2015-07-08 sprint

The original report is correct: this doesn't interfere with the running job at all. Crunch "expects" this error, and does some things differently based on whether or not it happens. We just need to suppress the error text.

#3 Updated by Brett Smith over 6 years ago

  • Story points set to 0.5

#4 Updated by Brett Smith over 6 years ago

  • Assigned To set to Brett Smith

#5 Updated by Brett Smith over 6 years ago

  • Status changed from New to In Progress

#6 Updated by Brett Smith over 6 years ago

6149-quiet-egg-info-stderr-wip is up for review. It captures egg_info's stderr, and since we're doing that anyway, it checks the stderr to see if the error message refers to git. If so, we go ahead and set a build tag to recover as before. Otherwise, we propagate stderr and abort.

Tested in a Docker cluster with a good Arvados SDK, and an Arvados SDK where setup.py was hacked to just raise Exception(). Logs look as you would expect.

#7 Updated by Tom Clegg over 6 years ago

The only suspicious thing I noticed is that $? != 0 does not guarantee $? >> 8 != 0, which means there's a possibility here (e.g., python command killed by signal) of abandoning the install process but exiting 0. Perhaps exit ($egg_info_exit || 1)?

      my $egg_info_exit = $? >> 8;
      foreach my $errline (@egg_info_errors) {
        print STDERR_ORIG $errline;
      }
      warn "python setup.py egg_info failed: exit $egg_info_exit";
      exit $egg_info_exit;

#8 Updated by Brett Smith over 6 years ago

  • Status changed from In Progress to Resolved
  • % Done changed from 0 to 100

Applied in changeset arvados|commit:f0a92e384da79e0a17efb42de17031f45f006e44.

Also available in: Atom PDF