Project

General

Profile

Actions

Bug #6497

closed

[Deployment] Separate package upload portions of run-build-packages.sh to a separate script

Added by Brett Smith over 7 years ago. Updated over 7 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
Deployment
Target version:
Start date:
07/07/2015
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)
Story points:
2.0

Description

The old way: we had a single shell script, run-build-packages.sh, that built .debs from our source repository, and uploaded them to an apt repository.

The new way: run-build-packages.sh builds packages for a single target distribution. Jenkins invokes this script multiple times in a dedicated Docker container for every distribution we support (see #6496). After that's done for every distribution, it runs the uploader script to publish the built packages.

The uploader should accept a list of distribution arguments that tell it what set(s) of packages to upload. There should be a shortcut that means "upload all known distributions." For each distribution, it should know how to find the built packages, scp them to another Web server, and run the necessary commands to publish them through an apt or yum repository.


Subtasks 3 (0 open3 closed)

Task #6502: Extract existing uploader code from run-build-packages.sh into a separate scriptResolvedBrett Smith07/07/2015

Actions
Task #6503: Parameterize the upload script to upload packages for the specified distribution(s)ResolvedBrett Smith07/07/2015

Actions
Task #6533: Review arvados-dev branch 6497-separate-package-upload-wipResolvedNico César07/08/2015

Actions
Actions #1

Updated by Brett Smith over 7 years ago

  • Description updated (diff)
Actions #2

Updated by Brett Smith over 7 years ago

  • Story points set to 2.0
Actions #3

Updated by Brett Smith over 7 years ago

  • Assigned To set to Ward Vandewege
Actions #4

Updated by Ward Vandewege over 7 years ago

  • Assigned To changed from Ward Vandewege to Brett Smith
Actions #5

Updated by Brett Smith over 7 years ago

  • Status changed from New to In Progress
Actions #6

Updated by Brett Smith over 7 years ago

I started manually testing in production, and with the latest fixups, it all works.

The first time it runs, it takes foorrrreeevvveeeerrrrr (10-15 minutes) because it uploads and publishes every .deb and .rpm we make. We can work around this by running something like touch -d '2 weeks ago' .last_upload to set a manual cutoff point if desired. Future runs will have a timestamp and go faster.

Actions #8

Updated by Brett Smith over 7 years ago

  • Status changed from In Progress to Resolved
  • % Done changed from 67 to 100

Applied in changeset arvados-dev|commit:c31cdc09ae6aefbbf6f9e14b3c6fdf910a729b29.

Actions

Also available in: Atom PDF