Bug #17012

[versioning] generate correct development version numbers for all packages after a major release

Added by Ward Vandewege 7 days ago. Updated 1 day ago.

Status:
In Progress
Priority:
Normal
Assigned To:
Category:
-
Target version:
Start date:
Due date:
% Done:

0%

Estimated time:
Story points:
-

Description

The build/version-at-commit.sh script doesn't handle the scenario right after a major release well.

For code that hasn't changed since the major release, it generates version numbers without taking the major release tag into account.

For instance; as of commit c6c2f3518bc745eed95b5f5b81db5d17db4366ff on master, which is after the fork point of the 2.1-dev branch, these packages were generated for debian10:

arvados-api-server_2.1.0-1_amd64.deb
arvados-client_2.1.0~dev20201012021212-1_amd64.deb
arvados-controller_2.1.0~dev20201012021212-1_amd64.deb
arvados-dispatch-cloud_2.1.0~dev20201012021212-1_amd64.deb
arvados-docker-cleaner_2.2.0~dev20201015145956-1_amd64.deb
arvados-git-httpd_2.1.0~dev20201012021212-1_amd64.deb
arvados-health_2.1.0~dev20201012021212-1_amd64.deb
arvados-src_2.2.0~dev20201015145956-1_all.deb
arvados-sync-groups_2.2.0~dev20201013141632-1_amd64.deb
arvados-workbench_2.1.0~dev20201012021212-1_amd64.deb
arvados-ws_2.1.0~dev20201012021212-1_amd64.deb
crunch-dispatch-local_2.1.0~dev20201012021212-1_amd64.deb
crunch-dispatch-slurm_2.1.0~dev20201012021212-1_amd64.deb
crunch-run_2.1.0~dev20201012021212-1_amd64.deb
crunchstat_2.1.0~dev20201012021212-1_amd64.deb
keep-balance_2.1.0~dev20201012021212-1_amd64.deb
keep-block-check_2.1.0~dev20201012021212-1_amd64.deb
keep-exercise_2.1.0~dev20201012021212-1_amd64.deb
keepproxy_2.1.0~dev20201012021212-1_amd64.deb
keep-rsync_2.1.0~dev20201012021212-1_amd64.deb
keepstore_2.1.0~dev20201012021212-1_amd64.deb
keep-web_2.1.0~dev20201012021212-1_amd64.deb
libpam-arvados-go_2.1.0~dev20201012021212-1_amd64.deb
python3-arvados-cwl-runner_2.2.0~dev20201015145956-1_amd64.deb
python3-arvados-fuse_2.2.0~dev20201015145956-1_amd64.deb
python3-arvados-python-client_2.2.0~dev20201015145956-1_amd64.deb
python3-crunchstat-summary_2.2.0~dev20201015145956-1_amd64.deb

You can see that the python packages have a version number that takes the release into account (2.2.0~dev20201015145956), which is because there are some commits in master (post the 2.1-dev branch fork) that touch the Python part of the codebase.

For other parts of the codebase, a version string is generated that starts with 2.1.0~dev, which is not ideal.

For the API server package, it even generated the version string 2.1.0, because the calculated version for that package and its dependencies happens to be the commit that is tagged with the 2.1.0 version.

History

#1 Updated by Ward Vandewege 7 days ago

  • Status changed from New to In Progress

#2 Updated by Ward Vandewege 7 days ago

  • Description updated (diff)

#3 Updated by Ward Vandewege 7 days ago

  • Description updated (diff)

#4 Updated by Ward Vandewege 7 days ago

  • Description updated (diff)

#5 Updated by Ward Vandewege 7 days ago

  • Description updated (diff)

#6 Updated by Ward Vandewege 7 days ago

After some reflection, I think I've convinced myself that this behavior - counterintuitive as it is - is actually OK. The packages that are being generated do actually correspond with the source code from that version.

There is, however, a different bug. This all came up because a branch was merged that changed the package version numbers for dev packages from .dev to ~dev.

The version number generation code did not recognize that this change (obviously!) was a reason to generate new package version numbers. That's because that code never took the 'build' directory into account when calculating the in-tree dependencies of each of our pieces of software, the git hashes and timestamps of which are used to generate an appropriate version number.

So; the real fix here is to add the 'build' directory to the list of directories that is checked as part of the version number calculation.

This is easy for the Go and Rails packages, and I've pushed a fix along those lines in e94ddba9d544b173e5b56b41c6ac76ea0b072a26

For the Python packages, the version calculation is done in 'arvados_version.py', and that happens inside a virtualenv. It's more complicated to take the 'build' directory into account there, so I'm going to punt on that for now.

#7 Updated by Ward Vandewege 1 day ago

  • Target version changed from 2020-10-21 Sprint to 2020-11-04 Sprint

Also available in: Atom PDF