Skip to content

MakingANewRelease

Jump to bottom
Brice Goglin edited this page Dec 5, 2023 · 13 revisions

How to Make a New Hwloc Release

There are several steps that are required to make a new hwloc release/prerelease and/or release series.

There are two cases:

  1. You are making a release on a brand-new release series (it's probably a pre-release, like an alpha, beta, or RC -- but hey, it could be a final release, too).
  2. You are making a release on an existing release series -- meaning that you've already followed the previous steps for initially creating the release series; now you're just putting a new release out on that existing series.

Announces

We usually announce the release to the following mailing lists:

Make a New Hwloc Release Series

This is the case where you need to make a release for version A.B.C where C==0 -- the release is probably a pre-release of some kind (e.g., an alpha, beta, or rc tarball). None of the infrastructure yet exists to make this new series.

Build System

Before making a branch for the new release series, it's usually a good idea to evaluate if upgrading to newer versions of the GNU Autotools is a good idea. Specifically: we make an effort to pick a tuple of Autotools for use with a release series and try very hard to use those same exact versions for every official release in that series.

If you end up upgrading the version of Autotools that you're using, it is best to edit source:/contrib/dist/make_dist_tarball (on the master -- before you make the Git branch for the new series) to set these versions. Run the script to ensure that these versions work properly with the master branch. Tune up anything that you need on the master branch to make these versions work. Once it's all working, commit the new version of make_dist_tarball with those version numbers back to the master branch.

API Version, etc.

If the API has changed, increasing HWLOC_API_VERSION in hwloc.h is usually a good idea. This should only be needed for major releases. Do it in master before branching.

Also update HWLOC_VERSION* in contrib/windows/hwloc_config.h.

If struct hwloc_topology has changed, or #define used inside it, or one of its children structures (internal distances, support, bitmap), HWLOC_TOPOLOGY_ABI should be bumped too.

If the plugin ABI has changed (include/hwloc/plugins.h), bumping HWLOC_COMPONENT_ABI might be necessary too.

Git

The main thing to do is to create a release branch in Git. Once you get the master branch in a good state, branch it. Note that there is a naming convention for branches:

$ git branch vA.B

where "A.B" is obviously the numbers that are relevant for your new release series (e.g., "1.1").

Once you make this branch, check it out and edit the VERSION file. Ensure that its VERSION numbers are set to A.B and the greek is set to something relevant (e.g., "a1" or "rc1").

Set the .so library version to the appropriate value (see the Libtool documentation for instructions: page 1 and page 2. And commit these changes.

Update the JenkinsFile using a different night time for scheduled extended builds.

Back on the master branch, edit the VERSION file and ensure that it is now set to version A.(B+1) and its greek is set to something like "a1".

Once everything is ready, push everything to github, as well as a tag on the new branch so that the nightly script (git describe) uses vA.B instead of dev as the basename.

$ git tag hwloc-A.B-branch vA.B
$ git push origin vA.B
$ git push origin hwloc-A.B-branch
$ git push origin master

Post the Tarballs

  1. Copy the tarballs to aws.open-mpi.org

  2. Log there, update your clone of ompi-scripts, and run something like:

~/ompi-scripts/dist/upload-release-to-s3.py --project hwloc --branch v1.11 --version 1.11.10 --date 2018/03/26 --files ~/hwloc-*1.11.10*

Web Site

First, get a clone of the Open MPI web site (because the hwloc tree is part of that web site) from https://github.com/open-mpi/ompi-www/.

NOTE it is a very large tree. As of August 2014, a Git clone of this tree takes about 4.3GB of disk space.

  1. Go into the software/hwloc tree.
    1. Copy the prior release series tree into a new tree: cp -a vA.(B-1) vA.B (e.g., cp v1.0 v1.1).
    2. Cd into the new vA.B tree.
    3. Edit the version.inc file and change all instances of vA_(B-1) to vA_B and all instances of vA.(B-1) to vA.B.
    4. Update the $releases or $prereleases variable with the new release.
    5. Check index.php in case you need to update anything there.
    6. If all that worked, you should be able to surf to your tree in software/hwloc/vA.B/ and see your release tarballs listed.
      • Double check that the download links work
      • Double check that the link to the Git NEWS file works

Post Doxygen Documentation

  1. Cd to the projects/hwloc/doc directory.
    1. Expand your release tarball and run configure on it. Cd into the doc directory.
    2. IMPORTANT: Ensure to have a recent version of Doxygen before continuing.
    3. IMPORTANT: You must do this from your expanded and configured tarball. Do NOT generate the doxygen docs from a Git checkout. The idea here is the GUARANTEE that the generated docs exactly match the release tarballs.
    4. Run doxygen www.open-mpi.org.cfg. This will generate a directory named www.open-mpi.org.
    5. Use contrib/dist/publish_doc from a GIT tree to copy the generated PDF and HTML directory to the website clone.
    6. Check the links displayed at the end of publish_doc script in case they need to be updated in the front page.
    7. Edit the projects/hwloc/doc/index.php file to update the arrays of docs to be displayed.
      • Browse to projects/hwloc/doc/ and ensure that your prerelease PDF and HTML links show up and work properly.
      • Browse to projects/hwloc/doc/vA.B/ and ensure that the HTML documentation shows up properly.

Update Left-Hand Navigation

  1. Edit the projects/hwloc/nav.inc file.
  2. Add a line for the new series. You might want to label it "(new)", "(beta)", or somesuch.

Git Commit

  1. Make sure to git add all new files.
  2. Then Git commit and push to your Github clone.
  3. Pull it on AWS
  4. Browse all the pages that you just added and ensure that they are all working properly, including all links.
  5. When you're sure everything is correct, do a PR and merge it.
  6. REMEMBER: The main Open MPI web site runs git pull every hour or so. So even if your changes aren't immediately visible on the main site, check back in an hour.

Make a New Hwloc (Pre)Release on an Existing Release Series

This is the case where you are making a release on a release series that has already been created (see below). Hence, all the infrastructure for the release series is already in place and you just need to make a new point release (potentially even the first formal release of the series, version A.B.C, where C==0, because the release series was created with a pre-release such as an alpha, beta, or rc tarball).

Misc

Check whether HWLOC_API_VERSION must be updated.

Update HWLOC_VERSION* in contrib/windows/hwloc_config.h

Make the Tarballs

  1. Edit the VERSION file and change at least 2 things:
    • Set the greek value to something relevant (e.g., "b1" or "rc1" or whatever -- keep bumping the number as you make successive pre-releases towards the final release).
    • Set the .so library version to the appropriate value (see the Libtool documentation for instructions: page 1 and page 2
    • Also update the HWLOC_VERSION in contrib/windows/private_config.h
    • You must commit the modified VERSION values back to Git
  2. NOTE: Each hwloc release branch requires a specific tuple of Autoconf/Automake/Libtool. You are strongly discouraged from changing this tuple for the life of a given series. The idea is to keep the build system fairly constant for the life of a series and only change the tuple at the next release series.
  3. Double check your Git tree and ensure that "git status" returns no locally-modified files.
  4. From the top-level source directory, run contrib/dist/make_dist_tarball.

If the process succeeds, you'll have 4 tarballs:

  • tar.gz and tar.bz2 for the pre-release/greek version.
  • tar.gz and tar.bz2 for the final version (i.e., exactly the same, but with the greek value set to the empty string).

The rationale is that you can test the greek tarballs, and if they're good, you have the final tarballs ready to ship. They're guaranteed to be exactly the same because they were all made at the same time.

Post the Tarballs (see above)

Post Doxygen Documentation

Follow the same steps as above: [#PostDoxygenDocumentation for posting the Doxygen documentation]. If you are releasing a production version, <greek> will be empty.

Additionally, if this is the first production release, then Git remove all the pre-release documentation directories. We do not continue hosting pre-release documentation after a final release is made.

Update Left-Hand Navigation

  • If this was the first production release of a series:
    • Set the $dir variable in software/hwloc/current/version.inc to point to the directory of the new release series. E.g., "v1.1".
    • Edit the projects/hwloc/nav.inc file.
    • Label the new series as "(stable)".
    • Update the labels on the prior series to indicate that they are "(older)", or "(ancient)", or whatever is relevant.
  • If this is not a pre-release
    • Edit projects/hwloc/doc/index.php
    • Add a new link for the new release series.

Check Everything and Git Push/PR/Merge

Follow the same steps as above for checking and merging everything.

Git

Tag the new release (some people also want tags for prereleases).

For example:

$ git tag -a hwloc-A.B.C
$ git push origin hwloc-A.B.C

where "A.B.C" is obviously the numbers that are relevant for your new release series (e.g., "1.1.0"). Note that we include the C value in the tag, even if it's zero.

NOTE: Be sure to use the -a option so that an annotated Git tag is created. See git-tag(1) for more details.

Once you have made this tag, edit the VERSION file on the branch and increment the version number so that it is now "A.(B+1).0a1". This ensures that we don't make new nightly snapshot tarballs that have the "A.B" version number. Specifically, version A.B has been released -- so we shouldn't make any more snapshot tarballs with that version number.

Github Release

Some users watch Github release notifications to update their hwloc, hence it's nice to update there too. However Github automatic releases only ship a snapshot of the tag without running autogen or generating the doc. Hence those are disabled (and the release page is hidden from the main page).

For non-RC releases, we manually publish a github release once the tag is pushed. Go to https://github.com/open-mpi/hwloc/releases and draft a new release:

  • select the appropriate tag
  • copy paste the big disclaimer from previous releases in the title an description
  • add our official tarballs/winballs

Android Release

Some releases deserve an Android app update (both Play store and F-Droid).

Clone this wiki locally