Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Linux package repository migration + automatic release #8

Open
wants to merge 2 commits into
base: develop
Choose a base branch
from

Conversation

rochala
Copy link

@rochala rochala commented Aug 21, 2023

PR implementing ongoing effort of migration from JFrog and securing sbt repository: https://contributors.scala-lang.org/t/roadmap-for-the-sbt-community-repository/6195

This PR focuses on 2 aspects:

  1. Migration from JFrog and setting up release process for new debian and rpm packages,
  2. Automating the process of the release (if this is something which is not necessary to have, we can delete it, but currently each version had to be released manually, this PR adds automatic release prompted by publishing one in sbt/sbt repository). This is important to have proper automation, as it won't require any input from the maintainers to have it released (right now version 1.9.3 of sbtn is not published nor released)

On top of that, this should answer a few reported issues such as:

  • added support for all architectures instead of specific ones directly in the configuration. This will solve the existing problem, where sbt couldn't be installed simply with apt install sbt but required an additional flag selecting the amd64 architecture, despite the fact sbtn is platform independent.
  • rpm repository is fully signed along with the metadata

This will require changes in sbt/sbt such as adding additional CI which will run the deployment in this repository and also update sbtn-version to the released one
https://github.com/sbt/sbt/blob/1.9.x/sbt#L27 with possible fallback. This PR may require additional changes to separate building and signing linux packages from native client building.

All the packages in this PR are signed with different key, as secrets are not yet setup in this repository.

@adpi2
Copy link
Member

adpi2 commented Aug 22, 2023

This will require changes in sbt/sbt such as adding additional CI which will run the deployment in this repository

So we would have to clone this repo from sbt/sbt CI to release sbtn.

and also update sbtn-version to the released one https://github.com/sbt/sbt/blob/1.9.x/sbt#L27 with possible fallback.

What about using an environment variable to achieve that? I think we use one already in the sbt/sbt CI.

as secrets are not yet setup in this repository.

Do we need to set them up here or in sbt/sbt?

@rochala
Copy link
Author

rochala commented Aug 22, 2023

So we would have to clone this repo from sbt/sbt CI to release sbtn.

It works a bit different. SBT only pings this repository to start the release automatically - https://github.com/rochala/sbt/blob/1.9.x/.github/workflows/release.yml
All of the release proccess happens here.

What about using an environment variable to achieve that? I think we use one already in the sbt/sbt CI

This is one solution and it can be extracted during CI to use previous release from this repository, as this is the source of sbt native client binaries.

Do we need to set them up here or in sbt/sbt?

In this repository we need various of secrets:

  • Private key to sign
  • Private key password to automate signing
  • Private key email, keyname and keygrip to automate signing
    This has to be private key from used in JFrog as each release need resigning of old packages.

And both repositories need

  • Private key for new bot
  • Bot ID
    The bot is responsible for first starting the job from sbt/sbt and secondly doing pull request, auto merging and creating release in this one.

I used Github App instead of personal access token as with Github app you can limit the repositories it can access and have more granular possible actions.

@rochala
Copy link
Author

rochala commented Aug 22, 2023

The one part I'm not sure if we can do is changing how our repository is structured. Currently we have separated source info for sbt < 1.5.0 called sbt_old.list and another one for new versions. This leads to well additional problems as you have to add 2.

echo "deb https://repo.scala-sbt.org/scalasbt/debian all main" | sudo tee /etc/apt/sources.list.d/sbt.list
echo "deb https://repo.scala-sbt.org/scalasbt/debian /" | sudo tee /etc/apt/sources.list.d/sbt_old.list

We can also add architecture in the source input:

echo "deb [arch=all] https://repo.scala-sbt.org/scalasbt/debian all main" | sudo tee /etc/apt/sources.list.d/sbt.list

which will solve the problem with missing support for aarch64 architecture.

The drawback of removing the separation of both repositories is that it will break on the user machines and they will need to update it. In my opinion it is worth.

@rochala rochala marked this pull request as ready for review August 30, 2023 14:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants