Release management

[daw3rd]

We need to define a release process to publish the following:

1. data-prep-toolkit library wheel to pypi
2. transform images to quay.io
3. data-prep-toolkit-kfp wheel to pypi
4. kfp image to quay.io

Eventually we might also want to add

1. wheels for each transform implementation to pypi

This would allow others to extend a transform implementation on their own outside of the repo.

Questions to address:

1. Should all wheels on pypi use the same version
2. Are the images published separately from the wheels (i.e. can they evolve independent of the libraries)
3. How do we trigger a release (on the website with a git action?)

[roytman]

Hi @daw3rd , thank you for raising it.
I think the project documentation should to be auto-generated per release. We did it in our previous project.
For your third question, I would say release creation should be started manually when an administrator thinks the new release is ready. However, release processing can be done as GH actions. (again, at least we did it in the past as an action)

[blublinsky]

@daw3rd here are my answers to the questions:

Should all wheels on pypi use the same version
Are the images published separately from the wheels (i.e. can they evolve independent of the libraries)

I would separate major version from minor. I think in our case the first two digits is Major, the last one is minor. Moving forward we might switch and consider only the first digit as Major.
With this, I would say that all the wheels for a given release and all the images should have the same Major version, while the minor one can change

[agoyal26]

@daw3rd are your questions satisfactorily answered ?
what are the actions we need to take/define for DPK developers from here ?