Change release process to update KServe Helm charts based on kustomize build dry-run

[ckadner]

Describe the problem

Currently, we update the KServe Helm charts for ModelMesh with each release in a rather tedious manual process, where we find all the changes made to the ModelMesh Serving kustomize manifests in the kserve/modelmesh-serving repo and update the corresponding Helm charts in the kserve/kserve repo.

However, the ModelMesh manifests are modified during the install process before being deployed to the cluster. Yet, we usually update the KServe Helm charts based on the unmodified YAML files in the config/ folder. Which means there are differences in an actual ModelMesh Serving deployment when installed using the ModelMesh Serving install script, vs the ModelMesh Serving deployment installed via the KServe Helm charts.

Proposed solution

We need to update the ModelMesh Serving release process to run a kustomize build (run the install script in dry-run mode) and use the generated final manifests as the basis to sync with the Helm charts in the kserve/kserve repo.

FYI @rafvasq @Jooho @rachitchauhan43

[spolloni]

Hey @ckadner! I landed on this issue while trying to decide how to install model mesh. I am very ignorant about kerserve and modelmesh at this stage -- trying to experiment with it for the first time. I do have a penchant for the helm charts though as this is how we install most things in our cluster.

there are differences in an actual ModelMesh Serving deployment when installed using the ModelMesh Serving install script, vs the ModelMesh Serving deployment installed via the KServe Helm charts.

should I worry about these differences in any way?

[spolloni]

it also seems the latest version in the chart is 0.11.0