Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/tir rm dyn data add helm status #1143

Open
wants to merge 40 commits into
base: master
Choose a base branch
from

Conversation

henrjk
Copy link
Member

@henrjk henrjk commented Jul 25, 2024

This is to supersede PR #1135. It is designed to be merged with the changes in opendevstack/ods-document-generation-templates#146.

Compared to PR#1135 some code was removed to reduce complexity and also addresses review comments.
The initial PR anticipated that we would want to take advantage of Pod related information in the helm status. However this information is not present for DeploymentConfig. Another change was that the helm release status is now surfaced with their own table replacing the deployment resource centric format of the TIR for tailor based deployment. Only a single helm release per component is anticipated and it is displayed like this:

Deployment Status

Table surfacing the releaseName, releaseRevision, namespace, deployStatus, deployDescription, lastDeployed and resources.

Deployment Mean

Table surfacing helm parameters.

Further change notes:

  • To avoid CPS issues in Jenkins all types were using their own file and the Immutable annotation was avoided.
  • In the deployment artifact the simplified helmStatus is kept under key 'helmStatus' in the deploymentMean information. This is also written to ods-deployments.json by DeploymentDescriptor.
  • PodData fields capturing dynamic data where removed, except for the podName which has a prefix useful for correlating dynamic data with pods. At the moment this is still needed for both tailor and helm deployments. The code which finds this named pods did require retry logic even for helm installs using --atomic.

henrjk and others added 15 commits July 1, 2024 11:45
If helm is used for deployment, the components deployed by helm are
no longer listed as toplevel elements in the TIR. Instead the
helm release is represented by its status so that users will see
tables under:

Deployment Resource: <helm-release-name>-deploymentMean
Deployment Resource: <helm-release-name>-deploymentStatus

The DeploymentStatus has rows surfacing the releaseName, releaseRevision, namespace, deployStatus, deployDescription, lastDeployed and resources.

Here an example from the resources:
"Deployment: backend-helm-monorepo-chart-component-a, backend- helm-monorepo-chart-component-b, Service: backend-helm-monorepo- chart"

Non-Helm deployments are surfaced as before.

Further change notes:
- To avoid CPS issues in Jenkins all types were using their own file
and the Immutable annotation was avoided.
- Type HelmStatusData captures helm status with high fidelity whereas
HelmStatusSimpleData captures shallower information in particular
omitting the details of resources except their names and kinds.
This approach simplifies the json Helm status json parsing code
while keeping the consuming code relatively lean.
In the deployment artifact the simplified helmStatus is kept under key
'helmStatus' in the deploymentMean information. Which is also written
to `ods-deployments.json` by DeploymentDescriptor.
- PodData fields capturing dynamic data where removed, except for the
podName which has a prefix useful for correlating dynamic data with
pods. At the moment this is still needed for both tailor and helm
deployments. The code which finds this named pods did require retry logic
even for helm installs using --atomic.
+ oc -n kraemerh-dev get pod -l app.kubernetes.io/instance=backend-helm-monorepo -o json
expected to call org.ods.util.JsonLogUtil.debug but wound up catching org.ods.util.Logger.getDebugMode; see: https://jenkins.io/redirect/pipeline-cps-method-mismatches/
@henrjk henrjk requested a review from jafarre-bi July 25, 2024 07:05
Copy link
Contributor

@jafarre-bi jafarre-bi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Several changes and simplifications are needed. Please, also check that all required unit tests are implemented. There is no unit test for HelmStatusSimpleData.fromJsonData.

src/org/ods/util/JsonLogUtil.groovy Outdated Show resolved Hide resolved
src/org/ods/component/HelmDeploymentStrategy.groovy Outdated Show resolved Hide resolved
src/org/ods/component/HelmDeploymentStrategy.groovy Outdated Show resolved Hide resolved
src/org/ods/component/HelmDeploymentStrategy.groovy Outdated Show resolved Hide resolved
src/org/ods/component/HelmDeploymentStrategy.groovy Outdated Show resolved Hide resolved
}

@NonCPS
Map<String, Object> toMap() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this really used? And if it is used, do we really need this? Having this class is a proper alternative to using maps.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this is needed at least in order to create a Map value for another Map that will be added to a Context.


@NonCPS
String toString() {
return toMap().toMapString()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can directly build the string instead. Probably the toMap method isn't necessary and will be removed.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is needed for GStrings, see: #1143 (comment)

}

@NonCPS
private static Tuple2<String, String> extractResource(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's very difficult to understand this method. There must be a much simpler and clearer way to do this, which may even not need a method returning a tuple. This needs rewriting.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The logic for this method is once again related to throwing exceptions with messages reporting what attributes are missing, see: #1143 (comment)


@SuppressWarnings(['Instanceof'])
@NonCPS
private static Tuple2<List<String>, List<String>> collectMissingStringAttributes(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If an attribute is missing, its value will be null. The users of this class will decide what to do with it.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as: #1143 (comment)

import groovy.transform.TypeChecked

@TypeChecked
class HelmStatusSimpleData {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is just a typical JavaBean holding a set of properties. It's reasonable to have a builder from a Map, but otherwise, no other logic or validation should be done. The users of this data will validate the property values when needed.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to ask for some clarification on this, regarding use cases for this HelmStatusSimpleData.
If this class is intended to be used as a "typed" JavaBean object in order to guarantee type-safety and data integrity, excluding the logic for converting to/from Map, what's the purpose for the rest of the logic?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @anteloro-boeh I am back now and if needed could answer questions about what I had in mind when writing this code in the first place. But perhaps this was already sorted out.?

@anteloro-boeh anteloro-boeh force-pushed the feature/tir-rm-dyn-data-add-helm-status branch from 8c9643b to c762b3b Compare October 14, 2024 09:48
src/org/ods/component/HelmDeploymentStrategy.groovy Outdated Show resolved Hide resolved
src/org/ods/orchestration/phases/DeployOdsComponent.groovy Outdated Show resolved Hide resolved

def modifier = { byte[] document ->
codeReviewReport
? this.pdf.merge(this.steps.env.WORKSPACE as String, [document] + codeReviewReport)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't this the same as [document, codeReviewReport]?

Copy link

@anteloro-boeh anteloro-boeh Nov 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not in this case, given that codeReviewReport is of type List<byte[]>
which gives:
[document] + codeReviewReport -> [byte[], byte[], byte[], ...] // we want this
[document, codeReviewReport] -> [byte[], List<byte[]>] // but not this

An alternative would be: [document, *codeReviewReport]
but spread operator '*' is not frequently used in this codebase.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants