jervis-2.0.1

jervis 2.0.1 - Oct 19th, 2023

• Critical bugfix: PUT requests would not properly send to servers. Exact root cause is unknown but the fix was validated as working.

See b275207 for the specific fix.

jervis-2.0

jervis 2.0 - Jun 27th, 2023

This is a new major release. From an end user perspective, all behavior for 1.x and 0.x YAML files is still supported. However, there are major API changes which warrant bumping the major to warn integrators who may be using code.

Migrating code

You can use GNU sed to migrate code. The following is a sedfile of expressions.

s/ \+$//
s/pipelineGeneratorTest/PipelineGeneratorTest/g
s/platformValidatorTest/PlatformValidatorTest/g
s/lifecycleGeneratorTest/LifecycleGeneratorTest/g
s/toolchainValidatorTest/ToolchainValidatorTest/g
s/jervisConfigsTest/JervisConfigsTest/g
s/lifecycleValidatorTest/LifecycleValidatorTest/g
s/lintJenkinsVarsTest/LintJenkinsVarsTest/g
s/securityIOTest/SecurityIOTest/g
s/pipelineGenerator/PipelineGenerator/g
s/lifecycleValidator/LifecycleValidator/g
s/toolchainValidator/ToolchainValidator/g
s/lifecycleGenerator/LifecycleGenerator/g
s/platformValidator/PlatformValidator/g
s/securityIO/SecurityIO/g
s/net\.gleske\.jervis\.lang\.[Ll]ifecycleGenerator\([ .]getObjectValue\)/net.gleske.jervis.tools.YamlOperator\1/g
s/generator\.getObjectValue/net.gleske.jervis.tools.YamlOperator.getObjectValue/g

Usage of the sedfile is the following.

find * -type f -name '*.groovy' -exec sed -i -f /tmp/sedfile {} +

Manually search for and change the following methods. See Major API changes
section for details.

# change id_rsa_keysize usage to rsa_keysize
grep -r '\([gs]et\)\{0,1\}[Ii]d_rsa_keysize' *

Several Jenkins shared pipline vars have been converted to NonCPS. This means upstream admin* functions must also be changed to NonCPS. The following is an example of a NonCPS var.

@NonCPS
def call() {
    // this method is NonCPS JIT compiled
}

If you define admin* vars you'll have to convert them to NonCPS. The following is a list of vars now requiring NonCPS annotation.

• vars/adminLibraryResource.groovy; refer to [loadCustomResource][loadCustomResource]

Migrating JSON to YAML

platforms, lifecycles, and toolchains have migrated from JSON to YAML. Users tend to define these files themselves. As a result, a migration script has been created. Run the migration script for your given prefix.

Major API changes

The following classes have been renamed.

Old name for imports	New name for imports
net.gleske.jervis.lang.lifecycleGenerator	net.gleske.jervis.lang.LifecycleGenerator
net.gleske.jervis.lang.lifecycleValidator	net.gleske.jervis.lang.LifecycleValidator
net.gleske.jervis.lang.pipelineGenerator	net.gleske.jervis.lang.PipelineGenerator
net.gleske.jervis.lang.platformValidator	net.gleske.jervis.lang.PlatformValidator
net.gleske.jervis.lang.toolchainValidator	net.gleske.jervis.lang.ToolchainValidator
net.gleske.jervis.tools.securityIO	net.gleske.jervis.tools.SecurityIO

The following methods and fields have been renamed or removed.

Class	Old method name	New method name
LifecycleValidator	load_JSON()	loadYamlFile()
LifecycleValidator	load_JSONString()	loadYamlString()
PlatformValidator	load_JSON()	loadYamlFile()
PlatformValidator	load_JSONString()	loadYamlString()
SecurityIO	getId_rsa_keysize()	getRsa_keysize()
SecurityIO	setId_rsa_keysize()	Removed
SecurityIO	id_rsa_keysize	Removed
ToolchainValidator	load_JSON()	loadYamlFile()
ToolchainValidator	load_JSONString()	loadYamlString()

The following methods have moved.

Method	Old class	New class
getObjectValue	net.gleske.jervis.lang.LifecycleGenerator	net.gleske.jervis.tools.YamlOperator

Warnings:

• Support for all vendors of JDK 1.8 is dropped in this release.
• OpenJDK11 or OpenJDK17 is build runtime and OpenJDK8 is the bytecode compatibility going forward to match the Jenkins project. Groovy 2.4 does not support higher than OpenJDK8 bytecode.
• Function getJervisYamlFiles(String owner, String repository) within class net.gleske.jervis.remotes.GitHubGraphQL used to default to master branch. It now defaults to main branch.
• Function getJervisYamlFiles(String repositoryWithOwner) within class net.gleske.jervis.remotes.GitHubGraphQL used to default to master branch. It now defaults to main branch.

New features:

Pipeline DSL scripts changes in the vars/ folder

• Jervis steps read from platforms, lifecycles, and toolchains YAML instead of JSON.
• Matrix building nodes reordered so it is wrapped in stages.
• New hasGlobalResource() step which can be used to conditionally load resources from libraryResource step. Allows a pipeline developer to only call libraryResource if it exists. Normally libraryResource step will throw an exception if the step doesn't exist. This is a fully NonCPS step and can be called from other NonCPS code blocks.
• New getBuildContextMap() which returns information about the current running pipeline such as how it was triggered, which part of Git workflow, and other meta info.
• New getJervisPipelineGenerators() which can read multiple repositories and return .jervis.yml pipeline objects for each repository in one API call.
• isBuilding() more reliable now that it is built into Jervis with unit tests. Several bugs were fixed while reaching 100% test coverage.
• loadCustomResource() has some new behavior. It first loads adminLibraryResource, then checks for the resource in the global config files plugin, and finally falls back to libraryResource. It can also skip looking for adminLibraryResource via a new boolean option:

  // skip loading adminLibraryResource
  loadCustomResource('resource-name', true)

• The following vars are now fully NonCPS. These vars can be called from within other NonCPS annotated methods in shared pipelines.
  • getBuildContextMap
  • getJervisPipelineGenerators
  • getUserBinding
  • hasGlobalResource
  • hasGlobalVar
  • isBuilding
  • isPRBuild
  • isTagBuild
  • [loadCustomResource][loadCustomResource]
  • prepareJervisLifecycleGenerator
  • prepareJervisPipelineGenerator

Jervis API changes in src/ folder

• [net.gleske.jervis.remotes.GitHubGraphQL][GitHubGraphQL] has a new sendGQL method. variables are now supported as a Map in addition to a String. The Map will be automatically converted to a String before being sent to GitHub as a query.
• [HashiCorp Vault][vault.io] support classes available. This will eventually lead to better native pipeline integration with Vault.
  • [VaultService][VaultService] class provides an easy to use communication class to KV Secrets Engine v1 and v2. AppRole authentication is recommended but any [TokenCredential][TokenCredential] type can be used.
  • AppRole authentication provided by [VaultAppRoleCredential][VaultAppRoleCredential]. It automatically renews leases and rotates credentials as leases run out. By default AppRole role_id and secret_id are resolved from [VaultRoleIdCredentialImpl][VaultRoleIdCredentialImpl], but custom credential resolver can be implented on [VaultRoleIdCredential][VaultRoleIdCredential] interface.
• Extend [net.gleske.jervis.remotes.StaticMocking][StaticMocking] test class to support recording mock API responses while calling Jervis dependent code.
• SimpleRestServiceSupport class changes. All REST services provided in net.gleske.jervis.remotes.* have new behaviors.
  • New HTTP header available on all REST services. Setting the Parse-JSON HTTP header on any REST service will override its default behavior. It can force-parse JSON or it can force returning plain text for JSON APIs instead of parsed JSON objects.
  • The default API response for SimpleRestService ias changed from a Map to a String. This means if there's no content response an empty String will be returned regardless of JSON parsing for the API. This used to return an empty HashMap.
• More flexibility has been added to static method net.gleske.jervis.remotes.SimpleRestService.apiFetch().
• Enhancements in net.gleske.jervis.tools.SecurityIO
  • Converted multiple functions to static to ease their use.
  • Added AES-256 encryption functions.
  • Added RS256 aglorithm for data signing and verification.
  • Added GitHub JSON Web Token (JWT) creation and verification support.
  • Added generic JWT verification.
  • avoidTimingAttack() static function available with usage documentation.
  • getRsa_keysize() always returns the calculated key size if any.
• Enhancements in [net.gleske.jervis.remotes.GitHub][GitHub]
  • Added support for adding headers to all requests via GitHub.headers field.
  • Updated client HTTP headers to match GitHub v3 REST API version 2022-11-28.
• New [CipherMap][CipherMap] utility class meant to transpare...

jervis-1.7

jervis 1.7 - Apr 14th, 2020

Bug fixes

• Bugfix: Additional toolchains loaded into a matrix build did not properly
  matrix. This bug has been fixed and tests added to avoid it.
• Bump snakeyaml to 1.26 to protect against billion laughs style attacks.
• Use SafeConstructor when parsing YAML to prevent remote code execution. See
  Documentation and Java Doc.

Breaking Job DSL changes

Jobs generated now use the SCM Filter Jervis YAML plugin
instead of the SCM Filter Branch PR plugin. If your Jenkins
instance does not have the SCM Filter Jervis YAML plugin installed, then you'll
get errors attempting to generate new jobs. This should not affect jobs that
already exist but it also means existing jobs can't be regenerated without the
plugin.

As a recommended migration path to convert all jobs to use the SCM Filter for
Jervis YAML, you can run a script console script to regenerate all
jbos

Deprecated pipeline steps

The following Jenkins pipeline steps provided by Jervis are deprecated and will
go away in a future release.

• isPRBuild() - use isBuilding('pr') instead.
• isTagBuild() - use `isBuilding('tag') instead.

As an admin, if you still want to support these steps then create your own steps
within your own shared pipeline library. Here are some examples:

Contents of vars/isPRBuild():

Boolean call() {
    isBuilding('pr')
}

Contents of vars/isTagBuild.groovy:

Boolean call() {
    isBuilding('tag')
}

Pipeline DSL scripts changes in the vars/ folder:

• New pipeline steps:
  • getMatrixAxes() - Spawned from a Jenkins blog post.
    It is not used directly by Jervis but is available to users of Jervis.
  • getUserBinding('somevar') - Users can set bindings in their pipeline
    runtime. This step allows pipeline shared libraries to retrieve bindings as
    opposed to having them passed as arguments to a step.
• isBuilding now supports manually triggered builds via
  isBuilding('manually'). manually takes several options
  • isBuilding(manually: false, combined: true) - a boolean where if true returns the
    username of the user who triggered the build and is boolean truthy. If
    false it will return true if the build was triggered by anything except a
    user, manually.
  • isBuilding(manually: 'samrocketman', combined: true) - returns true only
    if the build was manually triggered by user samrocketman.
• isBuilding now supports a combined boolean status of all filters for
  easing use in pipelines logic. Example of filtering for manually triggered
  tags is isBuilding(manually: true, tag: '/.*/', combined: true) which
  instead of returning a HashMap of the results for each filter it will return a
  single boolean. Returns true if all examples were true and false if any
  filter was not true.
• isBuilding now supports a List. See also documentation in the Jervis
  wiki.

See the full CHANGELOG...

Build environment

$ head -n1 /etc/issue
Ubuntu 18.04.4 LTS

$ lsb_release -d
Description:	Pop!_OS 18.04 LTS

$ uname -rms
Linux 5.3.0-7642-generic x86_64

$ java -version
openjdk version "1.8.0_242"
OpenJDK Runtime Environment (build 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08)
OpenJDK 64-Bit Server VM (build 25.242-b08, mixed mode)

$ ./gradlew  -version

------------------------------------------------------------
Gradle 5.6.3
------------------------------------------------------------

Build time:   2019-10-18 00:28:36 UTC
Revision:     bd168bbf5d152c479186a897f2cea494b7875d13

Kotlin:       1.3.41
Groovy:       2.5.4
Ant:          Apache Ant(TM) version 1.9.14 compiled on March 12 2019
JVM:          1.8.0_242 (Private Build 25.242-b08)
OS:           Linux 5.3.0-7642-generic amd64

Releases signed with 8D8BF0E242D8A068572EBF3CE8F732347257E65F.

jervis-1.6

jervis 1.6 - Nov 10th, 2019

New features:

• net.gleske.jervis.remotes.GitHubGraphQL has a new method
  getJervisYamlFiles which allows a caller to get multiple Jervis YAML files
  from multiple branches in a single API call. It is overloaded to return a
  list of files in each branch as well.

Pipeline DSL scripts changes in the vars/ folder:

• New pipeline step isBuilding() which provides a versatile conditional which
  allows a Jenkins pipeline user to filter the type of build that the current
  runtime is. e.g. Cron build, PR build, tag build, branch build... It supports
  checking tags and branches against a filter (literal or regex) so that it
  matches only tags and branches which have a specific name. See the header
  comment in vars/isBuilding.groovy which has
  documentation and examples for how to use the step.

Job DSL scripts changes in the jobs/ folder:

• Lots of code cleanup. Generating jobs just got a lot simpler! With the
  release of the scm-filter-jervis plugin we can now rely on on-the-fly branch
  detection. So we can generate jobs without pre-populating the repository with
  YAML. Rather than exhaustively listing everything that was deleted please
  review this git diff of code cleanup.

Other notes

• For tests, net.gleske.jervis.remotes.StaticMocking now supports mocking
  GraphQL and can render a different response depending on the query passed to
  the GraphQL mock. See GitHubGraphQLTest.groovy
  for an example of how it is used in tests.
• For tests, ./gradlew console now includes test classes in its classpath to
  help with debugging net.gleske.jervis.remotes.StaticMocking.

See the full CHANGELOG...

Releases signed with 8D8BF0E242D8A068572EBF3CE8F732347257E65F.

jervis-1.5

jervis 1.5 - Nov 9th, 2019

New features:

• AutoRelease support class added which provides automated bumping for versions.
  This is useful in Jenkins pipeline scripts by giving the next version if given
  a version and a set of existing Git tags. It supports unconventional
  continuos release and semantic versioning. It supports tags having prefixes
  like v1.0 (prefix being v) and suffixes like v1.0-beta. The API
  documentation for the AutoRelease class has several examples for how this new
  feature behaves.

See the full CHANGELOG...

Releases signed with 8D8BF0E242D8A068572EBF3CE8F732347257E65F.

Jervis 1.4

Release notes

See CHANGELOG.

Build Environment:

$ head -n1 /etc/issue
Ubuntu 18.04.3 LTS

$ lsb_release -d
Description:	Pop!_OS 18.04 LTS

$ uname -rms
Linux 5.0.0-31-generic x86_64

$ java -version
openjdk version "1.8.0_222"
OpenJDK Runtime Environment (build 1.8.0_222-8u222-b10-1ubuntu1~18.04.1-b10)
OpenJDK 64-Bit Server VM (build 25.222-b10, mixed mode)

$ ./gradlew  -version

------------------------------------------------------------
Gradle 5.6.3
------------------------------------------------------------

Build time:   2019-10-18 00:28:36 UTC
Revision:     bd168bbf5d152c479186a897f2cea494b7875d13

Kotlin:       1.3.41
Groovy:       2.5.4
Ant:          Apache Ant(TM) version 1.9.14 compiled on March 12 2019
JVM:          1.8.0_222 (Private Build 25.222-b10)
OS:           Linux 5.0.0-31-generic amd64

Releases signed with 8D8BF0E242D8A068572EBF3CE8F732347257E65F.

Jervis 1.3

Release notes

See CHANGELOG.

Build Environment:

$ head -n1 /etc/issue
Ubuntu 18.04.3 LTS

$ lsb_release -d
Description:	Pop!_OS 18.04 LTS

$ uname -rms
Linux 5.0.0-31-generic x86_64

$ java -version
openjdk version "1.8.0_222"
OpenJDK Runtime Environment (build 1.8.0_222-8u222-b10-1ubuntu1~18.04.1-b10)
OpenJDK 64-Bit Server VM (build 25.222-b10, mixed mode)

$ ./gradlew  -version

------------------------------------------------------------
Gradle 5.6.3
------------------------------------------------------------

Build time:   2019-10-18 00:28:36 UTC
Revision:     bd168bbf5d152c479186a897f2cea494b7875d13

Kotlin:       1.3.41
Groovy:       2.5.4
Ant:          Apache Ant(TM) version 1.9.14 compiled on March 12 2019
JVM:          1.8.0_222 (Private Build 25.222-b10)
OS:           Linux 5.0.0-31-generic amd64

Releases signed with 8D8BF0E242D8A068572EBF3CE8F732347257E65F.

Jervis 1.2

Release notes

See CHANGELOG.

Build Environment:

$ head -n1 /etc/issue
Ubuntu 16.04.4 LTS

$ uname -rms
Linux 4.4.0-130-generic x86_64

$ ./gradlew clean Jar
Using Groovy 2.4.15
Using Gradle 2.11
Using Java 1.8.0_131 (Oracle Corporation 25.131-b11)

$ java -version
java version "1.8.0_131"
Java(TM) SE Runtime Environment (build 1.8.0_131-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)

Releases signed with 8D8BF0E242D8A068572EBF3CE8F732347257E65F.

Jervis 1.1

Release notes

See CHANGELOG.

Build Environment:

$ head -n1 /etc/issue
Ubuntu 16.04.3 LTS

$ uname -rms
Linux 4.10.0-37-generic x86_64

$ ./gradlew clean check uploadArchives
Using Groovy 2.4.12
Using Gradle 2.11
Using Java 1.8.0_131 (Oracle Corporation 25.131-b11)

$ java -version
java version "1.8.0_131"
Java(TM) SE Runtime Environment (build 1.8.0_131-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)

Releases signed with 8D8BF0E242D8A068572EBF3CE8F732347257E65F.

Jervis 1.0

Release notes

See CHANGELOG.

Build Environment:

$ head -n1 /etc/issue
Ubuntu 16.04.3 LTS

$ uname -rms
Linux 4.10.0-37-generic x86_64

$ ./gradlew clean check uploadArchives
Using Groovy 2.4.12
Using Gradle 2.11
Using Java 1.8.0_131 (Oracle Corporation 25.131-b11)

$ java -version
java version "1.8.0_131"
Java(TM) SE Runtime Environment (build 1.8.0_131-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)

Releases signed with 8D8BF0E242D8A068572EBF3CE8F732347257E65F.