StackStorm v3.8.1 pre-release testing

[arm4b]

We're preparing the StackStorm v3.8.1 and starting pre-release testing.

StackStorm didn't have a release for a year! The focus in 3.8.1 was on updating the upstream dependencies and fixing bugs for stackstorm components like st2 core and orquesta workflow engine (pip), st2web Web UI and st2chatops (npm). Because lots of dependencies were updated during a long period of time, regressions are possible. Help us find them!

TL;DR

Install StackStorm v3.8.1 staging packages, try random things in different OS (CentOS/RedHat/RockyLinux 6 and 7, Ubuntu 18 and 20 LTS) and report any regressions found:

bash <(curl -sSL https://stackstorm.com/packages/install.sh) --user=st2admin --password=Ch@ngeMe --staging --stable

Release Process Preparation

Per Release Management Schedule @armab is the Release Manager and @nzlosh is the Release Assistant. We follow the StackStorm Release Process. Communication is happening in #releasemgmt and #development Slack channels.

Why Manual testing?

StackStorm has a lot of testing stages: Unit tests, Integration, Deployment/Integrity checks, Smoke tests and eventually end-2-end tests when automation spins up new AWS instance for each OS/flavor we support (see st2tests, st2ci, st2cd and st2cicd).

However it's not enough.
There are always unknowns to discover and edge cases. Hence, manual Exploratory Testing.

What to test?

• st2 core testing:
  • updated pip dependencies to fix upstream CVEs
  • many other bugfixes (15+)
  • Recommendation: test overall system stability
• st2chatops testing:
  • updated npm dependencies to fix upstream CVEs
  • Recommendation: test if update didn't broke anything with your chatops adapter
• st2web testing:
  • updates npm dependencies to fix upstream CVEs
  • Recommendation: test for Web UI for regressions or quirks
• Run st2-self-check
  https://docs.stackstorm.com/latest/troubleshooting/self_verification.html

Full Changelog

For reference, here is a full changelog. Recommended to explore, check and try in a random way.

st2

Fixed

• Fix proxy auth mode in HA environments #5766 #6049
  Contributed by @floatingstatic

• Fix issue with linux pack actions failed to run remotely due to incorrect python shebang. #5983 #6042
  Contributed by Ronnie Hoffmann (@ZoeLeah Schwarz IT KG)

• Fix CI usses #6015
  Contributed by Amanda McGuinness (@amanda11 intive)

• Bumped paramiko to 2.10.5 to fix an issue with SSH Certs - SHA2 support broke SHA1 support when server is OpenSSH <7.8 paramiko/paramiko#2017 (security)
  Contributed by @jk464

• Avoid logging sensitive information in debug (fix #5977)

• Fix codecov failures for stackstorm/st2 tests. #6035, #6046, #6048

• Fix #4676, edge case where --inherit-env is skipped if the action has no parameters

• Fix ST2 Client for Windows Clients. PWD is a Unix only Libary. #6071
  Contributed by (@philipphomberger Schwarz IT KG)

• Fix Snyk Security Finding Cross-site Scripting (XSS) in contrib/examples/sensors/echo_flask_app.py #6070
  Contributed by (@philipphomberger Schwarz IT KG)

• Update cryptography 3.4.7 -> 39.0.1, pyOpenSSL 21.0.0 -> 23.1.0, paramiko 2.10.5 -> 2.11.0 (security). #6055

• Bumped eventlet to 0.33.3 and gunicorn to 21.2.0 to fix RecursionError bug in setting SSLContext minimum_version property. (security) #6061
  Contributed by @jk464

• Update orquesta to v1.6.0 to fix outdated dependencies (security). #6050

• Fix KV value lookup in actions when RBAC is enabled #5934

• Update version 3.1.15 of gitpython to 3.1.18 for py3.6 and to 3.1.37 for py3.8 (security). #6063

• Update importlib-metadata from 3.10.1 to 4.8.3 for py3.6 and to 4.10.1 for py3.8 (security). #6072
  Contributed by @jk464

• For "local-shell-script" runner, on readonly filesystems, don't attempt to run chmod +x on script_action. Fixes #5591
  Contributed by @jk464

Added

• Move git clone to user_home/.st2packs #5845

• Error on st2ctl status when running in Kubernetes. #5851
  Contributed by @mamercad

• Continue introducing pants <https://www.pantsbuild.org/docs>_ to improve DX (Developer Experience)
  working on StackStorm, improve our security posture, and improve CI reliability thanks in part
  to pants' use of PEX lockfiles. This is not a user-facing addition.
  #5778 #5789 #5817 #5795 #5830 #5833 #5834 #5841 #5840 #5838 #5842 #5837 #5849 #5850
  #5846 #5853 #5848 #5847 #5858 #5857 #5860 #5868 #5871 #5864 #5874 #5884 #5893 #5891
  #5890 #5898 #5901 #5906 #5899 #5907 #5909 #5922 #5926 #5927 #5925 #5928 #5929 #5930
  #5931 #5932 #5948 #5949 #5950
  Contributed by @cognifloyd

• Added a joint index to solve the problem of slow mongo queries for scheduled executions. #5805

• Added publisher to ActionAlias to enable streaming ActionAlias create/update/delete events. #5763
  Contributed by @ubaumann

• Expose environment variable ST2_ACTION_DEBUG to all StackStorm actions.
  Contributed by @maxfactor1

• Python 3.9 support. #5730
  Contributed by Amanda McGuinness (@amanda11 intive)

• Run the st2 self-check in Github Actions and support the environment variable TESTS_TO_SKIP to skip tests when running st2-self-check. #5609
  Contributed by @winem

Changed

• Remove distutils dependencies across the project. #5992
  Contributed by @AndroxxTraxxon

Full list of changes: https://github.com/StackStorm/st2/blob/v3.8/CHANGELOG.rst

orquesta

Changed

• Update deprecated collections imports to collections.abc to be forward-compatible with Python3.10
  Contributed by @AndroxxTraxxon
• Migrate from nosetest to pytest for Python test runner.
  Contributed by @AndroxxTraxxon
• Add Python versions 3.9, 3.10, and 3.11 to the test matrix
  Contributed by @AndroxxTraxxon

Fixed

• Update networkx >=2.6 for Python 3.8 to fix insecure deserialization #255 (security fix)
  Contributed by @Stealthii
• Update jsonschema requirements to allow 3.2 (security fix)
  Contributed by @james-bellamy

Fore more info see https://github.com/StackStorm/orquesta/blob/master/CHANGELOG.rst#160

st2chatops

• Massive dependencies update to fix upstream CVEs Update dependencies - 2nd try (security)  st2chatops#184, Update dependencies - 1st try (security) st2chatops#183
• If you're using ChatOps - testing is highly recommended as all the ChatOps adapters/dependencies were updated!

st2web

Changed

• Updated various dependencies (security). #1009, #1020
  Contributed by @enykeev
• Updated NodeJS to v20 current (security). #1010
  Contributed by @enykeev

Fixed

• Fixed CircleCI tests by pinning [email protected]. #1008
  Contributed by @guzzijones

---

Please report if you did any testing and any share findings here.
Good luck!

[arm4b]

I've installed StackStorm staging packages on a fresh Ubuntu18 and Ubuntu20 VMs.
Verified the correct st2 v3.8.1 package version is present that includes correct pip dependencies that we updated (with no vulnerabilities under py3.8) and ran st2-self-check.

There were some leftower warnings about deprecated py3.6 coming from cryptography, which we didn't silence in full, but this is the latest release with py3.6 support so warnings shouldn't be that harmful, if they're not breaking any scripts/parsing.

TBD: need verifying if Web UI works as before as there were lots of dependency updates there.

[nzlosh]

Installed 3.8.1 packages and ran self-tests on CentOS7 and Rocky8. Both working

centos 7

SELF CHECK SUCCEEDED!
st2-self-check succeeded.

#############################################################
###################################################   #######
###############################################   /~\   #####
############################################   _- `~~~', ####
##########################################  _-~       )  ####
#######################################  _-~          |  ####
####################################  _-~            ;  #####
##########################  __---___-~              |   #####
#######################   _~   ,,                  ;  `,,  ##
#####################  _-~    ;'                  |  ,'  ; ##
###################  _~      '                    `~'   ; ###
############   __---;                                 ,' ####
########   __~~  ___                                ,' ######
#####  _-~~   -~~ _                               ,' ########
##### `-_         _                              ; ##########
#######  ~~----~~~   ;                          ; ###########
#########  /          ;                        ; ############
#######  /             ;                      ; #############
#####  /                `                    ; ##############
###  /                                      ; ###############
#                                            ################

[root@centos7 ~]# st2 --version
st2 3.8.1, on Python 3.6.8
[root@centos7 ~]# egrep 'NAME|VERSION' /etc/os-release 
NAME="CentOS Linux"
VERSION="7 (Core)"

Rocky8

SELF CHECK SUCCEEDED!
st2-self-check succeeded.

#############################################################
###################################################   #######
###############################################   /~\   #####
############################################   _- `~~~', ####
##########################################  _-~       )  ####
#######################################  _-~          |  ####
####################################  _-~            ;  #####
##########################  __---___-~              |   #####
#######################   _~   ,,                  ;  `,,  ##
#####################  _-~    ;'                  |  ,'  ; ##
###################  _~      '                    `~'   ; ###
############   __---;                                 ,' ####
########   __~~  ___                                ,' ######
#####  _-~~   -~~ _                               ,' ########
##### `-_         _                              ; ##########
#######  ~~----~~~   ;                          ; ###########
#########  /          ;                        ; ############
#######  /             ;                      ; #############
#####  /                `                    ; ##############
###  /                                      ; ###############
#                                            ################

[root@rocky8 ~]# st2 --version
st2 3.8.1, on Python 3.8.17
[root@rocky8 ~]# cat /etc/os-release 
NAME="Rocky Linux"
VERSION="8.9 (Green Obsidian)"

I'll look at doing some manual testing later

[winem]

st2-self-check as well as a bunch of manual tests went fine on Ubuntu 20.04.