Node add isn't consistently working

[gsstoykov]

To Reproduce

Initialisation steps from #727 and:

npm run solo -- node add --gossip-keys true --tls-keys true --release-tag v0.54.0-alpha.4 --namespace solo-e2e

Describe the bug

◼ Finalize
node:internal/process/promises:289
            triggerUncaughtException(err, true /* fromPromise */);
            ^

[UnhandledPromiseRejection: This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). The promise rejected with the reason "#<ErrorEvent>".] {
  code: 'ERR_UNHANDLED_REJECTION'
}

Node.js v21.7.1

I've seen fails with the error from #727 as well.

Describe the expected behavior

Node added and functioning. Does not happen every time but still it is not consistent for testing from our side.

Whole JUnit/CLI Logs

npm run solo -- node add --gossip-keys true --tls-keys true --release-tag v0.54.0-alpha.4 --namespace solo-e2e

> @hashgraph/[email protected] solo
> NODE_OPTIONS=--experimental-vm-modules node --no-deprecation solo.mjs node add --gossip-keys true --tls-keys true --release-tag v0.54.0-alpha.4 --namespace solo-e2e


******************************* Solo *********************************************
Version			: 0.31.0
Kubernetes Context	: kind-solo-e2e
Kubernetes Cluster	: kind-solo-e2e
Kubernetes Namespace	: solo-e2e
**********************************************************************************
✔ Initialize [0.1s]
✔ Check that PVCs are enabled
✔ Identify existing network nodes
  ✔ Check network pod: node1
✔ Determine new node account number
✔ Generate Gossip key [0.3s]
  ✔ Backup old files
  ✔ Gossip key for node: node2 [0.3s]
✔ Generate gRPC TLS key [0.4s]
  ✔ Backup old files
  ✔ TLS key for node: node2 [0.4s]
✔ Load signing key certificate
✔ Compute mTLS certificate hash
✔ Prepare gossip endpoints
✔ Prepare grpc service endpoints
✔ Prepare upgrade zip file for node upgrade process [2s]
✔ Check existing nodes staked amount [2s]
✔ Send node create transaction [2s]
✔ Send prepare upgrade transaction [4s]
✔ Send freeze upgrade transaction [2s]
✔ Download generated files from an existing node [0.5s]
✔ Prepare staging directory
  ✔ Copy Gossip keys to staging
  ✔ Copy gRPC TLS keys to staging
✔ Copy node keys to secrets [0.1s]
  ✔ Copy TLS keys [0.1s]
  ✔ Node: node1
    ✔ Copy Gossip keys
  ✔ Node: node2
    ✔ Copy Gossip keys
✔ Check network nodes are frozen [9s]
  ✔ Check network pod: node1  - status FREEZE_COMPLETE, attempt: 3/120 [9s]
✔ Get node logs and configs [2s]
✔ Deploy new network node [5s]
✔ Kill nodes to pick up updated configMaps
✔ Check node pods are running [58s]
  ✔ Check Node: node1
  ✔ Check Node: node2 [58s]
❯ Fetch platform software into all network nodes
  ⠇ Update node: node1 [ platformVersion = v0.54.0-alpha.4 ]
  ⠇ Update node: node2 [ platformVersion = v0.54.0-alpha.4 ]
◼ Download last state from an existing node
◼ Upload last saved state to new network node
◼ Setup new network node
◼ Start network nodes
◼ Enable port forwarding for JVM debugger
◼ Check all nodes are ACTIVE
◼ Check all node proxies are ACTIVE
◼ Stake new node
◼ Trigger stake weight calculate
◼ Finalize
node:internal/process/promises:289
            triggerUncaughtException(err, true /* fromPromise */);
            ^

[UnhandledPromiseRejection: This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). The promise rejected with the reason "#<ErrorEvent>".] {
  code: 'ERR_UNHANDLED_REJECTION'
}

Node.js v21.7.1

Additional Context

No response