If pendingPodConditions
isn't set, KEDA never detects any pending jobs
#6157
Labels
bug
Something isn't working
Report
If
pendingPodConditions
is not set on aScaledJob
with strategyaccurate
, KEDA appears to incorrectly set the number of pending jobs as always being 0. This means that if a job takes longer to start up than the poll time of the trigger, you end up with duplicate jobs.Expected Behavior
When using
scalingStrategy.strategy: accurate
, I'd expect KEDA to correctly count the number of pods that had been scheduled but are not yet running, and calculate the quantity to scale up by asQueueLength-RunningJobs-PendingJobs
.Actual Behavior
If
pendingPodConditions
is not set in yourScaledJob
, KEDA will always calculate the number of pending jobs as being 0, resulting in duplicate jobs if your job has a long startup time.Steps to Reproduce the Problem
pending
stateAdd 3 items to the trigger such that KEDA will scale up to 3 replicas. These 3 replicas should stay in the 'pending' state, however the KEDA operator will log
"Number of pending Jobs": 0
After 5 seconds (the next poll), KEDA will attempt to launch 3 additional replicas (bringing the total to 6) because it does not see the pending jobs.
Reset and modify your ScaledJob to add all possible
pendingPodConditions
:Logs from KEDA operator
15s poll time with
pendingPodConditions
unset, comments mine:15s poll time with
pendingPodConditions
set to all conditions, only part of the log but shows it's working correctly:KEDA Version
2.15.1
Kubernetes Version
1.28
Platform
Amazon Web Services
Scaler Details
SQS FIFO queue
Anything else?
I dug into this a tiny bit and I think the culprit is here. It looks like this function might be returning that a pod is running or complete when it's actually pending.
My conclusion that this is a bug is based on how the default behaviour is described here:
The text was updated successfully, but these errors were encountered: