Skip to content

Commit

Permalink
feat(autoscaling): Unified parameters to set scaled options for brows…
Browse files Browse the repository at this point in the history 
…er nodes

Signed-off-by: Viet Nguyen Duc <[email protected]>
  • Loading branch information
@VietND96
VietND96 committed Nov 2, 2023
1 parent cef5534 commit d14879b
Show file tree
Hide file tree
Showing 10 changed files with 124 additions and 98 deletions.
24 changes: 16 additions & 8 deletions charts/selenium-grid/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -109,8 +109,13 @@ This table contains the configuration parameters of the chart and their default
| `ingress.tls` | `[]` | TLS backend configuration for ingress resource |
| `autoscaling.enableWithExistingKEDA` | `false` | Enable autoscaling of browser nodes. |
| `autoscaling.enabled` | `false` | Same as above plus installation of KEDA |
| `autoscaling.scalingType` | `job` | Which typ of KEDA scaling to use: job or deployment |
| `autoscaling.scaledJobOptions` | See `values.yaml` | Options for KEDA ScaledJobs |
| `autoscaling.scalingType` | `job` | Which typ of KEDA scaling to use: `job` or `deployment` |
| `autoscaling.scaledOptions` | See `values.yaml` | Common options for KEDA scaled resources (both ScaledJobs and ScaledObjects) |
| `autoscaling.scaledOptions.minReplicaCount` | `0` | Min number of replicas that each browser nodes has when autoscaling |
| `autoscaling.scaledOptions.maxReplicaCount` | `8` | Max number of replicas that each browser nodes can auto scale up to |
| `autoscaling.scaledOptions.pollingInterval` | `10` | The interval to check each trigger on |
| `autoscaling.scaledJobOptions` | See `values.yaml` | Options for KEDA ScaledJobs (when `scalingType` is set to `job`) |
| `autoscaling.scaledObjectOptions` | See `values.yaml` | Options for KEDA ScaledObjects (when `scalingType` is set to `deployment`) |
| `autoscaling.deregisterLifecycle` | See `values.yaml` | Lifecycle applied to pods of deployments controlled by KEDA. Makes the node deregister from selenium hub |
| `chromeNode.enabled` | `true` | Enable chrome nodes |
| `chromeNode.deploymentEnabled` | `true` | Enable creation of Deployment for chrome nodes |
Expand Down Expand Up @@ -146,8 +151,9 @@ This table contains the configuration parameters of the chart and their default
| `chromeNode.hpa.url` | `{{ include "seleniumGrid.graphqlURL" . }}` | Graphql Url of the hub or the router |
| `chromeNode.hpa.browserName` | `chrome` | BrowserName from the capability |
| `chromeNode.hpa.browserVersion` | `` | BrowserVersion from the capability |
| `chromeNode.maxReplicaCount` | `8` | Max number of replicas that this browsernode can auto scale up to |
| `chromeNode.minReplicaCount` | `1` | Min number of replicas that this browsernode has when jobs are running |
| `chromeNode.scaledOptions` | See `values.yaml` | Override the global `autoscaling.scaledOptions` with specific scaled options for chrome nodes |
| `chromeNode.scaledJobOptions` | See `values.yaml` | Override the global `autoscaling.scaledJobOptions` with specific scaled options for chrome nodes |
| `chromeNode.scaledObjectOptions` | See `values.yaml` | Override the global `autoscaling.scaledObjectOptions` with specific scaled options for chrome nodes |
| `firefoxNode.enabled` | `true` | Enable firefox nodes |
| `firefoxNode.deploymentEnabled` | `true` | Enable creation of Deployment for firefox nodes |
| `firefoxNode.replicas` | `1` | Number of firefox nodes. Disabled if autoscaling is enabled. |
Expand Down Expand Up @@ -182,8 +188,9 @@ This table contains the configuration parameters of the chart and their default
| `firefoxNode.hpa.url` | `{{ include "seleniumGrid.graphqlURL" . }}` | Graphql Url of the hub or the router |
| `firefoxNode.hpa.browserName` | `firefox` | BrowserName from the capability |
| `firefoxNode.hpa.browserVersion` | `` | BrowserVersion from the capability |
| `firefoxNode.maxReplicaCount` | `8` | Max number of replicas that this browsernode can auto scale up to |
| `firefoxNode.minReplicaCount` | `1` | Min number of replicas that this browsernode has when jobs are running |
| `firefoxNode.scaledOptions` | See `values.yaml` | Override the global `autoscaling.scaledOptions` with specific scaled options for firefox nodes |
| `firefoxNode.scaledJobOptions` | See `values.yaml` | Override the global `autoscaling.scaledJobOptions` with specific scaled options for firefox nodes |
| `firefoxNode.scaledObjectOptions` | See `values.yaml` | Override the global `autoscaling.scaledObjectOptions` with specific scaled options for firefox nodes |
| `edgeNode.enabled` | `true` | Enable edge nodes |
| `edgeNode.deploymentEnabled` | `true` | Enable creation of Deployment for edge nodes |
| `edgeNode.replicas` | `1` | Number of edge nodes. Disabled if autoscaling is enabled. |
Expand Down Expand Up @@ -218,8 +225,9 @@ This table contains the configuration parameters of the chart and their default
| `edgeNode.hpa.url` | `{{ include "seleniumGrid.graphqlURL" . }}` | Graphql Url of the hub or the router |
| `edgeNode.hpa.browserName` | `edge` | BrowserName from the capability |
| `edgeNode.hpa.browserVersion` | `` | BrowserVersion from the capability |
| `edgeNode.maxReplicaCount` | `8` | Max number of replicas that this browsernode can auto scale up to |
| `edgeNode.minReplicaCount` | `1` | Min number of replicas that this browsernode has when jobs are running |
| `edgeNode.scaledOptions` | See `values.yaml` | Override the global `autoscaling.scaledOptions` with specific scaled options for edge nodes |
| `edgeNode.scaledJobOptions` | See `values.yaml` | Override the global `autoscaling.scaledJobOptions` with specific scaled options for edge nodes |
| `edgeNode.scaledObjectOptions` | See `values.yaml` | Override the global `autoscaling.scaledObjectOptions` with specific scaled options for edge nodes |
| `videoRecorder.enabled` | `false` | Enable video recorder for node |
| `videoRecorder.imageName` | `selenium/video` | Selenium video recoder image name |
| `videoRecorder.imageTag` | `ffmpeg-6.0-20231102` | Image tag of video recorder |
Expand Down
41 changes: 41 additions & 0 deletions charts/selenium-grid/templates/_helpers.tpl
View file
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,47 @@ Is autoscaling using KEDA enabled
{{- or .Values.autoscaling.enabled .Values.autoscaling.enableWithExistingKEDA | ternary "true" "" -}}
{{- end -}}

{{/*
Common autoscaling spec template
*/}}
{{- define "seleniumGrid.autoscalingTemplate" -}}
{{- $spec := toYaml (dict) -}}
{{/* Merge with precedence from right to left */}}
{{- with .Values.autoscaling.scaledOptions -}}
{{- $spec = mergeOverwrite ($spec | fromYaml) . | toYaml -}}
{{- end -}}
{{- with .node.scaledOptions -}}
{{- $spec = mergeOverwrite ($spec | fromYaml) . | toYaml -}}
{{- end -}}
{{- if eq .Values.autoscaling.scalingType "deployment" -}}
{{- with .Values.autoscaling.scaledObjectOptions -}}
{{- $spec = mergeOverwrite ($spec | fromYaml) . | toYaml -}}
{{- end -}}
{{- with .node.scaledObjectOptions -}}
{{- $spec = mergeOverwrite ($spec | fromYaml) . | toYaml -}}
{{- end -}}
{{- $spec = mergeOverwrite ($spec | fromYaml) (dict "scaleTargetRef" (dict "name" .name)) | toYaml -}}
{{- else if eq .Values.autoscaling.scalingType "job" -}}
{{- with .Values.autoscaling.scaledJobOptions -}}
{{- $spec = mergeOverwrite ($spec | fromYaml) . | toYaml -}}
{{- end -}}
{{- with .node.scaledJobOptions -}}
{{- $spec = mergeOverwrite ($spec | fromYaml) . | toYaml -}}
{{- end -}}
{{- $spec = mergeOverwrite ($spec | fromYaml) (dict "jobTargetRef" .podTemplate) | toYaml -}}
{{- end -}}
{{- if and $spec (ne $spec "{}") -}}
{{ tpl $spec $ }}
{{- end -}}
{{- if not .Values.autoscaling.scaledOptions.triggers }}
triggers:
- type: selenium-grid
{{- with .node.hpa }}
metadata: {{- tpl (toYaml .) $ | nindent 6 }}
{{- end }}
{{- end }}
{{- end -}}

{{/*
Common pod template
*/}}
Expand Down
13 changes: 4 additions & 9 deletions charts/selenium-grid/templates/chrome-node-hpa.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,13 +11,8 @@ metadata:
labels:
deploymentName: {{ template "seleniumGrid.chromeNode.fullname" . }}
spec:
maxReplicaCount: {{ .Values.chromeNode.maxReplicaCount }}
minReplicaCount: {{ .Values.chromeNode.minReplicaCount }}
scaleTargetRef:
name: {{ template "seleniumGrid.chromeNode.fullname" . }}
triggers:
- type: selenium-grid
{{- with .Values.chromeNode.hpa }}
metadata: {{- tpl (toYaml .) $ | nindent 8 }}
{{- end }}
{{- $podScope := deepCopy . -}}
{{- $_ := set $podScope "name" (include "seleniumGrid.chromeNode.fullname" .) -}}
{{- $_ = set $podScope "node" .Values.chromeNode -}}
{{- include "seleniumGrid.autoscalingTemplate" $podScope | nindent 2 }}
{{- end }}
24 changes: 6 additions & 18 deletions charts/selenium-grid/templates/chrome-node-scaledjobs.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -19,22 +19,10 @@ metadata:
{{- toYaml . | nindent 4 }}
{{- end }}
spec:
maxReplicaCount: {{ .Values.chromeNode.maxReplicaCount }}
{{- with .Values.autoscaling.scaledJobOptions -}}
{{ toYaml . | nindent 2 }}
{{- end }}
triggers:
- type: selenium-grid
{{- with .Values.chromeNode.hpa }}
metadata: {{- tpl (toYaml .) $ | nindent 8 }}
{{- end }}
jobTargetRef:
parallelism: 1
completions: 1
backoffLimit: 0
{{- $podScope := deepCopy . -}}
{{- $_ := set $podScope "name" "selenium-chrome-node" -}}
{{- $_ = set $podScope "node" .Values.chromeNode -}}
{{- $_ = set $podScope "uploader" (get .Values.videoRecorder (.Values.videoRecorder.uploader | toString)) -}}
{{- include "seleniumGrid.podTemplate" $podScope | nindent 4 }}
{{- $podScope := deepCopy . -}}
{{- $_ := set $podScope "name" "selenium-chrome-node" -}}
{{- $_ = set $podScope "node" .Values.chromeNode -}}
{{- $_ = set $podScope "uploader" (get .Values.videoRecorder (.Values.videoRecorder.uploader | toString)) -}}
{{- $_ = set $podScope "podTemplate" (include "seleniumGrid.podTemplate" $podScope | fromYaml) }}
{{- include "seleniumGrid.autoscalingTemplate" $podScope | nindent 2 }}
{{- end }}
13 changes: 4 additions & 9 deletions charts/selenium-grid/templates/edge-node-hpa.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,13 +11,8 @@ metadata:
labels:
deploymentName: {{ template "seleniumGrid.edgeNode.fullname" . }}
spec:
maxReplicaCount: {{ .Values.edgeNode.maxReplicaCount }}
minReplicaCount: {{ .Values.edgeNode.minReplicaCount }}
scaleTargetRef:
name: {{ template "seleniumGrid.edgeNode.fullname" . }}
triggers:
- type: selenium-grid
{{- with .Values.edgeNode.hpa }}
metadata: {{- tpl (toYaml .) $ | nindent 8 }}
{{- end }}
{{- $podScope := deepCopy . -}}
{{- $_ := set $podScope "name" (include "seleniumGrid.edgeNode.fullname" .) -}}
{{- $_ = set $podScope "node" .Values.edgeNode -}}
{{- include "seleniumGrid.autoscalingTemplate" $podScope | nindent 2 }}
{{- end }}
24 changes: 6 additions & 18 deletions charts/selenium-grid/templates/edge-node-scaledjob.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -19,22 +19,10 @@ metadata:
{{- toYaml . | nindent 4 }}
{{- end }}
spec:
maxReplicaCount: {{ .Values.edgeNode.maxReplicaCount }}
{{- with .Values.autoscaling.scaledJobOptions -}}
{{ toYaml . | nindent 2 }}
{{- end }}
triggers:
- type: selenium-grid
{{- with .Values.edgeNode.hpa }}
metadata: {{- tpl (toYaml .) $ | nindent 8 }}
{{- end }}
jobTargetRef:
parallelism: 1
completions: 1
backoffLimit: 0
{{- $podScope := deepCopy . -}}
{{- $_ := set $podScope "name" "selenium-edge-node" -}}
{{- $_ = set $podScope "node" .Values.edgeNode -}}
{{- $_ = set $podScope "uploader" (get .Values.videoRecorder (.Values.videoRecorder.uploader | toString)) -}}
{{- include "seleniumGrid.podTemplate" $podScope | nindent 4 }}
{{- $podScope := deepCopy . -}}
{{- $_ := set $podScope "name" "selenium-edge-node" -}}
{{- $_ = set $podScope "node" .Values.edgeNode -}}
{{- $_ = set $podScope "uploader" (get .Values.videoRecorder (.Values.videoRecorder.uploader | toString)) -}}
{{- $_ = set $podScope "podTemplate" (include "seleniumGrid.podTemplate" $podScope | fromYaml) }}
{{- include "seleniumGrid.autoscalingTemplate" $podScope | nindent 2 }}
{{- end }}
2 changes: 1 addition & 1 deletion charts/selenium-grid/templates/event-bus-configmap.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,6 @@ metadata:
{{- toYaml . | nindent 4 }}
{{- end }}
data:
SE_EVENT_BUS_HOST: {{ $eventBusHost }}
SE_EVENT_BUS_HOST: {{ $eventBusHost | quote }}
SE_EVENT_BUS_PUBLISH_PORT: {{ $eventBusPublishPort | quote }}
SE_EVENT_BUS_SUBSCRIBE_PORT: {{ $eventBusSubscribePort | quote }}
13 changes: 4 additions & 9 deletions charts/selenium-grid/templates/firefox-node-hpa.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,13 +11,8 @@ metadata:
labels:
deploymentName: {{ template "seleniumGrid.firefoxNode.fullname" . }}
spec:
maxReplicaCount: {{ .Values.firefoxNode.maxReplicaCount }}
minReplicaCount: {{ .Values.firefoxNode.minReplicaCount }}
scaleTargetRef:
name: {{ template "seleniumGrid.firefoxNode.fullname" . }}
triggers:
- type: selenium-grid
{{- with .Values.firefoxNode.hpa }}
metadata: {{- tpl (toYaml .) $ | nindent 8 }}
{{- end }}
{{- $podScope := deepCopy . -}}
{{- $_ := set $podScope "name" (include "seleniumGrid.firefoxNode.fullname" .) -}}
{{- $_ = set $podScope "node" .Values.firefoxNode -}}
{{- include "seleniumGrid.autoscalingTemplate" $podScope | nindent 2 }}
{{- end }}
24 changes: 6 additions & 18 deletions charts/selenium-grid/templates/firefox-node-scaledjob.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -19,22 +19,10 @@ metadata:
{{- toYaml . | nindent 4 }}
{{- end }}
spec:
maxReplicaCount: {{ .Values.firefoxNode.maxReplicaCount }}
{{- with .Values.autoscaling.scaledJobOptions -}}
{{ toYaml . | nindent 2 }}
{{- end }}
triggers:
- type: selenium-grid
{{- with .Values.firefoxNode.hpa }}
metadata: {{- tpl (toYaml .) $ | nindent 8 }}
{{- end }}
jobTargetRef:
parallelism: 1
completions: 1
backoffLimit: 0
{{- $podScope := deepCopy . -}}
{{- $_ := set $podScope "name" "selenium-firefox-node" -}}
{{- $_ = set $podScope "node" .Values.firefoxNode -}}
{{- $_ = set $podScope "uploader" (get .Values.videoRecorder (.Values.videoRecorder.uploader | toString)) -}}
{{- include "seleniumGrid.podTemplate" $podScope | nindent 4 }}
{{- $podScope := deepCopy . -}}
{{- $_ := set $podScope "name" "selenium-firefox-node" -}}
{{- $_ = set $podScope "node" .Values.firefoxNode -}}
{{- $_ = set $podScope "uploader" (get .Values.videoRecorder (.Values.videoRecorder.uploader | toString)) -}}
{{- $_ = set $podScope "podTemplate" (include "seleniumGrid.podTemplate" $podScope | fromYaml) }}
{{- include "seleniumGrid.autoscalingTemplate" $podScope | nindent 2 }}
{{- end }}
Loading

0 comments on commit d14879b

Please sign in to comment.