diff --git a/charts/logzio-metrics-collector/CHANGELOG.md b/charts/logzio-metrics-collector/CHANGELOG.md new file mode 100644 index 00000000..476bcac6 --- /dev/null +++ b/charts/logzio-metrics-collector/CHANGELOG.md @@ -0,0 +1,4 @@ +# Changelog + +- **0.0.1**: + - Initial release \ No newline at end of file diff --git a/charts/logzio-metrics-collector/Chart.yaml b/charts/logzio-metrics-collector/Chart.yaml new file mode 100644 index 00000000..63cac839 --- /dev/null +++ b/charts/logzio-metrics-collector/Chart.yaml @@ -0,0 +1,10 @@ +apiVersion: v2 +name: logzio-metrics-collector +version: 0.0.1 +description: Kubernetes metrics collection agent for Logz.io based on opentelemetry collector +type: application +home: https://logz.io/ +maintainers: + - name: Raul Gurshumov + email: raul.gurshumo@logz.io +appVersion: 0.106.1 diff --git a/charts/logzio-metrics-collector/README.md b/charts/logzio-metrics-collector/README.md new file mode 100644 index 00000000..c0d9c8e5 --- /dev/null +++ b/charts/logzio-metrics-collector/README.md @@ -0,0 +1,161 @@ +# logzio-metrics-collector + +Kubernetes metrics collection agent for Logz.io based on OpenTelemetry Collector. + +## Prerequisites + +- Kubernetes 1.24+ +- Helm 3.9+ + +Below is the extended README.md, the full configuration table based on the provided `values.yaml` is in [VALUES.md](VALUES.md) file, release updates posted in the [CHANGELOG.md](CHANGELOG.md) file. + + +* * * + +Logz.io Metrics Collector for Kubernetes +======================================== + +The `logzio-metrics-collector` Helm chart deploys a Kubernetes metrics collection agent designed to forward metrics from Kubernetes clusters to Logz.io. This solution leverages the OpenTelemetry Collector, providing a robust and flexible way to manage metric data, ensuring that your monitoring infrastructure scales with your application needs. + +It's pre-configured to send metrics to Logz.io, simplifying setup and integration. It also populates data for prebuilt content in the Logz.io platform. + +Getting Started +--------------- + +### Add Logz.io Helm Repository + +Before installing the chart, add the Logz.io Helm repository: + +``` +helm repo add logzio-helm https://logzio.github.io/logzio-helm +helm repo update +``` + +### Installation + +1. **Create the Logz.io Secret** + + If not managing secrets externally, create the Logz.io secret with your shipping token and other relevant information. + +2. **Install the Chart** + + Install `logzio-metrics-collector` from the Logz.io Helm repository, specifying the authentication values: + + ``` + helm install logzio-metrics-collector -n monitoring --create-namespace \ + --set enabled=true \ + --set secrets.logzioMetricsToken="<>" \ + --set secrets.logzioRegion="<>" \ + --set secrets.env_id="<>" \ + logzio-helm/logzio-metrics-collector + ``` + + Replace: + * `logzio-metrics-collector` with your release name + * `<>` with your Logz.io metrics shipping token + * `<>` with your Logz.io [account region code](https://docs.logz.io/docs/user-guide/admin/hosting-regions/account-region/) + * `<>` with a unique name assigned to your environment's identifier, to differentiate telemetry data across various environments + + +### Uninstalling the Chart + +To uninstall/delete the `logzio-metrics-collector` deployment: + +```shell +helm delete -n monitoring logzio-metrics-collector +``` + +### Configure customization options + +You can use the following options to update the Helm chart values [parameters](VALUES.md): + +* Specify parameters using the `--set key=value[,key=value]` argument to `helm install` + +* Edit the `values.yaml` + +* Overide default values with your own `my_values.yaml` and apply it in the `helm install` command. + +### Deploy metrics chart with Kuberenetes object logs correlation + +**Note**: `k8sObjectsLogs.enabled=true` will have no effect unless `enabled` is also set to `true` + +``` +helm install logzio-metrics-collector -n monitoring --create-namespace \ +--set enabled=true \ +--set k8sObjectsLogs.enabled=true \ +--set secrets.k8sObjectsLogsToken="<>" \ +--set secrets.logzioMetricsToken="<>" \ +--set secrets.logzioRegion="<>" \ +--set secrets.env_id="<>" \ +logzio-helm/logzio-metrics-collector +``` + +Replace: +* `logzio-metrics-collector` with your release name +* `<>` with your Logz.io metrics shipping token +* `<>` with your Logz.io logs shipping token +* `<>` with your Logz.io [account region code](https://docs.logz.io/docs/user-guide/admin/hosting-regions/account-region/) +* `<>` with a unique name assigned to your environment's identifier, to differentiate telemetry data across various environments + + +### For clusters with Windows Nodes + + +To extract and scrape metrics from Windows Nodes, a Windows Exporter service must be installed on the node host. This installation is accomplished by authenticating with a username and password via an SSH connection to the node through a job. + +By default, the Windows installer job will execute upon deployment and subsequently every 10 minutes, retaining the most recent failed and successful pods. +You can modify these settings in the `values.yaml` file: + +``` +windowsExporterInstallerJob: + interval: "*/10 * * * *" #In CronJob format + concurrencyPolicy: Forbid # Future cronjob will run only after current job is finished + successfulJobsHistoryLimit: 1 + failedJobsHistoryLimit: 1 +``` + +The default username for Windows Node pools is: `azureuser`. (This username and password are shared across all Windows node pools.) + +You can change the password for your Windows node pools in the AKS cluster using the following command (this will only affect Windows node pools): + +``` + az aks update \ + --resource-group $RESOURCE_GROUP \ + --name $CLUSTER_NAME \ + --windows-admin-password $NEW_PW +``` + +You can read more information at https://docs.microsoft.com/en-us/azure/aks/windows-faq, +under `How do I change the administrator password for Windows Server nodes on my cluster?` section. + + +###### Run the Helm deployment code for clusters with Windows Nodes: + +``` +helm install logzio-metrics-collector -n monitoring --create-namespace \ +--set enabled=true \ +--set secrets.windowsNodeUsername="<" \ +--set secrets.windowsNodePassword="<>" \ +--set secrets.logzioMetricsToken="<>" \ +--set secrets.logzioRegion="<>" \ +--set secrets.env_id="<>" \ +logzio-helm/logzio-metrics-collector +``` + +Replace: +* `logzio-metrics-collector` with your release name +* `<>` with your Logz.io metrics shipping token +* `<>` with your Logz.io [account region code](https://docs.logz.io/docs/user-guide/admin/hosting-regions/account-region/) +* `<>` with a unique name assigned to your environment's identifier, to differentiate telemetry data across various environments +* `<>` with the username for the Node pool you want the Windows exporter to be installed on. +* `<>` with the password for the Node pool you want the Windows exporter to be installed on. + + +### Handling image pull rate limit +In some cases (i.e spot clusters) where the pods/nodes are replaced frequently, the pull rate limit for images pulled from dockerhub might be reached, with an error: +`You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limits`. +In these cases we can use the following `--set` commands to use an alternative image repository: + +```shell +--set image.repository=ghcr.io/open-telemetry/opentelemetry-collector-releases/opentelemetry-collector-contrib +``` diff --git a/charts/logzio-metrics-collector/VALUES.md b/charts/logzio-metrics-collector/VALUES.md new file mode 100644 index 00000000..1476c021 --- /dev/null +++ b/charts/logzio-metrics-collector/VALUES.md @@ -0,0 +1,114 @@ +# logzio-metrics-collector + +The table below lists the configurable parameters of the `logzio-metrics-collector` chart and their default values. + +| Key | Description | Default Value | +|--------------------------|----------------------------------------------------------------------------------|----------------------------------------| +| `enabled` | Toggle for enabling the Helm chart deployment. | `true` | +| `nameOverride` | Override the default name for the deployment. | `""` | +| `fullnameOverride` | Set a full name override for the deployment. | `""` | +| `mode` | Deployment mode ("daemonset" or "standalone"). | `daemonset` | +| `namespaceOverride` | Override the namespace into which the resources will be deployed. | `""` | +| `secrets.enabled` | Toggle for creating and managing the Logz.io secret by this chart. | `true` | +| `secrets.name` | The name of the secret for Logz.io metrics collector. | `logzio-metric-collector-secrets` | +| `secrets.env_id` | Environment identifier attribute added to all metrics. | `my_env` | +| `secrets.logzioMetricsToken` | Secret with your Logz.io metrics shipping token. | `<>` | +| `secrets.logzioRegion` | Secret with your Logz.io region. | `us` | +| `secrets.k8sObjectsLogsToken` | Secret with your Logz.io logs shipping token, optional for Kuebrnetes object logs and metrics correlation, set `k8sObjectsLogs.enabled` to `true`. | `<>` | +| `secrets.customEndpoint` | Secret with your custom endpoint, overrides Logz.io region listener address. | `""` | +| `secrets.windowsNodeUsername` | Secret with your Windows node username. | `""` | +| `secrets.windowsNodePassword` | Secret with your Windows node password. | `""` | +| `configMap.create` | Specifies whether a configMap should be created. | `true` | +| `baseConfig` | Base collector configuration, supports templating. | Complex structure (see `values.yaml`) | +| `daemonsetConfig` | Configuration for OpenTelemetry Collector DaemonSet. | Complex structure (see `values.yaml`) | +| `standaloneConfig` | Configuration for standalone OpenTelemetry Collector. | Complex structure (see `values.yaml`) | +| `image.repository` | Docker image repository. | `otel/opentelemetry-collector-contrib` | +| `image.pullPolicy` | Image pull policy. | `IfNotPresent` | +| `image.tag` | Overrides the image tag. | `""` | +| `image.digest` | Pull images by digest. | `""` | +| `imagePullSecrets` | Specifies image pull secrets. | `[]` | +| `command.name` | OpenTelemetry Collector executable. | `otelcol-contrib` | +| `command.extraArgs` | Additional arguments for the command. | `[]` | +| `serviceAccount.create` | Specifies whether a service account should be created. | `true` | +| `serviceAccount.name` | The name of the service account to use. | `""` | +| `serviceAccount.annotations` | Annotations to add to the service account. | `{}` | +| `clusterRole.create` | Specifies whether a cluster role should be created. | `true` | +| `clusterRole.name` | The name of the cluster role to use. | `""` | +| `clusterRole.rules` | Access rules of the cluster role. | `[]` | +| `clusterRole.annotations` | Annotations to add to the cluster role. | `{}` | +| `clusterRole.clusterRoleBinding.annotations` | Annotations to add to the cluster role binding. | `{}` | +| `clusterRole.clusterRoleBinding.name` | The name of the cluster role binding to use. | `""` | +| `podSecurityContext` | Security context policies for the pod. | `{}` | +| `securityContext` | Security context policies for the container. | `{}` | +| `nodeSelector` | Node labels for pod assignment. | `{}` | +| `tolerations` | Tolerations for pod assignment. | `[]` | +| `affinity` | Affinity rules for pod assignment. | Complex structure (see `values.yaml`) | +| `priorityClassName` | Scheduler priority class name. | `""` | +| `extraEnvs` | Extra environment variables to set in the pods. | `[]` | +| `ports` | Defines ports configurations. | Complex structure (see `values.yaml`) | +| `resources` | CPU/memory resource requests/limits. | Default: `requests.cpu:50m`, `requests.memory:70Mi` | +| `podAnnotations` | Annotations to add to the pod | `{}` | +| `daemonsetCollector.configOverride` | Configuration override for DaemonSet collector. | `{}` | +| `daemonsetCollector.affinity`| Affinity rules for DaemonSet pod placement. | Complex structure (see `values.yaml`) | +| `daemonsetCollector.resources` | CPU/memory resource requests/limits for DaemonSet. | Default: `requests.cpu:50m`, `requests.memory:70Mi` | +| `daemonsetCollector.podLabels` | Labels to add to the DaemonSet pod. | `{}` | +| `daemonsetCollector.podAnnotations` | Annotations to add to the DaemonSet pod. | `{}` | +| `standaloneCollector.configOverride` | Configuration override for standalone collector. | `{}` | +| `standaloneCollector.replicas` | Number of replicas for the standalone collector. | `1` | +| `standaloneCollector.resources` | CPU/memory resource requests/limits for standalone collector. | Default: `requests.cpu:50m`, `requests.memory:70Mi` | +| `standaloneCollector.podLabels` | Labels to add to the standalone pod. | `{}` | +| `standaloneCollector.podAnnotations` | Annotations to add to the standalone pod. | `{}` | +| `applicationMetrics.enabled` | Enable sending application metrics. | `false` | +| `k8sObjectsLogs.enabled` | Enable Kubernetes objects logging. | `false` | +| `k8sObjectsLogs.config` | Configuration for Kubernetes objects logging. | Complex structure (see `values.yaml`) | +| `networkPolicy.enabled` | Enable NetworkPolicy creation. | `false` | +| `networkPolicy.annotations` | Annotations to add to the NetworkPolicy. | `{}` | +| `networkPolicy.allowIngressFrom` | Configure the 'from' clause of the NetworkPolicy. | `[]` | +| `networkPolicy.extraIngressRules` | Add additional ingress rules to specific ports. | `[]` | +| `networkPolicy.egressRules` | Restrict egress traffic from the OpenTelemetry collector pod. | `[]` | +| `useGOMEMLIMIT` | Set GOMEMLIMIT env var to a percentage of resources.limits.memory. | `false` | +| `opencost.enabled` | Enable OpenCost integration. | `false` | +| `opencost.config` | Configuration for OpenCost integration. | Complex structure (see `values.yaml`) | +| `enableMetricsFilter.gke` | Enable metrics filtering for Google Kubernetes Engine. | `false` | +| `enableMetricsFilter.eks` | Enable metrics filtering for Amazon Elastic Kubernetes Service. | `false` | +| `enableMetricsFilter.aks` | Enable metrics filtering for Azure Kubernetes Service. | `false` | +| `enableMetricsFilter.dropKubeSystem` | Drop kube-system metrics. | `false` | +| `prometheusFilters.metrics.infrastructure.keep.aks` | Metrics to keep for AKS infrastructure pipeline. | Complex structure (see `values.yaml`) | +| `prometheusFilters.metrics.infrastructure.keep.eks` | Metrics to keep for EKS infrastructure pipeline. | Complex structure (see `values.yaml`) | +| `prometheusFilters.metrics.infrastructure.keep.gke` | Metrics to keep for GKE infrastructure pipeline. | Complex structure (see `values.yaml`) | +| `prometheusFilters.metrics.infrastructure.drop.custom` | Custom metrics to drop for infrastructure pipeline. | `""` | +| `prometheusFilters.namespaces.infrastructure.keep.custom` | Custom namespaces to keep for infrastructure pipeline. | `""` | +| `prometheusFilters.namespaces.infrastructure.drop.kubeSystem` | Drop kube-system namespace. | `kube-system` | +| `initContainers` | List of init container specs. | `[]` | +| `extraContainers` | List of extra sidecars to add. | `[]` | +| `hostNetwork` | Use the host's network namespace. | `false` | +| `dnsPolicy` | Pod DNS policy. | `""` | +| `dnsConfig` | Custom DNS config. | `{}` | +| `hostAliases` | Adding entries to Pod /etc/hosts with HostAliases. | `[]` | +| `extraEnvsFrom` | Extra environment variables to set in the pods from a source. | `[]` | +| `extraVolumes` | Extra volumes to add to the pods. | `[]` | +| `extraVolumeMounts` | Extra volume mounts to add to the pods. | `[]` | +| `additionalLabels` | Common labels to add to all otel-collector resources. | `{}` | +| `podMonitor.enabled` | Enable the creation of a PodMonitor. | `false` | +| `podMonitor.metricsEndpoints` | Metrics endpoints configuration for PodMonitor. | Complex structure (see `values.yaml`) | +| `podMonitor.extraLabels` | Additional labels for the PodMonitor. | `{}` | +| `rollout.rollingUpdate` | Rolling update strategy for deployments. | `{}` | +| `rollout.strategy` | Deployment strategy for rolling updates. | `RollingUpdate` | +| `service.enabled` | Enable the creation of a Service. | `true` | +| `service.type` | Type of service to create. | `ClusterIP` | +| `service.annotations` | Annotations to add to the Service. | `{}` | +| `service.externalTrafficPolicy` | External traffic policy for LoadBalancer service. | `Cluster` | +| `service.internalTrafficPolicy` | Internal traffic policy for DaemonSet service. | `Local` | +| `service.loadBalancerIP` | LoadBalancer IP if `service.type` is `LoadBalancer`. | `""` | +| `service.loadBalancerSourceRanges` | Source ranges for LoadBalancer service. | `[]` | +| `ingress.enabled` | Enable ingress controller resource. | `false` | +| `ingress.annotations` | Annotations to add to the ingress. | `{}` | +| `ingress.hosts` | List of ingress hosts. | `[]` | +| `ingress.tls` | TLS configuration for the ingress. | `[]` | +| `ingress.ingressClassName` | Name of the ingress class to use. | `""` | +| `ingress.additionalIngresses` | Additional ingress configurations. | `[]` | +| `windowsExporterInstallerJob.interval` | Interval at which the Windows Exporter Installer Job runs. | `"*/10 * * * *"` | +| `windowsExporterInstallerJob.concurrencyPolicy` | Concurrency policy for the Windows Exporter Installer Job. | `"Forbid"` | +| `windowsExporterInstallerJob.successfulJobsHistoryLimit` | Number of successful Windows Exporter Installer jobs to retain. | `1` | +| `windowsExporterInstallerJob.failedJobsHistoryLimit` | Number of failed Windows Exporter Installer jobs to retain. | `1` | +| `windowsExporterInstallerJob.ttlSecondsAfterFinished` | Time to live in seconds for the Windows Exporter Installer Job. | `3600` | \ No newline at end of file diff --git a/charts/logzio-metrics-collector/templates/NOTES.txt b/charts/logzio-metrics-collector/templates/NOTES.txt new file mode 100644 index 00000000..7d2f5da5 --- /dev/null +++ b/charts/logzio-metrics-collector/templates/NOTES.txt @@ -0,0 +1,25 @@ +{{- if and (eq .Values.dnsPolicy "None") (not .Values.dnsConfig) }} +{{- fail "[ERROR] dnsConfig should be provided when dnsPolicy is None" }} +{{ end }} + +{{/* validate extensions must include health_check */}} +{{- if not (has "health_check" .Values.baseConfig.service.extensions) }} +{{ fail "[ERROR] The logzio-metrics-collector chart requires that the health_check extension to be included in the extension list." }} +{{- end}} + +{{- if not .Values.configMap.create }} +[WARNING] "configMap" will not be created and "config" will not take effect. +{{ end }} + +{{- if not .Values.resources }} +[WARNING] No resource limits or requests were set. Consider setter resource requests and limits for your collector(s) via the `resources` field. +{{ end }} + +{{- if and (eq .Values.mode "daemonset") (eq .Values.service.internalTrafficPolicy "Cluster") }} +[WARNING] Setting internalTrafficPolicy to 'Cluster' on Daemonset is not recommended. Consider using 'Local' instead. +{{ end }} + +{{- if and (.Values.useGOMEMLIMIT) (not ((((.Values.resources).limits).memory))) }} +[WARNING] "useGOMEMLIMIT" is enabled but memory limits have not been supplied, which means no GOMEMLIMIT env var was configured but the Memory Ballast Extension was removed. It is highly recommended to only use "useGOMEMLIMIT" when memory limits have been set. +{{ end }} + diff --git a/charts/logzio-metrics-collector/templates/_config.tpl b/charts/logzio-metrics-collector/templates/_config.tpl new file mode 100644 index 00000000..57bd3432 --- /dev/null +++ b/charts/logzio-metrics-collector/templates/_config.tpl @@ -0,0 +1,254 @@ +# Merge user supplied config. +{{- define "metrics-collector.baseConfig" -}} +{{- .Values.baseConfig | toYaml }} +{{- end }} + + +{{/* Build the list of port for service */}} +{{- define "metrics-collector.servicePortsConfig" -}} +{{- $ports := deepCopy .Values.ports }} +{{- range $key, $port := $ports }} +{{- if $port.enabled }} +- name: {{ $key }} + port: {{ $port.servicePort }} + targetPort: {{ $port.containerPort }} + protocol: {{ $port.protocol }} + {{- if $port.appProtocol }} + appProtocol: {{ $port.appProtocol }} + {{- end }} +{{- if $port.nodePort }} + nodePort: {{ $port.nodePort }} +{{- end }} +{{- end }} +{{- end }} +{{- end }} + +{{/* Build the list of port for pod */}} +{{- define "metrics-collector.podPortsConfig" -}} +{{- $ports := deepCopy .Values.ports }} +{{- range $key, $port := $ports }} +{{- if $port.enabled }} +- name: {{ $key }} + containerPort: {{ $port.containerPort }} + protocol: {{ $port.protocol }} + {{- if and $.isAgent $port.hostPort }} + hostPort: {{ $port.hostPort }} + {{- end }} +{{- end }} +{{- end }} +{{- end }} + +{{/* +Create pipeline job filters +Param 1: dict: "pipeline" infrastructure/applications & global context +*/}} +{{- define "metrics-collector.getPipelineFilters" -}} + +{{/*pipelines's metrics keep filters*/}} +{{- $pipeline := .pipeline -}} +{{- $metricsKeepFilters := (dict "source_labels" (list "__name__") "action" "keep") -}} +{{- if (and .Values.enableMetricsFilter.aks (eq $pipeline "infrastructure")) -}} + {{- $_ := set $metricsKeepFilters ("regex" ) .Values.prometheusFilters.metrics.infrastructure.keep.aks -}} +{{- else if (and .Values.enableMetricsFilter.eks (eq $pipeline "infrastructure")) -}} + {{- $_ := set $metricsKeepFilters ("regex" ) .Values.prometheusFilters.metrics.infrastructure.keep.eks -}} +{{- else if (and .Values.enableMetricsFilter.gke (eq $pipeline "infrastructure")) -}} + {{- $_ := set $metricsKeepFilters ("regex" ) .Values.prometheusFilters.metrics.infrastructure.keep.gke -}} +{{- end -}} +{{- $customKeep := index .Values "prometheusFilters" "metrics" $pipeline "keep" "custom" -}} +{{- if $customKeep -}} + {{- if (hasKey $metricsKeepFilters "regex" ) -}} + {{- $_ := set $metricsKeepFilters "regex" (print (get $metricsKeepFilters ("regex") ) "|" $customKeep ) -}} + {{- else -}} + {{- $_ := set $metricsKeepFilters "regex" $customKeep -}} + {{- end -}} +{{- end -}} + +{{/*pipeline's metrics drop filters*/}} +{{- $metricsDropFilters := (dict "source_labels" (list "__name__") "action" "drop") -}} +{{- $customDrop := index .Values "prometheusFilters" "metrics" $pipeline "drop" "custom" -}} +{{- if $customDrop -}} + {{- $_ := set $metricsDropFilters "regex" $customDrop -}} +{{- end -}} + +{{/*pipeline's namespace keep filters*/}} +{{- $namespaceKeepFilters := (dict "source_labels" (list "namespace") "action" "keep") -}} +{{- $customKeep = index .Values "prometheusFilters" "namespaces" $pipeline "keep" "custom" -}} +{{- if $customKeep -}} + {{- $_ := set $namespaceKeepFilters "regex" $customKeep -}} +{{- end -}} + +{{/*pipeline's namespace drop filters*/}} +{{- $namespaceDropFilters := (dict "source_labels" (list "namespace") "action" "drop") -}} +{{- if (and .Values.enableMetricsFilter.dropKubeSystem (eq $pipeline "infrastructure")) -}} + {{- $_ := set $namespaceDropFilters ("regex" ) .Values.prometheusFilters.namespaces.infrastructure.drop.kubeSystem -}} +{{- end -}} +{{- $customDrop = index .Values "prometheusFilters" "namespaces" $pipeline "drop" "custom" -}} +{{- if $customDrop -}} + {{- if (hasKey $namespaceDropFilters "regex" ) -}} + {{- $_ := set $namespaceDropFilters "regex" (print (get $namespaceDropFilters ("regex") ) "|" $customDrop ) -}} + {{- else -}} + {{- $_ := set $namespaceDropFilters "regex" $customDrop -}} + {{- end -}} +{{- end -}} + +{{/*pipeline's service keep filters - only valid for infrastructure pipelines!*/}} +{{- $serviceKeepFilters := (dict "source_labels" (list "__meta_kubernetes_service_name") "action" "keep") -}} +{{- $serviceDropFilters := (dict "source_labels" (list "__meta_kubernetes_service_name") "action" "drop") -}} +{{- if eq $pipeline "infrastructure" -}} + {{- $customKeep = index .Values "prometheusFilters" "services" $pipeline "keep" "custom" -}} + {{- if $customKeep -}} + {{- $_ := set $serviceKeepFilters "regex" $customKeep -}} + {{- end -}} + + {{/*pipeline's service drop filters*/}} + {{- if (and .Values.disableKubeDnsScraping (eq $pipeline "infrastructure")) -}} + {{- $_ := set $serviceDropFilters ("regex" ) .Values.prometheusFilters.services.infrastructure.drop.kubeDns -}} + {{- end -}} + {{- $customDrop = index .Values "prometheusFilters" "services" $pipeline "drop" "custom" -}} + {{- if $customDrop -}} + {{- if (hasKey $serviceDropFilters "regex" ) -}} + {{- $_ := set $serviceDropFilters "regex" (print (get $serviceDropFilters ("regex") ) "|" $customDrop ) -}} + {{- else -}} + {{- $_ := set $serviceDropFilters "regex" $customDrop -}} + {{- end -}} + {{- end -}} +{{- end -}} + +{{/*remove empty filters*/}} +{{/*use the "metric" prefix for dict keys to associate filter with "metric_relabel_config"*/}} +{{- $allFilters := dict "metric1" $metricsKeepFilters "metric2" $metricsDropFilters "metric3" $namespaceKeepFilters "metric4" $namespaceDropFilters "5" $serviceDropFilters "6" $serviceKeepFilters -}} +{{- $checkedFilters := dict -}} +{{- range $key,$filter := $allFilters -}} +{{/*check if regex key exists, if so filter also exist*/}} + {{- if (hasKey $filter "regex" ) -}} + {{- $_ := set $checkedFilters $key $filter -}} + {{- end -}} +{{- end -}} +{{- $res := $checkedFilters | toJson -}} +{{- $res -}} +{{- end -}} + +{{/* +Build config file for standalone OpenTelemetry Collector +*/}} +{{- define "metrics-collector.standaloneCollectorConfig" -}} +{{- $configData := dict }} +{{- $standaloneConfig := deepCopy .Values.standaloneConfig | mustMergeOverwrite }} +{{- $values := deepCopy .Values.standaloneCollector | mustMergeOverwrite (deepCopy .Values) }} +{{- $data := dict "Values" $values | mustMergeOverwrite (deepCopy .) }} +{{- $config := include "metrics-collector.baseConfig" $data | fromYaml }} +{{- $ctxParams := dict "pipeline" "infrastructure" -}} +{{- $ctxParams = merge $ctxParams $ -}} +{{- $infraFilters := include "metrics-collector.getPipelineFilters" $ctxParams -}} +{{- $ctxParams = dict "pipeline" "applications" -}} +{{- $ctxParams = merge $ctxParams $ -}} +{{- $applicationsFilters := include "metrics-collector.getPipelineFilters" $ctxParams -}} + +{{/* Handle opencost config */}} +{{- if .Values.opencost.enabled -}} +{{- $opencostConfig := deepCopy .Values.opencost.config | mustMergeOverwrite -}} +{{- $standaloneConfig = deepCopy $opencostConfig | merge $standaloneConfig | mustMergeOverwrite -}} +{{/* merge processor list for opencost*/}} +{{- $_ := set (index $standaloneConfig "service" "pipelines" "metrics/infrastructure") "processors" (concat (index $standaloneConfig "service" "pipelines" "metrics/infrastructure" "processors") (index $opencostConfig "service" "pipelines" "metrics/infrastructure" "processors" )) -}} +{{- end -}} + +{{/* Handle k8s objects config */}} +{{- if .Values.k8sObjectsLogs.enabled -}} +{{- $k8sObjectsLogsConfig := deepCopy .Values.k8sObjectsLogs.config | mustMergeOverwrite -}} +{{- $standaloneConfig = deepCopy $k8sObjectsLogsConfig | merge $standaloneConfig | mustMergeOverwrite -}} +{{- end -}} + +{{- if (and (eq .Values.mode "standalone") (.Values.enabled)) -}} +{{- $configData = $standaloneConfig }} +{{- end -}} + +{{- if (and (eq .Values.mode "standalone") (.Values.enabled)) -}} + {{- range $job := (index $configData "receivers" "prometheus/infrastructure" "config" "scrape_configs") -}} + {{- range $key,$filter := ($infraFilters | fromJson) -}} + {{- if contains "metric" $key -}} + {{- $_ := set $job ("metric_relabel_configs" | toYaml) ( append $job.metric_relabel_configs ($filter)) -}} + {{- else -}} + {{- $_ := set $job ("relabel_configs" | toYaml) ( append $job.relabel_configs ($filter)) -}} + {{- end -}} + {{- end -}} + {{- end -}} + + {{- range $job := (index $configData "receivers" "prometheus/applications" "config" "scrape_configs") -}} + {{- range $key,$filter := ($applicationsFilters | fromJson) -}} + {{- if contains "metric" $key -}} + {{- $_ := set $job ("metric_relabel_configs" | toYaml) ( append $job.metric_relabel_configs ($filter)) -}} + {{- else -}} + {{- $_ := set $job ("relabel_configs" | toYaml) ( append $job.relabel_configs ($filter)) -}} + {{- end -}} + {{- end -}} + {{- end -}} + {{- if .Values.applicationMetrics.enabled -}} + {{- $metricsApplications := dict "exporters" (list "prometheusremotewrite/applications") "processors" (list "attributes/env_id") "receivers" (list "prometheus/applications") -}} + {{- $_ := set .Values.standaloneConfig.service.pipelines "metrics/applications" $metricsApplications -}} + {{- end -}} +{{- end -}} +{{- .Values.standaloneCollector.configOverride | merge $configData | mustMergeOverwrite $config | toYaml}} +{{- end -}} + + +# Build config file for daemonset metrics Collector +{{- define "metrics-collector.daemonsetCollectorConfig" -}} +{{- $configData := dict }} +{{- $daemonsetConfig := deepCopy .Values.daemonsetConfig | mustMergeOverwrite }} +{{- $values := deepCopy .Values.daemonsetCollector | mustMergeOverwrite (deepCopy .Values) }} +{{- $data := dict "Values" $values | mustMergeOverwrite (deepCopy .) }} +{{- $config := include "metrics-collector.baseConfig" $data | fromYaml }} +{{- $ctxParams := dict "pipeline" "infrastructure" -}} +{{- $ctxParams = merge $ctxParams $ -}} +{{- $infraFilters := include "metrics-collector.getPipelineFilters" $ctxParams -}} +{{- $ctxParams = dict "pipeline" "applications" -}} +{{- $ctxParams = merge $ctxParams $ -}} +{{- $applicationsFilters := include "metrics-collector.getPipelineFilters" $ctxParams -}} + +{{/* Handle opencost config */}} +{{- if .Values.opencost.enabled -}} +{{- $opencostConfig := deepCopy .Values.opencost.config | mustMergeOverwrite -}} +{{- $daemonsetConfig = deepCopy $opencostConfig | merge $daemonsetConfig | mustMergeOverwrite -}} +{{/* merge processor list for opencost*/}} +{{- $_ := set (index $daemonsetConfig "service" "pipelines" "metrics/infrastructure") "processors" (concat (index $daemonsetConfig "service" "pipelines" "metrics/infrastructure" "processors") (index $opencostConfig "service" "pipelines" "metrics/infrastructure" "processors" )) -}} +{{- end -}} + +{{/* Handle k8s objects config */}} +{{- if .Values.k8sObjectsLogs.enabled -}} +{{- $k8sObjectsLogsConfig := deepCopy .Values.k8sObjectsLogs.config | mustMergeOverwrite -}} +{{- $daemonsetConfig = deepCopy $k8sObjectsLogsConfig | merge $daemonsetConfig | mustMergeOverwrite -}} +{{- end -}} + +{{- if (and (eq .Values.mode "daemonset") (.Values.enabled)) -}} +{{- $configData = $daemonsetConfig }} +{{- end -}} + +{{- if (and (eq .Values.mode "daemonset") (.Values.enabled)) -}} + {{- range $job := (index $configData "receivers" "prometheus/infrastructure" "config" "scrape_configs") -}} + {{- range $key,$filter := ($infraFilters | fromJson) -}} + {{- if contains "metric" $key -}} + {{- $_ := set $job ("metric_relabel_configs" | toYaml) ( append $job.metric_relabel_configs ($filter)) -}} + {{- else -}} + {{- $_ := set $job ("relabel_configs" | toYaml) ( append $job.relabel_configs ($filter)) -}} + {{- end -}} + {{- end -}} + {{- end -}} + + {{- range $job := (index $configData "receivers" "prometheus/applications" "config" "scrape_configs") -}} + {{- range $key,$filter := ($applicationsFilters | fromJson) -}} + {{- if contains "metric" $key -}} + {{- $_ := set $job ("metric_relabel_configs" | toYaml) ( append $job.metric_relabel_configs ($filter)) -}} + {{- else -}} + {{- $_ := set $job ("relabel_configs" | toYaml) ( append $job.relabel_configs ($filter)) -}} + {{- end -}} + {{- end -}} + {{- end -}} + {{- if .Values.applicationMetrics.enabled -}} + {{- $metricsApplications := dict "exporters" (list "prometheusremotewrite/applications") "processors" (list "attributes/env_id" ) "receivers" (list "prometheus/applications") -}} + {{- $_ := set .Values.daemonsetConfig.service.pipelines "metrics/applications" $metricsApplications -}} + {{- end -}} + +{{- end -}} + +{{- .Values.daemonsetCollector.configOverride | merge $configData | mustMergeOverwrite $config | toYaml}} +{{- end -}} \ No newline at end of file diff --git a/charts/logzio-metrics-collector/templates/_helpers.tpl b/charts/logzio-metrics-collector/templates/_helpers.tpl new file mode 100644 index 00000000..c2c9bb73 --- /dev/null +++ b/charts/logzio-metrics-collector/templates/_helpers.tpl @@ -0,0 +1,225 @@ +{{/* +Expand the name of the chart. +*/}} +{{- define "metrics-collector.name" -}} +{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }} +{{- end }} + +{{- define "metrics-collector.lowercase_chartname" -}} +{{- default .Chart.Name | lower }} +{{- end }} + +{{/* +Get component name +*/}} +{{- define "metrics-collector.component" -}} +{{- if eq .Values.mode "daemonset" -}} +component: metrics-collector +{{- else if eq .Values.mode "standalone" -}} +component: metrics-collector-standalone +{{- end -}} +{{- end }} + +{{/* +Create a default fully qualified app name. +We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec). +If release name contains chart name it will be used as a full name. +*/}} +{{- define "metrics-collector.fullname" -}} +{{- if .Values.fullnameOverride }} +{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }} +{{- else }} +{{- .Chart.Name | trunc 63 | trimSuffix "-" }} +{{- end }} +{{- end }} + +{{/* +Create chart name and version as used by the chart label. +*/}} +{{- define "metrics-collector.chart" -}} +{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }} +{{- end }} + +{{/* +Common labels +*/}} +{{- define "metrics-collector.labels" -}} +helm.sh/chart: {{ include "metrics-collector.chart" . }} +{{ include "metrics-collector.selectorLabels" . }} +{{- if .Chart.AppVersion }} +app.kubernetes.io/version: {{ .Chart.AppVersion | quote }} +{{- end }} +app.kubernetes.io/managed-by: {{ .Release.Service }} +{{ include "metrics-collector.additionalLabels" . }} +{{- end }} + +{{/* +Selector labels +*/}} +{{- define "metrics-collector.selectorLabels" -}} +app.kubernetes.io/name: {{ include "metrics-collector.name" . }} +app.kubernetes.io/instance: {{ .Release.Name }} +{{- end }} + +{{/* +Create the name of the service account to use +*/}} +{{- define "metrics-collector.serviceAccountName" -}} +{{- if .Values.serviceAccount.create }} +{{- default (include "metrics-collector.fullname" .) .Values.serviceAccount.name }} +{{- else }} +{{- default "default" .Values.serviceAccount.name }} +{{- end }} +{{- end }} + + +{{/* +Create the name of the clusterRole to use +*/}} +{{- define "metrics-collector.clusterRoleName" -}} +{{- default (include "metrics-collector.fullname" .) .Values.clusterRole.name }} +{{- end }} + +{{/* +Create the name of the clusterRoleBinding to use +*/}} +{{- define "metrics-collector.clusterRoleBindingName" -}} +{{- default (include "metrics-collector.fullname" .) .Values.clusterRole.clusterRoleBinding.name }} +{{- end }} + +{{- define "metrics-collector.podAnnotations" -}} +{{- if .Values.podAnnotations }} +{{- tpl (.Values.podAnnotations | toYaml) . }} +{{- end }} +{{- end }} + +{{- define "metrics-collector.podLabels" -}} +{{- if .Values.podLabels }} +{{- tpl (.Values.podLabels | toYaml) . }} +{{- end }} +{{- end }} + +{{- define "metrics-collector.additionalLabels" -}} +{{- if .Values.additionalLabels }} +{{- tpl (.Values.additionalLabels | toYaml) . }} +{{- end }} +{{- end }} + + +{{/* +Compute Service creation on mode +*/}} +{{- define "metrics-collector.serviceEnabled" }} + {{- $serviceEnabled := true }} + {{- if not (eq (toString .Values.service.enabled) "") }} + {{- $serviceEnabled = .Values.service.enabled -}} + {{- end }} + {{- if and (eq .Values.mode "daemonset") (not .Values.service.enabled) }} + {{- $serviceEnabled = false -}} + {{- end }} + + {{- print $serviceEnabled }} +{{- end -}} + + +{{/* +Compute InternalTrafficPolicy on Service creation +*/}} +{{- define "metrics-collector.serviceInternalTrafficPolicy" }} + {{- if and (eq .Values.mode "daemonset") (eq .Values.service.enabled true) }} + {{- print (.Values.service.internalTrafficPolicy | default "Local") -}} + {{- else }} + {{- print (.Values.service.internalTrafficPolicy | default "Cluster") -}} + {{- end }} +{{- end -}} + +{{/* +Allow the release namespace to be overridden +*/}} +{{- define "metrics-collector.namespace" -}} + {{- if .Values.namespaceOverride -}} + {{- .Values.namespaceOverride -}} + {{- else -}} + {{- .Release.Namespace -}} + {{- end -}} +{{- end -}} + +{{/* + This helper converts the input value of memory to Bytes. + Input needs to be a valid value as supported by k8s memory resource field. + */}} +{{- define "metrics-collector.convertMemToBytes" }} + {{- $mem := lower . -}} + {{- if hasSuffix "e" $mem -}} + {{- $mem = mulf (trimSuffix "e" $mem | float64) 1e18 -}} + {{- else if hasSuffix "ei" $mem -}} + {{- $mem = mulf (trimSuffix "e" $mem | float64) 0x1p60 -}} + {{- else if hasSuffix "p" $mem -}} + {{- $mem = mulf (trimSuffix "p" $mem | float64) 1e15 -}} + {{- else if hasSuffix "pi" $mem -}} + {{- $mem = mulf (trimSuffix "pi" $mem | float64) 0x1p50 -}} + {{- else if hasSuffix "t" $mem -}} + {{- $mem = mulf (trimSuffix "t" $mem | float64) 1e12 -}} + {{- else if hasSuffix "ti" $mem -}} + {{- $mem = mulf (trimSuffix "ti" $mem | float64) 0x1p40 -}} + {{- else if hasSuffix "g" $mem -}} + {{- $mem = mulf (trimSuffix "g" $mem | float64) 1e9 -}} + {{- else if hasSuffix "gi" $mem -}} + {{- $mem = mulf (trimSuffix "gi" $mem | float64) 0x1p30 -}} + {{- else if hasSuffix "m" $mem -}} + {{- $mem = mulf (trimSuffix "m" $mem | float64) 1e6 -}} + {{- else if hasSuffix "mi" $mem -}} + {{- $mem = mulf (trimSuffix "mi" $mem | float64) 0x1p20 -}} + {{- else if hasSuffix "k" $mem -}} + {{- $mem = mulf (trimSuffix "k" $mem | float64) 1e3 -}} + {{- else if hasSuffix "ki" $mem -}} + {{- $mem = mulf (trimSuffix "ki" $mem | float64) 0x1p10 -}} + {{- end }} +{{- $mem }} +{{- end }} + +{{- define "metrics-collector.gomemlimit" }} +{{- $memlimitBytes := include "metrics-collector.convertMemToBytes" . | mulf 0.8 -}} +{{- printf "%dMiB" (divf $memlimitBytes 0x1p20 | floor | int64) -}} +{{- end }} + + +{{/* +Calculate Logz.io listener address based on region +*/}} +{{- define "logzio.listenerAddress" -}} +{{- $region := .Values.secrets.logzioRegion -}} +{{- if eq $region "us" -}} +https://listener.logz.io:8053 +{{- else if eq $region "au" -}} +https://listener-au.logz.io:8053 +{{- else if eq $region "ca" -}} +https://listener-ca.logz.io:8053 +{{- else if eq $region "eu" -}} +https://listener-eu.logz.io:8053 +{{- else if eq $region "uk" -}} +https://listener-uk.logz.io:8053 +{{- else -}} +https://listener.logz.io:8053 # Default to us-east-1 region if no match +{{- end -}} +{{- end }} + +{{/* +Create k360 metrics list - will be used for K360 promethetus filters +If any OOB filters is being used the function return the OOB filter concatenated with custom keep infrastrucre filter +*/}} +{{- define "metrics-collector.k360Metrics" -}} +{{- $metrics := "" }} +{{- if .Values.enableMetricsFilter.aks }} + {{- $metrics = .Values.prometheusFilters.metrics.infrastructure.keep.aks }} +{{- else if .Values.enableMetricsFilter.gke}} + {{- $metrics = .Values.prometheusFilters.metrics.infrastructure.keep.gke }} +{{- else }} + {{- $metrics = .Values.prometheusFilters.metrics.infrastructure.keep.eks }} +{{- end -}} + +{{- if .Values.prometheusFilters.metrics.infrastructure.keep.custom }} + {{- $metrics = print $metrics "|" .Values.prometheusFilters.metrics.infrastructure.keep.custom }} +{{- end }} +{{- $metrics }} +{{- end }} \ No newline at end of file diff --git a/charts/logzio-metrics-collector/templates/_pod.tpl b/charts/logzio-metrics-collector/templates/_pod.tpl new file mode 100644 index 00000000..68a842fe --- /dev/null +++ b/charts/logzio-metrics-collector/templates/_pod.tpl @@ -0,0 +1,179 @@ +{{- define "metrics-collector.metricsPod" -}} +{{- with .Values.imagePullSecrets }} +imagePullSecrets: + {{- toYaml . | nindent 2 }} +{{- end }} +serviceAccountName: {{ include "metrics-collector.serviceAccountName" . }} +securityContext: + {{- toYaml .Values.podSecurityContext | nindent 2 }} +{{- with .Values.hostAliases }} +hostAliases: + {{- toYaml . | nindent 2 }} +{{- end }} +containers: + - name: {{ include "metrics-collector.lowercase_chartname" . }} + command: + - /{{ .Values.command.name }} + {{- if .Values.configMap.create }} + - --config=/conf/relay.yaml + {{- end }} + {{- range .Values.command.extraArgs }} + - {{ . }} + {{- end }} + securityContext: + {{- if not (.Values.securityContext) }} + runAsUser: 0 + runAsGroup: 0 + {{- else -}} + {{- toYaml .Values.securityContext | nindent 6 }} + {{- end }} + {{- if .Values.image.digest }} + image: "{{ ternary "" (print (.Values.global).imageRegistry "/") (empty (.Values.global).imageRegistry) }}{{ .Values.image.repository }}@{{ .Values.image.digest }}" + {{- else }} + image: "{{ ternary "" (print (.Values.global).imageRegistry "/") (empty (.Values.global).imageRegistry) }}{{ .Values.image.repository }}:{{ .Values.image.tag | default .Chart.AppVersion }}" + {{- end }} + imagePullPolicy: {{ .Values.image.pullPolicy }} + + {{- $ports := include "metrics-collector.podPortsConfig" . }} + {{- if $ports }} + ports: + {{- $ports | nindent 6}} + {{- end }} + env: + - name: MY_POD_IP + valueFrom: + fieldRef: + apiVersion: v1 + fieldPath: status.podIP + - name: KUBE_NODE_NAME + valueFrom: + fieldRef: + fieldPath: spec.nodeName + - name: K8S_360_METRICS + value: {{ include "metrics-collector.k360Metrics" . }} + - name: LOGZIO_AGENT_VERSION + value: {{.Chart.Version}} + - name: RELEASE_NAME + value: {{.Release.Name}} + - name: RELEASE_NS + value: {{.Release.Namespace}} + {{ if .Values.secrets.enabled}} + - name: LOGZIO_REGION + valueFrom: + secretKeyRef: + name: {{ .Values.secrets.name }} + key: logzio-listener-region + - name: LOGZIO_METRICS_TOKEN + valueFrom: + secretKeyRef: + name: {{ .Values.secrets.name }} + key: logzio-metrics-token + {{- if .Values.secrets.customEndpoint }} + - name: CUSTOM_ENDPOINT + valueFrom: + secretKeyRef: + name: {{ .Values.secrets.name }} + key: custom-endpoint + - name: LISTENER_URL + valueFrom: + secretKeyRef: + name: {{ .Values.secrets.name }} + key: custom-endpoint + {{- end -}} + {{- if .Values.secrets.k8sObjectsLogsToken }} + - name: LOGZIO_OBJECTS_LOGS_TOKEN + valueFrom: + secretKeyRef: + name: {{ .Values.secrets.name }} + key: logzio-k8s-objects-logs-token + {{- end -}} + {{- if .Values.secrets.env_id }} + - name: ENV_ID + valueFrom: + secretKeyRef: + name: {{ .Values.secrets.name }} + key: env-id + {{- end -}} + {{ end }} + - name: LISTENER_URL + value: {{ include "logzio.listenerAddress" . | quote }} + {{- if and (.Values.useGOMEMLIMIT) ((((.Values.resources).limits).memory)) }} + - name: GOMEMLIMIT + value: {{ include "metrics-collector.gomemlimit" .Values.resources.limits.memory | quote }} + {{- end }} + {{- with .Values.extraEnvs }} + {{- . | toYaml | nindent 6 }} + {{- end }} + {{- with .Values.extraEnvsFrom }} + envFrom: + {{- . | toYaml | nindent 6 }} + {{- end }} + {{- with .Values.resources }} + resources: + {{- toYaml . | nindent 6 }} + {{- end }} + volumeMounts: + {{- if .Values.configMap.create }} + - mountPath: /conf + name: {{ include "metrics-collector.lowercase_chartname" . }}-configmap + {{- end }} + - name: varlogpods + mountPath: /var/log/pods + readOnly: true + - name: varlibdockercontainers + mountPath: /var/lib/docker/containers + readOnly: true + - name: varlibotelcol + mountPath: /var/lib/otelcol + {{- if .Values.extraVolumeMounts }} + {{- toYaml .Values.extraVolumeMounts | nindent 6 }} + {{- end }} +{{- with .Values.extraContainers }} +{{- toYaml . | nindent 2 }} +{{- end }} +{{- if .Values.initContainers }} +initContainers: + {{- tpl (toYaml .Values.initContainers) . | nindent 2 }} +{{- end }} +{{- if .Values.priorityClassName }} +priorityClassName: {{ .Values.priorityClassName | quote }} +{{- end }} +volumes: + {{- if .Values.configMap.create }} + - name: {{ include "metrics-collector.lowercase_chartname" . }}-configmap + configMap: + name: {{ include "metrics-collector.fullname" . }}{{ .configmapSuffix }} + items: + - key: relay + path: relay.yaml + {{- end }} + - name: varlogpods + hostPath: + path: /var/log/pods + - name: varlibotelcol + hostPath: + path: /var/lib/otelcol + type: DirectoryOrCreate + - name: varlibdockercontainers + hostPath: + path: /var/lib/docker/containers + {{- if .Values.extraVolumes }} + {{- toYaml .Values.extraVolumes | nindent 2 }} + {{- end }} +{{- with .Values.nodeSelector }} +nodeSelector: + {{- toYaml . | nindent 2 }} +{{- end }} +{{- with .Values.affinity }} +affinity: + {{- toYaml . | nindent 2 }} +{{- end }} +{{- with .Values.tolerations }} +tolerations: + {{- toYaml . | nindent 2 }} +{{- end }} +{{- with .Values.topologySpreadConstraints }} +topologySpreadConstraints: + {{- toYaml . | nindent 2 }} +{{- end }} +{{- end }} diff --git a/charts/logzio-metrics-collector/templates/clusterrole.yaml b/charts/logzio-metrics-collector/templates/clusterrole.yaml new file mode 100644 index 00000000..011bfc49 --- /dev/null +++ b/charts/logzio-metrics-collector/templates/clusterrole.yaml @@ -0,0 +1,35 @@ +{{ if .Values.enabled}} +{{- if (.Values.clusterRole.create) -}} +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: {{ include "metrics-collector.clusterRoleName" . }} + labels: + {{- include "metrics-collector.labels" . | nindent 4 }} + {{- if .Values.clusterRole.annotations }} + annotations: + {{- range $key, $value := .Values.clusterRole.annotations }} + {{- printf "%s: %s" $key (tpl $value $ | quote) | nindent 4 }} + {{- end }} + {{- end }} +rules: + {{- if .Values.clusterRole.rules -}} + {{ toYaml .Values.clusterRole.rules | nindent 2 -}} + {{- end }} + - apiGroups: [""] + resources: ["events", "namespaces", "namespaces/status", "nodes", "nodes/stats", "nodes/spec","nodes/metrics", "pods", "pods/status", "pods/metrics", "resourcequotas","replicationcontrollers", "replicationcontrollers/status", "services","endpoints" ] + verbs: ["get", "list", "watch"] + - apiGroups: ["apps"] + resources: ["daemonsets", "deployments", "replicasets", "statefulsets"] + verbs: ["get", "list", "watch"] + - apiGroups: ["extensions"] + resources: ["daemonsets", "deployments", "replicasets"] + verbs: ["get", "list", "watch"] + - apiGroups: ["batch"] + resources: ["jobs", "cronjobs"] + verbs: ["get", "watch", "list"] + - apiGroups: ["autoscaling"] + resources: ["horizontalpodautoscalers"] + verbs: ["get", "watch", "list"] +{{- end }} +{{ end }} \ No newline at end of file diff --git a/charts/logzio-metrics-collector/templates/clusterrolebinding.yaml b/charts/logzio-metrics-collector/templates/clusterrolebinding.yaml new file mode 100644 index 00000000..fad40138 --- /dev/null +++ b/charts/logzio-metrics-collector/templates/clusterrolebinding.yaml @@ -0,0 +1,24 @@ +{{ if .Values.enabled}} +{{- if (.Values.clusterRole.create) -}} +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: {{ include "metrics-collector.clusterRoleBindingName" . }} + labels: + {{- include "metrics-collector.labels" . | nindent 4 }} + {{- if .Values.clusterRole.clusterRoleBinding.annotations }} + annotations: + {{- range $key, $value := .Values.clusterRole.clusterRoleBinding.annotations }} + {{- printf "%s: %s" $key (tpl $value $ | quote) | nindent 4 }} + {{- end }} + {{- end }} +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: {{ include "metrics-collector.clusterRoleName" . }} +subjects: +- kind: ServiceAccount + name: {{ include "metrics-collector.serviceAccountName" . }} + namespace: {{ include "metrics-collector.namespace" . }} +{{- end }} +{{ end }} diff --git a/charts/logzio-metrics-collector/templates/configmap-agent.yaml b/charts/logzio-metrics-collector/templates/configmap-agent.yaml new file mode 100644 index 00000000..8bc41c5b --- /dev/null +++ b/charts/logzio-metrics-collector/templates/configmap-agent.yaml @@ -0,0 +1,14 @@ +{{ if .Values.enabled}} +{{- if and (eq .Values.mode "daemonset") (.Values.configMap.create) -}} +apiVersion: v1 +kind: ConfigMap +metadata: + name: {{ include "metrics-collector.fullname" . }}-daemonset + namespace: {{ template "metrics-collector.namespace" . }} + labels: + {{- include "metrics-collector.labels" . | nindent 4 }} +data: + relay: | + {{- include "metrics-collector.daemonsetCollectorConfig" . | nindent 4 -}} +{{- end }} +{{ end }} \ No newline at end of file diff --git a/charts/logzio-metrics-collector/templates/configmap-standalone.yaml b/charts/logzio-metrics-collector/templates/configmap-standalone.yaml new file mode 100644 index 00000000..e7cd7475 --- /dev/null +++ b/charts/logzio-metrics-collector/templates/configmap-standalone.yaml @@ -0,0 +1,14 @@ +{{ if .Values.enabled}} +{{- if and (eq .Values.mode "standalone") (.Values.configMap.create) -}} +apiVersion: v1 +kind: ConfigMap +metadata: + name: {{ include "metrics-collector.fullname" . }}-standalone + namespace: {{ template "metrics-collector.namespace" . }} + labels: + {{- include "metrics-collector.labels" . | nindent 4 }} +data: + relay: | + {{- include "metrics-collector.standaloneCollectorConfig" . | nindent 4 -}} +{{- end }} +{{ end }} \ No newline at end of file diff --git a/charts/logzio-metrics-collector/templates/daemonset.yaml b/charts/logzio-metrics-collector/templates/daemonset.yaml new file mode 100644 index 00000000..96fc974e --- /dev/null +++ b/charts/logzio-metrics-collector/templates/daemonset.yaml @@ -0,0 +1,51 @@ +{{ if .Values.enabled}} +{{- if eq .Values.mode "daemonset" -}} +apiVersion: apps/v1 +kind: DaemonSet +metadata: + name: {{ include "metrics-collector.fullname" . }}-ds + namespace: {{ template "metrics-collector.namespace" . }} + labels: + {{- include "metrics-collector.labels" . | nindent 4 }} + {{- if .Values.annotations }} + annotations: + {{- range $key, $value := .Values.annotations }} + {{- printf "%s: %s" $key (tpl $value $ | quote) | nindent 4 }} + {{- end }} + {{- end }} +spec: + selector: + matchLabels: + {{- include "metrics-collector.selectorLabels" . | nindent 6 }} + {{- include "metrics-collector.component" . | nindent 6 }} + updateStrategy: + {{- if eq .Values.rollout.strategy "RollingUpdate" }} + {{- with .Values.rollout.rollingUpdate }} + rollingUpdate: + {{- toYaml . | nindent 6 }} + {{- end }} + {{- end }} + type: {{ .Values.rollout.strategy }} + template: + metadata: + annotations: + checksum/config: {{ include (print $.Template.BasePath "/configmap-agent.yaml") . | sha256sum }} + {{- include "metrics-collector.podAnnotations" . | nindent 8 }} + labels: + {{- include "metrics-collector.selectorLabels" . | nindent 8 }} + {{- include "metrics-collector.component" . | nindent 8 }} + {{- include "metrics-collector.podLabels" . | nindent 8 }} + spec: + {{- $podValues := deepCopy .Values }} + {{- $podData := dict "Values" $podValues "configmapSuffix" "-daemonset" "isAgent" true }} + {{- include "metrics-collector.metricsPod" ($podData | mustMergeOverwrite (deepCopy .)) | nindent 6 }} + hostNetwork: {{ .Values.hostNetwork }} + {{- with .Values.dnsPolicy }} + dnsPolicy: {{ . }} + {{- end }} + {{- with .Values.dnsConfig }} + dnsConfig: + {{- toYaml . | nindent 8 }} + {{- end }} +{{- end }} +{{ end }} diff --git a/charts/logzio-metrics-collector/templates/deployment.yaml b/charts/logzio-metrics-collector/templates/deployment.yaml new file mode 100644 index 00000000..9dceb0bf --- /dev/null +++ b/charts/logzio-metrics-collector/templates/deployment.yaml @@ -0,0 +1,42 @@ +{{ if .Values.enabled}} +{{- if eq .Values.mode "standalone" -}} +apiVersion: apps/v1 +kind: Deployment +metadata: + name: {{ include "metrics-collector.fullname" . }} + namespace: {{ template "metrics-collector.namespace" . }} + labels: + {{- include "metrics-collector.labels" . | nindent 4 }} + {{- if .Values.annotations }} + annotations: + {{- range $key, $value := .Values.annotations }} + {{- printf "%s: %s" $key (tpl $value $ | quote) | nindent 4 }} + {{- end }} + {{- end }} +spec: + replicas: {{ .Values.standaloneCollector.replicaCount }} + selector: + matchLabels: + {{- include "metrics-collector.selectorLabels" . | nindent 6 }} + template: + metadata: + annotations: + checksum/config: {{ include (print $.Template.BasePath "/configmap-standalone.yaml") . | sha256sum }} + {{- include "metrics-collector.podAnnotations" . | nindent 8 }} + labels: + {{- include "metrics-collector.selectorLabels" . | nindent 8 }} + {{- include "metrics-collector.podLabels" . | nindent 8 }} + spec: + {{- $podValues := deepCopy .Values }} + {{- $podData := dict "Values" $podValues "configmapSuffix" "-standalone" "isAgent" true }} + {{- include "metrics-collector.metricsPod" ($podData | mustMergeOverwrite (deepCopy .)) | nindent 6 }} + hostNetwork: {{ .Values.hostNetwork }} + {{- with .Values.dnsPolicy }} + dnsPolicy: {{ . }} + {{- end }} + {{- with .Values.dnsConfig }} + dnsConfig: + {{- toYaml . | nindent 8 }} + {{- end }} +{{- end }} +{{ end }} diff --git a/charts/logzio-metrics-collector/templates/ingress.yaml b/charts/logzio-metrics-collector/templates/ingress.yaml new file mode 100644 index 00000000..997386ff --- /dev/null +++ b/charts/logzio-metrics-collector/templates/ingress.yaml @@ -0,0 +1,57 @@ +{{ if .Values.enabled}} +{{- if .Values.ingress.enabled -}} +{{- $ingresses := prepend .Values.ingress.additionalIngresses .Values.ingress -}} +{{- range $ingresses }} +apiVersion: "networking.k8s.io/v1" +kind: Ingress +metadata: + {{- if .name }} + name: {{ printf "%s-%s" (include "metrics-collector.fullname" $) .name }} + {{- else }} + name: {{ include "metrics-collector.fullname" $ }} + {{- end }} + namespace: {{ template "metrics-collector.namespace" $ }} + labels: + {{- include "metrics-collector.labels" $ | nindent 4 }} + {{- include "metrics-collector.component" $ | nindent 4 }} + {{- if .annotations }} + annotations: + {{- range $key, $value := .annotations }} + {{- printf "%s: %s" $key (tpl $value $ | quote) | nindent 4 }} + {{- end }} + {{- end }} +spec: + {{- if .ingressClassName }} + ingressClassName: {{ .ingressClassName }} + {{- end -}} + {{- if .tls }} + tls: + {{- range .tls }} + - hosts: + {{- range .hosts }} + - {{ tpl . $ | quote }} + {{- end }} + {{- with .secretName }} + secretName: {{ . }} + {{- end }} + {{- end }} + {{- end }} + rules: + {{- range .hosts }} + - host: {{ tpl .host $ | quote }} + http: + paths: + {{- range .paths }} + - path: {{ .path }} + pathType: {{ .pathType }} + backend: + service: + name: {{ include "metrics-collector.fullname" $ }} + port: + number: {{ .port }} + {{- end }} + {{- end }} +--- +{{- end }} +{{- end }} +{{ end }} diff --git a/charts/logzio-metrics-collector/templates/job.yaml b/charts/logzio-metrics-collector/templates/job.yaml new file mode 100644 index 00000000..623101ed --- /dev/null +++ b/charts/logzio-metrics-collector/templates/job.yaml @@ -0,0 +1,94 @@ +{{- if .Values.enabled }} +{{- if and (.Values.secrets.windowsNodeUsername) (.Values.secrets.windowsNodePassword) -}} +apiVersion: batch/v1 +kind: Job +metadata: + name: windows-exporter-installer + namespace: {{ .Release.Namespace }} +spec: + ttlSecondsAfterFinished: {{ .Values.windowsExporterInstallerJob.ttlSecondsAfterFinished }} + template: + metadata: + annotations: + labels: + {{- include "metrics-collector.selectorLabels" . | nindent 8 }} + spec: + restartPolicy: Never + serviceAccountName: {{ include "metrics-collector.serviceAccountName" . }} + containers: + - name: windows-exporter-installer + image: "{{ .Values.windowsExporterInstallerImage.repository }}:{{ .Values.windowsExporterInstallerImage.tag }}" + env: + - name: SECRET_USERNAME + valueFrom: + secretKeyRef: + name: logzio-windows-exporter-installer-secrets + key: username + optional: true + - name: SECRET_PASSWORD + valueFrom: + secretKeyRef: + name: logzio-windows-exporter-installer-secrets + key: password + optional: true + args: ["$(SECRET_USERNAME)", + "$(SECRET_PASSWORD)"] + securityContext: + privileged: true + {{- toYaml .Values.containerSecurityContext | nindent 6 }} + nodeSelector: + kubernetes.io/os: linux + {{- if .Values.tolerations }} + tolerations: {{- .Values.tolerations | toYaml | trim | nindent 8 }} + {{- end }} + +--- + +apiVersion: batch/v1 +kind: CronJob +metadata: + name: windows-exporter-installer-scheduled + namespace: {{ .Release.Namespace }} +spec: + schedule: "{{ .Values.windowsExporterInstallerJob.interval }}" + successfulJobsHistoryLimit: {{ .Values.windowsExporterInstallerJob.successfulJobsHistoryLimit }} + failedJobsHistoryLimit: {{ .Values.windowsExporterInstallerJob.failedJobsHistoryLimit }} + concurrencyPolicy: "{{ .Values.windowsExporterInstallerJob.concurrencyPolicy }}" + jobTemplate: + spec: + template: + metadata: + annotations: + labels: + {{- include "metrics-collector.selectorLabels" . | nindent 12 }} + spec: + restartPolicy: Never + serviceAccountName: {{ include "metrics-collector.serviceAccountName" . }} + containers: + - name: windows-exporter-installer + image: "{{ .Values.windowsExporterInstallerImage.repository }}:{{ .Values.windowsExporterInstallerImage.tag }}" + env: + - name: SECRET_USERNAME + valueFrom: + secretKeyRef: + name: logzio-windows-exporter-installer-secrets + key: username + optional: true + - name: SECRET_PASSWORD + valueFrom: + secretKeyRef: + name: logzio-windows-exporter-installer-secrets + key: password + optional: true + args: ["$(SECRET_USERNAME)", + "$(SECRET_PASSWORD)"] + securityContext: + privileged: true + {{- toYaml .Values.containerSecurityContext | nindent 6 }} + nodeSelector: + kubernetes.io/os: linux + {{- if .Values.tolerations }} + tolerations: {{- .Values.tolerations | toYaml | trim | nindent 12 }} + {{- end }} +{{- end }} +{{- end }} \ No newline at end of file diff --git a/charts/logzio-metrics-collector/templates/networkpolicy.yaml b/charts/logzio-metrics-collector/templates/networkpolicy.yaml new file mode 100644 index 00000000..6b705cba --- /dev/null +++ b/charts/logzio-metrics-collector/templates/networkpolicy.yaml @@ -0,0 +1,41 @@ +{{ if .Values.enabled}} +{{- if .Values.networkPolicy.enabled }} +kind: NetworkPolicy +apiVersion: networking.k8s.io/v1 +metadata: + name: {{ include "metrics-collector.fullname" . }} + namespace: {{ template "metrics-collector.namespace" . }} + labels: + {{- include "metrics-collector.labels" . | nindent 4 }} + {{- if .Values.networkPolicy.annotations }} + annotations: + {{- range $key, $value := .Values.networkPolicy.annotations }} + {{- printf "%s: %s" $key (tpl $value $ | quote) | nindent 4 }} + {{- end }} + {{- end }} +spec: + podSelector: + matchLabels: + {{- include "metrics-collector.selectorLabels" . | nindent 6 }} + {{- include "metrics-collector.component" . | nindent 6 }} + ingress: + - ports: + {{- range $port := .Values.ports }} + {{- if $port.enabled }} + - port: {{ $port.containerPort }} + protocol: {{ $port.protocol }} + {{- end }} + {{- end }} + {{- if .Values.networkPolicy.allowIngressFrom }} + from: + {{- toYaml .Values.networkPolicy.allowIngressFrom | nindent 8 }} + {{- end }} + {{- if .Values.networkPolicy.extraIngressRules }} + {{- toYaml .Values.networkPolicy.extraIngressRules | nindent 4 }} + {{- end }} + {{- if .Values.networkPolicy.egressRules }} + egress: + {{- toYaml .Values.networkPolicy.egressRules | nindent 4 }} + {{- end }} +{{- end }} +{{ end }} diff --git a/charts/logzio-metrics-collector/templates/podmonitor.yaml b/charts/logzio-metrics-collector/templates/podmonitor.yaml new file mode 100644 index 00000000..0573505f --- /dev/null +++ b/charts/logzio-metrics-collector/templates/podmonitor.yaml @@ -0,0 +1,21 @@ +{{ if .Values.enabled}} +{{- if and .Values.podMonitor.enabled .Values.podMonitor.metricsEndpoints (eq .Values.mode "daemonset") }} +apiVersion: monitoring.coreos.com/v1 +kind: PodMonitor +metadata: + name: {{ include "metrics-collector.fullname" . }}-agent + namespace: {{ template "metrics-collector.namespace" . }} + labels: + {{- include "metrics-collector.labels" . | nindent 4 }} + {{- range $key, $value := .Values.podMonitor.extraLabels }} + {{- printf "%s: %s" $key (tpl $value $ | quote) | nindent 4 }} + {{- end }} +spec: + selector: + matchLabels: + {{- include "metrics-collector.selectorLabels" . | nindent 6 }} + {{- include "metrics-collector.component" . | nindent 6 }} + podMetricsEndpoints: + {{- toYaml .Values.podMonitor.metricsEndpoints | nindent 2 }} +{{- end }} +{{ end }} diff --git a/charts/logzio-metrics-collector/templates/secret.yaml b/charts/logzio-metrics-collector/templates/secret.yaml new file mode 100644 index 00000000..8c70ed71 --- /dev/null +++ b/charts/logzio-metrics-collector/templates/secret.yaml @@ -0,0 +1,37 @@ + +{{ if .Values.enabled}} +{{- if .Values.secrets.enabled }} +apiVersion: v1 +kind: Secret +metadata: + name: {{ .Values.secrets.name }} + namespace: {{ .Release.Namespace }} +type: Opaque +stringData: + env-id: {{.Values.secrets.env_id | quote}} + logzio-listener-region: {{ .Values.secrets.logzioRegion}} + {{ if .Values.secrets.logzioMetricsToken}} + logzio-metrics-token: {{ .Values.secrets.logzioMetricsToken }} + {{ end }} + {{ if .Values.secrets.customEndpoint}} + custom-endpoint: {{ .Values.secrets.customEndpoint}} + {{ end }} + {{ if .Values.secrets.k8sObjectsLogsToken}} + logzio-k8s-objects-logs-token: {{ .Values.secrets.k8sObjectsLogsToken}} + {{ end }} +{{- if .Values.secrets.windowsNodeUsername }} +{{- if .Values.secrets.windowsNodePassword }} +--- +apiVersion: v1 +kind: Secret +metadata: + name: logzio-windows-exporter-installer-secrets + namespace: {{ .Release.Namespace }} +type: kubernetes.io/basic-auth +stringData: + username: {{.Values.secrets.windowsNodeUsername}} + password: {{.Values.secrets.windowsNodePassword}} +{{- end }} +{{- end }} +{{- end }} +{{ end }} diff --git a/charts/logzio-metrics-collector/templates/service.yaml b/charts/logzio-metrics-collector/templates/service.yaml new file mode 100644 index 00000000..6fa80765 --- /dev/null +++ b/charts/logzio-metrics-collector/templates/service.yaml @@ -0,0 +1,44 @@ +{{ if .Values.enabled}} +{{- if or (eq (include "metrics-collector.serviceEnabled" .) "true") (.Values.ingress.enabled) -}} +apiVersion: v1 +kind: Service +metadata: + name: {{ include "metrics-collector.fullname" . }} + namespace: {{ template "metrics-collector.namespace" . }} + labels: + {{- include "metrics-collector.labels" . | nindent 4 }} + {{- include "metrics-collector.component" . | nindent 4 }} + {{- if .Values.service.annotations }} + annotations: + {{- range $key, $value := .Values.service.annotations }} + {{- printf "%s: %s" $key (tpl $value $ | quote) | nindent 4 }} + {{- end }} + {{- end }} +spec: + type: {{ .Values.service.type }} + {{- if .Values.service.clusterIP }} + clusterIP: {{ .Values.service.clusterIP }} + {{- end }} + {{- if and .Values.service.loadBalancerIP (eq .Values.service.type "LoadBalancer") }} + loadBalancerIP: {{ .Values.service.loadBalancerIP }} + {{- end }} + {{- if and .Values.service.loadBalancerSourceRanges (eq .Values.service.type "LoadBalancer") }} + loadBalancerSourceRanges: + {{- range .Values.service.loadBalancerSourceRanges }} + - {{ . }} + {{- end }} + {{- end }} + {{- $ports := include "metrics-collector.servicePortsConfig" . }} + {{- if $ports }} + ports: + {{- $ports | nindent 4}} + {{- end }} + selector: + {{- include "metrics-collector.selectorLabels" . | nindent 4 }} + {{- include "metrics-collector.component" . | nindent 4 }} + internalTrafficPolicy: {{ include "metrics-collector.serviceInternalTrafficPolicy" . }} + {{- if and (eq .Values.service.type "LoadBalancer") (.Values.service.externalTrafficPolicy) }} + externalTrafficPolicy: {{ .Values.service.externalTrafficPolicy }} + {{- end }} +{{- end }} +{{ end }} diff --git a/charts/logzio-metrics-collector/templates/serviceaccount.yaml b/charts/logzio-metrics-collector/templates/serviceaccount.yaml new file mode 100644 index 00000000..8a482b51 --- /dev/null +++ b/charts/logzio-metrics-collector/templates/serviceaccount.yaml @@ -0,0 +1,17 @@ +{{ if .Values.enabled}} +{{- if (.Values.serviceAccount.create) -}} +apiVersion: v1 +kind: ServiceAccount +metadata: + name: {{ include "metrics-collector.serviceAccountName" . }} + namespace: {{ template "metrics-collector.namespace" . }} + labels: + {{- include "metrics-collector.labels" . | nindent 4 }} + {{- if .Values.serviceAccount.annotations }} + annotations: + {{- range $key, $value := .Values.serviceAccount.annotations }} + {{- printf "%s: %s" $key (tpl $value $ | quote) | nindent 4 }} + {{- end }} + {{- end }} +{{ end }} +{{ end }} diff --git a/charts/logzio-metrics-collector/values.yaml b/charts/logzio-metrics-collector/values.yaml new file mode 100644 index 00000000..19e95e54 --- /dev/null +++ b/charts/logzio-metrics-collector/values.yaml @@ -0,0 +1,963 @@ +# Default values for opentelemetry-collector. +# This is a YAML-formatted file. +# Declare variables to be passed into your templates. + +# conditionally controll the deployment of this chart by a parent chart +enabled: true + +nameOverride: "" +fullnameOverride: "" + +# Valid values for metrics collection are "daemonset","standalone". Default is "daemonset" +mode: "daemonset" + +# Specify which namespace should be used to deploy the resources into +namespaceOverride: "" + + +secrets: + # When true, the logzio secret will be created and managed by this Chart. If you're managing the logzio secrets by yourself, set to false, note that in order for the default configuration to work proprly you need to create the following env variables: ENV_ID LOGZIO_REGION LOGZIO_METRICS_TOKEN + enabled: true + name: logzio-metric-collector-secrets + # environment indentifier attribute that will be added to all metrics + env_id: "my_env" + # Secret with your Logz.io metrics shipping token + logzioMetricsToken: "<>" + # Secret with your Logz.io region code - https://docs.logz.io/docs/user-guide/admin/hosting-regions/account-region/ + logzioRegion: "us" + # Secret with your custom endpoint, for example: http://endpoint:8050. Overrides secrets.logzioRegion listener adress + customEndpoint: "" + # Secret with your Logz.io logs shipping token, optional for Kuebrnetes object logs and metrics correlation, set `k8sObjectsLogs.enabled` to `true`. + k8sObjectsLogsToken: "<>" + # Secrets with Windows node username and password for windows node metrics collection + windowsNodePassword: "" + windowsNodeUsername: "" + +configMap: + # Specifies whether a configMap should be created (true by default) + create: true + +# Send application metrics, requires `enabled` flag, and the `prometheus.io/scrape: true` annotaion set to the relevant pods. +applicationMetrics: + enabled: false + +# OpenTelemetry Collector base configuration +baseConfig: + receivers: + k8s_cluster: + auth_type: serviceAccount + collection_interval: 30s + metadata_collection_interval: 1m + node_conditions_to_report: + - Ready + - MemoryPressure + - NetworkUnavailable + - DiskPressure + - PIDPressure + allocatable_types_to_report: + - cpu + - memory + - storage + - ephemeral-storage + resource_attributes: + k8s.pod.name: + enabled: true + k8s.deployment.name: + enabled: true + k8s.namespace.name: + enabled: true + k8s.node.name: + enabled: true + k8s.statefulset.name: + enabled: true + k8s.replicaset.name: + enabled: true + k8s.daemonset.name: + enabled: true + k8s.cronjob.name: + enabled: true + k8s.job.name: + enabled: true + k8s.pod.uid: + enabled: true + kubeletstats: + collection_interval: 30s + auth_type: "serviceAccount" + endpoint: "${env:KUBE_NODE_NAME}:10250" + insecure_skip_verify: true + prometheus/collector: + config: + global: + scrape_interval: 30s + scrape_timeout: 30s + scrape_configs: + # Job to collect opentelemetry collector metrics + - job_name: 'collector-metrics' + scrape_interval: 15s + static_configs: + - targets: [ "0.0.0.0:8888" ] + metric_relabel_configs: [] + exporters: + logging: + loglevel: info + prometheusremotewrite/applications: + timeout: 30s + endpoint: ${LISTENER_URL} + external_labels: + p8s_logzio_name: ${ENV_ID} + headers: + Authorization: "Bearer ${LOGZIO_METRICS_TOKEN}" + user-agent: "{{ .Chart.Name }}-{{ .Chart.Version }}-helm" + resource_to_telemetry_conversion: + enabled: true + prometheusremotewrite/infrastructure: + timeout: 30s + endpoint: ${LISTENER_URL} + external_labels: + p8s_logzio_name: ${ENV_ID} + headers: + Authorization: "Bearer ${LOGZIO_METRICS_TOKEN}" + user-agent: "{{ .Chart.Name }}-{{ .Chart.Version }}-helm" + resource_to_telemetry_conversion: + enabled: true + extensions: + health_check: {} + processors: + k8sattributes: + extract: + metadata: + - k8s.pod.name + - k8s.deployment.name + - k8s.namespace.name + - k8s.node.name + - k8s.statefulset.name + - k8s.replicaset.name + - k8s.daemonset.name + - k8s.cronjob.name + - k8s.job.name + - k8s.pod.uid + - k8s.pod.start_time + - k8s.pod.ip + filter: + node_from_env_var: KUBE_NODE_NAME + passthrough: false + pod_association: + - sources: + - from: resource_attribute + name: k8s.pod.ip + - sources: + - from: resource_attribute + name: k8s.pod.uid + - sources: + - from: connection + batch: {} + attributes/env_id: + actions: + - key: env_id + value: ${ENV_ID} + action: insert + - key: logzio_agent_version + value: "{{ .Chart.Version }}" + action: insert + service: + extensions: + - health_check + telemetry: + logs: + level: "debug" + + +# Configuration for OpenTelemetry Collector DaemonSet, enabled by default +daemonsetCollector: + + # prevent collector daemonset deployment on fargate nodes + affinity: + nodeAffinity: + requiredDuringSchedulingIgnoredDuringExecution: + nodeSelectorTerms: + - matchExpressions: + - key: eks.amazonaws.com/compute-type + operator: DoesNotExist + + resources: + requests: + cpu: 50m + memory: 70Mi + + + podLabels: {} + + podAnnotations: {} + + # DaemonSet Configuration override that will be merged into the daemonset default config + configOverride: {} + +# Configuration for standalone OpenTelemetry Collector deployment +standaloneCollector: + replicas: 1 + + resources: + requests: + cpu: 50m + memory: 70Mi + + + podLabels: {} + + podAnnotations: {} + + # Standalone collector configuration override that will be merged into the collector default config + configOverride: {} + +# OpenTelemetry Collector image +image: + # If you want to use the core image `otel/opentelemetry-collector`, you also need to change `command.name` value to `otelcol`. + repository: otel/opentelemetry-collector-contrib + pullPolicy: IfNotPresent + # Overrides the image tag whose default is the chart appVersion. + tag: "" + # When digest is set to a non-empty value, images will be pulled by digest (regardless of tag value). + digest: "" +imagePullSecrets: [] + +# OpenTelemetry Collector executable +command: + name: otelcol-contrib + extraArgs: [] + +serviceAccount: + # Specifies whether a service account should be created + create: true + # Annotations to add to the service account + annotations: {} + # The name of the service account to use. + # If not set and create is true, a name is generated using the fullname template + name: "" + +clusterRole: + # Specifies whether a clusterRole should be created + # Some presets also trigger the creation of a cluster role and cluster role binding. + # If using one of those presets, this field is no-op. + create: true + # Annotations to add to the clusterRole + # Can be used in combination with presets that create a cluster role. + annotations: {} + # The name of the clusterRole to use. + # If not set a name is generated using the fullname template + # Can be used in combination with presets that create a cluster role. + name: "" + # A set of rules as documented here : https://kubernetes.io/docs/reference/access-authn-authz/rbac/ + # Can be used in combination with presets that create a cluster role to add additional rules. + rules: [] + # - apiGroups: + # - '' + # resources: + # - 'pods' + # - 'nodes' + # verbs: + # - 'get' + # - 'list' + # - 'watch' + + clusterRoleBinding: + # Annotations to add to the clusterRoleBinding + # Can be used in combination with presets that create a cluster role binding. + annotations: {} + # The name of the clusterRoleBinding to use. + # If not set a name is generated using the fullname template + # Can be used in combination with presets that create a cluster role binding. + name: "" + +podSecurityContext: {} +securityContext: {} + +nodeSelector: {} +tolerations: [] +# Set affinity rules for the scheduler to determine where all DaemonSet pods can be placed. +affinity: {} +# Allows for pod scheduler prioritisation +priorityClassName: "" + +extraEnvs: [] +extraEnvsFrom: [] +extraVolumes: [] +extraVolumeMounts: [] + +ports: + metrics: + enabled: true + containerPort: 8888 + servicePort: 8888 + protocol: TCP + +resources: + requests: + cpu: 50m + memory: 70Mi + + +podAnnotations: {} + +podLabels: {} + +# Common labels to add to all otel-collector resources. Evaluated as a template. +additionalLabels: {} +# app.kubernetes.io/part-of: my-app + +# Host networking requested for this pod. Use the host's network namespace. +hostNetwork: false + +# Adding entries to Pod /etc/hosts with HostAliases +# https://kubernetes.io/docs/tasks/network/customize-hosts-file-for-pods/ +hostAliases: [] + # - ip: "1.2.3.4" + # hostnames: + # - "my.host.com" + +# Pod DNS policy ClusterFirst, ClusterFirstWithHostNet, None, Default, None +dnsPolicy: "" + +# Custom DNS config. Required when DNS policy is None. +dnsConfig: {} + + +annotations: {} + +# List of extra sidecars to add +extraContainers: [] +# extraContainers: +# - name: test +# command: +# - cp +# args: +# - /bin/sleep +# - /test/sleep +# image: busybox:latest +# volumeMounts: +# - name: test +# mountPath: /test + +# List of init container specs, e.g. for copying a binary to be executed as a lifecycle hook. +# Another usage of init containers is e.g. initializing filesystem permissions to the OTLP Collector user `10001` in case you are using persistence and the volume is producing a permission denied error for the OTLP Collector container. +initContainers: [] +# initContainers: +# - name: test +# image: busybox:latest +# command: +# - cp +# args: +# - /bin/sleep +# - /test/sleep +# volumeMounts: +# - name: test +# mountPath: /test +# - name: init-fs +# image: busybox:latest +# command: +# - sh +# - '-c' +# - 'chown -R 10001: /var/lib/storage/otc' # use the path given as per `extensions.file_storage.directory` & `extraVolumeMounts[x].mountPath` +# volumeMounts: +# - name: opentelemetry-collector-data # use the name of the volume used for persistence +# mountPath: /var/lib/storage/otc # use the path given as per `extensions.file_storage.directory` & `extraVolumeMounts[x].mountPath` + +service: + # Enable the creation of a Service. + enabled: true + + type: ClusterIP + # type: LoadBalancer + # loadBalancerIP: 1.2.3.4 + # loadBalancerSourceRanges: [] + + # By default, Service of type 'LoadBalancer' will be created setting 'externalTrafficPolicy: Cluster' + # unless other value is explicitly set. + # Possible values are Cluster or Local (https://kubernetes.io/docs/tasks/access-application-cluster/create-external-load-balancer/#preserving-the-client-source-ip) + + # externalTrafficPolicy: Cluster + + annotations: {} + + # By default, Service will be created setting 'internalTrafficPolicy: Local' on mode = daemonset + # unless other value is explicitly set. + # Setting 'internalTrafficPolicy: Cluster' on a daemonset is not recommended + + # internalTrafficPolicy: Cluster + +ingress: + enabled: false + # annotations: {} + # ingressClassName: nginx + # hosts: + # - host: collector.example.com + # paths: + # - path: / + # pathType: Prefix + # port: 4318 + # tls: + # - secretName: collector-tls + # hosts: + # - collector.example.com + + # Additional ingresses - only created if ingress.enabled is true + # Useful for when differently annotated ingress services are required + # Each additional ingress needs key "name" set to something unique + additionalIngresses: [] + # - name: cloudwatch + # ingressClassName: nginx + # annotations: {} + # hosts: + # - host: collector.example.com + # paths: + # - path: / + # pathType: Prefix + # port: 4318 + # tls: + # - secretName: collector-tls + # hosts: + # - collector.example.com + +podMonitor: + # The pod monitor by default scrapes the metrics port. + # The metrics port needs to be enabled as well. + enabled: false + metricsEndpoints: + - port: metrics + # interval: 15s + + # additional labels for the PodMonitor + extraLabels: {} + # release: kube-prometheus-stack + +rollout: + rollingUpdate: {} + # When 'mode: daemonset', maxSurge cannot be used when hostPort is set for any of the ports + # maxSurge: 25% + # maxUnavailable: 0 + strategy: RollingUpdate + +networkPolicy: + enabled: false + + # Annotations to add to the NetworkPolicy + annotations: {} + + # Configure the 'from' clause of the NetworkPolicy. + # By default this will restrict traffic to ports enabled for the Collector. If + # you wish to further restrict traffic to other hosts or specific namespaces, + # see the standard NetworkPolicy 'spec.ingress.from' definition for more info: + # https://kubernetes.io/docs/reference/kubernetes-api/policy-resources/network-policy-v1/ + allowIngressFrom: [] + # # Allow traffic from any pod in any namespace, but not external hosts + # - namespaceSelector: {} + # # Allow external access from a specific cidr block + # - ipBlock: + # cidr: 192.168.1.64/32 + # # Allow access from pods in specific namespaces + # - namespaceSelector: + # matchExpressions: + # - key: kubernetes.io/metadata.name + # operator: In + # values: + # - "cats" + # - "dogs" + + # Add additional ingress rules to specific ports + # Useful to allow external hosts/services to access specific ports + # An example is allowing an external prometheus server to scrape metrics + # + # See the standard NetworkPolicy 'spec.ingress' definition for more info: + # https://kubernetes.io/docs/reference/kubernetes-api/policy-resources/network-policy-v1/ + extraIngressRules: [] + # - ports: + # - port: metrics + # protocol: TCP + # from: + # - ipBlock: + # cidr: 192.168.1.64/32 + + # Restrict egress traffic from the OpenTelemetry collector pod + # See the standard NetworkPolicy 'spec.egress' definition for more info: + # https://kubernetes.io/docs/reference/kubernetes-api/policy-resources/network-policy-v1/ + egressRules: [] + # - to: + # - namespaceSelector: {} + # - ipBlock: + # cidr: 192.168.10.10/24 + # ports: + # - port: 1234 + # protocol: TCP + +# When enabled, the chart will set the GOMEMLIMIT env var to 80% of the configured +# resources.limits.memory +# If no resources.limits.memory are defined enabling does nothing. +# In a future release this setting will be enabled by default. +# See https://github.com/open-telemetry/opentelemetry-helm-charts/issues/891 +# for more details. +useGOMEMLIMIT: false + +# Opencost configuration +opencost: + enabled: false + config: + processors: + # opencost collects duplicates metrics from kube-state and cadvisor. + filter/opencost-exporter: + metrics: + datapoint: + - 'IsMatch(metric.name, "(${OPENCOST_DUPLICATES})") == true and attributes["app"] == "opencost"' + service: + pipelines: + metrics/infrastructure: + processors: + - filter/opencost-exporter + +# Filter only metrics relevant for prebuilt content +enableMetricsFilter: + gke: false # Google Kubernetes Engine + eks: false # Amazon Elastic Kubernetes Service + aks: false # Azure Kubernetes Service + dropKubeSystem: false # Drop kube-system metrics + +# Metrics names to be filtered +prometheusFilters: + # All values should be listed with | seperator, as regex. i.e: metric_1|metric_2|metric_3 + metrics: + # for infrastructure pipeline: metrics/infrastructure & metrics/cadvisor receivers + # (kubernetes-service-endpoints & cadvisor jobs) + infrastructure: + keep: + # need to also enable the flag: enableMetricsFilter.aks=true + aks: kube_daemonset_labels|kube_daemonset_status_number_ready|kube_daemonset_status_number_available|kube_daemonset_status_number_unavailable|kube_daemonset_status_current_number_scheduled|kube_daemonset_status_number_misscheduled|kube_daemonset_status_desired_number_scheduled|kube_job_labels|kube_job_complete|kube_job_status_failed|kube_job_status_succeeded|kube_job_complete|kube_job_status_failed|kube_job_status_completion_time|kube_replicaset_labels|kube_replicaset_spec_replicas|kube_replicaset_status_replicas|kube_replicaset_status_ready_replicas|kube_statefulset_replicas|kube_statefulset_status_replicas|kube_statefulset_status_replicas_updated|kube_statefulset_status_replicas_available|kube_pod_container_status_terminated_reason|kube_node_labels|kube_pod_container_status_waiting_reason|node_memory_Buffers_bytes|node_memory_Cached_bytes|kube_deployment_labels|container_cpu_usage_seconds_total|container_memory_working_set_bytes|container_network_receive_bytes_total|container_network_transmit_bytes_total|i:|kube_deployment_status_replicas|kube_deployment_status_replicas_available|kube_deployment_status_replicas_unavailable|kube_deployment_status_replicas_updated|kube_node_info|kube_node_spec_unschedulable|kube_node_status_allocatable|kube_node_status_capacity|kube_node_status_condition|kube_pod_container_info|kube_pod_container_resource_requests|kube_pod_container_resource_requests_cpu_cores|kube_pod_container_resource_requests_memory_bytes|kube_pod_container_status_ready|kube_pod_container_status_restarts_total|kube_pod_container_status_running|kube_pod_container_status_terminated|kube_pod_container_status_waiting|kube_pod_info|kube_pod_status_phase|machine_cpu_cores|namespace|node_boot_time_seconds|node_cpu_seconds_total|node_disk_io_time_seconds_total|node_filesystem_avail_bytes|node_filesystem_free_bytes|node_filesystem_size_bytes|node_memory_MemFree_bytes|node_memory_MemTotal_bytes|node_network_receive_bytes_total|node_network_transmit_bytes_total|node_time_seconds|p8s_logzio_name|windows_container_cpu_usage_seconds_total|windows_container_memory_usage_commit_bytes|windows_container_network_receive_bytes_total|windows_container_network_transmit_bytes_total|windows_cpu_time_total|windows_cs_hostname|windows_cs_physical_memory_bytes|windows_logical_disk_free_bytes|windows_logical_disk_read_seconds_total|windows_logical_disk_size_bytes|windows_logical_disk_write_seconds_total|windows_net_bytes_received_total|windows_net_bytes_sent_total|windows_os_physical_memory_free_bytes|windows_system_system_up_time|kube_pod_status_ready|kube_pod_container_status_restarts_total|kube_pod_container_resource_limits|container_memory_usage_bytes|container_network_transmit_packets_total|container_network_receive_packets_total|container_network_transmit_packets_dropped_total|container_network_receive_packets_dropped_total|kube_pod_created|kube_pod_owner|kube_pod_status_reason|node_cpu_seconds_total|node_memory_MemAvailable_bytes|kube_node_role|kube_node_created|node_load1|node_load5|node_load15|node_disk_reads_completed_total|node_disk_writes_completed_total|node_disk_read_bytes_total|node_disk_written_bytes_total|node_disk_read_time_seconds_total|node_disk_write_time_seconds_total|node_network_transmit_packets_total|node_network_receive_packets_total|node_network_transmit_drop_total|node_network_receive_drop_total|kube_replicaset_owner|kube_deployment_created|kube_deployment_status_condition|kube_deployment_spec_replicas|kube_namespace_status_phase|kubelet_volume_stats_inodes|kubelet_volume_stats_inodes_used|kubelet_volume_stats_used_bytes|kube_persistentvolumeclaim_info|kube_persistentvolumeclaim_resource_requests_storage_bytes|kube_job_owner|container_cpu_cfs_throttled_seconds_total + # need to also enable the flag: enableMetricsFilter.eks=true + eks: kube_daemonset_labels|kube_daemonset_status_number_ready|kube_daemonset_status_number_available|kube_daemonset_status_number_unavailable|kube_daemonset_status_current_number_scheduled|kube_daemonset_status_number_misscheduled|kube_daemonset_status_desired_number_scheduled|kube_job_labels|kube_job_complete|kube_job_status_failed|kube_job_status_succeeded|kube_job_complete|kube_job_status_failed|kube_job_status_completion_time|kube_replicaset_labels|kube_replicaset_spec_replicas|kube_replicaset_status_replicas|kube_replicaset_status_ready_replicas|kube_statefulset_replicas|kube_statefulset_status_replicas|kube_statefulset_status_replicas_updated|kube_statefulset_status_replicas_available|kube_pod_container_status_terminated_reason|kube_node_labels|kube_pod_container_status_waiting_reason|node_memory_Buffers_bytes|node_memory_Cached_bytes|kube_deployment_labels|container_cpu_usage_seconds_total|container_memory_working_set_bytes|container_network_receive_bytes_total|container_network_transmit_bytes_total|kube_deployment_status_replicas|kube_deployment_status_replicas_available|kube_deployment_status_replicas_unavailable|kube_deployment_status_replicas_updated|kube_node_info|kube_node_spec_unschedulable|kube_node_status_allocatable|kube_node_status_capacity|kube_node_status_condition|kube_pod_container_resource_requests|kube_pod_container_status_ready|kube_pod_container_status_restarts_total|kube_pod_container_status_running|kube_pod_container_status_terminated|kube_pod_container_status_waiting|kube_pod_info|kube_pod_status_phase|machine_cpu_cores|namespace|node_boot_time_seconds|node_cpu_seconds_total|node_disk_io_time_seconds_total|node_filesystem_avail_bytes|node_filesystem_free_bytes|node_filesystem_size_bytes|node_memory_MemFree_bytes|node_memory_MemTotal_bytes|node_network_receive_bytes_total|node_network_transmit_bytes_total|node_time_seconds|p8s_logzio_name|kube_pod_status_ready|kube_pod_container_status_restarts_total|kube_pod_container_resource_limits|container_memory_usage_bytes|container_network_transmit_packets_total|container_network_receive_packets_total|container_network_transmit_packets_dropped_total|container_network_receive_packets_dropped_total|kube_pod_created|kube_pod_owner|kube_pod_status_reason|node_cpu_seconds_total|node_memory_MemAvailable_bytes|kube_node_role|kube_node_created|node_load1|node_load5|node_load15|node_disk_reads_completed_total|node_disk_writes_completed_total|node_disk_read_bytes_total|node_disk_written_bytes_total|node_disk_read_time_seconds_total|node_disk_write_time_seconds_total|node_network_transmit_packets_total|node_network_receive_packets_total|node_network_transmit_drop_total|node_network_receive_drop_total|kube_replicaset_owner|kube_deployment_created|kube_deployment_status_condition|kube_deployment_spec_replicas|kube_namespace_status_phase|kubelet_volume_stats_inodes|kubelet_volume_stats_inodes_used|kubelet_volume_stats_used_bytes|kube_persistentvolumeclaim_info|kube_persistentvolumeclaim_resource_requests_storage_bytes|kube_job_owner|kube_pod_container_info|container_cpu_cfs_throttled_seconds_total + # need to also enable the flag: enableMetricsFilter.gke=true + gke: kube_daemonset_labels|kube_daemonset_status_number_ready|kube_daemonset_status_number_available|kube_daemonset_status_number_unavailable|kube_daemonset_status_current_number_scheduled|kube_daemonset_status_number_misscheduled|kube_daemonset_status_desired_number_scheduled|kube_job_labels|kube_job_complete|kube_job_status_failed|kube_job_status_succeeded|kube_job_complete|kube_job_status_failed|kube_job_status_completion_time|kube_replicaset_labels|kube_replicaset_spec_replicas|kube_replicaset_status_replicas|kube_replicaset_status_ready_replicas|kube_statefulset_replicas|kube_statefulset_status_replicas|kube_statefulset_status_replicas_updated|kube_statefulset_status_replicas_available|kube_pod_container_status_terminated_reason|kube_node_labels|kube_pod_container_status_waiting_reason|node_memory_Buffers_bytes|node_memory_Cached_bytes|kube_deployment_labels|container_cpu_usage_seconds_total|container_memory_working_set_bytes|container_network_receive_bytes_total|container_network_transmit_bytes_total|kube_deployment_status_replicas|kube_deployment_status_replicas_available|kube_deployment_status_replicas_unavailable|kube_deployment_status_replicas_updated|kube_node_info|kube_node_spec_unschedulable|kube_node_status_allocatable|kube_node_status_capacity|kube_node_status_condition|kube_pod_container_resource_requests|kube_pod_container_status_ready|kube_pod_container_status_restarts_total|kube_pod_container_status_running|kube_pod_container_status_terminated|kube_pod_container_status_waiting|kube_pod_info|kube_pod_status_phase|machine_cpu_cores|namespace|node_boot_time_seconds|node_cpu_seconds_total|node_disk_io_time_seconds_total|node_filesystem_avail_bytes|node_filesystem_free_bytes|node_filesystem_size_bytes|node_memory_MemFree_bytes|node_memory_MemTotal_bytes|node_network_receive_bytes_total|node_network_transmit_bytes_total|node_time_seconds|p8s_logzio_name|kube_pod_status_ready|kube_pod_container_status_restarts_total|kube_pod_container_resource_limits|container_memory_usage_bytes|container_network_transmit_packets_total|container_network_receive_packets_total|container_network_transmit_packets_dropped_total|container_network_receive_packets_dropped_total|kube_pod_created|kube_pod_owner|kube_pod_status_reason|node_cpu_seconds_total|node_memory_MemAvailable_bytes|kube_node_role|kube_node_created|node_load1|node_load5|node_load15|node_disk_reads_completed_total|node_disk_writes_completed_total|node_disk_read_bytes_total|node_disk_written_bytes_total|node_disk_read_time_seconds_total|node_disk_write_time_seconds_total|node_network_transmit_packets_total|node_network_receive_packets_total|node_network_transmit_drop_total|node_network_receive_drop_total|kube_replicaset_owner|kube_deployment_created|kube_deployment_status_condition|kube_deployment_spec_replicas|kube_namespace_status_phase|kubelet_volume_stats_inodes|kubelet_volume_stats_inodes_used|kubelet_volume_stats_used_bytes|kube_persistentvolumeclaim_info|kube_persistentvolumeclaim_resource_requests_storage_bytes|kube_job_owner|kube_pod_container_info|container_cpu_cfs_throttled_seconds_total + custom: + drop: + custom: + # for applications pipeline: applications job + applications: + keep: + custom: + drop: + custom: + + # Namespaces names to be filtered + # All values should be listed with | seperator, as regex. i.e: namespace_1|namespace_2|namespace_3 + namespaces: + # for infrastructure pipeline: metrics/infrastructure & metrics/cadvisor receivers + # (kubernetes-service-endpoints & cadvisor jobs) + infrastructure: + keep: + custom: + drop: + kubeSystem: kube-system # need to also enable the flag: enableMetricsFilter.kubeSystem=true + custom: + # for applications pipeline: applications job + applications: + keep: + custom: + drop: + custom: + + # Services names to filtered + # All values should be listed with | seperator, as regex. i.e: service_1|service_2|service_3 + services: + # for infrastructure pipeline: metrics/infrastructure & metrics/cadvisor receivers + # (kubernetes-service-endpoints & cadvisor jobs) + infrastructure: + keep: + custom: + drop: + kubeDns: kube-dns # need to also enable the flag: disableKubeDnsScraping=true + custom: + + +# OpenTelemetry configuration for DaemonSet collector +daemonsetConfig: + receivers: + prometheus/applications: + config: + global: + scrape_interval: 30s + scrape_timeout: 30s + scrape_configs: + - job_name: applications + honor_timestamps: true + honor_labels: true + metrics_path: /metrics + scheme: http + kubernetes_sd_configs: + - role: pod + selectors: + # only scrape data from pods running on the same node as the collector + - role: pod + field: "spec.nodeName=${env:KUBE_NODE_NAME}" + relabel_configs: + - action: keep + regex: true|"true" + source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape] + - action: replace + regex: (https?) + source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scheme] + target_label: __scheme__ + - action: replace + regex: (.+) + source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_path] + target_label: __metrics_path__ + - action: replace + regex: (.+?)(?::\d+)?;(\d+) + replacement: $$1:$$2 + source_labels: [__address__, __meta_kubernetes_pod_annotation_prometheus_io_port] + target_label: __address__ + - action: labelmap + regex: __meta_kubernetes_pod_annotation_prometheus_io_param_(.+) + replacement: __param_$$1 + - action: labelmap + regex: __meta_kubernetes_pod_label_(.+) + - action: replace + source_labels: [__meta_kubernetes_namespace] + target_label: namespace + - action: replace + source_labels: [__meta_kubernetes_pod_name] + target_label: pod + - source_labels: [__meta_kubernetes_pod_node_name] + action: replace + target_label: kubernetes_node + metric_relabel_configs: [] + prometheus/infrastructure: + config: + global: + scrape_interval: 30s + scrape_timeout: 30s + scrape_configs: + - job_name: windows-metrics + honor_timestamps: true + honor_labels: true + metrics_path: /metrics + scheme: http + kubernetes_sd_configs: + - role: pod + selectors: + # only scrape data from pods running on the same node as collector + - role: pod + field: "spec.nodeName=${env:KUBE_NODE_NAME}" + relabel_configs: + - source_labels: [__meta_kubernetes_pod_annotation_prometheus_windows_io_scrape] + action: keep + regex: true|"true" + metric_relabel_configs: [] + # Job to collect metrics from applications running on pods + - job_name: kubernetes-service-endpoints + honor_timestamps: true + honor_labels: true + metrics_path: /metrics + scheme: http + kubernetes_sd_configs: + - role: endpoints + selectors: + # only scrape data from pods running on the same node as collector + - role: pod + field: "spec.nodeName=${env:KUBE_NODE_NAME}" + relabel_configs: + # Adding a dummy job label to enable filtering for duplicate metrics using daemonset collector + # "job" label is only added in the prometheusremotewrite and cannot be added before + - action: replace + replacement: kubernetes-service-endpoints + target_label: job_dummy + - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_scrape] + action: keep + regex: true|"true" + - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_scheme] + action: replace + target_label: __scheme__ + regex: (https?) + - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_path] + action: replace + target_label: __metrics_path__ + regex: (.+) + - source_labels: [__address__, __meta_kubernetes_service_annotation_prometheus_io_port] + action: replace + target_label: __address__ + regex: ([^:]+)(?::\d+)?;(\d+) + replacement: $$1:$$2 + - action: labelmap + regex: __meta_kubernetes_service_label_(.+) + - source_labels: [__meta_kubernetes_namespace] + action: replace + target_label: namespace + - source_labels: [__meta_kubernetes_service_name] + action: replace + target_label: service + - source_labels: [__meta_kubernetes_pod_name] + action: replace + target_label: pod + - source_labels: [__meta_kubernetes_pod_node_name] + action: replace + target_label: kubernetes_node + - source_labels: [__meta_kubernetes_service_annotation_logz_io_app] + action: replace + target_label: logzio_app + metric_relabel_configs: [] + processors: + resourcedetection/all: + detectors: [ec2, azure, gcp] + filter/kubernetes360: + metrics: + datapoint: + - 'IsMatch(metric.name, "(${K8S_360_METRICS})") == true and attributes["logzio_app"] != "kubernetes360"' + # Workaround for an issue where metrics are scraped multiple times + - 'attributes["job_dummy"] == "kubernetes-service-endpoints" and attributes["kubernetes_node"] == nil' + # Removes label needed for duplicate metrics checks + attributes/remove_job_dummy: + actions: + - key: job_dummy + action: delete + service: + extensions: + - health_check + pipelines: + metrics/infrastructure: + exporters: + - prometheusremotewrite/infrastructure + processors: + - attributes/env_id + - batch + # - filter/kubernetes360 + - k8sattributes + # workaround for duplicate metrics sent from kubernetes-service-endpoints job + - attributes/remove_job_dummy + receivers: + - prometheus/infrastructure + - kubeletstats + - k8s_cluster + - prometheus/collector + +# OpenTelemetry configuration for Standalone collector +standaloneConfig: + receivers: + prometheus/applications: + config: + global: + scrape_interval: 60s + scrape_timeout: 60s + scrape_configs: + - job_name: applications + honor_timestamps: true + honor_labels: true + metrics_path: /metrics + scheme: http + kubernetes_sd_configs: + - role: pod + relabel_configs: + - action: keep + regex: true|"true" + source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape] + - action: replace + regex: (https?) + source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scheme] + target_label: __scheme__ + - action: replace + regex: (.+) + source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_path] + target_label: __metrics_path__ + - action: replace + regex: (.+?)(?::\d+)?;(\d+) + replacement: $$1:$$2 + source_labels: [__address__, __meta_kubernetes_pod_annotation_prometheus_io_port] + target_label: __address__ + - action: labelmap + regex: __meta_kubernetes_pod_annotation_prometheus_io_param_(.+) + replacement: __param_$$1 + - action: labelmap + regex: __meta_kubernetes_pod_label_(.+) + - action: replace + source_labels: [__meta_kubernetes_namespace] + target_label: namespace + - action: replace + source_labels: [__meta_kubernetes_pod_name] + target_label: pod + - source_labels: [__meta_kubernetes_pod_node_name] + action: replace + target_label: kubernetes_node + metric_relabel_configs: [] + prometheus/infrastructure: + config: + global: + scrape_interval: 30s + scrape_timeout: 30s + scrape_configs: + - job_name: windows-metrics + honor_timestamps: true + honor_labels: true + metrics_path: /metrics + scheme: http + kubernetes_sd_configs: + - role: pod + relabel_configs: + - source_labels: [__meta_kubernetes_pod_annotation_prometheus_windows_io_scrape] + action: keep + regex: true|"true" + metric_relabel_configs: [] + - job_name: kubernetes-service-endpoints + honor_timestamps: true + honor_labels: true + metrics_path: /metrics + scheme: http + kubernetes_sd_configs: + - role: endpoints + relabel_configs: + - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_scrape] + action: keep + regex: true|"true" + - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_scheme] + action: replace + target_label: __scheme__ + regex: (https?) + - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_path] + action: replace + target_label: __metrics_path__ + regex: (.+) + - source_labels: [__address__, __meta_kubernetes_service_annotation_prometheus_io_port] + action: replace + target_label: __address__ + regex: ([^:]+)(?::\d+)?;(\d+) + replacement: $$1:$$2 + - action: labelmap + regex: __meta_kubernetes_service_label_(.+) + - source_labels: [__meta_kubernetes_namespace] + action: replace + target_label: namespace + - source_labels: [__meta_kubernetes_service_name] + action: replace + target_label: service + - source_labels: [__meta_kubernetes_pod_name] + action: replace + target_label: pod + - source_labels: [__meta_kubernetes_pod_node_name] + action: replace + target_label: kubernetes_node + - source_labels: [__meta_kubernetes_service_annotation_logz_io_app] + action: replace + target_label: logzio_app + metric_relabel_configs: [] + processors: + filter/kubernetes360: + metrics: + datapoint: + - 'IsMatch(metric.name, "(${K8S_360_METRICS})") == true and attributes["logzio_app"] != "kubernetes360"' + service: + extensions: + - health_check + pipelines: + metrics/infrastructure: + exporters: + - prometheusremotewrite/infrastructure + processors: + - attributes/env_id + - batch + # - filter/kubernetes360 + - k8sattributes + receivers: + - prometheus/infrastructure + - kubeletstats + - k8s_cluster + - prometheus/collector + +# Kubernetes Object logs +k8sObjectsLogs: + enabled: false + config: + receivers: + # Watch for changes in Kubernetes objects + k8sobjects/watch: + objects: + - name: pods + mode: watch + exclude_watch_type: [ DELETED, BOOKMARK ] + - name: deployments + mode: watch + exclude_watch_type: [ DELETED, BOOKMARK ] + - name: daemonsets + mode: watch + exclude_watch_type: [ DELETED, BOOKMARK ] + - name: statefulsets + mode: watch + exclude_watch_type: [ DELETED, BOOKMARK ] + - name: jobs + mode: watch + exclude_watch_type: [ DELETED, BOOKMARK ] + - name: nodes + mode: watch + exclude_watch_type: [ DELETED, BOOKMARK ] + # Pull Kubernetes objects every 3 hours(default) + k8sobjects/pull: + objects: + - name: pods + mode: pull + interval: 180m + - name: deployments + mode: pull + interval: 180m + - name: daemonsets + mode: pull + interval: 180m + - name: statefulsets + mode: pull + interval: 180m + - name: jobs + mode: pull + interval: 180m + - name: nodes + mode: pull + interval: 180m + processors: + # Adds eventType key with value of type key, then sets type to k8s_object + transform/log_type: + error_mode: ignore + log_statements: + - context: log + statements: + - set(body["eventType"],body["type"]) where body["type"] != "k8s_object" + - set(body["type"], "k8s_object") + resource/env_id: + attributes: + # Adds env_id key with value from the secret + - key: env_id + action: insert + value: ${ENV_ID} + # For pulled objects, copy the log into object key + transform/pulled_object: + error_mode: ignore + log_statements: + - context: log + statements: + - set(body["object"],body) where body["object"] == nil + - keep_keys(body, ["object", "type", "eventType"]) + # Remove managed fields metadata key + transform/remove_managedfields: + error_mode: ignore + log_statements: + - context: log + statements: + - delete_key(body["object"]["metadata"], "managedFields") + exporters: + logzio/object_logs: + account_token: "${LOGZIO_OBJECTS_LOGS_TOKEN}" + region: "${LOGZIO_REGION}" + headers: + user-agent: "{{ .Chart.Name }}-{{ .Chart.Version }}-helm" + service: + pipelines: + logs/k8sobjects: + receivers: [k8sobjects/pull,k8sobjects/watch] + processors: [transform/pulled_object,transform/remove_managedfields,transform/log_type, resource/env_id] + exporters: [logzio/object_logs] + +windowsExporterInstallerJob: + interval: "*/10 * * * *" + concurrencyPolicy: Forbid # Future cronjob will run only after current job is finished + successfulJobsHistoryLimit: 1 + failedJobsHistoryLimit: 1 + ttlSecondsAfterFinished: 3600 # First job only (Not CronJob)