Add helm chart installation for OTel Agent

[dineshg13]

What this PR does / why we need it:

Adds OTel Agent container . OTel Agent is a Datadog's OpenTelemetry's collector distro that runs along with Agent.

The PR add example collector configuration and corresponding agent configuration.

To test the changes use the below helm command

helm install my ./charts/datadog --set-file datadog.otelCollector.config=examples/datadog/otel_collector_config.yaml -f examples/datadog/agent_otel_collector.yaml  --set datadog.apiKey=${DD_API_KEY}              

Special notes for your reviewer:

Checklist

[Place an '[x]' (no spaces) in all applicable fields. Please remove unrelated fields.]

• Chart Version bumped
• Documentation has been updated with helm-docs (run: .github/helm-docs.sh)
• CHANGELOG.md has been updated
• Variables are documented in the README.md
• For Datadog Operator chart or value changes update the test baselines (run: make update-test-baselines)

[clamoriniere]

a quick first review

[clamoriniere]

I think network policies need also to be updated. to expose the port.

and hostPort doesn't need to be set by default. see how it is done for the agent container

helm-charts/charts/datadog/templates/_container-agent.yaml

Lines 35 to 37 in 036f328

	{{- if .http.useHostPort }}
	hostPort: {{ .http.endpoint | regexFind ":[0-9]+$" | trimPrefix ":" }}
	{{- end }}

[dineshg13]

Updated.

[clamoriniere]

Hi @dineshg13

the changes present seems to be ok.
However some changes are missing:

• update the NetworkPolicy and Cilium Network Policy to allow the traffic on the new port.
• Add a test cas in https://github.com/DataDog/helm-charts/tree/main/charts/datadog/ci to test that the the generated manifests are valid and the agent start properly

Also I'm wondering if the work to be supported in GKE autopilot has been done already with the @DataDog/container-ecosystems team to define the new profil. If not maybe, the best for not is to disable OTEL container if GKE autopilot support is enable.

I let the container-ecosystem team responsible of the datadog chart the end of this PR review.

last, If you can add in the PR description more information about how to test it. For example the minimum values.yaml needed to install the chart with the OTEL container. 🙇

[clamoriniere]

@DataDog/container-ecosystems have you been able to update the Agent profile in GKE autopilot to support the additional OTEL container?

[dineshg13]

@clamoriniere how do i update Celium/Network policy ? How can we handle the case where customer might use a different port for OTLP receiver ?

[clamoriniere]

Can we make the default configuration shorter for otel support activation. like only to have to set datadog.otelcollector.enabled: true

When I saw the dash demo, it looks like only 2 parameters was needed.

I hope that we can simplify at least like

agents:
  image:
    tagSuffix: ot-beta
datadog:
  apiKey: "00000000000000000000000000000000"
  appKey: "0000000000000000000000000000000000000000"
  otelCollector:
    enabled: true

so my questions are, when datadog.otelcollector.enabled: true can we:

• generate a default configuration that use the API provide in the datadog.apiKey field?
• set agents.image.tagSuffix with ot-beta?
• inject the needed envvars in the core-agent?

[dineshg13]

This is a good feedback. Thanks for same. I have addressed same in the latest commit .
if datadog.otelcollector.enabled: true we do the following

1. set agents.image.tagSuffix with ot-beta if image tag isn't set
2. inject the needed envvar DD_OTELCOLLECTOR_ENABLED in the core-agent .
3. if datadog.otelcollector.config is not provided, use the default config.

[clamoriniere]

only 2 new comments

[clamoriniere]

Hi @dineshg13

Thank you for considering all my requests for changes. 🙇
I think the result is great the user experience for OTEL user is now good 👍

cheers

[dineshg13]

/merge

[dd-devflow]

❌ MergeQueue

You are not allowed to use the merge queue towards main.

If you need support, contact us on Slack #devflow with those details!

[levan-m]

/merge

[dd-devflow]

🚂 MergeQueue: waiting for PR to be ready

This merge request is not mergeable yet, because of pending checks/missing approvals. It will be added to the queue as soon as checks pass and/or get approvals.
Note: if you pushed new commits since the last approval, you may need additional approval.
You can remove it from the waiting list with /remove command.

Use /merge -c to cancel this operation!

[dd-devflow]

🚂 MergeQueue: pull request added to the queue

The median merge time in main is 38m.

Use /merge -c to cancel this operation!