Agent sidecar injection support via Admission Controller

[levan-m]

What this PR does / why we need it:

Adds support for agent sidecar injection configuration.

CECO-885

Which issue this PR fixes

(optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged)

• fixes #

Special notes for your reviewer:

First two commits sets up baselines, so easiest to review is go to third commit (or later).

Testing

Below we provide instructions how to test feature on Kind and Fargate clusters.

1. Agent will be installed in datadog-agent namespace. Application will be installed in fargate namespace.
2. Create secret in each namespace:

kubectl create secret generic datadog-secret -n datadog-agent --from-literal api-key=<YOUR_DATADOG_API_KEY> --from-literal token=<CLUSTER_AGENT_TOKEN>
kubectl create secret generic datadog-secret -n fargate --from-literal api-key=<YOUR_DATADOG_API_KEY> --from-literal token=<CLUSTER_AGENT_TOKEN>

3. Create below RBAC in fargate namespace:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: datadog-agent
  namespace: fargate
rules:
  - apiGroups:
      - ""
    resources:
      - nodes
      - namespaces
      - endpoints
    verbs:
      - get
      - list
  - apiGroups:
      - ""
    resources:
      - nodes/metrics
      - nodes/spec
      - nodes/stats
      - nodes/proxy
      - nodes/pods
      - nodes/healthz
    verbs:
      - get
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: datadog-agent
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: datadog-agent
subjects:
  - kind: ServiceAccount
    name: datadog-agent
    namespace: fargate
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: datadog-agent
  namespace: fargate

Kind

1. Create datadog.yaml values file for Helm installation.

datadog:
  apiKeyExistingSecret: datadog-secret
  clusterName: "kind-sidecar"

agents:
  enabled: false
    
clusterAgent:
  tokenExistingSecret: datadog-secret
  image:
    tag: 7.52.0-rc.2
  enabled: true
  admissionController:
    enabled: true
    agentSidecarInjection:
      enabled: true
      selectors:
        - objectSelector:
            matchLabels:
                "app": redis
      profiles:
        - env:
            - name: DD_KUBELET_TLS_VERIFY
              value: "false"

2. Install using helm install datadog ./charts/datadog -f datadog.yaml -n datadog-agent
3. Application manifest:

apiVersion: apps/v1
kind: Deployment
metadata:
 name: redis
spec:
 replicas: 1
 selector:
  matchLabels:
    app: redis
 template:
   metadata:
     labels:
       app: redis
       runsOn: nodeless
     name: redis
     annotations:
       ad.datadoghq.com/redis.check_names: '["redisdb"]'
       ad.datadoghq.com/redis.init_configs: '[{}]'
       ad.datadoghq.com/redis.instances: |
         [
           {
             "host": "%%host%%",
             "port": "6379"
           }
         ]         
   spec:
     serviceAccountName: datadog-agent
     containers:
     - name: redis
       image: redis:latest
       args:
         - "redis-server"
       ports:
       - containerPort: 6379

4. Apply manifest kubectl apply -f redis.yaml -n fargate.
5. Once pod is created it should be created with 2 containers, redis and datadog-agent-injected.

Fargate

1. Install Datadog Agent chart in datadog-agent namespace using:

helm install datadog ./charts/datadog -n datadog-agent
--set datadog.clusterName=cluster-name
--set agents.enabled=false
--set datadog.apiKeyExistingSecret=datadog-secret
--set clusterAgent.tokenExistingSecret=datadog-secret
--set clusterAgent.image.tag=7.52.0-rc.2
--set clusterAgent.admissionController.agentSidecarInjection.enabled=true
--set clusterAgent.admissionController.agentSidecarInjection.provider=fargate

2. Install an application in `fargate` namespace with a `agent.datadoghq.com/sidecar: "fargate"` pod label. Sample manifest:
```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
 name: redis
spec:
 replicas: 1
 selector:
  matchLabels:
    app: redis
 template:
   metadata:
     labels:
       app: redis
       agent.datadoghq.com/sidecar: "fargate"
     name: redis
     annotations:
       ad.datadoghq.com/redis.check_names: '["redisdb"]'
       ad.datadoghq.com/redis.init_configs: '[{}]'
       ad.datadoghq.com/redis.instances: |
         [
           {
             "host": "%%host%%",
             "port": "6379"
           }
         ]         
   spec:
     serviceAccountName: datadog-agent
     containers:
     - name: redis
       image: redis:latest
       args:
         - "redis-server"
       ports:
       - containerPort: 6379

3. Confirm redis pods are created with two containers.

Checklist

[Place an '[x]' (no spaces) in all applicable fields. Please remove unrelated fields.]

• Chart Version bumped
• Documentation has been updated with helm-docs (run: .github/helm-docs.sh)
• CHANGELOG.md has been updated
• Variables are documented in the README.md
• [] For Datadog Operator chart or value changes update the test baselines (run: make update-test-baselines)