Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add camunda exporter support to zeebe #2476

Draft
wants to merge 11 commits into
base: main
Choose a base branch
from

Conversation

drodriguez-305
Copy link
Contributor

Which problem does the PR fix?

Related to #2294

What's in this PR?

Adding support for the new camunda exporter.

Checklist

Please make sure to follow our Contributing Guide.

Before opening the PR:

  • In the repo's root dir, run make go.update-golden-only.
  • There is no other open pull request for the same update/change.
  • Tests for charts are added (if needed).
  • In-repo documentation are updated (if needed).

After opening the PR:

  • Did you sign our CLA (Contributor License Agreement)? It will show once you open the PR.
  • Did all checks/tests pass in the PR?

@drodriguez-305 drodriguez-305 added kind/enhancement New feature or request cycle/alpha1 Tasks will be done in alpha1 cycle labels Oct 15, 2024
@drodriguez-305 drodriguez-305 added this to the 8.7 Release Cycle milestone Oct 15, 2024
@drodriguez-305 drodriguez-305 self-assigned this Oct 15, 2024
@drodriguez-305 drodriguez-305 linked an issue Oct 15, 2024 that may be closed by this pull request
@drodriguez-305 drodriguez-305 marked this pull request as draft October 15, 2024 20:44
@drodriguez-305 drodriguez-305 changed the title enhancement: add camunda exporter support to zeebe feat: add camunda exporter support to zeebe Oct 15, 2024
@drodriguez-305
Copy link
Contributor Author

@aabouzaid I just did a first iteration which follows the structure of the other exporters as found here elasticsearch exporter config.

I am also passing any config available for elasticsearch or opensearch.

@aabouzaid
Copy link
Member

@drodriguez-305 LGTM 👍
Just let's check why the deployment failed.

@drodriguez-305 drodriguez-305 added test-persistent Don't delete the Helm deployment created by the PR and use fixed names and removed test-persistent Don't delete the Helm deployment created by the PR and use fixed names labels Oct 16, 2024
@drodriguez-305
Copy link
Contributor Author

drodriguez-305 commented Oct 17, 2024

@aabouzaid at first we noticed that the url was added with an extra line. After fixing the url template rendering issue, I am still encountering getting an error in zeebe.

@mustafadagher can you confirm if there is anything that might be missing?

│ 2024-10-17 21:03:55.691 [Broker-0] [zb-fs-workers-2] [Exporter-1] ERROR                                                                                                                 │
│       io.camunda.zeebe.broker.exporter - Failed to open exporter 'CamundaExporter'. Retrying...                                                                                         │
│ java.lang.IllegalArgumentException: HTTP Host may not be null

Confirmed by checking the configmap and it seems to be rendering the url correctly into the camunda exporter.

Name:         integration-zeebe-configuration
Namespace:    camunda-pr-2476-intg-alpha
Labels:       app=camunda-platform
              app.kubernetes.io/component=zeebe-broker
              app.kubernetes.io/instance=integration
              app.kubernetes.io/managed-by=Helm
              app.kubernetes.io/name=camunda-platform
              app.kubernetes.io/part-of=camunda-platform
              app.kubernetes.io/version=8.6.0
              helm.sh/chart=camunda-platform-0.0.0-ci-snapshot-alpha
Annotations:  meta.helm.sh/release-name: integration
              meta.helm.sh/release-namespace: camunda-pr-2476-intg-alpha

Data
====
startup.sh:
----
#!/usr/bin/env bash
set -eux -o pipefail
export ZEEBE_BROKER_CLUSTER_NODEID=${ZEEBE_BROKER_CLUSTER_NODEID:-$[${K8S_NAME##*-} * 1 + 0]}

if [ "$(ls -A /exporters/)" ]; then
  mkdir -p /usr/local/zeebe/exporters/
  cp -a /exporters/*.jar /usr/local/zeebe/exporters/
else
  echo "No exporters available."
fi
exec /usr/local/zeebe/bin/broker

application.yaml:
----
zeebe:
  broker:
    exporters:
      elasticsearch:
        className: "io.camunda.zeebe.exporter.ElasticsearchExporter"
        args:
          url: "http://integration-elasticsearch:9200"
          index:
            prefix: "zeebe-record"
      CamundaExporter:
        className: "io.camunda.exporter.CamundaExporter"
        args:
          connect:
            type: elasticsearch
            url: "http://integration-elasticsearch:9200"
          index:
            prefix: "zeebe-record"
          createSchema: true
    gateway:
      enable: true
      network:
        port: 26500
      security:
        enabled: false
        authentication:
          mode: none
    network:
      host: 0.0.0.0
      commandApi:
        port: 26501
      internalApi:
        port: 26502
      monitoringApi:
        port: "9600"
    cluster:
      clusterSize: "3"
      replicationFactor: "3"
      partitionsCount: "3"
      clusterName: integration-zeebe
    threads:
      cpuThreadCount: "3"
      ioThreadCount: "3"

# Camunda Database configuration
camunda.database:
  type: elasticsearch
  # Cluster name
  clusterName: elasticsearch
  # Elasticsearch full url
  url: "http://integration-elasticsearch:9200"

broker-log4j2.xml:
----


BinaryData
====

Events:  <none>

@mustafadagher
Copy link

@mustafadagher can you confirm if there is anything that might be missing?

@drodriguez-305 to me, the configuration you posted looks correct. but I'm not sure if there might be wrong indentation or something that causes the url not to be read!

@drodriguez-305 drodriguez-305 removed the test-persistent Don't delete the Helm deployment created by the PR and use fixed names label Oct 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component/zeebe cycle/alpha1 Tasks will be done in alpha1 cycle kind/enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[ENHANCEMENT] Execution Identity integration - iteration 2
3 participants