EMS Kafka Connector v1.8.8

Merge pull request #65 from celonis/fix/DP-0000-proxy_bug

Fixing a bug with proxy not being used while configured

EMS Kafka Connector v1.8.7

Add a new option, connect.ems.convert.lowercase.fields that, when set to true, will make the connector lowercase all the field names of the outgoing records.

EMS Kafka Connector v1.8.6

• DP-2782: Fix a bug that prevented optional fields to be preserved in some cases during schema evolution.
• DP-2915: Transform enum AVRO values into plain strings during flattening. This allows a more efficient uploading of parquet files when dealing with record-name-strategy AVRO topics with incompatible enum types.
• DP-2918: Introduce a new boolean configuration parameter connect.ems.error.policy.continue.on.invalid.input, defaulting to false. When set to true, the connector will not fail when failing transformation of single records (this includes problematic JSON that triggers failures during schema inference). Failed records will be sent to the DLQ, if configured.

EMS Kafka Connector v1.8.5

DP-2429 | Introduce incremental schema evolution (#60)

* introduced incremental schema evolution
* reset schema when evolution fails

EMS Kafka Connector v1.8.4

• Improve e2e tests
• Address vulnerabilities report on some dependencies
• Add validation for connect.ems.data.fallback.varchar.length config parameter, failing if the provided value is bigger than 65000

EMS Kafka Connector v1.8.3

Merge pull request #54 from celonis/fix/e2e

Fix for e2e tests

EMS Kafka Connector v1.8.2

Merge pull request #50 from celonis/bug/flat-nestion-union-null

Fix a bug for a flattener when a schema that have a nested schema as …

EMS Kafka Connector v1.8.1

DP-1710 | Introduce timeout for EmsSinkTask#put (#48)

* introduce timeout for EmsSinkTask#put
* add e2e test
* avoid ambiguity around java TimeoutException

EMS Kafka Connector v1.8.0

Key changes

• The limitation of commit.size.bytes has been reduced from >= 1MB to >= 100kB.
• The default value for row.group.size.bytes has been reduced to 10KB.
• A piece of config handling logic has been added to ensure that the value of row.size.bytes is constrained to MIN(row.group.size.bytes, commit.size.bytes). This is meant to prevent inconsistent and potentially problematic configurations.

EMS Kafka Connector v1.7.2

• Renamed connect.ems.parquet.write.flush.records option to connect.ems.parquet.row.group.size.bytes (name and documentation were misleading). Connector using old configuration key will end up using the new default value of 1024 * 1024
• [Internal] Use a single-instance of Http client per connector task
• [Internal] Add a 90s global timeout to the Http client