Skip to content

Commit

Permalink
[Doc] Add doc for includeTopicToMetadata configuration (#864)
Browse files Browse the repository at this point in the history
### Motivation

Add doc for #862
  • Loading branch information
RobertIndie authored Jan 16, 2024
1 parent ba24f85 commit 0e7c1e8
Show file tree
Hide file tree
Showing 4 changed files with 4 additions and 0 deletions.
1 change: 1 addition & 0 deletions docs/aws-s3-sink.md
Original file line number Diff line number Diff line change
Expand Up @@ -155,6 +155,7 @@ Before using the AWS S3 sink connector, you need to configure it. This table out
| `avroCodec` | String | False | false | snappy | Compression codec used when formatType=`avro`. Available compression types are: none (no compression), deflate, bzip2, xz, zstandard, snappy. |
| `parquetCodec` | String | False | false | gzip | Compression codec used when formatType=`parquet`. Available compression types are: none (no compression), snappy, gzip, lzo, brotli, lz4, zstd. |
| `jsonAllowNaN` | Boolean | False | false | false | Recognize 'NaN', 'INF', '-INF' as legal floating number values when formatType=`json`. Since JSON specification does not allow such values this is a non-standard feature and disabled by default. |
| `includeTopicToMetadata` | Boolean | False | false | false | Include the topic name to the metadata. |

## Advanced features

Expand Down
1 change: 1 addition & 0 deletions docs/azure-blob-storage-sink.md
Original file line number Diff line number Diff line change
Expand Up @@ -135,6 +135,7 @@ Before using the Azure Blob Storage sink connector, you need to configure it. Th
| `avroCodec` | String | False | false | snappy | Compression codec used when formatType=`avro`. Available compression types are: none (no compression), deflate, bzip2, xz, zstandard, snappy. |
| `parquetCodec` | String | False | false | gzip | Compression codec used when formatType=`parquet`. Available compression types are: none (no compression), snappy, gzip, lzo, brotli, lz4, zstd. |
| `jsonAllowNaN` | Boolean | False | false | false | Recognize 'NaN', 'INF', '-INF' as legal floating number values when formatType=`json`. Since JSON specification does not allow such values this is a non-standard feature and disabled by default. |
| `includeTopicToMetadata` | Boolean | False | false | false | Include the topic name to the metadata. |

There are three methods to authenticate with Azure Blob Storage:
1. `azureStorageAccountConnectionString`: This method involves using the Azure Blob Storage connection string for authentication. It's the simplest method as it only requires the connection string.
Expand Down
1 change: 1 addition & 0 deletions docs/cloud-storage-sink.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,7 @@ You can create a configuration file (JSON or YAML) to set the following properti
| `avroCodec` | String | False | snappy | Compression codec used when formatType=`avro`. Available compression types are: null (no compression), deflate, bzip2, xz, zstandard, snappy. |
| `parquetCodec` | String | False | gzip | Compression codec used when formatType=`parquet`. Available compression types are: null (no compression), snappy, gzip, lzo, brotli, lz4, zstd. |
| `jsonAllowNaN` | Boolean | False | false | Recognize 'NaN', 'INF', '-INF' as legal floating number values when formatType=`json`. Since JSON specification does not allow such values this is a non-standard feature and disabled by default. |
| `includeTopicToMetadata` | Boolean | False | false | Include the topic name to the metadata. |
The provided AWS credentials must have permissions to access AWS resources.
To use the Cloud Storage sink connector, the suggested permission policies for AWS S3 are:
Expand Down
1 change: 1 addition & 0 deletions docs/google-cloud-storage-sink.md
Original file line number Diff line number Diff line change
Expand Up @@ -139,6 +139,7 @@ Before using the Google Cloud Storage sink connector, you need to configure it.
| `avroCodec` | String | False | false | snappy | Compression codec used when formatType=`avro`. Available compression types are: none (no compression), deflate, bzip2, xz, zstandard, snappy. |
| `parquetCodec` | String | False | false | gzip | Compression codec used when formatType=`parquet`. Available compression types are: none (no compression), snappy, gzip, lzo, brotli, lz4, zstd. |
| `jsonAllowNaN` | Boolean | False | false | false | Recognize 'NaN', 'INF', '-INF' as legal floating number values when formatType=`json`. Since JSON specification does not allow such values this is a non-standard feature and disabled by default. |
| `includeTopicToMetadata` | Boolean | False | false | false | Include the topic name to the metadata. |

## Advanced features

Expand Down

0 comments on commit 0e7c1e8

Please sign in to comment.