New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add portable Mqtt source and sink transforms #32385

Open

ahmedabu98 wants to merge 1 commit into apache:master from ahmedabu98:mqtt_external

+468 −1

Open

Add portable Mqtt source and sink transforms #32385

External Mqtt source and sink

GitHub Actions / Test Results succeeded Aug 30, 2024 in 0s

All 143 tests pass, 1 skipped in 57m 7s

36 files 36 suites 57m 7s ⏱️
144 tests 143 ✅ 1 💤 0 ❌
637 runs 627 ✅ 10 💤 0 ❌

Results for commit 5008fc6.

Annotations

Check notice on line 0 in .github

github-actions / Test Results

1 skipped test found

There is 1 skipped test, see "Raw output" for the name of the skipped test.

Raw output


            org.apache.beam.sdk.io.kafka.KafkaIOTest ‑ testUnboundedSourceWithoutBoundedWrapper

Check notice on line 0 in .github

github-actions / Test Results

144 tests found

There are 144 tests, see "Raw output" for the full list of tests.

Raw output


            org.apache.beam.sdk.io.kafka.ConfluentSchemaRegistryDeserializerProviderTest ‑ testDeserialize
org.apache.beam.sdk.io.kafka.ConfluentSchemaRegistryDeserializerProviderTest ‑ testGetCoder
org.apache.beam.sdk.io.kafka.CustomTimestampPolicyWithLimitedDelayTest ‑ testCustomTimestampPolicyWithLimitedDelay
org.apache.beam.sdk.io.kafka.KafkaCommitOffsetTest ‑ testCommitOffsetDoFn
org.apache.beam.sdk.io.kafka.KafkaCommitOffsetTest ‑ testCommitOffsetError
org.apache.beam.sdk.io.kafka.KafkaCommitOffsetTest ‑ testKafkaOffsetComposite
org.apache.beam.sdk.io.kafka.KafkaCommitOffsetTest ‑ testKafkaOffsetCompositeLegacy
org.apache.beam.sdk.io.kafka.KafkaDlqTest ‑ testKafkaErrorFnFailure
org.apache.beam.sdk.io.kafka.KafkaDlqTest ‑ testKafkaErrorFnSuccess
org.apache.beam.sdk.io.kafka.KafkaIOExternalTest ‑ testConstructKafkaRead
org.apache.beam.sdk.io.kafka.KafkaIOExternalTest ‑ testConstructKafkaReadWithoutMetadata
org.apache.beam.sdk.io.kafka.KafkaIOExternalTest ‑ testConstructKafkaWrite
org.apache.beam.sdk.io.kafka.KafkaIOExternalTest ‑ testKafkaRecordToExternalKafkaRecord
org.apache.beam.sdk.io.kafka.KafkaIOExternalTest ‑ testKafkaRecordToExternalKafkaRecordWithNullKeyAndValue
org.apache.beam.sdk.io.kafka.KafkaIOIT ‑ testKafkaIOExternalRoundtripWithMetadataAndNullKeysAndValues
org.apache.beam.sdk.io.kafka.KafkaIOIT ‑ testKafkaIOReadsAndWritesCorrectlyInBatch
org.apache.beam.sdk.io.kafka.KafkaIOIT ‑ testKafkaIOSDFReadWithErrorHandler
org.apache.beam.sdk.io.kafka.KafkaIOIT ‑ testKafkaIOSDFResumesCorrectly
org.apache.beam.sdk.io.kafka.KafkaIOIT ‑ testKafkaIOWriteWithErrorHandler
org.apache.beam.sdk.io.kafka.KafkaIOIT ‑ testKafkaViaManagedSchemaTransformAvro
org.apache.beam.sdk.io.kafka.KafkaIOIT ‑ testKafkaViaManagedSchemaTransformJson
org.apache.beam.sdk.io.kafka.KafkaIOIT ‑ testKafkaWithDelayedStopReadingFunction
org.apache.beam.sdk.io.kafka.KafkaIOIT ‑ testKafkaWithDynamicPartitions
org.apache.beam.sdk.io.kafka.KafkaIOIT ‑ testKafkaWithStopReadingFunction
org.apache.beam.sdk.io.kafka.KafkaIOIT ‑ testWatermarkUpdateWithSparseMessages
org.apache.beam.sdk.io.kafka.KafkaIOReadImplementationCompatibilityTest ‑ testKafkaIOReadPropertiesEnumValuePresence
org.apache.beam.sdk.io.kafka.KafkaIOReadImplementationCompatibilityTest ‑ testPrimitiveKafkaIOReadPropertiesDefaultValueExistence
org.apache.beam.sdk.io.kafka.KafkaIOReadImplementationCompatibilityTest ‑ testReadTransformCreationWithBothImplementationBoundProperties
org.apache.beam.sdk.io.kafka.KafkaIOReadImplementationCompatibilityTest ‑ testReadTransformCreationWithLegacyImplementationBoundProperty
org.apache.beam.sdk.io.kafka.KafkaIOReadImplementationCompatibilityTest ‑ testReadTransformCreationWithSdfImplementationBoundProperty
org.apache.beam.sdk.io.kafka.KafkaIOTest ‑ testCommitOffsetsInFinalizeAndRedistributeErrors
org.apache.beam.sdk.io.kafka.KafkaIOTest ‑ testDeserializationWithHeaders
org.apache.beam.sdk.io.kafka.KafkaIOTest ‑ testExactlyOnceSink
org.apache.beam.sdk.io.kafka.KafkaIOTest ‑ testExactlyOnceSinkWithSendException
org.apache.beam.sdk.io.kafka.KafkaIOTest ‑ testKafkaWriteHeaders
org.apache.beam.sdk.io.kafka.KafkaIOTest ‑ testNumKeysIgnoredWithRedistributeNotEnabled
org.apache.beam.sdk.io.kafka.KafkaIOTest ‑ testReadAvroGenericRecordsWithConfluentSchemaRegistry
org.apache.beam.sdk.io.kafka.KafkaIOTest ‑ testReadAvroSpecificRecordsWithConfluentSchemaRegistry
org.apache.beam.sdk.io.kafka.KafkaIOTest ‑ testRecordsSink
org.apache.beam.sdk.io.kafka.KafkaIOTest ‑ testResolveDefaultApiTimeout
org.apache.beam.sdk.io.kafka.KafkaIOTest ‑ testRiskyConfigurationWarnsProperly
org.apache.beam.sdk.io.kafka.KafkaIOTest ‑ testSink
org.apache.beam.sdk.io.kafka.KafkaIOTest ‑ testSinkDisplayData
org.apache.beam.sdk.io.kafka.KafkaIOTest ‑ testSinkMetrics
org.apache.beam.sdk.io.kafka.KafkaIOTest ‑ testSinkProducerRecordsWithCustomPartition
org.apache.beam.sdk.io.kafka.KafkaIOTest ‑ testSinkProducerRecordsWithCustomTS
org.apache.beam.sdk.io.kafka.KafkaIOTest ‑ testSinkToMultipleTopics
org.apache.beam.sdk.io.kafka.KafkaIOTest ‑ testSinkWithSendErrors
org.apache.beam.sdk.io.kafka.KafkaIOTest ‑ testSinkWithSerializationErrors
org.apache.beam.sdk.io.kafka.KafkaIOTest ‑ testSourceDisplayData
org.apache.beam.sdk.io.kafka.KafkaIOTest ‑ testSourceWithExplicitPartitionsDisplayData
org.apache.beam.sdk.io.kafka.KafkaIOTest ‑ testSourceWithPatternDisplayData
org.apache.beam.sdk.io.kafka.KafkaIOTest ‑ testUnboundedReaderLogsCommitFailure
org.apache.beam.sdk.io.kafka.KafkaIOTest ‑ testUnboundedSource
org.apache.beam.sdk.io.kafka.KafkaIOTest ‑ testUnboundedSourceCheckpointMark
org.apache.beam.sdk.io.kafka.KafkaIOTest ‑ testUnboundedSourceCheckpointMarkWithEmptyPartitions
org.apache.beam.sdk.io.kafka.KafkaIOTest ‑ testUnboundedSourceCreateTimestamps
org.apache.beam.sdk.io.kafka.KafkaIOTest ‑ testUnboundedSourceCustomTimestamps
org.apache.beam.sdk.io.kafka.KafkaIOTest ‑ testUnboundedSourceLogAppendTimestamps
org.apache.beam.sdk.io.kafka.KafkaIOTest ‑ testUnboundedSourceMetrics
org.apache.beam.sdk.io.kafka.KafkaIOTest ‑ testUnboundedSourceRawSizeMetric
org.apache.beam.sdk.io.kafka.KafkaIOTest ‑ testUnboundedSourceSplits
org.apache.beam.sdk.io.kafka.KafkaIOTest ‑ testUnboundedSourceStartReadTime
org.apache.beam.sdk.io.kafka.KafkaIOTest ‑ testUnboundedSourceStartReadTimeException
org.apache.beam.sdk.io.kafka.KafkaIOTest ‑ testUnboundedSourceTimestamps
org.apache.beam.sdk.io.kafka.KafkaIOTest ‑ testUnboundedSourceWithExceptionInKafkaFetch
org.apache.beam.sdk.io.kafka.KafkaIOTest ‑ testUnboundedSourceWithExplicitPartitions
org.apache.beam.sdk.io.kafka.KafkaIOTest ‑ testUnboundedSourceWithPartiallyMatchedPattern
org.apache.beam.sdk.io.kafka.KafkaIOTest ‑ testUnboundedSourceWithPattern
org.apache.beam.sdk.io.kafka.KafkaIOTest ‑ testUnboundedSourceWithSingleTopic
org.apache.beam.sdk.io.kafka.KafkaIOTest ‑ testUnboundedSourceWithUnmatchedPattern
org.apache.beam.sdk.io.kafka.KafkaIOTest ‑ testUnboundedSourceWithWrongTopic
org.apache.beam.sdk.io.kafka.KafkaIOTest ‑ testUnboundedSourceWithoutBoundedWrapper
org.apache.beam.sdk.io.kafka.KafkaIOTest ‑ testUnreachableKafkaBrokers
org.apache.beam.sdk.io.kafka.KafkaIOTest ‑ testValuesSink
org.apache.beam.sdk.io.kafka.KafkaIOTest ‑ testWithInvalidConsumerPollingTimeout
org.apache.beam.sdk.io.kafka.KafkaIOTest ‑ testWithValidConsumerPollingTimeout
org.apache.beam.sdk.io.kafka.KafkaIOUtilsTest ‑ testOffsetConsumerConfigOverrides
org.apache.beam.sdk.io.kafka.KafkaReadSchemaTransformProviderTest ‑ testBuildTransformWithAvroSchema
org.apache.beam.sdk.io.kafka.KafkaReadSchemaTransformProviderTest ‑ testBuildTransformWithJsonSchema
org.apache.beam.sdk.io.kafka.KafkaReadSchemaTransformProviderTest ‑ testBuildTransformWithManaged
org.apache.beam.sdk.io.kafka.KafkaReadSchemaTransformProviderTest ‑ testBuildTransformWithProtoFormat
org.apache.beam.sdk.io.kafka.KafkaReadSchemaTransformProviderTest ‑ testBuildTransformWithProtoFormatWrongMessageName
org.apache.beam.sdk.io.kafka.KafkaReadSchemaTransformProviderTest ‑ testBuildTransformWithProtoSchemaFormat
org.apache.beam.sdk.io.kafka.KafkaReadSchemaTransformProviderTest ‑ testBuildTransformWithRawFormat
org.apache.beam.sdk.io.kafka.KafkaReadSchemaTransformProviderTest ‑ testBuildTransformWithoutProtoSchemaFormat
org.apache.beam.sdk.io.kafka.KafkaReadSchemaTransformProviderTest ‑ testFindTransformAndMakeItWork
org.apache.beam.sdk.io.kafka.KafkaReadSchemaTransformProviderTest ‑ testManagedMappings
org.apache.beam.sdk.io.kafka.KafkaReadSchemaTransformProviderTest ‑ testValidConfigurations
org.apache.beam.sdk.io.kafka.KafkaRecordCoderTest ‑ testCoderIsSerializableWithWellKnownCoderType
org.apache.beam.sdk.io.kafka.KafkaRecordCoderTest ‑ testKafkaRecordSerializableWithHeaders
org.apache.beam.sdk.io.kafka.KafkaRecordCoderTest ‑ testKafkaRecordSerializableWithNullValueHeader
org.apache.beam.sdk.io.kafka.KafkaRecordCoderTest ‑ testKafkaRecordSerializableWithoutHeaders
org.apache.beam.sdk.io.kafka.KafkaSchemaTransformTranslationTest ‑ testReCreateReadTransformFromRow
org.apache.beam.sdk.io.kafka.KafkaSchemaTransformTranslationTest ‑ testReadTransformProtoTranslation
org.apache.beam.sdk.io.kafka.KafkaSchemaTransformTranslationTest ‑ testRecreateWriteTransformFromRow
org.apache.beam.sdk.io.kafka.KafkaSchemaTransformTranslationTest ‑ testWriteTransformProtoTranslation
org.apache.beam.sdk.io.kafka.KafkaWriteSchemaTransformProviderTest ‑ testBuildTransformWithManaged
org.apache.beam.sdk.io.kafka.KafkaWriteSchemaTransformProviderTest ‑ testKafkaErrorFnProtoSuccess
org.apache.beam.sdk.io.kafka.KafkaWriteSchemaTransformProviderTest ‑ testKafkaErrorFnRawSuccess
org.apache.beam.sdk.io.kafka.KafkaWriteSchemaTransformProviderTest ‑ testKafkaErrorFnSuccess
org.apache.beam.sdk.io.kafka.KafkaWriteSchemaTransformProviderTest ‑ testManagedMappings
org.apache.beam.sdk.io.kafka.LocalDeserializerProviderTest ‑ testInferKeyCoder
org.apache.beam.sdk.io.kafka.LocalDeserializerProviderTest ‑ testInferKeyCoderFailure
org.apache.beam.sdk.io.kafka.ProducerRecordCoderTest ‑ testCoderIsSerializableWithWellKnownCoderType
org.apache.beam.sdk.io.kafka.ProducerRecordCoderTest ‑ testProducerRecordSerializableWithHeaders
org.apache.beam.sdk.io.kafka.ProducerRecordCoderTest ‑ testProducerRecordSerializableWithPartition
org.apache.beam.sdk.io.kafka.ProducerRecordCoderTest ‑ testProducerRecordSerializableWithTimestamp
org.apache.beam.sdk.io.kafka.ProducerRecordCoderTest ‑ testProducerRecordSerializableWithoutHeaders
org.apache.beam.sdk.io.kafka.ProducerRecordCoderTest ‑ testProducerRecordSerializableWithoutPartition
org.apache.beam.sdk.io.kafka.ProducerRecordCoderTest ‑ testProducerRecordSerializableWithoutTimestamp
org.apache.beam.sdk.io.kafka.ProducerRecordCoderTest ‑ testProducerRecordStructuralValueWithHeadersApi
org.apache.beam.sdk.io.kafka.ProducerRecordCoderTest ‑ testProducerRecordStructuralValueWithoutHeadersApi
org.apache.beam.sdk.io.kafka.ReadFromKafkaDoFnTest ‑ testBounded
org.apache.beam.sdk.io.kafka.ReadFromKafkaDoFnTest ‑ testConstructorWithPollTimeout
org.apache.beam.sdk.io.kafka.ReadFromKafkaDoFnTest ‑ testInitialRestrictionWhenHasStartOffset
org.apache.beam.sdk.io.kafka.ReadFromKafkaDoFnTest ‑ testInitialRestrictionWhenHasStartTime
org.apache.beam.sdk.io.kafka.ReadFromKafkaDoFnTest ‑ testInitialRestrictionWhenHasStopOffset
org.apache.beam.sdk.io.kafka.ReadFromKafkaDoFnTest ‑ testInitialRestrictionWhenHasStopTime
org.apache.beam.sdk.io.kafka.ReadFromKafkaDoFnTest ‑ testInitialRestrictionWithConsumerPosition
org.apache.beam.sdk.io.kafka.ReadFromKafkaDoFnTest ‑ testInitialRestrictionWithException
org.apache.beam.sdk.io.kafka.ReadFromKafkaDoFnTest ‑ testProcessElement
org.apache.beam.sdk.io.kafka.ReadFromKafkaDoFnTest ‑ testProcessElementWhenTopicPartitionIsRemoved
org.apache.beam.sdk.io.kafka.ReadFromKafkaDoFnTest ‑ testProcessElementWhenTopicPartitionIsStopped
org.apache.beam.sdk.io.kafka.ReadFromKafkaDoFnTest ‑ testProcessElementWithDeserializationExceptionDefaultRecordHandler
org.apache.beam.sdk.io.kafka.ReadFromKafkaDoFnTest ‑ testProcessElementWithDeserializationExceptionRecordingRecordHandler
org.apache.beam.sdk.io.kafka.ReadFromKafkaDoFnTest ‑ testProcessElementWithEmptyPoll
org.apache.beam.sdk.io.kafka.ReadFromKafkaDoFnTest ‑ testProcessElementWithException
org.apache.beam.sdk.io.kafka.ReadFromKafkaDoFnTest ‑ testRawSizeMetric
org.apache.beam.sdk.io.kafka.ReadFromKafkaDoFnTest ‑ testUnbounded
org.apache.beam.sdk.io.kafka.TopicPartitionCoderTest ‑ testEncodeDecodeRoundTrip
org.apache.beam.sdk.io.kafka.TopicPartitionCoderTest ‑ testToString
org.apache.beam.sdk.io.kafka.WatchForKafkaTopicPartitionsTest ‑ testGetAllTopicPartitions
org.apache.beam.sdk.io.kafka.WatchForKafkaTopicPartitionsTest ‑ testGetAllTopicPartitionsWithGivenPattern
org.apache.beam.sdk.io.kafka.WatchForKafkaTopicPartitionsTest ‑ testGetAllTopicPartitionsWithGivenTopics
org.apache.beam.sdk.io.kafka.WatchForKafkaTopicPartitionsTest ‑ testPartitionGrowth
org.apache.beam.sdk.io.kafka.WatchForKafkaTopicPartitionsTest ‑ testPartitionSingle
org.apache.beam.sdk.io.kafka.upgrade.KafkaIOTranslationTest ‑ testReCreateReadTransformFromRow
org.apache.beam.sdk.io.kafka.upgrade.KafkaIOTranslationTest ‑ testReCreateReadTransformWithTopics
org.apache.beam.sdk.io.kafka.upgrade.KafkaIOTranslationTest ‑ testReCreateWriteTransformFromRow
org.apache.beam.sdk.io.kafka.upgrade.KafkaIOTranslationTest ‑ testReadTransformRowIncludesAllFields
org.apache.beam.sdk.io.kafka.upgrade.KafkaIOTranslationTest ‑ testReadTransformURNDiscovery
org.apache.beam.sdk.io.kafka.upgrade.KafkaIOTranslationTest ‑ testWriteTransformRowIncludesAllFields
org.apache.beam.sdk.io.kafka.upgrade.KafkaIOTranslationTest ‑ testWriteTransformURNDiscovery

View more details on GitHub Actions