[destination-azure-blob-storage] Writes to Blob Store Work First Time Only

[amr-abouelleil-ic]

Connector Name

destination-azure-blob-storage

Connector Version

0.2.2

What step the error happened?

During the sync

Relevant information

Problem

Currently, writes to blob store destinations seem to only work the first time.

Steps to Reproduce

1. Add multiple files to an FTP source location
2. Set up a source connector to for this source location
3. Set up a destination connector to an Azure Blob Storage container
4. Set up source-destination connection
5. Run a sync, confirm data is in blob store
6. Run a second sync, it fails with a 'blobNotFound' error.

Relevant log output

│ orchestrator 2024-12-09 18:28:39 replication-orchestrator > failures: [ {                                                                                                                                                        │
│ orchestrator   "failureOrigin" : "destination",                                                                                                                                                                                  │
│ orchestrator   "failureType" : "system_error",                                                                                                                                                                                   │
│ orchestrator   "internalMessage" : "com.azure.storage.blob.models.BlobStorageException: Status code 404, \"<?xml version=\"1.0\" encoding=\"utf-8\"?><Error><Code>BlobNotFound</Code><Message>The specified blob does not exist. │
│ orchestrator   "externalMessage" : "Something went wrong in the connector. See the logs for more details.",                                                                                                                      │
│ orchestrator   "metadata" : {                                                                                                                                                                                                    │
│ orchestrator     "attemptNumber" : 0,                                                                                                                                                                                            │
│ orchestrator     "jobId" : 66,                                                                                                                                                                                                   │
│ orchestrator     "from_trace_message" : true,                                                                                                                                                                                    │
│ orchestrator     "connector_command" : "write"                                                                                                                                                                                   │
│ orchestrator   },

Contribute

• Yes, I want to contribute

[marcosmarxm]

@amr-abouelleil-ic, feel free to ping me if you need any assistance in contributing to fix the issue.

[amr-abouelleil-ic]

@amr-abouelleil-ic, feel free to ping me if you need any assistance in contributing to fix the issue.

Thanks, it does seem like this is a known issue. Is this not intended to be fixed by the creators of this connector?

These seem related:
#14662

https://discuss.airbyte.io/t/error-syncing-after-the-first-time-with-azure-blob-storage-destination/8276