Handle Severe errors: How to process severe errors in Spanner to BigQuery Change stream dataflow

[pradeepnr]

Related Template(s)

Spanner_Change_Streams_to_BigQuery

What feature(s) are you requesting?

Hi,
we are using Spanner Change streams to BigQuery template to capture cdc.

Occasionally, we have seen errors (not related to template) which are written Retry folder in GCS bucket. After few retrials they are written to severe folder. When the error is fixed we observe the transactions in Retry folder are reprocessed where as the transactions in severe folder are not.

What is the procedure you would recommend to handle transactions in severe folder? Is there any available template we can use? Thanks in Advance.

Regards