Ingest service exited with error DeltaTable { source: Protocol { source: Generic("filed to get tombstones") } }

[kpapadatos]

We are often seeing pod restarts with the following error. We run against ADLSv2 and Azure Event Hub for Kafka.

2024-08-11T01:30:05 [ERROR] - kafka_delta_ingest: Ingest service exited with error DeltaTable { source: Protocol { source: Generic("filed to get tombstones") } }
thread 'main' panicked at src/main.rs:185:14:
called `Result::unwrap()` on an `Err` value: DeltaTable { source: Protocol { source: Generic("filed to get tombstones") } }
stack backtrace:
   0:     0x55e168560daf - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h1e1a1972118942ad
   1:     0x55e1669b4e0b - core::fmt::write::hc090a2ffd6b28c4a
   2:     0x55e168529182 - std::io::Write::write_fmt::h8898bac6ff039a23
   3:     0x55e168562899 - std::sys_common::backtrace::print::ha96650907276675e
   4:     0x55e1685620be - std::panicking::default_hook::{{closure}}::h215c2a0a8346e0e0
   5:     0x55e168563333 - std::panicking::rust_panic_with_hook::hac8bdceee1e4fe2c
   6:     0x55e168562c02 - std::panicking::begin_panic_handler::{{closure}}::h00d785e82757ce3c
   7:     0x55e168562b59 - std::sys_common::backtrace::__rust_end_short_backtrace::h1628d957bcd06996
   8:     0x55e168562b46 - rust_begin_unwind
   9:     0x55e165fdfa02 - core::panicking::panic_fmt::hdc63834ffaaefae5
  10:     0x55e165fdfe55 - core::result::unwrap_failed::h82b551e0ff2b2176
  11:     0x55e167e81a3c - tokio::runtime::context::runtime::enter_runtime::h6708b0972b2db409
  12:     0x55e167e6a359 - kafka_delta_ingest::main::h70fa0c80617658b2
  13:     0x55e167e704e3 - std::sys_common::backtrace::__rust_begin_short_backtrace::h0da8ae65fe5dba64
  14:     0x55e167e6959f - main
  15:     0x7fcc3279124a - <unknown>
  16:     0x7fcc32791305 - __libc_start_main
  17:     0x55e16609c871 - _start
  18:                0x0 - <unknown>