"Storage is about to be deleted. Core dumped" exception when running chdb-node.

[jboga]

Hi there,
We have a ClickHouse Cloud service in production and we're trying to use chdb-node in our tests suite to simulate that service.
When running our tests suite we run into an issue where an exception is raised from the ClickHouse engine and we're having a hard time identifying what's wrong.

For each series of tests, we instantiate a new Session, perform some operations and then clean up the session.
After a few operations, we get the following error:

terminate called after throwing an instance of 'DB::Exception'
  what():  Storage is about to be deleted. Done active task as if it was aborted.
Aborted (core dumped)

We also get the following one (but less often):

terminate called after throwing an instance of 'DB::Exception'
  what():  Cancelled merging parts
Aborted (core dumped)

Configuration:

• NodeJS v20.11.0
• chdb-node v1.1.3

I'm curious if you guys ran into this issue in other context.
I also wrote a simple script that generates many sessions in series and for each session performs a few insert and select operations, and I didn't encounter the problem. So it could be something with our CI suite but so far I'm clueless.

I'm pretty positive I'm not deleting the session while it's been used but maybe I made a mistake.

Any pointers or ideas would be much appreciated. Also happy to re-post this somewhere else if it's more suitable there.
Thank you.

[jboga]

I confirm I'm not unintentionally calling session.cleanup() int he middle of the process.

[jboga]

We also noticed that this is always happening after performing 4 or 5 queries (no less, no more).

[jboga]

The following script triggers the error in my CI env:

import chdb from 'chdb';

Array.from(Array(10)).forEach(() => {
  const session = new chdb.Session();

  session.query('CREATE DATABASE local');

  session.query(`
  USE local; CREATE TABLE IF NOT EXISTS knowledge_base_portal_interface_event
  (
      timestamp                   DateTime64,
      company_id                  Int64,
      event_type                  String,
      locale                      String,
      article_id                  Int64 DEFAUlT 0,
  )
  ENGINE = MergeTree
  ORDER BY (company_id, locale, timestamp)
  COMMENT 'This table represents a store of knowledge base portal interface events';
  `);

  Array.from(Array(15)).forEach((_x, index) => {
    session.query(
      `USE local; INSERT INTO knowledge_base_portal_interface_event FORMAT JSONEachRow [{"company_id":${index},"locale":"en","timestamp":1717780952772,"event_type":"article_update","article_id":7},{"company_id":${
        index + 100
      },"locale":"en","timestamp":1717780952772,"event_type":"article_update","article_id":7}]`
    );
  });

  session.query('USE local; SELECT * FROM knowledge_base_portal_interface_event', 'JSONObjectEachRow');
  session.cleanup();
});

And below are the CI container info:

System information:
 Server Version: 24.0.9
 Storage Driver: overlay2
  Backing Filesystem: xfs
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Kernel Version: 5.15.0-1057-aws
 Operating System: Ubuntu 20.04.6 LTS
 OSType: linux
 Architecture: x86_64

[jboga]

Note that I also tried to use the latest version of chdb (1.4.1) and got the same error.

[auxten]

Can not reproduce it on my macOS + node v16. Testing on Linux

[auxten]

Reproduced on Linux + Node 20
It's a MergeTree background task terminating issue, will follow on chdb-io/chdb#226

[auxten]

PR: chdb-io/chdb#227

[auxten]

Fixed #12