Question: Do we need any logic on shutdown?

[ceckoslab]

I have been playing with chdb-node a lot lately - exploring what can I do with the project. I am working on a proof of concept a local emulator of a cloud based datawarehouse.

I programmed a lot logic in a single index.js file and I was running and stopping the index.js a few times a day.

Today I got a strange error after is was trying to start again the index.js with recent changes:

Attempt 1:

[1]    91711 abort      node index.js

Attempt 2::

node(91807,0x174d13000) malloc: *** error for object 0x6000001f4180: pointer being freed was not allocated
node(91807,0x174d13000) malloc: *** set a breakpoint in malloc_error_break to debug
node(91807,0x17470f000) malloc: *** error for object 0x600003a968c0: pointer being freed was not allocated
node(91807,0x17470f000) malloc: *** set a breakpoint in malloc_error_break to debug

After deleting the chdb data folder everything started working fine.

Here comes the question:

We we need to do or can we do something on SIGINT and SIGTERM ? I suppose that there could be cases where we could cause data corruption if we do not stop properly chdb while chdb is still processing data.

For example I noticed this code fragment in another project that takes requests and writes them to file system:

  // Handle close event
  process
    .once("message", (msg) => {
      if (msg === "shutdown") {
        shutdown(server);
      }
    })
    .once("SIGINT", () => shutdown(server))
    .once("SIGTERM", () => shutdown(server));
}

Full source code here: https://github.com/Azure/Azurite/blob/76f626284e4b4b58b95065bb3c92351f30af7f3d/src/blob/main.ts

[ceckoslab]

@auxten I was able to create a repro in a single file.

Here is an example:

const { query, Session } = require("chdb");

var ret;

// Test standalone query
ret = query("SELECT version(), 'Hello chDB', chdb()", "CSV");
console.log("Standalone Query Result:", ret);

// Test session query
// Create a new session instance
const session = new Session("./corrupt-on-shutdown");

session.query("CREATE DATABASE IF NOT EXISTS testdb;");

session.query("USE testdb;");

// Let's create a bunch of talbes
for (let t = 0; t < 20; t++) {
    console.log("Running create table num: " + t);
    session.query(`CREATE TABLE IF NOT EXISTS testtable_${t} (id UInt32) ENGINE = MergeTree() ORDER BY id;`)
}

// Let's insert bunch of data
for (let c = 0; c < 30; c++) {
    console.log("Running query num: " + c);
    session.query("INSERT INTO testtable_0 VALUES (1), (2), (3);");
} 

ret = session.query("SELECT * FROM testtable_0;")
console.log("Session Query Result:", ret);

// Handle the SIGINT signal (Ctrl+C)
process.on('SIGINT', () => {
    console.log("\nReceived SIGINT. Exiting gracefully...");
    process.exit(0);  // Exit with a success code
});

// Keep the script running indefinitely
console.log("Press Ctrl+C to stop the script.");
setInterval(() => {}, 1000);  // An empty interval to keep the event loop active

I get the error on every 2-3 runs.

I am waiting till I see the message "Press Ctrl+C to stop the script." and I press Ctrl+C few seconds after that.

System info:

• Apple M2 Pro
• macOS 14.5 (23F79)
• Node v20.9.0

  "dependencies": {
    "chdb": "^1.2.1"
  },

[auxten]

I should fix this issue first chdb-io/chdb#197