Apply patches for SQLCipher 9.6.0 / BearSSL 0.6

.gitignore

@@ -40,3 +40,6 @@ lib/binding
 temp/
 TODO
 .local
+
+# Generated binaries
+bin/

.gitlab-ci.yml

@@ -0,0 +1,34 @@
+workflow:
+  rules:
+    - if: $CI_COMMIT_TAG
+    - if: $CI_COMMIT_BRANCH
+
+image: docker.io/node:16-buster
+
+.cache: &cache
+  cache:
+    key:
+      files:
+        - package-lock.json
+    paths:
+      - .npm/
+      - node_modules/
+
+audit:
+  stage: test
+  <<: *cache
+  script:
+    - rm .npmrc
+    - npm install --ignore-scripts --cache .npm
+    - npm audit
+  allow_failure: true
+
+build_and_test:
+  stage: test
+  <<: *cache
+  script:
+    - npm install --ignore-scripts --cache .npm
+    - npm run build-debug
+    - sed -i 's/Release/Debug/' lib/database.js
+    - sed -i 's/Release/Debug/' test/10.database.open.js
+    - npm run test

README.md

@@ -1,92 +1,16 @@
-# better-sqlite3 [![Build Status](https://github.com/JoshuaWise/better-sqlite3/actions/workflows/build.yml/badge.svg)](https://github.com/JoshuaWise/better-sqlite3/actions/workflows/build.yml?query=branch%3Amaster)
+# better-sqlcipher
 
-The fastest and simplest library for SQLite3 in Node.js.
+[better-sqlite3](https://github.com/JoshuaWise/better-sqlite3) built with
+[SQLCipher](https://www.zetetic.net/sqlcipher/).
 
-- Full transaction support
-- High performance, efficiency, and safety
-- Easy-to-use synchronous API *(better concurrency than an asynchronous API... yes, you read that correctly)*
-- Support for user-defined functions, aggregates, virtual tables, and extensions
-- 64-bit integers *(invisible until you need them)*
-- Worker thread support *(for large/slow queries)*
+This is the better-sqlite3 library patched to use SQLCipher instead.
 
-## Help this project stay strong! 💪
+## Changes
 
-`better-sqlite3` is used by thousands of developers and engineers on a daily basis. Long nights and weekends were spent keeping this project strong and dependable, with no ask for compensation or funding, until now. If your company uses `better-sqlite3`, ask your manager to consider supporting the project:
-
-- [Become a GitHub sponsor](https://github.com/sponsors/JoshuaWise)
-- [Become a backer on Patreon](https://www.patreon.com/joshuawise)
-- [Make a one-time donation on PayPal](https://www.paypal.me/joshuathomaswise)
-
-## How other libraries compare
-
-|   |select 1 row  `get()` |select 100 rows   `all()`  |select 100 rows `iterate()` 1-by-1|insert 1 row `run()`|insert 100 rows in a transaction|
-|---|---|---|---|---|---|
-|better-sqlite3|1x|1x|1x|1x|1x|
-|[sqlite](https://www.npmjs.com/package/sqlite) and [sqlite3](https://www.npmjs.com/package/sqlite3)|11.7x slower|2.9x slower|24.4x slower|2.8x slower|15.6x slower|
-
-> You can verify these results by [running the benchmark yourself](./docs/benchmark.md).
-
-## Installation
-
-```bash
-npm install better-sqlite3
-```
-
-> You must be using Node.js v14.21.1 or above. Prebuilt binaries are available for [LTS versions](https://nodejs.org/en/about/releases/). If you have trouble installing, check the [troubleshooting guide](./docs/troubleshooting.md).
-
-## Usage
-
-```js
-const db = require('better-sqlite3')('foobar.db', options);
-
-const row = db.prepare('SELECT * FROM users WHERE id = ?').get(userId);
-console.log(row.firstName, row.lastName, row.email);
-```
-
-Though not required, [it is generally important to set the WAL pragma for performance reasons](https://github.com/WiseLibs/better-sqlite3/blob/master/docs/performance.md).
-
-```js
-db.pragma('journal_mode = WAL');
-```
-
-##### In ES6 module notation:
-
-```js
-import Database from 'better-sqlite3';
-const db = new Database('foobar.db', options);
-db.pragma('journal_mode = WAL');
-```
-
-## Why should I use this instead of [node-sqlite3](https://github.com/mapbox/node-sqlite3)?
-
-- `node-sqlite3` uses asynchronous APIs for tasks that are either CPU-bound or serialized. That's not only bad design, but it wastes tons of resources. It also causes [mutex thrashing](https://en.wikipedia.org/wiki/Resource_contention) which has devastating effects on performance.
-- `node-sqlite3` exposes low-level (C language) memory management functions. `better-sqlite3` does it the JavaScript way, allowing the garbage collector to worry about memory management.
-- `better-sqlite3` is simpler to use, and it provides nice utilities for some operations that are very difficult or impossible in `node-sqlite3`.
-- `better-sqlite3` is much faster than `node-sqlite3` in most cases, and just as fast in all other cases.
-
-#### When is this library not appropriate?
-
-In most cases, if you're attempting something that cannot be reasonably accomplished with `better-sqlite3`, it probably cannot be reasonably accomplished with SQLite3 in general. For example, if you're executing queries that take one second to complete, and you expect to have many concurrent users executing those queries, no amount of asynchronicity will save you from SQLite3's serialized nature. Fortunately, SQLite3 is very *very* fast. With proper indexing, we've been able to achieve upward of 2000 queries per second with 5-way-joins in a 60 GB database, where each query was handling 5–50 kilobytes of real data.
-
-If you have a performance problem, the most likely causes are inefficient queries, improper indexing, or a lack of [WAL mode](./docs/performance.md)—not `better-sqlite3` itself. However, there are some cases where `better-sqlite3` could be inappropriate:
-
-- If you expect a high volume of concurrent reads each returning many megabytes of data (i.e., videos)
-- If you expect a high volume of concurrent writes (i.e., a social media site)
-- If your database's size is near the terabyte range
-
-For these situations, you should probably use a full-fledged RDBMS such as [PostgreSQL](https://www.postgresql.org/).
-
-# Documentation
-
-- [API documentation](./docs/api.md)
-- [Performance](./docs/performance.md) (also see [benchmark results](./docs/benchmark.md))
-- [64-bit integer support](./docs/integer.md)
-- [Worker thread support](./docs/threads.md)
-- [Unsafe mode (advanced)](./docs/unsafe.md)
-- [SQLite3 compilation (advanced)](./docs/compilation.md)
-- [Contribution rules](./docs/contribution.md)
-- [Code of conduct](./docs/conduct.md)
-
-# License
-
-[MIT](./LICENSE)
+- Use SQLCipher instead of SQLite
+- Disable some SQLite features that we don't use to reduce the attack surface
+- Remove `prebuild` dependency and always build locally
+- Remove `bindings` dependency
+- Remove `sqlite`, `sqlite3` and `nodemark` dependencies (this breaks benchmarks)
+- Add type declarations
+- Add tests for the encryption functionality

binding.gyp

@@ -1,13 +1,13 @@
 # ===
-# This is the main GYP file, which builds better-sqlite3 with SQLite3 itself.
+# This is the main GYP file, which builds better-sqlcipher.
 # ===
 
 {
   'includes': ['deps/common.gypi'],
   'targets': [
     {
-      'target_name': 'better_sqlite3',
-      'dependencies': ['deps/sqlite3.gyp:sqlite3'],
+      'target_name': 'better_sqlcipher',
+      'dependencies': ['deps/sqlcipher.gyp:sqlcipher'],
       'sources': ['src/better_sqlite3.cpp'],
       'cflags_cc': ['-std=c++17'],
       'xcode_settings': {
@@ -31,8 +31,8 @@
     },
     {
       'target_name': 'test_extension',
-      'dependencies': ['deps/sqlite3.gyp:sqlite3'],
-      'conditions': [['sqlite3 == ""', { 'sources': ['deps/test_extension.c'] }]],
+      'dependencies': ['deps/sqlcipher.gyp:sqlcipher'],
+      'sources': ['deps/test_extension.c'],
     },
   ],
 }

deps/common.gypi

@@ -4,7 +4,10 @@
 # ===
 
 {
-  'variables': { 'sqlite3%': '' },
+  'variables': {
+    'sqlcipher_version%': '4.5.7', # Maps to SQLite 3.45.3
+    'bearssl_version%': '0.6',
+  },
   'target_defaults': {
     'default_configuration': 'Release',
     'msvs_settings': {

deps/defines.gypi

@@ -10,29 +10,28 @@
     'HAVE_UINT32_T=1',
     'HAVE_UINT8_T=1',
     'HAVE_USLEEP=1',
+    'SQLCIPHER_CRYPTO_BEARSSL',
     'SQLITE_DEFAULT_CACHE_SIZE=-16000',
     'SQLITE_DEFAULT_FOREIGN_KEYS=1',
     'SQLITE_DEFAULT_MEMSTATUS=0',
     'SQLITE_DEFAULT_WAL_SYNCHRONOUS=1',
     'SQLITE_DQS=0',
     'SQLITE_ENABLE_COLUMN_METADATA',
     'SQLITE_ENABLE_DESERIALIZE',
-    'SQLITE_ENABLE_FTS3',
-    'SQLITE_ENABLE_FTS3_PARENTHESIS',
-    'SQLITE_ENABLE_FTS4',
     'SQLITE_ENABLE_FTS5',
-    'SQLITE_ENABLE_GEOPOLY',
     'SQLITE_ENABLE_JSON1',
     'SQLITE_ENABLE_MATH_FUNCTIONS',
-    'SQLITE_ENABLE_RTREE',
     'SQLITE_ENABLE_STAT4',
     'SQLITE_ENABLE_UPDATE_DELETE_LIMIT',
+    'SQLITE_HAS_CODEC',
     'SQLITE_LIKE_DOESNT_MATCH_BLOBS',
     'SQLITE_OMIT_DEPRECATED',
+    'SQLITE_OMIT_GET_TABLE',
     'SQLITE_OMIT_PROGRESS_CALLBACK',
     'SQLITE_OMIT_SHARED_CACHE',
     'SQLITE_OMIT_TCL_VARIABLE',
-    'SQLITE_SOUNDEX',
+    'SQLITE_SECURE_DELETE',
+    'SQLITE_TEMP_STORE=3',
     'SQLITE_THREADSAFE=2',
     'SQLITE_TRACE_SIZE_LIMIT=32',
     'SQLITE_USE_URI=0',

deps/download.sh

@@ -1,27 +1,12 @@
 #!/usr/bin/env bash
+set -euo pipefail
 
-# ===
-# This script defines and generates the bundled SQLite3 unit (sqlite3.c).
-#
-# The following steps are taken:
-# 1. populate the shell environment with the defined compile-time options.
-# 2. download and extract the SQLite3 source code into a temporary directory.
-# 3. run "sh configure" and "make sqlite3.c" within the source directory.
-# 4. copy the generated amalgamation into the output directory (./sqlite3).
-# 5. export the defined compile-time options to a gyp file (./defines.gypi).
-# 6. update the docs (../docs/compilation.md) with details of this distribution.
-#
-# When a user builds better-sqlite3, the following steps are taken:
-# 1. node-gyp loads the previously exported compile-time options (defines.gypi).
-# 2. the copy.js script copies the bundled amalgamation into the build folder.
-# 3. node-gyp compiles the copied sqlite3.c along with better_sqlite3.cpp.
-# 4. node-gyp links the two resulting binaries to generate better_sqlite3.node.
-# ===
+BEARSSL_VERSION=$(grep bearssl_version common.gypi | sed -E "s/.*: '([^']+)'.*/\1/")
+SQLCIPHER_VERSION=$(grep sqlcipher_version common.gypi | sed -E "s/.*: '([^']+)'.*/\1/")
 
-YEAR="2024"
-VERSION="3450300"
+BEARSSL_URL="https://www.bearssl.org/bearssl-${BEARSSL_VERSION}.tar.gz"
+SQLCIPHER_REPO="[email protected]:clients/web/sqlcipher.git"
 
-# Defines below are sorted alphabetically
 DEFINES="
 HAVE_INT16_T=1
 HAVE_INT32_T=1
@@ -31,82 +16,77 @@ HAVE_UINT16_T=1
 HAVE_UINT32_T=1
 HAVE_UINT8_T=1
 HAVE_USLEEP=1
+SQLCIPHER_CRYPTO_BEARSSL
 SQLITE_DEFAULT_CACHE_SIZE=-16000
 SQLITE_DEFAULT_FOREIGN_KEYS=1
 SQLITE_DEFAULT_MEMSTATUS=0
 SQLITE_DEFAULT_WAL_SYNCHRONOUS=1
 SQLITE_DQS=0
 SQLITE_ENABLE_COLUMN_METADATA
 SQLITE_ENABLE_DESERIALIZE
-SQLITE_ENABLE_FTS3
-SQLITE_ENABLE_FTS3_PARENTHESIS
-SQLITE_ENABLE_FTS4
 SQLITE_ENABLE_FTS5
-SQLITE_ENABLE_GEOPOLY
 SQLITE_ENABLE_JSON1
 SQLITE_ENABLE_MATH_FUNCTIONS
-SQLITE_ENABLE_RTREE
 SQLITE_ENABLE_STAT4
 SQLITE_ENABLE_UPDATE_DELETE_LIMIT
+SQLITE_HAS_CODEC
 SQLITE_LIKE_DOESNT_MATCH_BLOBS
 SQLITE_OMIT_DEPRECATED
+SQLITE_OMIT_GET_TABLE
 SQLITE_OMIT_PROGRESS_CALLBACK
 SQLITE_OMIT_SHARED_CACHE
 SQLITE_OMIT_TCL_VARIABLE
-SQLITE_SOUNDEX
+SQLITE_SECURE_DELETE
+SQLITE_TEMP_STORE=3
 SQLITE_THREADSAFE=2
 SQLITE_TRACE_SIZE_LIMIT=32
 SQLITE_USE_URI=0
 "
 
-# ========== START SCRIPT ========== #
+GREEN="\033[0;32m"
+RESET="\033[0m"
+function log() {
+    echo -en "$GREEN"
+    echo -n "$1"
+    echo -e "$RESET"
+}
 
-echo "setting up environment..."
-DEPS="$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)"
+log "Setting up environment"
+DEPS="$PWD"
 TEMP="$DEPS/temp"
-OUTPUT="$DEPS/sqlite3"
 rm -rf "$TEMP"
-rm -rf "$OUTPUT"
 mkdir -p "$TEMP"
-mkdir -p "$OUTPUT"
-export CFLAGS=`echo $(echo "$DEFINES" | sed -e "/^\s*$/d" -e "s/^/-D/")`
+CFLAGS=$(echo "$DEFINES" | sed -e "/^\s*$/d" -e "s/^/-D/")
+export CFLAGS
 
-echo "downloading source..."
-curl -#f "https://www.sqlite.org/$YEAR/sqlite-src-$VERSION.zip" > "$TEMP/source.zip" || exit 1
+log "Downloading BearSSL"
+curl "$BEARSSL_URL" > bearssl.tar.gz
 
-echo "extracting source..."
-unzip "$TEMP/source.zip" -d "$TEMP" > /dev/null || exit 1
-cd "$TEMP/sqlite-src-$VERSION" || exit 1
+log "Cloning SQLCipher..."
+pushd "$TEMP"
+git clone $SQLCIPHER_REPO -b "bearssl-crypto-provider-${SQLCIPHER_VERSION}"
 
-echo "configuring amalgamation..."
-sh configure > /dev/null || exit 1
+log "Configuring amalgamation..."
+mkdir -p sqlcipher/build "sqlcipher/build/sqlcipher-amalgamation-${SQLCIPHER_VERSION}"
+pushd ./sqlcipher/build
+../configure --with-crypto-lib=bearssl --enable-tempstore="yes"
 
-echo "building amalgamation..."
-make sqlite3.c > /dev/null || exit 1
+log "Building amalgamation..."
+make sqlite3.c
+cp sqlite3.c sqlite3.h sqlite3ext.h "sqlcipher-amalgamation-${SQLCIPHER_VERSION}/"
+cp ../VERSION "sqlcipher-amalgamation-${SQLCIPHER_VERSION}/VERSION.txt"
+tar -czvf sqlcipher.tar.gz "sqlcipher-amalgamation-${SQLCIPHER_VERSION}"
+popd
+popd
+mv "$TEMP/sqlcipher/build/sqlcipher.tar.gz" .
 
-echo "copying amalgamation..."
-cp sqlite3.c sqlite3.h sqlite3ext.h "$OUTPUT/" || exit 1
-
-echo "updating gyp configs..."
+log "Writing GYP defines..."
 GYP="$DEPS/defines.gypi"
 printf "# THIS FILE IS AUTOMATICALLY GENERATED BY deps/download.sh (DO NOT EDIT)\n\n{\n  'defines': [\n" > "$GYP"
 printf "$DEFINES" | sed -e "/^\s*$/d" -e "s/\(.*\)/    '\1',/" >> "$GYP"
 printf "  ],\n}\n" >> "$GYP"
 
-echo "updating docs..."
-DOCS="$DEPS/../docs/compilation.md"
-MAJOR=`expr "${VERSION:0:1}" + 0`
-MINOR=`expr "${VERSION:1:2}" + 0`
-PATCH=`expr "${VERSION:3:2}" + 0`
-sed -Ei.bak -e "s/version [0-9]+\.[0-9]+\.[0-9]+/version $MAJOR.$MINOR.$PATCH/g" "$DOCS"
-sed -i.bak -e "/^SQLITE_/,\$d" "$DOCS"
-sed -i.bak -e "/^HAVE_/,\$d" "$DOCS"
-rm "$DOCS".bak
-printf "$DEFINES" | sed -e "/^\s*$/d" >> "$DOCS"
-printf "\`\`\`\n" >> "$DOCS"
-
-echo "cleaning up..."
-cd - > /dev/null || exit 1
+log "Cleaning up..."
 rm -rf "$TEMP"
 
-echo "done!"
+log "Done!"

deps/extract.js

@@ -0,0 +1,16 @@
+const tar = require("tar");
+
+const source = process.argv[2];
+const destination = process.argv[3];
+
+process.on("unhandledRejection", (err) => {
+  throw err;
+});
+
+/*
+ * This extracts the <$2> tar file and places the resulting files into the
+ * directory specified by <$3>.
+ */
+tar
+  .extract({ file: source, cwd: destination, onwarn: process.emitWarning })
+  .then(() => process.exit(0));