msgpackr may silently replace 64 byte+ strings with empty string after encoding 2GiB messages

[akumansley]

Hi!

We believe we've run into an issue where the msgpackr module can become "poisoned" after encoding a message longer than 2GiB.

Here's a test case that reproduces the issue on OS X Node.js v18.16.0. We've also encountered the issue on Linux Node.js v18.16.0.

const msgpackr = require('msgpackr');

const obj = {};

const largeString = 'a'.repeat(0x100);

const size = 2 ** 30; // 1GiB
const numberOfStringsRequiredFor1GB = Math.ceil(size / largeString.length);

for (let i = 0; i < numberOfStringsRequiredFor1GB; i++) {
    const key = i.toString();
    obj[key] = largeString;
}

// create a too-large target buffer
const packr = new msgpackr.Packr();

// incidentally, this value will be encoded correctly, because Buffer.utf8Write
// will only fail to write towards the beginning of the buffer
const _ = packr.pack([obj, obj]);

// now the packer module is poisoned
const newPackr = new msgpackr.Packr();

// must be at least a 64ch string to trigger the bug
const modestString = 'a'.repeat(64);
const simpleObj = {name: modestString};
const incorrectlyEncoded = newPackr.pack(simpleObj);
const decodedIncorrectlyEncoded = newPackr.unpack(incorrectlyEncoded);

if (decodedIncorrectlyEncoded.name !== modestString) {
    throw new Error("expected 64 'a's, but found ", decodedIncorrectlyEncoded.name);
}

As a workaround, we're calling packr.useBuffer(Buffer.allocUnsafeSlow(8192)) to force the packr to reset its internal target buffer.

[adembo]

We filed a bug report with NodeJS for what we believe to be the root cause.