You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The following sample code throws an MDB_KEYEXIST: Key/data pair already exists error when performing a put operation using the {append:true} option with a string containing character codes larger than 255, even though the key does not already exist:
I assume the reason is that node-lmdb internally uses a little-endian instead of a big-endian encoding for UTF16 strings. As soon as the character code hits 256 the first of the two bytes gets reset to 0, so for the lmdb the key is out of order as it already has keys whose first byte is up to 255. My assumption is based on the fact that I get the same error if I use a utf16le-encoded buffer as key:
...yet it works without error if the key is utf16be-encoded (since Node.js does not provide that encoding itself, I use swap16 on the utf16le-encoded buffer):
PS: The sample code has been tested with Node.js v12.19 and node-lmdb v0.9.4.
PPS: I know this is a niche issue. I don't expect it to get fixed, I just wanted to report so others can find the error code in a websearch and see the workaround.
The text was updated successfully, but these errors were encountered:
Your code will use the append mode only if val==someString, in which case the database already contains a value for the key 1. AFAIK, appending only works if the key is larger than any other existing key in the given database. It should even fail if the key you are overwriting is the largest one.
Your code will use the append mode only if val==someString, in which case the database already contains a value for the key 1. AFAIK, appending only works if the key is larger than any other existing key in the given database. It should even fail if the key you are overwriting is the largest one.
I completely misunderstood what append is for. I figured out what I actually wanted to know, which was either extending the existing entry (at which point I would just input the existing entry plus the new one together as the new input), or multiple entries on the same key (solved by modifying the db on creation with dupSort: true).
I appreciate the response and apologize for cluttering up the thread.
The following sample code throws an
MDB_KEYEXIST: Key/data pair already exists
error when performing aput
operation using the{append:true}
option with a string containing character codes larger than 255, even though the key does not already exist:I assume the reason is that node-lmdb internally uses a little-endian instead of a big-endian encoding for UTF16 strings. As soon as the character code hits 256 the first of the two bytes gets reset to 0, so for the lmdb the key is out of order as it already has keys whose first byte is up to 255. My assumption is based on the fact that I get the same error if I use a
utf16le
-encoded buffer as key:...yet it works without error if the key is
utf16be
-encoded (since Node.js does not provide that encoding itself, I useswap16
on theutf16le
-encoded buffer):PS: The sample code has been tested with Node.js v12.19 and node-lmdb v0.9.4.
PPS: I know this is a niche issue. I don't expect it to get fixed, I just wanted to report so others can find the error code in a websearch and see the workaround.
The text was updated successfully, but these errors were encountered: