You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I didn't know this package existed, and wrote my own, doh. But mine is scoped too heavily for my project. I want to switch to something a bit more flexible that has community support, and would love to get some of the reductions I implemented. In msgpackr for strings, I've been unable to get any boost from bundleStrings, which was odd.
Enhancements:
Allow custom string dictionary. An array of commonly used strings that is fed identically to both Packr and Unpackr. It should only take up two bytes per string to lookup against this table for dictionary length of 255.
Store location of repeated strings instead of encoding strings twice. If "hello" gets encoded at byte position 53, and the serializer sees "hello" again later, it should just encode the location position 53 for that 2nd "hello". Again, taking only 2 bytes or more if distance is greater than 255.
Feel free to see my own awful implementation, acos-json-encoder.
Edit: link goes to line where I implemented
The text was updated successfully, but these errors were encountered:
You might consider using CBOR packing, which was designed for this purpose: https://github.com/kriszyp/cbor-x?tab=readme-ov-file#cbor-packing
However, this will only find exact string value duplicates (no duplicates within string, it won't do any compression of {foo: 'hello', bar: 'hello world'}. For more general string deduping, that is kind of the whole point RLE compression, and there are plenty of great compression formats and tools which are much better than anything msgpack could offer.
I didn't know this package existed, and wrote my own, doh. But mine is scoped too heavily for my project. I want to switch to something a bit more flexible that has community support, and would love to get some of the reductions I implemented. In msgpackr for strings, I've been unable to get any boost from
bundleStrings
, which was odd.Enhancements:
Allow custom string dictionary. An array of commonly used strings that is fed identically to both
Packr
andUnpackr
. It should only take up two bytes per string to lookup against this table for dictionary length of 255.Store location of repeated strings instead of encoding strings twice. If "hello" gets encoded at byte position
53
, and the serializer sees "hello" again later, it should just encode the location position53
for that 2nd "hello". Again, taking only 2 bytes or more if distance is greater than 255.Feel free to see my own awful implementation, acos-json-encoder.
Edit: link goes to line where I implemented
The text was updated successfully, but these errors were encountered: