Skip to content

Instantly share code, notes, and snippets.

View kriszyp's full-sized avatar
💭
Working on @HarperDB

Kris Zyp kriszyp

💭
Working on @HarperDB
496 followers · 0 following
View GitHub Profile
@kriszyp
kriszyp / string-functions.csv
Last active January 2, 2021 15:16
String functions
Function Characters Time
Buffer.toString 3 180ns
Buffer.toString 10 200ns
Buffer.toString 2000 500ns
Buffer.toString('latin1') 2000 415ns
TextDecoder.decode 10 1700ns
String.fromCharCode(...) 3 16ns
String.fromCharCode(...) 10 46ns
String.fromCharCode(...) 2000 4300ns
String.fromCharCode.apply(array) 3 29ns
@kriszyp
kriszyp / serialization-performance.csv
Last active January 10, 2021 01:21
operation op ms op/s
buf = Buffer(JSON.stringify(obj)); 82000 5004 16386
obj = JSON.parse(buf); 88600 5000 17720
require('msgpackr').pack(obj); 161500 5002 32287
require('msgpackr').unpack(buf); 94600 5004 18904
msgpackr w/ shared structures: packr.pack(obj); 178400 5002 35665
msgpackr w/ shared structures: packr.unpack(buf); 376700 5000 75340
buf = require('msgpack-lite').encode(obj); 30100 5012 6005
obj = require('msgpack-lite').decode(buf); 16200 5001 3239
buf = require('notepack').encode(obj); 62600 5005 12507
@kriszyp
kriszyp / serialization-performance-small.csv
Last active January 10, 2021 03:51
operation op/s
buf = Buffer(JSON.stringify(obj)); 526300
obj = JSON.parse(buf); 527300
require('msgpackr').pack(obj); 1081280
require('msgpackr').unpack(buf); 675080
msgpackr w/ shared structures: packr.pack(obj); 1284720
msgpackr w/ shared structures: packr.unpack(buf); 2770860
@kriszyp
kriszyp / CBOR comparisons.md
Last active November 14, 2024 08:55
This is a comparison of different CBOR encoding techniques including packing and the proposed record structure tag (and combining both) with cbor-x JS library, and how affect size and performance.

The cbor-x's packed implementation only packs whole strings that occur multiple times, it does not search for repeated prefixes or postfixes, as they would almost certainly be vastly more expensive. Strings are packed if they occur multiples in a data structures. When using packed + record tags, strings as keys are not searched for string repetition (since it assumed repetition will mostly be eliminated by the structure reuse).

The table shows encoded size for each technique, and the encoding and decoding performance. The last column also includes the gzipped size for comparison sake (no gzip performance, but generally is about 2-4x slower with gzipping in my tests). The table compares plain CBOR encoding, packed, record structures with a 1+1 definition tag and 1+2 tag, and the combination of packed and record structures.

The first comparison test uses an 8KB JSON data structure from our database of medical studies, that has a fairly complicated and dynamic structure: https://github.com/kriszyp/cbor-x/blob/