CBOR and MessagePack initial sync profiling

gistfile1.txt

random interesting factoid:

lazyloading reduces most of my accounts initial sync by about 4-5x
i was wondering about ways to shrink further (short of paginated sync)
and so tried expressing it as CBOR
doing something like:

perl -MJSON::XS -MCBOR::XS -MFile::Slurp -e '$z=read_file(\*STDIN); print encode_cbor(decode_json($z));'

turns out for @matthew2's initial sync, this reduces 4.5MB of JSON to 3.7MB of CBOR.
however, the interesting bit is that when you gzip it, that turns into 1.2MB of gzipped JSON v. 1.3MB of gzipped CBOR.
i suspect this could be fixed with saner interning however.

still, kinda intersting
messagepack weighs in at 3.9MB raw, 1.3MB zipped, fwiw
via

cat sync-ll.json | perl -MJSON::PP -MData::MessagePack -MFile::Slurp -e '$JSON::PP::true=1; $JSON::PP::false=0; $z=read_file(\*STDIN); print Data::MessagePack->new()->pack(decode_json($z));' > sync-ll.mp

one observation is that the 64-bit ints in JSON (e.g. all those timestamps) get turned into strings by perl in those oneliners