Skip to content

Instantly share code, notes, and snippets.

@csabapalfi

csabapalfi/compression.md

Created September 6, 2018 15:53
Show Gist options
  • Save csabapalfi/9bfc375fc01f518a1d35bfae7085c633 to your computer and use it in GitHub Desktop.
Save csabapalfi/9bfc375fc01f518a1d35bfae7085c633 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
compression.md

gzip vs streaming HTML

curl -v -H 'Accept-Encoding: gzip' --limit-rate 2K 'https://www.homeaway.com/d/40/austin' | gzip --decompress

No, you don't need the complete response to decompress it. deflate compressed data (the format used in gzip streams) is a sequence of coded literal bytes and length/distance pairs (matches to previously uncompressed data), where those codes start producing uncompressed data immediately upon receipt. dynamic deflate blocks have about a 60 to 80 byte header describing the codes, so you only have to wait for the gzip header (10 bytes) and the first dynamic header to be able to start generating uncompressed data. Fixed-code deflate blocks use a pre-defined set of codes, and so you start getting data from those in the first two bytes.

HTTP streaming

https://gist.github.com/CMCDragonkai/6bfade6431e9ffb7fe88

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment