Skip to content

Instantly share code, notes, and snippets.

@fiddyspence

fiddyspence/gist:379a9c82cce23dc2c71d

Last active August 29, 2015 14:18
Show Gist options
  • Save fiddyspence/379a9c82cce23dc2c71d to your computer and use it in GitHub Desktop.
Save fiddyspence/379a9c82cce23dc2c71d to your computer and use it in GitHub Desktop.
Download ZIP
Forge gzip lies
Raw
gistfile1.txt
spenceware:~ fids$ curl -o foo https://forge.puppetlabs.com/api/v1/releases.json?module=puppetlabs/nodejs
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 7175 100 7175 0 0 11957 0 --:--:-- --:--:-- --:--:-- 11958
spenceware:~ fids$ file foo
foo: ASCII text, with very long lines, with no line terminators
spenceware:~ fids$ curl -o foo https://forge.puppetlabs.com/api/v1/releases.json?module=puppetlabs/nodejs
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 7175 100 7175 0 0 11523 0 --:--:-- --:--:-- --:--:-- 11535
spenceware:~ fids$ file foo
foo: ASCII text, with very long lines, with no line terminators
spenceware:~ fids$ curl -o foo https://forge.puppetlabs.com/api/v1/releases.json?module=puppetlabs/nodejs
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 601 0 601 0 0 988 0 --:--:-- --:--:-- --:--:-- 986
spenceware:~ fids$ file foo
foo: gzip compressed data, from Unix, last modified: Fri Apr 10 09:12:39 2015
spenceware:~ fids$ mv foo foo.gz
spenceware:~ fids$ gunzip foo.gz
spenceware:~ fids$ cat foo
{"puppetlabs/nodejs":[{"file":"/system/releases/p/puppetlabs/puppetlabs-nodejs-0.1.0.tar.gz","version":"0.1.0","dependencies":[["puppetlabs/apt",">= 0.0.2"]]},{"file":"/system/releases/p/puppetlabs/puppetlabs-nodejs-0.1.1.tar.gz","version":"0.1.1","dependencies":[["puppetlabs/apt",">= 0.0.3"]]},{"file":"/system/releases/p/puppetlabs/puppetlabs-nodejs-0.2.0.tar.gz","version":"0.2.0","dependencies":[["puppetlabs/apt",">= 0.0.3"],["puppetlabs/stdlib",">= 2.0.0"]]},{"file":"/system/releases/p/puppetlabs/puppetlabs-nodejs-0.2.1.tar.gz","version":"0.2.1","dependencies":[["puppetlabs/apt",">= 0.0.3"],["puppetlabs/stdlib",">= 2.0.0"]]},{"file":"/system/releases/p/puppetlabs/puppetlabs-nodejs-0.3.0.tar.gz","version":"0.3.0","dependencies":[["puppetlabs/apt",">= 0.0.3"],["puppetlabs/stdlib",">= 2.0.0"]]},{"file":"/system/releases/p/puppetlabs/puppetlabs-nodejs-0.4.0.tar.gz","version":"0.4.0","dependencies":[["puppetlabs/apt",">= 0.0.3"],["puppetlabs/stdlib",">= 2.0.0"]]},{"file":"/system/releases/p/puppetlabs/puppetlabs-nodejs-0.6.0.tar.gz","version":"0.6.0","dependencies":[["puppetlabs/apt",">=0.0.3"],["puppetlabs/stdlib",">=2.0.0"]]},{"file":"/system/releases/p/puppetlabs/puppetlabs-nodejs-0.6.1.tar.gz","version":"0.6.1","dependencies":[["puppetlabs/apt",">=0.0.3"],["puppetlabs/stdlib",">=2.0.0"]]},{"file":"/system/releases/p/puppetlabs/puppetlabs-nodejs-0.7.0.tar.gz","version":"0.7.0","dependencies":[["puppetlabs/apt",">=0.0.3"],["puppetlabs/stdlib",">=2.0.0"]]},{"file":"/system/releases/p/puppetlabs/puppetlabs-nodejs-0.7.1.tar.gz","version":"0.7.1","dependencies":[["puppetlabs/apt",">=0.0.3"],["puppetlabs/stdlib",">=2.0.0"]]}],"puppetlabs/stdlib":[{"file":"/system/releases/p/puppetlabs/puppetlabs-stdlib-2.0.0.tar.gz","version":"2.0.0","dependencies":[]},{"file":"/system/releases/p/puppetlabs/puppetlabs-stdlib-2.1.3.tar.gz","version":"2.1.3","dependencies":[]},{"file":"/system/releases/p/puppetlabs/puppetlabs-stdlib-2.2.0.tar.gz","version":"2.2.0","dependencies":[]},{"file":"/system/releases/p/puppetlabs/puppetlabs-stdlib-2.2.1.tar.gz","version":"2.2.1","dependencies":[]},{"file":"/system/releases/p/puppetlabs/puppetlabs-stdlib-2.3.0.tar.gz","version":"2.3.0","dependencies":[]},{"file":"/system/releases/p/puppetlabs/puppetlabs-stdlib-2.3.1.tar.gz","version":"2.3.1","dependencies":[]},{"file":"/system/releases/p/puppetlabs/puppetlabs-stdlib-2.3.2.tar.gz","version":"2.3.2","dependencies":[]},{"file":"/system/releases/p/puppetlabs/puppetlabs-stdlib-2.3.3.tar.gz","version":"2.3.3","dependencies":[]},{"file":"/system/releases/p/puppetlabs/puppetlabs-stdlib-2.4.0.tar.gz","version":"2.4.0","dependencies":[]},{"file":"/system/releases/p/puppetlabs/puppetlabs-stdlib-2.5.0.tar.gz","version":"2.5.0","dependencies":[]},{"file":"/system/releases/p/puppetlabs/puppetlabs-stdlib-2.5.1.tar.gz","version":"2.5.1","dependencies":[]},{"file":"/system/releases/p/puppetlabs/puppetlabs-stdlib-2.6.0.tar.gz","version":"2.6.0","dependencies":[]},{"file":"/system/releases/p/puppetlabs/puppetlabs-stdlib-3.0.0.tar.gz","version":"3.0.0","dependencies":[]},{"file":"/system/releases/p/puppetlabs/puppetlabs-stdlib-3.0.1.tar.gz","version":"3.0.1","dependencies":[]},{"file":"/system/releases/p/puppetlabs/puppetlabs-stdlib-3.1.0.tar.gz","version":"3.1.0","dependencies":[]},{"file":"/system/releases/p/puppetlabs/puppetlabs-stdlib-3.1.1.tar.gz","version":"3.1.1","dependencies":[]},{"file":"/system/releases/p/puppetlabs/puppetlabs-stdlib-3.2.0.tar.gz","version":"3.2.0","dependencies":[]},{"file":"/system/releases/p/puppetlabs/puppetlabs-stdlib-3.2.1.tar.gz","version":"3.2.1","dependencies":[]},{"file":"/system/releases/p/puppetlabs/puppetlabs-stdlib-3.2.2.tar.gz","version":"3.2.2","dependencies":[]},{"file":"/system/releases/p/puppetlabs/puppetlabs-stdlib-4.1.0.tar.gz","version":"4.1.0","dependencies":[]},{"file":"/system/releases/p/puppetlabs/puppetlabs-stdlib-4.2.0.tar.gz","version":"4.2.0","dependencies":[]},{"file":"/system/releases/p/puppetlabs/puppetlabs-stdlib-4.2.1.tar.gz","version":"4.2.1","dependencies":[]},{"file":"/system/releases/p/puppetlabs/puppetlabs-stdlib-4.2.2.tar.gz","version":"4.2.2","dependencies":[]},{"file":"/system/releases/p/puppetlabs/puppetlabs-stdlib-4.3.0.tar.gz","version":"4.3.0","dependencies":[]},{"file":"/system/releases/p/puppetlabs/puppetlabs-stdlib-4.3.2.tar.gz","version":"4.3.2","dependencies":[]},{"file":"/system/releases/p/puppetlabs/puppetlabs-stdlib-4.4.0.tar.gz","version":"4.4.0","dependencies":[]},{"file":"/system/releases/p/puppetlabs/puppetlabs-stdlib-4.5.0.tar.gz","version":"4.5.0","dependencies":[]},{"file":"/system/releases/p/puppetlabs/puppetlabs-stdlib-4.5.1.tar.gz","version":"4.5.1","dependencies":[]}],"puppetlabs/apt":[{"file":"/system/releases/p/puppetlabs/puppetlabs-apt-0.0.2.tar.gz","version":"0.0.2","dependencies":[["puppetlabs/stdlib",">= 2.2.1"]]},{"file":"/system/releases/p/puppetlabs/puppetlabs-apt-0.0.3.tar.gz","version":"0.0.3","dependencies":[["puppetlabs/stdlib",">= 2.2.1"]]},{"file":"/system/releases/p/puppetlabs/puppetlabs-apt-0.0.4.tar.gz","version":"0.0.4","dependencies":[["puppetlabs/stdlib",">= 2.2.1"]]},{"file":"/system/releases/p/puppetlabs/puppetlabs-apt-1.0.0.tar.gz","version":"1.0.0","dependencies":[["puppetlabs/stdlib","2.x"]]},{"file":"/system/releases/p/puppetlabs/puppetlabs-apt-1.0.1.tar.gz","version":"1.0.1","dependencies":[["puppetlabs/stdlib",">= 2.2.1"]]},{"file":"/system/releases/p/puppetlabs/puppetlabs-apt-1.1.0.tar.gz","version":"1.1.0","dependencies":[["puppetlabs/stdlib",">= 2.2.1"]]},{"file":"/system/releases/p/puppetlabs/puppetlabs-apt-1.1.1.tar.gz","version":"1.1.1","dependencies":[["puppetlabs/stdlib",">= 2.2.1"]]},{"file":"/system/releases/p/puppetlabs/puppetlabs-apt-1.2.0.tar.gz","version":"1.2.0","dependencies":[["puppetlabs/stdlib",">= 2.2.1"]]},{"file":"/system/releases/p/puppetlabs/puppetlabs-apt-1.3.0.tar.gz","version":"1.3.0","dependencies":[["puppetlabs/stdlib",">= 2.2.1"]]},{"file":"/system/releases/p/puppetlabs/puppetlabs-apt-1.4.0.tar.gz","version":"1.4.0","dependencies":[["puppetlabs/stdlib",">= 2.2.1"]]},{"file":"/system/releases/p/puppetlabs/puppetlabs-apt-1.4.1.tar.gz","version":"1.4.1","dependencies":[["puppetlabs/stdlib",">= 2.2.1"]]},{"file":"/system/releases/p/puppetlabs/puppetlabs-apt-1.4.2.tar.gz","version":"1.4.2","dependencies":[["puppetlabs/stdlib",">= 2.2.1"]]},{"file":"/system/releases/p/puppetlabs/puppetlabs-apt-1.5.0.tar.gz","version":"1.5.0","dependencies":[["puppetlabs/stdlib",">= 2.2.1"]]},{"file":"/system/releases/p/puppetlabs/puppetlabs-apt-1.5.1.tar.gz","version":"1.5.1","dependencies":[["puppetlabs/stdlib",">= 2.2.1"]]},{"file":"/system/releases/p/puppetlabs/puppetlabs-apt-1.5.2.tar.gz","version":"1.5.2","dependencies":[["puppetlabs/stdlib",">= 2.2.1"]]},{"file":"/system/releases/p/puppetlabs/puppetlabs-apt-1.6.0.tar.gz","version":"1.6.0","dependencies":[["puppetlabs/stdlib",">= 2.2.1"]]},{"file":"/system/releases/p/puppetlabs/puppetlabs-apt-1.7.0.tar.gz","version":"1.7.0","dependencies":[["puppetlabs/stdlib",">= 2.2.1"]]},{"file":"/system/releases/p/puppetlabs/puppetlabs-apt-1.8.0.tar.gz","version":"1.8.0","dependencies":[["puppetlabs/stdlib",">= 2.2.1"]]}]}spenceware:~ fids$
irb(main):022:0> open("https://forge.puppetlabs.com/api/v1/releases.json?module=puppetlabs/nodejs") {|f| f.meta }
=> {"server"=>"nginx", "date"=>"Fri, 10 Apr 2015 08:33:02 GMT", "content-type"=>"application/json;charset=utf-8", "transfer-encoding"=>"chunked", "connection"=>"keep-alive", "status"=>"200 OK", "cache-control"=>"private, must-revalidate, max-age=0", "x-node"=>"forge-app01-prod", "x-revision"=>"923bd24", "x-content-type-options"=>"nosniff", "vary"=>"Accept-Encoding", "content-encoding"=>"gzip"}
irb(main):023:0> open("https://forge.puppetlabs.com/api/v1/releases.json?module=puppetlabs/nodejs") {|f| f.meta }
=> {"server"=>"nginx", "date"=>"Fri, 10 Apr 2015 08:33:03 GMT", "content-type"=>"application/json;charset=utf-8", "content-length"=>"7175", "connection"=>"keep-alive", "status"=>"200 OK", "cache-control"=>"private, must-revalidate, max-age=0", "x-node"=>"forge-app03-prod", "x-revision"=>"923bd24", "x-content-type-options"=>"nosniff", "vary"=>"Accept-Encoding"}
irb(main):024:0> open("https://forge.puppetlabs.com/api/v1/releases.json?module=puppetlabs/nodejs") {|f| f.meta }
=> {"server"=>"nginx", "date"=>"Fri, 10 Apr 2015 08:33:04 GMT", "content-type"=>"application/json;charset=utf-8", "transfer-encoding"=>"chunked", "connection"=>"keep-alive", "status"=>"200 OK", "cache-control"=>"private, must-revalidate, max-age=0", "x-node"=>"forge-app02-prod", "x-revision"=>"923bd24", "x-content-type-options"=>"nosniff", "vary"=>"Accept-Encoding", "content-encoding"=>"gzip"}
irb(main):025:0> open("https://forge.puppetlabs.com/api/v1/releases.json?module=puppetlabs/nodejs") {|f| f.meta }
=> {"server"=>"nginx", "date"=>"Fri, 10 Apr 2015 08:33:08 GMT", "content-type"=>"application/json;charset=utf-8", "transfer-encoding"=>"chunked", "connection"=>"keep-alive", "status"=>"200 OK", "cache-control"=>"private, must-revalidate, max-age=0", "x-node"=>"forge-app04-prod", "x-revision"=>"923bd24", "x-content-type-options"=>"nosniff", "vary"=>"Accept-Encoding", "content-encoding"=>"gzip"}
irb(main):026:0> open("https://forge.puppetlabs.com/api/v1/releases.json?module=puppetlabs/nodejs") {|f| f.meta }
=> {"server"=>"nginx", "date"=>"Fri, 10 Apr 2015 08:33:09 GMT", "content-type"=>"application/json;charset=utf-8", "transfer-encoding"=>"chunked", "connection"=>"keep-alive", "status"=>"200 OK", "cache-control"=>"private, must-revalidate, max-age=0", "x-node"=>"forge-app02-prod", "x-revision"=>"923bd24", "x-content-type-options"=>"nosniff", "vary"=>"Accept-Encoding", "content-encoding"=>"gzip"}
irb(main):027:0> open("https://forge.puppetlabs.com/api/v1/releases.json?module=puppetlabs/nodejs") {|f| f.meta }
=> {"server"=>"nginx", "date"=>"Fri, 10 Apr 2015 08:33:10 GMT", "content-type"=>"application/json;charset=utf-8", "transfer-encoding"=>"chunked", "connection"=>"keep-alive", "status"=>"200 OK", "cache-control"=>"private, must-revalidate, max-age=0", "x-node"=>"forge-app04-prod", "x-revision"=>"923bd24", "x-content-type-options"=>"nosniff", "vary"=>"Accept-Encoding", "content-encoding"=>"gzip"}
irb(main):028:0> open("https://forge.puppetlabs.com/api/v1/releases.json?module=puppetlabs/nodejs") {|f| f.meta }
=> {"server"=>"nginx", "date"=>"Fri, 10 Apr 2015 08:33:12 GMT", "content-type"=>"application/json;charset=utf-8", "transfer-encoding"=>"chunked", "connection"=>"keep-alive", "status"=>"200 OK", "cache-control"=>"private, must-revalidate, max-age=0", "x-node"=>"forge-app04-prod", "x-revision"=>"923bd24", "x-content-type-options"=>"nosniff", "vary"=>"Accept-Encoding", "content-encoding"=>"gzip"}
irb(main):029:0> open("https://forge.puppetlabs.com/api/v1/releases.json?module=puppetlabs/nodejs") {|f| f.meta }
=> {"server"=>"nginx", "date"=>"Fri, 10 Apr 2015 08:33:13 GMT", "content-type"=>"application/json;charset=utf-8", "transfer-encoding"=>"chunked", "connection"=>"keep-alive", "status"=>"200 OK", "cache-control"=>"private, must-revalidate, max-age=0", "x-node"=>"forge-app04-prod", "x-revision"=>"923bd24", "x-content-type-options"=>"nosniff", "vary"=>"Accept-Encoding", "content-encoding"=>"gzip"}
And then a broken response
irb(main):030:0> open("https://forge.puppetlabs.com/api/v1/releases.json?module=puppetlabs/nodejs") {|f| f.meta }
=> {"server"=>"nginx", "date"=>"Fri, 10 Apr 2015 08:33:14 GMT", "content-type"=>"application/json;charset=utf-8", "content-length"=>"7175", "connection"=>"keep-alive", "status"=>"200 OK", "cache-control"=>"private, must-revalidate, max-age=0", "x-node"=>"forge-app03-prod", "x-revision"=>"923bd24", "x-content-type-options"=>"nosniff", "vary"=>"Accept-Encoding"}
irb(main):031:0>
spenceware:~ fids$ curl -o foo https://forge.puppetlabs.com/api/v1/releases.json?module=puppetlabs/nodejs
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 601 0 601 0 0 1014 0 --:--:-- --:--:-- --:--:-- 1013
spenceware:~ fids$ file foo
foo: gzip compressed data, from Unix, last modified: Fri Apr 10 14:50:51 2015
@scotje
Copy link

scotje commented Apr 10, 2015

Without Accept-Encoding:

$ for i in {1..20}; do curl -s -o output "https://forge.puppetlabs.com/api/v1/releases.json?module=rcole
man/motd"; file output; done
output: ASCII text, with very long lines, with no line terminators
output: ASCII text, with very long lines, with no line terminators
output: ASCII text, with very long lines, with no line terminators
output: ASCII text, with very long lines, with no line terminators
output: ASCII text, with very long lines, with no line terminators
output: ASCII text, with very long lines, with no line terminators
output: ASCII text, with very long lines, with no line terminators
output: ASCII text, with very long lines, with no line terminators
output: ASCII text, with very long lines, with no line terminators
output: ASCII text, with very long lines, with no line terminators
output: ASCII text, with very long lines, with no line terminators
output: ASCII text, with very long lines, with no line terminators
output: ASCII text, with very long lines, with no line terminators
output: ASCII text, with very long lines, with no line terminators
output: ASCII text, with very long lines, with no line terminators
output: ASCII text, with very long lines, with no line terminators
output: ASCII text, with very long lines, with no line terminators
output: ASCII text, with very long lines, with no line terminators
output: ASCII text, with very long lines, with no line terminators
output: ASCII text, with very long lines, with no line terminators

With Accept-Encoding:

$ for i in {1..20}; do curl -s -H "Accept-Encoding: gzip,deflate" -o output "https://forge.puppetlabs.com/a
pi/v1/releases.json?module=rcoleman/motd"; file output; done
output: gzip compressed data, from Unix, last modified: Fri Apr 10 11:24:49 2015
output: gzip compressed data, from Unix, last modified: Fri Apr 10 11:24:48 2015
output: gzip compressed data, from Unix, last modified: Fri Apr 10 11:24:48 2015
output: gzip compressed data, from Unix, last modified: Fri Apr 10 11:24:49 2015
output: gzip compressed data, from Unix, last modified: Fri Apr 10 11:24:49 2015
output: gzip compressed data, from Unix, last modified: Fri Apr 10 11:24:49 2015
output: gzip compressed data, from Unix, last modified: Fri Apr 10 11:24:48 2015
output: gzip compressed data, from Unix, last modified: Fri Apr 10 11:24:49 2015
output: gzip compressed data, from Unix, last modified: Fri Apr 10 11:24:48 2015
output: gzip compressed data, from Unix, last modified: Fri Apr 10 11:24:48 2015
output: gzip compressed data, from Unix, last modified: Fri Apr 10 11:24:48 2015
output: gzip compressed data, from Unix, last modified: Fri Apr 10 11:24:49 2015
output: gzip compressed data, from Unix, last modified: Fri Apr 10 11:24:48 2015
output: gzip compressed data, from Unix, last modified: Fri Apr 10 11:24:49 2015
output: gzip compressed data, from Unix, last modified: Fri Apr 10 11:24:48 2015
output: gzip compressed data, from Unix, last modified: Fri Apr 10 11:24:49 2015
output: gzip compressed data, from Unix, last modified: Fri Apr 10 11:24:49 2015
output: gzip compressed data, from Unix, last modified: Fri Apr 10 11:24:49 2015
output: gzip compressed data, from Unix, last modified: Fri Apr 10 11:24:49 2015
output: gzip compressed data, from Unix, last modified: Fri Apr 10 11:24:48 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment