Skip to content

Instantly share code, notes, and snippets.

@geopet

geopet/1_9_3_result.md

Last active December 18, 2015 12:29
Show Gist options
  • Save geopet/5782836 to your computer and use it in GitHub Desktop.
Save geopet/5782836 to your computer and use it in GitHub Desktop.
Download ZIP
Weather Underground API Debugging
Raw
1_9_3_result.md

Ruby MRI 1.9.3 Results

[gpetrie] $ ruby -v
ruby 1.9.3p429 (2013-05-15 revision 40747) [x86_64-darwin11.4.2]

[gpetrie] $ ruby wunderground.rb
Current temperature in Cedar Rapids is: 77.0
Raw
2_0_0_result.md

Ruby MRI 2.0.0 Results

This is a direct paste from a clean Vagrant Precise32 VM build using RVM as the Ruby version manager.

vagrant@precise32:~$ ruby -v
ruby 2.0.0p195 (2013-05-14 revision 40734) [i686-linux]
vagrant@precise32:~$ ruby /vagrant/wunderground.rb
/home/vagrant/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/net/http/response.rb:357:in `finish': incorrect header check (Zlib::DataError)
  from /home/vagrant/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/net/http/response.rb:357:in `finish'
	from /home/vagrant/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/net/http/response.rb:262:in `ensure in inflater'
	from /home/vagrant/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/net/http/response.rb:262:in `inflater'
	from /home/vagrant/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/net/http/response.rb:274:in `read_body_0'
	from /home/vagrant/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/net/http/response.rb:201:in `read_body'
	from /home/vagrant/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/open-uri.rb:328:in `block (2 levels) in open_http'
	from /home/vagrant/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/net/http.rb:1413:in `block (2 levels) in transport_request'
	from /home/vagrant/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/net/http/response.rb:162:in `reading_body'
	from /home/vagrant/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/net/http.rb:1412:in `block in transport_request'
	from /home/vagrant/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/net/http.rb:1403:in `catch'
	from /home/vagrant/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/net/http.rb:1403:in `transport_request'
	from /home/vagrant/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/net/http.rb:1376:in `request'
	from /home/vagrant/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/open-uri.rb:319:in `block in open_http'
	from /home/vagrant/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/net/http.rb:852:in `start'
	from /home/vagrant/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/open-uri.rb:313:in `open_http'
	from /home/vagrant/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/open-uri.rb:708:in `buffer_open'
	from /home/vagrant/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/open-uri.rb:210:in `block in open_loop'
	from /home/vagrant/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/open-uri.rb:208:in `catch'
	from /home/vagrant/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/open-uri.rb:208:in `open_loop'
	from /home/vagrant/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/open-uri.rb:149:in `open_uri'
	from /home/vagrant/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/open-uri.rb:688:in `open'
	from /home/vagrant/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/open-uri.rb:34:in `open'
	from /vagrant/wunderground.rb:3:in `<main>'
Raw
README.md

README

The Issue

This api call to the Weather Underground service works as expected using MRI ruby-1.9.3-p429 but not when using MRI ruby-2.0.0-p195.

This gist shows the code and the response when using 1.9.3 and 2.0.0.

The code sample is taken directly from the Weather Underground's documentation

Update

After working with @zph and @injekt there have been some interesting findings. I am including wunderground-working-2_0.rb as an example of a working version of the Weather Underground API using Ruby MRI 2.0.0.

The interesting piece was when @injekt showed that the rest-client gem could call the api without error. He then found that the gem was manually rescuing the Zlib::DataError (https://github.com/rest-client/rest-client/blob/master/lib/restclient/request.rb#L258).

So, for now, if I want to use the Weather Underground API and MRI 2.0.0, I'll be doing it with the rest-client gem.

Additionally interesting reading can be found with this Ruby core commit, where a significant update was made to the Zlib libraries.

Raw
wunderground-working-2_0.rb
require 'rest-client'
require 'json'
api_key = '' # required by producing your own API key from http://www.wunderground.com/weather/api/
url = "http://api.wunderground.com/api/#{api_key}/geolookup/conditions/q/IA/Cedar_Rapids.json"
res = RestClient.get url
parsed_json = JSON.parse(res)
location = parsed_json['location']['city']
temp_f = parsed_json['current_observation']['temp_f']
print "Current temperature in #{location} is: #{temp_f}\n"
Raw
wunderground.rb
require 'open-uri'
require 'json'
api_key = '' # required by producing your own API key from http://www.wunderground.com/weather/api/
open("http://api.wunderground.com/api/#{api_key}/geolookup/conditions/q/IA/Cedar_Rapids.json") do |f|
json_string = f.read
parsed_json = JSON.parse(json_string)
location = parsed_json['location']['city']
temp_f = parsed_json['current_observation']['temp_f']
print "Current temperature in #{location} is: #{temp_f}\n"
end
@zph
Copy link

zph commented Jun 14, 2013

@geopet,

From error message, I'm wondering if RVM 2.0.0 isn't getting compiled correctly with ZLIB...

I'll try it on my system.

@zph
Copy link

zph commented Jun 14, 2013

Looks like I hit this error while stepping through code in 2.0.0:

TypeError: no implicit conversion of nil into String
from /Users/zander/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/uri/generic.rb:1484:in `to_s'

@zph
Copy link

zph commented Jun 14, 2013

Believe that this was when I switched it to Net::HTTP lib.

Zlib::DataError: incorrect header check
from /Users/zander/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/net/http/response.rb:357:in `finish'
2.0.0 (main):0 > wtf?
Exception: Zlib::DataError: incorrect header check
--
0: /Users/zander/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/net/http/response.rb:357:in `finish'
1: /Users/zander/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/net/http/response.rb:357:in `finish'
2: /Users/zander/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/net/http/response.rb:262:in `ensure in inflater'
3: /Users/zander/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/net/http/response.rb:262:in `inflater'
4: /Users/zander/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/net/http/response.rb:274:in `read_body_0'
5: /Users/zander/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/net/http/response.rb:201:in `read_body'
6: /Users/zander/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/net/http/response.rb:226:in `body'
7: /Users/zander/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/net/http/response.rb:163:in `reading_body'
8: /Users/zander/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/net/http.rb:1412:in `block in transport_request'
9: /Users/zander/.rvm/rub

Or the more full response

Zlib::DataError: incorrect header check
from /Users/zander/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/net/http/response.rb:357:in `finish'
2.0.0 (main):0 > s
ArgumentError: uncaught throw :breakout_nav
from /Users/zander/.rvm/gems/ruby-2.0.0-p195/gems/pry-debugger-0.2.2/lib/pry-debugger/commands.rb:209:in `throw'
/Users/zander/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/net/http/response.rb:357:in `finish': incorrect header check (Zlib::DataError)
    from /Users/zander/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/net/http/response.rb:357:in `finish'
    from /Users/zander/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/net/http/response.rb:262:in `ensure in inflater'
    from /Users/zander/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/net/http/response.rb:262:in `inflater'
    from /Users/zander/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/net/http/response.rb:274:in `read_body_0'
    from /Users/zander/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/net/http/response.rb:201:in `read_body'
    from /Users/zander/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/net/http/response.rb:226:in `body'
    from /Users/zander/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/net/http/response.rb:163:in `reading_body'
    from /Users/zander/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/net/http.rb:1412:in `block in transport_request'
    from /Users/zander/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/net/http.rb:1403:in `catch'
    from /Users/zander/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/net/http.rb:1403:in `transport_request'
    from /Users/zander/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/net/http.rb:1376:in `request'
    from /Users/zander/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/net/http.rb:1278:in `request_get'
    from /Users/zander/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/net/http.rb:479:in `block in get_response'
    from /Users/zander/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/net/http.rb:852:in `start'
    from /Users/zander/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/net/http.rb:582:in `start'
    from /Users/zander/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/net/http.rb:477:in `get_response'
    from /Users/zander/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/net/http.rb:454:in `get'
    from wunderground.rb:11:in `block in <main>'
    from /Users/zander/.rvm/gems/ruby-2.0.0-p195/gems/interception-0.3/lib/interception.rb:71:in `call'
    from /Users/zander/.rvm/gems/ruby-2.0.0-p195/gems/interception-0.3/lib/interception.rb:71:in `listen'
    from /Users/zander/.rvm/gems/ruby-2.0.0-p195/gems/pry-rescue-1.1.1/lib/pry-rescue/core_ext.rb:74:in `enable_rescuing!'
    from /Users/zander/.rvm/gems/ruby-2.0.0-p195/gems/pry-rescue-1.1.1/lib/pry-rescue/core_ext.rb:94:in `with_rescuing'
    from /Users/zander/.rvm/gems/ruby-2.0.0-p195/gems/pry-rescue-1.1.1/lib/pry-rescue/core_ext.rb:15:in `block (2 levels) in rescue'
    from /Users/zander/.rvm/gems/ruby-2.0.0-p195/gems/pry-rescue-1.1.1/lib/pry-rescue/core_ext.rb:13:in `catch'
    from /Users/zander/.rvm/gems/ruby-2.0.0-p195/gems/pry-rescue-1.1.1/lib/pry-rescue/core_ext.rb:13:in `block in rescue'
    from /Users/zander/.rvm/gems/ruby-2.0.0-p195/gems/pry-rescue-1.1.1/lib/pry-rescue/core_ext.rb:12:in `loop'
    from /Users/zander/.rvm/gems/ruby-2.0.0-p195/gems/pry-rescue-1.1.1/lib/pry-rescue/core_ext.rb:12:in `rescue'
    from wunderground.rb:10:in `<main>'

shell returned 1

Press ENTER or type command to continue

@zph
Copy link

zph commented Jun 14, 2013

http://stackoverflow.com/questions/9211830/zlib-decompression-throws-header-error-ruby

@zph
Copy link

zph commented Jun 14, 2013

Here are the headers from a curl -is URL dump:

HTTP/1.1 200 OK
Date: Fri, 14 Jun 2013 16:06:18 GMT
Server: Apache/1.3.42 (Unix) PHP/5.3.2
Cache-Control: max-age=300
Expires: Fri, 14 Jun 2013 16:11:18 GMT
X-CreationTime: 0.120
Set-Cookie: DT=1371225978:10222:365-e6; path=/; expires=Fri, 01-Jan-2020 00:00:00 GMT; domain=.wunderground.com
Connection: close
Transfer-Encoding: chunked
Content-Type: application/json; charset=UTF-8

Maybe there's something malformed in the 'chunked' encoding that 1.9.3 is more tolerant of and 2.0.0 is stricter?

I'm a bit stumped and should probably wrap up w/ lunch ;).

If I get time later tonight, I'll give it another once over.

Good luck @geopet

@zph

@geopet
Copy link
Author

geopet commented Jun 14, 2013

I'm going to try using another ruby version manager to build out Ruby 2.0. I don't think that it is the problem, but it is probably good idea to rule it out.

The challenge with your findings, is that they point out another issue. Is this a problem with Ruby 2.0 being too strict or with the Weather Underground API response not being strict enough?

@zph
Copy link

zph commented Jun 14, 2013

I think it's unlikely to be an RVM issue, but always worth exploring.

I dug a bit deeper using pry-rescue and then threw a binding.pry into the line of the exception.

Interesting... I tracked it down to where Net::HTTP.response throws the exception.

Below is the Pry session output

From: /Users/zander/tmp/5782836/wunderground.rb @ line 11 :

     6:
     7: api_key = YAML.load_file(File.expand_path "~/.wunderground")['key']
     8: url = "http://api.wunderground.com/api/#{ api_key }/geolookup/conditions/q/IA/Cedar_Rapids.json"
     9: #
    10: Pry.rescue do
 => 11: open(url) do |f|
    12:   json_string = f.read
    13:   parsed_json = JSON.parse(json_string)
    14:   location = parsed_json['location']['city']
    15:   temp_f = parsed_json['current_observation']['temp_f']
    16:   print "Current temperature in #{location} is: #{temp_f}\n"

Zlib::DataError: incorrect header check
from /Users/zander/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/net/http/response.rb:368:in `inflate'
2.0.0 (main):0 > n
ArgumentError: uncaught throw :breakout_nav
from /Users/zander/.rvm/gems/ruby-2.0.0-p195/gems/pry-debugger-0.2.2/lib/pry-debugger/commands.rb:209:in `throw'
2.0.0 (main):0 > whereami

Frame number: 17/22
Frame type: block

From: /Users/zander/tmp/5782836/wunderground.rb @ line 11 :

     6:
     7: api_key = YAML.load_file(File.expand_path "~/.wunderground")['key']
     8: url = "http://api.wunderground.com/api/#{ api_key }/geolookup/conditions/q/IA/Cedar_Rapids.json"
     9: #
    10: Pry.rescue do
 => 11: open(url) do |f|
    12:   json_string = f.read
    13:   parsed_json = JSON.parse(json_string)
    14:   location = parsed_json['location']['city']
    15:   temp_f = parsed_json['current_observation']['temp_f']
    16:   print "Current temperature in #{location} is: #{temp_f}\n"
Press ENTER or type command to continue

2.0.0 (main):0 > open(url)
Zlib::DataError: incorrect header check
from /Users/zander/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/net/http/response.rb:357:in `finish'
2.0.0 (main):0 > edit Net::HTTP.response
2.0.0 (main):0 > edit Net::HTTP
2.0.0 (main):0 > open(url)
Zlib::DataError: incorrect header check
from /Users/zander/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/net/http/response.rb:357:in `finish'
2.0.0 (main):0 > exit
/Users/zander/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/net/http/response.rb:357:in `finish': incorrect header check (Zlib::DataError)
    from /Users/zander/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/net/http/response.rb:357:in `finish'
    from /Users/zander/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/net/http/response.rb:262:in `ensure in inflater'
    from /Users/zander/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/net/http/response.rb:262:in `inflater'
    from /Users/zander/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/net/http/response.rb:274:in `read_body_0'
    from /Users/zander/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/net/http/response.rb:201:in `read_body'
    from /Users/zander/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/open-uri.rb:328:in `block (2 levels) in open_http'
    from /Users/zander/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/net/http.rb:1413:in `block (2 levels) in transport_request'
    from /Users/zander/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/net/http/response.rb:162:in `reading_body'
    from /Users/zander/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/net/http.rb:1412:in `block in transport_request'
    from /Users/zander/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/net/http.rb:1403:in `catch'
    from /Users/zander/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/net/http.rb:1403:in `transport_request'
    from /Users/zander/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/net/http.rb:1376:in `request'
    from /Users/zander/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/open-uri.rb:319:in `block in open_http'
    from /Users/zander/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/net/http.rb:852:in `start'
    from /Users/zander/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/open-uri.rb:313:in `open_http'
    from /Users/zander/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/open-uri.rb:708:in `buffer_open'
    from /Users/zander/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/open-uri.rb:210:in `block in open_loop'
    from /Users/zander/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/open-uri.rb:208:in `catch'
    from /Users/zander/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/open-uri.rb:208:in `open_loop'
    from /Users/zander/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/open-uri.rb:149:in `open_uri'
    from /Users/zander/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/open-uri.rb:688:in `open'
    from /Users/zander/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/open-uri.rb:34:in `open'
    from wunderground.rb:11:in `block in <main>'
    from /Users/zander/.rvm/gems/ruby-2.0.0-p195/gems/interception-0.3/lib/interception.rb:71:in `call'
    from /Users/zander/.rvm/gems/ruby-2.0.0-p195/gems/interception-0.3/lib/interception.rb:71:in `listen'
    from /Users/zander/.rvm/gems/ruby-2.0.0-p195/gems/pry-rescue-1.1.1/lib/pry-rescue/core_ext.rb:74:in `enable_rescuing!'
    from /Users/zander/.rvm/gems/ruby-2.0.0-p195/gems/pry-rescue-1.1.1/lib/pry-rescue/core_ext.rb:94:in `with_rescuing'
    from /Users/zander/.rvm/gems/ruby-2.0.0-p195/gems/pry-rescue-1.1.1/lib/pry-rescue/core_ext.rb:15:in `block (2 levels) in rescue'
    from /Users/zander/.rvm/gems/ruby-2.0.0-p195/gems/pry-rescue-1.1.1/lib/pry-rescue/core_ext.rb:13:in `catch'
    from /Users/zander/.rvm/gems/ruby-2.0.0-p195/gems/pry-rescue-1.1.1/lib/pry-rescue/core_ext.rb:13:in `block in rescue'
    from /Users/zander/.rvm/gems/ruby-2.0.0-p195/gems/pry-rescue-1.1.1/lib/pry-rescue/core_ext.rb:12:in `loop'
    from /Users/zander/.rvm/gems/ruby-2.0.0-p195/gems/pry-rescue-1.1.1/lib/pry-rescue/core_ext.rb:12:in `rescue'
    from wunderground.rb:10:in `<main>'

shell returned 1

Press ENTER or type command to continue

Frame number: 0/27

From: /Users/zander/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/net/http/response.rb @ line 357 Net::HTTPResponse::Inflater#finish:

    356: def finish
 => 357:   require'pry';binding.pry
    358:   @inflate.finish
    359: end

2.0.0 (#<Net::HTTPResponse::Inflater:0x007f902c1b80c8>):0 > @inflate
=> #<Zlib::Inflate:0x007f902c1b8078 @dictionaries={}>
2.0.0 (#<Net::HTTPResponse::Inflater:0x007f902c1b80c8>):0 > cd @inflate
2.0.0 (#<Zlib::Inflate:0x007f902c1b8078>):1 > ls
Zlib::ZStream#methods:
  adler
  avail_out
  close
  data_type
  ended?
  finished?
  flush_next_out
  stream_end?
  total_out
  avail_in
  avail_out=
  closed?
  end
  finish
  flush_next_in
  reset
  total_in
Zlib::Inflate#methods: <<
add_dictionary
inflate
set_dictionary
sync
sync_point?
self.methods: __pry__
instance variables: @dictionaries
locals: _
__
_dir_
_ex_
_file_
_in_
_out_
_pry_
2.0.0 (#<Zlib::Inflate:0x007f902c1b8078>):1 > ? finish

From: ext/zlib/zlib.c (C Method):
Owner: Zlib::ZStream
Visibility: public
Signature: finish()
Number of lines: 3

Finishes the stream and flushes output buffer.  If a block is given each
chunk is yielded to the block until the input buffer has been flushed to
the output buffer.
2.0.0 (#<Zlib::Inflate:0x007f902c1b8078>):1 >

Exception appears to start from this file and line:

/Users/zander/.rvm/rubies/ruby-2.0.0-p195/lib/ruby/2.0.0/net/http/response.rb @ line 357

@zph
Copy link

zph commented Jun 14, 2013

This is where exception gets thrown:

  class Inflater # :nodoc:

    ##
    # Creates a new Inflater wrapping +socket+

    def initialize socket
      @socket = socket
      # zlib with automatic gzip detection
      @inflate = Zlib::Inflate.new(32 + Zlib::MAX_WBITS)
    end

    ##
    # Finishes the inflate stream.

    def finish
      @inflate.finish # EXCEPTION THROWN HERE!
    end

    ##
    # Returns a Net::ReadAdapter that inflates each read chunk into +dest+.
    #
    # This allows a large response body to be inflated without storing the
    # entire body in memory.

    def inflate_adapter(dest)
      block = proc do |compressed_chunk|
        @inflate.inflate(compressed_chunk) do |chunk|
          dest << chunk
        end
      end

      Net::ReadAdapter.new(block)
    end

From net/http/response.rb in 2.0.0 MRI.

@zph
Copy link

zph commented Jun 14, 2013

New Zlib options available in 2.0.0... wonder if they're the root of the issue.

See Zlib section about chunked responses:
http://globaldev.co.uk/2013/03/ruby-2-0-0-in-detail/

Also:
http://stackoverflow.com/questions/5279068/zlib-decompress-header-check-error?rq=1

@zph
Copy link

zph commented Jun 15, 2013

@geopet

Any further answers on this? I'm curious to see how it pans out 😄

@geopet
Copy link
Author

geopet commented Jun 18, 2013

@zph Nothing so far. I wasn't able to prove that it was poor compiling that was causing the issue either. So I'm going to chat about this in the #ruby IRC channel as soon as I have a moment.

@zph
Copy link

zph commented Jun 19, 2013

Interesting.

Wonder if @drbrain would have a better idea than we do about why rest-client succeeds with this request in MRI 2.0.0 when open-uri fails, possibly due to the ZLIB stuff.

Very interesting about @injekt finding that rest-client rescues the error and then succeeds as you note: https://github.com/rest-client/rest-client/blob/master/lib/restclient/request.rb#L258-L262.

@zph
Copy link

zph commented Jun 20, 2013

@geopet,

Check it out: https://github.com/zph/mri_2_0_0_zlib_troubleshooting

I've promoted it to a full repo for easier management and briefly wrote up my process and progress.

Seems like a corrupted HTTP response, ie corrupted Gzip compression.

Perhaps more avenues of exploration will appear as we mull on it.

@jasl
Copy link

jasl commented Jul 6, 2013

@zph
I trigger this issue when using omniauth do oauth through proxy, so I tried to monkey patch Net::HTTPResponse::Inflater https://gist.github.com/jasl/5939460

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment