Created
June 1, 2012 15:44
-
-
Save mynameisfiber/2853066 to your computer and use it in GitHub Desktop.
Golang http close body blocking problem
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
package main | |
import ( | |
"crypto/tls" | |
"net" | |
"net/http" | |
"time" | |
"fmt" | |
"errors" | |
) | |
func redirectPolicy(req *http.Request, via []*http.Request) error { | |
if len(via) >= 3 { | |
return errors.New("stopped after 3 redirects") | |
} | |
return nil | |
} | |
func main() { | |
transport := &http.Transport{TLSClientConfig: &tls.Config{InsecureSkipVerify: true}, | |
Dial: func(netw, addr string) (net.Conn, error) { | |
// we want to wait a maximum of 1.75 seconds... | |
// since we're specifying a 1 second connect timeout and deadline | |
// (read/write timeout) is specified in absolute time we want to | |
// calculate that time first (before connecting) | |
deadline := time.Now().Add(800 * time.Millisecond) | |
c, err := net.DialTimeout(netw, addr, time.Second) | |
if err != nil { | |
return nil, err | |
} | |
c.SetDeadline(deadline) | |
return c, nil | |
}} | |
httpclient := &http.Client{Transport: transport, CheckRedirect: redirectPolicy} | |
req, err := http.NewRequest("GET", "http://bukk.it/boss.gif", nil) | |
req.Header.Add("User-Agent", "Bitly ImgGtr - Saving your bandwith and our time") | |
resp, err := httpclient.Do(req) | |
if err != nil { | |
return | |
} | |
start := time.Now().UnixNano() | |
resp.Body.Close() | |
fmt.Println("Close took: ", time.Now().UnixNano() - start) | |
} |
Thanks for the info! At second glance, I shouldn't hit the same snags with my use case, since I should always be trying to fetch the entire body.
simplequeue seems like a cool idea; definitely something ripe for a Go port. What made you guys want to use an HTTP queueing system over something like Kestrel queues? Let me know if/when you guys open source it, I'd love to give it a whirl.
I just learned Go a couple weeks ago. First thing I did was port a toy service I built out for converting images into ascii art called skeeter. Now I'm playing around building out an http pinger and analytics gatherer.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Also, we're doing a bunch of fun things with go! We are making a new version of bitly/simplequeue that is native go with many more advanced features and message guarantees. We also have a sweet json library for saner json manipulation and I have a slightly more complete redis library in my repo in addition to an "image fetcher" that determines the resolution of an image based on the first several bytes! What are you working on?