Skip to content

Instantly share code, notes, and snippets.

@xiconet
Forked from cryptix/client.go
Last active October 25, 2015 03:43
Show Gist options
  • Save xiconet/818a897d3542e5e8443f to your computer and use it in GitHub Desktop.
Save xiconet/818a897d3542e5e8443f to your computer and use it in GitHub Desktop.
multipart upload with io.Pipe
package main
import (
"io"
"log"
"mime/multipart"
"net/http"
"os"
"path/filepath"
"runtime"
"github.com/cheggaaa/pb"
)
func main() {
if len(os.Args) != 2 {
log.Println("Usage: pipeUp <filename>\n")
os.Exit(1)
}
input, err := os.Open(os.Args[1])
check(err)
defer input.Close()
stat, err := input.Stat()
check(err)
pipeOut, pipeIn := io.Pipe()
fsize := stat.Size()
fname := stat.Name()
bar := pb.New(int(fsize)).SetUnits(pb.U_BYTES)
bar.ShowSpeed = true
writer := multipart.NewWriter(pipeIn)
// do the request concurrently
var resp *http.Response
done := make(chan error)
go func() {
// prepare request
req, err := http.NewRequest("POST", "http://localhost:9000/upload", pipeOut)
if err != nil {
done <- err
return
}
req.ContentLength = fsize // filesize
req.ContentLength += 232 // this value just works
req.ContentLength += int64(len(fname))
req.Header.Set("Content-Type", writer.FormDataContentType())
log.Println("Created Request")
bar.Start()
resp, err = http.DefaultClient.Do(req)
if err != nil {
done <- err
return
}
done <- nil
}()
part, err := writer.CreateFormFile("file", fname)
check(err)
out := io.MultiWriter(part, bar)
_, err = io.Copy(out, input)
check(err)
check(writer.Close())
check(pipeIn.Close()) // need to close the pipe to
check(<-done)
bar.FinishPrint("Upload done!")
}
func check(err error) {
_, file, line, _ := runtime.Caller(1)
if err != nil {
log.Fatalf("Fatal from <%s:%d>\nError:%s", file, line, err)
}
}
@xiconet
Copy link
Author

xiconet commented Oct 25, 2015

I had some hard time trying to find why I had a mismatch between the "announced" request content-length and the real length of the body when using this program to upload to pcloud.com.
In the end I found that req.ContentLength := filesize + 232 + len(filename) works.
Besides I find strange to use os.Args[1] (the file path) as a parameter for the request body length, unless filepath = filename of course. But the original program would not work even in this case. Weird.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment