Reading about how curl handle and encode parameters to understand what happened on requests with values containing spaces, like following

Today, during a pair programming session testing an endpoint sending a value to server and store it, a test case intrigue us while using curl.

We were trying to send a query string parameter containing spaces, and we got an unexpected behavior, like detailed on this gist.

Initial command with spaced value, was the following:

curl -vvv -X POST "http://localhost:4000/set?somekey=spaced value"

On server side, it only logs the following Malformed message, with not to much detail about the received message:

> INFO:tornado.general:Malformed HTTP message from ::1: Malformed HTTP request line

Inspecting curl generated HTTP request, the follwoing snippets shows the sent and received data:

> POST /set?key=k ey HTTP/1.1
> Host: localhost:4000
> User-Agent: curl/7.61.0
> Accept: */*
>
< HTTP/1.1 400 Bad Request

HTTP request URL is not fully enconded, specially the parameter. In this case, spaces are not allowed on request.

curl doesn't encode URL data automatically.

To send this as valid POST requests with query string parameters, as expected from the begining, it should modify or curl command as follow:

Split the querystring part and move to option --data-urlencode. Using this option, curl automatically consider this request as POST;
Add -G option to send all --data-urlencode value on querystring, as GET. It can still be combined with -XPOST option to send as POST.

The final command is:

curl -vvv -G -X POST "http://localhost:4000/set" --data-urlencode "somekey=spaced value"

Running this, we have the encoded data being sent on the query string part:

> POST /set?key=k%20%20y HTTP/1.1
> Host: localhost:4000
> User-Agent: curl/7.61.0
> Accept: */*
>
< HTTP/1.1 201 Created

On server LOG, now, we receive:

> INFO:tornado.access:201 POST /set?key=k%20%20y (::1) 0.91ms

References

Helpful links with guidance to understand better this behavior: