Skip to content

Instantly share code, notes, and snippets.

@maglietti

maglietti/HTTPie-blog.md

Last active July 8, 2019 23:57
Show Gist options
  • Save maglietti/1521324b1e615b912440 to your computer and use it in GitHub Desktop.
Save maglietti/1521324b1e615b912440 to your computer and use it in GitHub Desktop.
Download ZIP
Exploring Akamai OPEN APIs from the command line using `HTTPie` and `jq`
Raw
HTTPie-blog.md

Exploring Akamai OPEN APIs from the command line using HTTPie and jq

This article covers the following topics:

Introduction

When faced with a new interface, like the Akamai OPEN API catalog, developers don't usually start writing code from scratch. We like to play with the API first to get a feel for how it works. With RESTful APIs, developers typically reach for the curl command and start poking at the API making simple GET calls. Akamai APIs aren't compatible with curl out of the box due to security restrictions and the Akamai authorization header signing requirements.

Our first command line utility was edgegrid-curl. For over a year, egcurl has been the API evaluators tool of choice although the interface was not great.

A few weeks ago we created an authorization plugin for the popular HTTPie CLI HTTP tool using the official Akamai EdgeGrid signing module for python.

Installation

The first order of business is to get the httpie tool installed and operational on your development system. The easiest way to install this is to use pip, as it will install http, the edgegrid library and all requirements for you in a single command:

pip install httpie-edgegrid

Alternately, you can install it via git. The httpie-edgegrid library contains the necessary code if you want to examine the code in a local directory.

In a directory on your development machine, perform the following:

  • git clone https://github.com/akamai-open/httpie-edgegrid.git
  • cd httpie-edgegrid
  • python setup.py install # as root

This will build and install the HTTPie http command line tool to /usr/local/bin and enable the edgegrid authorization module.

Verify that the http command functions for regular HTTP calls by using the following command:

[~]$ http example.com
HTTP/1.1 200 OK
Accept-Ranges: bytes
Cache-Control: max-age=604800
Content-Length: 1270
Content-Type: text/html
Date: Wed, 22 Jul 2015 20:16:49 GMT
Etag: "359670651"
Expires: Wed, 29 Jul 2015 20:16:49 GMT
Last-Modified: Fri, 09 Aug 2013 23:54:35 GMT
Server: ECS (ewr/15BD)
X-Cache: HIT
x-ec-custom-error: 1

<!doctype html>
<html>
<head>
    <title>Example Domain</title>

    <meta charset="utf-8" />
    <meta http-equiv="Content-type" content="text/html; charset=utf-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1" />

    ...

</head>

<body>
<div>
    <h1>Example Domain</h1>
    <p>This domain is established to be used for illustrative examples in documents. You may use this
    domain in examples without prior coordination or asking for permission.</p>
    <p><a href="http://www.iana.org/domains/example">More information...</a></p>
</div>
</body>
</html>

[~]$

Configuration

Out of the box, http will make basic HTTP GET requests, or POST requests when there is data. It will return the results in a cleanly formatted and colorized result set. To explore the Akamai OPEN APIs, you'll need to add some extra command line options.

Akamai OPEN APIs require a set of API Credentials that you configure in the Luna Control Center. Follow the OPEN API Provisioning steps to set up API credentials. The httpie-edgegrid library uses the .edgerc credential file used by most of our akamai-open signing libraries. Follow the Configure your client instructions to get your credentials in a form that HTTPie will understand. You can use the gen_edgerc.py script in the httpie distribution to translate the Luna output into a configuration file.

Making your first call

You can now make a call to the diagnostic-tools API. A few command line options are required to utilize the edgegrid signing plugin. The format of an OPEN API Call is as follows:

% http --auth-type edgegrid -a <section_name>: :/<api_endpoint>

Making the first sample call from the introduction documentation on developer.akamai.com in HTTPie looks like this:

% http --auth-type edgegrid -a default: :/diagnostic-tools/v1/locations

This is great! The locations object was returned and colorized! However, I have a tendency to forget all of those command line options after a while, so you can put them into the HTTPie configuration file. All of the following examples assume that you have done this.

Edit your ~/.httpie/config.json file to include the default options from below:

{
    "__meta__": {
        "about": "HTTPie configuration file", 
        "help": "https://github.com/jakubroztocil/httpie#config", 
        "httpie": "0.9.2"
    }, 
    "default_options": [
        "--verbose",
        "--traceback", 
        "--auth-type=edgegrid", 
        "-adefault:"
    ], 
    "implicit_content_type": "json"
}

These default options enable the following:

  • --verbose Print the whole HTTP exchange (request and response).
  • --traceback Prints exception traceback should one occur.
  • --auth-type=edgegrid Selects the edgegrid authorization method.
  • -adefault Uses the credentials from the [default] section of ~/.edgerc.

Now the same command can be run simply:

[~]$ http :/diagnostic-tools/v1/locations

This is much easier and will benefit you while you start building up query parameters and submitting data in PUT or POST requests.

Making API Calls from the command line

As you can see from above, making API calls from the command line is simple with HTTPie. But what about when you need to include query parameters in the URI? The HTTPie utility makes sending data in the request very simple.

For example, you can make what appears to be a simple request using dig via the diagnostic-tools API. The call structure assumes that you have edited your config.json file as suggested above.

Once you've retrieved the locations, you can use one of them to select a server as the origin of the dig command. To do this, make a dig request for the A record for developer.akamai.com from a server in the United Kingdom.

[~]$ http -h :/diagnostic-tools/v1/dig hostname==developer.akamai.com \
location=="Chessington, United Kingdom" queryType==A

Adding -h to the command will tell HTTPie to only display the request headers sent to the API, as opposed to showing the pretty-printed JSON information that is printed by default.

GET /diagnostic-tools/v1/dig?hostname=developer.akamai.com&location=Chessington%2C+United+Kingdom&queryType=A HTTP/1.1
Accept: */*
Accept-Encoding: gzip, deflate
Authorization: EG1-HMAC-SHA256 client_token=akab-xxxxxxxxxxxxxxx-xxxxxxxxxxxxxxxx;access_token=akab-xxxxxxxxxxxxxxxx-xxxxxxxxxxxxxxxx;timestamp=20150722T21:30:50+0000;nonce=1c6a60c5-05df-4608-83f2-c720d52ab28e;signature=fpFagKeEEbnjZwv71wD6u3Qvza7wtOa4qxeQ3As0VwQ=
Connection: keep-alive
Host: akab-xxxxxxxxxxxxxxxx-xxxxxxxxxxxxxxxx.luna.akamaiapis.net
User-Agent: HTTPie/0.9.2
[~]$

Look at the GET line above and notice that http assembles the URI query parameters and takes care of URI encoding for you. The proper encoding the query parameters is one of the most frequently causes for confusion for developers new to using REST APIs in general and our goal is to provide tools that help you become successful quickly.

Parsing responses with jq

Sometimes getting a huge JSON object back from an API request can be quite confusing. I make a lot of API calls from the command line and have found the jq JSON parser to be invaluable when trying to see the data I want among all of the data that I receive back.

Installing jq is simple. Follow the instructions on the Download jq page. If you don't care to use a package manager, or are working on Windows or other Unix distributions, you can grab a precompiled binary from the jq website. Alternately, if you are a Mac user who uses homebrew, brew install jq will install the tool for you.

Calling the diagnostic-tools/v1/dig resource returns a large JSON object that contains the output from dig parsed into its components as well as the dig output itself. This JSON object scrolls right off my screen every time, and I am left searching back through the text in my terminal window to find what I requested. Try it for yourself and you will see what I mean.

[~]$ http -b :/diagnostic-tools/v1/dig hostname==developer.akamai.com \
location=="Chessington, United Kingdom" queryType==A

As with the -h option above, the -b option only displays the body from the API request response.

Lets look at just the dig command output using jq.

[~]$ http -b :/diagnostic-tools/v1/dig hostname==developer.akamai.com \
location=="Chessington, United Kingdom" queryType==A | jq '.dig.result'| xargs printf

; <<>> DiG 9.8.1-P1 <<>> developer.akamai.com -t A
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 27725
;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 8, ADDITIONAL: 8

;; QUESTION SECTION:
;developer.akamai.com.      IN  A

;; ANSWER SECTION:
developer.akamai.com.   300 IN  CNAME   developer.akamai.com.edgekey.net.
developer.akamai.com.edgekey.net. 21600 IN CNAME e8952.dscb.akamaiedge.net.
e8952.dscb.akamaiedge.net. 20   IN  A   2.16.89.105

;; AUTHORITY SECTION:
dscb.akamaiedge.net.    4000    IN  NS  n3dscb.akamaiedge.net.
dscb.akamaiedge.net.    4000    IN  NS  n0dscb.akamaiedge.net.
dscb.akamaiedge.net.    4000    IN  NS  n4dscb.akamaiedge.net.
dscb.akamaiedge.net.    4000    IN  NS  n5dscb.akamaiedge.net.
dscb.akamaiedge.net.    4000    IN  NS  n7dscb.akamaiedge.net.
dscb.akamaiedge.net.    4000    IN  NS  n6dscb.akamaiedge.net.
dscb.akamaiedge.net.    4000    IN  NS  n1dscb.akamaiedge.net.
dscb.akamaiedge.net.    4000    IN  NS  n2dscb.akamaiedge.net.

;; ADDITIONAL SECTION:
n0dscb.akamaiedge.net.  4000    IN  A   213.248.117.207
n1dscb.akamaiedge.net.  6000    IN  A   92.123.142.54
n2dscb.akamaiedge.net.  8000    IN  A   88.221.81.193
n3dscb.akamaiedge.net.  4000    IN  A   92.123.66.101
n4dscb.akamaiedge.net.  6000    IN  A   92.123.66.100
n5dscb.akamaiedge.net.  8000    IN  A   92.123.66.118
n6dscb.akamaiedge.net.  4000    IN  A   92.123.66.103
n7dscb.akamaiedge.net.  6000    IN  A   92.123.66.116

;; Query time: 10 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Wed Jul 22 21:05:10 2015
;; MSG SIZE  rcvd: 432

[~]$
  • Note that the pipe | is standard unix notation which pipes the result through another command. This should work correctly on OSX or other Unix based systems.
  • Windows users will need to stream the output into a file and run jq.exe against that file. The jq tutorial has information on how to do this.
  • The \ character is a Unix short hand that tells the shell that a command is not complete and will continue on the next line.
  • The jq '.dig.result' section only prints out the dig results.
  • The xargs printf command translates the \n characters in the output into newlines (OSX/Unix only).
  • The jq tutorial is a great resource for getting started with jq.

And, for example, lets just look at the A record for developer.akamai.com:

[~]$ http -b :/diagnostic-tools/v1/dig hostname==developer.akamai.com \
location=="Chessington, United Kingdom" queryType==A | \
jq '.dig.answerSection[] | select(.recordType == "A")'

{
  "domain": "e8952.dscb.akamaiedge.net.",
  "ttl": "20",
  "recordClass": "IN",
  "recordType": "A",
  "preferenceValues": null,
  "value": "2.16.89.105"
}
[~]$

You will likely find this output much easier to parse.

Posting JSON data with HTTPie

Requesting information via an API with GET methods can only get you so far. Eventually, you will need to make changes or interact with an API that requires you to send along a JSON object in the request body. HTTPie makes this easy as well.

Lets invalidate a file in the cache using the Content Control Utility API.

Using HTTPie, the JSON object is easily created on the command line.

[~]: http -a ccu: POST :/ccu/v2/queues/default \
objects:='["https://developer.akamai.com/stuff/Akamai_Time_Reference/AkamaiTimeReference.html"]' \
action=invalidate type=arl domain=production

As a side note, there are defaults for all action (remove), type (arl) and domain (production) so the above command is equivalent to this:

http -a ccu: POST :/ccu/v2/queues/default \
objects:='["https://developer.akamai.com/stuff/Akamai_Time_Reference/AkamaiTimeReference.html"]' \
action=invalidate

The JSON object is built from the data included in the request.

{
    "action": "invalidate", 
    "domain": "production", 
    "objects": [
        "https://developer.akamai.com/stuff/Akamai_Time_Reference/AkamaiTimeReference.html"
    ], 
    "type": "arl"
}

The CCU API responds with information about the request.

{
    "detail": "Request accepted.", 
    "estimatedSeconds": 240, 
    "httpStatus": 201, 
    "pingAfterSeconds": 240, 
    "progressUri": "/ccu/v2/purges/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx", 
    "purgeId": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx", 
    "supportId": "xxxxxxxxxxxxxxxxxxxx-xxxxxxxxx"
}

Using the progressUri you can easily manually check the status of the request until it is complete.

[~]$ http -b -a ccu: :/ccu/v2/purges/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx | \
jq .purgeStatus
"In-Progress"
[~]$  http -b -a ccu: :/ccu/v2/purges/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx | \
jq .purgeStatus
"Done"
[~]$

Summary

The Akamai Developer relations team is committed to constant improvement, creating tools, sample code and tutorials to get you started with the API as quickly as you can. We hope you find this new command line tool useful. Feedback and suggestions are always welcome in the Akamai Community Forum.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment