Skip to content

Instantly share code, notes, and snippets.

@mmcgrana

mmcgrana/gist.md

Last active June 15, 2023 21:25
Show Gist options
  • Save mmcgrana/10681030 to your computer and use it in GitHub Desktop.
Save mmcgrana/10681030 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
gist.md

HTTP API Design Guide

(Copy of internal Heroku document for sharing with collaborators.)

Introduction

This guide describes the design of RESTful HTTP+JSON APIs conforming to Heroku’s API conventions. It’s based on our design work for the Heroku Platform API.

This guide informs additions to that API and also guides new internal APIs at Heroku. We hope it’s also of interest to API designers outside of Heroku.

Our goals here are consistency and focusing on business logic while avoiding design bikeshedding. We’re looking for a good, consistent, well-documented way to design APIs, not necessarily the ideal or only way.

We assume you’re familiar with the basics of HTTP, REST, and JSON and won’t cover those in this guide.

Contents

Return appropriate status codes

Return appropriate HTTP status codes with each response. Successful responses should be coded according to this guide:

  • 200: Request succeeded for a GET calls, and for DELETE or PATCH calls that complete synchronously
  • 201: Request succeeded for a POST call that completes synchronously
  • 202: Request succeeded for a POST, DELETE, or PATCH call that will complete asynchronously
  • 206: Request succeeded on GET, but only a partial response returned: see above on ranges

Refer to the HTTP response code spec for guidance on status codes for user error and server error cases.

Provide full resources where available

Provide the full resource representation (i.e. the object with all attributes) whenever possible in the response. Always provide the full resource on 200 and 201 responses, including PUT/PATCH and DELETE requests, e.g.:

$ curl -X DELETE \  
  https://service.com/apps/1f9b/domains/0fd4

HTTP/1.1 200 OK
Content-Type: application/json;charset=utf-8
...
{
  "created_at": "2012-01-01T12:00:00Z",
  "hostname": "subdomain.example.com",
  "id": "01234567-89ab-cdef-0123-456789abcdef",
  "updated_at": "2012-01-01T12:00:00Z"
}

202 and 204 responses will not include the full resource representation, e.g.:

$ curl -X DELETE \  
  https://service.com/apps/1f9b/dynos/05bd

HTTP/1.1 202 Accepted
Content-Type: application/json;charset=utf-8
...
{}

Accept serialized JSON on request bodies

Accept serialized JSON on PUT/PATCH/POST request bodies, either instead of or in addition to form-encoded data. This creates symmetry with JSON-serialized response bodies, e.g.:

$ curl -X POST https://service.com/apps \
    -H "Content-Type: application/json" \
    -d '{"name": "demoapp"}'

{
  "id": "01234567-89ab-cdef-0123-456789abcdef",
  "name": "demoapp",
  "owner": {
    "email": "[email protected]",
    "id": "01234567-89ab-cdef-0123-456789abcdef"
  },
  ...
}

Provide resource (UU)IDs

Give each resource an id attribute by default. Use UUIDs unless you have a very good reason not to. Don’t use IDs that won’t be globally unique across instances of the service or other resources in the service, especially auto-incrementing IDs.

Render UUIDs in downcased 8-4-4-4-12 format, e.g.:

"id": "01234567-89ab-cdef-0123-456789abcdef"

Provide standard timestamps

Provide created_at and updated_at timestamps for resources by default, e.g:

{
  ...
  "created_at": "2012-01-01T12:00:00Z",
  "updated_at": "2012-01-01T13:00:00Z",
  ...
}

These timestamps may not make sense for some resources, in which case they can be omitted.

Use UTC times formatted in ISO8601

Accept and return times in UTC only. Render times in ISO8601 format, e.g.:

"finished_at": "2012-01-01T12:00:00Z"

Use consistent path formats

Prefer endpoint layouts that don’t need any special actions for individual resources, i.e. standard RESTful design. In cases where special actions are needed, place them under a standard actions prefix, to clearly delineate them:

/resources/:resource/actions/:action

e.g.

/runs/{run_id}/actions/stop

Downcase paths and attributes

Use downcased and dash-separated path names, for alignment with hostnames, e.g:

service-api.com/users
service-api.com/app-setups

Downcase attributes as well, but use underscore separators so that attribute names are valid JSON keys, e.g.:

"service_class": "first"

Nest foreign key relations

Serialize foreign key references with a nested object, e.g.:

{
  "name": "service-production",
  "owner": {
    "id": "5d8201b0..."
  },
  ...
}

Instead of e.g:

{
  "name": "service-production",
  "owner_id": "5d8201b0...",
  ...
}

This approach makes it possible to inline more information about the related resource without having to change the structure of the response or introduce more top-level response fields, e.g.:

{
  "name": "service-production",
  "owner": {
    "id": "5d8201b0...",
    "name": "Alice",
    "email": "[email protected]"
  },
  ...
}

Support non-id dereferencing for convenience

In some cases it may be inconvenient for end-users to provide IDs to identify a resource. For example, a user may think in terms of a Heroku app name, but that app may be identified by a UUID. In these cases you may want to accept both an id or name, e.g.:

$ curl https://service.com/apps/{app_id_or_name}
$ curl https://service.com/apps/97addcf0-c182
$ curl https://service.com/apps/www-prod

Do not accept only names to the exclusion of IDs.

Generate structured errors

Generate consistent, structured response bodies on errors. Include a machine-readable error id, a human-readable error message, and optionally a url pointing the client to further information about the error and how to resolve it, e.g.:

HTTP/1.1 429 Too Many Requests

{
  "id":      "rate_limit",
  "message": "Account reached its API rate limit.",
  "url":     "https://docs.service.com/rate-limits"
}

Document your error format and the possible error ids that clients may encounter.

Support caching with Etags

Include an ETag header in all responses, identifying the specific version of the returned resource. The user should be able to check for staleness in their subsequent requests by supplying the value in the If-None-Match header.

Trace requests with Request-Ids

Include a Request-Id header in each API response, populated with a UUID value. If both the server and client log these values, it will be helpful for tracing and debugging requests.

Paginate with Ranges

Paginate any responses that are liable to produce large amounts of data. Use Content-Range headers to convey pagination requests. Follow the example of the Heroku Platform API on Ranges for the details of request and response headers, status codes, limits, ordering, and page-walking.

Show rate limit status

Rate limit requests from clients to protect the health of the service and maintain high service quality for other clients. You can use a token bucket algorithm to quantify request limits.

Return the remaining number of request tokens with each request in the RateLimit-Remaining response header.

Version with Accepts header

Version the API from the start. Use the Accepts header to communicate the version, along with a custom content type, e.g.:

Accept: application/vnd.heroku+json; version=3

Prefer not to have a default version, instead requiring clients to explicitly peg their usage to a specific version.

Minimize path nesting

In data models with nested parent/child resource relationships, paths may become deeply nested, e.g.:

/orgs/{org_id}/apps/{app_id}/dynos/{dyno_id}

Limit nesting depth by preferring to locate resources at the root path. Use nesting to indicate scoped collections. For example, for the case above where a dyno belongs to an app belongs to an org:

/orgs/{org_id}
/orgs/{org_id}/apps
/apps/{app_id}
/apps/{app_id}/dynos
/dynos/{dyno_id}

Provide machine-readable JSON schema

Provide a machine-readable schema to exactly specify your API. Use prmd to manage your schema, and ensure it validates with prmd verify.

Provide human-readable docs

Provide human-readable documentation that client developers can use to understand your API.

If you create a schema with prmd as described above, you can easily generate Markdown docs for all endpoints with with prmd doc.

In addition to endpoint details, provide an API overview with information about:

  • Authentication, including acquiring and using authentication tokens.
  • API stability and versioning, including how to select the desired API version.
  • Common request and response headers.
  • Error serialization format.
  • Examples of using the API with clients in different languages.

Provide executable examples

Provide executable examples that users can type directly into their terminals to see working API calls. To the greatest extent possible, these examples should be usable verbatim, to minimize the amount of work a user needs to do to try the API, e.g.:

$ export TOKEN=... # acquire from dashboard
$ curl -is https://[email protected]/users

If you use prmd to generate Markdown docs, you will get examples for each endpoint for free.

Describe stability

Describe the stability of your API or its various endpoints according to its maturity and stability, e.g. with prototype/development/production flags.

See the Heroku API compatabilty policy for a possible stability and change management approach.

Once your API is declared production-ready and stable, do not make backwards incompatible changes within that API version. If you need to make backwards-incompatible changes, create a new API with an incremented version number.

Require SSL

Require SSL to access the API, without exception. It’s not worth trying to figure out or explain when it is OK to use SSL and when it’s not. Just require SSL for everything.

Pretty-print JSON by default

The first time a user sees your API is likely to be at the command line, using curl. It’s much easier to understand API responses at the command-line if they are pretty-printed. For the convenience of these developers, pretty-print JSON responses, e.g.:

{
  "beta": false,
  "email": "[email protected]",
  "id": "01234567-89ab-cdef-0123-456789abcdef",
  "last_login": "2012-01-01T12:00:00Z",
  "created_at": "2012-01-01T12:00:00Z",
  "updated_at": "2012-01-01T12:00:00Z"
}

Instead of e.g.:

{"beta":false,"email":"[email protected]","id":"01234567-89ab-cdef-0123-456789abcdef","last_login":"2012-01-01T12:00:00Z", "created_at":"2012-01-01T12:00:00Z","updated_at":"2012-01-01T12:00:00Z"}

Be sure to include a trailing newline so that the user’s terminal prompt isn’t obstructed.

For most APIs it will be fine performance-wise to pretty-print responses all the time. You may consider for performance-sensitive APIs not pretty-printing certain endpoints (e.g. very high traffic ones) or not doing it for certain clients (e.g. ones known to be used by headless programs).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment