JSON Based Hypermedia Structures - Notes & Comparisons

collection+json.md

Collection+JSON format

http://amundsen.com/media-types/collection/examples/

This format takes into consideration collections (much like Siren). It also takes into consideration versioning. However, I'm not certain version matters in the data set if it pertains to an API version. That is best left being in the API URL. It could pertain to collection version, but I'm not sure the point. Documentation says each collection should have a version - but says nothing more about meaning or why.

Items are clearly distinct in this format and are organizationally positioned separate from links.
This is by far the most collision free structure.

However, the data format itself is a bit strange...Every field being an array. Data is always an array of objects. I understand the flexibility this presents and am not 100% against it. I just believe it's perhaps not needed as things could change based on the client application.

QUERIES

The queries section is very nice and something not found in other formats. However, I don't see anything for pagination and I believe it would certainly make sense to include by now.

Collection+JSON puts a lot of emphasis on the display and intended usage of data. Moreso than any other format. This is nice, but unless the client application was attempting to display a layout automatically, I don't see the need. A human engineer can interpret and decide things just fine and perhaps different clients or even access control rules would dictate a different visual representation anyway. Therefore I believe this oversteps the duties a little bit. Still clever and guidance/definition is what we're after of course.

LINKS

I like how links are even more separate from the data here than any other format. Due to this structure, there is no need to prefix with underscores, etc. Links still carry the same "rel" descriptive values which are important for orientation.

However, I would still use "self" instead of "href" for the current item's link for consistency. Again, Collection+JSON puts some emphasis on how these links should appear. For example, the "render" key. However, many of these properties "render" and "prompt" etc. are optional anyway.

The only downside here is that given how much emphasis and careful consideration is put on how data is rendered, there seems to be a lack of emphasis for "actions" or forms.

The "template" section is actually intended to be used as a guide for which data should be sent to the API in these cases (POST and PUT for update, following convention). However, I don't believe it's particularly descriptive. Also, the format states there should only be one form. This isn't a bad rule of thumb, but I could see where an API response may return multiple forms every now and then (related data, etc.).

ERRORS

http://amundsen.com/media-types/collection/examples/#ex-error

Not shown here is another section called "errors" and I feel this is important and worth mentioning. It's very debatable that errors should be contained within HTTP reponse headers...But I believe that, often, additional detail is required and messages being returned in an API are good because it means those messages can be updated from one central location instead of many client application updates. Superior design for maintenance. Additionally, HTTP headers should be kept small and while a status code is obviously present, an entire message intended for a human is perhaps inappropriate.

{ "collection" :
  {
    "version" : "1.0",
    "href" : "http://example.org/friends/",
    
    "links" : [
      {"rel" : "feed", "href" : "http://example.org/friends/rss"}
    ],
    
    "items" : [
      {
        "href" : "http://example.org/friends/jdoe",
        "data" : [
          {"name" : "full-name", "value" : "J. Doe", "prompt" : "Full Name"},
          {"name" : "email", "value" : "[email protected]", "prompt" : "Email"}
        ],
        "links" : [
          {"rel" : "blog", "href" : "http://examples.org/blogs/jdoe", "prompt" : "Blog"},
          {"rel" : "avatar", "href" : "http://examples.org/images/jdoe", "prompt" : "Avatar", "render" : "image"}
        ]
      },
      
      {
        "href" : "http://example.org/friends/msmith",
        "data" : [
          {"name" : "full-name", "value" : "M. Smith", "prompt" : "Full Name"},
          {"name" : "email", "value" : "[email protected]", "prompt" : "Email"}
        ],
        "links" : [
          {"rel" : "blog", "href" : "http://examples.org/blogs/msmith", "prompt" : "Blog"},
          {"rel" : "avatar", "href" : "http://examples.org/images/msmith", "prompt" : "Avatar", "render" : "image"}
        ]
      },
      
      {
        "href" : "http://example.org/friends/rwilliams",
        "data" : [
          {"name" : "full-name", "value" : "R. Williams", "prompt" : "Full Name"},
          {"name" : "email", "value" : "[email protected]", "prompt" : "Email"}
        ],
        "links" : [
          {"rel" : "blog", "href" : "http://examples.org/blogs/rwilliams", "prompt" : "Blog"},
          {"rel" : "avatar", "href" : "http://examples.org/images/rwilliams", "prompt" : "Avatar", "render" : "image"}
        ]
      }      
    ],
    
    "queries" : [
      {"rel" : "search", "href" : "http://example.org/friends/search", "prompt" : "Search",
        "data" : [
          {"name" : "search", "value" : ""}
        ]
      }
    ],
    
    "template" : {
      "data" : [
        {"name" : "full-name", "value" : "", "prompt" : "Full Name"},
        {"name" : "email", "value" : "", "prompt" : "Email"},
        {"name" : "blog", "value" : "", "prompt" : "Blog"},
        {"name" : "avatar", "value" : "", "prompt" : "Avatar"}
        
      ]
    }
  } 
}

hal.md

HAL format

Borrowed from: https://gist.github.com/kevinswiber/3066768
The HAL draft: http://tools.ietf.org/html/draft-kelly-json-hal-06

LINKS

I don't like how "_links" are merged with other data because it's meta data added on top by the RESTful service and relates directly to the API. I feel this makes things slightly confusing and leaves the structure vulnerable to conflicts -- what if the "orders" had their own links field?

In this case it's a bit narrow too. Most people would return the customer details along with the order for example. There would not really be a need to make a second HTTP request to the API to get that data. This is because the client application would likely want to display that information along side. However, there are times when we might want to link out to another page or make another API call. In this specific example, perhaps for the "basket" data.

This dives into relationships and I think Siren is better equipped to handle them. Again, I believe there is an importance in not having to call the API an unnecessary amount of times. Ultimately it should be up to the engineer to decide as well because of special considerations like payload size.

The organization of data is a little confusing as well. This seems to be a listing of orders, and the "_embedded" orders items make sense...But "shippedToday" and "currentlyProcessing" do not make sense where they are positioned. We are to assume, by convention, that this has nothing to do with HAL format (no underscore), but the data fields (whether they came from the data source or were added) are at the same level as _links for example.

What if the data source also had a key name "_links" ? This opens the possibility of conflicts. A data source should never conflict with a conventional organization system and the onus should be on the system and not the data source. There are better organizational structures that can be used which will never conflict.

{
  "_links": {
    "self": { "href": "/orders" },
    "next": { "href": "/orders?page=2" },
    "find": { "href": "/orders{?id}", "templated": true }
  },
  "_embedded": {
    "orders": [{
        "_links": {
          "self": { "href": "/orders/123" },
          "basket": { "href": "/baskets/98712" },
          "customer": { "href": "/customers/7809" }
        },
        "total": 30.00,
        "currency": "USD",
        "status": "shipped",
      },{
        "_links": {
          "self": { "href": "/orders/124" },
          "basket": { "href": "/baskets/97213" },
          "customer": { "href": "/customers/12369" }
        },
        "total": 20.00,
        "currency": "USD",
        "status": "processing"
    }]
  },
  "currentlyProcessing": 14,
  "shippedToday": 20
}

old-unified-format.md

Unified Hypermedia JSON Format

The following is outdated. See the unified-format.go file for a more up to date draft of ideas.

---

Here's what my proposed format would look like (rough draft) which combines elements of the above 3 formats and provides a little more (optional) detail along with a little cleaner hierachy. By comparing with the other formats you should easily be able to see similarities and techniques borrowed from each. This format should suit a wider range of applications.

Note that there's a few things not solidified yet and there are many good approaches to those challenges...I just haven't researched which way makes most sense yet. Use your imagination there =)

Also note that this format does not prevent one from having multiple forms/actions in a single response. I believe this is another important characteristic when it comes to designing a flexible structure that works for many applications. I also believe the onus is on the engineer and system architect to ensure things don't get too crazy here with response payload size.

Keep in mind many fields are optional here intentionally to allow for judiciary performance optimizations and flexibility. The Hypermedia JSON format has two broad concerns or goals:

First Goal: Allow engineers to browse API JSON responses and navigate through the API.

Second Goal: Provide a client application guidance on how the server side application operates and how it should be handling data.

The format does not make assumptions about data or actions. While it would be very common to see your normal CRUD operations under _actions, it is not restricted to those operations. It is left open to accommodate the needs of the application.

I don't think there's anything wrong with flexibility to the degree that we would see different, optional, fields across different APIs. There is no one size fits all API. However, the required fields in this format do represent a common set of data that can be used across the vast majority of applications.

• I should note that this is very preliminary. I expect to roll some things around in my head and get some feedback to further detail this out.

{
 "_meta": {
  "dataCount": 25,
  "dataLimit": 25,
  "dataTotal": 100,
  "dataOrder": {"created": "desc"},
  "responseTime": 0.004,
  "message": "Everything looking good kemosabe."
 },
 "_http": {
  "code": 200,
  "name": "OK",
  "message": "Optional message with more details...More useful in the event of errors. This section allows for the extension of HTTP status codes (perhaps this is all frowned upon and that's ok, a more generic HTTP status could could be presented in the headers remaining consistent with standards and this _http block could present additional related details for the more adventurous). More importantly, this could handle language translations quite nicely."
 },
 "_links": [
  {"rel": "self", "href": "/posts"},
  {"rel": "next", "href": "/posts/page-2"},
  {"rel": "prev", "href": "/posts/page-1"}
  // NOTE: Route format not solidified, completely open to a good way to present arguments and regex matches, etc.
  {"rel": "skip", "href": "/posts/page-{pageNumber:[0-9]}"},
  // This is why we don't need another "_links" within each item below under "_data" - the client application should be able to logically put it together. This provides for a smaller payload as a nice bonus.
  {"rel": "read", "href": "/posts/read/{id}"}
  // ...additional related links as needed by the application. Also note there can be additional optional keys to help the client application, such as "_target" and HTML attributes, etc. While HTML won't always be the client application language (mobile apps lets say), these are optional and a client application can still assume intent (or disregard).
 ],
 "_data": {
  // A completely flexible data structure - separate from other meta data
  "posts": [
    {"_id": "01234ef", "title": "Boom"}
  ]
 },
 "_actions": {
  "_forms": {
   "formName": {
    // For the most part, HTML attributes dictate the keys here
    "name": "add-post",
    "title": "Add Blog Post",
    "method": "POST",
    "action": "/posts/add",
    "enctype": "application/x-www-form-urlencoded",
    "fields": [
     // Note: Validation convention still not solidified either, there are many good conventions here as well.
     { "name": "title", "type": "text", "validation": [{"rule": "[A-z0-9]\s", "message": "Must be alpha numeric."}] },
     { "name": "authorId", "type": "hidden", "value": "42" }
    ]
   }
  },
  "_errors": {
    "formName": [
     {"name": "title", "message": "Cannot be blank."}
    ]
  }
  
 }
}

siren.md

Siren format

https://github.com/kevinswiber/siren

Siren has the idea of "classes" or collections. This value can be an array of classes which is a little confusing. Top level entities can have sub-entities. In the following example, "properties" represents an entity's data (perhaps from a data source) and entities under it are related.

I like this entities approach better than other proposed structures. Even with the goofy classes...Do arrays represent relationships? If so, why would this not be expressed as key values with key names like "belongsTo"?
Why would the client even care about the relationships?

I haven't seen how Siren handles a top level result set. I imagine only an "entities" key exists with optional "actions" and "links" (where links may not be optional).

I think "entities" is too distinct in this case. I think the "orderNumber" and "status" from the main item should be in the same place as the related data. Yes, the related entities would be sub-items of the parent entity/ies. However, that's up to the application's data design. Let's say we wanted to show a listing of more than one order with all these related details? How would that look? "entities" under "entities" and then where would the top level "properties" go? Would "properties" become an array? If so, how would you ever relate the "entities" to the "properties" ? You couldn't other than perhaps keying things. So representing relationships should be done as children under the parent item and not in a new top level item.

ACTIONS

These are great and other formats will say "_forms" which is also acceptable. The only thing I would add here would be some more (optional) detail. How about field data types? Validation rules? I think validation rules are important because a front-end client can validation locally before sending to the server/API for server side validation.

While server-side validation is always required (to ensure it is not bypassed), relieving an API from unnecessary calls is always a win win for everyone. Faster user experience (less requests over the internet) and less traffic to the server so that it can better do its job and serve more clients.

LINKS

The links in this format contain "rel" values with arrays. I'm not sure the logic behind this. Would it also make sense for a class to go on the links as well? I'm not sure. However, the links are separate from the "properties" which is good. Most other proposed formats use an underscore "_links" to visually separate from other data...But I like things to be separate by hierarchary.

I also believe links shold carry pagination with them, another thing not often seen. Next/prev implies the sense of pagination, but there seems to be no "page" or "skip" link to jump to a specific section.

{
  "class": [ "order" ],
  "properties": { 
      "orderNumber": 42, 
      "itemCount": 3,
      "status": "pending"
  },
  "entities": [
    { 
      "class": [ "items", "collection" ], 
      "rel": [ "http://x.io/rels/order-items" ], 
      "href": "http://api.x.io/orders/42/items"
    },
    {
      "class": [ "info", "customer" ],
      "rel": [ "http://x.io/rels/customer" ], 
      "properties": { 
        "customerId": "pj123",
        "name": "Peter Joseph"
      },
      "links": [
        { "rel": [ "self" ], "href": "http://api.x.io/customers/pj123" }
      ]
    }
  ],
  "actions": [
    {
      "name": "add-item",
      "title": "Add Item",
      "method": "POST",
      "href": "http://api.x.io/orders/42/items",
      "type": "application/x-www-form-urlencoded",
      "fields": [
        { "name": "orderNumber", "type": "hidden", "value": "42" },
        { "name": "productCode", "type": "text" },
        { "name": "quantity", "type": "number" }
      ]
    }
  ],
  "links": [
    { "rel": [ "self" ], "href": "http://api.x.io/orders/42" },
    { "rel": [ "previous" ], "href": "http://api.x.io/orders/41" },
    { "rel": [ "next" ], "href": "http://api.x.io/orders/43" }
  ]
}

thoughts.md

Thoughts on Existing Formats

I think they are all well intended, but miss the mark a tiny bit. In fact, I would combine several aspects from each to be honest. To be 100% clear, I generally agree with and appreciate the proposals here, but I think more work is required to really polish things up.

When comparing these formats it's a little bit obvious that there were specific needs in mind. Collection+JSON was obviously targeted more toward NoSQL/schemaless data sources and had a more heavy emphasis on visual representation within the client application than the other formats. Many formats (and there's a few I didn't cover) have a strong emphasis on e-commerce. Obviously e-commerce applications are different than most other systems.

However, what's really important to take away from this is that I think a consolidated format could serve all of these needs and intended use cases. I think each time a new format was created it solved a different problem, but didn't think much about the previous problem being solved. I guarantee a few hours of putting some people's heads together is all we'd need to have a unified format.

Last, I don't think there's enough definition around the "actions" or forms in any of these formats. They all seem to focus on API orientation and navigation more than data managment. This is ok since we have API docs of course (and this format is by no means a replacment for documentation), but I think we can do better.

They all seem to have opened the door or perhaps have their hand on the knob...But they aren't walking through. They should. If you're providing insight on "what" to send back to the API (and under which method), then there should be some context around "how" it needs to look when it gets there. Especially if you want to dictate validation from a single point that can be easily updated without the need for client application updates. My reasoning here is because doing so is a win win win scenario:

Win #1: The API is hit less frequently which allows it to serve more clients without the need to scale so you're winning on costs too in the long run (win 1.5), but this means more stability for your API.

Win #2: Again, less maintenance. Faster development cycles. You do not need to update the client applications if they obey the rules coming from the JSON response. How many client applications might you have? This could be a major improvement on the maintenence considerations of your application and API.

Win #3: It's faster for the end user! I know we live in a broadband world, but if you think about it in terms of mobile apps or even complex JavaScript browser apps, your application will be more reponsive if it doesn't need to make that round trip request to validate data (as often). Obviously this is not intended to be a guarantee so you should always keep server-side data validation in place, but the more you can do up-front, the more responsive things will be. Better user experience. More importantly you now know any front-end (JavaScript or otherwise) validation is in sync with your back-end validation because you aren't redefining rules in JavaScript or something on the front-end. You're just reading them. Go look at some jQuery form validation plugins and what people have done in the past, all the duplicate work, and you'll quickly appreciate this direction from the API.

My Combinations

So what would I combine? Well, I like the underscore notation. I think while there may be fields in one's data source with underscores (think MongoDB's _id for example off the top of my head - and I personally use them in MongoDB to denote relationships), it is still good to use for visual distinction for an engineer. Especially when these fields are top level items in the JSON response. An engineer can very quickly see that it's part of the Hypermedia convention and isn't something coming from the data source (or perhaps it is - but they'll easily know when).

Separation of Data

This brings me to hierarchy. It's super important. Mixing data from the data source with additional meta data is a bad idea. Especially the common term "links" ... Of course there could be a "links" key from the data source with all sorts of values and meaning. If one of the goals of Hypermedia JSON is to present the engineer with an easy to understand (and navigatable) response, then I think any confusion would put the format at risk of failing to meet that goal. In some cases, it may even become impossible to use the format with the data set unless aliases were used for field names.

While a complete data dump may be an unfavorable design practice, I still don't believe the API response and the data source should be that far apart. While the end-user engineer may not care (or know any better), the internal engineers who are developing the system should also not need to be bothered with any unnecssary field mapping or aliasing. What got moved where for this particular API and nothing else? What do they call this field in this particular response? etc. It introduces a maintenance problem. Anytime there's confusion and additional maintenance time, business goals and costs suffer. No matter how awesome an engineer may believe something to be, there are real world factors and goals and budgets that must be cosidered.

So keep meta data and application data organizationally separate and distinct. It's extremely important. This is why most formats separate out "links" in the first place. However, if you see them embedded within each item, I still think it's a problem (unless each item is an object with a "data" key and a "links" key as with Collection+JSON). I would prefer instead, routing rules like what HAL suggests.

So I would provide a ``_data``` wrapper for the items coming back from the data source. Essentially like "items" in Collection+JSON. I would not go so far as to show relationships in the data like Siren or HAL. I think this assumes too much and not every application will have the same needs or schema. I don't think we can ever replace the need for API documentation or a general understanding of the application's data. Likewise, APIs don't point guns to people's heads saying you must use every bit of data I give you! =)

Actions & Forms

I think Siren put the most thought into forms or actions. There is no confusion about the HTTP method that the API will accept and the response and this is nice because it means your API is truly more navigatable than most others. Think about the time you'll be saving not referencing up the API docs for example. This leads to faster and easier development. Plus, your client application can be a bit more dynamic and take instruction so that you may end up with less maintenance.

So I would take the way Siren handles actions and replace Collection+JSON's "template" section which is a little confusing anyway. When we see "template" we don't really think about forms and actions. Plus, there's more detail within the action items.

However, I would add to this even more (of course optional) information for validation rules. I think Siren hits the nail on the head here in terms of providing form element types and default values...But I would also add optional placeholder keys. Though the validation rules I think are super important. These can even come with messages. If all this comes from the API then maintenance is easy on the server side and again there's that win win win situation here for user experience and API health.

I would also add an _errors key to the forms/actions section. All forms would be given a name and this object would then have named forms with their errors so that a client application knew exactly how to display and handle the errors. Collection+JSON introduces errors, but they seem to be unrelated to the "template" section.

HTTP Errors

Yes, this is what HTTP headers are for. However, I believe additional information may be desired by certain APIs. Anything outside of HTTP response header standards, etc. I also don't believe the headers are an appropriate place for lengthy messages. I think HTTP status codes are great, but they are short codes in the first place for a reason. I would include an _http key with an object value that contained additional error details, similar to Collection+JSON and perhaps even more depending on the needs of the application. This provides better flexibility.

Messages

One thing you don't see a lot of here are messages. "Flash" messages are quite common in applications to help notify the user that something saved properly or there was an error, etc. Of course with form validation we also have the need for messages. They should all be strings and any internationalization considerations should be taken care of in the API URL. /en/posts or /es/posts for example. While one might consider putting translations in the response itself, I think this would unecessarily increase the payload size and most client applications would have no need to display more than one language at a time.

Meta

I would add a _meta key with values for extra things like pagination, response time, flash message, etc. If you're dealing with a listing of items, this is a great place for a total count, sorting options, pagination, etc. It's also nice to get a response time from an API. This should always be held separate from the other data.

Links

Again, I would keep _links separate from the data. This is data added on top of the application data that has more to do with the API than it does the actual application. So there's absolutely no reason to mix it in with the data. Also, I think it's ok to use routing logic here like HAL does. However, HAL mixes in "_links" as if it was data coming straight from the data source along with everything else. I believe to avoid conflicts and to stay better organized, it should just be moved a little bit. Simple change and Collection+JSON does this.

Versioning

I think we should follow semantic versioning in our APIs. Maybe the level of detail is a bit silly at times, but loosely we should follow it. I don't believe in the "v1" we commonly see and I think I'm not alone in this. If you look at Twitter's API, you'll notice they don't use "v" in their version number.

The API version number is contained in the API URL and it is the first item. I agree with this positioning and format. While we never see a real need for "patch" I think "major" and "minor" are more common enough. I think the version number should be kept out of the response as well. Though it wouldn't hurt to optionally add it to the meta data, it does belong in the URL.

Locale/Internationalization

Locale also belongs in the URL. One should not POST arguments for the language. It should be set (preferable immediately following API version number) in the URL. This could even be optional with a default to a specific language. If the API does not support internationalization, then of course we wouldn't expect to see it.

unified-format.go

// Inspired by a few hypermedia formats, this should allow for HAL compatible JSON. However, it deals with more than resources and links.
// It puts more structure around two way communication whereas HAL just tells you what resources are available and where to find them....
// This unified format takes the concerns from formats like Siren and adds them on top of HAL. Yes, with some slight modifications all around.

// This HAL inspired resource includes "_links" and "_embedded" but also adds "_data", "_meta", and "_forms" for enhancements and keeping things tidy.
// Additionally, there's "_curies" which pulls HAL's "curies" out from "_links" and up to its parent. Why? Mainly because "curies" isn't a valid IANA link relation.
// Another reason for breaking it out of "_links" is that CURIEs can be used elsewhere. For example: <html xmlns:news="..."> etc. See: http://www.w3.org/TR/curie
// So now each resource defines the CURIEs it will use. This removes a limitation by HAL.
type HypermediaResource struct {
	Meta     HypermediaMeta                `json:"_meta"`
	Links    map[string]HypermediaLink     `json:"_links,omitempty"`
	Curies   map[string]HypermediaCurie    `json:"_curies,omitempty"`
	Data     map[string]interface{}        `json:"_data,omitempty"`
	Embedded map[string]HypermediaResource `json:"_embedded,omitempty"`
	Forms    map[string]HypermediaForm     `json:"_forms,omitempty"`
}

// The Meta structure helps with common (but optional) info useful for working with data (pagination) and handling errors.
type HypermediaMeta struct {
	Success      bool              `json:"success"`
	Message      string            `json:"message"`
	ResponseTime float32           `json:"responseTime"`
}

// HAL compatible link structure plain and simple (somewhat compatible with http://tools.ietf.org/html/rfc5988).
// NOTE: in HAL format, links can be an array with aliases - our format has no such support, but this doens't break HAL compatibility.
// Why not support it? Because that changes from {} to [] and I don't want to change data types. If it's common enough then maybe always array/slice.
// Also, each "_links" key name using this struct should be one of: http://www.iana.org/assignments/link-relations/link-relations.xhtml unless using CURIEs.
type HypermediaLink struct {
	Href        string `json:"href"`
	Type        string `json:"type,omitempty"`
	Deprecation string `json:"deprecation,omitempty"`
	Name        string `json:"name,omitempty"`
	Profile     string `json:"profile,omitempty"`
	Title       string `json:"title,omitempty"`
	Hreflang    string `json:"hreflang,omitempty"`
}

// Defines a CURIE
type HypermediaCurie struct {
	Name      string `json:"name,omitempty"`
	Href      string `json:"href,omitempty"`
	Templated bool   `json:"templated,omitempty"`
}

// Form structure defines attributes that match HTML (whereas "links" and "resources" do not).
// Any attribute not found in HTML should be prefixed with an underscore (for example, "_fields").
// NOTE: Not in the HAL spec, this extends the hypermedia format to allow for additional CRUD support.
type HypermediaForm struct {
	Name          string                         `json:"name,omitempty"`
	Method        string                         `json:"method,omitempty"`
	Enctype       string                         `json:"enctype"`
	AcceptCharset string                         `json:"accept-charset,omitempty"`
	Target        string                         `json:"target,omitempty"`
	Action        string                         `json:"action,omitempty"`
	Autocomplete  bool                           `json:"autocomplete,omitempty"`
	Fields        map[string]HypermediaFormField `json:"_fields,omitempty"`
}

// Defines properties for a field (HTML attributes) as well as holds the "_errors" and validation "_rules" for that field.
// "_rules" have key names that map to HypermediaFormField.Name, like { "fieldName": HypermediaFormFieldRule } and the rules themself are named.
// "_errors" have key names that also map to HypermediaFormField.Name
type HypermediaFormField struct {
	Name         string                              `json:"name,omitempty"`
	Value        string                              `json:"value,omitempty"`
	Type         string                              `json:"type,omitempty"`
	Src          string                              `json:"src,omitempty"`
	Checked      bool                                `json:"checked,omitempty"`
	Disabled     bool                                `json:"disabled,omitempty"`
	ReadOnly     bool                                `json:"readonly,omitempty"`
	Required     bool                                `json:"required,omitempty"`
	Autocomplete bool                                `json:"autocomplete,omitempty"`
	Tabindex     int                                 `json:"tabindex,omitempty"`
	Multiple     bool                                `json:"multiple,omitempty"`
	Accept       string                              `json:"accept,omitempty"`
	Errors       map[string]HypermediaFormFieldError `json:"_errors,omitempty"`
	Rules        map[string]HypermediaFormFieldRule  `json:"_rules,omitempty"`
}

// Error messages from validation failures (optional) "name" is the HypermediaFormFieldRule.Name in this case and "message" is returned on failure.
type HypermediaFormFieldError struct {
	Name    string `json:"name"`
	Failed  bool   `json:"name"`
	Message string `json:"message,omitempty"`
}

// Validation rules! Easily nested into "_rules" on "_fields" (optional).
type HypermediaFormFieldRule struct {
	Name        string `json:"name"`
	Description string `json:"description,omitempty"`
	// Careful, regular expressions need to be escaped properly, but it can provide for front-end validation before data is submitted (of course not to be trusted)
	Pattern string `json:"pattern"`
	// This isn't for JSON, it's for Go (because we dont't trust front-end validation)
	Function func(value string) (fail bool, message string)
}

Yet another format, http://json-schema.org/latest/json-schema-hypermedia.html ... Pretty similar to the others. Introduces a "mediaType" for the links which can be nice. Also uses "encType" for forms which is nice. However this format has forms under the "links" section. The data format under "properties" is nice to include types, but perhaps a little too detailed. Data types are usually more important for form handling and so I think it's fine to use {fieldName: value} in the data section. Plus, it's a little confusing to see a title that's different in the top level of the JSON response and then a different title under the "properties" section. What's the example here showing? A related item? The same? Why different titles?

In general, all of these formats really share some similarities...But I feel they are all trying to solve specific problems instead of general problems. This makes certain formats better for certain applications and, again, I don't believe there's any reason why a more general format can't serve everyone's needs.

Take a look at Mason: https://github.com/JornWildt/Mason - it is an attempt to combine some of the good stuff from the various formats you mention. Similar to what you suggest yourself. I wrote a piece on my design goals here: http://soabits.blogspot.com/2014/02/implementing-hypermedia-apis-and-rest.html

Mason avoids any UI details completely (as any attempt to do this will be way too complex and end up implementing HTML 5 again), uses JSON for data in actions and relies on JSON-Schema for optional client side validation. It also adds error information and informational elements directed at client developers.

We are probably going to disagree on where the data is stored. Mason merges hypermedia elements with data - as opposed to embedding it in @DaTa or (_data if you wish). Take a look at the attachment array in the example at http://soabits.blogspot.com/2014/02/implementing-hypermedia-apis-and-rest.html - how would you "escape" the data in order to add hypermedia if you cannot merge hypermedia directly in the data? For me hypermedia elements should be first class citizens of API data.

I do like Mason and the use of @ ... I think it's much better than my underscore notation. My only fear is RDF confusion (JSON-LD http://www.w3.org/TR/json-ld)...But I'm trying to think when there might be a potential collision there and I can't, as of right now, find one. The more I go to use RDF perhaps the more I'll be put in that position. Of course @data at the root of the response would not be interpreted as RDF and anything within it could safely be parsed as RDF or whatever else. So I guess there's no problem...Unless someone blindly parsed the entire response as RDF.

Also noteworthy: https://rawgit.com/mamund/media-types/master/uber-hypermedia.html ... One point of "Uber" is to account for there potentially not being any "HTTP" which my above format also handles since _http is broken out.

Links are within the data (a faux pas in my mind) like many other formats...Contextual links being within the data is fine, but when talking about a class of objects -- it's silly to read the data to pick up a pattern and then wasteful to repeat what otherwise could be a simple regex in someone's code. The only benefit to such repetition is that someone browsing the JSON responses in a web browser could click around the RESTful API.

BUT. IF we are truly saying that the API may not come via HTTP -- then you aren't clicking around anyway. So get rid of self documenting links within the data. There's really no need.

After playing around with my format, I'm also thinking about ditching _actions OR keep it (optional) and stick that plus _links under _http since they are HTTP related items. Then another protocol can be at the same level as _http with link children that make sense for the protocol.

Nice summary.. You should also have a look at Hydra which plays well with JSON-LD and other RDF serialization formats. Would love to hear your thoughts @tmaiaroto.

I think JSON-LD is pretty important stuff for the future. The idea of semantic data (RDF) and a giant graph is a slow moving one - but I think one that will shape the future of the internet as we all become connected. Right now we're on all these little islands. The internet is loosely connected and what's coming in the future will make things like Facebook and OpenGraph and Twitter etc. all look like children's toys. In fact, if they don't adapt, they'll simply not be a part of the future web (rule #1 of the internet - nothing lasts forever).

...And for the rest of us. Bloggers and creators, we'll be using things like JSON-LD and microformats and RDF ...and. Hypermedia APIs to programmatically speak to and connect with one another.

I did see Hydra and I'll revisit it and add it to the above comparisons. Thanks.

Coming back a bit later here, I'm working with HAL again and like it for the most part because it fits into what I'm doing and what I need can remain compatible with HAL. My "unified" format has changed a bit (simpler) and I will re-post soon. Actually, I'm gonna re-post it as Go structs because that's helped with design.

In response to your post:

Constructive enlightenment incoming…

1. Your unified format now requires a specialized client to use it. It’s not typical JSON objects - it’s JSON objects nested inside of a structure. When you PUT/PATCH an object - do you patch it “in the raw” or does it also come back in this structure?
2. Your attempt to avoid _links per item in your _data section causes a lot of problems. In my API I routinely have multiple levels of arrays of embedded resources and I rely on their individualized links heavily. This also decreases human readability (not that important) — but you’re right it may reduce overall payload size (but at what cost?).
3. You don’t mention how to embed resources so I’m assuming that’s either not supported or will be as naively implemented as _links
4. Your _meta, _http, and _actions sections aren’t universally useful enough to be part of a spec (imo) — and I don’t see why you couldn’t just bolt those on to an existing spec like HAL
5. For your “Route Format” I highly recommend URI Templates
6. Some of your example link relations are invalid. They should be one of the IANA registered rels or urls (presumably to documentation describing the rel). For instance “skip” - skip what? Many HAL based API’s use CURIEs for this.

no curies can’t be _curies just because some people get confused.

First, if there's any JSON formatting errors it was an oversight (these are thoughts not actual code). The intention is typical JSON of course.

I was thinking about #2 in some of the alterations I'm making now and decided to only support one level of "embedded resources" for now. With the above format I was not even using embedded resources (to answer #3). My goal was basic CRUD. Again, I'm designing for a different/broader case now. My thinking on keeping one level of embedded resources is that anything deeper and more complex would require another HTTP request. A single level allows for navigation. My fear with having a very deep response is performance and confusion. Everywhere performance. Parsing. Generating. Response payload size. This may change in the future though.
#4 I'm ditching _http ... I'm ditching _actions (consolidating to _links) but keeping _meta because many APIs do return things like "response time" etc. I believe there is a universal use case here, but it's completely optional of course. I want good support and good flexibility. So having "_meta" as a misc. bucket for things outside the concern of resources and links made sense to me.
#5 Yup, I need to look into the best way to do that (I was jotting down ideas borrowed from some routers in various frameworks). Thanks for the link. That's super awesome.
#6 Makes sense. I was not using CURIEs before, but am going to now. As for IANA that's fine it's supported here. People can plug those in as desired. I don't care that "skip" isn't in IANA. It doesn't make the format any less valid. It may make more sense to use in a CURIE and that's cool.

Moreover, "curies" isn't on that IANA list either. So again... _curies and not curies fits better in my opinion.

Thanks for the notes. Super helpful and much appreciated.
I do still wonder why curies can't be _curies ... Given it's more for HAL compatibility I may stick without the underscore, but I do feel it creates a minor inconsistency with the format I'd like to use.

Eh, I found a way to engineer this that I like better and it accommodates an unlimited nesting better. So stay tuned. I think I'll allow _embedded to an infinite depth. Though I still have my concerns about that being abused and creating performance problems for parses and unnecessarily large response payloads. It's a delicate balance, but I regard flexibility quite highly so I'm thinking about letting up on that.

I put up a refactored version of the unification of formats like HAL and Siren. I looked over Mason and Hydra too for some ideas. If Mason is a superset of HAL then it should also play nicely with this unified format too. I think all of the "class" level stuff in some of the other formats can be handled by "_meta" for the most part. There may be the need for additional keys in the resource. We'll have to see.

The new file I added here is a representation in Go. I'm doing this because I find it easier to architect for starters, but also I'm working on a new project in Go. So it's right within reach for me. Plus Go is great for JSON.

However, this leaves things a little awkward to read because structs are broken up. You'll have to see which ones go where for now. I will get some JSON examples in the future.

Strongly considering changing _meta to _state.
Also thinking about dropping some direct HTML attribute mappings within _forms (potentially renaming _forms even) in order to make the API compatible with more than just web applications (ie. consider mobile apps).

imo, you should create a new media type because you have a specific problem to solve that isn't serviced appropriately by any of the existing types - not because you feel like the existing ones need to be either consolidated or "fixed up".

Just to add my work to the list, I have created a media type that aims to address all of the areas other media types can handle called Verbose:

http://verbose.readthedocs.org/en/latest/index.html

I created it with the purpose of having a model for converting to and from other media types. While that is the main goal, it can be used as a media type, though it's in development right now, so proceed with caution!

@mikekelly puts it perfectly - there's never going to be a one and only standard. so it doesn't really matter. i mean, that's the moral of the internet i suppose.

There a TON of hypermedia formats and people are mentioning more when commenting here. That's awesome for research. Here's another great resource: http://ruben.verborgh.org/phd/hypermedia/

It goes on to discuss "read" and "write" and I feel that's important. Not all hypermedia formats address this, but some do. I think (automatically) understanding how to work with the described resources and how to modify state is well warranted.

So that's why I'm trying to do based on the collective thoughts of others. Just to be clear. I'm not trying so much to "change" any of the formats. Though I figured I'd present some thoughts given the RFC nature of everything. Anything I find to be more in a "finalized" state, I'll simply hold off on commenting. Not a big deal. I still want comments though. I might even go so far (one day, right now I'm super busy) catalogue everything in a better system than a gist. It's really nice to compare and contrast things that have worked for others. For everyone. For every new project. Because I'm in the camp of believing that there's no one size fits all solution and over time, things change. Let us not forget many of these "standards" were written years ago which is ancient for the web. How I created web sites in the 90's is not at all how I build them today.
(edited to keep comments shorter)

@smizell I really find the use of "affordances" in Verbose fun. Good Fielding reference. I like how "properties" of state are separated too. I'm strongly thinking about keeping "form" type stuff outside of "links" or in your case "affordances" ... Did you find any problems with having that information inside "affordances"? Given the word, I assume the goal was to eliminate any confusion with "web links".

I started out separating the idea of links and actions/forms myself. The difficult part is that given a resource, there are several actions that can be done to that particular resource. For example, if you use the "edit" link relation from AtomPub, it says, "the href IRI can be used to retrieve, update, and delete the Resource represented by that Entry." So in that one link relation, which most media types would consider a "link," you actually have one "link" and two "actions" that you can do. So is that a link or an action? Maybe both?

For me, an affordance says here's a resource, and here are the things you can do with that resource. Specifying one action (like a form for Edit Customer) seems to be limiting, so with an affordance in Verbose, you can specify what all can be done to that URI.

Also, I'm not 100% on sticking with "affordances," but it does provide a way to convey both the idea of a link and an action/form. And it's a fun word!

That's a good point, hmm... With the struct I defined in Go, I have a map[string]HypermediaForm so in other words, each form comes with a key. That key can define a "_link" (affordance)... So I'd have to make that an array/slice of forms then to match what Verbose may support. I see how you aren't even saying "different forms" - it's more like different request methods and request types. That's interesting. More things to think about =)