Skip to content

Instantly share code, notes, and snippets.

@jhliberty
Forked from richmolj/jsonapi-overview.md
Created May 1, 2017 18:05
Show Gist options
  • Save jhliberty/0f88d1daaf1f5622b90323a18b12e76b to your computer and use it in GitHub Desktop.
Save jhliberty/0f88d1daaf1f5622b90323a18b12e76b to your computer and use it in GitHub Desktop.

Table of contents

Concepts

A request comes in to the server.

Underneath-the-hood, that request gets normalized into a Query object.

A Resource configures how to resolve a Query when given a base Scope. A scope can be any object - from an ActiveRecord::Relation to an HTTP client to an empty hash.

The user may request a resource and its relationships in one API call. This is called sideloading. Resources are re-used for sideloading, sideloading simply defines the relationships between Resources.

Once we've resolved the query into objects, we want to serialize those objects as JSON (which may include formatting/display logic). We use jsonapi-rb for this; the API is very similar to active_model_serializers.

Reads

Basic Setup (Master)

We'll be creating an API for an Employee Directory. An Employee has many positions (one of which is the current position), and a Position belongs to a Department.

Let's start with a basic foundation: an index endpoint (list multiple entities) and a show (single entity) endpoint for an Employee model.

Code:

class EmployeesController < ApplicationController
  jsonapi resource: EmployeeResource

  def index
    render_jsonapi(Employee.all)
  end
  
  def show
    scope = jsonapi_scope(Employee.where(id: params[:id]))
    render_jsonapi(scope.resolve.first, scope: false)
  end
end
class EmployeeResource < ApplicationResource
  type :employees
end
class SerializableEmployee < JSONAPI::Serializable::Resource
  type :employees

  attribute :first_name
  attribute :last_name
  attribute :age
end

Tests:

RSpec.describe 'v1/employees#index', type: :request do
  let!(:employee1) { create(:employee) }
  let!(:employee2) { create(:employee) }

  it 'lists employees' do
    get '/api/v1/employees'
    expect(json_ids(true)).to eq([employee1.id, employee2.id])
    assert_payload(:employee, employee1, json_items[0])
  end
end
RSpec.describe 'v1/employees#show', type: :request do
  let!(:employee) { create(:employee) }

  it 'returns relevant employee' do
    get "/api/v1/employees/#{employee.id}"
    assert_payload(:employee, employee, json_item)
  end
end

A note on testing: these are full-stack request specs. We seed the database using factory_girl, randomizing data with faker, then assert on the resulting JSON using spec helpers.

You won't have to write all the tests you see here, some are simply for demonstrating the functionality.

Filtering

One line of code allows simple WHERE clauses. If the user tried to filter on something not whitelisted here, an error would be raised.

https://github.com/jsonapi-suite/employee_directory/compare/master...step_1_add_filter

Custom Filtering

Sometimes WHERE clauses are more complex, such as prefix queries. Here we'll query all employees whose age is greater than or equal to a given number.

https://github.com/jsonapi-suite/employee_directory/compare/step_1_add_filter...step_2_add_custom_filter

Sorting

Sorting comes for free, but here's a test for it. We would decide as a team if we actually need to write a spec here, or if it's considered tested within the libraries.

https://github.com/jsonapi-suite/employee_directory/compare/step_2_add_custom_filter...step_3_basic_sorting

Custom Sorting

Sometimes we need more than a simple ORDER BY clause, for example maybe we need to join on another table. In this example, we switch from Postgres's default case-sensitive query to a case in-sensitive one...but only for the first_name field.

https://github.com/jsonapi-suite/employee_directory/compare/step_3_basic_sorting...step_4_custom_sorting

Pagination

Pagination also comes for free, so once again we'll have to decide if writing a spec like this is worth the bother.

https://github.com/jsonapi-suite/employee_directory/compare/step_4_custom_sorting...step_5_pagination

Custom Pagination

By default we use the Kaminari library for pagination. This shows how we could instead sub-out Kaminari and replace it with will_paginate

https://github.com/jsonapi-suite/employee_directory/compare/step_5_pagination...step_6_custom_pagination

Statistics

For default statistics, (count, sum, average, maximum and minimum), simply specify the field and statistic.

https://github.com/jsonapi-suite/employee_directory/compare/step_6_custom_pagination...step_7_stats

Custom Statistics

Here we add a median statistic to show non-standard custom statistic usage.

https://github.com/jsonapi-suite/employee_directory/compare/step_7_stats...step_8_custom_stats

Custom Serialization

Let's say we wanted the employee's age to serialize Thirty-Two instead of 32 in JSON. Here we use a library to get the friendly-word doppleganger, and change the test to recognize this custom logic.

https://github.com/jsonapi-suite/employee_directory/compare/master...custom-serialization

Has-Many Association

Get employees and their positions in one call.

https://github.com/jsonapi-suite/employee_directory/compare/master...step_9_has_many

Belongs-To Association

Get employees, positions, and the department for those positions in one call:

https://github.com/jsonapi-suite/employee_directory/compare/step_9_has_many...step_10_belongs_to

Many to Many

In this example an Employee has many Teams and a Team has many Employees.

https://github.com/jsonapi-suite/employee_directory/compare/step_13_error_handling...many-to-many

Resource Re-Use

In prior steps we created PositionResource and DepartmentResource. These objects may have custom sort logic, filter whitelists, etc - this configuration can be re-used if we need to add /api/v1/positions and /api/v1/departments endpoints.

https://github.com/jsonapi-suite/employee_directory/compare/step_10_belongs_to...step_11_resource_reuse

Filter/Sort/Paginate Associations

This comes for free. As long as the associated Resource knows how to do something, we can re-use that logic.

https://github.com/jsonapi-suite/employee_directory/compare/step_11_resource_reuse...step_12_fsp_associations

Error Handling

In this example we add global error handling, so any random error will return a JSONAPI-compatible error response. Then we customize that response for a specific scenario (the requested employee does not exist).

https://github.com/jsonapi-suite/employee_directory/compare/step_12_fsp_associations...step_13_error_handling

Writes

Basic Create

Basic example without validations or strong parameters.

https://github.com/jsonapi-suite/employee_directory/compare/bump_gemfile_for_writes...step_14_create

Validations

Validations are basic, vanilla Rails code. When there is a validation error, we return a jsonapi-compatible error respone.

https://github.com/jsonapi-suite/employee_directory/compare/step_14_create...step_15_validations

Strong Resources

The biggest problem with strong_parameters is that we might want to create an employee from the /employees endpoint, or we might want to create a position with an employee at the same time from /positions. Maintaining the same strong parameter hash across a number of places is difficult.

Instead we use strong_resources to define the parameter template once, and re-use. This has the added benefit of being built on top of stronger_parameters, which gives us type checking and coercion.

Note: strong_resources requires Rails.

https://github.com/jsonapi-suite/employee_directory/compare/step_15_validations...step_16_strong_resources

Basic Update

Looks very similar to create.

https://github.com/jsonapi-suite/employee_directory/compare/step_16_strong_resources...step_17_basic_update

Basic Destroy

More or less basic Rails.

https://github.com/jsonapi-suite/employee_directory/compare/step_17_basic_update...step_18_basic_destroy

Customizing Persistence

So far we've shown ActiveRecord. What if we wanted to use a different ORM, or ElasticSearch? What if we wanted 'side effects' such as "send a confirmation email after creating the user"?

This code shows how to customize create/update/destroy. In this example we're simply logging the action, but you could do whatever you want here as long as you return an instance of the object. Just like with reads, if any of this code becomes duplicative across Resource objects you could move it into a common Adapter.

https://github.com/jsonapi-suite/employee_directory/compare/step_18_basic_destroy...step_19_custom_persistence

Association Writes

Nested Creates

Think Rails' accepts_nested_attributes_for, but not coupled to Rails or ActiveRecord. Here we create an Employee, a Position for the employee, and a Department for the position in one call. This is helpful when dealing with nested forms!

Once again, note how our strong_resources can be shared across controllers.

https://github.com/jsonapi-suite/employee_directory/compare/step_19_custom_persistence...step_20_association_create

Nested Updates

We got this for free, here's a spec!

https://github.com/jsonapi-suite/employee_directory/compare/step_20_association_create...step_21_association_update

Nested Destroys

We get this for free, though we have to explicitly tell strong_resources that destroys are allowed from this endpoint.

Note destroy will do two things: delete the object, and make the foreign key on the corresponding child in the payload null.

https://github.com/jsonapi-suite/employee_directory/compare/step_21_association_update...step_22_association_destroy

Disassociations

destroy actually deletes objects, what if we want to simply disassociate the objects by making the foreign key null? We get this for free, too.

https://github.com/jsonapi-suite/employee_directory/compare/step_22_association_destroy...step_23_disassociation

Non-ActiveRecord

Let's say the departments come from a service call. Here's the change to the /departments endpoint:

class DepartmentsController < ApplicationResource
  jsonapi resource: DepartmentResource
  
  def index
-    render_jsonapi(Department.all)
+    render_jsonapi({})
  end
end
class DepartmentResource < ApplicationResource
  type :departments
+  use_adapter JsonapiCompliable::Adapters::Null  
  
+  def resolve(scope)
+    Department.where(scope)
+  end
end

Department.where is our contract for resolving the scope. The underlying Department code could use an HTTP client, alternate datastore, what-have-you.

Let's also change our code for sideloading departments at /api/v1/employees?include=departments:

class PositionResource < ApplicationResource
  type :positions
  
-   belongs_to :department,
-     scope: -> { Department.all },
-     foreign_key: :department_id,
-     resource: DepartmentResource
+  allow_sideload :department, resource: DepartmentResource do
+    scope do |employees|
+      Department.where(employee_id: employees.map(&:id))
+    end
+
+    assign do |employees, departments|
+      employees.each do |e|
+        e.department = departments.find { |d| d.employee_id == e.id }
+      end
+    end
+  end
end
class Position < ApplicationRecord
-  belongs_to :department, optional: true
+  attr_accessor :department
end

ElasticSearch

Similar to a service call, here's how we might incorporate the elasticsearch trample gem.

class EmployeesController < ApplicationController
  jsonapi resource: EmployeeResource
  
  def index
-    render_jsonapi(Employee.all)
+    render_jsonapi(Search::Employee.new)
  end
end
class EmployeeResource < ApplicationResource
  type :employees
+  use_adapter JsonapiCompliable::Adapters::Trample

+  allow_filter :first_name
+  allow_filter :first_name_prefix do |scope, value|
+    scope.condition(:first_name).starts_with(value)
+  end

+  def resolve(scope)
+    scope.query!
+    scope.results
+  end
end

Client-Side

JSORM Javascript Client

There are number of jsonapi clients in a variety of languages. Here we'll be using jsorm - an ActiveRecord-style ORM that can be used from Node or the browser.

This will fetch an employee with id 123. their last 3 positions where the title starts with 'dev', and the departments for those positions.

First define our models (additional client-side business logic can go in these classes):

class Employee extends Model {
  static jsonapiType: 'people';

  firstName: attr();
  lastName: attr();
  age: attr();
  
  positions: hasMany();
}

class Position extends Model {
  static jsonapiType: 'positions';

  title: attr();
  
  department: belongsTo();
}

class Department extends Model {
  static jsonapiType: 'departments';
  
  name: attr();
}

Fetch the data in one call:

let positionScope = Position.where({ title_prefix: 'dev' }).order({ created_at: 'dsc' });

let scope = Employee.includes({ positions: 'department' }).merge({ positions: positionScope});
scope.find(123).then (response) => {
  let employee = response.data;
  // access data like so in HTML:
  // employee.positions[0].department.name
}

Read the JSORM documentation here

Sample Application

JSORM can be used with the client-side framework of your choice. To give an example of real-world usage, we've created a demo application using Glimmer. Glimmer is super-lightweight (you can learn it in 5 minutes) and provides the bare-bones we need to illustrate JSONAPI and JSORM in action.

Still, we want to demo JSONAPI, not Glimmer. To that end, we've created a base glimmer application that will take care of styling and glimmer-specific helpers.

Finally, this points to a slightly tweaked branch of the server-side API above.

Now let's create our Employee Directory:

demo

Client-Side Datagrid

We'll start by adding our models and populating a simple table:

https://github.com/jsonapi-suite/employee-directory/compare/master...step_1_basic_search

Client-Side Filtering

Now add some first name/last name search filters to the grid:

https://github.com/jsonapi-suite/employee-directory/compare/step_1_basic_search...step_2_add_filtering

Client-Side Pagination

Pretty straightforward: we add pagination to our scope, with some logic to calculate forward/back.

https://github.com/jsonapi-suite/employee-directory/compare/step_2_add_filtering...step_3_add_pagination

Client-Side Stats

Here we'll add a "Total Count" above our grid, and use this value to improve our pagination logic:

https://github.com/jsonapi-suite/employee-directory/compare/step_3_add_pagination...step_4_stats

Client-Side Sorting

https://github.com/jsonapi-suite/employee-directory/compare/step_4_stats...step_5_sorting

Client-Side Nested Create

Let's add a form that will create an Employee, their Positions and associated Departments in one go:

https://github.com/jsonapi-suite/employee-directory/compare/step_5_sorting...step_6_basic_create

Client-Side Nested Update

Let's add some glimmer-binding so that we can click an employee in the grid, and edit that employee in the form:

https://github.com/jsonapi-suite/employee-directory/compare/step_6_basic_create...step_7_update

Client-Side Nested Destroy

Remove employee positions. Since only one position is 'current', we'll do some recalculating as the data changes.

https://github.com/jsonapi-suite/employee-directory/compare/step_7_update...step_8_destroy

Client-Side Validations

Of course, no form is complete without nested, server-backed validations. Here we'll highlight the main fields in red, and also give an example of adding a note explaining the error to the user.

The 'age' field is an exception. If the user submits a string instead of a number, the server will response with a 500. This is to show off our stronger_parameters integration

https://github.com/jsonapi-suite/employee-directory/compare/step_8_destroy...step_9_validations

Documentation

Because all of this is configuration via DSL, we can introspect that DSL and auto-generate documentation. Previously I've used swagger for this, but the UI gets awkward when many filters and relationships are present. Instead, we'll be releasing a custom alternative shortly.

That said, you may disagree with me. On the other side of this spectrum is NodeJS and GraphQL. This is still an improvement over BAS services. If we go this route, I would recommend rewriting the JSONAPI libraries in Node and calling jsonapi_scope to fulfill the resolve function. I suggest the same solution if you like the GraphQL payload but are against NodeJS.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment