Playing with Orbit.js and Ember.js

This presenation is an exploration of rolling your own data persistence for and Ember.js Application

During this discovery I attempt to roll my own data solution. I've selected to use alpha software, Orbit.js. So not also choosing other alpha software at the some time. Not covering es6, broccoli, ember-cli etc.

The exploratory app is based on converting my blog app (uses REST) to use Web sockets with a Node.js backend. I am using some build tools brunch.io also borrowing from tapas-with-ember. The modules in the source code are CommonJS modules.

What Options Exist for Persistance

Ember.js Persistence Foundation
- Emphasis on correctness and stability.
- Feature parity with Ember-Data with an easy migration path.
Ember Model
- A lightweight model library for Ember.js
- Provide primitives on top of $.ajax
Ember Data (Beta)
Roll your own, (Alpha)

I'm not going to take any time to compare but may callout some contrasts to choice I'm making as I work on a DIY data persistence solution.

What I'm currently exploring

Web Sockets (TCP)
- Potentially closer to realtime interations compared to REST
- Ships with All modern (popular) browsers, see caniuse.com/#search=Socket
- Gets around CORS
Smaller payloads (partial representation of a resource)
- JSONPatch, jsonpatch.com
Connecting multiple storage adapters
- Memory <-> localStorage <-> Web Socket (remote db)
High availability (distributed computing system)
- Weak consistency with higher availability
- Choosing liveness (eventually there) over Safety (always right)

JSONPatch

See: jsonpatch.com ...

Simple example

The original document:

{
  "baz": "qux",
  "foo": "bar"
}

The patch:

[
  { "op": "replace", "path": "/baz", "value": "boo" },
  { "op": "add", "path": "/hello, "value": ["world"] },
  { "op": "remove, "path": "/foo}
]

The result:

{
   "baz": "boo",
   "hello": ["world"]
}

Operations

Add, Remove, Replace, Copy, Copy, Test

Add

{"op": "add", "path": "/biscuits/1", "value": {"name": "Ginger Nut"}}

Adds a value to an object or inserts it into an array. In the case of an array the value is inserted before the given index. The - character can be used instead of an index to insert at the end of an array.

Remove

{"op": "remove", "path": "/biscuits"}

Removes a value from an object or array.

Replace

{"op": "replace", "path": "/biscuits/0/name", "value": "Chocolate Digestive"}

Replaces a value, equivalent to a “remove” followed by an “add”.

Libraries:

What is Orbit.js ?

Let's take a look at the ember-orbit-example demo app.

Watch Dan give an intoduction to Orbit.js. (YouTube video from the Jan '14 Boston Ember Meetup)

How it works

Orbit requires that every data source support one or more common interfaces. These interfaces define how data can be both accessed and transformed.
The methods for accessing and transforming data return promises.
Multiple data sources can be involved in a single action

Orbit.js uses JSON / JSONPatch by default.

Sends partial data
Uses multiple stores

Orbit interfaces:

Requestable
- for managing requests for data via methods such as find, create, update and destroy
Transformable
- Keep data sources in sync through low level transformations (using JSON PATCH spec)
- transform is the method your data source (prototype) object needs to implement

Requestable

Events associated with an action:

assistFind, rescueFind, didFind, didNotFind

Transformable

A single method, transform, which can be used to change the contents of a source.

{op: 'add', path: 'planet/1', value: {__id: 1, name: 'Jupiter', classification: 'gas giant'}
{op: 'replace', path: 'planet/1/name', value: 'Earth'}
{op: 'remove', path: 'planet/1'}

Orbit Common Library

Contains a set of compatible data sources, currently including: an in-memory cache, a local storage source, and a source for accessing JSON API compliant APIs with AJAX.

You can define your own data sources that will work with Orbit as long as they support Orbit's interfaces.

Example:

// Create data sources with a common schema
var schema = new OC.Schema({
  idField: '__id',
  models: {
    planet: {
      attributes: {
        name: {type: 'string'},
        classification: {type: 'string'}
      }
    }
  }
});
var memorySource = new OC.MemorySource(schema);
var restSource = new OC.JSONAPISource(schema);
var localSource = new OC.LocalStorageSource(schema);

// Connect MemorySource -> LocalStorageSource (using the default blocking strategy)
var memToLocalConnector = new Orbit.TransformConnector(memorySource, localSource);

// Connect MemorySource <-> JSONAPISource (using the default blocking strategy)
var memToRestConnector = new Orbit.TransformConnector(memorySource, restSource);
var restToMemConnector = new Orbit.TransformConnector(restSource, memorySource);

TransformConnector

A TransformConnector watches a transformable source and propagates any transforms to a transformable target.

Each connector is "one way", so bi-directional synchronization between sources requires the creation of two connectors.

RequestConnector

A RequestConnector observes requests made to a primary source and allows a secondary source to either "assist" or "rescue" those requests.

The mode of a RequestConnector can be either "rescue" or "assist" ("rescue" is the default).

Document

Document is a complete implementation of the JSON PATCH spec detailed in RFC 6902.

It can be manipulated via a transform method that accepts an operation, or with methods add, remove, replace, move, copy and test.

Data at a particular path can be retrieved from a Document with retrieve().

Code Walkthrough:

Topics:

How to integrate Orbit.js with an Ember.js app
Register singleton services
queryFactory() <- builds up a query
Injected storage object
- Instead of this.store.find -> this.dataStore.find('modelName', query)
Creating a storeMeta as meta on a route. Pull out metadata on a route.
OC.MemorySource already exists. Decorating it. More of a strategy.
MemorySource tries to resolve request, if that (promise) fails it sends the request to the SocketSource to find.

Orbit.js custom Store for communication over a Web Socket

Show code examples for socket source, and socket service

MemoryStore + SocketStore + Ember.js

Show code examples for setting up strategy for request / transform operations

Discoveries (so far...)

Why am I using this instead of ember data?
I'm building a blog -> content management system... web sockets + orbit.js + ember.js would be really cool to use as a content management system instead of WordPress.
When editing content I can send the property change instead of the complete resource.
Speak JSONPatch over TCP is nice

Goals for continuing development

Dump the db resources into the client as JSON and allow the Socket communication more info.
Load from localStorage. Check timestamps/etags and send data operations if updated upstream.

pixelhandler/playing-with-orbit-js-and-ember-js.md