Control Your Templates

control-your-templates.md

Control Your Templates!

@mcculloughsean

Actually

Control Your Templates!

Scratch that. This talk is really about...

Useful Abstractions vs Implementation Hiding

@mcculloughsean

Pivot

This talk was supposed to be just about Bulk Hogan and a better way to implement a view engine in Express.js.

But the real story here is about how a nasty bit of implementation hiding made what should have been a simple story about views in Express.js very difficult to explain.

Last night..

I spent 3 hours looking through the Express.js code to explain how the view engine worked.

Afterwards I realized my mental model of how the view system worked was completely wrong.

What's an Abstraction?

Developers create abstractions to help them focus on as few pieces of a problem at one time.

An example in Express

app.get, app.post, etc are abstractions for listening to a HTTP server onRequest event.

Plain HTTP server

http = require 'http'

options =
  host: "127.0.0.1"
  port: 22344

onRequest = (req, res) ->
  req.on "error", onError
  res.on "error", onError

  if req.method is 'GET'
    if req.url is '/'
      body = Array(1024).join("x")
      headers = "Content-Length": "" + body.length

    if req.url is '/cats'
      body = Array(1024).join("cats")
      headers = "Content-Length": "" + body.length

  res.writeHead 200, headers
  res.end body

onError = (err) ->
  console.error err.stack

http.createServer(onRequest).listen options.port, options.host

Express.js

express = require 'express'
app = express.createServer()
app.get "/", (req, res) ->
  res.send Array(1024).join('x')

app.get "/cats", (req, res) ->
  res.send Array(1024).join('x')

app.listen 3000

Express' Routing Abstractions

Both pieces of code do the same thing, but the Express implementation reduces the amount of code you have to write to do the same thing.

Express' Routing Abstractions

The router does not hide the fact that you're responding to a HTTP request or sending strings to a client

Express' Routing Abstractions

Abstractions let us worry about solving the problem we have (e.g. sending 'x' to a browser 1024 times) instead of focusing on all secondary details of getting that job done.

Jobs of Express

• Start up a server listening on a port
• Route requests to some driver code
• Simplify the req and res api
• Mutating req and res objects conditionally (middleware)
• Render HTML from Templates

Only the router code is an abstraction

Reasoning

The Express router allows you to reason about what's happening behind the scenes in the same way the vanilla HTTP server does.

There's more to it, but the general idea of "When the URL and method match a set of qualifiers, run this code" is the same.

The rest is something different

Case Study: Express View Engine

Jobs of a View Engine

• First find the templates on the filesystem
• Read them into the application as a string
• Compile them into a function that can be called with data
• Pass data to the function to return a rendered template
• Nest a view in a layout
• Send the rendered template to the browser via HTTP

Templates

• Layouts
• Views
• Partials

Conventions

• Naming by convention
  • e.g. layout, _partial
• Nested structure
  • /views/page/index
  • /views/page/_partial

How does it work?

View filenames take the form <name>.<engine>, where <engine> is the name of the module that will be required.

From the Express.js 2.x guide

How does it work?

For example the view layout.ejs will tell the view system to require(‘ejs’), the module being loaded must export the method exports.compile(str, options)...

From the Express.js 2.x guide

How does it work?

... and return a Function to comply with Express. To alter this behavior app.register() can be used to map engines to file extensions, so that for example “foo.html” can be rendered by ejs."

From the Express.js 2.x guide

Configuration

# Set Default Engine
app.set 'view engine', 'jade'

# No Layout by Default
app.set 'view options', layout: false

Rendering

# Basic View Rendering
app.get "/", (req, res, next) ->
  # renders /views/index.jade inside /views/layout.jade
  res.render "index",
    title: "hello world!"

# renders /views/special_page.jade with no layout
res.render 'special_page', layout: false

Pretty easy, right?

Things get weird, quick.

How would you render a template to a string?

res.render "index",
  title: 'hello world'
, (err, html) ->
  console.log html

What if you don't have a res object?

Well, you're out of luck. All of the template rendering is bound to a request lifecycle in Express.

Yes, there are ways around this but it's far too much work

Want to use Mustache?

There's no Express.js interface for Hogan, so we need to roll our own:

hogan = require 'hogan.js'
app.set "view engine", "hogan.js"
app.register "hogan.js",
  compile: ->
    t = hogan.compile.apply(hogan, arguments)
    return ->
      t.render.apply t, arguments

Some templates

<!-- layout -->
<html>
<head>
<title> Hi! </title>
</head>
<body>
{{{body}}}
</body>
</html>

<!-- index -->

<h1>{{title}}</h1>

Rendering a Mustache Template

# Basic View Rendering
app.get "/", (req, res, next) ->
  res.render "index",
    title: "hello world!"

What's the filename of index?

/views/index.hogan.js

Have fun with that.

What happened to the layout?

That's a convention of the template library and not a convention of Express.

Your view is just a string set as the body local variable when rendering the layout.

So we have to reimplement that too

# Basic View Rendering
app.get "/", (req, res, next) ->
  res.render "layout",
    # Warning: pseudocode
    body: indexTemplate.render(title: "hello world!")

How do we get access to a compiled template for the view template?

Where is the view template?

This is all in Express' View engine...

View Engine: Abstraction vs Hiding

The Express.js view engine doesn't abstract some of it's job away from you, it hides it all in such a way that you can't easily reason about what's happening behind the scenes.

So let's make a view engine with some useful abstractions

We'll need to reimplement:

• Template lookup
• Compilation
• Managing helpers
• Sending rendered HTML to the client

Quick Thanks

All of this code is either written by or inspired by Myles Byrne (@quackingduck).

Jobs of a View Engine

• First find the templates on the filesystem
• Read them into the application as a string
• Compile them into a function that returns another string
• Pass data to the function to return a rendered template
• Nest a view in a layout
• Send the rendered template to the browser via HTTP

Express' view engine does all of this

The Pain Points

• No way to access templates outside a request
• No intuition of how a layout interacts with a view
• Bad model of how the files on disk are translated into template functions

A Better Way

Let's reason through a solution that makes all of the jobs of a view engine easy to use without hiding the implementation too much

Allow the developer to reason about how all the pieces fit together without needing to know all the implementation details

Bulk-Hogan

by Myles Byrne (@quackingduck)

https://github.com/quackingduck/bulk-hogan

A simple module to read files from disk, compile them, and give you easy access to it.

Bulk-Hogan

templates = require 'bulk-hogan'

templates.dir = __dirname + '/templates'
templates.modulesDir = __dirname + '/modules'

templates.render 'index', { title: "hello world" }, (err, html) ->
  throw err if err?
  console.log html

What, no layouts?

templates.render 'index', { title: "hello world" }, (err, html) ->
  throw err if err?
  templates.render 'layout', { body: html }, (err, html) ->
    throw err if err?
    console.log html

Look how easy that is!

• Simple rendering to a string
• Templates have a simple naming convention:
  • /templates/index.html.mustache -> index
  • /templates/layout.html.mustache -> layout
  • /modules/users/main.html.mustache -> users
  • /modules/users/widget.html.mustache -> users_widget

Partials

All the templates are accessible as Mustache partials

<!-- templates/index -->
{{>users}}
{{>users_widget}}

<!-- modules/users/main -->

{{>layout}} <!-- if you want, there's nothing stopping you -->

Responding to a Request

# Basic View Rendering
app.get "/", (req, res, next) ->
  templates.render 'index', { title: "hello world" }, (err, html) ->
    next err if err?
    templates.render 'layout', { body: html }, (err, html) ->
      next err if err?
      res.header 'Content-Length', (new Buffer html).length
      res.contentType 'text/html'
      res.end html

Thats a lot more code than res.render

Create a render module

render =
  # Renders the page's template wrapped in it's layout (if it has one)
  pageWithLayout: (template, context, renderCallback) ->
    @pageWithoutLayout template, context, (err, html) ->
      return renderCallback err if err?
      templates.render 'layout', { body: html }, renderCallback

  # Renders just the page's template, no layout
  pageWithoutLayout: (template, context, renderCallback) ->
    templates.render template, context, renderCallback

Try again

app.get "/", (req, res, next) ->
  render.pageWithLayout 'index', {title: "Hello World"}, (err, html) ->
    next err if err?
    res.header 'Content-Length', (new Buffer html).length
    res.contentType 'text/html'
    res.end html

That's still pretty verbose

View Data

A lot of this code is the 'view data':

• Reference to the template
• Data going into the template
• Decision about whether to use the layout or not

Let's move it into a View Model

class Page

  constructor: (@templateName, attrs = {}) ->
    @layout = 'layout'
    @set attrs

  set: (attrs) ->
    for name, value of attrs
      this[name] = @attrs[name] unless this[name]?

viewModel = new View 'index', title: 'Hello World'

The View Model is now the context for the template rendering

Refactor the render module

render =
  # Renders the page's template wrapped in it's layout (if it has one)
  pageHtml: (pageView, renderCallback) ->
    @pageWithoutLayout pageView, (err, html) ->
      return renderCallback err if err?

      if pageView.layout is off
        renderCallback noErr, html
      else
        pageView.body = html
        templates.render pageView.layout, pageView, renderCallback

# Renders just the page's template, no layout
  pageWithoutLayout: (pageView, renderCallback) ->
    templates.render pageView.templateName, pageView, renderCallback

Responding to a Request

app.get "/", (req, res, next) ->
  page = new View 'index', title: 'Hello World'
  render.pageHtml page, (err, html) ->
    next err if err?
    res.header 'Content-Length', (new Buffer html).length
    res.contentType 'text/html'
    res.end html

Kill the boilerplate

We're going to use this callback a lot

(err, html) ->
    next err if err?
    res.header 'Content-Length', (new Buffer html).length
    res.contentType 'text/html'
    res.end html

The Respond Module

# Returns a function suitable for use as the callback to some rendering
# function. If the render succeeds the resulting string is written to the
# response object and then it's closed.
respond = (res, next) ->
  (err, html) ->
    return next err if err?
    res.header 'Content-Length', (new Buffer html).length
    res.contentType 'text/html'
    res.end html

This route started looking like this

# Basic View Rendering
app.get "/", (req, res, next) ->
  templates.render 'index', { title: "hello world" }, (err, html) ->
    next err if err?
    templates.render 'layout', { body: html }, (err, html) ->
      next err if err?
      res.header 'Content-Length', (new Buffer html).length
      res.contentType 'text/html'
      res.end html

So now our route will look like

app.get "/", (req, res, next) ->
  page = new View 'index', title: 'Hello World'
  render.pageHtml page, respond(res, next)

Hide the HTTP details

The respond callback encapsulates all the HTTP response details, so all interfaces to rendering a template can stay agnostic of how data is sent.

Yes, it's more than res.render

But this code is much more flexible!

• Rendering HTML is no longer tied to HTTP in any way
• Very easy to separate the jobs of "making HTML" and "sending HTML to the browser"

Rendering HTML inside a JSON response

app.get "/foo.json", (req, res, next) ->
  page = new View 'index', title: 'Hello World'
  render.pageHtml page, (err, html) ->
    next err if err?

    myJsonResponse = { html, foo: 'bar' }
    stringifiedResponse = JSON.stringify myJsonResponse

    res.header 'Content-Length', (new Buffer stringifedResponse).length
    res.contentType 'application/json'

    res.end stringifiedResponse

Yes, it's more than res.render

Moreover, it allows you to think about each part of that proces separately. You don't need to think about rendering templates at the same time you're thinking about HTTP requests to your server.

Modular View Engine

• First find the templates on the filesystem (Bulk-Hogan)
• Read them into the application as a string (Bulk-Hogan)
• Compile them into a function that returns another string (Bulk-Hogan)
• Pass data to the function to return a rendered template (View class)
• Nest a view in a layout (render)
• Send the rendered template to the browser via HTTP (respond)

Extending View Models

Use CoffeeScript's executable class bodies to create a 'mixin' pattern.

URL Helpers

urlHelpers = (constructorFn) ->
  extend constructorFn.prototype, urlHelpers

extend urlHelpers,
  cssPath: -> (file, render) =>
    path = '/css' + file

  jsPath: -> (file, render) =>
    path = '/js' + file

  pathTo: -> (location, render) =>
    '/' + location

  imagePath: -> (image, render) =>
    '/images' + image

Mix it in

class Page

  urlHelpers @

<!-- index -->
{{#pathTo}}someCool/resource{{/pathTo}}

Think about URL helpers without thinking about the whole View model

Better Abstraction and Simpler Design

• Bulk Hogan defines simple patterns for laying out templates on the filesystem.
• It makes reasoning about accessing and rendering those templates simple.
• But that's all it does. It leaves the rest of the jobs up to you.

Better Abstraction and Simpler Design

• View models allow a developer to easily reason about the context in which templates are rendered. It's simple and declarative.
• The relationship between a layout and a view is clear. It's implemented in our code instead of a library.
• Sending the rendered HTML is now a separate step from generating it, so when we're worried about template rendering we don't have to worry about delivery at the same time.

Thank you!

Padding

Padding

Padding

myles-byrne.jpg