Rendering templates obsolete

rendering_templates_obsolete.md

(tl;dr DOM builders like domo trump HTML templates on the client.)

Like all web developers, I've used a lot of template engines. Like most, I've also written a few of them, some of which even fit in a tweet.

The first open-source code I ever wrote was also one of the the first template engines for node.js, [a port][node-tmpl] of the mother of all JavaScript template engines, [John Resig][jresig]'s [micro-templates][tmpl]. Of course, these days you can't swing a dead cat without hitting a template engine; one in eight packages on npm ([2,220][npm templates] of 16,226 as of 10/19) involve templates.

John's implementation has since evolved and [lives on in Underscore.js][underscore], which means it's the default choice for templating in Backbone.js. And for a while, it's all I would ever use when building a client-side app.

But I can't really see the value in client-side HTML templates anymore.

I recently built a small (under 1k minizipped) DOM library called domo. All it really does is pull the semantics of CSS and HTML into the syntax of JavaScript, which means you can write your entire client app in one language. It looks like this:

// this code returns an HTML DOM element
HTML({lang: "en"},
  HEAD(
    TITLE("domo"),
    STYLE({type: "text/css"},
      CSS("#container", {backgroundColor: "#eee"})
    )
  ),

  BODY(
    DIV({id: "container"},
      "For more details about domo, see the source: ",
      A({href: "//github.com/jed/domo/blob/master/domo.js"}, "View source")
    )
  )
)

It doesn't replace CSS or HTML at all; it just lets you write them both in JavaScript. So instead of relying on the arbitrary design choices of SASS/LESS or HAML/Mustache for making your CSS and HTML more dynamic, you rely only on the design choices of JavaScript, which while subjectively less arbitrary, are objectively more known.

I think there's so much win in this approach that I'm surprised it's not more popular. Or to turn it around, I'm surprised most web developers tolerate the usual workflow for templating:

1. Write HTML templates written in some arbitrary frankenlanguage,
2. put them in several DOM script tags with a non-standard type,
3. compile it into a function that takes data, HTML-escapes it, and returns HTML, and
4. use jQuery to turn the html string into a proper DOM element.

when all we really need is a friendlier way to wrap the overly imperative and verbose DOM API into JavaScript code that mimics the resulting DOM.

Sure, using a DOM builder like domo instead of both a CSS preprocessor and HTML template engine means you lose the superficial syntax of both CSS and HTML, but you still preserve the underlying semantics and gain the following in return:

• Reduce your exposure to XSS attacks. You don't need to rely on a library to sanitize your rendered data because the browser does it for you automatically with createTextNode and setAttribute.

• Eliminate a separate compile step. With DOM builders, "compilation" is done in the same JavaScript process as your code, which means that any syntax errors are thrown with line numbers in the same JavaScript process.

• Write your DOM code where it makes sense. If you're writing a view that renders a DOM node, you can write it directly within the rendering code for convenience, and then pull it out into its own file when the size of your app requires it. You don't need to let implentations drive architectural decisions.

• Use JavaScript syntax everywhere. Instead of remembering which symbols map to which behaviors like looping or escaping or conditionals or negation, just use JavaScript.

• Decouple your syntax sugar. Instead of choosing a template engine with the ideal syntax, just use JavaScript, and do your sweetening on a more reliable level. CoffeeScript can go a long way in [making DOM building code look like HAML][browserver], for example.

• Reduce the number of moving parts. Instead of shipping HTML containing strings that compile into JavaScript functions that return HTML that's used to create DOM nodes, just use JavaScript to create DOM nodes, eliminating underscore/jQuery dependencies at the same time.

• Reuse existing infrastructure. Any tools you use in your JavaScript workflow, such as minification or packaging, can now be used for your styles and markup too. You can even use something like [browserify][browserify] to easily discover all app dependencies and maintain code modularity.

• Lessen the burden of context switching. Whether using JavaScript on both the client and server enables code reuse is debatable, but that it prevents the overhead of switching contexts between languages is less so. It may be subjective, but I think using one language everywhere reduces the congitive overhead for web app development.

As more and more logic moves to the browser, I really think this approach makes a lot of sense and would love to see more developers give it a try. I'd love to [hear your feedback][hn].

[node-tmpl]: https://github.com/jed/tmpl-node/commit/33cfc11f54b0e6c33b9de7589c62057dae82848b) [jresig]: http://ejohn.org/blog [tmpl]: http://ejohn.org/blog/javascript-micro-templating/ [underscore]: http://documentcloud.github.com/underscore/#template [npm templates]: https://encrypted.google.com/search?q=template+OR+templating+site:npmjs.org [browserver]: https://github.com/jed/browserver.org/blob/master/app.coffee [browserify]: https://github.com/substack/node-browserify [hn]: http://news.ycombinator.com/item?id=4672353

I've liked this way of creating contents since I first saw it in MochiKit, then discovered Dan Webb's standalone version.

Once you're able to generate HTML strings from the same code, another benefit is reusable components - I'm using a DOMBuilder for all output creation in newforms and since the Node.js version of DOMBuilder defaults to HTML output, I can use it in my Node projects and reuse Form objects on both sides, e.g. these are my Jade form rendering mixins for a CRM.

You can also do templating this way - I have a mode plugin for DOMBuilder which apes the features and implementation of Django's template language (primarily for template inheritance, which since Django has been my must-have feature for any template language), bonuses being that you don't need to write a parser (#lazyweb), new "templatetags" are just functions and you can use JavaScript functions to generate chunks of template instead of putting everything in template logic.

Examples:

• Defining templates
• Extending templates
• Switching the base template for a full-page version on the server side

This is also a gateway drug to using comma-first, as if you're just using a text editor it's the only way to stay sane with the kind of nested structures you end up writing, as you don't have to scan a ragged edge to find missing commas and they all line up nicely to indicate the nesting level.

I found that once you've been doing this for a while and you're used to dealing with the nesting, nested array representations of elements start to become bearable - I added a method to DOMBuilder support using nested arrays with its DOM and HTML output modes, but for Backbone.js projects at the moment, I'm liking Tim Caswell's dombuilder more than anything else, as it makes it so easy to pull out references to generated elements as you're defining the structure.

I attempted something similar using arrays and strings (incurs more characters but removes the need for defining so many global functions):

lmd(['div', ['h1', {class: 'big'}, 'Hello World']])

https://github.com/rudenoise/LM.JS

One possible advantage being that (as long as functions are not in-lined) you could transport it as JSON.

It was inspired by https://github.com/sgrove/vana-templating

Very similar to _why's markaby.

I personally don't like this for several reasons:

1. There is also the assumption that one would only want to use a template engine for html/css, when building more complex applications one can use templates to generate JavaScript, JSON and XML, whatever they need.

2. String templates can be loaded and serialised via JSON parse/stringify, with this method you would need to use eval to parse your template, rather than turning a string into a function.

3. Any attempt at code reuse will still re

   however, the scope of what a template should be able to generate
   I personally prefer plain old html for tem

Sorry about the last post, pressed some key which obviously meant "add comment"! Be nice if you could remove your own comments... :P

Very similar to _why's markaby.

I personally don't prefer this style for several reasons:

1. There is assumption that one would only want to use a template engine for html/css, when building more complex applications one can use templates to generate JavaScript, JSON, XML and/or whatever they need.
2. String templates can be more easily loaded and serialised via JSON parse/stringify, with this method – if you wanted to load in your templates only as they were needed – you would need to use eval or JSONP or requireJS to parse your template, rather than turning a string into a function. So that under 1kb has maybe just gained a few dependancies?
3. Any attempt at template reuse will still result in adding the template code inside of a function, so what's the difference?
4. There is nothing inherently wrong with creating a string that is then parsed into a DOM tree – you also don't need to use jquery to do this – also, last time I checked it was still faster than creating individual DOM nodes and appending them to a tree.
5. Readability and making changes are not improved by this method, where missing closing tags can be forgiven by most user agents a missing closing function parenthesis is not and may take longer to debug.
6. HTML is much more easy to edit for those unfamiliar with JavaScript, so changing look/feel is not limited to those with JavaScript knowledge.
7. If/when the HTML syntax changes others will be reliant on the maintainer of the library to keep it current, a template engine that can generate strings is not bound by this constraint.

I think that's all. Btw. I'm not trying to attack or belittle your idea, if I came across this way, I apologise in advance as it is due to my lack of eloquence rather than anger. :)

Just thinking out loud (i.e. without processing what I'm about to say):

Personally I think that:

a) this approach will take off rapidly at some time in the future to a point where we hardly interact with the DOM at all. I mean: why do we do this? Why do we effectively have two representations of our program (javascript / application and DOM). Sooner or later the line between them is going to blur and this is the start of it.

b) I predict that within ten years HTML and the DOM will die. Let's be honest: they're rubbish. HTML5 is putting lipstick on a pig. I have no idea what form the replacement will take but the coding process will be similar to the above: there will be no distinction between the application and the rendering engine / API.

Just my 0.02

This is interesting although I think template libraries have sprung up for a good reason: to separate control logic from presentation logic. And, as previously mentioned, to enable non-coders to have some control over the presentation layer. This approach seems like it would impede a clean separation of concerns.

This is not clear, this is not cool-looking for designers, this creates so many global functions, this is hard to use with iterations ... I'd rather not use this.

@jed I understand, the purpose of my feedback before, and here, is not to debate, but just to have a constructive discussion for clarification sake. Your comments definitely helped me understand better what your solution is going for, so that's a good thing. If nothing else, hopefully this helps you identify places where your solution can benefit from better "documentation". :)

...implement an identical API to dom-o whose output is an HTML string instead of a DOM node, or provide both toHTML and toDOM methods of a lightweight "node" class

Interesting thought. If your approach could either generate DOM or markup, what then would be the reason why you would use the DOM approach? If the concern is just avoiding having to set innerHTML because that sort of code is ugly, I'm sure why insertBefore(node,node.parent) messiness is all that much better. Besides, your API could easily just do the innerHTML insertion for them, instead of DOM appending.

And there have been a number of example performance tests done which, IIRC, show that in most cases innerHTML insertion is faster. Of course, it does have caveats of overriding event handlers, but that's something you have to be careful about no matter how you're modifying the page.

So is there another benefit to DOM vs. markup insertion that I'm not seeing?

...replace the current node upon generation for elements of which there should only be one of in the DOM (HTML, HEAD, BODY).

So, for cases where I did DIV({id:"container"} ...), would it automatically replace that element under the same sort of assumption? That sounds nice, but then it creates more disparity if you do DIV({class:"foo"} ...) and there are more than one of them in the page... does it replace all of them, or none of them (and force you to fall back to your own code to inject)?

What I was getting at with the original point is, when you use a traditional string-based template engine, you always do the page insertion in the same way (inserting via innerHTML), so there's a consistency and predictability to that pattern. Your approach seems to suggest that there may end up being several (2 or more) different ways I have to modify the page depending on the type of element I'm dealing with.

It would be nice if there didn't have to be very different approaches, or special rules of when it was automatic and when you had to do it yourself, etc.

also, i'm unfamiliar with what SPA means.

Sorry, SPA = Single Page Application

not sure what css pre-processing / templates you're using, but with some exceptions, the error output for the ones i've used is pretty horrible.

I don't really use much by way of css pre-processors (not on that bandwagon, at least yet), but I do use my template engine grips for markup generation. I took great pains in making sure that it was very robust/friendly /helpful in its debug error messages, both during compilation and during run-time. It always shows you what symbol in the original template is the offender, and gives a line and column reference. This is true even when the "error" that happens is a run-time javascript exception in the compiled code.

If the template engines you're familiar with don't take this level of care for debugging utility, perhaps take a look at grips. :)

CSS and HTML folks are already learning and using things like LESS and HAML respectively.

That's a fair point, but it brings up another concern that I have, a very strong one that was actually the primary motivator for me designing grips the way it is. I say in the docs that I want grips to be remarkable not for what it can do, but for what it cannot do. It's a restrained and simple DSL syntax for templating, which aims to give you only what tools you need, and not let you do the things that people often end up trying to squeeze into their template code which clearly violates SoC (separation of concerns).

SoC is so important to me, I had to go to the trouble of building a template engine that enforced it, because human/dev nature is so overriding that time and time again, in various teams and various dev cultures, if you can do something in a template, invariably some dev does do that in the template.

Example: it's really "nice and convenient" to format currency inside your template, so you just call out to some function (because your template engine can call functions and perform other general coding tasks). Then it's not too long before that formatting function is doing business logic (like checking a user's preferences in addition to their locale), and then it's not too long before you wrap your formatting call in some boolean operation combined conditional check, and ...

The point is, general coding syntax inside a template is the gateway drug to violation of SoC, and it has been one of the worst offenders in all my various dev experiences to poor app architecture, complicated code maintenance, etc.

Your approach throws that concern to the wind (you're definitely not the only "offender" -- many do!), and says basically, "hey, it's all code, intermingle at will, and we'll trust you to do the right thing..." :)

what about rendering that requires client-side runtime information? are you going to render server templates that render client templates (i've done that, it sucks).

I don't think having the exact same template engine and templates, and letting an application decide, based on a variety of factors (including performance, capability of the connected user agent, etc), where to actually invoke the render(), is necessarily bad complexity. I think it's a flexibility and power that represents a next generation of "responsive" apps.

What I would do, and what I often do, is have a general full-page template that's rendered on the server, and if there are parts or chunks of the page which must respond to user environment for their rendering, then those parts are simply left with empty placeholders, and then client side template rendering is invoked to generate the innerHTML for that placeholder. I'm not sure why that's seen as a bad pattern, it worked well on a number of my projects. But anyway, that's how I'd handle it.

there's nothing to stop you from writing a JS-based stylesheet in its own resource (even rendered on the server and loaded via LINK), and a JS-based dom or other logic in another.

Sure, that's one valid approach, but it wasn't hinted at in your example, and instead the rather anti-pattern of "eschew the external stylesheet and drop all your CSS into an inline <style> tag" is what you showed, so that's what I was objecting to.

Of course, to do that, you'd need for the "templating language" in question to be able to render out text for the CSS from the server (like you mentioned above with a toHTML()), and we get back to the above question of why we'd need both the capability to render CSS as DOM and as HTML? What would rendering CSS as DOM really buy us?

what's to prevent you from validating the rendered CSS?

The "rendered CSS" in your above example is never in a string form suitable for linting/validation, it's directly injected into the DOM, so it's never in a form that would be easily lintable by current tools.

If it was instead compiled/built into a string CSS representation, in a regular CSS file, on the server, during a build-step, then yes you definitely would be able to lint/validate it, but then... haven't we come back to one of the main things you were trying to avoid, which is to have a build step associated with your page generation?

The much more likely scenario is that if people wanted to lint this kind of CSS and markup, they'd need separate tools that were built to understand and inspect the JS DSL you've created. That's not abjectly a bad thing, but it's definitely an extra "cost" to your approach as opposed to sticking with traditional separation of markup and CSS into their own files, which lets them more easily be validated by existing tools.

Wait, so this isn't some arbitrary Frankenlanguage?

DIV({id: "container"},
  "For more details about dom-o, see the source: ",
  A({href: "//github.com/jed/dom-o/blob/master/dom-o.js"}, "View source")
)

I've seen this attempted for 15 years in Perl, PHP, and now JS and they never gain any traction.

👍

"Sugared DOM" is similar but feels even more JavaScripty to me. I also consider the approach with child elements as an array to be more powerful.

Reminds me of DOMQL ;)

INSERT INTO (CREATE ELEMENT HTML (
          lang 'en'
        )) VALUES (
          CREATE ELEMENT TITLE (
            innerText 'dom-o'
          ),
          CREATE ELEMENT STYLE (
            type 'text/css'
          )
      )

More about Sugared DOM and alternatives to templates: http://blog.fastmail.fm/2012/02/20/building-the-new-ajax-mail-ui-part-2-better-than-templates-building-highly-dynamic-web-pages/

My two cents is that this approach doesn't improve upon external templates (in their own files), but it does improve upon generating HTML from within JavaScript. The current methods for this (I'm looking at you jQuery) are not great.

I'm no fan of templates, but obsessive with reuse. I crafted buildr for this kind of thing. It uses jQuery.

http://mlanza.github.com/buildr/

I like this approach and used it in a personal library I have called TBone. I originally wrote it to explore some ideas from the NPR.org tech blog. Originally I tried to keep all the dom like parts separate. In my recent rewrite I abandoned this approach use extended string. In my rewrite (e.g. T.div("inner content here") and an attribute function (T.attr()). It's chainable and all the tags support variable number of arguments. This let me do things like-

   var TBone = require("tbone"),
         T = new Tbone.HTML();

    console.log(T.div(
         T.h1("Hello World),
         T.p("blah blah blah").attr({"class": "sensless", "id":"do-i-care"})
    ).attr({"class", "my-stuff"}).toString());

The nice thing is no dependencies. Words in NodeJS, Works in the Mongo DB shell, works in the browser (though I don't have a real reason to use it there). Loops are just a JS loop or a map and sticking the results in another container. Server side I think you can use modules like memoize (https://github.com/medikoo/memoize.git) to improve performance and cache functional results.

I would encourage you to keep exploring this approach.

All the best.

Happy to see that the idea of writing everything with one language is starting to spread. For about a year now, I'v been using only CoffeeScript to write my templates and CSS (with CoffeeKup and CCSS). I have a system where i have a single file for a template, style definitions- and functionality (event bindings, etc.) of that template. It gives you a nice modularity where you can take these template-components and throw them around from one project to another and everything just works, since all you need for that component is a single file. Here's one template: https://github.com/jussiry/houCe/blob/master/client/templates/header.templ

I'll reserve any personal investment of time into new templating systems until the W3 Web Components spec is implemented (Firefox will be landing Custom Elements shortly). I can nearly guarantee that our collective perception of, and approach to, the DOM and Templates will change _dramatically_ over the next 18 months. Let's tee it up Milton Friedman-style: "Underlying most arguments against the DOM is a lack of belief in the DOM itself"

I don't expect it does many things with template in just 1K of code. dom-o is an intuitive way to create small HTML snippets in code, better than chaining jQuery objects. Thanks.

Enyo JS does something similar. It's focused on building a reusable component tree, but the rendering of any component results in a JS object hierarchy being turned into HTML. At the moment, our main rendering approach is to generate a string then use innerHtml to set a large part of the DOM at one time, but I could see us switching to use document fragment creation if that was shown to be faster in the environments we care about.

Appreciate the point/counter-point dialogue going on here. Dont have much to offer except the encouragement to keep it up as I'm evaluating templating options for a pretty plain-vanilla application. I'm attempting to enforce a standard template for "show" and "index" views and have been fairly successful so far. I'm still early enough to adopt this approach though, if I can get my head around it. Thanks, all.

The problem is designers and tools. They handle the layouts and mockups and they are not interested in building layouts from javascript arrays or objects. Works for us, but not them.

Basically, it would be better for many artists if they all drew on a computer tablet with vectors and unlimited "undo" history - but some of them still like painting on canvas, sidewalks, buildings, etc.. and we just need to work around that.