NodeJS at GDS.md

title
Using Node.js at GDS

This document describes how we write Node.js code at GDS. It is a list of guidelines that developers should follow in order to standardise code across all Node.js projects, making it easy for developers to change projects or start new ones.

Important Note

No new project at GDS should use Node.js without prior consultation with the Deputy Director Technology Operations. The programming language recommendations section has more detail on how we currently use Node.js at GDS and why this decision has been made.

Introduction

The guidance set out here should be followed in conjunction with the advice on the main programming languages manual page.

The advice here is specifically about Node.js. Generic guidelines on writing code are out of scope.

If you want to contribute to this document please see the Updating this manual section below.

Most guidelines listed here are recommendations, and the authors acknowledge that there will sometimes be valid exceptions.

Node versions

Only use Long Term Support (LTS) versions of Node.js.

These are even-numbered versions (for example, Node.js 6.x or 8.x). However, it is important to keep an eye on the Node.js LTS Schedule as to when versions move in and out of LTS.

Don't use the --harmony or in-progress feature flags

Staged or in-progress features aren't considered stable by the Node implementors.

Don't use language features beyond EcmaScript2016

Even though ES2017 is final, some features aren't yet implemented in LTS versions. Check using node.green.

Source formatting and linting

Use the JavaScript Standard Style (StandardJS)

StandardJS includes a built in formatter, linter and automatic code fixer (auto-fix). To make sure everything stays clean we recommend running this tool before each commit. It is also possible to setup a pre-commit hook to automate this process. When StandardJS automatically fixes errors it has clear and concise messaging and it also tells you when an automatic fix isn't possible.

We’ve encapsulated our recommended process into a separate repository where everything is controlled by a few lines of config within your package.json file. Once you have run npm install, any time you make a commit, StandardJS runs. It will then format and lint your code automatically.

Most editors can check StandardJS syntax through plugins that in some cases comes pre-installed. Check the editor plugin list.

Project directory structure

Organise files around features, not roles

The following structure means you don’t require lots of context switching to find related functions and your require statements don’t need overly complicated paths.

├── product
|   ├── index.js
|   ├── product.js
|   ├── product.spec.js
|   └── product.njk
├── user
|   ├── index.js
|   ├── user.js
|   ├── user.spec.js
|   └── user.njk

Don’t put logic in index.js files

Use these files to require all the modules functions.

// product/index.js
const product = require('./product')

module.exports = {
  create: product.create
}

Store test files within the implementation

Keeping tests in the same directory makes them easier to find and more obvious when a test is missing. Keep the project's global test config and setup scripts in a separate test directory.

├── test
|   └── setup.spec.js
├── product
|   ├── index.js
|   ├── product.js
|   ├── product.spec.js
|   └── product.njk

See also:

• Node.js Project Structure Tutorial on Rising Stack

Language constructs

Declarations

Use const and let, avoid var

Embrace immutability (more below) and don't let hoisting confuse you.

Functions

Prefer function for top-level function declarations and => for function literals

// Use:
const foo = function (x) {
  ...
}

// avoid:
const foo = x => {
  ...
}


// Use:
map(item => Math.sqrt(item), array)

// avoid:
map(function (item) { return Math.sqrt(item) }, array)

Acceptable exceptions include single expression functions, such as

const collatz = n => (n % 2) ? (n / 2) : (3 * n + 1)

or curryied functions, which are more readable using the arrow notation:

const foo = x => y => z => x*y+z

Be aware that because anonymous functions don't have a name, a stack trace will be harder to read when debugging. Also remember that arrow functions keep their context's this.

Classes

Use the class keyword to define classes

There are many different ways to define classes in JavaScript. class has been added to the standard specifically to resolve this. In short, use class for classes and function for functions.

class Rectangle {
  constructor(height, width) {
    this.height = height
    this.width = width
  }
  area () {
    return this.width * this.height
  }
}

Asynchronous code

Use asynchronous versions of the Node.js API functions whenever possible.

For instance, avoid readFileSync but instead use readFile. Even if your code is less readable as a result and that particular piece of code doesn't need to be asynchronous (because you can't proceed until you've read that file anyway), it won't block other server threads. However if your program doesn't use concurrency, synchronous versions are sometimes preferable as they are more readable and less heavy on the operating system.

Avoid inline callbacks

// Prefer:
const pwdReadCallback = function (err, data) { ... }
fs.readFile('/etc/passwd', pwdReadCallback)

// over:
fs.readFile('/etc/passwd', (err, data) => {
...
}


// Another example:

const done = function (resolve, reject) {
  return () => {
    try {
      // Do something that might fail
      resolve('all went well')
    } catch (e) {
      reject(e)
    }
  }
}

const waitAndSee = function (resolve, reject) {
  setTimeout(done(resolve, reject), 2500)
}

const log = status => message => console.log(status + ': ' + message)

new Promise(waitAndSee)
  .then(log('Success'))
  .catch(log('Failure'))

This will avoid "callback hell" and encourage organising callback functions linearly, in an event-driven fashion. It also makes it easier to write unit tests for those functions.

const pwdReadCallback = function (err, data) { ... }
const userLoggedInCallback = function (authToken) { ... }
const dbRequestResultCallback = function (req, res) { ... }

Separating out a callback may pose a problem when it needs its original scope. This can be handled by currying the callback:

fs.readFile(filename, callback(filename))

const callback = filename => (err, data) =>
  if (err) {
    console.log(`failed reading ${filename}.`)
    throw err
  }
  ...
}

Functional programming

Use JavaScript's functional programming features

JavaScript has the advantage that it offers functional programming concepts natively, like functions as first-class objects, higher-order functions (like map, reduce or apply) or pattern matching.

Following functional programming principles, such as immutable data structures and pure functions, produces code that is easier to test, less prone to runtime errors and is often more performant. Write functions as expressions that return values of a single type.

Side effects in a function can have bad consequences, especially within map or reduce. this is better avoided, as it can refer to different objects depending on the context in which a function is executed and lead to unexpected side-effects.

Remain aware that JavaScript is not just a functional language and you will most probably have to mix functional and object-oriented concepts, which can be tricky to get right.

More guidance:

• An Introduction to Functional JavaScript
• JavaScript Allongé

Errors

Make sure you handle all errors

Envisage all error scenarios, in particular in asynchronous callback functions or Promises, and have a fallback for any situation. Consider programmer errors (bugs) as well as operational errors (arising from external circumstances, like a missing file). Bugs that are caught by exceptions should be logged and the execution stopped, and the supervisor will restart the process. Use the built-in Error object instead of custom types. It makes logging easier.

Your application shouldn't trust any input, for example from a file or an API. So the application should handle all operational errors and recover from them.

Node.js's HTTP server

Offload Node's server as much as possible

Don't expose Node's HTTP server to the public. Use a reverse proxy to serve static assets, and cache content as much as possible. Implement adequate supervising: pm2 is recommended for Node.js-specific servers.

Transpiling

Stick to JavaScript

Avoid anything that compiles to JavaScript (except for static type checking, see below). Examples to avoid include CoffeeScript, PureScript and many others.

Use TypeScript

In order to avoid runtime exceptions and catch errors before execution, TypeScript is recommended. Flow is also acceptable, but anything that departs from the standard JavaScript syntax in a way that a Node developer would have trouble reading your code is advised against.

Frameworks

Use Express for web applications, avoid lesser-known frameworks

Express is very common for writing web applications that talk to back-end APIs. If you find yourself looking for an MVC framework or if you need an ORM to manage records in a database, you probably shouldn't be using Node.js in the first place.

Libraries

Avoid libraries that produce esoteric code, or that have a steep learning curve

Using advanced libraries can make code very hard to read for developers not familiar with them, as useful as they may be. For instance, Ramda lets you write:

R.cond([
 [R.is(Number), R.identity],
 [R.is(String), parseInt],
 [R.T, R.always(NaN)]
])

which is compact and useful, but not easily understood up by a developer not familiar with it.

Generally, readable code is better than compact or advanced code. Optimisation can lead to very arcane code, so should only be used when necessary.

// Prefer:
for (let i=0; i<10; i+=1) {
  for (let j=0; j<10; j+=1) {
    console.log(i, j)
  }
}

// Over:
for (let i=0,j=0;i<10 && j<10;j++,i=(j==10)?i+1:i,j=(j==10)?j=0:j,console.log(i,j)){}

Node Package Manager (NPM)

Limit your reliance on external code, and check the NPM packages you use

Using other people's code is a risk. NPM is no exception and is arguably worse than other package managers as the small granularity of packages leads to a very large number of dependencies. Your application may easily end up relying on software written by hundreds of different people, most of whom you don't know.

However, given that it's impossible not to depend on foreign code, you should do your best to mitigate the risks involved:

• avoid relying on packages for functionality you could implement yourself without too much effort
• Empirically assess how reliable any foreign code you are using is: look up the source, check how trustworthy a package is, or its author
• Reduce the number of dependencies in your applications to a minimum
• Use tools like Snyk to check packages for vulnerabilities

Use npm init, npm start, npm script, etc.

Making full use of NPM's features will simplify your continuous integration and deployment.

Lock down dependencies

Avoid incompatible upgrades that may break your application. NPM 5 does it by default, but be careful when upgrading or adding a new package.

Regularly check for unused dependencies, and make sure you don't have dev dependencies in production

Don't slow down your deployments with unused code

If you build modules you think could be useful for others, publish them on npm

Because Open Source is a Good Thing

See also:

• Controlling the Node.js security risk of npm dependencies
• I’m harvesting credit card numbers and passwords from your site. Here’s how.
• you might not need gulp or grunt.

Further reading

In the GDS context, the guidelines provided in the links below are advisory only. The guidance provided here takes precedence in case of conflicts.

• [Node.js best practices] — A comprehensive guide to Node.js best practices
• node.cool — A curated list of Node.js packages and resources by Sindre Sorhus
• JS documentation — MDN's definitive JS docs

Updating this manual

This manual is not presumed to be infallible or beyond dispute. If you think something is missing or if you'd like to see something changed then:

1. (optional) Start with the #Nodejs community's Slack channel to see what other developers think. You will then understand how likely it is that your proposal will be accepted as a pull request before you complete any work.
2. Check out the making changes section of the GDS Tech repo
3. Create a pull request against GDS Tech repo

Maybe link to the Deputy Director Technology Operations on people finder (or write their name)

Good point @TobyRet we need clarification around that point. There is an issue open that I raised in the GDS Way. Need to discuss it again with @rboulton.