Better local require() paths for Node.js

better-nodejs-require-paths.md

Better local require() paths for Node.js

Problem

When the directory structure of your Node.js application (not library!) has some depth, you end up with a lot of annoying relative paths in your require calls like:

const Article = require('../../../../app/models/article');

Those suck for maintenance and they're ugly.

Possible solutions

Ideally, I'd like to have the same basepath from which I require() all my modules. Like any other language environment out there. I'd like the require() calls to be first-and-foremost relative to my application entry point file, in my case app.js.

There are only solutions here that work cross-platform, because 42% of Node.js users use Windows as their desktop environment (source).

0. The Alias

1. Install the module-alias package:

   npm i --save module-alias
   

2. Add paths to your package.json like this:

   {
       "_moduleAliases": {
           "@lib": "app/lib",
           "@models": "app/models"
       }
   }

3. In your entry-point file, before any require() calls:

   require('module-alias/register')

4. You can now require files like this:

   const Article = require('@models/article');

1. The Container

1. Learn all about Dependency Injection and Inversion of Control containers. Example implementation using Electrolyte here: github/branneman/nodejs-app-boilerplate

2. Create an entry-point file like this:

   const IoC = require('electrolyte');
   IoC.use(IoC.dir('app'));
   IoC.use(IoC.node_modules());
   IoC.create('server').then(app => app());

3. You can now define your modules like this:

   module.exports = factory;
   module.exports['@require'] = [
       'lib/read',
       'lib/render-view'
   ];
   function factory(read, render) { /* ... */ }

More detailed example module: app/areas/homepage/index.js

2. The Symlink

Stolen from: focusaurus / express_code_structure # the-app-symlink-trick

1. Create a symlink under node_modules to your app directory:
   Linux: ln -nsf node_modules app
   Windows: mklink /D app node_modules

2. Now you can require local modules like this from anywhere:

   const Article = require('models/article');

Note: you can not have a symlink like this inside a Git repo, since Git does not handle symlinks cross-platform. If you can live with a post-clone git-hook and/or the instruction for the next developer to create a symlink, then sure.

Alternatively, you can create the symlink on the npm postinstall hook, as described by scharf in this awesome comment. Put this inside your package.json:

"scripts": {
    "postinstall" : "node -e \"var s='../src',d='node_modules/src',fs=require('fs');fs.exists(d,function(e){e||fs.symlinkSync(s,d,'dir')});\""
  }

3. The Global

1. In your entry-point file, before any require() calls:

   global.__base = __dirname + '/';

2. In your very/far/away/module.js:

   const Article = require(`${__base}app/models/article`);

4. The Module

1. Install some module:

   npm install app-module-path --save

2. In your entry-point file, before any require() calls:

   require('app-module-path').addPath(`${__dirname}/app`);

3. In your very/far/away/module.js:

   const Article = require('models/article');

Naturally, there are a ton of unmaintained 1-star modules available on npm: 0, 1, 2, 3, 4, 5

5. The Environment

Set the NODE_PATH environment variable to the absolute path of your application, ending with the directory you want your modules relative to (in my case .).

There are 2 ways of achieving the following require() statement from anywhere in your application:

const Article = require('app/models/article');

5.1. Up-front

Before running your node app, first run:

Linux: export NODE_PATH=.
Windows: set NODE_PATH=.

Setting a variable like this with export or set will remain in your environment as long as your current shell is open. To have it globally available in any shell, set it in your userprofile and reload your environment.

5.2. Only while executing node

This solution will not affect your environment other than what node preceives. It does change your application start command.

Start your application like this from now on:
Linux: NODE_PATH=. node app
Windows: cmd.exe /C "set NODE_PATH=.&& node app"

(On Windows this command will not work if you put a space in between the path and the &&. Crazy shit.)

6. The Start-up Script

Effectively, this solution also uses the environment (as in 5.2), it just abstracts it away.

With one of these solutions (6.1 & 6.2) you can start your application like this from now on:
Linux: ./app (also for Windows PowerShell)
Windows: app

An advantage of this solution is that if you want to force your node app to always be started with v8 parameters like --harmony or --use_strict, you can easily add them in the start-up script as well.

6.1. Node.js

Example implementation: https://gist.github.com/branneman/8775568

6.2. OS-specific start-up scripts

Linux, create app.sh in your project root:

#!/bin/sh
NODE_PATH=. node app.js

Windows, create app.bat in your project root:

@echo off
cmd.exe /C "set NODE_PATH=.&& node app.js"

7. The Hack

Courtesy of @joelabair. Effectively also the same as 5.2, but without the need to specify the NODE_PATH outside your application, making it more fool proof. However, since this relies on a private Node.js core method, this is also a hack that might stop working on the previous or next version of node.

In your app.js, before any require() calls:

process.env.NODE_PATH = __dirname;
require('module').Module._initPaths();

8. The Wrapper

Courtesy of @a-ignatov-parc. Another simple solution which increases obviousness, simply wrap the require() function with one relative to the path of the application's entry point file.

Place this code in your app.js, again before any require() calls:

global.rootRequire = name => require(`${__dirname}/${name}`);

You can then require modules like this:

const Article = rootRequire('app/models/article');

Another option is to always use the initial require() function, basically the same trick without a wrapper. Node.js creates a new scoped require() function for every new module, but there's always a reference to the initial global one. Unlike most other solutions this is actually a documented feature. It can be used like this:

const Article = require.main.require('app/models/article');

Since Node.js v10.12.0 there's a module.createRequireFromPath() function available in the stdard library:

const { createRequireFromPath } = require('module')
const requireUtil = createRequireFromPath('../src/utils')

requireUtil('./some-tool')

Conclusion

0. The Alias
Great solution, and a well maintained and popular package on npm. The @-syntax also looks like something special is going on, which will tip off the next developer whats going on. You might need extra steps for this solution to work with linting and unit testing though.

1. The Container
If you're building a slightly bigger application, using a IoC Container is a great way to apply DI. I would only advise this for the apps relying heavily on Object-oriented design principals and design patterns.

2. The Symlink
If you're using CVS or SVN (but not Git!), this solution is a great one which works, otherwise I don't recommend this to anyone. You're going to have OS differences one way or another.

3. The Global
You're effectively swapping ../../../ for __base + which is only slightly better if you ask me. However it's very obvious for the next developer what's exactly happening. That's a big plus compared to the other magical solutions around here.

4. The Module
Great and simple solution. Does not touch other require calls to node_modules.

5. The Environment
Setting application-specific settings as environment variables globally or in your current shell is an anti-pattern if you ask me. E.g. it's not very handy for development machines which need to run multiple applications.

If you're adding it only for the currently executing program, you're going to have to specify it each time you run your app. Your start-app command is not easy anymore, which also sucks.

6. The Start-up Script
You're simplifying the command to start your app (always simply node app), and it gives you a nice spot to put your mandatory v8 parameters! A small disadvantage might be that you need to create a separate start-up script for your unit tests as well.

7. The Hack
Most simple solution of all. Use at your own risk.

8. The Wrapper
Great and non-hacky solution. Very obvious what it does, especially if you pick the require.main.require() one.

@branneman On solution 8 "The wrapper", the createRequireFromPath() has been deprecated. See here for deprecation. It suggests using createRequire() instead, which was introduced in v12.2.0. See here for documentation.

i would suggest taking a look into npm v7 workspaces feature
https://docs.npmjs.com/cli/v7/using-npm/workspaces/

it works like a charm for me

thanks @SeanDunford

@W2AlharbiMe i'm not really fond of having to make a package.json inside every folder that i want to turn into a workspace. Do you find it annoying at all?

@W2AlharbiMe i'm not really fond of having to make a package.json inside every folder that i want to turn into a workspace. Do you find it annoying at all?

no in fact it has more benefits for monorepos take a look here
https://dev.to/limal/simplify-your-monorepo-with-npm-7-workspaces-5gmj

i'm currently using it here
https://github.com/W2AlharbiMe/fastify-boilerplate

(for Yarn) How about: https://yarnpkg.com/features/protocols#why-is-the-link-protocol-recommended-over-aliases-for-path-mapping ?
Just define it in dependencies as such

"dependencies": {
  "src": "link:./src"
}

My IDE was easilly able to parse my linked src folder with zero configuration

(for Yarn) How about: https://yarnpkg.com/features/protocols#why-is-the-link-protocol-recommended-over-aliases-for-path-mapping ?
Just define it in dependencies as such

"dependencies": {
  "src": "link:./src"
}

My IDE was easilly able to parse my linked src folder with zero configuration

NICE, thanks for sharing

Another ugly solution (only for ESM modules) came with v16.12.0

Documentation

Here's the simplest example I personally use:

// my_project/loader.js
import path from 'path';
import fs from 'fs';
import { fileURLToPath } from 'url';

export async function resolve(specifier, context, defaultResolve) {
  if (specifier.startsWith('/')) {
    // try resolve absolute path imports relative to
    // project root (closest "package.json" location)
    let root = context.parentURL ? path.dirname(fileURLToPath(context.parentURL)) : process.cwd();
    while (root !== '/' && root !== '.') {
      try {
        await fs.promises.access(path.resolve(root, 'package.json'), fs.constants.R_OK);
        specifier = path.resolve(root, specifier.replace(/^\/+/, ''));
        break;
      } catch {
        root = path.dirname(root);
      }
    }
  }
  return defaultResolve(specifier, context, defaultResolve);
};

To make use of the loader you have to run node with flag:

cd my_project
node --experimental-loader ./loader.js start.js

And now

// my_project/deep/nested/script.js
import util from '/lib/util.js'; // -> resolves to my_project/lib/util.js

Or you can implement any other kind of resolution logic.

PROS: Native, no deps, full control, completely customizable
CONS: --experimental-loader flag, API is subject to change, only for ESM modules, possible side-effects??

Great post. Thanks.

If the project support .env file, I use NODE_PATH solution (5) , like this:

.env:

NODE_PATH=.
...

Thanks for your sharing