Secure random values (in Node.js)

random.md

Not all random values are created equal - for security-related code, you need a specific kind of random value.

A summary of this article, if you don't want to read the entire thing:

• Don't use Math.random(). There are extremely few cases where Math.random() is the right answer. Don't use it, unless you've read this entire article, and determined that it's necessary for your case.
• Don't use crypto.getRandomBytes directly. While it's a CSPRNG, it's easy to bias the result when 'transforming' it, such that the output becomes more predictable.
• If you want to generate random tokens or API keys: Use uuid, specifically the uuid.v4() method. Avoid node-uuid - it's not the same package, and doesn't produce reliably secure random values.
• If you want to generate random numbers in a range: Use random-number-csprng.

You should seriously consider reading the entire article, though - it's not that long :)

Types of "random"

There exist roughly three types of "random":

• Truly random: Exactly as the name describes. True randomness, to which no pattern or algorithm applies. It's debatable whether this really exists.
• Unpredictable: Not truly random, but impossible for an attacker to predict. This is what you need for security-related code - it doesn't matter how the data is generated, as long as it can't be guessed.
• Irregular: This is what most people think of when they think of "random". An example is a game with a background of a star field, where each star is drawn in a "random" position on the screen. This isn't truly random, and it isn't even unpredictable - it just doesn't look like there's a pattern to it, visually.

Irregular data is fast to generate, but utterly worthless for security purposes - even if it doesn't seem like there's a pattern, there is almost always a way for an attacker to predict what the values are going to be. The only realistic usecase for irregular data is things that are represented visually, such as game elements or randomly generated phrases on a joke site.

Unpredictable data is a bit slower to generate, but still fast enough for most cases, and it's sufficiently hard to guess that it will be attacker-resistant. Unpredictable data is provided by what's called a CSPRNG.

Types of RNGs (Random Number Generators)

• CSPRNG: A Cryptographically Secure Pseudo-Random Number Generator. This is what produces unpredictable data that you need for security purposes.
• PRNG: A Pseudo-Random Number Generator. This is a broader category that includes CSPRNGs and generators that just return irregular values - in other words, you cannot rely on a PRNG to provide you with unpredictable values.
• RNG: A Random Number Generator. The meaning of this term depends on the context. Most people use it as an even broader category that includes PRNGs and truly random number generators.

Every random value that you need for security-related purposes (ie. anything where there exists the possibility of an "attacker"), should be generated using a CSPRNG. This includes verification tokens, reset tokens, lottery numbers, API keys, generated passwords, encryption keys, and so on, and so on.

Bias

In Node.js, the most widely available CSPRNG is the crypto.randomBytes function, but you shouldn't use this directly, as it's easy to mess up and "bias" your random values - that is, making it more likely that a specific value or set of values is picked.

A common example of this mistake is using the % modulo operator when you have less than 256 possibilities (since a single byte has 256 possible values). Doing so actually makes lower values more likely to be picked than higher values.

For example, let's say that you have 36 possible random values - 0-9 plus every lowercase letter in a-z. A naive implementation might look something like this:

let randomCharacter = randomByte % 36;

That code is broken and insecure. With the code above, you essentially create the following ranges (all inclusive):

• 0-35 stays 0-35.
• 36-71 becomes 0-35.
• 72-107 becomes 0-35.
• 108-143 becomes 0-35.
• 144-179 becomes 0-35.
• 180-215 becomes 0-35.
• 216-251 becomes 0-35.
• 252-255 becomes 0-3.

If you look at the above list of ranges you'll notice that while there are 7 possible values for each randomCharacter between 4 and 35 (inclusive), there are 8 possible values for each randomCharacter between 0 and 3 (inclusive). This means that while there's a 2.64% chance of getting a value between 4 and 35 (inclusive), there's a 3.02% chance of getting a value between 0 and 3 (inclusive).

This kind of difference may look small, but it's an easy and effective way for an attacker to reduce the amount of guesses they need when bruteforcing something. And this is only one way in which you can make your random values insecure, despite them originally coming from a secure random source.

So, how do I obtain random values securely?

In Node.js:

• If you need a sequence of random bytes: Use crypto.randomBytes.
• If you need individual random numbers in a certain range: use crypto.randomInt.
• If you need a random string: You have two good options here, depending on your needs.
  1. Use a v4 UUID. Safe ways to generate this are crypto.randomUUID, and the uuid library (only the v4 variant!).
  2. Use a nanoid, using the nanoid library. This also allows specifying a custom alphabet to use for your random string.

Both of these use a CSPRNG, and 'transform' the bytes in an unbiased (ie. secure) way.

In the browser:

• When using the Node.js options, your bundler should automatically select equivalently safe browser implementations for all of these.
• If not using a bundler:
  • If you need a sequence of random bytes: Use crypto.getRandomValues with a Uint8Array. Other array types will get you numbers in different ranges.
  • If you need a random string: You have two good options here, depending on your needs.
    1. Use a v4 UUID, with the crypto.randomUUID method.
    2. Use a nanoid, using the standalone build of the nanoid library. This also allows specifying a custom alphabet to use for your random string.

However, it is strongly recommended that you use a bundler, in general.

Related: n-digit-token:

This tiny module generates an n-digit cryptographically strong pseudo-random token in constant time and avoids modulo bias.

https://github.com/almasen/n-digit-token

Wrote this

/* eslint-disable no-constant-condition */

const util = require('util');
const assert = require('assert');
const crypto = require('crypto');

const random_bytes = util.promisify(crypto.randomBytes);

/**
 * Random integer between 1 to 256
 *
 * @param {number} max
 * @returns {number}
 */
const random_integer = async (max) => {
  assert(typeof max === 'number');
  assert(Number.isFinite(max) === true);
  assert(Number.isInteger(max) === true);
  assert(max > 1);
  assert(max <= 256);
  const range = max;
  const range_floor = Math.floor(256 / range);
  const range_product = range * range_floor;
  while (true) {
    const bytes = await random_bytes(1);
    const byte = bytes[0];
    if (byte < range_product) {
      let value = byte + 1;
      if (value > range) {
        const remainder = byte % range;
        value = remainder === 0 ? range : remainder;
      }
      return value;
    }
  }
};

process.nextTick(async () => {
  const random_int = await random_integer(256);
  console.log({ random_int });
});

Example distribution

  max: 256,
  range: 256,
  iterations: 100000,
  hits: {
    '1': 363,
    '2': 399,
    '3': 375,
    '4': 439,
    '5': 344,
    '6': 385,
    '7': 379,
    '8': 376,
    '9': 401,
    '10': 343,
    '11': 395,
    '12': 374,
    '13': 391,
    '14': 410,
    '15': 404,
    '16': 429,
    '17': 414,
    '18': 395,
    '19': 397,
    '20': 377,
    '21': 387,
    '22': 394,
    '23': 366,
    '24': 395,
    '25': 401,
    '26': 404,
    '27': 406,
    '28': 386,
    '29': 413,
    '30': 372,
    '31': 367,
    '32': 370,
    '33': 406,
    '34': 383,
    '35': 387,
    '36': 372,
    '37': 377,
    '38': 361,
    '39': 400,
    '40': 407,
    '41': 390,
    '42': 397,
    '43': 383,
    '44': 425,
    '45': 366,
    '46': 390,
    '47': 407,
    '48': 391,
    '49': 394,
    '50': 371,
    '51': 351,
    '52': 380,
    '53': 387,
    '54': 385,
    '55': 405,
    '56': 409,
    '57': 389,
    '58': 366,
    '59': 357,
    '60': 401,
    '61': 424,
    '62': 371,
    '63': 383,
    '64': 420,
    '65': 382,
    '66': 366,
    '67': 430,
    '68': 383,
    '69': 376,
    '70': 394,
    '71': 418,
    '72': 403,
    '73': 405,
    '74': 387,
    '75': 430,
    '76': 422,
    '77': 429,
    '78': 400,
    '79': 386,
    '80': 371,
    '81': 382,
    '82': 350,
    '83': 368,
    '84': 437,
    '85': 357,
    '86': 417,
    '87': 391,
    '88': 379,
    '89': 414,
    '90': 394,
    '91': 406,
    '92': 418,
    '93': 402,
    '94': 367,
    '95': 410,
    '96': 359,
    '97': 380,
    '98': 397,
    '99': 407,
    '100': 374,
    '101': 381,
    '102': 386,
    '103': 430,
    '104': 373,
    '105': 400,
    '106': 433,
    '107': 372,
    '108': 390,
    '109': 385,
    '110': 404,
    '111': 375,
    '112': 368,
    '113': 427,
    '114': 373,
    '115': 377,
    '116': 384,
    '117': 384,
    '118': 394,
    '119': 386,
    '120': 388,
    '121': 384,
    '122': 386,
    '123': 385,
    '124': 401,
    '125': 384,
    '126': 375,
    '127': 370,
    '128': 405,
    '129': 411,
    '130': 370,
    '131': 372,
    '132': 408,
    '133': 390,
    '134': 362,
    '135': 377,
    '136': 364,
    '137': 390,
    '138': 403,
    '139': 410,
    '140': 421,
    '141': 400,
    '142': 395,
    '143': 414,
    '144': 349,
    '145': 390,
    '146': 402,
    '147': 390,
    '148': 389,
    '149': 372,
    '150': 380,
    '151': 440,
    '152': 371,
    '153': 392,
    '154': 405,
    '155': 407,
    '156': 408,
    '157': 373,
    '158': 423,
    '159': 382,
    '160': 387,
    '161': 396,
    '162': 416,
    '163': 375,
    '164': 362,
    '165': 415,
    '166': 398,
    '167': 386,
    '168': 361,
    '169': 391,
    '170': 378,
    '171': 368,
    '172': 427,
    '173': 375,
    '174': 375,
    '175': 394,
    '176': 385,
    '177': 363,
    '178': 396,
    '179': 367,
    '180': 393,
    '181': 393,
    '182': 424,
    '183': 388,
    '184': 397,
    '185': 397,
    '186': 386,
    '187': 407,
    '188': 399,
    '189': 408,
    '190': 389,
    '191': 378,
    '192': 422,
    '193': 361,
    '194': 391,
    '195': 384,
    '196': 380,
    '197': 414,
    '198': 381,
    '199': 414,
    '200': 369,
    '201': 379,
    '202': 382,
    '203': 402,
    '204': 382,
    '205': 400,
    '206': 390,
    '207': 396,
    '208': 414,
    '209': 401,
    '210': 397,
    '211': 401,
    '212': 349,
    '213': 379,
    '214': 397,
    '215': 366,
    '216': 402,
    '217': 408,
    '218': 392,
    '219': 370,
    '220': 356,
    '221': 399,
    '222': 433,
    '223': 411,
    '224': 406,
    '225': 408,
    '226': 384,
    '227': 380,
    '228': 365,
    '229': 379,
    '230': 372,
    '231': 387,
    '232': 392,
    '233': 383,
    '234': 418,
    '235': 434,
    '236': 371,
    '237': 383,
    '238': 386,
    '239': 419,
    '240': 402,
    '241': 403,
    '242': 383,
    '243': 381,
    '244': 381,
    '245': 393,
    '246': 368,
    '247': 419,
    '248': 387,
    '249': 399,
    '250': 391,
    '251': 390,
    '252': 354,
    '253': 410,
    '254': 376,
    '255': 351,
    '256': 412
  }
}

It's quite concerning not to illustrate why those libraries are safer than others. I was looking for the reason why uuid and random-number-csprng are safer.
No one should read this article and just think "well, I'm gonna use these libraries because some post on the internet said so."

@joshxyzhimself what are you attempting to do here? you created 256 random numbers that don't fit in a requested range. These are also biased using the very method this gist aims to solve: transforming numbers using the modulus operator (and/or multiplication/division).

if (byte < range_product) {
      let value = byte + 1;
      if (value > range) {
        const remainder = byte % range;
        value = remainder === 0 ? range : remainder;
      }
      return value;
    }

it still took 100,000 iterations to come up with these, and they don't fit into an input range like: 1-255... they just create the amount of requested randoms... but in what range? looks like you're using a range of 256 to 512, based on the input.... all of these result are above the requested maximum of 256.

@crowjdh there is a very easy way to do this without any libraries. the issue comes when looking for ranges of random, by transforming the random result (as i explained above), rather than simply trying again when the random is not in range. I broke down the topic pretty well above and left a sandbox example https://gist.github.com/joepie91/7105003c3b26e65efcea63f3db82dfba#gistcomment-3595666. Since then I have created a super efficient version that chooses the array size based on the input range.

You need the lib to prevent you from making obvious mistakes, if you understand the right way to implement it, you don't need a lib. Here's my Browser implementation, for node you'll need to rework slightly as the 2 crypto APIs differ slightly (see my comment above for a link to a node sandbox version), all of this free to reuse anywhere:

const getCSPRNG = (min, max) => {
      // validate
      if (min < 0) {
        return console.error("min must be at least 0");
      }
      if (min > max) {
        return console.error("min must be less than max");
      }
      let array;
      // choose the correct array type
      switch (true) {
        case max < 1:
          console.error("max must be at least 1");
          return;
        case max < 256: // 1-255
          array = new Uint8Array(1);
          break;
        case max < 65536: // 256-65535
          array = new Uint16Array(1); // default
          break;
        case max < 4294967296: // 65536-4294967295
          array = new Uint32Array(1);
          break;
        default:
          console.error("max must be no more than 4294967295");
          return;
      }
      let randomNumber = crypto.getRandomValues(array)[0];
      let attempts = 1;
      while (randomNumber > max || randomNumber < min) {
        randomNumber = crypto.getRandomValues(array)[0]; // retry until result is in range
        attempts++; // track failed generations
      }
      console.info({
        randomNumber,
        attempts,
      });
      return randomNumber;
    };

if "crypto safe" isn't exactly what you need, just "better random numbers with less repetition", there is another simpler method below.

again in this case, written for browser crypto api.

// first make math random use crypto instead
    Math.random = crypto.getRandomValues(new Uint32Array(1))[0] / 4294967295;

// typical biased range 
    const getMathRandomRange = (min, max) => {
      min = Math.ceil(min); // minimum is inclusive
      max = Math.floor(max); // maximum is inclusive
      return Math.floor(Math.random() * (max - min + 1)) + min;
    };

For the API keys or secret keys we can use crypto.pbkdf2(secret, salt, iteration, length, algorithm) with just single iteration. We can use different secret and salt according to our needs and different length. As it is cryptographically generated string values, just single iteration is sufficient so performance is not a problem and secret & salt makes it more random and unique.

@bipinstha7 This is bad practice. Never derive "random" keys from any other kind of data - it will never make it more secure than truly random data, all it can do is weaken the unpredictability of the key. "Cryptographically generated" also means nothing if you're using the wrong cryptographic tool for the job.

The only correct way to obtain a secure random value is from a cryptographically-secure random source, not a key derivation function.

@bipinstha7 This is bad practice. Never derive "random" keys from any other kind of data - it will never make it more secure than truly random data, all it can do is weaken the unpredictability of the key. "Cryptographically generated" also means nothing if you're using the wrong cryptographic tool for the job.

The only correct way to obtain a secure random value is from a cryptographically-secure random source, not a key derivation function.

@joepie91 Thank you for correcting me. I wasn't aware of its weakness. Could you please point us about how it weakens the predictability despite having secret key and salt?
UUIDv4 is one of the best and nanoid looks promising. Beyond these tools, what could be the native way for the API keys or secret keys?

@bipinstha7 So there's a general principle in cryptography: never add more complexity than is actually needed to effectively solve the problem. This is because every bit of complexity you add, increases the chance of failure; whether it's because of a bug in the implementation, or because you put the pieces together incorrectly. And because a single tiny cryptographic failure can compromise an entire system, it's crucial to reduce this risk to the absolute minimum.

It's for this same reason that when you see a cryptographic tool advertising itself with how complex it is ("triple hashing", "double encryption", "proprietary algorithm", etc.), you should treat it as completely insecure. When someone presents complexity as a feature, that's a sure sign that they don't understand how to handle risk in cryptography. More about that and related topics here.

But to get back to your case: consider that in your approach, you still need to have a secret key, which needs to be randomly generated, and a salt, which also needs to be randomly generated. So in the end, you still need a random source, you're just adding a bunch of cryptographic complexity on top of that. That means more things that could break.

Consider, for example, what would happen if a weakness were found in the key derivation function that allows someone with 10 outputs (with 10 different salts) to determine the original secret key. Suddenly, your key security got a lot weaker; a problem that would not have existed if you had just used a CSPRNG with a simple encoding scheme.

You're not avoiding the encoding scheme either - you say that it produces "string values", but it doesn't, at least not originally! Like all cryptographic operations, PBKDF2 operates on series of bytes, and produces a series of bytes too. It likely just gets encoded (hex-encoded, base64-encoded, etc.) before it is returned to you.

So why not just take a random string from the CSPRNG, and encode it in some simple and non-biased way, completely skipping the whole "cryptographic key derivation with a salt" dance inbetween those two steps? And that is indeed exactly what UUID and nanoid implementations do.

You may also find this a useful read.

@joepie91 Thank you, that's precious information. You cleared the misunderstanding I had with cryptography. Thank you :)

@Joepie I'm not able to derive the same percentages that you quote regarding the relative chance of generating a certain number mod 36:

2.64% chance of getting a value between 4 and 35 (inclusive), there's a 3.02% chance of getting a value between 0 and 3 (inclusive).

By my calculation the numbers should be a ~2.73% chance of getting a value between 4 and 35 (inclusive) and a ~3.13% chance of getting a value between 0 and 3 inclusive.

I calculate that there for 0-3 range, we can say there are 8 chances of getting that value out of a total of 256 or 0.03125, and for the 4-35 range we have 7 chances out of 256 or 0.02734375.

This is also borne out empirically by this test code:

const crypto = require("crypto")

// Create an array of 0's to hold a count of the number of times a given number is observed
counts = [];
for (let i = 0; i < 36; i++) {
    counts[i] = 0;
}

// Do insecure modulo method NUM_OBSERVATIONS_TO_MAKE times and count how many times each number is observed
const NUM_OBSERVATIONS_TO_MAKE = 1000000
for (let i = 0; i < NUM_OBSERVATIONS_TO_MAKE; i++) {
    const randomByte = crypto.randomBytes(1).readUInt8(0);
    const randomNumber = randomByte%36;
    counts[randomNumber]++;
}

// Calculate the ratio of the number of times a number was observed out of the total number of observations made
const ratios = counts.map(c => c / NUM_OBSERVATIONS_TO_MAKE)
console.log(ratios[0]) // Approximately 0.03125
console.log(ratios[5]) // Approximately 0.02734375

Very possible I'm missing something about we are supposed to derive these numbers!

I also realize that this doesn't affect your higher level points, but I think accurate numbers are also important.

Thank you for taking the time to put this post together!

import crypto from 'crypto';
crypto.randomUUID();

https://developer.mozilla.org/en-US/docs/Web/API/Crypto/randomUUID

I'm thinking of using 32 bytes randomBytes for raffle tickets. Is it safe to use? I will order them and take first 100. My concern is can ordering make it biased?