Skip to content

Instantly share code, notes, and snippets.

@twirrim
Created November 15, 2024 20:27
Show Gist options
  • Save twirrim/a26ae4f60fff3d4180d48a48d4ec5c2f to your computer and use it in GitHub Desktop.
Save twirrim/a26ae4f60fff3d4180d48a48d4ec5c2f to your computer and use it in GitHub Desktop.
Making Random Data really Fast

I stumbled across this trick elsewhere several years ago, but I don't remember where. I needed to generate 10TiB of hard-to-compress data, and /dev/urandom is somewhat slow (400-ish MB/s)

Most major CPUs over the past several years have acceleration for AES-256 ciphers. So one way to produce lots of random-ish, hard to compress data is to leverage openssl.

For OpenSSL 1.1 onwards, using current epoch as the seed:

$ openssl enc -aes-256-ctr -pbkdf2 -pass pass:"$(date '+%s')" < /dev/zero

Note you could replace that call out to date with a straight password instead, but you'll always get the same data each time out the other side:

$ openssl enc -aes-256-ctr -pbkdf2 -pass pass:"pass" < /dev/zero

According to my notes the original version used to feed in some data from /dev/urandom piped through base64 as the seed, but that feelse like overkill to me:

$ openssl enc -aes-256-crt -pbkdf2 -pass pass:"$(dd if=/dev/urandom bs=128 count=1 2>/dev/null | base64)" < /dev/zero

The original version used to use the -nosalt argument, I'm not sure that provides value either.

Note: On older versions of OpenSSL you may need to drop the -pbkdf2 argument to get it to work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment