If you are like many (most?) of us, you have encountered Rails
Credentials/Secrets and secret_key_base and may have been
left a bit (or more) confused.
This post is an attempt to remove some of that confusion by taking a comprehensive look at Rails Secrets and Credentials.
This post covers Rails Secrets and Rails Credentials, how
they are different, and their history, as well as the different
Rails secret_key_bases. Finally, it presents an approach
for setting the secret_key_base used by Rails without using
Rails Secrets or Credentials files.
🚢 Captain Obvious says that the general term "secrets" is used to describe those configuration values that are sensitive like usernames, passwords, and API keys
⏱️This post is current as of and based upon Rails 7.0.4.2
One of the first areas of potential confusion is that Rails
Credentials and Secrets are two different and separate things
that can be interconnected in the case of the Rails
secret_key_base (which will be covered later in this post).
To better understand why Rails Credentials and Secrets both exist, consider their history and the need for some backward compatibility.
-
Rails 4.1 (2Q2014) introduced the automatic generation and use of the file
config/secrets.ymlwhich contained the application'ssecret_key_baseby default. The secrets in this file were made accessible throughRails.application.secrets(e.g.Rails.application.secrets.secret_key_base). This file was not encrypted and should not be checked into the source code repository or made public. -
Rails 5.1 (2Q2017) introduced Encrypted Secrets (based on the
sekretsgem). Runningbin/rails secrets:setupcreated the encrypted secrets fileconfig/secrets.yml.encand the encryption key fileconfig/secrets.yml.key(instead of "always going [to] theRAILS_MASTER_KEY"). This is also when the requirement for the specification of a master key was added in order to start the Rails server. The encryptedconfig/secrets.yml.encfile could now be checked into the source code repository, but the plain textconfig/secrets.yml.keyfile should not be. For more information, see the PR. -
Rails 5.2 (2Q2018) introduced (encrypted) Credentials and the automatically generated files
config/credentials.yml.encfor the encrypted credentials andconfig/master.keyfor the encryption key. Per the release notes, "[t]his will eventually replaceRails.application.secretsand the encrypted secrets introduced in Rails 5.1". Like the similarconfig/secrets.yml.enc, the encryptedconfig/credentials.yml.enccould be checked into the source code repository (but not the unencryptedconfig/master.key). For more information, see the PR. -
Rails 6.0 (3Q2019) introduced support for Multi-Environment Credentials with environment-specific files
config/credentials/<rails-env>.yml.encfor the encrypted credentials andconfig/credentials/<rails-env>.keyfor the encryption key (e.g.config/credentials/production.yml.encandconfig/credentials/production.key). Also introduced were the configuration settingsconfig.credentials.content_pathandconfig.credentials.key_pathfor overriding the default locations of the credentials and key files. For more information, see the PR.
🚫 Rails Secrets are deprecated as of Rails 5.2 and are only supported for backwards compatibility. If you want Rails to manage your sensitive configuration, you should use Rails Credentials.
Rails Secrets were introduced in Rails 4.1 and Encrypted Secrets were introduced in Rails 5.1. As of Rails 7.0.4.2 both are still (somewhat) supported.
-
Unencrypted secrets use the file
config/secrets.ymlto store the secretsdevelopment: secret_key_base: 49d3f3de9ed86c74b94ad6bd0... my_custom_secret: some-value test: secret_key_base: 707db01ac0f0d50d62c14a3ce... my_custom_secret: some-value production: secret_key_base: 4795c8bffcdf8e54b327f117d... my_custom_secret: some-value
Note that the same file is used to define the secrets for all environments and that you can define your own application-specific secrets in it.
Since it is not encrypted, this file should never be checked into a source code repository or made public.
-
Encrypted secrets use the two files...
-
config/secrets.yml.encto store the encrypted secretsNote that like
config/secrets.yml, this same file is used to define the secrets for all environments and that you can define your own application-specific secrets in it.Since this secrets file is encrypted, you could check it into a source code repository.
-
config/secrets.yml.keyto store the unencrypted (i.e. plain text) key used to encrypt/decrypt the encrypted secrets inconfig/secrets.yml.encSince this key file is not encrypted, this file should never be checked into a source code repository or made public.
The environment variable
RAILS_MASTER_KEYcan also be used to specify the encryption key instead of theconfig/secrets.yml.keyfile. -
Encrypted secrets are not active by default in Rails 5.1. To use encrypted secrets in Rails 5.1, add the following configuration to your application...
config.read_encrypted_secrets = trueTo create your encrypted secrets in Rails 5.1, run the following command...
bundle exec bin/rails secrets:setup
This will generate the two files config/secrets.yml.enc and
config/secrets.yml.key.
The generated config/secrets.yml.enc will be empty, so you
must edit it to add your secrets.
👉 Note that with the Rails 5.2 introduction of encrypted credentials the
bin/rails secrets:setupcommand is deprecated and can no longer be used. It returns the messageEncrypted secrets is deprecated in favor of credentials. Run:rails credentials:help
In order to edit your encrypted secrets you must have...
- The key used to create the encrypted secrets file either
in file
config/secrets.yml.keyor in the environment variableRAILS_MASTER_KEY - An editor set in the system environment variable
EDITOR(e.g.EDITOR=vim)
To edit your encrypted secrets file config/secrets.yml.enc,
run the following command...
bundle exec bin/rails secrets:edit
You can use both unencrypted secrets in config/secrets.yml
and encrypted secrets in config/secrets.yml.enc and Rails
will merge them together during initialization.
Secrets can be accessed inside of your application (or in the Rails console) at...
Rails.application.secretsFor example, Rails.application.secrets.secret_key_base
Rails.application.secrets contains the merged secrets from
config/secrets.yml and config/secrets.yml.enc.
For more information on Rails Secrets...
- Source Code
- Rails 4.1 with Manual Generation of Secrets Release Notes
- Encrypted Secrets Pull Request
- Rails 5.1 with Encrypted Secrets Release Notes
- Engine Yard article on Encrypted Rails Secrets on Rails 5.1
Rails Credentials were introduced in Rails 5.2 and multi-environment credentials were introduced in Rails 6.
Rails Credentials are the preferred and fully-supported mechanism if you want Rails to manage your sensitive configuration.
Rails Credentials are always encrypted and are generated
automatically when creating a rails project with rails new.
Like Rails Encrypted Secrets, Rails Credentials use two types of files...
-
The encrypted credential file(s):
-
config/credentials.yml.encwhich is the generated default shared across all rails environments# aws: # access_key_id: 123 # secret_access_key: 345 # Used as the base secret for all MessageVerifiers in Rails, including the one protecting cookies. secret_key_base: 14b6462b8dc4366e1266baab0...
-
config/credentials/<rails-env>.yml.encwhich are for Multi-Environment Credentials and are specific to a given Rails environment (e.g.config/credentials/production.yml.enc)
Like Rails Secrets, you can define your own application-specific credentials in this file type and check this file type into your source code repository.
👉 Note that Embedded RuBy (ERB) is not supported in the Rails encrypted credential files
-
-
The encryption key file(s) containing the unencrypted (i.e. plain text) encryption key(s) used to encrypt (and decrypt) the credential file(s):
-
config/master.keywhich is the generated default shared across all rails environments -
config/credentials/<rails-env>.keywhich are for multi-environment credentials and are specific to a given Rails environment (e.g.config/credentials/production.key)Like Rails Encrypted Secrets, this key file is not encrypted and should never be checked into a source code repository or made public.
-
Like Rails Encrypted Secrets, the environment variable
RAILS_MASTER_KEY can also be used to specify the
encryption key instead of the config/master.key file.
👉 Note that the
RAILS_MASTER_KEYenvironment variable takes precedence over the credentials key file
Encrypted credentials are active and generated by default in Rails versions 5.2 and later and no additional configuration is needed.
However, for Rails version 6.0 and later, you can configure the locations of the credential and key files...
-
config.credentials.content_pathspecifies the location of the encrypted credentials file (e.g.config.credentials.content_path = config/devtest.credentials.yml.enc) -
config.credentials.key_pathspecifies the location of the credential's encryption key file (e.g.config.credentials.key_path = config/devtest.key)
You can require the use of a master key with the configuration...
config.require_master_key = trueThis master key requirement is the default in production and
does not seem to be able to be disabled.
As mentioned in Rails versions 5.2 and later, the Rails
credential files config/credentials.yml.enc and
config/master.key are created automatically by rails new.
The rails credentials:edit command is used to both
edit existing credentials and create new ones. When
editing existing credentials, you must supply the encryption
key either in the appropriate key file or with the
RAILS_MASTER_KEY environment variable.
If you lose your encryption key, you can regenerate your credentials. However, you will have to delete your encrypted credential file and readd your application-specific credentials.
To regenerate your credential files...
- Delete
config/master.keyif it existsrm -f config/master.key - Delete
config/credentials.yml.encrm -f config/credentials.yml.enc - Ensure that the
EDITORsystem environment variable is set (e.g.EDITOR="code --wait") or specify it on the command line - Run the
rails credentials:editcommand to generate the new credential files
or to specify the editor to use (for examplebundle exec bin/rails credentials:editvim) on the command lineEDITOR=vim bundle exec bin/rails credentials:edit
👀 For more information on the
rails credentials:editcommand run the command...bundle exec bin/rails credentials:edit -h
To create and edit credentials for a specific environment
use the rails credentials:edit command with the
--environment option to specify the desired environment.
Again, you will need to either have the EDITOR system
environment variable already set or specify it on the command
line.
For example to generate credentials for the production
environment...
EDITOR=vim bundle exec bin/rails credentials:edit --environment production
This will generate the two credential files
config/credentials/production.yml.enc and
config/credentials/production.key.
🤷 Per Pull Request 47107 it appears that the Rails Credentials commands (e.g.
rails credentials:edit) use theconfig.credentials.content_pathandconfig.credentials.key_pathconfiguration for the specified environment
Multi-environment credentials take precedence over the default credentials and there is no "fall back" or merging.
If you want to use Multi-environment credentials and use the
default credential files for an environment, you will need
to use the config.credentials.content_path and
config.credentials.key_path configuration to specify them
for that environment. For example in
config/environments/development.rb
Rails.application.configure do
...
config.credentials.content_path = 'config/credentials.yml.enc'
config.credentials.key_path = 'config/master.key'
...Credentials can be accessed inside of your application (or in the Rails console) at...
Rails.application.credentialsFor example, Rails.application.credentials.secret_key_base
To display the contents of your Rails Credentials file(s),
use the rails credentials:show command...
bundle exec bin/rails credentials:show
For more information on Rails Credentials...
- Source Code
- Rails Credentials Pull Request
- Rails 5.2 with Rails Credentials Release Notes
- Rails Multi-Environment Credentials Pull Request
- Rails 6.0 with Multi-Environment Credentials Release Notes
- Rails Guide on Securing Rails Applications
- Big Binary article on Rails 6 adds support for Multi Environment credentials
- Ruby on Rails discussion on The status of simultaneous support for secrets, encrypted credentials and multi-environment encrypted credentials is confusing
There is always at least one sensitive configuration key and
value which is automatically generated by Rails when it
generates Secrets or Credentials. This sensitive
configuration is the Rails secret_key_base.
According to the
Ruby on Rails API,
the Rails secret_key_base "is used as the input secret to
the application's key generator, which in turn is used to
create all ActiveSupport::MessageVerifier and
ActiveSupport::MessageEncryptor instances, including the
ones that sign and encrypt cookies."
What can be very confusing is that there are five
different secret_key_bases:
-
There is the
Rails.application.secret_key_basemethod which is the one used by Rails for its key generator (ActiveSupport::MessageVerifierandActiveSupport::MessageEncryptorinstances) -
There is a
secret_key_baseattribute for Rails Credentials (i.e.Rails.application.credentials.secret_key_base) -
There is a
secret_key_baseattribute for Rails Secrets (i.e.Rails.application.secrets.secret_key_base) -
There is the
Rails.application.config.secret_key_baseconfiguration -
Finally, there is the Rails environment variable
SECRET_KEY_BASE(i.e.ENV['SECRET_KEY_BASE'])
Some of these secret_key_bases are interconnected and some
are completely separate.
The following analysis of these different secret_key_bases
is based upon inspecting the
Rails source code
and actual structured exploratory testing as of Rails 7.0.4.2
and "edge" (pre Rails 7.1) circa 1Q2023.
Rails.application.secret_key_base is the method actually
used by Rails itself for its encryption of things like
cookies. It is described in the
Ruby on Rails API.
Generally, this is the one that you should use in your application. It is also the most complicated and its value depends directly on the...
- Rails environment
- Value in the automatically generated temporary file
tmp/development_secret.txt - Value in
ENV['SECRET_KEY_BASE'] - Value in
Rails.application.credentials.secret_key_base - Value in
Rails.application.secrets.secret_key_base
Given its direct dependencies, the value of
Rails.application.secret_key_base also indirectly depends
on the value of the Rails.application.config.secret_key_base
configuration.
Specifically and in order of precedence, the value of
Rails.application.secret_key_base is determined as
follows...
-
In the Rails
developmentortestenvironments,Rails.application.secret_key_basereturns...- The value of
Rails.application.secrets.secret_key_baseif it is set for that environment in the fileconfig/secrets.yml.*, for example...development: secret_key_base: from-secrets-yml-development
- Otherwise, the value in the temporary file
tmp/development_secret.txtand if that file does not exist, Rails automatically creates it with a value
🤦 As a side-effect of initialization, if there is not a value for the environment in
config/secrets.yml.*, Rails setsRails.application.secrets.secret_key_baseto the value intmp/development_secret.txt.👉 Note that the
Rails.application.secret_key_basemethod does not use Rails Credential at all in thedevelopmentortestenvironments. - The value of
-
In any other Rails environment other than
developmentortest(e.gproduction),Rails.application.secret_key_basereturns...- The value in the Rails
SECRET_KEY_BASEenvironment variable (i.e.ENV['SECRET_KEY_BASE']) if it is set - Otherwise, the value in
Rails.application.credentials.secret_key_baseif set from the Rails Credentials files (with any multi-environment credential file taking precedence) - Otherwise, the value in
Rails.application.secrets.secret_key_baseif set from the Rails Secrets files
💥 If none of these sources have a value, then Rails raises the following error...
Missing `secret_key_base` for 'production' environment, set this string with `bin/rails credentials:edit` (ArgumentError) - The value in the Rails
The Rails.application.credentials.secret_key_base attribute
is set from the Rails Credentials files described previously
in this post.
Specifically and in order of precedence, the value of
Rails.application.credentials.secret_key_base is determined
as follows...
- If configured, from the encrypted credentials file specified
in
config.credentials.content_path - Otherwise, from the encrypted credentials file
config/credentials/<rails-env>.yml.encif it exists - Otherwise, from the default encrypted credentials file
config/credentials.yml.encif it exists - Otherwise it is set to
nil
The key used to decrypt the value for the
Rails.application.credentials.secret_key_base just described
is determined as follows in order of precedence...
- From the value in the Rails
RAILS_MASTER_KEYenvironment variable (i.e.ENV['RAILS_MASTER_KEY']) if set - Otherwise, if configured, from the key file specified in
config.credentials.key_path - Otherwise, from the key file
config/credentials/<rails_env>.keyif it exists - Otherwise, from the default credentials key file
config/master.keyif it exists
💥 If using Rails Credentials for the value of
the Rails.application.secret_key_base method, you must
have a master key value otherwise Rails will raise a
MissingKeyError. You may encounter this issue in the
Rails production environment during Asset Precompile.
🤷 Since I was using Rails 7.0.4.2 which no longer supports creating Rails Encrypted Secrets, I was unable to test specific behavior associated with them versus unencrypted secrets
Unlike the Rails.application.credentials.secret_key_base
attribute, the Rails.application.secrets.secret_key_base
attribute is not set from just the Rails Secrets files.
Thus, Rails.application.secrets.secret_key_base is more
complex and configurable.
As mentioned previously in this section, in the Rails
development and test environments, the value of
the Rails.application.secrets.secret_key_base is
interconnected with the value of the
Rails.application.secret_key_base method.
Specifically and in order of precedence, the value of
Rails.application.secrets.secret_key_base is determined as
follows...
-
In the Rails
developmentortestenvironments...- If present, from the environment configuration in the
secrets file(s)
config/secrets.yml.*, for example...development: secret_key_base: from-secrets-yml-development
- Otherwise, if present, from the value specified in
Rails.application.config.secret_key_baseconfiguration - Otherwise, from the value in the Rails-generated
temporary file
tmp/development_secret.txt
- If present, from the environment configuration in the
secrets file(s)
-
In any other Rails environment other than
developmentortest(e.gproduction)...- If present, from the environment configuration in the
secrets file(s)
config/secrets.yml.* - Otherwise, if present, from the value specified in
Rails.application.config.secret_key_baseconfiguration - Otherwise it is set to
nil
- If present, from the environment configuration in the
secrets file(s)
👉 Recall that with Rails Secrets files, the encrypted secrets file takes precedence and is merged with the unencrypted secrets file
The Rails.application.config.secret_key_base configuration
is simply a setting in the Rails application configuration.
👀 For more information, see Configuring Rails Applications in the Rails Guide
This is simply the Rails-specific environment variable
for specifying a secret_key_base value referenced
inside of a Rails application by ENV['SECRET_KEY_BASE'].
You may have one or more good reasons for not using Rails Secrets or Credentials for your application's sensitive configuration, for example...
-
Your application's sensitive configuration is managed by another team and/or secrets manager such as Hashicorp Vault, AWS Secrets Manager, Azure Key Vault, or Kubernetes Secrets
-
Your applications follows the twelve-factor methodology as described in the Twelve-Factor App where configuration is stored in environment variables
-
You want less complexity and similar behavior across all Rails environments
While you have complete control over your application-specific
"secrets", you will have to accommodate and address the
Rails.application.secret_key_base method in order to even
start your application (i.e. Rails server) in the Rails
production environment and/or precompile assets.
Here is an approach for addressing
Rails.application.secret_key_base consistently in all
Rails environments without using or needing Rails Secrets
or Credentials. Here you will use the Rails SECRET_KEY_BASE
environment variable to specify the value.
- Remove any custom
config.credentials.content_pathandconfig.credentials.key_pathconfiguration being sure to note the custom locations of your credential files - Delete all Rails Credentials files, for example...
rm -rf config/credentials* rm config/master.key - Delete all Rails Secrets files
rm config/secrets.yml* - Configure your Rails application for all environments to
set
Rails.application.config.secret_key_basefromENV['SECRET_KEY_BASE'], for example in fileconfig/application.rbHere you are usingclass Application < Rails::Application ... # Do not use Rails Secrets or Credentials for secret_key_base # Set the secret_key_base used by Rails key generators from ENV config.secret_key_base = ENV.fetch('SECRET_KEY_BASE') ... end
ENV.fetchwithout a default value or block so that an obvious error occurs if theSECRET_KEY_BASEenvironment variable is not set.
Here are some Rails console sessions, showing the different
secret_key_bases' values using this approach and how they
are the same for all environments...
-
For the Rails
development(andtest) environment...$ RAILS_ENV=development SECRET_KEY_BASE='from-env-var' bundle exec bin/rails c Loading development environment (Rails 7.0.4.2) >> Rails.application.secret_key_base => "from-env-var" >> Rails.application.credentials.secret_key_base => nil >> Rails.application.secrets.secret_key_base => "from-env-var" >> Rails.application.config.secret_key_base => "from-env-var" >> ENV['SECRET_KEY_BASE'] => "from-env-var"
-
For the Rails
productionenvironment...$ RAILS_ENV=production SECRET_KEY_BASE='from-env-var' bundle exec bin/rails c Loading production environment (Rails 7.0.4.2) >> Rails.application.secret_key_base => "from-env-var" >> Rails.application.credentials.secret_key_base => nil >> Rails.application.secrets.secret_key_base => "from-env-var" >> Rails.application.config.secret_key_base => "from-env-var" >> ENV['SECRET_KEY_BASE'] => "from-env-var"
Finally, if you need to generate an encryption key like or for
secret_key_base, you can use the rails secret command...
bundle exec bin/rails secret
🤦 Unfortunately, it appears through testing that the
rails secret command requires secret_key_base. So if you
are using the approach presented in
How to NOT Use Rails Secrets or Credentials, you will need
to supply a valid dummy value for the SECRET_KEY_BASE
environment variable.
Here is a janky workaround if you want to dynamically generate
an encryption key for the SECRET_KEY_BASE environment
variable on the command line...
export SECRET_KEY_BASE="a-chicken-to-lay-the-first-egg"; SECRET_KEY_BASE=$(bundle exec bin/rails secret) bundle exec bin/rails c
This brings a lot of light in all the details of this API. Thanks for the outstanding explanation!