Django Configuration Loader Strawman

strawman.md

A generic configuration loader for Django. Configuration is the term I’m using for environment-specific settings values. Most settings are static and don’t change from environment-to-environment, but some like DATABASES or SECRET_KEY should.

Inspired by dataclasses and Pydantic.

Users should define a class which inherits from a Configuration class:

class Config(Configuration):
    DEBUG: bool = False  # value with default
    SECRET_KEY: str      # value with no default (raises an error if not provided)
    ALLOWED_HOSTS: list[str] = field(description="A list of domains allowed to access the site")

A field function can be used to add additional functionality that can’t be expressed via Python type hints and assignments.

Instantiating the config class will load the variables from an external source. A list of loader classes should be provided which define how to fetch the values from the external sources. Variables could be loaded from environment variables with:

config = Config(loaders=[EnvironmentConfigLoader()])

Loader classes can take additional arguments that define how the loader should behave. For example an environment variable loader might take a prefix argument which looks for environment variables under a given name prefix. For example:

config = Config(loaders=[EnvironmentConfigLoader(prefix="DJANGO_")])

Loader classes can be stacked where values loaded later in the list take precedence over earlier loaders. Imagine a DockerSecretsLoader, you might have:

config = Config(loaders=[EnvironmentConfigLoader(), DockerSecretsLoader()])

In this case, any values defined in environment variables would be overridden by Docker secrets (if the value exists there).

Once instantiated, the loaded values can be accessed as you’d expect via the class attributes. For example, in settings.py you would have:

DEBUG = config.DEBUG

Motivation

Django’s default settings encourages multiple settings files and hard-coding potentially secret values in those files.

Best practice is to keep those values in a secure storage that is loaded at startup. In practice, environment variables are the most common method here (even though security is debatable). Django should encourage the use of these systems by providing an API like it does for storages, databases, etc.

Additionally, Django does not ship with a production-ready configuration. It should ship with configuration optimized for both development and production to promote good security practices and facilitate the deployment process. Some sort of “toggle” is needed to flip between the two modes. NODE_ENV and RAILS_ENV are examples of this in other ecosystems. This would hopefully pave the way for providing more secure defaults in the future.

Notes

I expect using type hints to be controversial. Not sure if it is more important to follow the pattern of Django models or to do what’s more common in “modern” Python. These wouldn't be needed for most loaders and could be omitted entirely if the user preferred to have everything cast a string.

It would be nice if you could define loaders at execution time. That would make projects more portable. We have DJANGO_SETTINGS_MODULE as an example of this style behavior, but not sure if there’s an appetite to do more of that.

Hey @ipmb — this looks great — the field() option is good. Two initial reactions:

1. If we can’t use type hints in this kind of case, we can’t use them anywhere. Surely everyone is using data classes by now… — I think we should make that case.
2. On implementation I’d quite like to explore a decorator based approach (extending attrs maybe) before settling on inheritance. I know everything in Django uses inheritance, but that’s just because it’s 20 years old. A decorator (composition) based approach would be good to explore. (But that’s to get into details already, which is a buying sign, not an objection.)

I have no concerns with a decorator approach 👍