Understanding compose profiles

ComposeProfiles.md

IOTstack: Understanding compose profiles

You are probably familiar with running a command like:

$ docker-compose up -d

You probably expect that command to bring up a container for every service definition mentioned in your docker-compose.yml. That's its default behaviour.

Compose profiles are a very useful way of grouping service definitions into sets so that you can achieve more fine-grained control. The classic example is a set of containers for "production" and another set for a "test" environment.

Adding profiles to your compose file

Consider the following snippet from a docker-compose.yml:

services:

  container1:
    container_name: container1
    image: something
    …
    
  container2:
    container_name: container2
    image: something
    profiles:
      - production
    …

  container3:
    container_name: container3
    image: something
    profiles:
      - test
    …

  container4:
    container_name: container4
    image: something
    profiles:
      - production
      - test
    …

The only real difference between those service definitions is container1 does not have a profiles: clause while the others are members of "production" and/or "test" profiles.

Key point:

• profile names are just strings. There is nothing magical about "production" or "test". They are not predefined names with their own special meanings. You can invent whatever names you need.

Controlling "up" and "down" commands with profiles

In any service definition, the effect of a profiles: clause is governed by the value of the COMPOSE_PROFILES environment variable. Here are some examples:

1. When COMPOSE_PROFILES has not been set:

   $ unset COMPOSE_PROFILES
   $ docker-compose up -d

   ☞   Only container 1 will come up.

   It is not usually necessary to "unset" the variable explicitly. This is just making the point that, when the environment variable is undefined, only containers without a profiles: clause will be affected.

   $ docker-compose down

   ☞   Container 1 will be taken down.

2. When COMPOSE_PROFILES is set to "production":

   $ export COMPOSE_PROFILES=production
   $ docker-compose up -d

   ☞   Containers 1, 2 and 4 will come up.

   $ docker-compose down

   ☞   Containers 1, 2 and 4 will be taken down.

3. When COMPOSE_PROFILES is set to "test":

   $ export COMPOSE_PROFILES=test
   $ docker-compose up -d

   ☞   Containers 1, 3 and 4 will come up.

   $ docker-compose down

   ☞   Containers 1, 3 and 4 will be taken down.

4. When COMPOSE_PROFILES is set to both "production" and "test":

   $ export COMPOSE_PROFILES=production,test
   $ docker-compose up -d

   ☞   All four containers will come up.

   $ docker-compose down

   ☞   All four containers will be taken down.

5. When COMPOSE_PROFILES is set to some other value:

   $ export COMPOSE_PROFILES=neitherProductionNorTest
   $ docker-compose up -d

   ☞   Only container 1 will come up.

   $ docker-compose down

   ☞   Container 1 will be taken down.

   In other words, the case where COMPOSE_PROFILES has a value that is not found in any profiles: clause is the same as when the variable is undefined.

Mix and match

$ export COMPOSE_PROFILES=test
$ docker-compose up -d

☞   Containers 1, 3 and 4 will come up.

$ export COMPOSE_PROFILES=production
$ docker-compose up -d

☞   Container 2 will come up as well.

$ export COMPOSE_PROFILES=test
$ docker-compose down

☞   Containers 1, 3 and 4 will be taken down but container 2 will stay up.

$ export COMPOSE_PROFILES=production
$ docker-compose down

☞   Container 2 will be taken down.

One-shots

The earlier examples follow a two-line pattern:

$ export COMPOSE_PROFILES=something
$ docker-compose …

A side-effect of using two-line syntax is that COMPOSE_PROFILES remains set until you change it again (or logout).

Suppose the starting position is:

$ export COMPOSE_PROFILES=production
$ docker-compose up -d

The result is containers 1, 2 and 4 are running.

Now execute the following:

$ COMPOSE_PROFILES=test docker-compose up -d

Container 3 comes up. However:

$ echo $COMPOSE_PROFILES
production

In other words, with the one-line syntax, COMPOSE_PROFILES only changes to "test" for the duration of the associated command, and reverts to its prior value once the command completes.

Overriding the effect of profiles

Assume:

$ export COMPOSE_PROFILES=production
$ docker-compose up -d

By now, you should understand that containers 1, 2 and 4 will be running.

Suppose you want to start container 3 as well. You could follow the earlier example of setting COMPOSE_PROFILES=test but you can also just name the container on the "up" command:

$ docker-compose up -d container3

Key point:

• docker-compose up -d ignores COMPOSE_PROFILES when you name a container.

Of course, COMPOSE_PROFILES=production will still be in effect when you issue the "down" command:

$ docker-compose down

so container 3 will be left running. Once again, you have a choice. You could set COMPOSE_PROFILES=test. Alternatively, you could use docker commands to stop the container, by name:

$ docker stop container3
$ docker rm container3

Key point:

• docker-compose rm respects COMPOSE_PROFILES so this is one situation where neither of the following commands will work if the named container has a profiles: clause and it does not include a matching profile name:

   $ docker-compose rm --force --stop -v container3
   $ TERMINATE container3

  TERMINATE is just an alias to docker-compose rm so those are actually the same command.

Other possibilities

The way I use profiles is to add the following line to my .bashrc:

export COMPOSE_PROFILES=$(hostname -s)

On Raspbian, you can use $HOSTNAME rather than $(hostname -s). It's just that the second form also works on macOS where $HOSTNAME contains the fully-qualified domain name.

Then, my profiles: clauses list of machine names:

profiles:
- iot-hub
- test-hub
- …

I can use a single docker-compose.yml across all my hosts and then add/remove host names from profiles: clauses to control which containers come up on each machine.

Reference

• docs.docker.com/compose/profiles