Olivier Mehani: Optional Docker services and dependencies

Like many, have become my go-to tool to create software that can be conveniently deployed to production with a limited amount of headache. However, many tasks, and sometimes whole services, pertain only to the development side of the workflow, and need to stay there.

Moreover, some tasks, such as time-consuming provisioning tasks, are only on-demand one-offs. They shouldn’t run at all most of the time, but they should slot into the dependency graph correctly when needed.

tl;dr: I realised that . Profiles are also useful to package actions and triggers to run on demand, so they are not started by default.

We can start with a simple setup where our long-running main service depends on an init service to perform preliminary steps. This can be setup with depends_on the compose.yaml.

services:

  main:

    image: debian:latest

    command: "sh -c 'while : ; do echo main; sleep 10; done"

    depends_on:

      init:

        condition: service_completed_successfully



  init:

    image: debian:latest

    command: sh -c 'echo init; sleep 10'

Even when run ning the main container, we get the right dependency (and delay). So far so good (though up will show the output from all containers.

But what if we have another, much more time consuming, initialisation step?

services:

   [...]

   opt-init:

    image: debian:latest

    command: sh -c 'echo opt-init; sleep 100'

Perhaps we are lucky, and while it needs to run once, we don’t need it to run everytime (think: database setup).

.

services:

  main:

    [...]

      opt-init:

        condition: service_completed_successfully

        required: false

And that’s really all there is to it: with the right profile, the optional dependency is started in the desired order, but its absence is otherwise transparently ignored. Both docker compose up and docker compose --profile opt work as desired.

Profiles afford us another useful trick: on-demand tasks not started by default. This can be handy for maintenance tasks (data cleanup, garbage collection, …) or test scripts (running test workload, sending message, …). Those are handy during development, but would not be necessary, or take a different form, in other deployments.

services:

  [...]

  say-hello:

    image: debian:latest

    profiles:

      - hello

    command: echo hello

    depends_on:

      main:

        condition: service_started

Conveniently, when explicitly running a service, it is not necessary to request a matching profile, keeping the command line lean: docker compose run say-hello.

So here we are. Compose profiles allow us to control which services get started, and mark some as conditional. This, coupled with the ability to mark some depends_on rules as not required is a good way to seamlessly prevent heavy or otherwise time consuming services from starting when not needed, while retaining proper dependency ordering when enabled.

For completeness, the full, final, compose.yaml looks as follow.

services:

  main:

    image: debian:latest

    command: "sh -c 'while : ; do echo main; sleep 10; done'"

    depends_on:

      init:

        condition: service_completed_successfully

      opt-init:

        condition: service_completed_successfully

        required: false



  init:

    image: debian:latest

    command: sh -c 'echo init; sleep 10'



  opt-init:

    image: debian:latest

    profiles:

      - opt

    command: sh -c 'echo opt-init; sleep 100'



  say-hello:

    image: debian:latest

    profiles:

      - hello

    command: echo hello

    depends_on:

      main:

        condition: service_started

The post .