Docker Compose – Profiles, Overrides and Multiple Files

The demo API can use Elasticsearch for product search, a message broker for order events, and an SMTP sink for receipt emails. None of them is required to run the application.

If they are all in the compose file, docker compose up starts four containers and several gigabytes of JVM heap to serve a pizza menu. If they are not in it, there is nowhere to record how to run them. Profiles are the answer to that.

Profiles

  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:9.1.0
    container_name: pizza-elasticsearch
    profiles: ["search", "all"]

A service with a profiles key does not start unless one of its profiles is requested. A service without one always starts.

docker compose up -d                          # MySQL only
docker compose --profile search up -d         # + Elasticsearch
docker compose --profile messaging up -d      # + the Artemis broker
docker compose --profile mail up -d           # + Mailpit
docker compose --profile all up -d            # everything

Listing two profiles — ["search", "all"] — means the service is included by either, which is how all works without repeating anything.

The naming is worth stealing: each compose profile is named after the Spring profile that needs it. So the two commands line up, and nobody has to remember a mapping:

docker compose --profile search up -d
./mvnw spring-boot:run -Dspring-boot.run.profiles=local,search

The same mechanism in the full-stack file keeps the second frontend out of the default build:

  web-angular:
    profiles: ["angular"]

Two frontends against one API is useful to have available and wasteful to build every time.

Two things that will catch you

down needs the profile too. docker compose down only tears down services in the active profiles, so a container started with --profile search is left running by a bare down. Use docker compose --profile all down, or --profile matching what you started.

A dependency does not activate a profile. If an always-on service declares depends_on on a profiled one, the dependency is skipped rather than pulled in — and you get a start-up failure that does not mention profiles at all.

The override file

Compose reads compose.yaml and then, if present, compose.override.yaml, merging the second over the first. Automatically, with no flags.

The usual arrangement is a base file with what is true everywhere, and an override — gitignored, or committed as the development default — with what is true here:

# compose.yaml — committed, true everywhere
services:
  api:
    build:
      context: ./pizza-springboot-backend
    ports:
      - "8086:8085"

# compose.override.yaml — local only
services:
  api:
    environment:
      LOGGING_LEVEL_COM_PIZZA: DEBUG
    volumes:
      - ./pizza-springboot-backend/src:/src:ro

Nobody edits the shared file to change a log level, and nobody accidentally commits their personal debugging setup.

How the merge actually works

This is the part that surprises people, and it is worth knowing before you rely on it.

Scalars are replaced. image, restart, a single value — the override wins.

Maps are merged, key by key. environment, labels. The override adds and replaces individual keys; it does not discard the rest.

Sequences are concatenated, not replaced. ports, volumes, command arguments. So an override that "changes" a port actually adds a second one:

# base
ports: ["8086:8085"]
# override
ports: ["9000:8085"]
# result: BOTH published — 8086 and 9000

There is no way to remove a list entry from an override. If you need a genuinely different list, the base file must not declare one — which is a real argument for keeping ports out of the base and in the per-environment files.

docker compose config prints the merged result. Use it rather than reasoning about the rules.

Stacking files with -f

For more than two environments, name them and be explicit:

docker compose -f compose.yaml -f compose.prod.yaml up -d

Later files win. ⚠️ Passing any -f disables the automatic compose.override.yaml — which is a feature (your production command does not silently pick up someone's local debugging file) and a trap the first time a -f compose.yaml makes your usual overrides stop applying.

One caveat on relative paths: they resolve against the first file's directory, not each file's own. Splitting compose files across directories produces build contexts that point somewhere unexpected.

Variables and .env

      STRIPE_SECRET_KEY: ${STRIPE_SECRET_KEY:-}
      STRIPE_PUBLISHABLE_KEY: ${STRIPE_PUBLISHABLE_KEY:-}

Interpolation happens on the host, before any container exists — Compose is doing string substitution on the YAML. A .env file next to the compose file is read automatically for these values.

${VAR}            empty if unset
${VAR:-default}   default if unset OR empty
${VAR-default}    default only if UNSET (an empty value stays empty)
${VAR:?message}   fail with `message` if unset — for a required value

${VAR:?message} is underused. It turns "the stack came up but nothing works" into a clear error before anything starts.

⚠️ Do not confuse this .env with the container's environment. This one configures Compose; to give a container a file of variables, that is env_file: on the service. Lesson 12.

Which mechanism for what

profiles         optional SERVICES — search, messaging, a second frontend
override files   different SETTINGS for the same services — dev vs prod
variables        different VALUES — hostnames, keys, tags

Next: the whole application, one command.