Troubleshoot a Docker Compose deployment

The API, worker, and the two one-shot containers run a distroless image with no shell, so there is nothing to exec into and no in-container health command to read. Every diagnosis starts from logs:

docker compose ps -a
docker compose logs migrate seed
docker compose logs --tail 100 api

Norn writes one JSON object per line on standard output. A container that exited non-zero has its reason in the last few lines.

The stack starts and then nothing is running

Read the exit codes before the logs:

docker compose ps -a

migrate, seed, and storage-init are one-shot containers and should read Exited (0). Anything else stops the containers that depend on them, so the API, worker, dashboard, and Caddy never start. That is one failure, not five.

Container	Check
storage-init	the Docker daemon can write to the blob-data volume
migrate	the connection string, database availability, and migration error output
seed	database access and Valkey availability; the policy seed needs both

The database connection string is rejected

parse postgres dsn: failed to parse as URL (invalid port ...)

The password contains a character that is not valid in a URL. Regenerate it with openssl rand -hex 32, or percent-encode the existing one, and set it in .env.

The API refuses to start

Norn validates its whole configuration before it opens a listener, and names the setting it rejected. The pairs that catch most deployments:

• NORN_SESSION_SECURE must be true while NORN_APP_BASE_URL is https
• NORN_SMTP_HOST and NORN_SMTP_FROM_ADDRESS are set together or not at all
• NORN_HTTP_CLIENT_IP_HEADER requires NORN_HTTP_TRUSTED_PROXIES
• with S3 storage, the storage host must differ from the host in NORN_APP_BASE_URL

Every setting and every rule of this kind is listed in the environment variable reference.

Caddy cannot obtain a certificate

docker compose logs caddy

Issuance needs the DNS record for NORN_APP_BASE_URL to resolve to this host already, and ports 80 and 443 reachable from the internet. A host firewall, a cloud security group, or another process holding port 80 all produce the same failure.

Certificates live in the norn_caddy-data volume. Removing that volume discards the account key and starts issuance again, which counts against the issuer's rate limits.

An instance on a private network cannot be issued a public certificate. Use a plain-http origin instead.

Norn opens but sign-in or forms return 403

The browser URL must match Norn's configured public origin exactly, including the scheme. The dashboard sends NORN_APP_BASE_URL as the Origin header on every write, and the API compares it against its own configuration; a mismatch fails writes while reads keep working.

Check that one value:

grep NORN_APP_BASE_URL .env

Changing it means recreating the containers that read it, which docker compose up -d does.

Realtime updates disconnect

/v1/workspaces/<workspace>/events is a Server-Sent Events stream. The bundled Caddy is already configured for it: the route is matched before the rest of the API, flushed immediately, and left out of the compression allow-list.

Reconnects at a consistent interval point at something else in the path — a company proxy, a CDN in front of the host, or a load balancer with a response read timeout. Confirm the stream is clean at the host itself:

curl -N https://norn.example.com/v1/workspaces/<workspace>/events

A working stream answers content-type: text/event-stream, carries no content-encoding, and sends its first bytes immediately.

Attachments fail

By default, attachment bytes travel through the API and land on the blob-data volume, mounted into both the API and the worker. Check that the volume is writable by the container user:

docker compose run --rm --no-deps storage-init
docker compose up -d api worker

With S3-compatible storage, the browser transfers bytes directly to the endpoint instead. Check the failing request in the browser network panel: the endpoint must be browser-reachable with a valid certificate, its CORS policy must allow the exact Norn origin to use GET, HEAD, and PUT, allow content-type, and expose ETag, and the credentials must reach the configured bucket.

Email is not delivered

Check the API and worker logs for SMTP errors. Verify the provider's host, port, sender verification, authentication type, and TLS policy. Norn supports authentication types none, plain, login, and cram-md5; TLS policies are none, opportunistic, and mandatory.

While mail is unconfigured, sign-up and invitation links appear on screen rather than failing silently. Seeing a link where an email was expected means NORN_SMTP_HOST and NORN_SMTP_FROM_ADDRESS did not both take effect.

Every session records the same address

Norn attributes a request to the client address forwarded by the proxy, and trusts that header only from NETWORK_SUBNET. If sessions all show one container address, the real address was inside the trusted range and was skipped as another hop.

That is expected when you test from the Docker host itself. It is a problem when a proxy outside the compose network fronts the deployment: add its address to NORN_HTTP_TRUSTED_PROXIES in a compose override.

A changed setting has no effect

Settings the compose file owns cannot be changed from .env; they are listed in where configuration lives and are changed through compose.override.yaml.

Everything else takes effect on the next docker compose up -d, which recreates the containers whose environment changed. ./setup.sh restart does the same thing. Plain docker compose restart does not: it restarts the existing containers with the environment they were created with.

Share diagnostics safely

Include the Norn version, the Docker and Compose versions, the host operating system, whether the bundled or external data services are in use, and redacted logs. Never share .env, connection strings, credentials, encryption keys, cookies, or licence keys.