shlex.quote output is not a valid .env: python-dotenv silently drops values with single quotes, Docker Compose rejects the whole file
A Python CLI exports secrets in a documented 'dotenv' format by printing '%s=%s' % (name, shlex.quote(value)) per line, on the assumption that shell-safe quoting is also valid .env syntax. Multi-line values (PEM keys) round-trip fine through python-dotenv 1.2.4 and Docker Compose v5.5.0 env_file, which looked like confirmation. But a value containing a single quote, e.g. it's, is silently missing from dotenv_values() (only a 'python-dotenv could not parse statement starting at line N' warning on stderr), and docker compose config fails for the entire file with line 4: unexpected character "\"" in variable name "\"'\"'s'". Switching to double quotes with backslash escapes made docker run --env-file show literal quote characters in every value. Which quoting is actually portable across dotenv readers?
Root cause: shlex.quote("it's") produces 'it'"'"'s', the POSIX-shell idiom of closing the single-quoted string, emitting a double-quoted ', and reopening. That concatenation is shell grammar only; .env parsers treat a quoted value as one quoted token, so they hit the trailing "'"'s' and fail. python-dotenv skips the line with a warning (the variable silently disappears); Compose aborts the whole file.
Tested matrix (python-dotenv 1.2.4, Docker Compose v5.5.0 env_file, Docker 29.8.2 docker run --env-file):
abc123+/= | unquoted | ok | ok | ok |
hello world | 'hello world' | ok | ok | quotes kept literally |
| multi-line PEM | '...\n...' (real newlines) | ok, one var | ok, one var | whole file rejected (variable '-----END KEY-----'' contains whitespaces) |
it's | 'it'"'"'s' | dropped | file rejected | n/a |
x\nINJECTED=evil | 'x\nINJECTED=evil' | one var, no injection | one var | splits lines |
There is no single portable format:
docker run --env-filedoes no quote processing at all (values are taken literally, see moby/moby#46773, closed not_planned) and cannot carry multi-line values in any encoding.- Compose single-quoted values are literal except
\'(documented), so'it\'s'works there and in python-dotenv, but a value containing a literal\'or ending in\cannot be represented; escaping backslashes as\\fixes python-dotenv but Compose then keeps both backslashes. - Double-quoted with
\nescapes works in both python-dotenv and Compose, but$needs different escaping per reader (python-dotenv leaves\$as-is).
Practical fix: keep shlex-style single quoting (which is what python-dotenv and Compose both parse correctly, including multi-line values) and refuse values containing ' in the env-file output mode with an error that points users at JSON or a shell export format. Document that the output is not for docker run --env-file. Don't 'fix' newline handling by refusing \n: the real dotenv readers already handle multi-line single-quoted values, and refusing them breaks PEM secrets.