Docker container has malformed DNS setup

January 15, 2023

How it started

I could not log in to Nginx proxy manager after docker container restart. UI returned 502 Bad Gateway

After investigation and browsing docker container logs

docker ps
docker logs --follow nginx-proxy-manager-app-1
docker logs --follow nginx-proxy-manager-db-1

It turned out that nginx app raised error from pip execution

Command failed: pip install certbot-dns-cloudflare==$(certbot --version | grep -Eo '[0-9](\.[0-9]+)+') cloudflare
  WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7f7ca970c2e8>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': (...)

I tried to install pip package on docker host which ended succesfully but it failed within docker container

Actually, /etc/resolv.conf in container had changed having following records:

search szajowski.dev
nameserver 127.0.0.11
options ndots:0

When I changed /etc/resolv.conf to docker host config, problem was resolved.

Unforunately problem has propagated to rest of the containers, just created metube container suffered from the very same issue

Found solution

There is already a topic https://serverfault.com/a/1024840/1000770 regarding DNS within docker. I simply removed docker that was installed using snap and installed again with following command

sudo snap remove docker
curl -fsSL https://get.docker.com -o get-docker.sh
sh get-docker.sh

In the end containers /etc/resolv.conf remained the same (using nameserver 127.0.0.11 )

Lessons learned

snap docker config deviates from the one installed using get-docker.com
create VM backup before substantial changes in OS. (sudo snap remove docker removed external volumes)
https://dockerlabs.collabnix.com/intermediate/networking/Configuring_DNS.html