Connect to a service on a Docker host from a Docker container
When using the default networking bridge mode, you must make a service on the Docker host listen (also) to docker0.
If you have some Docker problem and search the Internet, you will almost certainly stumble upon the question: How can I connect to localhost? What people mean is: How can I connect to a service on the Docker host from a Docker container. When I started using Docker I was also struggling with this.
I am using Linux, Ubuntu, and when finally host.docker.internal was available for Linux many people thought that all their problems were over. But nothing changed. The problem remains the same.
There are many posts on the internet about this, and this is another one. I will be discussing only the default networking bridge mode, and not the 'network=host' mode.
Tools used
In this post I use a few command-line tools, which you may want to install. To get the latest version, or add them to the repository on your machine:
sudo apt update
netcat
Used to create a listener service.
sudo apt install netcat
netstat
Used to show listeners on the host.
sudo install net-tools
nmap
Used to scan (for open) ports.
sudo apt install nmap
Listener services and networking
Let's spin up a listener service in a terminal window, listen to port 2345 on localhost, where localhost is 127.0.0.1. We add the -k (keep open) option, to prevent netcat from terminating when you disconnect:
netcat -k -l 127.0.0.1 2345
Check that we have a netcat service listening:
sudo netstat -tunlp
Or, if you have a very long list:
sudo netstat -tunlp | grep 2345
Result:
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
...
tcp 0 0 127.0.0.1:2345 0.0.0.0:* LISTEN 2961983/netcat
...
This means that netcat is listening on IP address 127.0.0.1 to port 2345.
Now start a Busybox Docker container in another terminal window:
docker run -it --add-host=host.docker.internal:host-gateway --rm busybox
And then in the container try to access our listener service:
telnet host.docker.internal 2345
Result:
telnet: can't connect to remote host (172.17.0.1): Connection refused
If telnet does not respond, check UFW
What also can happen is that telnet hangs, and after a very long time responds with:
telnet: can't connect to remote host (172.17.0.1): Connection timed out
In this case, the request from Docker is probably being blocked by UFW (Uncomplicated Firewall). You can check if UFW is blocking the request by looking at the UFW entries in /var/log/syslog. In another terminal window, type the following:
tail -f /var/log/syslog | grep UFW
In the Docker container, restart the telnet command. If UFW blocks your request, you will see messages like:
Aug 11 09:48:05 myra kernel: [247791.035929] [UFW BLOCK] IN=docker0 OUT= PHYSIN=vethe3b7837 MAC=02:42:80:b6:f4:ea:02:42:ac:11:00:02:08:00 SRC=172.17.0.2 DST=172.17.0.1 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=9274 DF PROTO=TCP SPT=35690 DPT=2345 WINDOW=64240 RES=0x00 SYN URGP=0
The important information here is:
SRC=172.17.0.2 <--- SOURCE
DST=172.17.0.1 <--- DESTINATION
SPT=35690 <--- SOURCE PORT
DPT=2345 <--- DESTINATION PORT
Note that port 2345 is blocked by UFW. You can do one of two things here:
- Disable UFW during these tests, or
- Add a rule to UFW to allow access to port 2345:
sudo ufw allow from 172.17.0.0/16 to any port 2345.
This allow networks 172.17.x.y to access port 2345.
For the moment I suggest you disable UFW. After doing this, telnet should respond as mentioned before:
telnet: can't connect to remote host (172.17.0.1): Connection refused
Ok, so why is the connection refused?
For this you need to understand some networking basics. The listener service said it is listening to 'Local Address': 127.0.0.1:2345. And telnet in the Docker container said: can't connect to remote host (172.17.0.1).
Note that 127.0.0.1 and 172.17.0.1 are on different networks. This means a connection can never be established!
From the post 'How to Connect to Localhost Within a Docker Container', see links below:
Exactly. This means that our listener service should not only listen to 127.0.0.1 but also to 172.17.0.1.
Let's restart our listener service, and make it listen to 172.17.0.1:
netcat -l 172.17.0.1 2345
Start telnet again in the Docker container:
telnet host.docker.internal 2345
Result:
Connected to host.docker.internal
Bingo! This means that the service on the Docker host also must listen to docker0. You can type some text and it will be echo-ed by netcat.
What we also learned here is that host.docker.internal is NOT equivalent to localhost
host.docker.internal is just a name for docker0. Docker0 is a virtual interface, a bridge, created by Docker, and host.docker.internal is the (domain) name of the docker0 bridge. All Docker containers are connected to the docker0 bridge by default. Although the IP address of docker0 is often 172.17.0.1, it can be anything from the private range defined by RFC 1918. That is why we should not use the IP address but refer to docker0 as host.docker.internal.
+-------------+ +-------------+
| container A | | container B |
+-------------+ +-------------+
| |
| |
+------------------------------+
+----| docker0 |---+
| +------------------------------+ |
| |
| Host |
| |
+---------------------------------------+
|
network
Again, how can I connect to a service on localhost from a Docker container?
When the listener service was listening on 127.0.0.1 we could not reach it from our docker container. But when the listener service was listening on 172.17.0.1 we could connect from our docker container.
In other words, the cause of the problem is that the service on the host is NOT listening to 172.17.0.1.
In Linux, most services can be made to listen to all interfaces, networks. We do this by using the address '0.0.0.0'. We can start netcat with or without IP address '0.0.0.0', it is the same:
netcat -l 2345
or,
netcat -l 0.0.0.0 2345
Checking the listening ports
sudo netstat -tunlp | grep 2345
Result:
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
...
tcp 0 0 0.0.0.0:2345 0.0.0.0:* LISTEN 1606615/netcat
...
Local Address '0.0.0.0' means that it will listen to all networks including 127.0.0.1 and 172.17.0.1.
Start telnet in the container:
telnet host.docker.internal 2345
Result:
Connected to host.docker.internal
As expected the connection succeeded.
Nice, but what must I do to connect to a service on localhost from a Docker container?
It's all about making sure that the service on localhost is (also) listening to 172.17.0.1, or docker0. On Ubuntu, the IP address of docker0 is 172.17.0.1, but Docker says it also can be randomly choosen, see also above.
This means that restarting Docker may give a different docker0 IP address, maybe not today, but maybe in a few months, or next year(?). Since we cannot rely on the docker0 IP address, the only other option is to have your service on the host listen to '0.0.0.0'. It is possible to set the bridge IP address and subnet, but that is beyond the scope of this post.
To access MySQL on the Docker host from a container, you can change a line in /etc/mysql/my.cnf:
From:
bind-address = 127.0.0.1
To:
bind-address = 0.0.0.0
Note that there are solutions on the Internet that make the service on the Docker host listen on the docker0 IP address. They rely on the fact that this IP address will not change. Unless you know what you are doing I cannot recommend this.
Create a separate Docker network for the service on the host
You can isolate the service on the host by creating a Docker network for it, for example:
docker network create --driver=bridge --subnet=172.17.33.0/24 --gateway=172.17.33.10 my_subnet
Start the service on the host:
netcat -k -l 172.17.33.10 2345
Then start the Docker container:
docker run -it --add-host=my_host:172.17.33.10 --network=my_subnet --rm busybox
And access the service from the Docker container:
telnet my_host 2345
Access a service on the Docker host using a Unix socket
Finally, there is a totally different way to access a service on the Docker host, but it depends on whether the service supports it. Some services use a 'unix socket' a file. For example, MySQL uses the file:
/run/mysqld/mysqld.sock
This means we can also access MySQL from a container using volume mapping, in Docker-Compose:
volumes:
# connect to mysql via unix socket
- /var/run/mysqld:/var/run/mysqld
This works fine. I have applications on the host and applications in Docker container all using MariaDb on the host.
Docker IP addresses and UFW rules
This is important because you are probably using UFW to control access to and from the machine. As mentioned above, each time a container is created, the IP address can change and some UFW rules will no longer work!
By default, Docker assigns an IP address to a container from one of the following network ranges for bridge networks:
- 172.17.0.0/16
- 172.18.0.0/16
- 172.19.0.0/16
- 172.20.0.0/14
- 172.24.0.0/14
- 172.28.0.0/14
- 192.168.0.0/16
This means that if we specified a UFW rule with '172.17.0.0/16', this rule will not work anymore if Docker assigns the next time an IP address from the network '192.168.0.0/16'.
One way of solving this is to specify the default address pools for Docker networks ourselves, as I described in the post 'Docker containers suddenly using 192.168.0.0/16 instead of 172.17.0.0/16: services lost', see links below.
We can specify default address pools for our Docker networks by creating the file (it was not there):
/etc/docker/daemon.json
This example tells Docker to use only IP address from the network 172.17.0.0/16:
{
"default-address-pools":
[
{"base":"172.17.0.0/16","size":24}
]
}
Always scan your ports after making changes!
After making changes, always scan the ports on your machine and the production server. Especially when using Docker we need to make sure that no ports have been opened by mistake!
If we want to do this ourselves we can use the nmap command. The problem is that the results depend on the way you connect to the Internet. You need to check the firewall on your machine and the firewall on your Internet access point. Also, your Internet provider's firewall may block connections. The only way to scan your production server for open ports is to use another server (VPS) on the same network, or on the Internet, that has no outbound restrictions.
sudo apt install nmap
An example of this command to scan ports 80-10000:
nmap -p 80-10000 www.example.com
You can also use the IP address of your development machine, and run nmap on this machine, but I found that the results are not correct.
nmap -p 80-10000 <your-development-machine-ip-address>
It showed that port 0.0.0.0:2345 was open. Scanning from another computer in my local network did not show this.
Summary
In this post I discussed how you can connect to a service on a Docker host from a Docker container. There are limitations but most of the time it is possible. Be aware of Docker assigned IP addresses, they can change every time a container is created. And always check your systems for open ports after you made changes!
Links / credits
Change default docker0 bridge ip address
https://stackoverflow.com/questions/52225493/change-default-docker0-bridge-ip-address
Connecting Docker to localhost PostgreSQL on an Ubuntu 18.04 Host
https://gregtczap.com/blog/docker-postgres-ubuntu-localhost
Docker containers suddenly using 192.168.0.0/16 instead of 172.17.0.0/16: services lost
https://www.peterspython.com/en/blog/docker-containers-suddenly-using-192-168-0-0-16-instead-of-172-17-0-0-16-services-lost
From inside of a Docker container, how do I connect to the localhost of the machine?
https://stackoverflow.com/questions/24319662/from-inside-of-a-docker-container-how-do-i-connect-to-the-localhost-of-the-mach
How to Connect to Localhost Within a Docker Container
https://www.howtogeek.com/devops/how-to-connect-to-localhost-within-a-docker-container
What's the `docker0` there?
https://stackoverflow.com/questions/63948879/whats-the-docker0-there
Read more
Docker
Recent
Most viewed
- Using PyInstaller and Cython to create a Python executable
- Reducing page response times of a Flask SQLAlchemy website
- Using Python's pyOpenSSL to verify SSL certificates downloaded from a host
- Connect to a service on a Docker host from a Docker container
- Using UUIDs instead of Integer Autoincrement Primary Keys with SQLAlchemy and MariaDb
- SQLAlchemy: Using Cascade Deletes to delete related objects