angle-uparrow-clockwisearrow-counterclockwisearrow-down-uparrow-leftatcalendarcard-listchatcheckenvelopefolderhouseinfo-circlepencilpeoplepersonperson-fillperson-plusphoneplusquestion-circlesearchtagtrashx

Connect to a service on a Docker host from a Docker container

When using the default networking bridge mode, you must make a service on the Docker host listen (also) to docker0.

11 August 2022 Updated 12 August 2022
In Docker
post main image
https://www.pexels.com/@vladbagacian/

If you have some Docker problem and search the Internet, you will almost certainly stumble upon the question: How can I connect to localhost? What people mean is: How can I connect to a service on the Docker host from a Docker container. When I started using Docker I was also struggling with this.

I am using Linux, Ubuntu, and when finally host.docker.internal was available for Linux many people thought that all their problems were over. But nothing changed. The problem remains the same.

There are many posts on the internet about this, and this is another one. I will be discussing only the default networking bridge mode, and not the 'network=host' mode.

Tools used

In this post I use a few command-line tools, which you may want to install. To get the latest version, or add them to the repository on your machine:

sudo apt update

netcat

Used to create a listener service.

sudo apt install netcat

netstat

Used to show listeners on the host.

sudo install net-tools

nmap

Used to scan (for open) ports.

sudo apt install nmap

Listener services and networking

Let's spin up a listener service in a terminal window, listen to port 2345 on localhost, where localhost is 127.0.0.1. We add the -k (keep open) option, to prevent netcat from terminating when you disconnect:

netcat -k -l 127.0.0.1 2345

Check that we have a netcat service listening:

sudo netstat -tunlp

Or, if you have a very long list:

sudo netstat -tunlp | grep 2345

Result:

Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
...
tcp        0      0 127.0.0.1:2345          0.0.0.0:*               LISTEN      2961983/netcat
...

This means that netcat is listening on IP address 127.0.0.1 to port 2345.

Now start a Busybox Docker container in another terminal window:

docker run -it --add-host=host.docker.internal:host-gateway --rm busybox

And then in the container try to access our listener service:

telnet host.docker.internal 2345

Result:

telnet: can't connect to remote host (172.17.0.1): Connection refused

If telnet does not respond, check UFW

What also can happen is that telnet hangs, and after a very long time responds with:

telnet: can't connect to remote host (172.17.0.1): Connection timed out

In this case, the request from Docker is probably being blocked by UFW (Uncomplicated Firewall). You can check if UFW is blocking the request by looking at the UFW entries in /var/log/syslog. In another terminal window, type the following:

tail -f /var/log/syslog | grep UFW

In the Docker container, restart the telnet command. If UFW blocks your request, you will see messages like:

Aug 11 09:48:05 myra kernel: [247791.035929] [UFW BLOCK] IN=docker0 OUT= PHYSIN=vethe3b7837 MAC=02:42:80:b6:f4:ea:02:42:ac:11:00:02:08:00 SRC=172.17.0.2 DST=172.17.0.1 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=9274 DF PROTO=TCP SPT=35690 DPT=2345 WINDOW=64240 RES=0x00 SYN URGP=0

The important information here is:

SRC=172.17.0.2   <--- SOURCE
DST=172.17.0.1   <--- DESTINATION
SPT=35690        <--- SOURCE PORT
DPT=2345         <--- DESTINATION PORT

Note that port 2345 is blocked by UFW. You can do one of two things here:

  • Disable UFW during these tests, or
  • Add a rule to UFW to allow access to port 2345:
    sudo ufw allow from 172.17.0.0/16 to any port 2345.
    This allow networks 172.17.x.y to access port 2345.

For the moment I suggest you disable UFW. After doing this, telnet should respond as mentioned before:

telnet: can't connect to remote host (172.17.0.1): Connection refused

Ok, so why is the connection refused?

For this you need to understand some networking basics. The listener service said it is listening to 'Local Address': 127.0.0.1:2345. And telnet in the Docker container said: can't connect to remote host (172.17.0.1).

Note that 127.0.0.1 and 172.17.0.1 are on different networks. This means a connection can never be established!

From the post 'How to Connect to Localhost Within a Docker Container', see links below:

One pitfall of this approach is you might not be able to connect to services which bind directly to localhost. You’ll need to make sure your services are listening for connections on your Docker bridge IP, as well as localhost and 127.0.0.1. Otherwise you’ll see connection refused or similar errors within your container.

Exactly. This means that our listener service should not only listen to 127.0.0.1 but also to 172.17.0.1.
Let's restart our listener service, and make it listen to 172.17.0.1:

netcat -l 172.17.0.1 2345

Start telnet again in the Docker container:

telnet host.docker.internal 2345

Result:

Connected to host.docker.internal

Bingo! This means that the service on the Docker host also must listen to docker0. You can type some text and it will be echo-ed by netcat.

What we also learned here is that host.docker.internal is NOT equivalent to localhost

host.docker.internal is just a name for docker0. Docker0 is a virtual interface, a bridge, created by Docker, and host.docker.internal is the (domain) name of the docker0 bridge. All Docker containers are connected to the docker0 bridge by default. Although the IP address of docker0 is often 172.17.0.1, it can be anything from the private range defined by RFC 1918. That is why we should not use the IP address but refer to docker0 as host.docker.internal.

      +-------------+        +-------------+
      | container A |        | container B |
      +-------------+        +-------------+
             |                      |
             |                      |
         +------------------------------+ 
    +----|           docker0            |---+
    |    +------------------------------+   |
    |                                       |
    |                  Host                 |
    |                                       |
    +---------------------------------------+
                         |
                      network
                         

Again, how can I connect to a service on localhost from a Docker container?

When the listener service was listening on 127.0.0.1 we could not reach it from our docker container. But when the listener service was listening on 172.17.0.1 we could connect from our docker container.

In other words, the cause of the problem is that the service on the host is NOT listening to 172.17.0.1.

In Linux, most services can be made to listen to all interfaces, networks.  We do this by using the address '0.0.0.0'. We can start netcat with or without IP address '0.0.0.0', it is the same:

netcat -l 2345

or,

netcat -l 0.0.0.0 2345

Checking the listening ports

sudo netstat -tunlp | grep 2345

Result:

Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
...
tcp        0      0 0.0.0.0:2345            0.0.0.0:*               LISTEN      1606615/netcat
...

Local Address '0.0.0.0' means that it will listen to all networks including 127.0.0.1 and 172.17.0.1.

Start telnet in the container:

telnet host.docker.internal 2345

Result:

Connected to host.docker.internal

As expected the connection succeeded.

Nice, but what must I do to connect to a service on localhost from a Docker container?

It's all about making sure that the service on localhost is (also) listening to 172.17.0.1, or docker0. On Ubuntu, the IP address of docker0 is 172.17.0.1, but Docker says it also can be randomly choosen, see also above.

This means that restarting Docker may give a different docker0 IP address, maybe not today, but maybe in a few months, or next year(?). Since we cannot rely on the docker0 IP address, the only other option is to have your service on the host listen to '0.0.0.0'. It is possible to set the bridge IP address and subnet, but that is beyond the scope of this post.

To access MySQL on the Docker host from a container, you can change a line in /etc/mysql/my.cnf:
From:

bind-address = 127.0.0.1

To:

bind-address = 0.0.0.0

Note that there are solutions on the Internet that make the service on the Docker host listen on the docker0 IP address. They rely on the fact that this IP address will not change. Unless you know what you are doing I cannot recommend this.

Create a separate Docker network for the service on the host

You can isolate the service on the host by creating a Docker network for it, for example:

docker network create --driver=bridge --subnet=172.17.33.0/24 --gateway=172.17.33.10 my_subnet

Start the service on the host:

netcat -k -l 172.17.33.10 2345

Then start the Docker container:

docker run -it --add-host=my_host:172.17.33.10 --network=my_subnet --rm busybox

And access the service from the Docker container:

telnet my_host 2345

Access a service on the Docker host using a Unix socket

Finally, there is a totally different way to access a service on the Docker host, but it depends on whether the service supports it. Some services use a 'unix socket' a file. For example, MySQL uses the file:

/run/mysqld/mysqld.sock

This means we can also access MySQL from a container using volume mapping, in Docker-Compose:

    volumes:
      # connect to mysql via unix socket 
      - /var/run/mysqld:/var/run/mysqld

This works fine. I have applications on the host and applications in Docker container all using MariaDb on the host.

Docker IP addresses and UFW rules

This is important because you are probably using UFW to control access to and from the machine. As mentioned above, each time a container is created, the IP address can change and some UFW rules will no longer work!

By default, Docker assigns an IP address to a container from one of the following network ranges for bridge networks:

  • 172.17.0.0/16
  • 172.18.0.0/16
  • 172.19.0.0/16
  • 172.20.0.0/14
  • 172.24.0.0/14
  • 172.28.0.0/14
  • 192.168.0.0/16

This means that if we specified a UFW rule with '172.17.0.0/16', this rule will not work anymore if Docker assigns the next time an IP address from the network '192.168.0.0/16'.

One way of solving this is to specify the default address pools for Docker networks ourselves, as I described in the post 'Docker containers suddenly using 192.168.0.0/16 instead of 172.17.0.0/16: services lost', see links below.

We can specify default address pools for our Docker networks by creating the file (it was not there):

/etc/docker/daemon.json

This example tells Docker to use only IP address from the network 172.17.0.0/16:

{
	"default-address-pools":
	[
		{"base":"172.17.0.0/16","size":24}
	]
}

Always scan your ports after making changes!

After making changes, always scan the ports on your machine and the production server. Especially when using Docker we need to make sure that no ports have been opened by mistake!

If we want to do this ourselves we can use the nmap command. The problem is that the results depend on the way you connect to the Internet. You need to check the firewall on your machine and the firewall on your Internet access point. Also, your Internet provider's firewall may block connections. The only way to scan your production server for open ports is to use another server (VPS) on the same network, or on the Internet, that has no outbound restrictions.

sudo apt install nmap

An example of this command to scan ports 80-10000:

nmap -p 80-10000 www.example.com

You can also use the IP address of your development machine, and run nmap on this machine, but I found that the results are not correct.

nmap -p 80-10000 <your-development-machine-ip-address>

It showed that port 0.0.0.0:2345 was open. Scanning from another computer in my local network did not show this.

Summary

In this post I discussed how you can connect to a service on a Docker host from a Docker container. There are limitations but most of the time it is possible. Be aware of Docker assigned IP addresses, they can change every time a container is created. And always check your systems for open ports after you made changes!

Links / credits

Change default docker0 bridge ip address
https://stackoverflow.com/questions/52225493/change-default-docker0-bridge-ip-address

Connecting Docker to localhost PostgreSQL on an Ubuntu 18.04 Host
https://gregtczap.com/blog/docker-postgres-ubuntu-localhost

Docker containers suddenly using 192.168.0.0/16 instead of 172.17.0.0/16: services lost
https://www.peterspython.com/en/blog/docker-containers-suddenly-using-192-168-0-0-16-instead-of-172-17-0-0-16-services-lost

From inside of a Docker container, how do I connect to the localhost of the machine?
https://stackoverflow.com/questions/24319662/from-inside-of-a-docker-container-how-do-i-connect-to-the-localhost-of-the-mach

How to Connect to Localhost Within a Docker Container
https://www.howtogeek.com/devops/how-to-connect-to-localhost-within-a-docker-container

What's the `docker0` there?
https://stackoverflow.com/questions/63948879/whats-the-docker0-there

Read more

Docker

Leave a comment

Comment anonymously or log in to comment.

Comments (1)

Leave a reply

Reply anonymously or log in to reply.

avatar

Спасибо! Мне очень помогла статья