HAProxy Network Error “cannot bind socket” often triggers when another process is listening on the same interface and TCP port combination that HAProxy uses.

As a part of our Server Management Services, we help our Customers to fix Load balancer related issues regularly.

Let us today discuss the possible causes and fixes for this error.

What causes HAProxy Network Error “cannot bind socket”?

An HAProxy “cannot bind socket” error triggers in either of the following cases:

• Another process listens on the same interface and TCP port combination set for HAProxy.
• HAProxy attempts to use an IP address that is not assigned to a network interface.

The first step in troubleshooting the error is to examine the list of currently used sockets and IP addresses in the system.

We will need to examine systemctl and journalctl output to determine the IP address and port combination that are causing the error.

Troubleshooting HAProxy Network Error “cannot bind socket” with systemctl

Initially, we need to check HAProxy’s status with systemctl. On CentOS, Fedora, and RedHat-derived systems, we can use this systemctl command to examine HAProxy’s status:

$ sudo systemctl status haproxy.service -l --no-pager

The -l flag will ensure that systemctl outputs the entire contents of a line. The –no-pager flag will output the entire log to the screen.

The output may look like:

● haproxy.service - HAProxy Load Balancer
Loaded: loaded (/usr/lib/systemd/system/haproxy.service; disabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Wed 2020-08-19 14:57:05 UTC; 3s ago
Process: 138738 ExecStart=/usr/sbin/haproxy -Ws -f $CONFIG -p $PIDFILE (code=exited, status=1/FAILURE)
Process: 138736 ExecStartPre=/usr/sbin/haproxy -f $CONFIG -c -q (code=exited, status=0/SUCCESS)
Main PID: 138738 (code=exited, status=1/FAILURE)

Aug 19 14:57:05 92214d8ff5e2 systemd[1]: Starting HAProxy Load Balancer...
Aug 19 14:57:05 92214d8ff5e2 haproxy[138738]: [ALERT] 231/145705 (138738) : Starting frontend main: cannot bind socket [0.0.0.0:80]
. . .
Aug 19 14:57:05 92214d8ff5e2 systemd[1]: Failed to start HAProxy Load Balancer.

Here, the line “cannot bind socket [0.0.0.0:80]” describes the socket that HAProxy is trying to use (0.0.0.0:80).

The last line indicates the status of the HAProxy process, which in the case of a cannot bind socket error will show Failed to start HAProxy Load Balancer.

[Troubled with HAProxy Network Error “cannot bind socket”? – Our Server experts can help you fix it.]

Troubleshooting HAProxy Network Error “cannot bind socket” Using journalctl Logs

For Ubuntu or a Debian-derived Linux distribution, systemctl does not include specifics about the “cannot bind socket” error message.

Thus, we should proceed with using the journalctl command to examine systemd logs for HAProxy.

Ubuntu:

$ sudo journalctl -u haproxy.service --since today --no-pager

CentOS:

$ sudo journalctl -u haproxy.service --since today --no-pager

Search through the output for lines that are similar to the following log entries:

-- Logs begin at Wed 2020-08-19 19:38:12 UTC, end at Wed 2020-08-19 19:53:53 UTC. --
. . .
Aug 19 19:39:21 92214d8ff5e2 systemd[1]: Starting HAProxy Load Balancer...
Aug 19 19:39:21 92214d8ff5e2 haproxy[135]: [ALERT] 231/193921 (135) : Starting frontend main: cannot bind socket [0.0.0.0:80]
Aug 19 19:39:21 92214d8ff5e2 haproxy[135]: [ALERT] 231/193921 (135) : Starting frontend main: cannot bind socket [:::80]
Aug 19 19:39:21 92214d8ff5e2 systemd[1]: haproxy.service: Main process exited, code=exited, status=1/FAILURE
Aug 19 19:39:21 92214d8ff5e2 systemd[1]: haproxy.service: Failed with result 'exit-code'.
Aug 19 19:39:21 92214d8ff5e2 systemd[1]: Failed to start HAProxy Load Balancer.
. . .

The line of output “cannot bind socket [0.0.0.0:80]” indicates that HAProxy cannot bind to port 80 on all available IPv4 interfaces.

Troubleshooting with ss and ps Utilities

Now that we know that some other process is listening to the port, let us now try to find the details of the process.

We can use the command below to determine the name of the process that is bound to an IPv4 interface on port 80. Ensure to replace 80 with the exact port number from the error message:

$ sudo ss -4 -tlnp | grep 80

We should receive an output as given below:

LISTEN 0 511 0.0.0.0:80 0.0.0.0:* users:(("nginx",pid=40,fd=6))

The important fields in the output includes the fourth (0.0.0.0:80) and the last users:((“nginx”,pid=40,fd=6)), specifically the pid=40 portion.

Similarly, for IPv6 addresses, use the format below:

$ sudo ss -6 -tlnp |grep 80

Output for it may look like the one given below:

LISTEN 0 511 [::]:80 [::]:* users:(("nginx",pid=40,fd=7))

The output in both cases indicates that there is a program with process ID 40 bound to the 0.0.0.0:80 and [::]:80 interfaces respectively. This process is preventing HAProxy from starting since it already owns the port.

To determine the name of the program, use the ps utility substituting the process ID from the output in place of 40 in this example:

$ sudo ps -p 40

The output looks like:

PID TTY TIME CMD
40 ? 00:00:00 nginx

From the output, Nginx is the process that is listening on the interfaces.

Now that we have the name of the program, we could either reconfigure Nginx to listen on a different interface/port or reconfigure HAProxy to avoid the port collision.

If the output of the ss commands does not have a line with a matching port, then the “cannot bind socket” error may be related to the use of an IP address that is not assigned to a network interface.

Troubleshooting with the IP Utility

To determine whether the IP not assigned to a network interface is causing the error “cannot bind socket error”, we will examine both the IPv4 and IPv6 network interfaces on the system using the IP command.

$ sudo ip -4 -c address show

We will see an output like the one below:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
inet xxx.xxx.xxx.1/24 brd xxx.xxx.xxx.255 scope global eth0
valid_lft forever preferred_lft forever
inet xxx.xxx.xxx.1/24 brd xxx.xxx.xxx.225 scope global eth0
valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
inet xxx.xxx.xxx.1/24 brd xxx.xxx.xxx.255 scope global eth1
valid_lft forever preferred_lft forever

Make a note of the IP addresses in the output.

Similarly, for IPv6 addresses use the command below:

$ sudo ip -6 -c address show

Once we have a list of addresses that are assigned to the system, we can try to find a matching IP address that corresponds to the cannot bind socket [x.x.x.x:80] error.

To resolve the error, we will need to edit your /etc/haproxy/haproxy.cfg file. Change the bind addresses to an available IP address based on the output of the IP command.

Once we have edited /etc/haproxy/haproxy.cfg with the correct IP address, restart it using the systemctl command:

$ sudo systemctl restart haproxy.service

[Need any further assistance in fixing HAProxy Network Error cannot bind socket errors? – We’re available 24*7]

Conclusion

In short, HAProxy Network Error “cannot bind socket” can trigger due to two different reasons. Today, we saw the steps that our Support Engineers follow to find the exact reason for the error.