Are your websites showing blank pages? Is your server load high? Do you see hundreds of Apache process in your admin console?

You are not alone.

“Too many connections” is a common issue in LAMP (Linux-Apache-MySQL-PHP) servers that’s most commonly used in Web Hosting.

Here at Bobcares, our Hosting Support Engineers support LAMP servers of web hosts, digital marketers, etc., and we’ve seen these reasons for Apache’s too many connections error:

– Unoptimized Apache settings

The default settings of Apache can induce a heavy load on memory if the sites are content rich dynamic pages (eg. WordPress). If the memory usage is not carefully calibrated and balanced between HTTP and Database servers, the server will freeze.

– Brute force attacks

The internet is infested with Bots and Bot-masters that try to infect websites and spread their malware everywhere. They do this by exploiting website vulnerabilities and cracking login details through brute forcing. When such a large scale attack occurs, the server can freeze.

– Database server (eg. MySQL) limits

Database servers also have built-in connection limits that can fail to rise up to the traffic, causing “too many connections” error.

Now, let’s take a look at the ways in which we resolve these issues in our customer servers:

1. Fix Apache configuration

Misconfigured Apache is the most common reason we’ve seen for the “too many connections” error.

We’ve seen these common configuration issues while troubleshooting customer servers:

• Unsuitable MPM for the server – Apache servers use Prefork MPM (Multi-Processing Module) by default. This takes too much memory and suited for low traffic, simple (text based) websites. Content rich dynamic websites (eg. WordPress) are better served by Worker MPM (or Event MPM for very busy sites).
• Insufficient memory allocation – Most web servers also run Database services, DNS services and Mail services. So unless the MaxClient (MaxRequestWorkers in Apache 2.4) and StartServer settings are carefully calibrated to match available memory, Apache will resort to using the slower swap space, and the whole server will grind to a halt.
• Too liberal KeepAlive values – Apache uses a feature called “KeepAlive” to make connections faster, but malicious bots and slow devices can hang on to those connections for a long time, and starve server resources.

When we are called in to fix an unoptimized server, we estimate the (1) average & max traffic on the server, (2) memory usage per server request, and (3) the trend of resource usage in all services.

Based on this data, we tweak the server settings to keep the server load less than 1.0.

We follow-up with our 24/7 monitoring service and make continual adjustments to the configuration until the server can comfortably take care of the traffic directed at it.

2. Block DoS and brute force attacks using firewalls

Most attacks behave in predictable ways.

Some send hundreds of connections in seconds, some establish no-response connections, and some others use abuse-listed IPs.

Our dedicated support engineers fight this in 4 ways:

• Harden the network & kernel settings – Some attacks such as Smurf attacks, Slow Loris attacks and Syn Flood attacks can be blocked by hardening the basic network settings. So, that’s the first step in our network security measures.
• Setup a strong firewall – Once the kernel is secure, we configure the firewall to detect port flooding, port scanning, and other such behavior that’s indicative of abusive behavior. We then set these IPs to be automatically blocked, so that legitimate requests are not affected.
• Enable anti-DoS modules in Apache – As the final layer of defense, we setup anti-DoS modules in Apache such as mod_evasive and mod_qos which limits the bandwidth per visitor. In this way, the server won’t be affected even if one user initiates a lot of connections (common to DoS attacks and bots).
• Setup a web application firewall – Web Application Firewalls such as mod_security blocks many common attacks (such as XSS, CSRF, SQLi, etc.) based on connection signatures. We configure the system to detect and block attacker IPs, so that attackers won’t be able to upload a malware, much less execute it.

3. Use a reverse proxy or caching server

The most effective solution that we’ve found against Apache overload issues is to use a Reverse Proxy server in front of Apache.

The reality is, Apache is not all that great in handling multiple simultaneous connections.

So, we put caching service such as Nginx or Varnish in front of Apache that reduces the hits sent to the Apache service.

These services store a copy of all pages served in the memory, and when another request comes for the same page, it quickly sends back the cached copy.

In this way, we’ve been able to cut down the load to Apache to less than 40%, while using less resources than the previous Apache stand-alone process.

4. Optimize database settings

In a few rare cases, we’ve seen MySQL settings to be unsuitable for the traffic the server was receiving.

When we detect MySQL issues, we use the same approach as we did with Apache. We log the requests sent to the database, and find out the actual number of connections.

Once we know the exact load exerted on the database, we tweak the Buffer size, Cache size, Sort table size, Max user connections, and more to eliminate aborted queries.

The server performance is observed for a few days to make sure the resource allocation works well with Apache settings and incoming traffic.

If needed, we add PHP opcode caches to further reduce database server load.

Conclusion

Apache’s “too many connections” is caused by memory overage, brute force attacks, MySQL limits, and more. Today we’ve seen the top causes for this error, and how Dedicated Support Engineers at Bobcares resolve and prevent it from happening again.