Magento is arguably the most popular eCommerce application, but its Achilles heel is performance. Sooner or later every webmaster finds their site to be slow, and look for Magento speed optimization methods.

The natural tendency for most webmasters is to upgrade server resources or buy expensive commercial software. But it’s only a temporary fix. Unless you identify specific bottlenecks, and fix them, you’ll continue to face performance issues.

Bobcares helps eCommerce sites maintain high speed websites. In our web server management services, we keep Magento super fast by using Magento optimized servers and optimizing Magento settings.

See how we speed up Magento websites!

Today we’ll see what makes Magento slow, how to build a high speed Magento system using open source software, and how to keep your Magento site blazing fast.

Why is Magento so slow?

Magento is popular because it fits the needs of a wide range of businesses. It’s built to be flexible and customizable. Performance was not a core consideration in its design. This caused bottlenecks to form in page rendering, database design etc. Here are the top ones:

• Multiple database queries – For a single page access, Magento has to construct a complex database query that touches multiple tables and rows. Databases reside in server disks, and disks are the slowest component in a server. So, to fix this bottle neck, the disk has to be insanely fast, or clever caching systems need to be enabled.
• Large list of configuration settings – Magento uses a variety of configuration files (system config, layout config, app config, etc.) which requires huge XML trees to be built up in system memory. For each access, these trees are queried, which takes significant CPU time and memory. Unless efficient caching is enabled, these accesses alone can slow the site to a crawl.
• Additional code blocks through customization – Magento’s flexibility comes from its provisions to override default settings, but the downside is that, for each access Magento checks if the needed file sections are overridden at some point. This extra checking adds to the woes.
• Complex templates loading multiple files – To display a single Magento page, multiple template files need to be executed. These files in turn has calls to other PHP files, making a single page display an amalgamation of at least 10 different files. This adds to the resource usage. Proper caching at block and page levels is needed to fix this.

If you thought these were pretty big issues, well, we are just getting started. There are other site maintenance and server infrastructure level issues that compounds the performance issues.

[ Want to speed up your Magento sites for better performance? Our website technicians will audit and fix your websites in no time. ]  

Website maintenance issues & Magento server bottlenecks

• Poor web server optimization – To serve an app as heavy as Magento, the web server needs to be as fast as possible. We’ve seen sites using Apache (instead of Nginx), compression not enabled, expires headers not enabled, Etags enabled, and more such settings, that puts needless pressure on the website.
• Poor PHP optimization – Not all PHP modules are created equal. We’ve seen sites that use Apache mod_php, where as there exists high performance alternatives such as PHP-FPM and HHVM. Even if the right module is used, it is important to give it the right settings such as memory size, execution time, max variable limit, etc.
• Slow database server – Magento is dependent heavily on the database. So, even a small issue with optimzation can have a large impact on performance. We’ve seen sites that had unoptimized MySQL settings such as query cache, table cache size, thread cache size, etc. We’ve even seen databases where log sizes exceeded 3 GB, that needlessly dragged the site.
• Poor cache systems – Just like choosing the right PHP engine, it is important to choose the right caching system for your website’s scale of operations. For eg., in a large website an HTML cache such as Varnish maybe required on top of a memory object cache such as MemCached. The choice must be made depending on the number of hits, and the size of your catalogs.
• Heavy Magento extensions – Every website falls for it at one time or another. You need an awesome feature that’s provided by your competitors. You search Magento Connect, and install one that looks good. But more often than not, it imposes a performance penalty – some much more heavily than others.
• Skipping periodic optimization – New products, categories, pages, or even stores get added to a Magento site all the time. Such changes negates any optimization you did on the website. Periodic performance audits need to be done on the website, and suitable changes (including upgrades) need to be performed to keep the site running smooth. See more details.

So, as you can see, Magento speed optimization is covers a wide range of activities that include Magento settings optimization, web server optimization, PHP optimization, database optimization, cache system optimization and periodic performance tuning.

How to speed up Magento?

A quick way to improve Magento speed is to tweak its admin settings. Just by enabling proper cache settings, enabling automatic log cleaning, or re-indexing the product list, you can gain significant improvement in website speed.

If you’re not sure your admin panel settings are optimized, here’s a full listing of all the performance settings.

The admin panel tweaks can delver a noticeable improvement in performance, but all those optimizations can be undermined by a poorly optimized server infrastructure.

To deliver web pages within 3 seconds (beyond which visitors start bouncing), your servers should be optimized for Magento. Let’s see how to do it.

Building Magento optimized servers

As we mentioned earlier, Magento’s performance-unfriendly design causes repeated calls to databases, and recursive file loading. These issues can be resolved in part by using the right Magento settings, and in part by building a server that’s optimized for fast access.

Depending on the number of transactions your site gets, the size of your catalog, and the number of stores your site has, your server design will change. For eg., for a site that receives hundreds of catalog views and orders per day, it is best to use 2 or 3 web and database servers to share the load.

To demonstrate this point, I’ll use the example of an apparel website that used Magento. The website ran on a single dedicated server, but recent success of their brand more than doubled their traffic. The single server solution was no longer sufficient to support future growth.

Conventional logic said that we should just get another dedicated server, and make a cluster of web and database services, but that would make it difficult to shift to a new network (like co-location or cloud) if the growth demanded it.

So, we used a light weight server virtualization system called LXD/LXC, which would negate the vendor locking on hardware. We could migrate the entire infrastructure with minimal downtime, if needed.

We needed a system that could handle thousands of transactions per day at a speed of at least 38 transactions/sec (which was the fastest in Nov 2015). For that, we used the Magento server architecture shown below.

This architecture was implemented using two physical servers with redundant hardware (dual-power, dual NIC, RAID 10 SSDs, etc.) to ensure fault tolerance. LXC was setup in both servers, and services were split evenly between the two.

The system consisted of the following components:

Light weight server virtualization for easy infrastructure scaling

The infrastructure needed to be flexible enough to quickly scale up or scale out to accommodate future growth. Using a server virtualization system helped us create an operating system layer that is not dependent on hardware.

It gave us the option to move to a cloud system or a co-located hosting system in the future without any changes to the architecture.

Nginx reverse proxy load balancer to evenly split HTTP traffic

As the web traffic was on an increasing trend, we needed a way to evenly split the number of visitors to multiple web servers. This would make sure no single server is over-loaded.

In future if the traffic increases, it’s just a matter of adding one more web server to the load balancing cluster. For this we used an Nginx reverse proxy because it can handle 10,000 simultaneous connections (600,000 hits/hr), and made efficient use of resources.

[ Optimize your Magento sites for better experience! Our server technicians will audit and fix your websites at affordable pricing. ]

Optimized Nginx web servers for lightning fast page delivery

Nginx web servers use very low memory, and uses something called “asynchronous I/O” to avoid delays in processing a request. When serving hundreds of visitors per day, this makes a huge difference in performance. This made us choose Nginx over Apache as the favored web server.

PHP 7 compiler for fast page creation

We did an analysis of all high performance PHP compilers out there, that included PHP-FPM, HipHop Virtual Machine (HHVM), and PHP 7.

We found PHP 7 to be the fastest of them all. PHP 7 was twice as fast as PHP 5.6, with 50% better memory consumption, and it served more concurrent users. PHP 7 was then tweaked with optimum values for realpath_cache_size, realpath_cache_ttl, memory_limit, etc. that improved performance.

Varnish cache reverse proxy to minimize disk access

Varnish Cache is a caching HTTP reverse proxy, or HTTP accelerator, that helps to serve contents to users in very less time. As it caches responses from a web or application server in memory, future requests for the same content can be served instantly. We installed and configured Varnish caching to improve performance of Magento application.

Varnish was configured to listen to port 80 for service the webserver requests. But Magento requires SSL for secure online transactions. As Varnish doesn’t support SSL directly, our design was tweaked to configure Nginx to handle SSL requests and then pass the request to Varnish. We configured Nginx to use 443 for the Frontend (SSL) and port 8000 for the backend (8000).

Whenever a request came to Varnish, it would check it’s cache and send the data to Nginx if it is there in the cache. If the data was not in its cache, Varnish would forward the request to Nginx backend, which would give the required data to Varnish. The data would be then forwarded to Nginx Frontend for SSL encapsulation and presented it back to the client.

Varnish configuration setting for cache size was set to an optimal value based on the total available RAM size. The Varnish stats was monitored over a period of time and the cache size was further adjusted to deliver a high cache hits to miss ratio.

HAProxy database load balancer to evenly distribute database load

We needed a way to evenly distribute database requests to a large number of servers. For this, we used a load balancer called HAProxy. We’ve found it to be a high performance load balancer for database servers, and evenly distributes read accesses to all servers using round-robin methodology.

Percona XtraDB database cluster for fast database performance

As mentioned earlier, we expected a growth in traffic to the website. So, we wanted a database system that’ll allow fast, simultaneous database writes. The InnoDB technology of MySQL was a good option, but we chose a clustered implementation of it called Percona XtraDB. It ensured that even if the number of database servers grew, performance wont be impacted.

With these steps, we were able to achieve a speed of 37.3 transactions/sec, which was one among the top Magento performance speeds as reported by magepseedtest.com.

Keeping Magento blazing fast

Now that we know how to build a top performing Magento website, we need to address a greater challenge – how to remain on top.

In Nov 2015, the top Magento websites reported a speed of 38 transactions/sec. Now, in Feb 2016, when I’m writing this post, the top speed reported is 41 transactions/sec. That’s how competitive the market is.

So, what do you need to do to maintain high performance?

We maintain several small, mid-size and large Magento web infrastructures, and have seen our share of performance bottlenecks that develop over time. Here’s how we keep performance issues from cropping up:

Optimize Magento database to keep it lean and mean

Every website visit leaves a record in the Magento database. Even a small business website issues thousands of database operations per day.

So, it is natural that the database size changes, its memory requirements changes, and its processing overhead increases. If left un-attended, these factors can quickly add up to form a performance bottleneck.

To counter this, we use several methods:

1. Periodically audit the database for performance issues. We manually check the optimization levels of each table, check if the database settings (such as query_cache size) are adequate for the changed table sizes, and see if there are any anomalies in operations (such as slow queries, aborted queries, etc.). Any noted issues are fixed on the spot.
2. Clean the database regularly to minimize the size. A large database quickly brings down the site performance. We work with the store manager to identify unwanted products, categories and other pages. These are deleted along with old logs to keep the database lean and mean.
3. Monitor the database for serious performance issues. We keep an eye on several critical database performance metrics (such as query_count, query_time, queue_wait, etc.) that gives us an idea about database health. If we detect an anomaly, we quickly find the root cause and fix the issue before it can affect website performance.

Optimize web server and PHP settings to accommodate changing traffic load

As the site traffic changes, various web server and PHP settings such as memory limit, concurrent connection limit, connection keepalive settings, etc. need to be adjusted to ensure smooth operations.

We periodically analyze the web server traffic data, and check if there are delays in serving web pages. Any noted anomaly is traced to relevant server settings, and fixed.

In addition to this, there would be cases of missing files (such as missing images, java script files, etc.) that causes significant delay in page loads. We analyze the web server logs, identify the cause of all kinds of errors, and fix them as soon as they are detected.

Optimized Magento contents and extensions to cut flab

A leading cause of site slowness is un-optimized content. This can include large media files, high number of CSS or Javascript files, or even resource hogging Magento extensions.

We periodically audit the loading speed of the most popular pages, and identify page components that causes the maximum delay. We then build and implement software or server administration solutions to remove those bottlenecks.

eCommerce customers expect a website to load within 2 seconds. In this aspect, Magento shops are intrinsically at a disadvantage because of its complex design.

Today we’ve seen how to do Magento speed optimization using open source software. Bobcares helps business websites of all sizes achieve world-class performance and uptime, using tried and tested website architectures.

If you’d like to know how to make your Magento store more reliable, we’d be happy to talk to you.