How a UK web host turned brickbats into bouquets using 24/7 support and preventive server maintenance
Spring 2013 – Domino Host* (name changed), a UK based web hosting company was getting pounded by bad reviews in popular webmaster forums.
“Worst service ever! Mail’s been down for 10 hours now, and no response yet from Domino support. Are these guys in business still?”
“This is the 3rd time this month my site has gone down. Domino has really gone down the hill in the past year. Would NOT recommend!“
Things were not looking good for Matt Davis*, the founder of Domino Host.
It was a completely different picture in 2012. Business was good. Orders were coming in from all over the country – Matt had to add 3 new servers to accommodate the increased user base. Domino offered all the latest features in web hosting market, and users loved the service.
However, by late 2012, things started looking grim. The servers were plagued with performance issues. Every day, one or two servers needed reboots. Support tickets kept piling up in help desk with responses taking 12 hours or more. The phone rang off the hook affording Matt zero time with family or to attend other business matters.
To make matters worse, by early 2013, negative reviews started coming up in popular web host review websites and web master forums. Account cancellations were a daily affair, and new signups had all but dried up.
Bad customer reviews – How it all started
Like many shared hosting providers, Domino Host had humble beginnings with a limited service portfolio. Matt started off as a website designer, grew to a reseller web host, and finally took his own server to host shared hosting accounts.
The servers were configured to host only WordPress websites. A basic control panel was used which provided PHP, MySQL, DNS and Mail management. Things were not complicated. The company owner Matt knew how to take care of server issues.
As business boomed, Domino started getting customers from a much more diverse background, such as busy forums, large blogs, gallery owners, etc. With a diverse user base, came requests for more features like one-touch-installers, point-in-time backups, webmail, web analytics, etc. Domino was only happy to oblige.
However, all these additional features came at a cost. Servers were frequently under high load. Customers faced frequent web and mail outages. In addition, many customers asked for detailed assistance in using new features such as spam filter configuration, app installers, webmail, and so on.
Matt found himself firefighting all the time, and no matter how hard he tried, new issues kept piling up.
The search for a solution
Things came to a head in April 2013, and Matt decided to hire Bobcares to fix the operations. He contacted Bobcares with a specific request – “Help me reverse the trend of negative reviews, and get me positive reviews“.
The task was cut out for Hamish Oscar, the account manager from Bobcares for Domino Host.
From Matt’s account, it was pretty clear that something had happened in the past 6 months that led to this steep rise in customer complaints. To find a lasting solution, I had to find out what’s causing these issues.
Sr. Systems Engineer, Bobcares
Hamish started off by sifting through support requests, negative reviews, server event logs, and software change logs in search of a common thread. By the end of that week, he had zeroed in on two core complaints of Domino customers, and a two-pronged strategy to fix them:
- Prevent recurring server issues with preventive maintenance – A significant number of negative reviews stated, “This is the Nth time my service went down“. To fix this, Hamish planned to use 24/7 Server Monitoring, to quickly recover from server faults and Preventive Server Maintenance to prevent server faults.
- Reverse the reputation of “slow tech support” by 24/7, expert tech support – Many account cancellations cited slow or zero response. Statements like this was typical – “All my mails were bouncing. I couldn’t reach my business contacts, and Domino did NOTHING to help me.” Hamish planned to fix this by ensuring Expert, 24/7 Tech Support, staffed by engineers who can resolve issues in as little as 5 mins.
Building the right team
The first order of business was to build a team that could fix server issues and handle tech support round the clock. Hamish used a “Dedicated Seat” team configuration, which guaranteed the availability of an engineer at any point of time. “By using a Dedicated Seat, I had 6 engineers to take care of support” Hamish says. “3 engineers would work on 8 hour shifts covering 24 hours. During weekends, I could fall back on the 3 stand-by engineers to cover the shifts.”
With the kind of challenges faced by Domino Host, the team knew it would be an uphill task.
Prior to this I had worked with many other hosting companies. From the brief delivered by Hamish, it was clear that I could put my knowledge to full use. The support goals called for recovering hacked websites, tracing spam attacks, fixing broken web apps, and more – each to be completed in 10 mins or less! It sure was challenging.
Support Engineer, Bobcares
Sambhu’s team mate Balaji Pai shines a light at the bigger picture – “Domino’s customers are mostly small businesses. A bounced mail, or a broken website could easily mean lost business opportunities. In the teams I worked before, customers used to get back to me saying how our timely support helped them bag a deal, or save expensive website overhaul. Moments like that reinforce customer loyalty, and that’s the difference I wanted to bring to Domino Host.”
Defining the support goals
Once the team was in place, Hamish broke down the support strategy into several measurable support goals:
- Achieve zero server downtime through 24/7 monitoring and emergency administration
- Achieve zero recurring complaints through preventive server maintenance
- Respond to 100% support requests within 30 mins
- Resolve at least 50% support tickets in the first contact
- Achieve zero negative feedback on support experience
1. Zero service downtime
Here’s the thing about server downtimes – it doesn’t happen in an instant. It takes several minutes for the server load to shoot up, services to stop responding, and finally the server to become inaccessible. This is where 24/7 availability of a server expert makes a difference. An experienced engineer can detect early signs of server trouble, and prevent a server crash by killing the root source of the issue.
Sambhu explains – “I always keep an eye on the server resource utilization trends. The moment a resource such as I/O or memory is used higher than normal, I login to the server and investigate. Many a time, I was able to quarantine resource abusers or malware that threatened server stability. Such early intervention protects the uptime of countless businesses hosted in the server.”
2. Zero recurring server issues
One of the most frustrating things for a hosting customer is to report the same server issue over and over again. In the case of Domino, issues like mail bounce was common. Every week, websites in shared servers used to get hacked, and spam sent out from them. It resulted in the server IPs being blocked by Hotmail, Gmail, etc. This affected all customers hosted in that server. The plan called for such issues to be prevented.
There was this one time when I had to call my phone company 4 times to get an issue fixed. I was so frustrated” Balaji says. “I know that’s what’s going through the mind of each customer when they contact us for a recurring complaint. We planned daily security and performance audits on the servers, and implemented solutions to methodically cut down instances of mail, web and database downtimes.
Former Support Engineer, Bobcares
3. 30 mins response for 100% of tickets
“No one likes to wait, especially when something is not working as promised” says Hamish. “While working with many other hosting companies, I realized that the chance of a positive review significantly improved if a solution was delivered within 30 mins.”
When Hamish laid out his plan, we immediately understood the significance of making customers not just happy, but truly delighted with the support experience. Super fast response was key. All of us in the team could resolve issues in as little as 10 minutes. So, even if we had to work on 2-3 tickets at a time, we were confident of resolving all issues within 30 mins.
Priya Agni (Team Lead)
Sr. Support Engineer, Bobcares
4. 50% tickets to be resolved in first contact
“We as humans seek closure or completeness,” says Hamish, “and it is important to understand that the customer would be waiting impatiently at the other end until an issue is fully resolved. No matter how fast you respond, unless the issue is resolved, it’s forcing the customer to spend more time and effort to keep the ticket live. It’s not going to create happy customers, which is why I wanted 50% First Contact Resolutions.”
It was a challenging target, but not impossible. Balaji says, “We planned on responding to each customer within 30 mins, but with the 50% First Contact Resolution goal, we’d be trying to resolve 50% issues within 30 mins. Yes, it was a challenge, but we brainstormed and came up with ways to reduce followups with customers. For eg., we found ways to capture the customer’s IP from the ticketing system, which saves a follow-up asking for connecting IP.”
5. Zero negative feedback on support
Matt’s request to Bobcares was to help him reduce customer complaints, and get him positive reviews. Hamish converted this vision into a support goal so that he can measure on a day-to-day basis how close they are to achieving this objective.
“A person has to take extra effort to send in a Thank You message,” says Priya, “and to make a customer do that, you’ll need to meet and exceed their expectations from support. All engineers have been doing it in their previous teams, and all I had to do was to give specific directions on what more could be done. For eg., along with fixing an issue, we’d give customers easy steps on how to avoid it next time. That extra step makes them feel valued, and they’d be happy to give a positive review.”
Path to recovery
In the next 8 weeks, the team made rapid progress in achieving the goals set forth in the solution strategy. The number of customer complaints came down, and the servers became stable.
The effect of 24/7 server monitoring and maintenance
At the time the team took over Domino support, there were more than 40 serious server incidents per day that led to degraded service. With Hamish’s team keeping a watch on the servers 24/7, such server incidents were prevented from developing into full blown server downtimes.
Further, the causes for recurring server issues were systematically cut down through daily performance and security audits. A series of mitigation steps such as resource limits, mail rate limits, file upload scanning, web access scanning, server connection limits, and more was implemented as part of the preventive maintenance.
Taken together, these steps resulted in stable servers that never needed emergency reboots.
How 24/7 expert support made a difference
Before Bobcares came into the scene, Domino tickets used to lie un-responded for up to 12 hours. By reducing the number of tickets through preventive server maintenance, and by having experienced engineers manning support 24/7, the average response time was brought down to 18 minutes within 16 weeks.
As per Hamish’s support strategy, a critical focus of the team was to increase the number of positive feedbacks. Customer ratings were audited on a daily basis, and support interactions that didn’t receive positive feed backs were analyzed for opportunities for improvement. Many nuances like referring to previous interactions, adding related hosting tips, etc. were introduced to show customers that we truly cared about their ticket.
The effectiveness of each measure was constantly measured and tweaked, until the percentage of customers leaving a Thank You message rose to 12/day at the end of 16 weeks.
Effect on customer perception
The new support process started showing results as early as 4 weeks, but continued improving for 12 weeks. By the end of June, Hamish introduced an option to leave a thank you note on Domino Host’s Facebook page, Google Business and other social forums. Positive reviews slowly started coming in, and flooded out the earlier negative reviews that were present in the page.
Domino Host has come a long way since then, and is now a popular web host offering Shared, VPS, Dedicated, Cloud and specialized services. Hamish’s team is still an integral part of Domino’s success story.
*Names of client and company changed as per our Non Disclosure Agreement. Bobcares provides 100% white labeled support.