“Noisy neighbors” are customers who use up more than their fair share of resources in a virtualized server. Such over-use of resources (aka resource hogging) affects server performance, especially if a VPS provider runs high-density servers. Though server virtualization provides cost-effective hosting, it involves sharing of resources such as CPU, RAM, I/O, network, etc. among the VPS accounts.

To ensure that every account gets its fair share, these resources have to be controlled per VPS instance. Recently, we setup a high density server virtualization system for a VPS provider. QoS (Quality of Service) for CPU and memory was a key feature in this system. QoS enforces fair usage of resources for every customer.

In our server virtualization solution, LXC containers were used to host VPS accounts. For resource control, we used a feature of Linux kernel called cgroups (control groups), which limits the resources used by processes. Here is how we did it.

Memory QoS

To limit memory usage per container, the parameter “lxc.cgroup.memory.limit_in_bytes” was configured in the container config file.

root@host:/var/lib/lxc/Ubuntu-01# echo "lxc.cgroup.memory.limit_in_bytes=512M" >> config

To set the memory usage including the swap memory, “lxc.cgroup.memory.memsw.limit_in_bytes” was used. When this limit is reached for a container, it would automatically kill applications and save other containers from failing.

We set the memory limit per container based on the plan purchased by that user. We also set a memory soft limit for each container, using the parameter “lxc.cgroup.memory.soft_limit_in_bytes”, to limit memory in case of a memory contention.

CPU QoS

CPU usage per VPS can be set in terms of cores and shares. The parameter “lxc.cgroup.cpuset.cpus” in the container’s config file was used to limit the CPU cores allotted to each VPS.

CPU resources were configured based on the VPS plan. For a container that required a single core of CPU for processing, the following setting was used in its config file.

root@host:/var/lib/lxc/Ubuntu-01# echo "lxc.cgroup.cpuset.cpus=2" >> config

To divide the CPU utilization time between each container, we configured the parameter “lxc.cgroup.cpu.shares”. For instance, container 1 could be allotted 100 shares and container 2 could be given 50 shares. This would provide container 1 with double the CPU than container 2.

root@host:/var/lib/lxc/Ubuntu-01# echo "lxc.cgroup.cpu.shares=50" >> config

Resource requirements of a VPS may depend on various matters. Anything, like a memory-hogging application or a sudden increase in business, can cause a spike in resource usage. To identify and mitigate such spikes, we also configured a resource monitoring script.

When a VPS was seen as always hitting its limit, the monitoring script helped us to identify the applications that hogged the memory or CPU. If the applications were not optimized, our 24/7 team tweaked them for optimal memory and performance. If the resource usage was due to increased business, we initiated plan upgrades for those VPS owners and allotted more resources to that VPS.

Resource control and constant performance monitoring is crucial for smooth running of a VPS hosting business. In this post, we discussed how we configured resource control for QoS, in a server virtualization solution that used LXC containers. Bobcares helps hosting providers deliver high-performance and cost-effective solutions through custom configuration, quick provisioning, resource control, security patching and preventive maintenance of server virtualization solutions.