As a server management company, we are often consulted by online service providers for a variety of business decisions – which technology to use, what hardware to deploy, how to keep the system profitable, etc. Profitability of a hosting company or an IaaS provider directly depends on the number of customers that can be supported on a given infrastructure. For instance, if a technology enables putting 40 customers on a server, that can otherwise hold only 30, it can deliver better ROI (Return on Investment).

Virtualization is the de-facto method by which hosting providers and IaaS companies deliver multi-tenant hosting. Of these technologies, some offer greater server density than others. For example, container virtualization technologies such as OpenVZ can achieve a higher server density than true virtualization hypervisors like Xen or KVM. It’s then not a surprise that hosting businesses that cater to retail market often choose container virtualization systems.

For a long time, the container virtualization market was ruled by OpenVZ and its commercial variant, Virtuozzo. In May 2015, Canonical published a study that claimed that their new LXD/LXC container virtualization system achieved 14.5 times better server density than KVM. This was great news for hosting providers, as container virtualization systems usually offered a density of 1.5 – 2.0 times the server capacity, and LXD showed greater promise.

Recently, we were contacted by a VPS host to implement a high density server virtualization system using LXD/LXC. Infrastructure setup is a part of our server administration services. This is the story of how we created a systems design, built a high density system, and our observations on server density.

LXD/LXC server virtualization – design considerations

The use of LXD and LXC is mostly limited to the DevOps world, and as such, its default settings are not oriented for VPS hosting. So, for LXD to be a viable VPS hosting solution, we wanted the following to be resolved:

1. Resource limitation – It should be easy for us to set and change disk space limit, CPU limit and memory limit on LXC containers.
2. Quick provisioning based on templates – We should be able to provision a new container in a matter of seconds and we should be able to choose from a wide variety of server templates like Ubuntu, CentOS, LAMP, LNMP, etc.
3. Services exposed through public IP – The default LXD configuration didn’t expose the containers to a public IP. We wanted the containers to be visible on a public IP and customers to access Web, Mail, FTP etc.
4. Backups – We should be able to automatically take daily backups of containers and put them in a central backup repository. This is needed to restore service in case of a hardware failure.

Setting up a high density VPS server

The basic setup was pretty straight forward. LXD is included in Ubuntu 15.04. So, it was only a matter of running “apt-get install lxd" to get the hypervisor running.

Note: In older Ubuntu systems, the “ppa:ubuntu-lxc/lxd-stable” needs to be added to install LXD.

Putting the container on a public IP

Now, we had an LXD server, but its default configuration is to assign private IPs to containers that is not visible from the internet. To be able to assign public IPs, the default network interface of the host server should be bridged to the containers. For that, we converted the server ethernet (eth0) to a bridge (br0), disabled USE_LXC_BRIDGE in /etc/default/lxc-net, and set the lxc.network.link as br0 in the default LXC profile.

A server visible on the internet need to be secure. As is standard with all our VPS deployments, a set of security rules were then added to the network and firewall settings so that the VPS customer would be immune to a slew of common attacks prevalent in the internet.

Customizing the container and creating images

Now we had a container running a stable server image. What we now needed was images customized for various purposes such as Mail server, Web server, VPN server, etc., so that we could commission special purpose servers in seconds. For this, relevant packages were installed on the base server image, configuration settings were optimized for container environment, and firewall rules were updated to ensure smooth connectivity. Then these images were saved using the “lxc publish" command.

Note:  Based on the purpose, additional settings may have to be applied to the LXC container to cover all possible usage scenarios. For eg., LXC mount entries need to be adjusted to allow GUI (XOrg) display on VPN servers.

Configuring resource limits

Based on different VPS plans, the resources available need to be limited. The CPU and memory limits were changed for containers using the commands:

# lxc config set server01 limits.cpus 1
# lxc config set server01 limits.memory 500

The above command set the number of cores available to the container server01 to 1, and the memory to 500 MB. We saw that some container images had issues in booting up with a low memory setting. This was fixed by adjusting the swap space in the LXC container profiles.

For new containers, the resource settings were defined in profiles under /etc/lxc/ and those profiles were referred to, while creating new containers.

To limit the disk space, LXC containers were created on an LVM volume (which merits an entire article on its own), and the create command looked like this:

# lxc-create -t ubuntu1403-LNMP -n server03 -B lvm --fssize=5G

The above command created a container with disk space 5 GB.

Taking VPS backups

To take backups, we used the lxc-clone command:

# lxc-clone -P /path/to/backup/drive/ server01 server01-$(date)

This created a backup of the container server01 in our backup drive. Restoring this backup is as simple as copying this directory to /var/lib/lxd/containers/. A backup script was created to automate this process, and space was saved by compressing the archive.

Right now, the production server has 20 VPS instances and everything work like a charm. It is far from the capacity of the server, and going by the test results in our lab, this server (with 16 GB RAM) can easily take up to 30 VPS or more.

Conclusion

Server density has always been a critical factor in running a VPS hosting business. For long, container technology software like OpenVZ and Virtuozzo have ruled the roost in delivering economical VPS hosting solutions. Now, with the release of LXD, the VPS hosting business has a strong new choice which is as good as or even better than others. If the steady development is anything to go by, LXD could be the de-facto high-density VPS hosting solution sooner rather than later.