Many developers now use Docker to host their apps in production servers.

But Docker in production servers often cause resource bottlenecks – especially Docker container memory overhead.

Here at Bobcares.com, we help web developers and digital marketers to setup and maintain Docker based web hosting servers.

And a common issue we resolve is containers crashing due to “Out of memory” errors as a result of traffic spikes, spam attacks, etc.

Here’s a typical error log:

Jan 31 15:58:03 dockerhost1 kernel: [5615240.584133] Out of memory: Kill process 24707 (mysqld) score 19 or sacrifice child
Jan 31 15:58:03 dockerhost1 kernel: [5615240.588688] Killed process 24707 (mysqld) total-vm:5116432kB, anon-rss:653964kB, file-rss:0kB, shmem-rss:0kB
Jan 31 15:58:03 dockerhost1 kernel: [5615241.070421] oom_reaper: reaped process 24707 (mysqld), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB

What causes Docker container memory overhead?

By default, docker does not impose a limit on the memory used by containers.

When the traffic in one container increases, it’ll grab more memory from the Docker host to run its processes.

When the Docker host runs out of memory, it’ll kill the largest memory consumer (usually the MySQL process), which results in websites going offline.

How to prevent out of memory errors

Here at Bobcares, we help our customers prevent Docker crashes by setting up memory limits.

By assigning a limit to each container, we limit the damage due to traffic spike or resource abuse to just one website.

How to determine the memory limit

Now, the question is how do you find out the normal usage boundary for a container?

For that we use the statistics function of Docker, and chart the memory usage over a period of 1 week.

A sample output looks like this:

The memory values are written into a file, and the 99 percentile value is calculated after 1 week.

A good sample period if 1 week, where we take into account traffic patterns of businesses through all weekdays, work hours and week ends.

We filter out the the outlier values (solitary spikes), and determine what’s the memory value where the site remains functional 99.9% of the time.

Setting the Docker memory limit

Once we know the safe memory limit for each container, we modify the Docker Compose files with the setting mem_limit like so:

container_name: wpsite1.com
image: wordpress-custom
mem_limit: 650m
environment:
  WORDPRESS_DB_HOST: 10.0.0.21
  WORDPRESS_DB_NAME: wpsite1db
  WORDPRESS_DB_USER: wpsite1db_user
  WORDPRESS_DB_PASSWORD: 'KJHu63deks'
  VIRTUAL_HOST: wpsite1.com, www.wpsite1.com

A quick restart of the container will limit the memory to the set value.

Use caution while setting limits

We’ve seen Docker users set arbitrary memory limits based on how big their database is.

This often leads to frequent website crashes in containers with very low memory limits.

Similarly, we’ve seen developers use bulky images (eg. CentOS) to host applications.

Many times, we’ve reduced memory usage just by choosing a lean OS like Alpine Linux and building a custom Docker image.

If you are not sure how to fix the memory issues in your Docker host, click here to talk to our Docker experts. We’re online 24/7 and can attend your request in a few minutes.

Conclusion

Containers crashing due to “Out of memory” errors is a common issue in Docker servers. Today we’ve seen how to use memory limits to prevent Docker container memory overhead.