cPanel Log Rotation – Effective solution to log file growth

A log file can grow without bounds if it is not managed well. Larger log files are relatively harder to manipulate and also increase in the size of log file can result in file systems to run out of space. Opening, closing and manipulating data for very large file size consumes system resources and will therefore slow down the server.

An efficient solution to this problem is log rotation. It allows automatic rotation, compression, removal and mailing of log files. Log files can be rotated daily, weekly, monthly or when it grows too large.

Normally, log rotate is run as daily cron job (read more about cron management here), it will not modify a log file multiple times in a day unless the criteria for that log file is based on log file’s size or force option is being used. Log files can be rotated on daily or weekly basis by using the log rotate package in WHM. Files will only be rotated when they grow larger than the threshold value of 300MB.

cPanel log rotation configuration feature allows to choose which log files should be manipulated by cPanel’s cpanellogd daemon. This daemon will take specified log files, compress them, and store them to /usr/local/cpanel/logs/archive/ with a file for each month. The files stored in that directory by cpanellogd will be left there until it is deleted manually or through an automated script.

The cpanellogd daemon uses /etc/logrotate.conf as its primary configuration file. Log rotation instructions for specific log files can be loaded from /etc/logrotate.d directory. Some of these directives are:

• compress – old versions of log files are compressed
• monthly, weekly, daily – intervals at which files are to be rotated
• compresscmd – specifies which command is to be used to compress log files (by default it is gzip)
• copy – makes a copy of the log file, but doesn’t change the original at all
• delaycompress – postpone compression of the previous log file to the next rotation cycle
• copytruncate – truncate the original log file in place after creating a copy, instead of moving the old log file and optionally creating a new one
• mail address – when a log is rotated out-of-existence, it is mailed to a mail address.

An example of a configuration file is given below:

$ cat /etc/logrotate.d/apache-error
/usr/local/apache/logs/error_log {
 weekly
 rotate 4
 size 300M
 compress
 sharedscripts
 postrotate
 /sbin/service httpd reload > /dev/null 2>/dev/null || true
 endscript
}

In the above configuration, Apache error log will be rotated weekly provided it has a size of above 300 MB. 4 versions of old logs will be kept, and they will be compressed to save space. Once the log is rotated, Apache will be restarted to register the new log file.

Not sure where the various log files are? Here’s a complete list log file locations in a cPanel server.