A large Nagios installation creates an enormous amount of disk activity. RAM Disk in Nagios XI can help cure this.

Here at Bobcares, we assist our customers with several Nagios-related issues as part of our Server Management Services.

Today, we’ll see how to free an enormous amount of disk activity, by adding RAM disks to the local filesystem.

 

Nagios XI and RAM Disk

Although enough CPU power on a Nagios server is important, a hardware limitation to a Nagios system is disk I/O.

If the hard disk cannot keep up with the constant traffic flow, a large number of CPUs are going to wait for the disk in order to write new information to the disk.

This can cause check latencies to soar even though the CPU usage appears within a safe range.

Sometimes using creative file mounts on separate partitions or purchasing extremely fast disks for servers help.

Today, we will discuss how to use a RAM disk to boost performance on a Nagios XI server

 

Utilizing RAM Disk in Nagios XI

Along with utilizing RAM Disk in Nagios XI let us focus on various aspects of its disk activity.

• Key Files That Affect Disk I/O

Now let us look at some of the key files that create disk activity on a Nagios install.

/usr/local/nagios/var/status.dat

This file contains all the “live” information on the monitoring environment. They update every 10-20 seconds (as specified in nagios.cfg) with all current status information.

/usr/local/nagios/var/objects.cache

This file stores all object configuration data for Nagios. It only updates upon a restart of the Nagios process.

/usr/local/nagios/var/host-perfdata & service-perfdata

These files may be in a different location for Core install. However, they function as an intermediary file for PNP’s NPCD daemon that processes performance data results. These files update about every 10-15 seconds.

/usr/local/nagios/var/spool

This directory tree acts as a dropbox for all incoming check results. The disk activity is almost constant since both Nagios and NPCD continually creates result files.

/usr/local/nagios/var/*.log

These log-files are worth mentioning. Minimize all unnecessary logging, particularly with performance data processing.

We can minimize logging by editing: /usr/local/nagios/etc/pnp/npcd.cfg and /usr/local/nagios/etc/pnp/process_perfdata.cfg, and setting the log level to 0.

• Automatic RAM Disk Installation

The easiest way to install RAM disk on the Nagios XI is to use install_ramdisk.sh script. We establish a terminal session to Nagios XI server as the root user and execute:

# cd /tmp
# wget https://assets.nagios.com/downloads/nagiosxi/scripts/install_ramdisk.sh
# chmod +x install_ramdisk.sh
# ./install_ramdisk.sh

The script will prompt for the size of the RAM Disk.

The install_ramdisk.sh script works for the following operating systems:

1. Red Hat Enterprise Linux (RHEL), CentOS, Oracle Linux
   ◦ 6.x 32-bit and 64-bit
   ◦ 7.x 64-bit
2. Debian
   ◦ 8.x 32-bit and 64-bit
   ◦ 9.x 32-bit and 64-bit
3. Ubuntu
   ◦ All active Long Term Support (TLS)
   ◦ 32-bit and 64-bit

If we have an older distribution, we need to follow the manual install instructions. The same is valid when we have a “non-standard” Nagios XI instance or if we want to set up the RAM disk in a “non-default” location.

• Manual RAM Disk Installation

This should only be followed if using the install_ramdisk.sh script is not possible.

Initially, we need to determine the size of the RAM disk. We can determine the recommended size of the RAM disk based on the total number and hosts and services that we are monitoring.

Total Number of Hosts + Services              RAM Disk Size
Less than 1000                                                  100 MB
1000 to 5000                                                      300 MB
More than 5000                                                500 MB

It is always a good idea to use a higher number than the “recommended size” in order to give some leeway and allow for growth.

[Need help to proceed? We are available 24*7]

 

INIT or SYSTEMD and RAM Disk in Nagios XI

The method we use to implement RAM disk is different on different OS versions due to how services are controlled in the back-end.

Let us look in detail at the steps depending on our OS version.

Each method requires us to establish a terminal session to the Nagios XI server as a root user to complete these steps.

• INIT

a) Create a RAM Disk Directory

We execute these commands to create the base directory:

# mkdir /var/nagiosramdisk/
# chown -R nagios.nagios /var/nagiosramdisk

b) Editing INIT Files

The /etc/init.d/nagios script should have the following few lines added to it (around line 69):

if test -f /etc/sysconfig/nagios; then
. /etc/sysconfig/nagios
fi

If these lines are missing, we add them to the top of the /etc/init.d/nagios file below the comments.

Then we create a file called nagios in the /etc/sysconfig/ directory and set permissions:

# mkdir -p /etc/sysconfig
# cd /etc/sysconfig/
# touch nagios
# chown nagios:nagios nagios
# chmod 775 nagios

Next, we open the nagios file in vi:

# vi nagios

We add the following lines:

USE_RAMDISK=1
RAMDISK_DIR=/var/nagiosramdisk
RAMDISK_SIZE=100
if [ “`mount |grep “${RAMDISK_DIR} type tmpfs”`”X == “X” ]; then
mount -t tmpfs -o size=${RAMDISK_SIZE}m tmpfs ${RAMDISK_DIR}
fi
mkdir -p -m 775 ${RAMDISK_DIR} ${RAMDISK_DIR}/tmp ${RAMDISK_DIR}/spool
${RAMDISK_DIR}/spool/checkresults ${RAMDISK_DIR}/spool/xidpe ${RAMDISK_DIR}/spool/perfdata
chown -R nagios:nagios ${RAMDISK_DIR}

The command starting with mkdir and ending with perfdata is all on one line

Finally, save, and close the file.

• SYSTEMD

a) Create RAM Disk Directory

We execute these commands to create the base directory:

# mkdir /var/nagiosramdisk/
# chown -R nagios.nagios /var/nagiosramdisk

b) Create SYSTEMD Service

The following steps will create a ramdisk.service file, the location of this file differs depending on our OS. We change into the correct directory as per:

1. RHEL, CentOS, Oracle Linux
   ◦ cd /usr/lib/systemd/system/
2. Ubuntu, Debian
   ◦ cd /lib/systemd/system/

To create the ramdisk.service file and set permissions:

# touch ramdisk.service
# chown nagios:nagios ramdisk.service
# chmod 775 ramdisk.service

Then we open the ramdisk.service file in vi:

# vi ramdisk.service

Next, we need to add lines on the following page:

[Unit]
Description=Ramdisk
Requires=local-fs.target
After=local-fs.target
Before=nagios.service
[Service]
Type=simple
RemainAfterExit=yes
Restart=always
ExecStartPre=/usr/bin/mkdir -p -m 775 /var/nagiosramdisk
ExecStartPre=/usr/bin/mount -t tmpfs -o size=100m tmpfs /var/nagiosramdisk
ExecStartPre=/usr/bin/mkdir -p -m 775 /var/nagiosramdisk/tmp
/var/nagiosramdisk/spool /var/nagiosramdisk/spool/checkresults
/var/nagiosramdisk/spool/xidpe /var/nagiosramdisk/spool/perfdata
ExecStart=/usr/bin/chown -R nagios:nagios /var/nagiosramdisk
[Install]
WantedBy=multi-user.target

The third instance of the ExecStartPre line ending with perfdata is all on one line

Finally, we save and close the file.

Then we need to issue this command for the OS to detect the new service:

# systemctl daemon-reload
# systemctl enable ramdisk.service

 

Editing Configuration Files to utilize RAM Disk in Nagios XI

Furthermore, Let us see how our Support Techs edit the configuration files.

• /usr/local/nagios/etc/nagios.cfg

First we open up the /usr/local/nagios/etc/nagios.cfg file in vi:

# vi /usr/local/nagios/etc/nagios.cfg

Then we update the following lines:

service_perfdata_file=/var/nagiosramdisk/service-perfdata
host_perfdata_file=/var/nagiosramdisk/host-perfdata
check_result_path=/var/nagiosramdisk/spool/checkresults
object_cache_file=/var/nagiosramdisk/objects.cache
status_file=/var/nagiosramdisk/status.dat
temp_path=/var/nagiosramdisk/tmp

Once done, we save and close the file.

• /usr/local/nrdp/server/config.inc.php

Initially we open /usr/local/nrdp/server/config.inc.php in vi:

# vi /usr/local/nrdp/server/config.inc.php

Then we update the following line:

cfg[“check_results_dir”]=”/var/nagiosramdisk/spool/checkresults”;

Finally, save and close the file.

• /usr/local/nagiosxi/html/config.inc.php

To begin with, we open /usr/local/nagiosxi/html/config.inc.php in vi:

# vi /usr/local/nagiosxi/html/config.inc.php

Then we update the following lines:

$cfg[‘xidpe_dir’] = ‘/var/nagiosramdisk/spool/xidpe/’;
$cfg[‘perfdata_spool’] = ‘/var/nagiosramdisk/spool/perfdata/’;

In the end, we save and close the file.

• /usr/local/nagios/etc/pnp/npcd.cfg

First we open up /usr/local/nagios/etc/pnp/npcd.cfg in vi:

# vi /usr/local/nagios/etc/pnp/npcd.cfg

Next, we update the following line:

perfdata_spool_dir = /var/nagiosramdisk/spool/perfdata/

When finished, save and close the file.

• /usr/local/nagiosmobile/include.inc.php

Initially, we open up /usr/local/nagiosmobile/include.inc.php in vi:

# vi /usr/local/nagiosmobile/include.inc.php

Then we update the following lines:

$STATUS_FILE = “/var/nagiosramdisk/status.dat”;
$OBJECTS_FILE = “/var/nagiosramdisk/objects.cache”;

Once done, we save and close the file.

[Coonfused with the edits? We’d be happy to assist]

 

Update Performance Data Commands

We login to Nagios XI and navigate to Configure > Core Config Manager > Commands > >_ Commands.

In the search field, we type file-bulk and press Enter.

The screen will update with the two commands process-host-perfdata-file-bulk and processservice-perfdata-file-bulk.

Make a copy of the existing commands, prior to modifying them. Leave the “old” commands (copies) inactive. This can be helpful if we need to go back.

Both of these commands need to be updated with the new directory locations as follows:

process-host-perfdata-file-bulk:

# /bin/mv /var/nagiosramdisk/host-perfdata /var/nagiosramdisk/spool/xidpe/$TIMET$.perfdata.host

process-service-perfdata-file-bulk:

# /bin/mv /var/nagiosramdisk/service-perfdata /var/nagiosramdisk/spool/xidpe/$TIMET$.perfdata.service

After making these changes we click the Apply Configuration button.

It is possible that the Apply Configuration will fail the first time. Simply click the Try Again button and it will succeed the second time.

 

Restart Services

After making all of these changes, we execute the following commands in the terminal session:

INIT

# service nagios restart
# service httpd restart
# service npcd restart

SYSTEMD

# systemctl restart nagios.service
# systemctl restart npcd.service

1. RHEL, CentOS, Oracle Linux
   ◦ systemctl restart httpd.service
2. Ubuntu, Debian
   ◦ systemctl restart apache2.service

[Can’t figure out how to utilize RAM Disk in Nagios XI? We are here for you!]

 

Conclusion

To conclude, making use of a RAM disk can provide huge performance improvements on larger systems or any system where check latencies are greater than 2 seconds. Today we saw how our Support Engineers go about with this.