In our Server administration services, we perform a wide range of tasks from server hardening to service optimization. Kernel updates are vital in our services, especially in scenarios where a vulnerability or exploit make the servers susceptible to hacks.

We perform kernel updates in all Linux flavours, and the procedure varies with the type of the OS installed in the server and its version. If not done with proper planning and caution, a kernel update can mess up with your server.

4 steps that we follow to update Centos 6 kernel

If the updated kernel is by any means incompatible with the server, it can fail to boot. This can lead to kernel panic and server downtime, and you may have to contact the data centre to bring it back.

But with some safety measures and fallback configuration, it is possible to update Centos 6 kernel without causing a downtime for the server. Here’s how we do it with our 4-point update procedure.

1. Take a copy of the grub configuration

Before the kernel upgrade, it is always safe to take a copy of the grub.conf file. This is the config file that lists the operating systems to boot in GRUB’s menu interface during a server reboot.

If the grub.conf gets corrupt during edits, the server can fail to boot. The backup can come handy to retrieve the entries, in case something goes wrong while editing it.

2. Perform the kernel upgrade

In CentOS servers, the yum utility can be used to update kernel easily, by fetching the latest available versions from the yum repository. Before updating, we ensure that the repository is defined correctly, or else it may install wrong kernel.

To ensure that customers are not affected by the downtime involved during reboots, we always schedule the updates to off-peak hours of the customer and confirm that the ‘konsole’ access to the server is available, in case an emergency pops up.

3. Configure fallback entry in grub

Most server administrators perform the reboot to load the new CentOS 6 kernel, soon after the yum update. While majority of the servers reboot fine with the new kernel, we’ve seen instances where some servers fail to boot with the new kernel.

Since it is not easy to predict which server would fail to boot after an update, we always configure a fall-back entry in the grub to prevent a server downtime. We’ll see how.

A typical grub.conf in CentOS 6 server has entries like these:

title CentOS (2.6.18-238.19.1.el5PAE)
root (hd0,0)
kernel /vmlinuz-2.6.18-238.19.1.el5PAE ro root=LABEL=/
initrd /initrd-2.6.18-238.19.1.el5PAE.img
title CentOS (2.6.18-238.9.1.el5PAE)
root (hd0,0)
kernel /vmlinuz-2.6.18-238.9.1.el5PAE ro root=LABEL=/
initrd /initrd-2.6.18-238.9.1.el5PAE.img
title CentOS (2.6.18-194.32.1.el5PAE)
root (hd0,0)
kernel /vmlinuz-2.6.18-194.32.1.el5PAE ro root=LABEL=/
initrd /initrd-2.6.18-194.32.1.el5PAE.img

These entries denote the kernels that are configured to load during a reboot. The first “title” is given the number 0, next “title” 1 and so on. Thus there are three kernels available in this case.

In the grub.conf file, there is an entry that is used to define the “default” kernel, which is booted when no option is specified. This is edited and set to the running kernel number instead of the latest updated kernel.

For the newly updated kernel version, we set “panic=5″ in the line which starts with “kernel” corresponding to the newest kernel entry. This will reboot the server on kernel panic after 5 seconds.

Next, we access the grub console and set the default kernel to boot. Since the default entry is set to previous running kernel, it would fall back to that kernel and boot the server, thus avoiding any downtime.

4. Reboot and validate the new kernel

Once the fall-back entry is set in grub.conf, we reboot the server. If there are no issues with the new kernel, the server will boot into the new one or else, it would load the previous one.

After confirming that the server is working fine with the new kernel, we edit the /etc/grub.conf again and set it as the default one and remove the panic field in the entry.

[ Worried about breaking your server? Click here to update your kernel with our expert help right now – we’re online 24/7. ]

In short..

Kernel updates require proper planning and precautionary measures to be taken, inorder to avoid a server downtime. At Bobcares, we examine the server and prepare a custom checklist to keep the server software updated with minimal downtime.