EC2: Fail to boot due to issues with the GRUB2 BLS Configuration File? We can help you.

We can use the grubby tool to manage the blscfg files and retrieve information from the /boot/loader/entries/.

Here, at Bobcares, we assist our customers with several AWS queries as part of our AWS Support Services.

Today, let us see how to recover the BLS configuration (blscfg) file found under /boot/loader/entries/ if it is corrupt.

EC2: Fail to boot due to issues with the GRUB2 BLS Configuration File

GRUB2 in RHEL 8 and Centos 8 uses blscfg files and entries in /boot/loader for the boot configuration.

Suppose the blscfg files are missing from this location or corrupted. In that case, grubby doesn’t show any results.

How to resolve this?

Moving ahead, let us see the methods our Support Techs recommend to resolve this query.

Since this procedure requires a stop and start of the instance, any data on the instance can be lost. So we make sure not to perform this procedure on an instance store-backed instance.

Attach the root volume to a rescue EC2 instance

Initially, we create an EBS snapshot of the root volume.

2. Then we open the Amazon EC2 console.

3. From the navigation pane, we select Instances > Impaired instance.

4. After that we go to, Actions > Instance State > Stop.

5. Under the Root device in the Description tab, we select /dev/sda1, and then the EBS ID.

6. Then we select Actions > Detach Volume > Yes, Detach. Here, we need to note the Availability Zone.

7. In the same Availability Zone we need to launch a similar rescue EC2 instance. This becomes the rescue instance.

8. Once it launches, we select Volumes and then choose the detached root volume of the impaired instance.

9. Eventually, we go to Actions > Attach Volume.

10. Here we need to select the rescue instance ID, and then set an unused device.

Mount the volume of the impaired instance

1. First, we connect to the rescue instance via SSH.

2. Then we run the lsblk command to view the available disk devices.

[ec2-user@ip-10-10-1-111 /]s lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
xvda 202:0 0 10G 0 disk
├─xvda1 202:1 0 1M 0 part
└─xvda2 202:2 0 10G 0 part /
xvdf 202:80 0 10G 0 disk
├─xvdf1 202:81 0 1M 0 part
└─xvdf2 202:82 0 10G 0 part

3. After that, we create a mount directory, and then mount the root partition of the mounted volume to this new directory.

In the above example, /dev/xvdf2 is the root partition.

sudo mkdir /mount
sudo mount /dev/xvdf2 /mount

4. We mount /dev, /run, /proc, and /sys of the rescue instance to the same paths as the new.

sudo mount -o bind /dev /mount/dev
sudo mount -o bind /run /mount/run
sudo mount -o bind /proc /mount/proc
sudo mount -o bind /sys /mount/sys

5. Finally, we start the chroot environment.

sudo chroot /mount

Regenerate the blscfg files

1. Here, we run the rpm command and take note of the available kernels in the instance.

[root@ip-10-10-1-111 ~]# rpm -q --last kernel
kernel-4.18.0-147.3.1.el8_1.x86_64 Tue 21 Jan 2020 05:11:16 PM UTC
kernel-4.18.0-80.4.2.el8_0.x86_64 Tue 18 Jun 2019 05:06:11 PM UTC

2. After that, to recreate the blscfg file, we run:

sudo kernel-install add 4.18.0-147.3.1.el8_1.x86_64 /lib/modules/4.18.0-147.3.1.el8_1.x86_64/vmlinuz

Make sure to use your kernel version number.

The blscfg for the designated kernel regenerates under /boot/loader/entries/.

[root@ip-10-10-1-111 ~]# ls /boot/loader/entries/
2bb67fbca2394ed494dc348993fb9b94-4.18.0-147.3.1.el8_1.x86_64.conf

3. If necessary, we can repeat step 2 for other kernels on the instance. However, the latest kernel is set to the default kernel.

4. Then to see the current default kernel we run the grubby command –-default kernel.

sudo grubby --default-kernel

5. Eventually,we exit from chroot and unmount the /dev, /run, /proc, and /sys mounts.

Exit
sudo umount /mount/dev
sudo umount /mount/run
sudo umount /mount/proc
sudo umount /mount/sys
sudo umount /mount

6. Later, we mount the device back to the original instance with the correct block device mapping. Now the device boots with the default kernel.

[Stuck in between? We are here to assist you]

Conclusion

In short, we saw how our Support Techs go about EC2: Fail to boot due to issues with the GRUB2 BLS Configuration File.