One of the most common queries we get from our OnApp customers  is something along the lines of ” how to resolve onapp backup failure due to multipath?” We can help you with fixing this issue!

As part of our Server Virtualization Technologies and Services, we assist our customers with several OnApp queries.

Today, let see how our Support Engineers help our customers to fix this OnApp backup failure issue.
 

OnApp backup failure due to multipath

 
Our customers often report to us that they receive backup failure error messages while taking backup on the server.

Now, let see the error messages and the resolution for the onapp backup failure issue.

We will recieve the following errors while taking backup due to multipath issue:

Running: mount -t ext4 -o noatime,user_xattr,acl /dev/sdk1 /mnt/onapp-backup-yolzefqirpdpek
mount: special device /dev/sdk1 does not exist
Fatal: Execution of mount -t ext4 -o noatime,user_xattr,acl /dev/sdk1 /mnt/onapp-backup-yolzefqirpdpek failed

Running: ntfsclone -q -s -O /onapp/backups/i/i/iignblsablnmyt /dev/sdl1 > /tmp/onapp-tmp-ntfsclone-sdl.log 2>&1 || ( cat /tmp/onapp-tmp-ntfsclone-sdl.log; rm -f /tmp/onapp-tmp-ntfsclone-sdl.log; exit 1 )
ntfsclone v2016.2.22 (libntfs-3g)
ERROR(2): Failed to check '/dev/sdl1' mount state: No such file or directory
Fatal: Execution of ntfsclone -q -s -O /onapp/backups/i/i/iignblsablnmyt /dev/sdl1 > /tmp/onapp-tmp-ntfsclone-sdl.log 2>&1 || ( cat /tmp/onapp-tmp-ntfsclone-sdl.log; rm -f /tmp/onapp-tmp-ntfsclone-sdl.log; exit 1 ) failed

Both error mentioned above is because the multipath is trying to interfere with a temporary iSCSI connection between the HV and the backup server.

We can resolve this issue by editing the /etc/multipath.conf file on the backup server.

On the configuration file, we should add the following lines under the blacklist section.

device {
vendor "IET"
product "VIRTUAL-DISK"
}

For example, please refer to the following to see how to add the above lines in the multipath.conf file:

# cat /etc/multipath.conf
## Use user friendly names, instead of using WWIDs as names.

defaults {
user_friendly_names yes
find_multipaths yes
#from IBM
#polling_interval 30
}

blacklist {
wwid 26353900f02796769
devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"
devnode "^hd[a-z]"
devnode "sda"

device {
vendor "IET"
product "VIRTUAL-DISK"
}
}

After adding the lines on the configuration file, we need to reload the multipathd service by using the following command:

service multipathd reload

The issue will be resolved after reloading the service multipathd.

Also note that this is applicable only for the environments such as OnApp version 3.2 or later, LVM datastore and dedicated backup server.

[Do you need more assistance with OnApp queries? We would be happy to do it for you]
 

Conclusion

 
In short, today we saw how our Support Techs help our customers to fix backup failure with onapp.