• Ever heard of a situation where a security expert/data-center asks to re-install the OS of a server for security reasons? That sounds like(and in fact is) a drastic situation, and it does happen. The risk of living with a server that was once compromised is very high. The risk it is due to the possibility of a rootkit implanted in the server. Unfortunately the only proven recovery method would be a clean install of the OS.

What is Rootkit

Root-kit typically is a malware that is stealthy in nature. They are usually hard to detect, and harder to remove. A rootkit can potentially hide almost any software; including files, botnets, key-loggers and back-doors. Root-kits are implanted in a system by an attacker who gains access to the system using some inherent vulnerability in the system.

Once the system is rooted, the options for remedying the scenario is limited. Due to the stealthy nature of rootkits, it is almost impossible to ascertain the level to which the system is compromised. Many a times the system binaries and libraries are tampered with, hence one could no longer rely on these infested binaries. The only safe course, is a complete wipe of system and a fresh install of OS.

How RootKits are detected

Rootkit detectors use few standard methods for detecting the presence of rootkits :

-> Based on Signature – The idea is to look for specific files and binaries that are part of known root-kits. This is not fool-proof for the very fact that signatures of known malware could change every now and then. Also, newer rootkits would never be identified, as they are still an unknown entity.

-> Based on behavior – Behavior-based detectors identify rootkits by finding hidden elements, which is mostly finding an anomalous behavior of a server. Say the network interface entering promiscuous mode, or a similar activity.

-> Comparison/Integrity based – Here comparison of present output with the desired output is used to detect the presence of a rootkit. Say the command ps gives one output, while the status of processes in /proc has a different tale to tell. Also checking the sanity of system tools with methods like MD5 hash compare, and changed attributes of system files.

How to mitigate the rootkit

Taking a few steps can ensure the health and integrity of the servers. The process involves :

-> Proper security measures to protect against unauthorized access to server/data.
-> Proper means to detect rootkits; in case security is compromised at some point of time.
-> Being ready for a recovery, if need arises

There are many tools available, that help in detecting a possible rootkit. Do keep an eye out for posts on a few of them in the coming weeks.

About the Author :

Sankar works as a Senior Software Engineer in Bobcares. He joined Bobcares back in April 2006. He loves grooming/mentoring people. During his free time, he listens to music, and enjoys singing..