Here at Bobcares, our Dedicated Support Engineers maintain servers of web hosts, web designers, and other online businesses.

In cPanel and Plesk servers, a common error we see from LFD (Login Failure Daemon) is Suspicious process running under user XXXX.

A majority of these errors are harmless, as the default settings of LFD tend to classify cPanel and spamd services as malicious.

BUT, every once in a while LFD does detect a real malicious process.

This post is about how we traced a malicious process in a Plesk server to a malware and secured the system.

LFD alert : Suspicious process running under user XXXX

We closely monitor alerts from all servers we maintain for our customers.

One day, LFD from a server sends this mail:

Subject: lfd: Suspicious process running under user XXXX

Time: Tue Jun 26 19:32:52 2018 +0800
PID: 12012 (Parent PID:31052)
Account: XXXX
Uptime: 64 seconds

Executable:

/var/www/vhosts/XXXX/httpdocs/.xmr/apache2

That executable sure looked suspicious.

So, immediately we knew this was a valid alert.

Validating our suspicions – Is it really malicious?

The first thing we do when we receive an alert is to validate if it is correct and estimate the severity.

Our server administrators do this by logging into the server within minutes and inspecting the process behavior.

Since this process was running for a long time and was using high CPU, we used the top command.

Sure enough, the said process was using 416% of the CPU, which almost never happens for a legitimate service.

The command line says “/var/www/apache2” but that can be easily faked by a malicious process.

At this point, we knew we were dealing with a malicious process. Quickly we checked what it was doing.

The process seemed to be receiving instructions from an external server as highlighted here.

This is the usual behavior of bots, where malware contacts an external control server to receive commands from a bot master.

Outgoing connections almost never happens on a shared web hosting server.

To know what kind of beast we’re dealing with here, we checked the folder in which the malware was uploaded.

The malware file names are faked to mimic legitimate server processes.

A quick look at the file called apache2 showed that it was a Crypto Currency Miner.

Designed to suck the CPU out of servers, and make the bot master some money.

Rescue action – Killing the bot, Blocking the IP and Neutralizing the malware

Our immediate concern is to rescue the server from high load and possible IP blacklisting.

So, once we identified the malware and its source, we:

• Killed the bot process (masquerading as apache2).
• Blocked the IPs to which the bots communicated (most possibly bot controller IPs).
• Made the malware files non-executable and non-readable (which prevents the process from starting again).

We waited a couple of minutes to see if the bot process came back again, and if the server load went down.

And it sure did.

Which confirmed that we killed the malware source.

What’s the extend of the damage? Finding all locations of malware

Now, we know that when an attacker gains access to a server, they don’t leave malware only in one location.

They’ll spread it as wide and deep as possible.

So, we then looked in these areas:

• Newly inserted code within legitimate web files.
• Malware hidden as images, or other such innocuous files.
• Crontab, which is used to re-trigger bot process if one is killed.

We found this particular malware traces in all three above locations which suggested a serious vulnerability either in the web application or in the user account details.

How did the malware get here?

Our next priority is to find out which vulnerability in the website or server allowed the attacker to get in.

For this, we looked at the file creation time of each malware file and correlated that with server access logs in HTTP, FTP and Plesk file manager.

Sure enough, we found that the time correlated with a weird looking access to the Drupal website.

We checked the Drupal source, and found that it was an old version which had vulnerabilities.

So, now we knew how the attacker got it.

We cross-verified the FTP and File Manager logs for good measure to rule out multiple paths of entry.

Killing the malware & Cleaning the server

Now that we had all the information we needed, we cleaned all malware off the account, and restored any needed file from backup archives.

Patching the CMS and securing the account

We then patched the Drupal website to prevent any future exploits.

To be doubly careful, we reset all passwords to prevent attacks using stolen credentials (people tend to re-use passwords).

Once the system was secured, we let the website owner know about the security issue and actions taken.

Summing up

“Suspicious process running under user” is a common error message sent by LFD running in cPanel and Plesk servers. Today we’ve seen how Dedicated Support Engineers here at Bobcares dug to the bottom of one such error, and cleaned malware from a server.