License check is an important function in commercial applications. Every major software vendor implements license verification systems to make sure all online systems are valid.

A popular way to implement license check is to have the system talk to a licensing server frequently. The licensing server can then verify the system’s license to see if its still valid. Each successful verification extends the validity of the license for a certain period of time (say 1 week), before which the system should check in again to keep everything working.

So, what happens if the system is unable to connect to a licensing server? Without the confirmation of a licensing server, the validity of a license expires in a short period. At that point, systems could stop working, and users would no longer be able to use the servers.

It is important for service providers to keep tabs on system status to make sure everything is working properly. As part of our dedicated support services, we help data centers and cloud providers monitor their infrastructure 24/7 and quickly resolve service errors.

In a few data centers we support, OnApp cloud management system is used to manage their cloud infrastructure. OnApp uses a licensing server to verify licenses in OnApp systems. If an OnApp system is unable to connect to the licensing server, a license error is shown. One day, we noticed this error in a data center:

Error: Unable to connect to licensing server

If not fixed, the issue could lead to an expiry of the license, and break down of all OnApp management functions.

A break in communication between two servers could happen due to many reasons. In this post we’ll go through how we resolved this license error.

Resolving “Error: Unable to connect to licensing server”

There were two possibilities for this error to happen:

1. The licensing service had crashed.
2. Port 5555 in the OnApp control panel was blocked.

We tested the above two possibilities to troubleshoot the issue.

Checking if license client was working

This error usually happens when a service called “Passenger AppPreloader” crashes in the OnApp management server. To check this, we looked for this service in the list of active programs.

onapp  5235  0.1  1.3  2089196  436200  ?  Sl  Jul02   6:18  Passenger AppPreloader: /onapp/interface

The service was indeed working. So, it was not a case of “Passenger AppPreloader” crashing.

Quick Tip : If the service is not listed, restart it by going to Settings > Licensing > Restart License Client in the OnApp control panel. If that didn’t work, you can restart it by executing these commands in the OnApp Management Server:
# service onapp stop
# touch /onapp/interface/tmp/restart.txt
# service onapp start

Checking if port 5555 was open

The next possibility was that the license client service “Passenger AppPreloader” was unable to communicate with OnApp licensing server. The service connects to port 5555 of the OnApp licensing server to check the license status. In this case it seemed that outgoing connection to port 5555 was blocked.

OnApp Management Server unable to connect to licensing server port 5555

To verify this, we tried to connect to port 5555 of 89.238.146.217 (which is an IP of OnApp licensing server). The connection failed, which pointed to a possible issue with the firewall rules in the OnApp management server.

For situations exactly such as this, we keep backups of critical configuration files in each server. We restored the firewall configuration file from the daily backup and restarted the firewall service. We were then able to connect to port 5555 of 89.238.146.217, which meant the block was removed. Now, we restarted the License Client service (aka Passenger AppPreloader), and within 30 mins, the error shown in the control panel was gone.

OnApp systems were back on ALL GREEN status. 🙂

Network errors can happen due to a variety of reasons. Here we’ve covered a couple of common causes for license errors in OnApp systems. Networks can fail if any of its sub-components fail. To quickly troubleshoot network downtimes, it is important to be aware of how network components interact with each other. Bobcares helps data centers and cloud service providers minimize service downtimes through proactive systems audits, 24/7 monitoring, and 24/7 emergency administration.