Is your server showing Starting Elasticsearch server failed error message? Here’s the fix.

When the Elasticsearch server does not run, it generates an error in the search application.

At Bobcares, we help customers with Elasticsearch related queries as part of our Server Management Services.

Today, we’ll see how we fixed the error and made Elasticsearch working for one of our customers.

What is Elasticsearch?

Let’s first get an idea of Elasticsearch.

In simple words, Elasticsearch is a search engine. It allows us to store, search and analyze big data volumes easily. With the data in the JSON format, search results will be really quick. That’s the main advantage of Elasticsearch.

Most server owners use Elasticsearch in managing log files.

What causes starting Elasticsearch server failed error

We just saw the basics of Elasticsearch.

Usually, for the search results to show up fine, the underlying Elasticsearch server should work correctly. However, for many reasons, the Elasticsearch server can go down.

Now it’s time to check the causes for the Elasticsearch server failure.

1. Permission problems

From our experience in managing Elasticsearch servers, we often see the permission problems causing errors. When the ‘elasticsearch‘ user lacks privileges on the folders, the service can go down.

Therefore, it is necessary to give permissions on the entire elasticsearch directory. Additionally, it should have permission to access the /var/lib/elasticsearch folder where elasticsearch stores data.

Also, there should be enough rights in the file /var/run/elasticsearch that contains the process id of elasticsearch.

2. Dependency failure cause starting Elasticsearch server failed error

Similarly, starting Elasticsearch server failed error can happen due to dependency failure too. This can be related to the JAVA version or the version of the plugins. Very often this can be due to incompatible versions. Usually, an upgrade fixes such errors.

3. Configuration errors

Usually, elasticsearch holds the configuration in the elasticsearch.yml file. When there are errors in this file, it can also result in Elasticsearch server failed error.

How we fixed starting Elasticsearch server failed error

Till now, we saw different reasons for service failure.

Moving on, let’s check how our Support Engineers resolved the Elasticsearch server failed error for the customer.

Hi, our ElasticSearch server is down. Can you please investigate why it happened?

That was the customer request in our Helpdesk.

As the first step, we checked the status of the Elasticsearch on the server. We found that it was in failed status.

[root@xxx elasticsearch]# systemctl status elasticsearch.service
● elasticsearch.service - Elasticsearch
   Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; enabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since Thu 2020-03-05 17:46:34 PST; 4min 41s ago
     Docs: http://www.elastic.co
  Process: 56711 ExecStart=/usr/share/elasticsearch/bin/elasticsearch -p ${PID_DIR}/elasticsearch.pid --quiet (code=exited, status=1/FAILURE)
 Main PID: 56711 (code=exited, status=1/FAILURE)

The server tried restarting the failed service but the Elasticsearch service was crashing automatically.

On checking the error logs, we could see the below error.

java.lang.IllegalArgumentException: Plugin [mapper-size] was built for Elasticsearch version 6.8.6 but version 6.8.7 is running

Here, the problem was with the ‘mapper-size‘ plugin. The installed plugin was for Elasticsearch version 6.8.6. But, the currently running version in the server was 6.8.7.

These Mapper plugins allow new field datatypes to be added to Elasticsearch.

So we upgraded the plugin ‘mapper-size‘ for Elasticsearch version 6.8.7.

That resolved the error and the service started working fine.

Finally, we created monitoring for Elasticsearch service too. This helped to track the service continuously.

[Is your Elasticsearch server going down frequently? We’ll fix it for you.]

Conclusion

In short, starting Elasticsearch server failed error happens due to dependency errors, incorrect permissions, etc. Today, we saw how our Support Engineers fix Elasticsearch service and make it work.