Digitalocean elasticsearch cluster | Easy Method

Digitalocean elasticsearch cluster article will guide you through installing Elasticsearch, configuring your use case, and securing installation with help of our DigitalOcean Managed Services.

Digitalocean elasticsearch cluster on Ubuntu

Elasticsearch is a popular platform for real time distributed search and analysis of data. It’s due to its usability, powerful features, and scalability.

Prerequisites

An Ubuntu server with 2GB RAM and 2 CPUs set up with a non-root sudo user.

We will start working with the minimum amount of CPU and RAM required for the Elasticsearch run. The amount of CPU, RAM, and storage that your Elasticsearch server required depending on the volume of logs that you expect.

Step 1 — Installing and Configuring Elasticsearch

To begin, use CURL. The commands line tool for transferring data with URLs. To import the Elasticsearch public GPG key into APT, you need to Pipe the output to the

gpg --dearmorCopy Code

command. This converts the key format that apt can use to verify downloaded packages.

curl -fsSL https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo gpg --dearmor -o /usr/share/keyrings/elastic.gpg
Copy Code

Next, add the Elastic source list to the directory

"sources.list.d"Copy Code

. Here

aptCopy Code

will search for new sources:

echo "deb [signed-by=/usr/share/keyrings/elastic.gpg] https://artifacts.elastic.co/packages/7.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-7.x.list
Copy Code

The

[signed-by=/usr/share/keyrings/elastic.gpg]Copy Code

portion of the file instructs apt to use the key that was downloaded to verify repository and file information for Elasticsearch packages.

Now, update your package lists and APT will read the new Elastic source by using the below command:

sudo apt update
Copy Code

Then install Elasticsearch:

sudo apt install elasticsearch
Copy Code

Press 

YCopy Code

when prompted to confirm the installation. If you are prompted to restart any services, then press

ENTERCopy Code

to accept the defaults and continue. Elasticsearch is now ready to get started.

Step 2 — Configuring Elasticsearch

Further to configure Elasticsearch. We will edit its main configuration file

elasticsearch.ymlCopy Code

, where the configuration options stored. This file will locate in the directory”

/etc/elasticsearchCopy Code

“.

Edit Elasticsearch’s configuration file:

sudo nano /etc/elasticsearch/elasticsearch.ymlCopy Code

The file

elasticsearch.ymlCopy Code

provides configuration options for your cluster, node, memory, network, and gateway. Most of these options are preconfigured. However, you can make changes according to your needs.

Elasticsearch listens for traffic from port

9200Copy Code

. To restrict access and to increase security, find the line that specifies

network.hostCopy Code

, uncomment it and replace its value with

localhostCopy Code

so it reads like as shown below:

/etc/elasticsearch/elasticsearch.ymlCopy Code

. . .
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
network.host: localhost
. . .
Copy Code

We have specified

localhostCopy Code

so Elasticsearch listens to all interfaces and bound IPs. If you want it to listen only on a specific interface then specify its IP in place of

localhostCopy Code

. Later Save and close

elasticsearch.ymlCopy Code

.

These are the settings you can start to use Elasticsearch. Now you can further start Elasticsearch .

Start the Elasticsearch service with

systemctlCopy Code

.

sudo systemctl start elasticsearchCopy Code

Next, run the below command to enable Elasticsearch for server boots:

sudo systemctl enable elasticsearch
Copy Code

Step 3 — Securing Elasticsearch

If you want to allow remote access to the HTTP API. You should limit the network exposure with Ubuntu’s default firewall, UFW. Make a note that this firewall should enable.

We will now configure the firewall to allow access to the default Elasticsearch HTTP API port  9200 for the remote host. Generally, the server used in a single-server setup, such as

198.51.100.1Copy Code

. To allow access, type the below command:

sudo ufw allow from 198.51.100.1 to any port 9200
Copy Code

Once completed, you can enable UFW with the command:

sudo ufw enable
Copy Code

Finally, check the status of UFW with the command:

sudo ufw status
Copy Code

If the rules are specified properly, you will receive output as shown below:

OutputStatus: active

To                         Action      From
--                         ------      ----
9200                       ALLOW      198.51.100.0
22                         ALLOW       Anywhere
22 (v6)                    ALLOW       Anywhere (v6)
Copy Code

The UFW will now enable and set up to protect Elasticsearch port 9200.

Step 4 — Testing Elasticsearch

By now, Elasticsearch will run on port 9200. You can test with CURL and a GET request.

curl -X GET 'http://localhost:9200'
Copy Code

You should receive the following response:

Output{
  "name" : "elastic-22",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "DEKKt_95QL6HLaqS9OkPdQ",
  "version" : {
    "number" : "7.17.1",
    "build_flavor" : "default",
    "build_type" : "deb",
    "build_hash" : "e5acb99f822233d62d6444ce45a4543dc1c8059a",
    "build_date" : "2022-02-23T22:20:54.153567231Z",
    "build_snapshot" : false,
    "lucene_version" : "8.11.1",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}
Copy Code

If you have received a similar response to the above, then Elasticsearch is working properly.

To perform more checks of Elasticsearch execute the below command:

curl -X GET 'http://localhost:9200/_nodes?pretty'
Copy Code

You can verify all the current settings for the node, cluster, application paths, modules, and more from the above output command.

Step 5 — Using Elasticsearch

For using Elasticsearch, let’s first add some data. Elasticsearch uses a RESTful API, which responds to the usual commands: create, read, update, and delete. To work on this, we will use the CURL command again.

You can add your first entry like:

curl -XPOST -H "Content-Type: application/json" 'http://localhost:9200/tutorial/helloworld/1' -d '{ "message": "Hello World!" }'
Copy Code

Next will receive the response like this:

Output{"_index":"tutorial","_type":"helloworld","_id":"1","_version":2,"result":"updated","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":1,"_primary_term":1}
Copy Code

Can check the first entry with an HTTP GET request.

curl -X GET -H "Content-Type: application/json" 'http://localhost:9200/tutorial/helloworld/1'
Copy Code

This should given the output as:

Output{"_index":"tutorial","_type":"helloworld","_id":"1","_version":1,"found":true,"_source":{ "message": "Hello, World!" }}
Copy Code

To modify any existing entry. You can manage with HTTP PUT requests.

curl -X PUT -H "Content-Type: application/json"  'localhost:9200/tutorial/helloworld/1?pretty' -d '
{
  "message": "Hello, People!"
}'
Copy Code

Elasticsearch will modify results like this:

Output{
  "_index" : "tutorial",
  "_type" : "helloworld",
  "_id" : "1",
  "_version" : 2,
  "result" : "updated",
  "_shards" : {
    "total" : 2,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 1,
  "_primary_term" : 1
}
Copy Code

From the above example, we have modified 

messageCopy Code

of the first entry to “Hello, People!”. With this update, the version will automatically increase to the value

2Copy Code

.

You can notice that the extra argument

prettyCopy Code

in the above request. This enables a human-readable format. So that we can write each data field on a new row. Can also “prettify” your results while retrieving data to get a proper readable output by command:

curl -X GET -H "Content-Type: application/json" 'http://localhost:9200/tutorial/helloworld/1?pretty'
Copy Code

Now the response will format as:

Output{
  "_index" : "tutorial",
  "_type" : "helloworld",
  "_id" : "1",
  "_version" : 2,
  "_seq_no" : 1,
  "_primary_term" : 1,
  "found" : true,
  "_source" : {
    "message" : "Hello, People!"
  }
}
}Copy Code

Now we have successfully added and data in Elasticsearch.

[Looking for a solution to another query? We are just a click away.]

Conclusion

To sum up, from this article, you have installed, configured, and begun to use Elasticsearch.