AWS Elasticsearch 429 Too Many Requests error can be resolved easily with these troubleshooting tips by our experts. 

At Bobcares, we offer solutions for every query, big and small, as a part of our AWS Support Services.

Let’s take a look at how our AWS Support Team is ready to help customers resolve AWS Elasticsearch 429 Too Many Requests error.

About the AWS Elasticsearch 429 Too Many Requests error

The 429 error message is a clear indication of a bulk queue error. As per our AWS experts, the bulk queue on each node has a capacity between 50 and 200 requests. This depends on the Elasticsearch version in use. Interestingly, many of our customers have come across the AWS Elasticsearch 429 error due to using an older version of Elasticsearch.

The Elasticsearch 429 error results in the following error message:

error":"elastic: Error 429 (Too Many Requests): rejected execution of org.elasticsearch.transport.TransportService$7@b25fff4 on 
EsThreadPoolExecutor[bulk, queue capacity = 200, org.elasticsearch.common.util.concurrent.EsThreadPoolExecutor@768d4a66[Running, 
pool size = 2, active threads = 2, queued tasks = 200, completed tasks = 820898]] [type=es_rejected_execution_exception]"

Reason={"type":"es_rejected_execution_exception","reason":"rejected execution of org.elasticsearch.transport.TcpTransport$RequestHandler@3ad6b683 on EsThreadPoolExecutor[search, queue capacity = 1000, org.elasticsearch.common.util.concurrent.EsThreadPoolExecutor@bef81a5[Running, pool size = 25, active threads = 23, queued tasks = 1000, completed tasks = 440066695]]"

Reasons behind the AWS Elasticsearch 429 Too Many Requests error

According to our AWS Support Team, the 429 error message may also appear as es_rejected_execution_exception. Here are some variables that may be the reason behind the error:

• Instance metrics with high values
• Data node instance types as well as search or write limits
• Active and Queue threads

Furthermore, the 429 error may also appear from a single node or multiple nodes of the cluster.

How to resolve the AWS Elasticsearch 429 error

First, let’s take a look at the instance metrics values to look for high values. Some of the variables that we need to keep an eye out for include:

• SearchRate
• lndexingRate
• PrimaryWriteRejected
• CoordinatingWriteRejected
• ReplicaWriteRejected
• ThreadpoolWriteRejected
• ThreadpoolWriteQueue
• ThreadpoolSearchQueue
• ThreadpoolSearchRejected

The ThreadpoolSearchQueue and ThreadpoolWriteQueue metrics in Amazon CloudWatch can help us keep track of our queue threads, according to our AWS specialists.

Moreover, we can look for search rejections on the Active and Queen thread via the following command:

GET /_cat/thread_pool/search?v&h=id,name,active,queue,rejected,completed

Additionally, we can look for write rejections on the Active and Queen thread via the following command:

GET /_cat/thread_pool/write?v&h=id,name,active,queue,rejected,completed

Furthermore, there is a limit on the number of searches or writes per that we can perform. This limit depends on the thread pool definition as well as the Elasticsearch version.

If the above troubleshooting tips did not help, drop us a line and our AWS Support Team will resolve AWS Elasticsearch 429 Too Many Requests error in no time.

[Need assistance with another query? We are available 24/7.]

Conclusion

In a nutshell, our skilled AWS Support Engineers at Bobcares demonstrated how to resolve AWS Elasticsearch 429 Too Many Requests error.