Cloudflare whitelist user agent enables the Cloudflare crawlers to archive the site, When we integrate our system with the Cloudflare firewall.

In the course of providing our Server management service, Bobcares will respond to all inquiries, no matter how minor.

Let’s look more closely at how our Support team handles whitelisting Cloudflare UA.

Cloudflare whitelist user agent

Specific browser or web application User-Agent request headers are blocked by User Agent Blocking (UA) rules. Instead of specific subdomains, the entire domain is subject to UA rules. Permitting an IP address via Zone Lockdown avoids UA rules because they are applied after Zone Lockdown rules. Depending on the type of plan, it permits a certain number of UA rules.

To enable our crawlers to archive the site, we might need to whitelist the UA depending on the Cloudflare settings. It is best practise to whitelist Better Uptime IPs and UA if our system is using a Cloudflare firewall. By doing this, Cloudflare won’t deny Better Uptime’s monitoring requests and and won’t generate false incident alerts.

Steps to WhiteList user agent in Cloudflare

1. Firstly, log in to the Cloudflare account and select the Security tab from the left menu.
2. Then, select WAF.  As a result it shows the Web Application Firewall security settings.
3. Go to the Firewall rules tab.
4. Then, click on Create firewall Rule.
5. Then, we gave a name to the Firewall rule.
6. Set Field to User Agent, Equals as the Operator, and Value to

   Better Uptime Bot Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36
7. Then, turn on the Allow.
8. Finally, click Deploy in the bottom-right corner after completing the form.

[Looking for a solution to another query? We are just a click away.]

Conclusion

In conclusion, our websites won’t appear during crawling if the user agent is not in whitelist in Cloudflare.