Cloudflare provides multiple ways to block bots from accessing your site while allowing legitimate users and search engines through. Here are the most effective methods:

1. Enable Cloudflare Bot Management (Advanced)

• If you’re on a Pro, Business, or Enterprise plan, Cloudflare provides Bot Management, which automatically blocks bad bots and allows good ones.


• You can enable Bot Fight Mode under Security > Bots to mitigate harmful automated traffic.


• Use supervised challenge or managed challenge to let Cloudflare decide when to challenge visitors.

2. Use Firewall Rules to Block Bad Bots

You can create custom rules to filter out bots based on behavior, user-agent, or IP.

1. Go to Security > WAF (Web Application Firewall)


2. Click Create Firewall Rule


3. Set conditions such as:
   
   
   • Known bots: Block or challenge requests that don’t match known good bots (Googlebot, Bingbot, etc.).
   
   
   • User-Agent strings: Block common spam bots using User-Agent matches.
   
   
   • Countries: Block entire regions known for bot traffic.
   
   
   • Request frequency: Block IPs making too many requests in a short time.
   
   
   
   

Example firewall rule:

If User-Agent contains "MJ12bot" or "AhrefsBot" then Block

3. Block Traffic from Data Centers

Many bots come from cloud hosting services (AWS, DigitalOcean, Linode, etc.).

• Create a firewall rule to block ASN (Autonomous System Numbers) from these providers.

Example rule:

Field: ASN | Operator: equals | Value: 16509 (Amazon AWS) → Action: Block

Use ipinfo.io/AS16509 to find more ASNs.

4. Rate Limiting to Stop Scrapers

You can configure rate limits to block excessive requests from single IPs.

1. Go to Security > WAF > Rate Limiting Rules


2. Set a rule like:
   
   
   • If a single IP makes more than 20 requests per second, block it for 10 minutes.
   
   
   • If an IP makes too many POST requests, block it.
   
   
   
   

Example:

If Path contains "/wp-login.php" and requests > 10 per minute → Block

5. JavaScript Challenges for Suspicious Traffic

To deter automated bots that don’t process JavaScript:

1. Go to Security > WAF > Firewall Rules.


2. Set a rule like:
   
   
   • If Threat Score > 20, apply JavaScript Challenge.
   
   
   • If Bot Score < 30, apply Captcha Challenge.
   
   
   
   

6. Use Super Bot Fight Mode (Free & Paid Plans)

• Enable Bot Fight Mode under Security > Bots.


• It automatically blocks known bots and challenges others.


• On Pro plans and above, you can fine-tune the level of bot detection.

7. Use Custom Page Rules

• Redirect bots to a fake page using Page Rules.


• Add a rule: If User-Agent contains "bot" -> Redirect to /fake-page

8. Monitor and Adjust Regularly

• Use Analytics > Security Events to track blocked bot attempts.


• Adjust rules based on real traffic data.

Bonus: Block Bots at the Application Level

• If you use WordPress, install plugins like Cloudflare Turnstile or Wordfence to further filter bots.


• Use robots.txt to disallow bots from indexing pages that they shouldn’t crawl.

Conclusion

By combining Bot Fight Mode, WAF rules, rate limiting, and user-agent filtering, you can block most bots without harming real users or search engines. Regularly check Cloudflare’s Security Events to refine your approach. 🚀