How to Configure Cloudflare to Allow Nytro Crawling Bot?

How to Configure Cloudflare to Allow Nytro Crawling Bot?

The Nytro Crawling Bot helps with SEO auditing and site analysis by crawling your website. To ensure it can properly access your site without being blocked by Cloudflare security rules, follow the steps below to configure your Cloudflare settings.


Step 1: Open the Security Rules Page

  1. Log in to your Cloudflare dashboard.

  2. In the left-hand menu, navigate to Security → WAF (Web Application Firewall).

  3. Click on Security Rules.



Step 2: Create a New Security Rule

  1. Click the Create rule button.

  2. In the rule builder, enter the following details:

    • Field: User Agent

    • Operator: equals

    • Value:(Nytro User Agent: Mozilla/5.0 (compatible; NSCrawlerBot/1.0; +https://nytroseo.com/)


Notes
This user agent string ensures that Cloudflare recognizes and allows the official Nytro Crawling Bot.


Step 3: Deploy the Rule

  1. Review your configuration to confirm that it matches the user agent above.

  2. Click Deploy to activate the rule.