How to Configure Cloudflare to Allow Nytro Crawling Bot?

How to Configure Cloudflare to Allow Nytro Crawling Bot?

The Nytro Crawling Bot helps with SEO auditing and site analysis by crawling your website. To ensure it can properly access your site without being blocked by Cloudflare security rules, follow the steps below to configure your Cloudflare settings.


Step 1: Open the Security Rules Page

  1. Log in to your Cloudflare dashboard.

  2. In the left-hand menu, navigate to Security → WAF (Web Application Firewall).

  3. Click on Security Rules.



Step 2: Create a New Security Rule

  1. Click the Create rule button.

  2. In the rule builder, enter the following details:

    • Field: User Agent

    • Operator: equals

    • Value:(Nytro User Agent: Mozilla/5.0 (compatible; NSCrawlerBot/1.0; +https://nytroseo.com/)


Notes
This user agent string ensures that Cloudflare recognizes and allows the official Nytro Crawling Bot.


Step 3: Deploy the Rule

  1. Review your configuration to confirm that it matches the user agent above.

  2. Click Deploy to activate the rule.

    Submit a Ticket

      • Related Articles

      • Nytro Crawler Bot

        A bot, also known as a web robot, web spider, or web crawler, is a software application designed to perform repetitive tasks in an automated, structured, and efficient manner. Web crawling, the most common use of bots, involves browsing websites in a ...

      • Troubleshooting: Solving Common Issues with Nytro JS Snippet

        If you're experiencing issues with the Nytro JS snippet not working or not being verified, it could be due to interference from third-party speed and optimization plugins or browser-related issues. Follow the guide below to troubleshoot and resolve ...

      • Installing and Verifying the Nytro SEO JS Snippet on BigCommerce

        1: Locating the JS Snippet Go to the Nytro SEO dashboard. From the left sidebar, select “Websites” ➡️ “Manage.” Click the symbol ‘</>’ to view the JS snippet for your website. Copy the snippet code. Note: If you haven’t added your website yet, check ...

      • Understanding Canonical Web Pages and How Nytro SEO Optimizes Them

        What is a Canonical Web Page? A canonical web page is the preferred version of a page when multiple URLs contain similar or identical content. Without a designated canonical page, search engines might treat each variation as a separate page, ...

      • Installing and Verifying the Nytro SEO JS Snippet on WordPress

        The Nytro SEO JS Optimization Snippet allows for the automatic SEO optimization of your webpage code. To enable this feature, you need to copy and paste the JS snippet into the <HEAD> section of your website. Here’s a detailed guide to help you ...