If you've recently added a website to our system and are experiencing issues with crawling, identification, or analysis, this guide will help you pinpoint potential causes and understand the limitations and supported features of Nytro Systems.

System Compatibility

Nytro Systems is engineered to seamlessly analyze a broad spectrum of websites, ensuring comprehensive insights. However, understanding the types of websites we support and the common issues that can disrupt crawling is key to resolving any challenges.

Supported Website Types

Nytro Systems currently supports the following types of websites:

• Standard HTML-based Websites: Websites where content is served directly from the server in HTML format.

• Server Side Rendering (SSR): Websites that dynamically generate content on the server side before sending it to the client in HTML format.

• Standard CMS Platforms: Examples include WordPress, Wix, Squarespace and other popular content management systems.

Website supported upon request

There are certain types of websites and technologies that Nytro Systems does not currently support by default. These include:

1. Client-Side Rendering (CSR) Websites: Websites that rely on JavaScript to render content in the browser. Our system is currently unable to crawl by default and analyze these websites. Like JavaScript sites that use the app shell model where the initial HTML does not contain the actual content and NytroSEO needs to execute JavaScript before being able to see the actual page content that JavaScript generates. Source

Common Issues Preventing Website Crawling

If your website is not being crawled or analyzed, it may be due to one of the following issues:

• Robots.txt Blocking: If your website's robots.txt file disallows crawling, our system will be unable to access the site.

• Crawling Blocked by Firewalls or Security Services: Websites protected by firewalls or services like Cloudflare may block our crawling attempts, preventing analysis.

• Empty or Linkless Home Page: A home page without content or internal links can hinder our system from initiating the crawling process.

• Home Page Errors:

• 404 Not Found: The home page is missing.

• Server Down or Not Responding: The server hosting the website is unresponsive.

• 403 Access Denied: The system is restricted from accessing the home page.

• Other HTTP Errors: Any other HTTP error codes that prevent the home page from loading.

• Expired Domain: If the domain of the website has expired, crawling is not possible.

• Single, Non-Canonical Page: Websites with only one page that is not marked as canonical may cause issues with crawling and identification.

• Unexpected Failures in Our System:

• Server Downtime: Temporary unavailability of our system may prevent crawling.

• Logical Errors in the Crawling Process: Errors within the system's crawling logic might cause failures.

• Website Offline: If your website is not connected to the internet or the server is down, crawling cannot be initiated.

Steps to Resolve Crawling Issues

If you encounter any of the issues listed above, we recommend the following steps:

1. Check Your Website Configuration: Ensure that your website’s robots.txt file, security settings, and server configuration allow for crawling by our system.

2. Review HTTP Status Codes: Verify that your home page is accessible and not returning error codes that would block crawling.

3. Ensure Website Availability: Make sure that your website is online and the domain is active.