Crawler
Crawler Info
The crawler reads your sitemap.xml file and stores the content of each discovered URL in the database for further analysis.
How it works
- Enter the sitemap.xml URL of the website you want to analyze
- The crawler reads all URLs from the sitemap
- Each URL is fetched and its content is stored
- The data is then available for SEO analyses