If you add a URL in the first form, the SiteCozy site scanner will scan your webpage, discover links and review every webpage that are discovered on a domain. You must ensure that this URL is the real URL of the domain. For example, you may enter http://example.com and the right URL is http://www.example.com. However, if a redirection is in place, the scanner should follow the link.
On every webpage, the scanner will discover links once again and review every webpage until no pages are left. The SiteCozy site scanner can review up to 100000 links including webpages, images and iframe.
You can add URLs to exclude a list of URLs. This list of URLs should be exactly the ones that we can find on the website. They should be full URL with HTTP(s) at the beginning. Don’t be misled by the size of the exclusion field, you can paste many URLs here, one per line as seen on the screenshot.
Once you are ready, click on the “submit” button which will start the scanning process. 1 progress bar should be displayed after pressing “submit”. When the progress bar is visible it means the crawler is working. You can press “clear and stop” if you want to cancel a running scanning process or if you want to clear the url field.
Here, you can add a list of URLs to be scanned. You must add the URL on every line. After pressing submit, the site scanner will only review the URLs that were provided in the field. It will check and report URL errors. You can submit many URLs.
Counter & Console
The counter displays the number of links (image URLs, links, Iframes) that are collected by the crawler in real time. The console displays the list or URLs that are crawled. This can be useful to see how the URLs looks like when they are crawled.
URL error report
The URL error report lists all the URL errors found on the webpages as soon as the site scanner finds it. This report includes 4xx or 5xx URL errors, mixed content errors, malformatted URLs, timeout URLs, redirection loops and redirection when they exceed 15 redirections (internal & external). You can widen the report width by clicking on the fullscreen button at the top right corner.
For your information, you can scroll the report in horizontal and vertical mode. For example, if there are very long URLs, they may not fit the screen. In this case, in order to see the error you must scroll the screen to the right.
In the first block, you can read the number of internal HREF links or external HREF links, the number of internal and external SRC Urls, the number of internal and external IFRAME reports on the scanned website or the list of webpages.
In the second block, you can see the number of unique internal and external links, the number of internal and external image URLs and Iframe.
List of URLs
At the end of the scanning process,The site scanner will display the list of unique URLs that have been discovered on the website: the list of URLs from the A HREF tag, the list of image links from the IMG SRC tag and the list of iframe links from the IFRAME tag. If you want to export it to MS Excel click here for a tutorial. In order to display the list of URLS you are interested in, click on the tab in the black header.