SiteCozy

  • My account

Server friendly approach of the SiteCozy broken link checker

Numerous crawlers aggressively visit web server by sending many simultaneous requests to a web server in a small time frame. We think that it is not the right thing to do. Right now, the SiteCozy broken link scanner visits one resource at a time.

There are several reasons why we think that offering a very fast broken link checker is not the right thing to do for several reasons:

  1. If there is already a significant amount of web traffic to a web server, it is unwanted to stress the web server. I especially think about web servers with little resources.
  2. If firewall rules are like rate limiter are in place, they may refuse some requests by “thinking” that the web server is under a Denial-of-service attack. Some legitimate requests from regular visitors can be refused by the web server.
  3. We don’t want the Sitecozy broken link scanner application to be used as a denial-of-service attacker against a web server. Imagine that the SiteCozy broken link scanner would send as many simultaneous requests as it could. An individual could install the app on several machines and perform a DDOS attack. We already saw that a crawler can take down a web server
  4. If we stress servers from main shared web hosting providers, they will block our user agent.
Download our Broken link checker freeware here Buy a license key for the Sitecozy broken link checker

Customer Login

Lost password?

Categories

  • Webmaster advice
  • SEO advice
  • Web hosting
  • SiteCozy link checker KB
  • WordPress theme & plugin reviews
  • All articles
Disclosure: We are a professional review site that receives compensation from the companies whose products we review. We test each product thoroughly and give high marks to only the very best. We are independently owned and the opinions expressed here are our own.

Copyright Sitecozy 2018