Who We Are
If you are viewing this page, your website was probably visited by the Cazoodle crawler. Its purpose is to collect information of next-generation Web search and integration solutions.
What We Do
The crawler follows the robots exclusion protocol, which allows site administrators to prevent robots from visiting all or certain parts of their website. You can learn more about this protocol here. The robots.txt file must be located at www.yoursite.com/robots.txt. There are many ways to block only certain robots or only certain areas of your site, but here is a brief example of a robots.txt entry which would block our crawler from visiting every page on your site:
User-agent: CazoodleBot
Disallow: /private-data
Allow: /
In addition, the crawler delays at least 10 seconds between requests to the same website so that it does not overload the server.
Contact Us
We would like the crawler to be as friendly as possible, so if you feel the crawler is not behaving properly or have any other concerns, we would like to hear about it. Please email us at cazoodlebot@cazoodle.com with a description of the problem and any other relevant information such as access times or server logs. Or you can go to our contact information page.