CukBot is CompaniesInTheUK.co.uk's web crawling bot (sometimes also called a "spider"). Crawling is the process by which CompaniesInTheUK.co.uk discovers the websites of UK Companies.  We are not harvesting email addresses, or comment spam, we do not have any evil intent crawling your website.

CukBot uses an algorithmic process: computer programs determine which sites to crawl, how often, and how many pages to fetch from each site.

For webmasters: CukBot and your site

How CukBot accesses your site

CukBot's crawl process visits at most 10 pages on your website, and will not visit more often than once per day.  We will not access these ten pages more than once per second.

Blocking CukBot from content on your site

If you want to prevent CukBot from crawling content on your site, use a robots.txt file to block access to files and directories on your server.  CukBot is compatible with GoogleBot and follows the same rules for the robots.txt as documented on Google, including wildcard matching.

The useragent string to use for CukBot in your robots.txt file is (surprisingly?) : CukBot 

The IP addresses used by the CukBot are 176.9.136.4, 176.9.146.202, 176.9.99.42, 5.9.97.100, 5.9.142.210 and 176.9.118.9 if we are not using these IP address then you were visited by a bot is pretending to be us, please block it!

support@companiesintheuk.co.uk