Search results

Mosh had to give up development on the Spiders list due to medical reasons in May of 2010, I decided in January 2014 to update the list, and try and stay on top of it now. He originally took it over from Dream in 2008 which consisted of 503 spiders. Mosh's list once he was done with it...

Please add the TrueBot spider. 192.99.149.88 Mozilla/5.0 (compatible; TrueBot/1.0)

Please add the DTS Agent spider to the list. 41.189.39.147 - Beijing Express Email Address Extractor via DHCP Data Transport Services (DTS) Mozilla/4.0 (compatible; MSIE 5.0; Windows NT; DigExt; DTS Agent

Please add the NetShelter ContentScan spider. ec2-184-73-74-14.compute-1.amazonaws.com - http://whois.domaintools.com/184.73.74.14 Mozilla/5.0 (Windows; U; Windows NT 6.1; en-GB; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 (NetShelter ContentScan)

Can you add the madaali spider please. 188.138.90.244 madaali.de (Version: 1.06, powered by www.madaali.de +http://www.madaali.de/pfadzurbotseite/bot.html)

Can you add the tags2dir spider to the list.

Please ad the CA Crawler spider to the list.

Please add the Techmixx spider during the next update.

Please add the Artmixx spider to the list. :)

Please add the FunWebProducts bot to the list. 27.150.243.37 - Whois ( There is a bunch of different IP's ) Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; FunWebProducts; .NET CLR 1.1.4322; PeoplePal 6.2) Everything I have read about this bot is it is not a good bot.

Please add the BrokenLinkCheck spider. 74.91.20.214 - Whois Mozilla/4.0 (compatible; MSIE 7.0; Windows; Windows NT 5.1) BrokenLinkCheck.com/1.1

Please can you add the SharewarePlaza spider.

Can you add the Adaxas spider to the list.

Can you add the A1 Sitemap Generator to the list please. :)

Please add the A1 Keyword Research spider.

Pleas can you add the 2Bone_LinkChecker Spider to the list. :)

Please add the ltbot 130.83.167.219 - Whois Mozilla/5.0 (compatible; ltbot/3.2.0.10 +http://www.kdsl.tu-darmstadt.de/de/kdsl/research-program/crawling-and-semantic-structuring/)

I would like to see the Sygol spider added to the list. :)

Please add the UnChaos spider. :)

It would be cool if you added this spider. 54.174.17.24 - Whois ltx71 - ( http://ltx71.com/)