source: trunk/EventBenchConsole/misc/robotfilter.txt

Last change on this file was 233, checked in by sherbold, 13 years ago
  • updated robotfilter
  • Property svn:mime-type set to text/plain
File size: 885 bytes
RevLine 
[72]1findlinks
2discobot
3Googlebot
4Slurp
5YandexBot
[78]6Spider
7ScholarUniverse
8Baiduspider
9Exabot
10Robot
11MetaGer-Bot
12YandexImages
13Gigabot
14SiteBot
15bingbot
16Ezooms
17Jeeves/Teoma
[110]18msnbot
19DotBot
20changedetection.com/bot.html
21FAST Enterprise Crawler 6
22psbot
23http://ws.daum.net/aboutWebSearch.html
24NerdByNature.Bot
25Sogou web spider
26ssearch_bot
27Purebot
28http://www.icjobs.de
29scoutjet
30Netcraft Web Server Survey
31TurnitinBot
32ia_archiver
[178]33MJ12bot
34Domnutch-Bot
35Eurobot
36GarlikCrawler
37CMS Crawler
38MSIECrawler
39NaverBot
4080legs
41AhrefsBot
42SISTRIX Crawler
43NetcraftSurveyAgent
44Search17Bot
45Semager
46YandexFavicons
47heritrix
48suggybot
49Netluchs
50Ocelli
51PHPCrawl
52SolomonoBot
53Sosospider
54Xerka WebBot
55YahooCacheSystem
56Xenu Link Sleuth
57cmsworldmap
58suchen.de
59amaredo.com/de/suche.html
60ibot
61w3af.sourceforge.net
[180]62w3af.sf.net
[233]63yacybot
64larbin2
65t-h-u-n-d-e-r-s-t-o-n-e
66sqlmap
Note: See TracBrowser for help on using the repository browser.