No bot name found - WhichBrowser 2.0.10

• 3x detected (1x unique)
  Applebot

  Example user agent
  Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/600.2.5 (KHTML, like Gecko) Version/8.0.2 Safari/600.2.5 (Applebot/0.1; +http://www.apple.com/go/applebot)

• 2x detected (2x unique)
  Crawllybot,Crawlboy

  Example user agent
  Crawllybot/0.1/0.1 (Crawllybot/0.1; http://www.crawlly.com; crawler@crawlly.com)

• 3x detected (3x unique)
  Almaden,Fake Browser,www.almaden.ibm.com

  Example user agent
  http://www.almaden.ibm.com/cs/crawler

• 2x detected (2x unique)
  NameProtect,abot

  Example user agent
  abot/0.1 (abot; http://www.abot.com; abot@abot.com)

• 3x detected (3x unique)
  spbot,OpenLinkProfiler,SEOprofiler

  Example user agent
  Mozilla/5.0 (compatible; spbot/4.0.9; +http://OpenLinkProfiler.org/bot )

• 1x detected (1x unique)
  AdnormCrawler

  Example user agent
  AdnormCrawler www.adnorm.com/crawler

• 2x detected (1x unique)
  MetaGeneratorCrawler

  Example user agent
  MetaGeneratorCrawler/1.3.4 (www.metagenerator.info)

• 0x detected (0x unique)

  Example user agent
  Feedfetcher-feedraider; (+http://feedraider.com/help/feedfetcher.html; X subscribers; Allow like Gecko)

• 1x detected (1x unique)
  FatBot

  Example user agent
  Mozilla/5.0 (compatible; FatBot 2.0; http://www.thefind.com/crawler)

• 1x detected (1x unique)
  InfuzApp

  Example user agent
  Mozilla/5.0 (compatible; InfuzApp/1.0; +http://www.infuz.com/bot.html)

• 1x detected (1x unique)
  wume_crawler

  Example user agent
  wume_crawler/1.1 (http://wume.cse.lehigh.edu/~xiq204/crawler/)

• 2x detected (2x unique)
  Cydral Web Image Search,CydralSpider

  Example user agent
  CydralSpider/3.2 (Cydral Image Search; http://www.cydral.com)

• 3x detected (1x unique)
  Screaming Frog SEO Spider

  Example user agent
  Screaming Frog SEO Spider/2,22

• 1x detected (1x unique)
  Squzer

  Example user agent
  Mozilla/5.0 (compatible; Squzer/0.0.1; +http://www.declum.com/)

• 3x detected (3x unique)
  80bot,80legs/0.71,80Legs

  Example user agent
  Mozilla/5.0 (compatible; 80bot/0.71; http://www.80legs.com/spider.html;) Gecko/2008032620

• 1x detected (1x unique)
  Diffbot

  Example user agent
  Diffbot/0.1

• 2x detected (2x unique)
  Easou Spider,EasouSpider

  Example user agent
  Mozilla/5.0 (iPhone; U; CPU iPhone OS 3_0 like Mac OS X; en-us; EasouSpider; +http://www.easou.com/search/spider.html)

• 1x detected (1x unique)
  VM-Crawler

  Example user agent
  Mozilla/4.0, VM-Crawler/cs version info ofni noisrev sc

• 1x detected (1x unique)
  RDF crawler

  Example user agent
  OpenLink Virtuoso RDF crawler

• 1x detected (1x unique)
  SRCCN!Spider

  Example user agent
  Mozilla/5.0 (compatible; SRCCN!Spider/1.1; +http://site.srccn.com/spider.html)

• 1x detected (1x unique)
  Beehive Crawler

  Example user agent
  ZoomInfo::Beehive Crawler

• 4x detected (3x unique)
  YodaoBot,YodaoBot-Image,YodaoBot-Image/1.0

  Example user agent
  Mozilla/5.0 (compatible;YodaoBot-Image/1.0;http://www.youdao.com/help/webmaster/spider/;)

• 3x detected (2x unique)
  Semrush Bot,SemrushBot

  Example user agent
  Mozilla/5.0 (compatible; SemrushBot-SA/0.97; +http://www.semrush.com/bot.html)

• 2x detected (2x unique)
  WinWebBot,WinWebBot/1.0

  Example user agent
  WinWebBot/1.0; (Balaena Ltd, UK); http://www.balaena.com/winwebbot.html; winwebbot@balaena.com;)

• 2x detected (1x unique)
  SMTBot

  Example user agent
  Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A5376e Safari/8536.25 (compatible; SMTBot/1.0; +http://www.similartech.com/smtbot)

• 3x detected (1x unique)
  Zookabot

  Example user agent
  Zookabot/2.4; +http://zookabot.com

• 3x detected (2x unique)
  FyberSpider,FyberSpider/1.3

  Example user agent
  FyberSpider/1.3 (http://www.fybersearch.com/fyberspider.php)

• 3x detected (1x unique)
  rogerbot

  Example user agent
  rogerbot/1.1 (http://moz.com/help/pro/what-is-rogerbot-, rogerbot-crawler+pr1-crawler-14@moz.com)

• 3x detected (1x unique)
  NetSeer crawler

  Example user agent
  Mozilla/5.0 (compatible; NetSeer crawler/2.0; +http://www.netseer.com/crawler.html; crawler@netseer.com)

• 2x detected (2x unique)
  Infohelfer,Infohelfer/1.2.0

  Example user agent
  Mozilla/5.0 (compatible; Infohelfer/1.2.0; +http://www.infohelfer.de/crawler.php)

• 3x detected (1x unique)
  DotBot

  Example user agent
  dotbot

• 2x detected (2x unique)
  com/bot,SEOdiver Bot

  Example user agent
  Mozilla/5.0 (compatible; SEOdiver/1.0; +http://www.seodiver.com/bot)

• 2x detected (2x unique)
  UptimeRobot,Uptime Robot

  Example user agent
  Mozilla/5.0+(compatible; UptimeRobot/2.0; http://www.uptimerobot.com/)

• 1x detected (1x unique)
  Crawler

  Example user agent
  Mozilla/5.0 (compatible; crawler/3.0.0 +http://www.notconfigured.com/)

• 5x detected (3x unique)
  YoudaoBot,YoudaoBot/1.0,Youdao Bot

  Example user agent
  Mozilla/5.0 (compatible; YoudaoBot/1.0; http://www.youdao.com/help/webmaster/spider/; )

• 1x detected (1x unique)
  it-Crawler

  Example user agent
  Liquida.it-Crawler/1.0 ( crawler@liquida.it +http://www.liquida.it )

• 1x detected (1x unique)
  CosmixCrawler

  Example user agent
  CosmixCrawler/0.1

• 1x detected (1x unique)
  Content Crawler

  Example user agent
  Content Crawler

• 1x detected (1x unique)
  kgbody

  Example user agent
  Mozilla/5.0 (compatible; kgbody/2.0; +http://kids.goo.ne.jp/tool/kgbody.php)

• 1x detected (1x unique)
  TridentSpider

  Example user agent
  Mozilla/5.0 (compatible; TridentSpider/3.1)

• 2x detected (2x unique)
  FAST PartnerSite,PartnerSite Crawler

  Example user agent
  Fast PartnerSite Crawler

• 1x detected (1x unique)
  Door Spider

  Example user agent
  FFC Trap Door Spider

• 1x detected (1x unique)
  loc-crawl

  Example user agent
  Mozilla/5.0 (compatible; loc-crawl/1.10.1 +http://www.google.com)

• 3x detected (3x unique)
  SEOprofiler,OpenLinkProfiler,spbot

  Example user agent
  Mozilla/5.0 (compatible; spbot/4.1.0; +http://OpenLinkProfiler.org/bot )

• 1x detected (1x unique)
  Crawler

  Example user agent
  1NET/Crawler (compatible; 1NET; robot; http://www.1NET.gr)

• 1x detected (1x unique)
  BOT for JCE

  Example user agent
  BOT/0.1 (BOT for JCE)

• 2x detected (2x unique)
  Sputnik Bot,SputnikBot

  Example user agent
  Mozilla/5.0 (compatible; SputnikBot/2.2)

• 4x detected (1x unique)
  thumbshots-de-bot

  Example user agent
  thumbshots-de-bot (+http://www.thumbshots.de/)

• 2x detected (2x unique)
  SpiderLing,Generic Research Crawler

  Example user agent
  Mozilla/5.0 (compatible; SpiderLing (a SPIDER for LINGustic research); +http://nlp.fi.muni.cz/projects/biwec/)

• 3x detected (1x unique)
  woriobot

  Example user agent
  Mozilla/5.0 (compatible; woriobot +http://worio.com)

• 2x detected (1x unique)
  GoGuidesBot

  Example user agent
  GoGuidesBot/0.0.1 (GoGuides Indexing Spider; http://www.goguides.org/spider.html)

• 2x detected (1x unique)
  GarlikCrawler

  Example user agent
  GarlikCrawler/1.2 (http://garlik.com/

• 1x detected (1x unique)
  ADJUSTbot

  Example user agent
  Mozilla/5.0 (compatible; ADJUSTbot/2.0; +http://www.ad-just.jp/)

• 1x detected (1x unique)
  LikaholixCrawler

  Example user agent
  Mozilla/5.0 (compatible; LikaholixCrawler/1.0; +http://mylikes.com/about/crawler)

• 2x detected (1x unique)
  AskPeterBot

  Example user agent
  Mozilla/5.0 (compatible; AskPeterBot; +http://www.askpeter.de/bot.html)

• 1x detected (1x unique)
  GematchCrawler

  Example user agent
  GematchCrawler/2.1 (http://www.gematch.com/crawler.html)

• 2x detected (2x unique)
  LSSRocketCrawler,Lightspeed Systems RocketCrawler

  Example user agent
  LSSRocketCrawler/1.0 LightspeedSystems

• 1x detected (1x unique)
  MXit WebBot

  Example user agent
  Opera/8.01 (J2ME/MIDP; MXit WebBot/1.8.3.119) Opera Mini/3.1

• 2x detected (1x unique)
  HyperCrawl

  Example user agent
  Mozilla/5.0 (compatible; HyperCrawl/0.2; +http://www.seograph.net/bot.html)

• 3x detected (3x unique)
  Website Thumbnail Generator,WebThumb,WebThumbnail

  Example user agent
  Mozilla/5.0 (compatible; WebThumbnail/2.2; Website Thumbnail Generator; +http://webthumbnail.org)

• 5x detected (2x unique)
  DotBot,DotBot/1.1

  Example user agent
  Mozilla/5.0 (compatible; DotBot/1.1; http://www.dotnetdotcom.org/, crawler@dotnetdotcom.org)

• 1x detected (1x unique)
  Infocrawler

  Example user agent
  Mozilla/4.0 (compatible; MSIE 5.5; Windows 98; Infocrawler; Alexa Toolbar)

• 1x detected (1x unique)
  WangIDSpider

  Example user agent
  Mozilla/5.0 (compatible; WangIDSpider/1.0; +http://www.wangid.com/spider.html)

• 2x detected (1x unique)
  IssueCrawler

  Example user agent
  IssueCrawler

• 2x detected (2x unique)
  WebTarantula,com Crawler

  Example user agent
  WebTarantula.com Crawler

• 1x detected (1x unique)
  PeerFactor Crawler

  Example user agent
  PeerFactor Crawler

• 1x detected (1x unique)
  SpeedySpider

  Example user agent
  Mozilla/4.0 (compatible; SpeedySpider; www.entireweb.com)

• 2x detected (1x unique)
  UniversalFeedParser

  Example user agent
  UniversalFeedParser/5.1.3 +https://code.google.com/p/feedparser/

• 2x detected (1x unique)
  sReleaseBot

  Example user agent
  Mozilla/5.0 (compatible; sReleaseBot/6.1; +http://www.srelease.com/)

• 1x detected (1x unique)
  ReadPath_Spider

  Example user agent
  ReadPath_Spider

• 3x detected (2x unique)
  Linkdex Bot,linkdexbot

  Example user agent
  Mozilla/5.0 (compatible; linkdexbot/2.0; +http://www.linkdex.com/about/bots/)

• 3x detected (3x unique)
  General Crawlers,SmartAndSimpleWebCrawler,Java

  Example user agent
  SmartAndSimpleWebCrawler/1.3 (https://crawler.dev.java.net),gzip(gfe),gzip(gfe)

• 1x detected (1x unique)
  Overture-WebCrawler

  Example user agent
  Overture-WebCrawler/3.8/Fresh (atw-crawler at fast dot no; http://fast.no/support/crawler.asp)

• 2x detected (2x unique)
  imrbot,Mignify Bot

  Example user agent
  Mozilla/5.0 (compatible; imrbot/1.10.8 +http://www.mignify.com)

• 1x detected (1x unique)
  HttpSpider

  Example user agent
  HttpSpider/0.91

• 2x detected (2x unique)
  Internet Archive,UA crawler

  Example user agent
  Mozilla/2.0 (compatible; ARCHIVE.ORG.UA crawler)

• 0x detected (0x unique)

  Example user agent
  Mozilla/5.0 (Linux; U; Android 4.2.2; it-it; S308 Build/CUBOT) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30

• 0x detected (0x unique)

  Example user agent
  SearQuBot/SearQuBot v1.0

• 1x detected (1x unique)
  PAD-bot

  Example user agent
  Mozilla/5.0 (compatible; PAD-bot/9.0; +http://www.descargarprogramagratis.com/)

• 3x detected (1x unique)
  MQBOT

  Example user agent
  MQbot http://metaquerier.cs.uiuc.edu/crawler

• 1x detected (1x unique)
  Symfony Spider

  Example user agent
  Symfony Spider (http://symfony.com/spider)

• 2x detected (1x unique)
  memorybot

  Example user agent
  Mozilla/5.0 (compatible; memorybot/1.20.41 +http://archivethe.net/en/index.php/about/internet_memory1 on behalf of DNB)

• 3x detected (2x unique)
  discoverybot,discoverybot/2.0

  Example user agent
  Mozilla/5.0 (compatible; discoverybot/2.0; +http://discoveryengine.com/discoverybot.html)

• 1x detected (1x unique)
  it2media-domain-crawler

  Example user agent
  it2media-domain-crawler/1.0 on crawler-prod.it2media.de

• 1x detected (1x unique)
  WebmasterCoffee

  Example user agent
  Mozilla/5.0 (compatible; WebmasterCoffee/0.7; +http://webmastercoffee.com/about)

• 2x detected (1x unique)
  JobdiggerSpider

  Example user agent
  Mozilla/5.0 (compatible; JobdiggerSpider +http://www.jobdigger.nl/spider)

• 1x detected (1x unique)
  WebCrunch

  Example user agent
  Mozilla/5.0 (compatible; WebCrunch/1.2; +http://webcrunch.net/crawler)

• 1x detected (1x unique)
  CamontSpider

  Example user agent
  CamontSpider/1.0 http://epweb2.ph.bham.ac.uk/user/slater/camont/info.html

• 2x detected (1x unique)
  LoadTimeBot

  Example user agent
  Mozilla/5.0 (compatible; LoadTimeBot/0.9; +http://www.loadtime.net/bot.html)

• 1x detected (1x unique)
  audioCrawlerBot

  Example user agent
  audioCrawlerBot/1.0 (http://www.audiocrawler.com/)

• 3x detected (3x unique)
  Amfibibot/0.07,Amfibibot,Amfibi

  Example user agent
  Amfibibot/0.07 (Amfibi Robot; http://www.amfibi.com; agent@amfibi.com)

• 3x detected (2x unique)
  yrspider,Yunyun Bot

  Example user agent
  Mozilla/5.0 (compatible; YRSpider; +http://www.yunrang.com/yrspider.html)

• 2x detected (2x unique)
  Research Projects,CMS crawler

  Example user agent
  CMS crawler ( http://buytaert.net/crawler/)

• 2x detected (1x unique)
  Socialradarbot

  Example user agent
  Mozilla/5.0 (compatible; Linux; Socialradarbot/2.0; en-US; crawler@infegy.com)

• 2x detected (1x unique)
  MarketwireBot

  Example user agent
  Mozilla/5.0 (compatible; MSIE 9.0; Windows NT; MarketwireBot +http://www.marketwire.com)

• 4x detected (2x unique)
  SISTRIX Crawler,sistrix

  Example user agent
  Mozilla/5.0 (compatible; SISTRIX Crawler; http://crawler.sistrix.net/)

• 3x detected (3x unique)
  Google Bot,GooglePlusBot,Googlebot

  Example user agent
  Google (+https://developers.google.com/+/web/snippet/)

• 2x detected (1x unique)
  XoviBot

  Example user agent
  Mozilla/5.0 (compatible; XoviBot/2.0; +http://www.xovibot.net/)

• 1x detected (1x unique)
  Axtaris

  Example user agent
  Axtaris Web Crawler/Axtaris-1.0.1 (Hello from Axtaris.org | Web-Crawler; http://axtaris.org; crawler at axtaris dot org)

• 2x detected (2x unique)
  mixi-crawler,mixi

  Example user agent
  mixi-crawler/2.00 (http://mixi.jp/)

• 1x detected (1x unique)
  Google Structured-Data TestingTool

  Example user agent
  Mozilla/5.0 (compatible; X11; Linux x86_64; Google-StructuredDataTestingTool; +http://www.google.com/webmasters/tools/richsnippets)

• 3x detected (1x unique)
  Rogerbot

  Example user agent
  rogerbot/1.0 (http://www.moz.com/dp/rogerbot, rogerbot-crawler@moz.com)

• 2x detected (1x unique)
  ShowyouBot

  Example user agent
  ShowyouBot (http://showyou.com/crawler)

• 1x detected (1x unique)
  ProloCrawler

  Example user agent
  ProloCrawler/1.0 (http://www.prolo.com)

• 3x detected (2x unique)
  Esribot,Esribot/1.0

  Example user agent
  Mozilla/5.0 (compatible; Esribot/1.0; http://www.esrihu.hu/)

• 2x detected (2x unique)
  Fake Browser,compatible crawler

  Example user agent
  http://www.Syntryx.com/ ANT Chassis 9.27; Mozilla/4.0 compatible crawler

• 1x detected (1x unique)
  METASpider

  Example user agent
  Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0) METASpider

• 2x detected (1x unique)
  LinkpadBot

  Example user agent
  Mozilla/5.0 (compatible; LinkpadBot/1.06; +http://www.linkpad.ru)

• 1x detected (1x unique)
  KIT webcrawler

  Example user agent
  KIT webcrawler/0.2.4

• 1x detected (1x unique)
  DotSpotsBot

  Example user agent
  DotSpotsBot/0.2 (crawler; support at dotspots.com)

• 2x detected (1x unique)
  GSiteCrawler

  Example user agent
  GSiteCrawler/v1.12 rev. 260 (http://gsitecrawler.com/)

• 3x detected (2x unique)
  SurveyBot,Survey Bot

  Example user agent
  Mozilla/5.0 (Windows; U; Windows NT 5.1; en; rv:1.9.0.13) Gecko/2009073022 Firefox/3.5.2 (.NET CLR 3.5.30729) SurveyBot/2.3 (DomainTools)

• 1x detected (1x unique)
  PsSpider

  Example user agent
  PsSpider/0.7 (spider@bildkiste.de)

• 4x detected (2x unique)
  Zookabot,Zookabot/2.5

  Example user agent
  Zookabot/2.5;++http://zookabot.com

• 1x detected (1x unique)
  Gnam Spider

  Example user agent
  Gnam Gnam Spider

• 2x detected (2x unique)
  QuickFinder Crawler,QuickFinder

  Example user agent
  QuickFinder Crawler

• 2x detected (2x unique)
  Finderbots,bixo

  Example user agent
  Mozilla/5.0 (compatible; Finderbots finder bot; +http://wiki.github.com/bixo/bixo/bixocrawler; bixo-dev@yahoogroups.com)

• 1x detected (1x unique)
  Axadine Crawler

  Example user agent
  axadine/ (Axadine Crawler; http://www.axada.de/; )

• 2x detected (1x unique)
  EveryoneSocialBot

  Example user agent
  Mozilla/5.0 (compatible; EveryoneSocialBot/1.0; support@everyonesocial.com http://everyonesocial.com/)

• 2x detected (2x unique)
  Jambot,Jambot/0.1.1

  Example user agent
  Jambot/0.1.1 (Jambot; http://www.jambot.com/blog; crawler@jambot.com)

• 2x detected (1x unique)
  BabalooSpider

  Example user agent
  BabalooSpider/1.2 (BabalooSpider; http://www.babaloo.si; spider@babaloo.si)

• 2x detected (1x unique)
  grub

  Example user agent
  Grub/2.0 (Grub.org crawler; http://www.grub.org/; bot@grub.org)

• 3x detected (2x unique)
  TestCrawler,LDSpider

  Example user agent
  ldspider (BTC 2011 crawl, harth@kit.edu, http://code.google.com/p/ldspider/wiki/Robots)

• 1x detected (1x unique)
  archive_crawler

  Example user agent
  Mozilla/5.0 (compatible; archive_crawler/3.0.0-SNAPSHOT-20091205.013431 http://www.archive.org/details/archive_crawler)

• 2x detected (2x unique)
  com crawler,semantic-visions.com crawler

  Example user agent
  Mozilla/5.0 (X11; compatible; semantic-visions.com crawler; HTTPClient 3.1)

• 3x detected (2x unique)
  yrspider,Yunyun Bot

  Example user agent
  yrspider (Mozilla/5.0 (compatible; YRSpider; http://www.yunrang.com/yrspider.html))

• 3x detected (3x unique)
  Seznam screenshot-generator 2.0,Seznam Bot,Seznam Screenshot Generator

  Example user agent
  Mozilla/5.0 (compatible; Seznam screenshot-generator 2.0; +http://fulltext.sblog.cz/screenshot/)

• 2x detected (1x unique)
  AjaxSnapBot

  Example user agent
  Mozilla/5.0 (compatible; AjaxSnapBot; +http://ajaxsnapshots.com/bot.html)

• 2x detected (2x unique)
  Slackbot,Slackbot-Link-Expanding

  Example user agent
  Slackbot-LinkExpanding+(+https://api.slack.com/robots)

• 2x detected (2x unique)
  DNSGroup,DNS Group Crawler

  Example user agent
  DNSGroup/0.1 (DNS Group Crawler; http://www.dnsgroup.com/; crawler@dnsgroup.com)

• 2x detected (1x unique)
  5Bot

  Example user agent
  Mozilla/5.1 (compatible; +5Bot/1.0; +http://plus5files.net)

• 1x detected (1x unique)
  Engage Crawler

  Example user agent
  Adknowledge Engage Crawler

• 4x detected (3x unique)
  AcoonBot/4.11.1,AcoonBot,Acoon

  Example user agent
  Mozilla/5.0 (compatible; AcoonBot/4.11.1; +http://www.acoon.de/robot.asp)

• 5x detected (2x unique)
  Gigabot,Gigabot/3.0

  Example user agent
  Gigabot/3.0 (http://www.gigablast.com/spider.html)

• 2x detected (1x unique)
  Steeler

  Example user agent
  Steeler/1.3 (http://www.tkl.iis.u-tokyo.ac.jp/~crawler/)

• 2x detected (1x unique)
  m65bot

  Example user agent
  DoCoMo/2.0 P904i( m65bot/0.1; c; http://m65.jp/bot.html )

• 2x detected (2x unique)
  DuckDuckGo-Favicons-Bot,DuckDuck Favicons Bot

  Example user agent
  Mozilla/5.0 (compatible; DuckDuckGo-Favicons-Bot/1.0; +http://duckduckgo.com)

• 2x detected (2x unique)
  SpokeSpider/1.0,SpokeSpider

  Example user agent
  SpokeSpider/1.0 (http://support.spoke.com/webspider/) Mozilla/5.0 (not really)

• 1x detected (1x unique)
  Automattic Analytics Crawler

  Example user agent
  Automattic Analytics Crawler/0.2;+http://wordpress.com/crawler/

• 1x detected (1x unique)
  MegaIndex Bot

  Example user agent
  Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +https://www.megaindex.ru/?tab=linkAnalyze)

• 3x detected (2x unique)
  sproose,Sproose/1.0beta

  Example user agent
  sproose/1.0beta (sproose bot; http://www.sproose.com/bot.html; crawler@sproose.com)

• 2x detected (1x unique)
  HRCrawler

  Example user agent
  HRCrawler/2.0 (XF86; MacOS x86_64) AppleWebKit/537.31 (KHTML, like Gecko)

• 1x detected (1x unique)
  Denodo IECrawler

  Example user agent
  Denodo IECrawler/4.5,gzip(gfe),gzip(gfe)

• 2x detected (1x unique)
  NerdyBot

  Example user agent
  NerdyBot

• 1x detected (1x unique)
  blackspider

  Example user agent
  blackspider

• 1x detected (1x unique)
  Index crawler

  Example user agent
  IOI/2.0 (ISC Open Index crawler; http://index.isc.org/; bot@index.isc.org)

• 2x detected (2x unique)
  gsa-crawler,Google Search Appliance

  Example user agent
  gsa-crawler

• 2x detected (2x unique)
  woobot,WooRank

  Example user agent
  woobot/1.1

• 1x detected (1x unique)
  boitho crawler

  Example user agent
  boitho crawler

• 1x detected (1x unique)
  info Crawler

  Example user agent
  visaduhoc.info Crawler

• 1x detected (1x unique)
  Moreover

  Example user agent
  Mozilla/5.0 Moreover/5.1 (+http://www.moreover.com; webmaster@moreover.com)

• 3x detected (2x unique)
  netEstate NE Crawler,NE Crawler

  Example user agent
  netEstate NE Crawler (+http://www.sengine.info/)

• 1x detected (1x unique)
  DomainAppender Bot

  Example user agent
  Mozilla/5.0 (compatible; DomainAppender /1.0; +http://www.profound.net/domainappender)

• 1x detected (1x unique)
  MetaInspector

  Example user agent
  MetaInspector/3.0.0 (+https://github.com/jaimeiniesta/metainspector)

• 3x detected (2x unique)
  SemrushBot,Semrush Bot

  Example user agent
  Mozilla/5.0 (compatible; SemrushBot-SI/0.97; +http://www.semrush.com/bot.html)

• 1x detected (1x unique)
  DoubleVerify Crawler

  Example user agent
  DoubleVerify Crawler

• 4x detected (2x unique)
  Google Bot,Googlebot

  Example user agent
  Googlebot

• 2x detected (1x unique)
  ArchitextSpider

  Example user agent
  ArchitextSpider

• 4x detected (1x unique)
  TweetedTimes Bot

  Example user agent
  Mozilla/5.0 (compatible; TweetedTimes Bot/1.0 (resolver); +http://tweetedtimes.com)

• 4x detected (1x unique)
  DotBot

  Example user agent
  DotBot/1.0.1 (http://www.dotnetdotcom.org/, crawler@dotnetdotcom.org)

• 3x detected (2x unique)
  Steeler,Steeler/3.5

  Example user agent
  Mozilla/5.0 (compatible; Steeler/3.5; http://www.tkl.iis.u-tokyo.ac.jp/~crawler/)

• 1x detected (1x unique)
  Sosoblogspider

  Example user agent
  Sosoblogspider+(+http://help.soso.com/soso-blog-spider.htm)

• 2x detected (1x unique)
  lbot

  Example user agent
  Mozilla/5.0 (lbot/2.6.4) lbot@mailme.not

• 3x detected (3x unique)
  redditbot,Reddit Bot,Reddit

  Example user agent
  Mozilla/5.0 (compatible; redditbot/1.0; +http://www.reddit.com/feedback)

• 0x detected (0x unique)

  Example user agent
  Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_8) AppleWebKit/534.50 (KHTML, like Gecko) Version/5.1 Instapaper/4.0 (+http://www.instapaper.com/)

• 2x detected (1x unique)
  Uipbot

  Example user agent
  uipbot/1.0 (uipbot@semasio.net)

• 1x detected (1x unique)
  Fake Browser

  Example user agent
  Mozzila/5.0 (compatible; Sonic/1.0; http://www.yama.info.waseda.ac.jp/~crawler/info.html)

• 2x detected (1x unique)
  AntBot

  Example user agent
  AntBot/1.0 (http://www.ant.com)

• 3x detected (1x unique)
  WeSEE:Search

  Example user agent
  WeSEE:Search/0.1 (Alpha, http://www.wesee.com/bot/)

• 3x detected (2x unique)
  Sogou Orion spider,sogou spider

  Example user agent
  Sogou Orion spider/4.0( http://www.sogou.com/docs/help/webmasters.htm#07)

• 1x detected (1x unique)
  ptd-crawler

  Example user agent
  Mozilla/5.0 (compatible; ptd-crawler; http://bixolabs.com/crawler/ptd/; crawler@bixolabs.com)

• 3x detected (3x unique)
  Ex-Crawler,Java,General Crawlers

  Example user agent
  Mozilla/5.0 (compatible; Ex-Crawler/v0.1.5ALPHA SVN rev58; powered by ex-crawler; +http://www.ex-crawler.de/) Java/1.6.0_20,gzip(gfe),gzip(gfe)

• 2x detected (1x unique)
  MixBot

  Example user agent
  MixBot (+http://t.co/GSRLLKex24)

• 1x detected (1x unique)
  BlogRangerCrawler

  Example user agent
  BlogRangerCrawler/1.0

• 3x detected (3x unique)
  Ex-Crawler,Java,General Crawlers

  Example user agent
  Mozilla/5.0 (compatible; Ex-Crawler/0.1.5a; powered by ex-crawler; +http://www.ex-crawler.de/) Java/1.6.0_20,gzip(gfe),gzip(gfe)

• 3x detected (2x unique)
  Openindex Spider,OpenindexSpider

  Example user agent
  Mozilla/5.0 (compatible; OpenindexSpider; +http://www.openindex.io/en/webmasters/spider.html)

• 1x detected (1x unique)
  GoGuidesBot

  Example user agent
  Mozilla/5.0 (compatible; GoGuidesBot; http://www.goguides.org/spider.html)

• 1x detected (1x unique)
  Jomjaibot

  Example user agent
  Jomjaibot/1.0 Crawl (+http://www.jomjaibot.com/)

• 3x detected (2x unique)
  omgilibot,Omgili bot

  Example user agent
  omgilibot/0.3 +http://www.omgili.com/Crawler.html

• 2x detected (2x unique)
  AppCodes crawler,TestCrawler

  Example user agent
  AppCodes crawler - looking for iOS app mentions. More info: support@appcodes.com. Robots.txt id: AppCodesCrawler

• 1x detected (1x unique)
  Logict IPv6 Crawler

  Example user agent
  Logict IPv6 Crawler/1.0 (http://ipv6search.logict.net),gzip(gfe),gzip(gfe)

• 3x detected (1x unique)
  Scrubby

  Example user agent
  Mozilla/5.0 (compatible; Scrubby/3.2; +http://seotools.scrubtheweb.com/webpage-analyzer.html)

• 1x detected (1x unique)
  fastlwspider

  Example user agent
  fastlwspider/1.0

• 2x detected (1x unique)
  ca-crawler

  Example user agent
  Mozilla/5.0 (compatible; ca-crawler/1.0)

• 4x detected (3x unique)
  MJ12bot,MJ12bot/v1.4.3,MJ12 Bot

  Example user agent
  Mozilla/5.0 (compatible; MJ12bot/v1.4.3; http://www.majestic12.co.uk/bot.php?+)

• 4x detected (1x unique)
  MojeekBot

  Example user agent
  Mozilla/5.0 (compatible; MojeekBot/0.6; http://www.mojeek.com/bot.html)

• 1x detected (1x unique)
  449 Overture-WebCrawler

  Example user agent
  449 Overture-WebCrawler/3.8/Fresh (atw-crawler at fast dot no; http://fast.no/support/crawler.asp

• 2x detected (2x unique)
  PageFreezer,pagefreezer crawler

  Example user agent
  PageFreezer (pagefreezer crawler; http://pagefreezer.com/pagefreezer-crawler/; info@pagefreezer.com)

• 2x detected (1x unique)
  BOTW Spider

  Example user agent
  Mozilla/4.0 (compatible; BOTW Spider; http://botw.org)

• 1x detected (1x unique)
  Rakutenbot

  Example user agent
  Mozilla/5.0 (compatible; Rakutenbot/1.0; +http://dynamic.rakuten.co.jp/bot.html)

• 4x detected (2x unique)
  ScoutJet,ScoutJet old

  Example user agent
  Mozilla/5.0 (compatible; ScoutJet; +http://www.scoutjet.com/)

• 1x detected (1x unique)
  kizasi-spider

  Example user agent
  kizasi-spider/1.0 (+http://kizasi.jp/)

• 1x detected (1x unique)
  BeijingCrawler

  Example user agent
  BeijingCrawler

• 1x detected (1x unique)
  vkShare

  Example user agent
  Mozilla/5.0 (compatible; vkShare; +http://vk.com/dev/Share)

• 1x detected (1x unique)
  BejiBot

  Example user agent
  BejiBot Crawler (BNL Services; http://www.bejjan.net/crawler/)

• 2x detected (2x unique)
  woobot,WooRank

  Example user agent
  woobot/2.0

• 1x detected (1x unique)
  BuildCMS crawler

  Example user agent
  BuildCMS crawler (http://www.buildcms.com/crawler)

• 3x detected (1x unique)
  Screaming Frog SEO Spider

  Example user agent
  Screaming Frog SEO Spider/2,01

• 1x detected (1x unique)
  X-Crawler

  Example user agent
  X-Crawler

• 5x detected (3x unique)
  BingBot,Locubot,Googlebot

  Example user agent
  Locubot (compatible; Googlebot; msnbot)

• 2x detected (2x unique)
  Nutch,GrowerIdeas Crawler

  Example user agent
  GrowerIdeas Crawler/GrowerIdeas-nutch-1.6 (Crawls URLs for indexing content for our new search startup which aims to provide simple and smart search across curated content. For more info please contact helloworld@growerideas.com. If you think

• 2x detected (2x unique)
  DealGates Bot/1.1,DealGates Bot

  Example user agent
  DealGates Bot/1.1 by Luc Michalski (http://spider.dealgates.com/bot.html)

• 2x detected (1x unique)
  WebAlta Crawler

  Example user agent
  WebAlta Crawler/2.0 (http://www.webalta.net/ru/about_webmaster.html) (Windows; U; Windows NT 5.1; ru-RU)

• 3x detected (1x unique)
  Screaming Frog SEO Spider

  Example user agent
  Screaming Frog SEO Spider/2.40

• 1x detected (1x unique)
  Yahoo-VerticalCrawler-FormerWebCrawler

  Example user agent
  Yahoo-VerticalCrawler-FormerWebCrawler/3.9 crawler at trd dot overture dot com; http://www.alltheweb.com/help/webmaster/crawler

• 1x detected (1x unique)
  Yahoo! Japan

  Example user agent
  Mozilla/5.0 (Linux; U; Android 2.3.3; ja-jp; sdk Build/GRI34; Y!J-BRZ/YATSHA crawler; http://help.yahoo.co.jp/help/jp/search/indexing/indexing-15.html) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1

• 1x detected (1x unique)
  dtSearchSpider

  Example user agent
  dtSearchSpider

• 3x detected (3x unique)
  Baidu Mobile Search,Baidu Spider,BaiduMobaider

  Example user agent
  DoCoMo/2.0 P05A(c100;TB;W24H15) (compatible; BaiduMobaider/1.0; http://www.baidu.jp/spider/)

• 5x detected (3x unique)
  SeznamBot,SeznamBot/3.0,Seznam Bot

  Example user agent
  SeznamBot/3.0 (+http://fulltext.sblog.cz/)

• 1x detected (1x unique)
  CrawlBot

  Example user agent
  Mozilla/5.0 (compatible; CrawlBot/1.0.0)

• 2x detected (2x unique)
  webceo Bot,online-webceo-bot

  Example user agent
  Mozilla/5.0 (compatible; online-webceo-bot/1.0; +http://online.webceo.com)

• 1x detected (1x unique)
  wminer

  Example user agent
  wminer/1.2 (North America Web Crawler; http://www.wminer.com/bot; info at wminer dot com)

• 1x detected (1x unique)
  bendercrawler

  Example user agent
  Mozilla/5.0 (compatible; Bender; http://sites.google.com/site/bendercrawler)

• 3x detected (1x unique)
  Screaming Frog SEO Spider

  Example user agent
  Screaming Frog SEO Spider/2.22

• 1x detected (1x unique)
  haupia-crawler

  Example user agent
  haupia-crawler

• 1x detected (1x unique)
  WSDL Crawler

  Example user agent
  WSDL Crawler

• 1x detected (1x unique)
  semantics webbot

  Example user agent
  semantics webbot/1.0

• 5x detected (2x unique)
  ExaBot,Exabot/3.0/BiggerBetter

  Example user agent
  Mozilla/5.0 (compatible; Exabot/3.0 (BiggerBetter); +http://www.exabot.com/go/robot)

• 2x detected (1x unique)
  WIRE

  Example user agent
  WIRE/1.0 (Linux;i686;Bot,Robot,Spider,Crawler)

• 2x detected (2x unique)
  EventGuruBot/1.0,EventGuruBot

  Example user agent
  Mozilla/5.0 (compatible; EventGuruBot/1.0; +http://www.eventguru.com/spider.html)

• 4x detected (2x unique)
  sogou spider,Sogou Pic Spider

  Example user agent
  Sogou Pic Spider/3.0( http://www.sogou.com/docs/help/webmasters.htm#07)

• 3x detected (1x unique)
  BitlyBot

  Example user agent
  bitlybot

• 2x detected (2x unique)
  HubSpot Webcrawler,Webcrawler

  Example user agent
  Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.116 Safari/537.36 HubSpot Webcrawler

• 1x detected (1x unique)
  None-Crawler

  Example user agent
  Mozilla/5.0 (compatible; .None-Crawler/0.1 +http://domains.ericbinek.None/)

• 1x detected (1x unique)
  ScSpider

  Example user agent
  ScSpider/0.2

• 4x detected (2x unique)
  Rogerbot,rogerbot/1.0

  Example user agent
  rogerbot/1.0 (http://moz.com/help/pro/what-is-rogerbot-, rogerbot-crawler+shiny@moz.com)

• 4x detected (2x unique)
  Seznam Bot,SeznamBot

  Example user agent
  Mozilla/5.0 (compatible; SeznamBot/3.2-test1; +http://fulltext.sblog.cz/)

• 2x detected (1x unique)
  DomainCrawler

  Example user agent
  DomainCrawler/2.0 (info@domaincrawler.com; http://www.domaincrawler.com/bot

• 2x detected (2x unique)
  SputnikImageBot,Sputnik Bot

  Example user agent
  Mozilla/5.0 (compatible; SputnikImageBot/2.2)

• 2x detected (1x unique)
  MonoBot

  Example user agent
  Mozilla/5.1 (compatible; MonoBot/1.0; +http://mono.name)

• 3x detected (1x unique)
  URLAppendBot

  Example user agent
  Mozilla/5.0 (compatible; URLAppendBot/1.0; +http://www.profound.net/urlappendbot.html)

• 2x detected (1x unique)
  PluckItCrawler

  Example user agent
  PluckItCrawler/1.0 (compatible; Mozilla 4.0; MSIE 5.5; http://www.pluck.com;)

• 1x detected (1x unique)
  tCrawler

  Example user agent
  tCrawler/0.1,gzip(gfe),gzip(gfe),gzip(gfe)

• 3x detected (2x unique)
  Sogou Head Spider,sogou spider

  Example user agent
  Sogou head spider/3.0( http://www.sogou.com/docs/help/webmasters.htm#07)

• 1x detected (1x unique)
  TAMU_CS_IRL_CRAWLER

  Example user agent
  TAMU_CS_IRL_CRAWLER/1.0

• 2x detected (2x unique)
  ownCloud Server Crawler,Server Crawler

  Example user agent
  ownCloud Server Crawler

• 3x detected (2x unique)
  envolk/1.7,Envolk

  Example user agent
  envolk/1.7 (+http://www.envolk.com/envolkspiderinfo.html)

• 2x detected (1x unique)
  diffbot

  Example user agent
  Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.15 (KHTML, like Gecko; +http://www.diffbot.com) Chrome/24.0.1295.0 Safari/537.15

• 2x detected (1x unique)
  Slackbot

  Example user agent
  Slackbot (+https://api.slack.com/robots)

• 1x detected (1x unique)
  Qwantify

  Example user agent
  Mozilla/5.0 (compatible; Qwantify/2.0; +https://www.qwant.com/)

• 1x detected (1x unique)
  FunkyCrawler

  Example user agent
  Mozilla/5.0 (compatible; FunkyCrawler; )

• 2x detected (1x unique)
  Pete-Spider

  Example user agent
  Pete-Spider/1.1

• 4x detected (2x unique)
  Googlebot,acapbot

  Example user agent
  Mozilla/5.0 (compatible;acapbot/0.1.;treat like Googlebot)

• 4x detected (1x unique)
  TurnitinBot

  Example user agent
  TurnitinBot/1.5 (http://www.turnitin.com/robot/crawlerinfo.html)

• 2x detected (1x unique)
  noxtrumbot

  Example user agent
  noxtrumbot/1.0 (crawler@noxtrum.com)

• 2x detected (2x unique)
  bl.uk_lddc_bot,uk_lddc_bot

  Example user agent
  bl.uk_lddc_bot/3.3.0-SNAPSHOT-2014-10-07T09:33:31Z (+http://www.bl.uk/aboutus/legaldeposit/websites/websites/faqswebmaster/index.html)

• 2x detected (1x unique)
  Crawler

  Example user agent
  Mozilla/5.0 (compatible; Crawler/0.9; http://linkfluence.net/)

• 2x detected (2x unique)
  NetWhatCrawler,NetWhatCrawler/0.06-dev

  Example user agent
  NetWhatCrawler/0.06-dev (NetWhatCrawler from NetWhat.com; http://www.netwhat.com; support@netwhat.com)

• 1x detected (1x unique)
  Host-Spy Crawler

  Example user agent
  Mozilla/5.0 (compatible; Host-Spy Crawler/1.2; +http://www.host-spy.com/)

• 2x detected (1x unique)
  SoftListBot

  Example user agent
  Mozilla/5.0 (compatible; SoftListBot/2.2; +http://www.softlist.us/)

• 2x detected (1x unique)
  ScollSpider

  Example user agent
  Mozilla/3.0 (compatible; ScollSpider; http://www.webwobot.com)

• 3x detected (1x unique)
  YioopBot

  Example user agent
  Mozilla/5.0 (compatible; YioopBot; +http://173.13.143.74/bot.php)

• 2x detected (2x unique)
  GroupHigh Bot,com/bot

  Example user agent
  Mozilla/5.0 (compatible; GroupHigh/1.0; +http://www.grouphigh.com/bot.html)

• 3x detected (3x unique)
  WebThumb,WebThumbnail,Website Thumbnail Generator

  Example user agent
  Mozilla/5.0 (compatible; WebThumbnail/3.x; Website Thumbnail Generator; +http://webthumbnail.org)

• 2x detected (2x unique)
  RedBot1.0,redbot

  Example user agent
  RedBot/redbot-1.0 (Rediff.com Crawler; redbot at rediff dot com)

• 1x detected (1x unique)
  Webspider

  Example user agent
  Webspider/1.0 (web spider; )

• 1x detected (1x unique)
  SMXCrawler

  Example user agent
  SMXCrawler/1.0 (www.socialmetrix.com)

• 2x detected (2x unique)
  bl.uk_lddc_bot,uk_lddc_bot

  Example user agent
  Mozilla/5.0 (bl.uk_lddc_bot; Linux x86_64) PhantomJS/1.9.7 (+http://www.bl.uk/aboutus/legaldeposit/websites/websites/faqswebmaster/index.html)

• 2x detected (1x unique)
  NalezenCzBot

  Example user agent
  NalezenCzBot/1.0 (http://www.nalezen.cz/about-crawler)

• 2x detected (2x unique)
  Flatland,flatlandbot

  Example user agent
  flatlandbot/allspark (Flatland Industries Web Spider; http://www.flatlandindustries.com/flatlandbot; jason@flatlandindustries.com)

• 4x detected (2x unique)
  Wotbox,Wotbox/2.01

  Example user agent
  Wotbox/2.01 (+http://www.wotbox.com/bot/)

• 1x detected (1x unique)
  Crowsnest/0.5

  Example user agent
  Crowsnest/0.5 (+http://www.crowsnest.tv/)

• 4x detected (1x unique)
  DotBot

  Example user agent
  Mozilla/5.0 (compatible; DotBot/1.1; http://www.opensiteexplorer.org/dotbot, help@moz.com)

• 1x detected (1x unique)
  CowCrawler

  Example user agent
  CowCrawler/CowCrawler-dev (+http://beta.cow.com),gzip(gfe),gzip(gfe)

• 3x detected (3x unique)
  SEOENGWorldBot,SEOENGBot/1.0,SEOENGBot

  Example user agent
  SEOENGWorldBot/1.0 (+http://www.seoengine.com/seoengbot.htm)

• 2x detected (2x unique)
  socialbm_bot,socialbm_bot/1.0

  Example user agent
  Mozilla/5.0 (compatible; socialbm_bot/1.0; +http://spider.socialbm.net)

• 1x detected (1x unique)
  RSSIncludeBot

  Example user agent
  RSSIncludeBot/1.0 (http://www.rssinclude.com/spider)

• 2x detected (1x unique)
  BuiBui-Bot

  Example user agent
  BuiBui-Bot/1.0 (email: buibui[at]dadapro[dot]com)

• 2x detected (1x unique)
  hivaBot

  Example user agent
  hivaBot/hivaBot-0.2 (info@gorgor.ir)

• 2x detected (2x unique)
  focuseekbot,seekbot

  Example user agent
  Mozilla/4.0 (compatible; focuseekbot)

• 2x detected (2x unique)
  EnaBot,EnaBot/1.2

  Example user agent
  EnaBot/1.2 (http://www.enaball.com/crawler.html)

• 1x detected (1x unique)
  Sangfor Spider

  Example user agent
  Sangfor Spider/0.7

• 2x detected (2x unique)
  fastbot crawler,fastbot

  Example user agent
  fastbot crawler beta 4.0 (+http://www.fastbot.de)

• 3x detected (1x unique)
  Applebot

  Example user agent
  Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/600.2.5 (KHTML, like Gecko) Version/8.0.2 Safari/600.2.5 (Applebot/0.1)

• 3x detected (1x unique)
  UnisterBot

  Example user agent
  Mozilla/5.0 (compatible; UnisterBot; crawler@unister.de)

• 1x detected (1x unique)
  EtaoSpider

  Example user agent
  Mozilla/5.0 (compatible; EtaoSpider/1.0; http://open.etao.com/dev/EtaoSpider)

• 2x detected (1x unique)
  DBot

  Example user agent
  Mozilla/5.1 (compatible; DBot/7.5.6; +http://a14download.com)

• 1x detected (1x unique)
  SeekOn Spider

  Example user agent
  SeekOn Spider 1.9(+http://www.seekon.com/spider.html)

• 2x detected (2x unique)
  Nett.io bot,io bot

  Example user agent
  Mozilla/5.0 (compatible; Nett.io bot/1.0; +http://nett.io/)

• 1x detected (1x unique)
  OpenX Spider

  Example user agent
  Mozilla/5.0 (compatible; OpenX Spider; http://www.openx.org)

• 2x detected (1x unique)
  WNMbot

  Example user agent
  WNMbot/0.4.0

• 1x detected (1x unique)
  Fluffy the spider

  Example user agent
  Mozilla/3.0 (compatible; Fluffy the spider; http://www.searchhippo.com/; info@searchhippo.com)

• 2x detected (2x unique)
  Cityreview Robot,cityreview

  Example user agent
  Cityreview Robot (+http://www.cityreview.org/crawler/)

• 2x detected (1x unique)
  Lycos

  Example user agent
  Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; LYCOSA;+http://lycosa.se)

• 4x detected (2x unique)
  discobot,discobot/1.0

  Example user agent
  Mozilla/5.0 (compatible; discobot/1.0; +http://discoveryengine.com/discobot.html)

• 1x detected (1x unique)
  Iframely Bot

  Example user agent
  Iframely/0.7.1 (+http://iframe.ly/;)

• 3x detected (2x unique)
  yrspider,Yunyun Bot

  Example user agent
  Mozilla/5.0 (Android 3.0; YRSpider; +http://www.yunrang.com/yrspider.html)

• 3x detected (1x unique)
  Superfeedr Bot

  Example user agent
  Superfeedr bot/2.0 http://superfeedr.com - Make your feeds realtime: get in touch

• 4x detected (4x unique)
  GooglePlusBot,Google Web Snippet,Googlebot,Googlebot snippet

  Example user agent
  Mozilla/5.0 (Windows NT 6.1; rv:6.0) Gecko/20110814 Firefox/6.0 Google (+https://developers.google.com/+/web/snippet/)

• 3x detected (2x unique)
  Semrush Bot,SemrushBot

  Example user agent
  Mozilla/5.0 (compatible; SemrushBot/0.97sa; +http://www.semrush.com/bot.html)

• 4x detected (1x unique)
  Gigabot

  Example user agent
  Gigabot/2.0/gigablast.com/spider.html

• 1x detected (1x unique)
  Sangfor Spider

  Example user agent
  Sangfor Spider

• 2x detected (1x unique)
  Microsoft Bing Mobile SocialStreams Bot

  Example user agent
  Microsoft Bing Mobile SocialStreams Bot

• 1x detected (1x unique)
  URLfilterDB-crawler

  Example user agent
  Mozilla/5.0 (compatible; Linux i686; en-US; URLfilterDB-crawler/1.0) ufdb/1.0

• 3x detected (2x unique)
  Sogou Spider,New-Sogou-Spider

  Example user agent
  New-Sogou-Spider/1.0 (compatible; MSIE 5.5; Windows 98)

• 2x detected (2x unique)
  robot,Werbefreie Deutsche Suchmaschine

  Example user agent
  Mozilla/5.0 (compatible; DeuSu/0.1.0; +https://deusu.de/robot.html)

• 3x detected (1x unique)
  woriobot

  Example user agent
  Mozilla/5.0 (compatible; woriobot support [at] zite [dot] com +http://zite.com)

• 1x detected (1x unique)
  Kyoto-Crawler

  Example user agent
  Kyoto-Crawler/2.0 (Mozilla-compatible; kyoto-crawler-contact(at)nlp(dot)kuee(dot)kyoto-u(dot)ac(dot)jp; http://nlp.ist.i.kyoto-u.ac.jp/)

• 4x detected (2x unique)
  Google Bot,Googlebot

  Example user agent
  Googlebot v2.1 (+http://www.google.com/bot.html) (http://www.openwebspider.org/)

• 1x detected (1x unique)
  nyobot

  Example user agent
  nyobot/1.1 (Noyb.com search engine crawler; http://www.noyb.com; bot@noyb.com),gzip(gfe),gzip(gfe)

• 1x detected (1x unique)
  tivraSpider

  Example user agent
  tivraSpider/1.0 (crawler@tivra.com)

• 1x detected (1x unique)
  PicSpider

  Example user agent
  PicSpider/1.1 (spider@217-20-118-26.internetserviceteam.com; http://www.bildkiste.de)

• 1x detected (1x unique)
  jrCrawler

  Example user agent
  jrCrawler/1.0b

• 2x detected (1x unique)
  JoobleBot

  Example user agent
  JoobleBot (compatible; http://jooble.org/jooble-bot)

• 2x detected (1x unique)
  MSR-ISRCCrawler

  Example user agent
  MSR-ISRCCrawler

• 3x detected (3x unique)
  Nutch,Nutch-based Bot,Nutch; http:

  Example user agent
  SapphireWebCrawler/1.0 (Sapphire Web Crawler using Nutch; http://boston.lti.cs.cmu.edu/crawler/; mhoy@cs.cmu.edu)

• 3x detected (2x unique)
  Shim-Crawler,Shim Crawler

  Example user agent
  Shim-Crawler

• 1x detected (1x unique)
  HiveCrawler

  Example user agent
  Mozilla/5.0 (compatible; HiveCrawler/1.2 http://www.businessinsider.com)

• 3x detected (1x unique)
  MetaJobBot

  Example user agent
  Mozilla/5.0 (compatible; MetaJobBot; http://www.metajob.at/crawler)

• 1x detected (1x unique)
  cis455crawler

  Example user agent
  cis455crawler

• 1x detected (1x unique)
  PipeLine Spider

  Example user agent
  pipeLiner/0.10 (PipeLine Spider; http://www.pipeline-search.com/webmaster.html)

• 2x detected (1x unique)
  WebAlta Crawler

  Example user agent
  WebAlta Crawler/1.3.18 (http://www.webalta.net/ru/about_webmaster.html) (Windows; U; Windows NT 5.1; ru-RU)

• 2x detected (1x unique)
  IstellaBot

  Example user agent
  Mozilla/5.0 (compatible; IstellaBot/1.18.81 +http://www.tiscali.it/)

• 2x detected (2x unique)
  Seznam Bot,Seznam Screenshot Generator

  Example user agent
  Mozilla/5.0 PhantomJS (compatible; Seznam screenshot-generator 2.1; +http://fulltext.sblog.cz/screenshot/)

• 2x detected (1x unique)
  memoryBot

  Example user agent
  Mozilla/5.0 (compatible; memorybot/1.20.71 +http://archivethe.net/en/index.php/about/internet_memory1 on behalf of DNB)

• 1x detected (1x unique)
  Liquida Spider

  Example user agent
  Liquida Spider 1.0 +http://liquida.com/

• 2x detected (2x unique)
  Jaxified Crawler,Jaxified

  Example user agent
  Jaxified Crawler 1.0a (+http://www.jaxified.com/),gzip(gfe),gzip(gfe)

• 3x detected (1x unique)
  ICC-Crawler

  Example user agent
  ICC-Crawler/2.0 (Mozilla-compatible; ; http://kc.nict.go.jp/project1/crawl.html; zurukko1552995316;)

• 1x detected (1x unique)
  LargeSmall Crawler

  Example user agent
  LargeSmall Crawler

• 3x detected (2x unique)
  suggybot,suggybot/0.01a

  Example user agent
  Mozilla/5.0 (compatible; suggybot v0.01a, http://blog.suggy.com/was-ist-suggy/suggy-webcrawler/)

• 1x detected (1x unique)
  WebMasterAid

  Example user agent
  Mozilla/5.0 (compatible; WebMasterAid/1.0; +http://wmaid.com/explore/)

• 3x detected (2x unique)
  200PleaseBot,200PleaseBot/1.0

  Example user agent
  Mozilla/5.0 (compatible; 200PleaseBot/1.0; +http://www.200please.com/bot)

• 1x detected (1x unique)
  agbot

  Example user agent
  agbot/1.0 (AgHaven.com search engine crawler; http://search.aghaven.com; bot@aghaven.com)

• 1x detected (1x unique)
  Feed Crawler

  Example user agent
  www2.apserver.net ASP-Ranker Feed Crawler

• 2x detected (1x unique)
  lbot

  Example user agent
  Mozilla/5.0 (lbot/2.6.4) (lbot@mailme.not)

• 2x detected (2x unique)
  Gazeta Bot,Yandex Bot

  Example user agent
  Mozilla/5.0 (compatible; Linux i686; Yandex.Gazeta Bot/1.0; +http://gazeta.yandex.ru)

• 2x detected (2x unique)
  Webbot.ru,Webbot

  Example user agent
  Mozilla/5.0 (compatible; Webbot/0.1; http://www.webbot.ru/bot.html)

• 3x detected (2x unique)
  Orangebot,Orange Bot

  Example user agent
  Mozilla/5.0 (compatible; OrangeBot/2.0; support.orangebot@orange.com)

• 2x detected (1x unique)
  LinkStats Bot

  Example user agent
  LinkStats Bot

• 3x detected (2x unique)
  Northern Light Web Search,NLCrawler

  Example user agent
  Mozilla/5.0 (compatible; NLCrawler/2.0.15; Linux 2.6.3-7; i686; en_US)KHTML/3.4.89 (like Gecko)

• 1x detected (1x unique)
  Web-Robot

  Example user agent
  Web-Robot/5.0 (en-US; web-robot.com/policy.html) Web-Robot Crawler/2.0.3

• 4x detected (4x unique)
  TinEye Crawler,TinEye-bot,TinEye Bot,TinEye

  Example user agent
  TinEye-bot/0.02 (see http://www.tineye.com/crawler.html)

• 5x detected (3x unique)
  SeoMoz rogerbot,rogerbot,rogerbot/1.0

  Example user agent
  rogerbot/1.0 (http://www.seomoz.org/dp/rogerbot, rogerbot-crawler@seomoz.org)

• 1x detected (1x unique)
  Top10Ranking Spider

  Example user agent
  Top10Ranking Spider/3.1 ( http://www.top10Ranking.nl/, Top10ranking.nl heeft op een aantal woorden uw posities in Google gecheckt)

• 3x detected (2x unique)
  rogerbot/1.0,Rogerbot

  Example user agent
  Mozilla/5.0 (compatible; rogerBot/1.0; UrlCrawler; http://www.seomoz.org/dp/rogerbot)

• 4x detected (2x unique)
  CareerBot,CareerBot/1.1

  Example user agent
  Mozilla/5.0 (compatible; CareerBot/1.1; +http://www.career-x.de/bot.html)

• 1x detected (1x unique)
  SlugBug Spider

  Example user agent
  SlugBug Spider/0.1 beta (SlugBug.com search engine; http://www.slugbug.com)

• 3x detected (2x unique)
  TweetmemeBot,Tweetmeme Bot

  Example user agent
  Mozilla/5.0 (TweetmemeBot/4.0; +http://datasift.com/bot.html) Gecko/20100101 Firefox/31.0

• 4x detected (3x unique)
  CazoodleBot,CazoodleBot b,MQBOT

  Example user agent
  CazoodleBot/0.1 (CazoodleBot Crawler; http://www.cazoodle.com; mqbot@cazoodle.com)

• 1x detected (1x unique)
  Inspyder-Crawler

  Example user agent
  Mozilla/4.0 (compatible; Inspyder-Crawler; http://www.inspyder.com)

• 1x detected (1x unique)
  lb-spider

  Example user agent
  lb-spider

• 1x detected (1x unique)
  HPI FeedCrawler

  Example user agent
  HPI FeedCrawler/0.1 (+http://www.hpi.uni-potsdam.de/meinel/bross/feedcrawler)

• 1x detected (1x unique)
  Brekiri crawler

  Example user agent
  Brekiri crawler/1.0

• 2x detected (2x unique)
  Ilse,INGRID

  Example user agent
  INGRID/2.0 (http://spsearch.ilse.nl/; Startpagina dochter links spider)

• 5x detected (3x unique)
  SeznamBot,Seznam Bot,SeznamBot/3.1-test

  Example user agent
  Mozilla/5.0 (compatible; SeznamBot/3.1-test1; +http://fulltext.sblog.cz/)

• 1x detected (1x unique)
  PArchiveCrawler

  Example user agent
  PArchiveCrawler/1.0

• 5x detected (4x unique)
  Naver Yeti,TestCrawler,Yeti/Naverbot,Yeti

  Example user agent
  Yeti-FeedItemCrawler/1.0 (NHN Corp.;+http://help.naver.com/robots/)

• 1x detected (1x unique)
  dCrawlBot

  Example user agent
  dCrawlBot/1.0.1120

• 1x detected (1x unique)
  Searchspider

  Example user agent
  Searchspider/1.2 (SearchSpider; http://www.searchspider.com; webmaster@searchspider.com)

• 5x detected (3x unique)
  BingBot,MSRBOT,MSRBOT c

  Example user agent
  MSRBOT (http://research.microsoft.com/research/sv/msrbot)

• 1x detected (1x unique)
  Thesis_php_crawler

  Example user agent
  Thesis_php_crawler

• 2x detected (1x unique)
  Vagabondo

  Example user agent
  Mozilla/5.0 (compatible; Vagabondo/Wapspider; webcrawler at wise-guys dot nl; http://webagent.wise-guys.nl/)

• 1x detected (1x unique)
  Medialab Spider

  Example user agent
  Mozilla/4.0 (compatible); MSIE 5.0; Medialab Spider

• 5x detected (3x unique)
  rogerbot/1.0,rogerbot,SeoMoz rogerbot

  Example user agent
  rogerbot/1.0 (http://www.seomoz.org/dp/rogerbot, rogerbot-crawler+shiny@seomoz.org)

• 2x detected (2x unique)
  Slackbot-Link-Expanding,Slackbot

  Example user agent
  Slackbot-LinkExpanding (+https://api.slack.com/robots)

• 3x detected (3x unique)
  Mail.Ru,Mail.RU_Bot,Mail.Ru Bot

  Example user agent
  Mozilla/5.0 (compatible; Linux x86_64; Mail.RU_Bot/Robots; +http://go.mail.ru/help/robots)

• 2x detected (1x unique)
  AOLbot

  Example user agent
  Mozilla/5.0 (compatible; AOLbot/4.0; +http://www.aol-soft.com/)

• 1x detected (1x unique)
  favicon_archive

  Example user agent
  FaviconArchiver/1.0 (+http://moblur.org/workshop/favicon_archive/)

• 4x detected (2x unique)
  YodaoBot,YodaoBot/1.0

  Example user agent
  Mozilla/5.0 (compatible; YodaoBot/1.0; http://www.yodao.com/help/webmaster/spider/; )

• 1x detected (1x unique)
  Apexoo Spider

  Example user agent
  Apexoo Spider (http://www.apexoo.com/spider/)

• 2x detected (2x unique)
  Sosoimagespider,Soso Spider

  Example user agent
  Mozilla/5.0 (compatible; Sosoimagespider/2.0; +http://help.soso.com/soso-image-spider.htm)

• 1x detected (1x unique)
  CSimpleSpider

  Example user agent
  CSimpleSpider Robot

• 2x detected (1x unique)
  nbot

  Example user agent
  Mozilla/5.0 (compatible; nbot/2.0)

• 3x detected (2x unique)
  BabalooSpider,BabalooSpider/1.3

  Example user agent
  BabalooSpider/1.3 (BabalooSpider; http://www.babaloo.si; spider@babaloo.si)

• 2x detected (2x unique)
  007AC9 Crawler,Crawler

  Example user agent
  Mozilla/5.0 (compatible; 007ac9 Crawler; http://crawler.007ac9.net/)

• 2x detected (2x unique)
  Coccoc bot,coccoc/1.0

  Example user agent
  Mozilla/5.0 (compatible; coccoc/1.0; +http://help.coccoc.com/)

• 2x detected (2x unique)
  Yunyun Bot,YRSpider

  Example user agent
  Mozilla/5.0 (compatible; YRSpider; +http://www.yunyun.com/SiteInfo.php?r=about)

• 1x detected (1x unique)
  FreeWebMonitoring SiteChecker

  Example user agent
  FreeWebMonitoring SiteChecker/0.1 (+http://www.freewebmonitoring.com)

• 2x detected (2x unique)
  SNK Screenshot Bot,Save n Keep Screenshot Bot

  Example user agent
  SNK Screenshot Bot/0.20 ( abuse at savenkeep dot_com )

• 4x detected (2x unique)
  ZumBot,ZumBot/1.0

  Example user agent
  ZumBot/1.0 (ZUM Search; http://help.zum.com/inquiry)

• 3x detected (2x unique)
  oBot/2.3.1,oBot

  Example user agent
  Mozilla/5.0 (compatible; oBot/2.3.1; +http://filterdb.iss.net/crawler/)

• 3x detected (1x unique)
  OpenWebSpider

  Example user agent
  OpenWebSpider/0.6 (http://www.openwebspider.org)

• 2x detected (1x unique)
  HuaweiSymantecSpider

  Example user agent
  HuaweiSymantecSpider/1.0 DSE-support@huaweisymantec.com (compatible; MSIE 7.0; Windows NT 5.1; Trident/4.0; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR ; http://www.huaweisymantec.com/cn/IRL/spider)

• 2x detected (2x unique)
  Lipperhey SEO Service,Lipperhey Spider

  Example user agent
  Mozilla/5.0 (compatible; Lipperhey Spider; http://www.lipperhey.com/)

• 1x detected (1x unique)
  SearchSpider

  Example user agent
  SearchSpider/1.2.10

• 1x detected (1x unique)
  Noobot

  Example user agent
  noobot Spider/Noobot-1.2 (noobot-bot; http://www.noobot.fr)

• 2x detected (1x unique)
  seebot

  Example user agent
  seebot/2.0 (+http://www.seegnify.com/bot)

• 3x detected (2x unique)
  GrapeshotCrawler,GrapeshotCrawler/2.0

  Example user agent
  Mozilla/5.0 (compatible; GrapeshotCrawler/2.0; +http://www.grapeshot.co.uk/crawler.php)

• 1x detected (1x unique)
  BlinkaCrawler

  Example user agent
  Mozilla/5.0 (compatible; BlinkaCrawler/1.0; http://www.blinka.jp/crawler/)

• 1x detected (1x unique)
  Hailoobot

  Example user agent
  Mozilla/5.0 (compatible; Hailoobot/1.2; http://www.hailoo.com/spider.html)

• 2x detected (2x unique)
  CloudServerMarketSpider,CloudServerMarketSpider/1.0

  Example user agent
  Mozilla/5.0 (compatible; CloudServerMarketSpider/1.0; +http://www.cloudservermarket.com/spider.html)

• 3x detected (3x unique)
  WI Job Roboter,Job Roboter Spider 3,Job Roboter

  Example user agent
  Mozilla/5.0 (compatible;WI Job Roboter Spider Version 3;+http://www.webintegration.at)

• 2x detected (2x unique)
  URLfilterDB Crawler,URLfilterDB-crawler

  Example user agent
  Mozilla/5.0 (URLfilterDB-crawler/1.1) ufdb/1.0

• 1x detected (1x unique)
  MPICrawler

  Example user agent
  MPICrawler/0.1 +http://kermit.news.cs.nyu.edu/crawler.html,gzip(gfe),gzip(gfe)

• 1x detected (1x unique)
  EverbeeCrawler

  Example user agent
  EverbeeCrawler

• 2x detected (2x unique)
  YowedoBot/1.0,Yowedo

  Example user agent
  YowedoBot/Yowedo 1.0 (Search Engine crawler for yowedo.com; http://yowedo.com/en/partners.html; crawler@yowedo.com)

• 2x detected (2x unique)
  EuripBot,Europe Internet Portal

  Example user agent
  EuripBot/0.5 (+http://www.eurip.com) PreCheck

• 3x detected (3x unique)
  Mail.Ru,Mail.RU_Bot,Mail.Ru Bot

  Example user agent
  Mozilla/5.0 (compatible; Linux x86_64; Mail.RU_Bot/Fast/2.0; +http://go.mail.ru/help/robots)

• 1x detected (1x unique)
  ExactSeekCrawler

  Example user agent
  ExactSeekCrawler/1.0

• 1x detected (1x unique)
  8bo Crawler Bot

  Example user agent
  8bo Crawler Bot

• 2x detected (1x unique)
  oBot

  Example user agent
  Mozilla/5.0 (compatible; oBot/2.3.1; http://filterdb.iss.net/crawler/)

• 1x detected (1x unique)
  SuperSpider

  Example user agent
  Mozilla/4.0 (compatible; MSIE 5.5; SuperSpider/139; Windows 98; Win 9x 4.90)

• 3x detected (2x unique)
  YoudaoBot,Youdao Bot

  Example user agent
  Mozilla/5.0 (compatible; YoudaoBot-rts/1.0; http://www.youdao.com/help/webmaster/spider/; )

• 1x detected (1x unique)
  BIXOCRAWLER

  Example user agent
  Mozilla/5.0 (compatible; BIXOCRAWLER; +http://wiki.github.com/bixo/bixo/bixocrawler; bixo-dev@yahoogroups.com)

• 2x detected (1x unique)
  LinkAider

  Example user agent
  LinkAider (http://linkaider.com/crawler/)

• 1x detected (1x unique)
  EDI

  Example user agent
  EDI/1.6.0 (Edacious & Intelligent Web Crawler)

• 3x detected (1x unique)
  Superfeedr Bot

  Example user agent
  Superfeedr bot/2.0 http://superfeedr.com - Make your feeds realtime: get in touch!

• 2x detected (1x unique)
  OmniExplorer_Bot

  Example user agent
  OmniExplorer_Bot/6.70 ( http://www.omni-explorer.com) WorldIndexer

• 3x detected (2x unique)
  CazoodleBot-0.1,CazoodleBot

  Example user agent
  CazoodleBot/CazoodleBot-0.1 (CazoodleBot Crawler; http://www.cazoodle.com/cazoodlebot; cazoodlebot@cazoodle.com)

• 2x detected (1x unique)
  ConveraMultiMediaCrawler

  Example user agent
  ConveraMultiMediaCrawler/0.1 ( http://www.authoritativeweb.com/crawl)

• 1x detected (1x unique)
  BurstFindCrawler

  Example user agent
  BurstFindCrawler/1.1 (crawler.burstfind.com; http://crawler.burstfind.com; crawler@burstfind.com)

• 1x detected (1x unique)
  Prlog

  Example user agent
  Mozilla/5.0 (compatible; Prlog/1.0; +http://prlog.ru/)

• 2x detected (1x unique)
  zspider

  Example user agent
  Zspider (+http://www.zhanzhangsou.com/index.htm)

• 3x detected (2x unique)
  SEOkicks Robot,SEOkicks-Robot

  Example user agent
  Mozilla/5.0 (compatible; SEOkicks-Robot +http://www.seokicks.de/robot.html)

• 1x detected (1x unique)
  deepcrawler

  Example user agent
  deepcrawler/3.1 (http://www.queusearch.com/whatis_deepcrawler.php),gzip(gfe),gzip(gfe)

• 2x detected (2x unique)
  Mozilla crawl,Mozilla Crawler

  Example user agent
  Mozilla crawl/5.0 (compatible; fairshare.cc +http://fairshare.cc)

• 2x detected (1x unique)
  MerchantCentricBot

  Example user agent
  Mozilla/5.0 (compatible; MerchantCentricBot/1.0)

• 1x detected (1x unique)
  persomm-spider

  Example user agent
  persomm-spider/v1.0

• 4x detected (3x unique)
  MJ12bot,MJ12bot/v1.4.4,MJ12 Bot

  Example user agent
  Mozilla/5.0 (compatible; MJ12bot/v1.4.4; http://www.majestic12.co.uk/bot.php?+)

• 1x detected (1x unique)
  Wocodi Web Crawler

  Example user agent
  WocBot/Mozilla/5.0 (Wocodi Web Crawler 1.0; http://www.wocodi.com/crawler; crawler@wocodi.com)

• 3x detected (2x unique)
  NE Crawler,netEstate NE Crawler

  Example user agent
  netEstate NE Crawler (+http://www.website-datenbank.de/)

• 3x detected (3x unique)
  IIS Site Analysis,IIS Site Analysis Web Crawler,iisbot

  Example user agent
  iisbot/1.0 (+http://www.iis.net/iisbot.html)

• 1x detected (1x unique)
  Comodo HTTP(S) Crawler

  Example user agent
  Comodo HTTP(S) Crawler - http://www.instantssl.com/crawler

• 1x detected (1x unique)
  lemurwebcrawler

  Example user agent
  Mozilla/5.0 (compatible; lemurwebcrawler admin@lemurproject.org; +http://boston.lti.cs.cmu.edu/crawler_12/)

• 4x detected (1x unique)
  TweetedTimes Bot

  Example user agent
  Mozilla/5.0 (compatible; TweetedTimes Bot/1.0; +http://tweetedtimes.com)

• 2x detected (2x unique)
  com WebBot,EmeraldShield.com WebBot

  Example user agent
  EmeraldShield.com WebBot

• 1x detected (1x unique)
  Student Spider

  Example user agent
  PKU Student Spider

• 4x detected (2x unique)
  Sogou Pic Spider,Sogou Spider

  Example user agent
  Sogou Pic Spider/3.0(+http://www.sogou.com/docs/help/webmasters.htm

• 4x detected (2x unique)
  WeSEE:Search/0.1 (Alpha),WeSEE:Search

  Example user agent
  WeSEE:Search/0.1 (Alpha, http://www.wesee.com/en/support/bot/)

• 2x detected (1x unique)
  Socialradarbot

  Example user agent
  Mozilla/5.0 (compatible; Linux; Socialradarbot/2.0; en-US; bot@infegy.com)

• 3x detected (2x unique)
  Peew,Peew/1.0

  Example user agent
  Mozilla/5.0 (compatible; Peew/1.0; http://www.peew.de/crawler/)

• 2x detected (1x unique)
  GIDBot

  Example user agent
  GIDBot/2.0 (+http://www.gidnetwork.com/tools/gzip-test.php)

• 1x detected (1x unique)
  PathDefender

  Example user agent
  Mozilla/5.0 (compatible; PathDefender/1.0; +http://www.pathdefender.com/help/crawler)

• 1x detected (1x unique)
  SiteExplorer

  Example user agent
  Mozilla/5.0 (compatible; SiteExplorer/1.0b; +http://SiteExplorer.Info/)

• 3x detected (2x unique)
  Zao-Crawler,Zao

  Example user agent
  Zao-Crawler

• 1x detected (1x unique)
  Tasapspider

  Example user agent
  Tasapspider/0.9

• 1x detected (1x unique)
  SimpleCrawler

  Example user agent
  SimpleCrawler/0.1

• 2x detected (1x unique)
  memoryBot

  Example user agent
  Mozilla/5.0 (compatible; memoryBot/1.20.235 +http://internetmemory.org/en/)

• 2x detected (1x unique)
  DomainSigmaCrawler

  Example user agent
  Mozilla/5.0 (compatible; DomainSigmaCrawler/0.1; +http://domainsigma.com/robot)

• 3x detected (2x unique)
  Sogou-Test-Spider,Sogou-Test-Spider/4.0

  Example user agent
  Sogou-Test-Spider/4.0 (compatible; MSIE 5.5; Windows 98)

• 1x detected (1x unique)
  SiteExplorer

  Example user agent
  Mozilla/5.0 (compatible; SiteExplorer/1.0b; +http://siteexplorer.info/)

• 2x detected (1x unique)
  GarlikCrawler

  Example user agent
  GarlikCrawler/1.2 (http://garlik.com/, crawler@garlik.com)

• 1x detected (1x unique)
  SAS Crawler

  Example user agent
  Teragram/SAS Crawler

• 2x detected (1x unique)
  ScollSpider

  Example user agent
  ScollSpider/2.0 ( http://www.webwobot.com/ScollSpider.php)

• 2x detected (2x unique)
  DomainScan Server Monitoring,robot

  Example user agent
  Mozilla/5.0 (compatible; DomainSCAN/2.0; +http://domainscan.info/robot/)

• 2x detected (1x unique)
  Dazoobot

  Example user agent
  Mozilla/5.0 (compatible; Dazoobot/0.1; +http://dazoo.fr)

• 1x detected (1x unique)
  TECOMAC-Crawler

  Example user agent
  TECOMAC-Crawler/0.3

• 1x detected (1x unique)
  1470.net crawler

  Example user agent
  1470.net crawler

• 3x detected (1x unique)
  PiplBot

  Example user agent
  Mozilla/5.0+(compatible;+PiplBot;++http://www.pipl.com/bot/)

• 1x detected (1x unique)
  Superarama.com - BOT

  Example user agent
  Superarama.com - BOT/v.0.1

• 2x detected (2x unique)
  mixi-news-crawler,mixi

  Example user agent
  mixi-news-crawler/1.00 (http://mixi.jp/)

• 1x detected (1x unique)
  itsscan

  Example user agent
  Mozilla/5.0 (compatible; Muenster University of Applied Sciences; +http://fb02itsscan.fh-muenster.de)

• 1x detected (1x unique)
  Punk Spider

  Example user agent
  Punk Spider/PunkSPIDER-v0.1

• 2x detected (2x unique)
  ru Screenshot Bot,Screenshot Bot

  Example user agent
  pr-cy.ru Screenshot Bot

• 2x detected (1x unique)
  DBot

  Example user agent
  Mozilla/5.1 (compatible; DBot/7.0; +http://getdownload.ws)

• 2x detected (2x unique)
  MetamojiCrawler/1.0,MetamojiCrawler

  Example user agent
  Mozilla/5.0 (compatible; MetamojiCrawler/1.0; +http://www.metamoji.com/jp/crawler.html

• 3x detected (2x unique)
  Peeplo Screenshot Bot,Peeplo Screenshot Bot/0.20

  Example user agent
  Peeplo Screenshot Bot/0.20 ( abuse at peeplo dot_com )

• 2x detected (1x unique)
  pub-crawler

  Example user agent
  Mozilla/5.0 (compatible; pub-crawler; +http://wiki.github.com/bixo/bixo/bixocrawler; bixo-dev@yahoogroups.com)

• 2x detected (1x unique)
  YisouSpider

  Example user agent
  YisouSpider

• 1x detected (1x unique)
  AcquiloSpider

  Example user agent
  AcquiloSpider/1.0,gzip(gfe),gzip(gfe)

• 1x detected (1x unique)
  blog_crawler

  Example user agent
  blog_crawler/1.0

• 2x detected (1x unique)
  Scopia Crawler

  Example user agent
  Mozilla/5.0 (compatible; Scopia Crawler 1.2; +http://www.scopia.co)

• 3x detected (2x unique)
  Easou Spider,EasouSpider

  Example user agent
  Mozilla/5.0 (compatible; EasouSpider; +http://www.easou.com/search/spider.html)

• 2x detected (2x unique)
  SWEBot/1.0,SWEBot

  Example user agent
  Mozilla/5.0 (compatible; SWEBot/1.0; +http://swebot-crawler.net)

• 1x detected (1x unique)
  CheckSite Verification Agent

  Example user agent
  CheckSite Verification Agent (+http://www.checksite.us)

• 2x detected (2x unique)
  EasouSpider,Easou Spider

  Example user agent
  EasouSpider

• 2x detected (2x unique)
  LDSpider,lworldspider

  Example user agent
  lworldspider

• 1x detected (1x unique)
  Altresium

  Example user agent
  Altresium/4.6 (+http://www.altresium.com/bot.html)

• 1x detected (1x unique)
  T3census-Crawler

  Example user agent
  T3census-Crawler/1.0

• 1x detected (1x unique)
  FlaxCrawler

  Example user agent
  FlaxCrawler/1.0,gzip(gfe),gzip(gfe)

• 2x detected (2x unique)
  Orgbybot,OrgbyBot/1.2

  Example user agent
  Orgbybot/OrgbyBot v1.2 (Spidering the net for Orgby; http://www.orgby.com/ ; Orgby.com Search Engine)

• 1x detected (1x unique)
  Lightningspider

  Example user agent
  Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7) Gecko/20040707 Lightningspider/0.9.2

• 4x detected (2x unique)
  ICC-Crawler/2.0,ICC-Crawler

  Example user agent
  ICC-Crawler/2.0 (Mozilla-compatible; ; http://kc.nict.go.jp/project1/crawl.html)

• 4x detected (3x unique)
  BaiDu,BaiDuSpider,Baidu Spider

  Example user agent
  BaiDuSpider

• 2x detected (2x unique)
  Automattic Analytics Crawler/0.1,Automattic Analytics Crawler

  Example user agent
  Automattic Analytics Crawler/0.1; http://wordpress.com/crawler/

• 2x detected (2x unique)
  SensikaBot,Sensika Bot

  Example user agent
  SensikaBot/x.33 (+http://sensika.com)

• 4x detected (3x unique)
  MJ12bot/v1.4.0,MJ12bot,MJ12 Bot

  Example user agent
  Mozilla/5.0 (compatible; MJ12bot/v1.4.0; http://www.majestic12.co.uk/bot.php?+)

• 1x detected (1x unique)
  PaxleFramework

  Example user agent
  Mozilla/5.0 (compatible; PaxleFramework/0.1.1; http://www.paxle.net/en/bot)

• 4x detected (2x unique)
  Sogou Pic Spider,Sogou Spider

  Example user agent
  Sogou Pic Spider/3.0(+http://www.sogou.com/docs/help/webmasters.htm#07)

• 4x detected (3x unique)
  BLEXBot Crawler,BLEXBot/1.0,BLEXBot

  Example user agent
  Mozilla/5.0 (compatible; BLEXBot/1.0; +http://webmeup.com/crawler.html)

• 3x detected (2x unique)
  spider,proximic

  Example user agent
  Mozilla/5.0 (compatible; proximic; +http://www.proximic.com/info/spider.php)

• 1x detected (1x unique)
  Yet-Another-Spider

  Example user agent
  Mozilla/4.0 (compatible; Yet-Another-Spider; )

• 0x detected (0x unique)

  Example user agent
  Mozilla/5.0 (Linux; U; Android 4.2.2; it-it; S222 Build/CUBOT) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30

• 1x detected (1x unique)
  PHP Server Monitor

  Example user agent
  Mozilla/5.0 (compatible; phpservermon/3.1.1; +http://www.phpservermonitor.org)

• 2x detected (2x unique)
  VisBot,Visbot/2.0

  Example user agent
  VisBot/2.0 (Visvo.com Crawler; http://www.visvo.com/bot.html; bot@visvo.com)

• 2x detected (2x unique)
  Webspider,RankFlex

  Example user agent
  RankFlex.com Webspider

• 3x detected (1x unique)
  Screaming Frog SEO Spider

  Example user agent
  Screaming Frog SEO Spider/1.10

• 2x detected (2x unique)
  Lijit,Lijit Crawler

  Example user agent
  Lijit Crawler (+http://www.lijit.com/robot/crawler)

• 1x detected (1x unique)
  WSDLSpider

  Example user agent
  WSDLSpider/1.0 (http://www.wsdlworld.com)

• 1x detected (1x unique)
  VinjaVideoSpider

  Example user agent
  VinjaVideoSpider/1.1

• 1x detected (1x unique)
  WebCompanyCrawler

  Example user agent
  WebCompanyCrawler

• 2x detected (1x unique)
  Digincore bot

  Example user agent
  Mozilla/5.0 (compatible; Digincore bot; https://www.digincore.com/crawler.html for rules and instructions.

• 2x detected (2x unique)
  stq_bot,Searchteq Bot

  Example user agent
  stq_bot (+http://www.searchteq.de)

• 2x detected (2x unique)
  ExB Language Crawler 2.1.5,ExB Language Crawler

  Example user agent
  ExB Language Crawler 2.1.5 (+http://www.exb.de/crawler)

• 5x detected (2x unique)
  magpie-crawler,magpie-crawler/1.1

  Example user agent
  magpie-crawler/1.1 (U; Linux amd64; en-GB; +http://www.brandwatch.net)

• 4x detected (2x unique)
  ZumBot,ZumBot/1.0

  Example user agent
  Mozilla/5.0 (compatible; ZumBot/1.0; http://help.zum.com/inquiry)

• 1x detected (1x unique)
  StreamScraper

  Example user agent
  Mozilla/5.0 (compatible; StreamScraper/1.0; +http://code.google.com/p/streamscraper/)

• 3x detected (2x unique)
  robot,Apercite

  Example user agent
  Mozilla/5.0 (compatible; Apercite; +http://www.apercite.fr/robot/index.html)

• 1x detected (1x unique)
  Votay bot

  Example user agent
  Mozilla/5.0 (Votay bot/4.0; http://www.votay.com/arts/comics/)

• 1x detected (1x unique)
  NodejsSpider

  Example user agent
  NodejsSpider/1.0

• 3x detected (2x unique)
  Sogou Develop Spider,sogou spider

  Example user agent
  Sogou develop spider/4.0( http://www.sogou.com/docs/help/webmasters.htm#07)

• 1x detected (1x unique)
  RSpider

  Example user agent
  RSpider/1.0

• 2x detected (1x unique)
  Simpy

  Example user agent
  Simpy/1.1 (Simpy; http://www.simpy.com/?ref=bot; feedback at simpy dot com)

• 2x detected (1x unique)
  memoryBot

  Example user agent
  Mozilla/5.0 (compatible; memorybot/1.20.207 +http://mignify.com/bot.html)

• 4x detected (2x unique)
  Seznam Bot,SeznamBot

  Example user agent
  Mozilla/5.0 (compatible; SeznamBot/3.2; +http://fulltext.sblog.cz/)

• 1x detected (1x unique)
  Wireless Crawler

  Example user agent
  EricssonR320/R1A (Fast Wireless Crawler)

• 3x detected (2x unique)
  BLEXBot,BLEXBot Crawler

  Example user agent
  Mozilla/5.0 (compatible; BLEXBot/1.0; +http://webmeup-crawler.com/)

• 2x detected (2x unique)
  Kalooga,KaloogaBot

  Example user agent
  kalooga/KaloogaBot (Kalooga; http://www.kalooga.com/info.html?page=crawler)

• 2x detected (2x unique)
  AboutUsBot,AboutUs Bot

  Example user agent
  AboutUsBot/Harpy (Website Analysis; http://www.aboutus.org/Aboutus:Bot; help@aboutus.org)

• 3x detected (2x unique)
  furlbot,FurlBot/Furl Search 2.0

  Example user agent
  Mozilla/4.0 compatible FurlBot/Furl Search 2.0 (FurlBot; http://www.furl.net; wn.furlbot@looksmart.net)

• 1x detected (1x unique)
  Iplexx Spider

  Example user agent
  Mozilla/4.0 (compatible; Iplexx Spider/1.1)

• 1x detected (1x unique)
  DRKSpider

  Example user agent
  DRKSpider - Website link validator - http://www.drk.com.ar/spider/,gzip(gfe),gzip(gfe),gzip(gfe)

• 2x detected (2x unique)
  Phantom.js bot,js bot

  Example user agent
  Phantom.js bot

• 1x detected (1x unique)
  Embedly

  Example user agent
  Mozilla/5.0 (compatible; Embedly/0.2; snap; +http://support.embed.ly/)

• 1x detected (1x unique)
  Yahoo! Japan

  Example user agent
  Mozilla/5.0 (iPhone; Y!J-BRY/YATSH crawler; http://help.yahoo.co.jp/help/jp/search/indexing/indexing-15.html)

• 4x detected (1x unique)
  TurnitinBot

  Example user agent
  TurnitinBot/3.0 (http://www.turnitin.com/robot/crawlerinfo.html)

• 1x detected (1x unique)
  ZillaCrawler

  Example user agent
  ZillaCrawler

• 2x detected (2x unique)
  SputnikBot,Sputnik Bot

  Example user agent
  Mozilla/5.0 (compatible; SputnikBot/2.3; +http://corp.sputnik.ru/webmaster)

• 4x detected (3x unique)
  Alexabot,Alexa Crawler,Exabot

  Example user agent
  Mozilla/5.0 (compatible; Alexabot/1.0; +http://www.alexa.com/help/certifyscan; certifyscan@alexa.com)

• 2x detected (2x unique)
  TestCrawler,MultiCrawler

  Example user agent
  MultiCrawler, http://sw.deri.org/2006/04/multicrawler/robots.html

• 4x detected (3x unique)
  PaperLiBot/2.1,PaperLiBot,Paper.li Bot

  Example user agent
  Mozilla/5.0 (compatible; PaperLiBot/2.1; http://support.paper.li/entries/20023257-what-is-paper-li)

• 2x detected (2x unique)
  Livelap Crawler,LivelapBot

  Example user agent
  LivelapBot/0.2 (http://site.livelap.com/crawler)

• 1x detected (1x unique)
  NationalDirectory-WebSpider

  Example user agent
  NationalDirectory-WebSpider/1.3

• 1x detected (1x unique)
  BebopBot

  Example user agent
  BebopBot/2.5.1 (compatible; media crawler V1; http://www.apassion4jazz.net/bebopbot.html;)

• 1x detected (1x unique)
  Autonomy Spider

  Example user agent
  Autonomy Spider

• 3x detected (2x unique)
  pmoz.info ODP link checker,doc/botinfo

  Example user agent
  Mozilla/5.0 (compatible; pmoz.info ODP link checker; +http://pmoz.info/doc/botinfo.htm)

• 2x detected (2x unique)
  GingerCrawler/1.0,GingerCrawler

  Example user agent
  GingerCrawler/1.0 (Language Assistant for Dyslexics; www.gingersoftware.com/crawler_agent.htm; support at ginger software dot com)

• 3x detected (3x unique)
  Ant-Nutch,Nutch,Nutch-based Bot

  Example user agent
  Ant/Ant-Nutch-1.1 (Ant Nutch Crawler; http://www.ant.com; crawler@ant.com)

• 1x detected (1x unique)
  Analyzer Crawler

  Example user agent
  www.website-analyzer.net Website Analyzer Crawler

• 3x detected (1x unique)
  TweetedTimes Bot

  Example user agent
  TweetedTimes Bot/1.0 (Mozilla/5.0 Compatible, +http://tweetedtimes.com)

• 2x detected (2x unique)
  Arachnida Web Crawler,Web Crawler

  Example user agent
  Arachnida Web Crawler

• 1x detected (1x unique)
  Spider

  Example user agent
  Spider/5.0

• 1x detected (1x unique)
  IPv4Scan

  Example user agent
  IPv4Scan (+http://ipv4scan.com)

• 3x detected (2x unique)
  Steeler,Steeler/3.3

  Example user agent
  Steeler/3.3 (http://www.tkl.iis.u-tokyo.ac.jp/~crawler/)

• 1x detected (1x unique)
  NewzCrawler

  Example user agent
  NewzCrawler/1.9 (compatible; MSIE 6.00; Newz Crawler 1.9; http://www.newzcrawler.com/ )

• 3x detected (3x unique)
  ProjectWF-java-test-crawler,Java,General Crawlers

  Example user agent
  ProjectWF-java-test-crawler

• 2x detected (1x unique)
  TestCrawler

  Example user agent
  Mozilla/5.0 (compatible; TestCrawler)

• 1x detected (1x unique)
  kulturarw3

  Example user agent
  Mozilla/5.0 (compatible; kulturarw3 +http://www.kb.se/om/projekt/Svenska-webbsidor---Kulturarw3/)

• 1x detected (1x unique)
  YoonoCrawler

  Example user agent
  YoonoCrawler/1.0 (crawler@yoono.com)

• 2x detected (2x unique)
  Zeusbot,Zeus

  Example user agent
  Zeusbot/0.8.1 (Ulysseek's web-crawling robot; http://www.zeusbot.com; agent@zeusbot.com)

• 1x detected (1x unique)
  loc-crawler

  Example user agent
  Mozilla/5.0 (compatible; loc-crawler/0.11.0 +http://loc.gov),gzip(gfe),gzip(gfe),gzip(gfe)

• 1x detected (1x unique)
  Lipperhey SEO Service

  Example user agent
  Mozilla/5.0 (compatible; Lipperhey-Kaus-Australis/5.0; +https://www.lipperhey.com/en/about/)

• 1x detected (1x unique)
  StoneSunSpider

  Example user agent
  Mozilla/5.0 (compatible; StoneSunSpider/1.1)

• 2x detected (2x unique)
  PagesInventory Bot,PagesInventory

  Example user agent
  PagesInventory (robot +http://www.pagesinventory.com)

• 2x detected (1x unique)
  Netseer crawler

  Example user agent
  compatible; Netseer crawler/2.0; +http://www.netseer.com/crawler.html; crawler@netseer.com

• 1x detected (1x unique)
  RavenCrawler

  Example user agent
  RavenCrawler

• 2x detected (2x unique)
  Hatena Crawler,Hatena::Crawler

  Example user agent
  Hatena::Crawler/0.01

• 1x detected (1x unique)
  Arikus_Spider

  Example user agent
  Arikus_Spider

• 3x detected (2x unique)
  Orangebot,Orange Bot

  Example user agent
  Mozilla/5.0 (compatible; Orangebot/2.0; support.orangebot@orange.com)

• 2x detected (1x unique)
  OpenHoseBot

  Example user agent
  Mozilla/5.0 (compatible; OpenHoseBot/2.1; +http://www.openhose.org/bot.html)

• 3x detected (2x unique)
  Mixrank Bot,MixrankBot

  Example user agent
  Mozilla/5.0 (compatible; MixrankBot; crawler@mixrank.com)

• 0x detected (0x unique)

  Example user agent
  Nokia6680/1.0 ((4.04.07) SymbianOS/8.0 Series60/2.6 Profile/MIDP-2.0 Configuration/CLDC-1.1 (for mobile crawler) )

• 4x detected (3x unique)
  meanpathbot,meanpathbot/1.0,Meanpath Bot

  Example user agent
  Mozilla/5.0 (compatible; meanpathbot/1.0; +http://www.meanpath.com/meanpathbot.html)

• 2x detected (1x unique)
  SMTBot

  Example user agent
  Mozilla/5.0 (Windows NT 6.1) (compatible; SMTBot/1.0; +http://www.similartech.com/smtbot)

• 3x detected (2x unique)
  Lycos_Spider_,Lycos

  Example user agent
  Lycos_Spider_(modspider)

• 2x detected (1x unique)
  KomodiaBot

  Example user agent
  Mozilla/5.0 (Windows NT 6.1; Win64; x64) KomodiaBot/1.0

• 2x detected (1x unique)
  SMTBot

  Example user agent
  Mozilla/5.0 (compatible; SMTBot/1.0; +http://www.similartech.com/smtbot)

• 1x detected (1x unique)
  ExactSeek_Spider

  Example user agent
  ExactSeek_Spider

• 1x detected (1x unique)
  LEIA

  Example user agent
  LEIA/3.01pr (LEIAcrawler; leia@gseek.com; http://www.gseek.com)

• 2x detected (2x unique)
  Adbeat Bot,adbeat_bot

  Example user agent
  adbeat_bot

• 3x detected (2x unique)
  TweetmemeBot,Tweetmeme Bot

  Example user agent
  Mozilla/5.0 (compatible; TweetmemeBot/3.0; +http://tweetmeme.com/)

• 1x detected (1x unique)
  TouTrix crawler

  Example user agent
  TouTrix crawler/1.0

• 1x detected (1x unique)
  moba-crawler

  Example user agent
  DoCoMo/2.0 N902iS(c100;TB;W24H12)(compatible; moba-crawler; http://crawler.dena.jp/)

• 3x detected (1x unique)
  BingBot

  Example user agent
  bingbot

• 2x detected (1x unique)
  NLCrawler

  Example user agent
  nlcrawler/1.0 (+http://northernlight.com/)

• 2x detected (2x unique)
  NikiBot,niki-bot

  Example user agent
  niki-bot

• 1x detected (1x unique)
  Begun Advertising Bot

  Example user agent
  Mozilla/5.0 (compatible; BegunAdvertising/3.0; +http://begun.ru/begun/technology/indexer/)

• 5x detected (3x unique)
  Picsearch bot,psbot,psbot/0.1

  Example user agent
  psbot/0.1 (+http://www.picsearch.com/bot.html)

• 1x detected (1x unique)
  RyzeCrawler

  Example user agent
  RyzeCrawler/1.1.0 (+http://www.ryze.nl/crawler/)

• 3x detected (2x unique)
  Linkdex Bot,linkdexbot

  Example user agent
  Mozilla/5.0 (compatible; linkdexbot/2.0; +http://www.linkdex.com/bots/)

• 4x detected (4x unique)
  Mail.RU_Bot,Mail.Ru,Mail.RU_Bot/2.0,Mail.Ru Bot

  Example user agent
  Mozilla/5.0 (compatible; Mail.RU_Bot/2.0; +http://go.mail.ru/help/robots)

• 4x detected (3x unique)
  IRLbot/3.0,IRLbot,Research Projects

  Example user agent
  IRLbot/3.0 (compatible; MSIE 6.0; http://irl.cs.tamu.edu/crawler)

• 4x detected (2x unique)
  psbot,Picsearch Bot

  Example user agent
  psbot-image (+http://www.picsearch.com/bot.html)

• 3x detected (1x unique)
  IRLbot

  Example user agent
  IRLbot/1.0 ( http://irl.cs.tamu.edu/crawler)

• 2x detected (1x unique)
  i1searchbot

  Example user agent
  i1searchbot/2.0 (i1search web crawler; http://www.i1search.com; crawler@i1search.com)

• 4x detected (1x unique)
  MojeekBot

  Example user agent
  Mozilla/5.0 (compatible; MojeekBot/0.6; +https://www.mojeek.com/bot.html)

• 1x detected (1x unique)
  clumboot

  Example user agent
  Mozilla/5.0 (compatible; clumboot; +http://localhost/clumpit/bot.php)

• 2x detected (1x unique)
  YRL_ODP_CRAWLER

  Example user agent
  YRL_ODP_CRAWLER

• 5x detected (3x unique)
  Speedy Spider,Entireweb,Speedy

  Example user agent
  Mozilla/5.0 (compatible; Speedy Spider; http://www.entireweb.com/about/search_tech/speedy_spider/)

• 1x detected (1x unique)
  search crawler

  Example user agent
  camoca/camoca-n.1.2 (super-agent; search crawler; info at does not exist dot com)

• 1x detected (1x unique)
  OpenCrawler

  Example user agent
  Mozilla/5.0 (compatible; OpenCrawler/0.1.6.6; http://code.google.com/p/opencrawler/),gzip(gfe),gzip(gfe)

• 1x detected (1x unique)
  Impressumscrawler

  Example user agent
  netEstate Impressumscrawler (+http://www.netestate.de/De/Loesungen/Impressumscrawler)

• 1x detected (1x unique)
  bmdspider

  Example user agent
  Mozilla/4.0 (compatible; bmdspider; windows 5.1)

• 4x detected (1x unique)
  magpie-crawler

  Example user agent
  magpie-crawler/1.1 (U; Linux amd64; en-GB; http://www.brandwatch.net)

• 1x detected (1x unique)
  Tailsweep

  Example user agent
  TailsweepBlogCrawler/Tailsweep-2.9-SNAPSHOT (http://www.tailsweep.com/; bot at [tailsweep] dot com)

• 1x detected (1x unique)
  picmole Bot

  Example user agent
  Mozilla/5.0 (compatible;picmole/1.0 +http://www.picmole.com)

• 1x detected (1x unique)
  Yahoo-Newscrawler

  Example user agent
  Yahoo-Newscrawler/3.9 (news-search-crawler at yahoo-inc dot com)

• 1x detected (1x unique)
  Engine Spider

  Example user agent
  ByWebSite Search Engine Spider

• 1x detected (1x unique)
  ProoXiBot

  Example user agent
  ProoXiBot

• 1x detected (1x unique)
  Comodo-Certificates-Spider

  Example user agent
  Comodo-Certificates-Spider

• 4x detected (2x unique)
  holmes/3.12.4 - morfeo,Holmes

  Example user agent
  holmes/3.12.4 (http://morfeo.centrum.cz/bot)

• 1x detected (1x unique)
  LB-Crawler

  Example user agent
  LB-Crawler/1.0

• 5x detected (3x unique)
  BingBot,msnbot,adidxbot

  Example user agent
  adidxbot/2.0 (+http://search.msn.com/msnbot.htm)

• 3x detected (2x unique)
  TweetmemeBot,Tweetmeme Bot

  Example user agent
  Mozilla/5.0 (compatible; TweetmemeBot/2.11; +http://tweetmeme.com/)

• 1x detected (1x unique)
  Baidu Mobile Search

  Example user agent
  Mozilla/5.0 (Linux;u;Android 2.3.7;zh-cn;) AppleWebKit/533.1 (KHTML,like Gecko) Version/4.0 Mobile Safari/533.1 (compatible; +http://www.baidu.com/search/spi_der.html)

• 1x detected (1x unique)
  ERACrawler

  Example user agent
  ERACrawler/1.0

• 1x detected (1x unique)
  PortalBSpider

  Example user agent
  PortalBSpider/2.0 (spider@portalb.com)

• 3x detected (2x unique)
  googlebot,Google Bot

  Example user agent
  mozilla/5.0 (iphone; cpu iphone os 8_3 like mac os x) applewebkit/600.1.4 (khtml, like gecko) version/8.0 mobile/12f70 safari/600.1.4 (compatible; googlebot/2.1; +http://www.google.com/bot.html)

• 2x detected (2x unique)
  chlooe Bot,bot-pge.chlooe.com/1.0.0

  Example user agent
  bot-pge.chlooe.com/1.0.0 (+http://www.chlooe.com/)

• 3x detected (3x unique)
  AboutUs Bot Johnny5,AboutUsBot Johnny5/2.0,AboutUsBot

  Example user agent
  Mozilla/5.0 (compatible; AboutUsBot Johnny5/2.0; +http://www.AboutUs.org/)

• 2x detected (1x unique)
  zitebot

  Example user agent
  Mozilla/5.0 (compatible; zitebot support [at] zite [dot] com +http://zite.com)

• 2x detected (1x unique)
  SemanticBot

  Example user agent
  Mozilla/5.0 (compatible; SemanticBot/1.1)

• 2x detected (1x unique)
  AdvBot

  Example user agent
  Mozilla/5.0 (compatible; AdvBot/2.0; +http://advbot.net/bot.html)

• 2x detected (2x unique)
  CompSpyBot,CompSpyBot/1.0

  Example user agent
  Mozilla/5.0 (compatible; CompSpyBot/1.0; +http://www.compspy.com/spider.html)

• 1x detected (1x unique)
  zyklop-crawler

  Example user agent
  Mozilla/5.0 (compatible; zyklop; +http://www.seoratio.de/zyklop-crawler/)

• 2x detected (1x unique)
  CrawlRobot

  Example user agent
  Mozilla/5.0 (compatible; CrawlRobot; +http://localhost/bot.php)

• 1x detected (1x unique)
  archive.bibalex.org_bot

  Example user agent
  Mozilla/5.0 (compatible; archive.bibalex.org_bot; +http://www.test.com)

• 2x detected (2x unique)
  DueDil Crawler,our-crawler

  Example user agent
  Mozilla/5.0 (compatible; electricmonk/3.2.0 +https://www.duedil.com/our-crawler/)

• 2x detected (1x unique)
  MetaGeneratorCrawler

  Example user agent
  MetaGeneratorCrawler/1.3.8 (www.metagenerator.info)

• 2x detected (2x unique)
  robot,Werbefreie Deutsche Suchmaschine

  Example user agent
  Mozilla/5.0 (compatible; DeuSu/5.0.2; +https://deusu.de/robot.html)

• 2x detected (2x unique)
  Visual Site Mapper Crawler,VSMCrawler

  Example user agent
  Mozilla/5.0 (compatible; VSMCrawler; http://www.visualsitemapper.com/crawler/)

• 5x detected (2x unique)
  Exabot,Exabot/3.0

  Example user agent
  Mozilla/5.0 (compatible; Exabot/3.0; +http://www.exabot.com/go/robot)

• 2x detected (2x unique)
  Fake IE,MSIECrawler

  Example user agent
  Mozilla/4.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/4.0; FDM; MSIECrawler; Media Center PC 5.0)

• 1x detected (1x unique)
  Aleksika Spider

  Example user agent
  Aleksika Spider/1.0 ( http://www.aleksika.com/)

• 3x detected (2x unique)
  ConveraCrawler,Convera

  Example user agent
  ConveraCrawler/0.9c ( http://www.authoritativeweb.com/crawl)

• 4x detected (3x unique)
  SemrushBot/0.97,Semrush Bot,SemrushBot

  Example user agent
  Mozilla/5.0 (compatible; SemrushBot/0.97; +http://www.semrush.com/bot.html)

• 3x detected (1x unique)
  Screaming Frog SEO Spider

  Example user agent
  Screaming Frog SEO Spider/2,55

• 3x detected (2x unique)
  WBSearchBot,WBSearchBot/1.1

  Example user agent
  Mozilla/5.0 (compatible; WBSearchBot/1.1; +http://www.warebay.com/bot.html)

• 4x detected (2x unique)
  Google Bot,Googlebot

  Example user agent
  googlebot

• 1x detected (1x unique)
  NMG Spider

  Example user agent
  NMG Spider/0.3 (szukanko.com)

• 3x detected (1x unique)
  MetaHeadersBot

  Example user agent
  MetaHeadersBot (+http://www.metaheaders.com/bot.html)

• 2x detected (1x unique)
  DomainsBot

  Example user agent
  domainsbot (+http://www.domainsbot.com)

• 1x detected (1x unique)
  MSIECrawler

  Example user agent
  Mozilla/4.0 (compatible; MSIE 4.0; MSIECrawler; Windows 95)

• 1x detected (1x unique)
  LesnikBot

  Example user agent
  Mozilla/4.0 (compatible; MSIE 6.0; Windows compatible LesnikBot)

• 2x detected (2x unique)
  Bountii Bot,BountiiBot

  Example user agent
  Mozilla/5.0 (compatible; BountiiBot/1.1; +http://bountii.com/contact.php)

• 1x detected (1x unique)
  8webcrawler

  Example user agent
  Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.8webcrawler) http://skateboarddirectory.com

• 2x detected (1x unique)
  PWBot

  Example user agent
  Mozilla/5.1 (compatible; PWBot/1.0; +http://opendownload.pw)

• 2x detected (2x unique)
  baiduspider,Baidu Spider

  Example user agent
  Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; .NET CLR 1.1.4322; .NET CLR 2.0.50215; baiduspider)

• 2x detected (2x unique)
  Thinglink ImageBot,Elmer

  Example user agent
  Elmer, the Thinglink ImageBot (http://www.thinglink.com/help/ThinglinkImageBot)

• 2x detected (2x unique)
  Fake IE,CrawlDaddy

  Example user agent
  Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; CrawlDaddy v0.3.0 abot v1.1.1.0 http://code.google.com/p/abot)

• 3x detected (2x unique)
  fr_bot,bnf.fr_bot

  Example user agent
  Mozilla/5.0 (compatible; bnf.fr_bot; +http://www.bnf.fr/fr/outils/a.dl_web_capture_robot.html)

• 1x detected (1x unique)
  JikeSpider

  Example user agent
  JikeSpider; +http://shoulu.jike.com/spider.html

• 2x detected (1x unique)
  Thumbshots.ru

  Example user agent
  Mozilla/5.0 (compatible; Thumbshots.ru; +http://thumbshots.ru/bot) Firefox/3

• 1x detected (1x unique)
  PornSpider

  Example user agent
  Mozilla/5.0 (compatible; PornSpider/1.0; +http://www.pornspider.net)

• 1x detected (1x unique)
  AnomaliesBot

  Example user agent
  AnomaliesBot/0.06-dev (The Anomalies Network Search Spider; http://www.anomalies.net; info@anomalies.net)

• 4x detected (4x unique)
  updated,Updated!,updated crawler,updated/0.1-alpha

  Example user agent
  updated/0.1-alpha (updated crawler; http://www.updated.com; crawler@updated.com)

• 1x detected (1x unique)
  bot

  Example user agent
  bot

• 2x detected (1x unique)
  fr-crawler

  Example user agent
  Mozilla/5.0 (compatible; fr-crawler/1.1)

• 1x detected (1x unique)
  SheenBot

  Example user agent
  SheenBot/SheenBot-1.0.4 (Sheen web crawler)

• 2x detected (1x unique)
  JUST-CRAWLER

  Example user agent
  JUST-CRAWLER( http://www.justsystems.com/jp/tech/crawler/)

• 3x detected (2x unique)
  Sogou Push Spider,sogou spider

  Example user agent
  Sogou Push Spider/3.0( http://www.sogou.com/docs/help/webmasters.htm#07)

• 2x detected (1x unique)
  Checklinks

  Example user agent
  Checklinks/1.3 (pywikipedia robot; http://toolserver.org/~dispenser/view/Checklinks)

• 5x detected (2x unique)
  ScoutJet,Blekkobot

  Example user agent
  Mozilla/5.0 (compatible; Blekkobot; ScoutJet; +http://blekko.com/about/blekkobot)

• 4x detected (4x unique)
  Mail.RU_Bot,Mail.RU_Bot/2.0,Mail.Ru Bot,Mail.Ru

  Example user agent
  Mozilla/5.0 (compatible; Linux x86_64; Mail.RU_Bot/2.0; +http://go.mail.ru/help/robots)

• 3x detected (2x unique)
  WinkBot,WinkBot/0.06

  Example user agent
  WinkBot/0.06 (Wink.com search engine web crawler; http://www.wink.com/Wink:WinkBot; winkbot@wink.com)

• 1x detected (1x unique)
  cloudforecastbot

  Example user agent
  cloudforecastbot

• 1x detected (1x unique)
  Crayon Crawler

  Example user agent
  Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; DigExt; Crayon Crawler)

• 1x detected (1x unique)
  FlipboardProxy

  Example user agent
  Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:28.0) Gecko/20100101 Firefox/28.0 (FlipboardProxy/1.1; +http://flipboard.com/browserproxy)

• 1x detected (1x unique)
  TwitterCrawler

  Example user agent
  Mozilla/5.0 (compatible; TwitterCrawler)

• 2x detected (2x unique)
  ANTIPLAG SPIDER,DAWINCI ANTIPLAG SPIDER

  Example user agent
  Mozilla/4.0 (compatible; DAWINCI ANTIPLAG SPIDER)

• 3x detected (1x unique)
  Wotbox

  Example user agent
  Wotbox/2.01 ( http://www.wotbox.com/bot/)

• 2x detected (1x unique)
  Facebot

  Example user agent
  Facebot/1.0

• 2x detected (2x unique)
  ProCogBot,ProCogBot/1.0

  Example user agent
  Mozilla/5.0 (compatible; ProCogBot/1.0; +http://www.procog.com/spider.html)

• 1x detected (1x unique)
  VodpodCrawler

  Example user agent
  Mozilla/5.0 (compatible; VodpodCrawler/1.0; +http://vodpod.com/site/help)

• 3x detected (2x unique)
  ssearch_bot,sSearch Crawler

  Example user agent
  ssearch_bot (sSearch Crawler; http://www.semantissimo.de)

• 1x detected (1x unique)
  TwitSpider

  Example user agent
  TwitSpider

• 2x detected (1x unique)
  metager2-verification-bot

  Example user agent
  Mozilla/5.0 (compatible; metager2-verification-bot; +http://metager2.de/technology.php)

• 3x detected (2x unique)
  envolk[ITS]spider,Envolk

  Example user agent
  envolk[ITS]spider/1.6 ( http://www.envolk.com/envolkspider.html)

• 2x detected (1x unique)
  NetzCheckBot

  Example user agent
  NetzCheckBot/1.0 (security seal; private-krankenversicherung.de.netzcheck.com; bot@netzcheck.com)

• 3x detected (1x unique)
  Applebot

  Example user agent
  Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10 _1) AppleWebKit/600.2.5 (KHTML, like Gecko) Version/8.0.2 Safari/600.2.5 (Applebot/0.1; +http://www.apple.com/go/applebot)

• 1x detected (1x unique)
  XYZ Spider

  Example user agent
  XYZ Spider

• 1x detected (1x unique)
  PercolateCrawler

  Example user agent
  PercolateCrawler/4 (ops@percolate.com)

• 1x detected (1x unique)
  ZemlyaCrawl

  Example user agent
  Mozilla/5.0 (compatible; ZemlyaCrawl/1.0; +http://zemlyaozer.com/bot)

• 2x detected (2x unique)
  Yunyun Bot,YYSpider

  Example user agent
  Mozilla/5.0 (compatible; YYSpider; +http://www.yunyun.com/spider.html)

• 3x detected (2x unique)
  WeSEE:Ads,WeSEE:Search

  Example user agent
  WeSEE:Ads/PageBot (http://www.wesee.com/bot/)

• 2x detected (1x unique)
  GLBot

  Example user agent
  Mozilla/5.1 (compatible; GLBot/1.0; +http://galaxydownloads.co)

• 1x detected (1x unique)
  IXE Crawler

  Example user agent
  IXE Crawler

About this comparison