2x detected (1x unique)
YOURLS
Example user agent
YOURLS v1.7 +http://yourls.org/ (running on http://shortr.at) 4x detected (1x unique)
Scrubby
Example user agent
Mozilla/5.0 (compatible; Scrubby/2.1; +http://www.scrubtheweb.com/abs/meta-check.html) 2x detected (1x unique)
YOURLS
Example user agent
YOURLS v1.7.1 +http://yourls.org/ (running on http://sresc.io) 3x detected (1x unique)
sReleaseBot
Example user agent
Mozilla/5.0 (compatible; sReleaseBot/6.1; +http://www.srelease.com/) 3x detected (2x unique)
Perl,libwww
Example user agent
libwww-perl/5.53 3x detected (1x unique)
SMTBot
Example user agent
Mozilla/5.0 (Windows NT 6.1) (compatible; SMTBot/1.0; +http://www.similartech.com/smtbot) 5x detected (1x unique)
MojeekBot
Example user agent
Mozilla/5.0 (compatible; MojeekBot/0.6; +https://www.mojeek.com/bot.html) 2x detected (1x unique)
DomainAppender Bot
Example user agent
Mozilla/5.0 (compatible; DomainAppender /1.0; +http://www.profound.net/domainappender) 3x detected (1x unique)
diffbot
Example user agent
Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.15 (KHTML, like Gecko; +http://www.diffbot.com) Chrome/24.0.1295.0 Safari/537.15 2x detected (1x unique)
Smartsite HTTPClient
Example user agent
Smartsite HTTPClient - Mozilla/4.0 (compatible; MSIE 6.0) 3x detected (2x unique)
Perl,libwww
Example user agent
libwww-perl/5.820 2x detected (1x unique)
YOURLS
Example user agent
YOURLS v1.7 +http://yourls.org/ (running on http://z-url.us) 8x detected (3x unique)
Googlebot,Google Bot,Google
Example user agent
Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A5376e Safari/8536.25 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) 3x detected (2x unique)
Socialradarbot,Socialradar Bot
Example user agent
Mozilla/5.0 (compatible; Linux; Socialradarbot/2.0; en-US; bot@infegy.com) 3x detected (2x unique)
uk_lddc_bot,bl.uk_lddc_bot
Example user agent
Mozilla/5.0 (bl.uk_lddc_bot; Linux x86_64) PhantomJS/1.9.7 (+http://www.bl.uk/aboutus/legaldeposit/websites/websites/faqswebmaster/index.html) 2x detected (1x unique)
YOURLS
Example user agent
YOURLS v1.6 +http://yourls.org/ (running on http://2u4.us) 2x detected (1x unique)
Adbeat Bot
Example user agent
Mozilla/5.0 (X11; Linux x86_64) adbeat.com/policy AppleWebKit/537.21 (KHTML, like Gecko) Firefox/34.0 Safari/537.21 2x detected (1x unique)
Exploratodo Bot
Example user agent
Mozilla/5.0 (compatible; Exploratodo/1.0; +http://www.exploratodo.com 2x detected (1x unique)
Webmaster Tips Bot
Example user agent
Mozilla/5.0 (compatible; wmtips.com/1.0; +http://www.wmtips.com/tools/) 4x detected (1x unique)
Zookabot
Example user agent
Zookabot/2.4; +http://zookabot.com 2x detected (1x unique)
Windows Phone Search
Example user agent
Windows Phone Search (Windows Phone OS 7.10;NOKIA;Lumia 900;7.10;8862) 2x detected (1x unique)
Werbefreie Deutsche Suchmaschine
Example user agent
Mozilla/5.0 (compatible; DeuSu/5.0.2; +https://deusu.de/robot.html) 4x detected (3x unique)
spbot,SEOprofiler,OpenLinkProfiler
Example user agent
Mozilla/5.0 (compatible; spbot/4.1.0; +http://OpenLinkProfiler.org/bot ) 3x detected (2x unique)
Perl,libwww
Example user agent
libwww-perl/5.808 4x detected (2x unique)
BLEXBot,BLEXBot Crawler
Example user agent
Mozilla/5.0 (compatible; BLEXBot/1.0; +http://webmeup-crawler.com/) 3x detected (1x unique)
SemanticBot
Example user agent
Mozilla/5.0 (compatible; SemanticBot/1.1) 4x detected (1x unique)
Applebot
Example user agent
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/600.2.5 (KHTML, like Gecko) Version/8.0.2 Safari/600.2.5 (Applebot/0.1) 3x detected (2x unique)
Java,Java Standard Library
Example user agent
Java1.2.2 2x detected (1x unique)
007AC9 Crawler
Example user agent
Mozilla/5.0 (compatible; 007ac9 Crawler; http://crawler.007ac9.net/) 3x detected (1x unique)
MarketwireBot
Example user agent
Mozilla/5.0 (compatible; MSIE 9.0; Windows NT; MarketwireBot +http://www.marketwire.com) 3x detected (2x unique)
ANTIPLAG SPIDER,DAWINCI ANTIPLAG SPIDER
Example user agent
Mozilla/4.0 (compatible; DAWINCI ANTIPLAG SPIDER) 3x detected (1x unique)
lbot
Example user agent
Mozilla/5.0 (lbot/2.6.4) (lbot@mailme.not) 4x detected (3x unique)
Mail.RU_Bot,Mail.Ru,Mail.Ru Bot
Example user agent
Mozilla/5.0 (compatible; Linux x86_64; Mail.RU_Bot/Robots; +http://go.mail.ru/help/robots) 2x detected (1x unique)
Sitemap Generator
Example user agent
Mozilla/5.0 (compatible; Sitemap Generator/1.3; +http://www.check-domains.com/sitemap/index.php) 2x detected (1x unique)
YOURLS
Example user agent
YOURLS v1.6 +http://yourls.org/ (running on http://pagn.info) 2x detected (1x unique)
Digg Feed Fetcher
Example user agent
Digg Feed Fetcher 1.0 (Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_1) AppleWebKit/534.48.3 (KHTML, like Gecko) Version/5.1 Safari/534.48.3) 2x detected (1x unique)
YOURLS
Example user agent
YOURLS v1.7 +http://yourls.org/ (running on http://jim-s.com) 2x detected (1x unique)
TrendWinHttp
Example user agent
TrendWinHttp/ 2x detected (1x unique)
WebMasterAid
Example user agent
Mozilla/5.0 (compatible; WebMasterAid/1.0; +http://wmaid.com/explore/) 3x detected (2x unique)
Perl,libwww
Example user agent
libwww-perl/5.48 2x detected (1x unique)
YOURLS
Example user agent
YOURLS v1.6 +http://yourls.org/ (running on http://sgfx.co) 8x detected (3x unique)
msnbot,adidxbot,BingBot
Example user agent
adidxbot/2.0 (+http://search.msn.com/msnbot.htm) 3x detected (2x unique)
Perl,libwww
Example user agent
libwww-perl/5.50 2x detected (1x unique)
kgbody
Example user agent
Mozilla/5.0 (compatible; kgbody/2.0; +http://kids.goo.ne.jp/tool/kgbody.php) 3x detected (2x unique)
woobot,WooRank
Example user agent
woobot/2.0 4x detected (2x unique)
FDSE,FDSE robot
Example user agent
Mozilla/4.0 (compatible: FDSE robot) 5x detected (2x unique)
TweetmemeBot,Tweetmeme Bot
Example user agent
Mozilla/5.0 (TweetmemeBot/4.0; +http://datasift.com/bot.html) Gecko/20100101 Firefox/31.0 4x detected (3x unique)
Perl,lwp-trivial,lwp
Example user agent
lwp-trivial/1.38 2x detected (1x unique)
RMSnapKit
Example user agent
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10) AppleWebKit/600.1.25 (KHTML, like Gecko) RMSnapKit/1.0 8x detected (2x unique)
Bing,bingbot
Example user agent
Mozilla/5.0 (iPhone; CPU iPhone OS 7_0 like Mac OS X) AppleWebKit/537.51.1 (KHTML, like Gecko) Version/7.0 Mobile/11A465 Safari/9537.53 (compatible; bingbot/2.0; http://www.bing.com/bingbot.htm) 3x detected (1x unique)
GLBot
Example user agent
Mozilla/5.1 (compatible; GLBot/1.0; +http://galaxydownloads.co) 3x detected (1x unique)
KomodiaBot
Example user agent
Mozilla/5.0 (Windows NT 6.1; Win64; x64) KomodiaBot/1.0 3x detected (2x unique)
Perl,libwww
Example user agent
libwww-perl/5.803 3x detected (1x unique)
LinkStats Bot
Example user agent
LinkStats Bot 2x detected (1x unique)
Web Glance
Example user agent
Mozilla/5.0 (X11; U; Linux; C -) AppleWebKit/527+ (KHTML, like Gecko, Safari/419.3) Webglance/0.1 5x detected (2x unique)
Python,Python-urllib
Example user agent
Python-urllib/2.5 2x detected (1x unique)
Superarama.com - BOT
Example user agent
Superarama.com - BOT/v.0.1 2x detected (1x unique)
BrokenLinkCheck
Example user agent
Mozilla/5.0 (Windows NT 5.1) BrokenLinkCheck.com/1.1 3x detected (2x unique)
Java,Java Standard Library
Example user agent
Java1.3.1 2x detected (1x unique)
Rival IQ Bot
Example user agent
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9) AppleWebKit/537.71 (KHTML, like Gecko) Version/7.0 Safari/537.71 (Rival IQ, rivaliq.com) 2x detected (1x unique)
TweetedTimes Bot
Example user agent
Mozilla/5.0 (compatible; +http://tweetedtimes.com) 2x detected (1x unique)
vkShare
Example user agent
Mozilla/5.0 (compatible; vkShare; +http://vk.com/dev/Share) 3x detected (1x unique)
Curl
Example user agent
curl/7.35.0 3x detected (1x unique)
GreatNews
Example user agent
GreatNews/1.0 4x detected (3x unique)
iisbot,IIS Site Analysis Web Crawler,IIS Site Analysis
Example user agent
iisbot/1.0 (+http://www.iis.net/iisbot.html) 3x detected (2x unique)
Seznam Screenshot Generator,Seznam Bot
Example user agent
Mozilla/5.0 PhantomJS (compatible; Seznam screenshot-generator 2.1; +http://fulltext.sblog.cz/screenshot/) 2x detected (1x unique)
YOURLS
Example user agent
YOURLS v1.5 +http://yourls.org/ (running on http://oerk.at) 3x detected (2x unique)
AboutUsBot,AboutUs Bot
Example user agent
AboutUsBot/Harpy (Website Analysis; http://www.aboutus.org/Aboutus:Bot; help@aboutus.org) 5x detected (2x unique)
Python,Python-urllib
Example user agent
Python-urllib/1.15 2x detected (1x unique)
YOURLS
Example user agent
YOURLS v1.5.1 +http://yourls.org/ (running on http://u.i4x3.tk) 3x detected (1x unique)
MixBot
Example user agent
MixBot (+http://t.co/GSRLLKex24) 2x detected (1x unique)
Powermarks
Example user agent
Mozilla/4.0 (compatible; Powermarks/3.5; Windows 95/98/2000/NT) 3x detected (2x unique)
com Crawler,WebTarantula
Example user agent
WebTarantula.com Crawler 2x detected (1x unique)
YOURLS
Example user agent
YOURLS v1.6 +http://yourls.org/ (running on http://revmkt.in) 3x detected (2x unique)
Perl,libwww
Example user agent
libwww-perl/5.75 2x detected (1x unique)
WebmasterCoffee
Example user agent
Mozilla/5.0 (compatible; WebmasterCoffee/0.7; +http://webmastercoffee.com/about) 3x detected (2x unique)
stq_bot,Searchteq Bot
Example user agent
stq_bot (+http://www.searchteq.de) 3x detected (1x unique)
memorybot
Example user agent
Mozilla/5.0 (compatible; memorybot/1.20.41 +http://archivethe.net/en/index.php/about/internet_memory1 on behalf of DNB) 5x detected (2x unique)
Python,Python-urllib
Example user agent
Python-urllib/1.17 3x detected (2x unique)
URLfilterDB-crawler,URLfilterDB Crawler
Example user agent
Mozilla/5.0 (URLfilterDB-crawler/1.1) ufdb/1.0 7x detected (3x unique)
Googlebot,Google PageSpeed Insights,Google
Example user agent
Mozilla/5.0 (iPhone; CPU iPhone OS 6_0_1 like Mac OS X) AppleWebKit/537.36 (KHTML, like Gecko; Google Page Speed Insights) Version/6.0 Mobile/10A525 Safari/8536.25 GoogleBot/2.1 3x detected (1x unique)
LinkpadBot
Example user agent
Mozilla/5.0 (compatible; LinkpadBot/1.06; +http://www.linkpad.ru) 2x detected (1x unique)
YOURLS
Example user agent
YOURLS v1.0-rc +http://yourls.org/ (running on http://epi.my) 3x detected (1x unique)
seebot
Example user agent
seebot/2.0 (+http://www.seegnify.com/bot) 3x detected (2x unique)
SNK Screenshot Bot,Save n Keep Screenshot Bot
Example user agent
SNK Screenshot Bot/0.20 ( abuse at savenkeep dot_com ) 2x detected (1x unique)
Google Image Proxy
Example user agent
Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.0.7) Gecko/2009021910 Firefox/3.0.7 (via ggpht.com) 3x detected (1x unique)
BuiBui-Bot
Example user agent
BuiBui-Bot/1.0 (email: buibui[at]dadapro[dot]com) 4x detected (3x unique)
DuckDuckGo-Favicons-Bot,DuckDuck Favicons Bot,DuckDuckGo
Example user agent
Mozilla/5.0 (compatible; DuckDuckGo-Favicons-Bot/1.0; +http://duckduckgo.com) 3x detected (2x unique)
io bot,Nett.io bot
Example user agent
Mozilla/5.0 (compatible; Nett.io bot/1.0; +http://nett.io/) 4x detected (2x unique)
Yahoo Mobile App,Yahoo
Example user agent
YahooMobile/1.0 (Yahoo! Search; 3.1.3) (Apple; iPhone; iPhone OS/7.1.2); YHOO_Search_App/3.1.3 3x detected (1x unique)
zitebot
Example user agent
Mozilla/5.0 (compatible; zitebot support [at] zite [dot] com +http://zite.com) 2x detected (1x unique)
Begun Advertising Bot
Example user agent
Mozilla/5.0 (compatible; BegunAdvertising/3.0; +http://begun.ru/begun/technology/indexer/) 3x detected (1x unique)
Google Keyword Suggestion
Example user agent
Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/534.16 (KHTML, like Gecko, Google Keyword Suggestion) Chrome/10.0.648.127 Safari/534.16 2x detected (1x unique)
Second Live Client
Example user agent
Mozilla/5.0 (Windows; U; ; en-US) AppleWebKit/533.3 (KHTML, like Gecko) SecondLife/3.7.23.297296 (Second Life Release; default skin) Safari/533.3 3x detected (1x unique)
metager2-verification-bot
Example user agent
Mozilla/5.0 (compatible; metager2-verification-bot; +http://metager2.de/technology.php) 4x detected (1x unique)
Screaming Frog SEO Spider
Example user agent
Screaming Frog SEO Spider/2.40 7x detected (5x unique)
Google Ads Bot,AdsBot-Google,AdsBot Google-Mobile,Googlebot,AdsBot Google
Example user agent
AdsBot-Google-Mobile (+http://www.google.com/mobile/adsbot.html) Mozilla (iPhone; U; CPU iPhone OS 3 0 like Mac OS X) AppleWebKit (KHTML, like Gecko) Mobile Safari 2x detected (1x unique)
WebCorp
Example user agent
Mozilla/5.0 (compatible; WebCorp/5.0; +http://www.webcorp.org.uk) 2x detected (1x unique)
Cyberduck
Example user agent
Cyberduck/4.6.4 (16610) (Windows XP/5.1) (x86) 2x detected (1x unique)
python-requests
Example user agent
python-requests/2.0.0 CPython/2.6.6 Linux/2.6.32-431.el6.x86_64 2x detected (1x unique)
Incoming Links - WordPress Plugin
Example user agent
MonitorBacklinksWP (+http://monitorbacklinks.com/blog/incoming-links/) 3x detected (2x unique)
Perl,libwww
Example user agent
libwww-perl/6.05 3x detected (2x unique)
5Bot,Plus5Bot
Example user agent
Mozilla/5.1 (compatible; +5Bot/1.0; +http://plus5files.net) 3x detected (1x unique)
LoadTimeBot
Example user agent
Mozilla/5.0 (compatible; LoadTimeBot/0.9; +http://www.loadtime.net/bot.html) 2x detected (1x unique)
URL Grabber
Example user agent
urlgrabber/3.9.1 yum/3.2.29 3x detected (2x unique)
Java,Java Standard Library
Example user agent
Java1.3.0 2x detected (1x unique)
EventMachine HttpClient
Example user agent
EventMachine HttpClient 6x detected (3x unique)
Exabot,Alexabot,Alexa Crawler
Example user agent
Mozilla/5.0 (compatible; Alexabot/1.0; +http://www.alexa.com/help/certifyscan; certifyscan@alexa.com) 3x detected (2x unique)
Mozilla crawl,Mozilla Crawler
Example user agent
Mozilla crawl/5.0 (compatible; fairshare.cc +http://fairshare.cc) 4x detected (2x unique)
Bloglines,BlogLines Web
Example user agent
Bloglines/3.1 (http://www.bloglines.com) 2x detected (1x unique)
kulturarw3
Example user agent
Mozilla/5.0 (compatible; kulturarw3 +http://www.kb.se/om/projekt/Svenska-webbsidor---Kulturarw3/) 2x detected (1x unique)
python-requests
Example user agent
python-requests/2.3.0 CPython/2.7.6 Linux/3.13.0-24-generic 4x detected (3x unique)
Bing Preview,BingPreview,BingBot
Example user agent
Mozilla/5.0 (iPhone; CPU iPhone OS 6_0_1 like Mac OS X) AppleWebKit/536.26 (KHTML; like Gecko) Version/6.0 Mobile/10A523 Safari/8536.25 BingPreview/1.0b 6x detected (2x unique)
Baiduspider,Baidu Spider
Example user agent
BaiDuSpider 6x detected (3x unique)
Googlebot,Google Bot,Google
Example user agent
Googlebot 4x detected (2x unique)
New-Sogou-Spider,Sogou Spider
Example user agent
New-Sogou-Spider/1.0 (compatible; MSIE 5.5; Windows 98) 4x detected (2x unique)
PagesInventory ,PagesInventory Bot
Example user agent
PagesInventory (robot +http://www.pagesinventory.com) 2x detected (1x unique)
YOURLS
Example user agent
YOURLS v1.5 +http://yourls.org/ (running on http://jmp.su) 5x detected (2x unique)
Python,Python-urllib
Example user agent
Python-urllib/3.1 5x detected (2x unique)
Python,Python-urllib
Example user agent
Python-urllib/1.16 4x detected (1x unique)
GurujiBot
Example user agent
Mozilla/5.0 GurujiBot/1.0 (+http://www.guruji.com/en/WebmasterFAQ.html) 3x detected (1x unique)
CrawlRobot
Example user agent
Mozilla/5.0 (compatible; CrawlRobot; +http://localhost/bot.php) 5x detected (2x unique)
Python,Python-urllib
Example user agent
Python-urllib/2.4 3x detected (1x unique)
WNMbot
Example user agent
WNMbot/0.4.0 3x detected (2x unique)
Java,Java Standard Library
Example user agent
Java1.4.0 4x detected (1x unique)
URLAppendBot
Example user agent
Mozilla/5.0 (compatible; URLAppendBot/1.0; +http://www.profound.net/urlappendbot.html) 6x detected (3x unique)
googlebot,Google Bot,Google
Example user agent
mozilla/5.0 (iphone; cpu iphone os 8_3 like mac os x) applewebkit/600.1.4 (khtml, like gecko) version/8.0 mobile/12f70 safari/600.1.4 (compatible; googlebot/2.1; +http://www.google.com/bot.html) 2x detected (1x unique)
PycURL
Example user agent
PycURL/7.22.0 2x detected (1x unique)
FreeWebMonitoring SiteChecker
Example user agent
FreeWebMonitoring SiteChecker/0.1 (+http://www.freewebmonitoring.com) 4x detected (1x unique)
wonderbot
Example user agent
wonderbot/JS 1.0 2x detected (1x unique)
jig browser web
Example user agent
Mozilla/4.0 (jig browser web; 1.0.4; N02A) 3x detected (1x unique)
Curl
Example user agent
curl/7.15.5 (x86_64-redhat-linux-gnu) libcurl/7.15.5 OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5 5x detected (2x unique)
pub-crawler,Yahoo
Example user agent
Mozilla/5.0 (compatible; pub-crawler; +http://wiki.github.com/bixo/bixo/bixocrawler; bixo-dev@yahoogroups.com) 3x detected (2x unique)
Perl,libwww
Example user agent
libwww-perl/5.63 8x detected (2x unique)
msnbot,BingBot
Example user agent
MSNBOT/0.1 (http://search.msn.com/msnbot.htm) 3x detected (1x unique)
Digincore bot
Example user agent
Mozilla/5.0 (compatible; Digincore bot; https://www.digincore.com/crawler.html for rules and instructions. 3x detected (1x unique)
HRCrawler
Example user agent
HRCrawler/2.0 (XF86; MacOS x86_64) AppleWebKit/537.31 (KHTML, like Gecko) 4x detected (1x unique)
SimplePie
Example user agent
SimplePie/1.2.1-dev (Feed Parser; http://simplepie.org; Allow like Gecko) Build/20111015034325 4x detected (1x unique)
Facebot
Example user agent
Facebot/1.0 3x detected (2x unique)
uk_lddc_bot,bl.uk_lddc_bot
Example user agent
bl.uk_lddc_bot/3.3.0-SNAPSHOT-2014-10-07T09:33:31Z (+http://www.bl.uk/aboutus/legaldeposit/websites/websites/faqswebmaster/index.html) 3x detected (2x unique)
our-crawler,DueDil Crawler
Example user agent
Mozilla/5.0 (compatible; electricmonk/3.2.0 +https://www.duedil.com/our-crawler/) 7x detected (5x unique)
Google Mediapartners,Mediapartners-Google,AdSense Bot,Googlebot,MediaPartners Google
Example user agent
Mozilla/5.0 (iPhone; U; CPU iPhone OS 4_1 like Mac OS X; en-us) AppleWebKit/532.9 (KHTML, like Gecko) Version/4.0.5 Mobile/8B117 Safari/6531.22.7 (compatible; Mediapartners-Google/2.1; +http://www.google.com/bot.html) 2x detected (1x unique)
Google Adwords DisplayAds WebRender
Example user agent
Mozilla/5.0 (en-us) AppleWebKit/537.36(KHTML, like Gecko; Google-Adwords-DisplayAds-WebRender;) Chrome/27.0.1453Safari/537.36 2x detected (1x unique)
YOURLS
Example user agent
YOURLS v1.6 +http://yourls.org/ (running on http://clickand.co) 2x detected (1x unique)
Embedly
Example user agent
Mozilla/5.0 (compatible; Embedly/0.2; snap; +http://support.embed.ly/) 3x detected (2x unique)
js bot,Phantom.js bot
Example user agent
Phantom.js bot 4x detected (1x unique)
Superfeedr bot
Example user agent
Superfeedr bot/2.0 http://superfeedr.com - Make your feeds realtime: get in touch 6x detected (3x unique)
Googlebot,acapbot,Google
Example user agent
Mozilla/5.0 (compatible;acapbot/0.1.;treat like Googlebot) 3x detected (2x unique)
Webspider,RankFlex
Example user agent
RankFlex.com Webspider 5x detected (1x unique)
TweetedTimes Bot
Example user agent
Mozilla/5.0 (compatible; TweetedTimes Bot/1.0; +http://tweetedtimes.com) 2x detected (1x unique)
PagePeeker Screenshot Maker
Example user agent
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/538.1 (KHTML, like Gecko) domain.com Safari/538.1 4x detected (1x unique)
OpenHoseBot
Example user agent
Mozilla/5.0 (compatible; OpenHoseBot/2.1; +http://www.openhose.org/bot.html) 2x detected (1x unique)
BOT for JCE
Example user agent
BOT/0.1 (BOT for JCE) 2x detected (1x unique)
Daumoa
Example user agent
Mozilla/5.0 (compatible; MSIE or Firefox mutant; not on Windows server;) 3x detected (2x unique)
LSSRocketCrawler,Lightspeed Systems RocketCrawler
Example user agent
LSSRocketCrawler/1.0 LightspeedSystems 2x detected (1x unique)
Shrook
Example user agent
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10) AppleWebKit/600.1.25 (KHTML, like Gecko) Shrook/434l 4x detected (2x unique)
WeSEE:Ads,WeSEE:Search
Example user agent
WeSEE:Ads/PageBot (http://www.wesee.com/bot/) 3x detected (1x unique)
AOLbot
Example user agent
Mozilla/5.0 (compatible; AOLbot/4.0; +http://www.aol-soft.com/) 5x detected (2x unique)
SeznamBot,Seznam Bot
Example user agent
Mozilla/5.0 (compatible; SeznamBot/3.2; +http://fulltext.sblog.cz/) 5x detected (3x unique)
Speedy Spider,Entireweb,Speedy
Example user agent
Speedy Spider (Entireweb; Beta/1.3; http://www.entireweb.com/about/search_tech/speedyspider/) 2x detected (1x unique)
CheckSite Verification Agent
Example user agent
CheckSite Verification Agent (+http://www.checksite.us) 2x detected (1x unique)
Feeddler RSS Reader
Example user agent
FeeddlerRSS 2.4 (iPad; iPhone OS 7.1; en_GB) 2x detected (1x unique)
Moreover
Example user agent
Mozilla/5.0 Moreover/5.1 (+http://www.moreover.com; webmaster@moreover.com) 2x detected (1x unique)
python-requests
Example user agent
python-requests/2.5.1 CPython/2.7.6 Linux/3.13.0-36-generic 2x detected (1x unique)
Lipperhey Kaus Australis
Example user agent
Mozilla/5.0 (compatible; Lipperhey-Kaus-Australis/5.0; +https://www.lipperhey.com/en/about/) 5x detected (2x unique)
SeznamBot,Seznam Bot
Example user agent
Mozilla/5.0 (compatible; SeznamBot/3.2-test1; +http://fulltext.sblog.cz/) 4x detected (2x unique)
ArchiveBot,ArchiveTeam ArchiveBot
Example user agent
ArchiveTeam ArchiveBot/20141009.02 (wpull 0.1002a1) and not Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/38.0.2125.101 Safari/537.36 5x detected (2x unique)
Daumoa,Daum
Example user agent
Mozilla/5.0 (compatible; MSIE or Firefox mutant; not on Windows server;) Daumoa 4.0 3x detected (2x unique)
niki-bot,NikiBot
Example user agent
niki-bot 2x detected (1x unique)
Prlog
Example user agent
Mozilla/5.0 (compatible; Prlog/1.0; +http://prlog.ru/) 3x detected (1x unique)
PWBot
Example user agent
Mozilla/5.1 (compatible; PWBot/1.0; +http://opendownload.pw) 3x detected (2x unique)
woobot,WooRank
Example user agent
woobot/1.1 2x detected (1x unique)
YOURLS
Example user agent
YOURLS v1.5 +http://yourls.org/ (running on http://onj.me) 3x detected (2x unique)
Java,Java Standard Library
Example user agent
Java1.4.0_03 3x detected (2x unique)
Google Image Proxy,Gmail Image Proxy
Example user agent
Mozilla/5.0 (Windows NT 5.1; rv:11.0) Gecko Firefox/11.0 (via ggpht.com GoogleImageProxy) 3x detected (1x unique)
ca-crawler
Example user agent
Mozilla/5.0 (compatible; ca-crawler/1.0) 2x detected (1x unique)
WbSrch Web Search
Example user agent
Mozilla/5.0 (compatible; WbSrch/1.1 +http://wbsrch.com) 2x detected (1x unique)
YOURLS
Example user agent
YOURLS v1.5.1 +http://yourls.org/ (running on http://rlink.eu) 3x detected (2x unique)
Perl,libwww
Example user agent
libwww-perl/5.805 3x detected (1x unique)
Curl
Example user agent
curl/7.29.0 3x detected (2x unique)
seekbot,focuseekbot
Example user agent
Mozilla/4.0 (compatible; focuseekbot) 5x detected (2x unique)
Python,Python-urllib
Example user agent
Python-urllib/3.0 2x detected (1x unique)
YOURLS
Example user agent
YOURLS v1.6 +http://yourls.org/ (running on http://www.easylink.mobi) 3x detected (1x unique)
DomainSigmaCrawler
Example user agent
Mozilla/5.0 (compatible; DomainSigmaCrawler/0.1; +http://domainsigma.com/robot) 3x detected (1x unique)
PiplBot
Example user agent
Mozilla/5.0 (compatible; PiplBot; http://www.pipl.com/bot/) 2x detected (1x unique)
YOURLS
Example user agent
YOURLS v1.5 +http://yourls.org/ (running on http://mv2.me) 3x detected (1x unique)
FyberSpider
Example user agent
FyberSpider (+http://www.fybersearch.com/fyberspider.php) 2x detected (1x unique)
YOURLS
Example user agent
YOURLS v1.7.1 +http://yourls.org/ (running on https://ivyti.es) 5x detected (2x unique)
psbot,Picsearch Bot
Example user agent
psbot-image (+http://www.picsearch.com/bot.html) 3x detected (2x unique)
Java,Java Standard Library
Example user agent
Java1.2.1 2x detected (1x unique)
Windows-Update-Agent
Example user agent
Windows-Update-Agent/7.9.9600.17729 Client-Protocol/1.21 3x detected (2x unique)
Java,Java Standard Library
Example user agent
Java1.3.1_04 3x detected (1x unique)
memorybot
Example user agent
Mozilla/5.0 (compatible; memorybot/1.20.71 +http://archivethe.net/en/index.php/about/internet_memory1 on behalf of DNB) 4x detected (1x unique)
NetSeer Crawler
Example user agent
Mozilla/5.0 (compatible; NetSeer crawler/2.0; +http://www.netseer.com/crawler.html; crawler@netseer.com) 2x detected (1x unique)
YOURLS
Example user agent
YOURLS v1.5.1 +http://yourls.org/ (running on http://tty.nu) 3x detected (2x unique)
Slackbot,Slackbot-Link-Expanding
Example user agent
Slackbot-LinkExpanding+(+https://api.slack.com/robots) 5x detected (2x unique)
Magpie RSS,MagpieRSS
Example user agent
MagpieRSS/0.72 \(+http://magpierss.sf.net\) 5x detected (3x unique)
Butterfly,Butterfly Robot,topsy Butterfly
Example user agent
Mozilla/5.0 (Macintosh; Butterfly/1.0; +http://labs.topsy.com/butterfly/) Gecko/2009032608 Firefox/3.0.8 3x detected (1x unique)
MonoBot
Example user agent
Mozilla/5.1 (compatible; MonoBot/1.0; +http://mono.name) 5x detected (1x unique)
TweetedTimes Bot
Example user agent
Mozilla/5.0 (compatible; TweetedTimes Bot/1.0 (resolver); +http://tweetedtimes.com) 4x detected (2x unique)
ssearch_bot,sSearch Crawler
Example user agent
ssearch_bot (sSearch Crawler; http://www.semantissimo.de) 3x detected (1x unique)
lbot
Example user agent
Mozilla/5.0 (lbot/2.6.4) lbot@mailme.not 2x detected (1x unique)
YOURLS
Example user agent
YOURLS v1.7.1 +http://yourls.org/ (running on http://bp7.org) 3x detected (2x unique)
Webcrawler,HubSpot Webcrawler
Example user agent
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.116 Safari/537.36 HubSpot Webcrawler 3x detected (1x unique)
domainsbot
Example user agent
domainsbot (+http://www.domainsbot.com) 4x detected (2x unique)
SemrushBot,Semrush Bot
Example user agent
Mozilla/5.0 (compatible; SemrushBot-SA/0.97; +http://www.semrush.com/bot.html) 2x detected (1x unique)
MegaIndex Bot
Example user agent
Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +https://www.megaindex.ru/?tab=linkAnalyze) 4x detected (2x unique)
Google Page Speed,Google PageSpeed Insights
Example user agent
Mozilla/5.0 (iPhone; CPU iPhone OS 6_0_1 like Mac OS X) AppleWebKit/537.36 (KHTML, like Gecko; Google Page Speed Insights) Version/6.0 Mobile/10A525 Safari/8536.25 3x detected (1x unique)
YisouSpider
Example user agent
YisouSpider 4x detected (3x unique)
Bing Preview,BingPreview,BingBot
Example user agent
Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0; WOW64; Trident/5.0; BingPreview/1.0b) 4x detected (1x unique)
GurujiBot
Example user agent
Mozilla/5.0 GurujiBot/1.0 ( http://www.guruji.com/en/WebmasterFAQ.html) 4x detected (3x unique)
Bing Preview,BingPreview,BingBot
Example user agent
Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.2; Trident/6.0) BingPreview/1.0b 2x detected (1x unique)
YOURLS
Example user agent
YOURLS v1.5.1 +http://yourls.org/ (running on http://fhort.com) 2x detected (1x unique)
Second Live Commerce Client
Example user agent
Mozilla/4.0 (compatible; MSIE 7.0b; Windows NT 6.0 ; .NET CLR 2.0.50215; SL Commerce Client v1.0; Tablet PC 2.0 4x detected (1x unique)
WeSEE:Search
Example user agent
WeSEE:Search/0.1 (Alpha, http://www.wesee.com/bot/) 2x detected (1x unique)
YOURLS
Example user agent
YOURLS v1.7 +http://yourls.org/ (running on http://u.vsil.org) 3x detected (2x unique)
Google Image Proxy,Gmail Image Proxy
Example user agent
Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.0.7) Gecko/2009021910 Firefox/3.0.7 (via ggpht.com GoogleImageProxy) 3x detected (1x unique)
Scopia Crawler
Example user agent
Mozilla/5.0 (compatible; Scopia Crawler 1.2; +http://www.scopia.co) 2x detected (1x unique)
Windows-Update-Agent
Example user agent
Windows-Update-Agent/7.9.9600.18094 Client-Protocol/1.21 3x detected (1x unique)
Typhoeus
Example user agent
Typhoeus - https://github.com/typhoeus/typhoeus 5x detected (3x unique)
Speedy Spider,Entireweb,Speedy
Example user agent
Speedy Spider (http://www.entireweb.com/about/search_tech/speedyspider/) 3x detected (1x unique)
memoryBot
Example user agent
Mozilla/5.0 (compatible; memoryBot/1.20.235 +http://internetmemory.org/en/) 2x detected (1x unique)
YOURLS
Example user agent
YOURLS v1.7 +http://yourls.org/ (running on http://www.h-link.de) 2x detected (1x unique)
MetaInspector
Example user agent
MetaInspector/3.0.0 (+https://github.com/jaimeiniesta/metainspector) 3x detected (2x unique)
Java,Java Standard Library
Example user agent
Java1.4.0_01 3x detected (1x unique)
uipbot
Example user agent
uipbot/1.0 (uipbot@semasio.net) 2x detected (1x unique)
Windows-RSS-Platform
Example user agent
Windows-RSS-Platform/2.0 (IE 11.0; Windows NT 6.3) 3x detected (1x unique)
Slackbot
Example user agent
Slackbot (+https://api.slack.com/robots) 5x detected (1x unique)
360spider
Example user agent
Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0); 360Spider(compatible; HaosouSpider; http://www.haosou.com/help/help_3_2.html) 3x detected (2x unique)
ru Screenshot Bot,Screenshot Bot
Example user agent
pr-cy.ru Screenshot Bot 2x detected (1x unique)
Iframely Bot
Example user agent
Iframely/0.7.1 (+http://iframe.ly/;) 3x detected (2x unique)
Perl,libwww
Example user agent
libwww-perl/5.69 2x detected (1x unique)
Google Structured-Data TestingTool
Example user agent
Mozilla/5.0 (compatible; X11; Linux x86_64; Google-StructuredDataTestingTool; +http://www.google.com/webmasters/tools/richsnippets) 4x detected (2x unique)
SemrushBot,Semrush Bot
Example user agent
Mozilla/5.0 (compatible; SemrushBot-SI/0.97; +http://www.semrush.com/bot.html) 3x detected (1x unique)
WinHTTP
Example user agent
WinHTTP 5x detected (2x unique)
Google Web Preview,Googlebot
Example user agent
Mozilla/5.0 (iPad; CPU OS 5_0 like Mac OS X) AppleWebKit/537.36 (KHTML, like Gecko; Google Web Preview) Version/5.1 Mobile/9A334 Safari/7534.48.3 3x detected (2x unique)
Perl,libwww
Example user agent
libwww-perl/5.76 3x detected (2x unique)
adbeat_bot,Adbeat Bot
Example user agent
adbeat_bot 3x detected (2x unique)
com/bot,GroupHigh Bot
Example user agent
Mozilla/5.0 (compatible; GroupHigh/1.0; +http://www.grouphigh.com/bot.html) 2x detected (1x unique)
SiteExplorer
Example user agent
Mozilla/5.0 (compatible; SiteExplorer/1.0b; +http://SiteExplorer.Info/) 4x detected (1x unique)
Screaming Frog SEO Spider
Example user agent
Screaming Frog SEO Spider/2,22 3x detected (2x unique)
Thinglink ImageBot,Elmer
Example user agent
Elmer, the Thinglink ImageBot (http://www.thinglink.com/help/ThinglinkImageBot) 5x detected (4x unique)
TinEye,TinEye-bot,TinEye Bot,TinEye Crawler
Example user agent
TinEye-bot/0.02 (see http://www.tineye.com/crawler.html) 2x detected (1x unique)
YOURLS
Example user agent
YOURLS v1.5 +http://yourls.org/ (running on http://shelby.im) 3x detected (2x unique)
Slackbot,Slackbot-Link-Expanding
Example user agent
Slackbot-LinkExpanding (+https://api.slack.com/robots) 2x detected (1x unique)
YOURLS
Example user agent
YOURLS v1.6 +http://yourls.org/ (running on http://urlmicro.me) 4x detected (2x unique)
MixrankBot,Mixrank Bot
Example user agent
Mozilla/5.0 (compatible; MixrankBot; crawler@mixrank.com) 3x detected (1x unique)
UniversalFeedParser
Example user agent
UniversalFeedParser/5.1.3 +https://code.google.com/p/feedparser/ 3x detected (2x unique)
com/bot,SEOdiver Bot
Example user agent
Mozilla/5.0 (compatible; SEOdiver/1.0; +http://www.seodiver.com/bot) 8x detected (5x unique)
Googlebot Mobile,Googlebot-Mobile,Google Bot Mobile,Googlebot,Google
Example user agent
Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A5376e Safari/8536.25 (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html) 2x detected (1x unique)
URL Grabber
Example user agent
urlgrabber/3.10 yum/3.4.3 4x detected (3x unique)
Perl,lwp-trivial,lwp
Example user agent
lwp-trivial/1.36 4x detected (2x unique)
Bloglines,BlogLines Web
Example user agent
Bloglines/3.0-rho (http://www.bloglines.com; 3 subscribers) 4x detected (2x unique)
GomezAgent,Gomez Site Monitor
Example user agent
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:13.0; GomezAgent 3.0) Gecko/20100101 Firefox/13.0.1 4x detected (3x unique)
WebThumb,Website Thumbnail Generator,WebThumbnail
Example user agent
Mozilla/5.0 (compatible; WebThumbnail/3.x; Website Thumbnail Generator; +http://webthumbnail.org) 3x detected (1x unique)
AskPeterBot
Example user agent
Mozilla/5.0 (compatible; AskPeterBot; +http://www.askpeter.de/bot.html) 2x detected (1x unique)
YOURLS
Example user agent
YOURLS v1.7 +http://yourls.org/ (running on http://www.vrbas.be/url) 4x detected (1x unique)
CatchBot
Example user agent
CatchBot/1.0; http://www.catchbot.com 2x detected (1x unique)
open-webkit-sharp
Example user agent
Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/534+ (KHTML, like Gecko) OpenWebKitSharp 4x detected (1x unique)
EveryoneSocialBot
Example user agent
Mozilla/5.0 (compatible; EveryoneSocialBot/1.0; support@everyonesocial.com http://everyonesocial.com/) 4x detected (3x unique)
WeSEE_Bot,WeSEE Bot,WeSEE:Search
Example user agent
WeSEE_Bot:we_help_monitize_your_site (http://www.wesee.com/bot/) 3x detected (1x unique)
NerdyBot
Example user agent
NerdyBot 2x detected (1x unique)
Slinky
Example user agent
Slinky v1.0 +http://dentedreality.com.au/projects/slinky/ 3x detected (2x unique)
Perl,libwww
Example user agent
libwww-perl/5.814 3x detected (2x unique)
Perl,libwww
Example user agent
libwww-perl/5.36 2x detected (1x unique)
dead link checker
Example user agent
www.deadlinkchecker.com XMLHTTP/1.0 Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.86 Safari/537.36 4x detected (2x unique)
Google Page Speed,Google PageSpeed Insights
Example user agent
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko; Google Page Speed Insights) Chrome/27.0.1453 Safari/537.36 3x detected (1x unique)
memorybot
Example user agent
Mozilla/5.0 (compatible; memorybot/1.20.207 +http://mignify.com/bot.html) 3x detected (1x unique)
hivaBot
Example user agent
hivaBot/hivaBot-0.2 (info@gorgor.ir) 2x detected (1x unique)
picmole Bot
Example user agent
Mozilla/5.0 (compatible;picmole/1.0 +http://www.picmole.com) 7x detected (3x unique)
Yahoo! Slurp,Inktomi Slurp,Yahoo
Example user agent
Mozilla/5.0 (iPhone; CPU iPhone OS 7_1 like Mac OS X) AppleWebKit (KHTML, like Gecko) Mobile (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) 2x detected (1x unique)
vSentry
Example user agent
Mozilla/5.0 (Windows; chromeframe/2.4.8.5746) AppleWebKit/1.0 (KHTML, like Gecko) Bromium Safari/1.0 2x detected (1x unique)
DuckDuck App
Example user agent
DDG-Android-3.0.11 4x detected (2x unique)
Daumoa,Daum
Example user agent
Mozilla/5.0 (compatible; MSIE or Firefox mutant; not on Windows server;) Daumoa/4.0 4x detected (2x unique)
Yahoo Mobile App,Yahoo
Example user agent
YahooMobile/1.0 (Yahoo! Search; 3.1.2) (Apple; iPod touch; iPhone OS/7.1.1); YHOO_Search_App/3.1.2 2x detected (1x unique)
Adbeat Bot
Example user agent
Mozilla/5.0 (X11; U; Linux x86; de-de) adbeat.com/policy Gecko/20100423 Ubuntu/10.04 (lucid) Firefox/3.6.3 AppleWebKit/532.4 Safari/532.4 3x detected (1x unique)
XoviBot
Example user agent
Mozilla/5.0 (compatible; XoviBot/2.0; +http://www.xovibot.net/) 4x detected (2x unique)
Java,Java Standard Library
Example user agent
Java/1.17 3x detected (2x unique)
LivelapBot,Livelap Crawler
Example user agent
LivelapBot/0.2 (http://site.livelap.com/crawler) 2x detected (1x unique)
itsscan
Example user agent
Mozilla/5.0 (compatible; Muenster University of Applied Sciences; +http://fb02itsscan.fh-muenster.de) 2x detected (1x unique)
YOURLS
Example user agent
YOURLS v1.7 +http://yourls.org/ (running on http://mrk.im) 2x detected (1x unique)
profiller
Example user agent
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.21 (KHTML, like Gecko) profiller Safari/537.21 2x detected (1x unique)
WebThumb
Example user agent
Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.9.2.19) Gecko WebThumb/1.0 2x detected (1x unique)
Web Glance
Example user agent
Mozilla/5.0 (X11; U; Linux; C -) AppleWebKit/527+ (KHTML, like Gecko, Safari/419.3) Webglance/0.1 2x detected (1x unique)
IPv4Scan
Example user agent
IPv4Scan (+http://ipv4scan.com) 2x detected (1x unique)
SiteExplorer
Example user agent
Mozilla/5.0 (compatible; SiteExplorer/1.0b; +http://siteexplorer.info/) 3x detected (2x unique)
Java,Java Standard Library
Example user agent
Java1.4.0_02 5x detected (2x unique)
Python,Python-urllib
Example user agent
Python-urllib/2.0a1 3x detected (1x unique)
GIDBot
Example user agent
GIDBot/2.0 (+http://www.gidnetwork.com/tools/gzip-test.php) 4x detected (2x unique)
SemrushBot,Semrush Bot
Example user agent
Mozilla/5.0 (compatible; SemrushBot/0.97sa; +http://www.semrush.com/bot.html) 2x detected (1x unique)
python-requests
Example user agent
python-requests/2.2.1 CPython/2.7.3 Linux/3.2.0-56-generic 2x detected (1x unique)
YOURLS
Example user agent
YOURLS v1.7 +http://yourls.org/ (running on http://linkcut.eu) 4x detected (2x unique)
EasouSpider,Easou Spider
Example user agent
Mozilla/5.0 (compatible; EasouSpider; +http://www.easou.com/search/spider.html) 5x detected (3x unique)
Speedy Spider,Entireweb,Speedy
Example user agent
Speedy Spider (Entireweb; Beta/1.0; http://www.entireweb.com/about/search_tech/speedyspider/) 3x detected (2x unique)
SkypeUriPreview,Skype URI Preview
Example user agent
Mozilla/5.0 (Windows NT 6.1; WOW64) SkypeUriPreview Preview/0.5 5x detected (1x unique)
DotBot
Example user agent
Mozilla/5.0 (compatible; DotBot/1.1; http://www.opensiteexplorer.org/dotbot, help@moz.com) 2x detected (1x unique)
YOURLS
Example user agent
YOURLS v1.6 +http://yourls.org/ (running on http://3uves.com) 2x detected (1x unique)
Anonymizied
Example user agent
Mozilla/5.0 (Randomized by FreeSafeIP.com/upgrade-to-remove; compatible; MSIE 9.0; Windows NT 5.0WOW64) Chrome/18.0.1229.79 5x detected (3x unique)
Speedy Spider,Entireweb,Speedy
Example user agent
Speedy Spider (Entireweb; Beta/1.1; http://www.entireweb.com/about/search_tech/speedyspider/) 2x detected (1x unique)
Adbeat Bot
Example user agent
Mozilla/5.0 (X11; U; Linux x86; nl-NL) adbeat.com/policy Gecko/20100423 Ubuntu/10.04 (lucid) Firefox/3.6.3 AppleWebKit/532.4 Safari/532.4 4x detected (3x unique)
Perl,lwp-trivial,lwp
Example user agent
lwp-trivial/1.35 2x detected (1x unique)
node-fetch
Example user agent
node-fetch/1.0 (+https://github.com/bitinn/node-fetch) 6x detected (3x unique)
Googlebot,Google Bot,Google
Example user agent
googlebot 2x detected (1x unique)
YOURLS
Example user agent
YOURLS v1.7 +http://yourls.org/ (running on http://ezurl.me) 4x detected (3x unique)
Mail.RU_Bot,Mail.Ru,Mail.Ru Bot
Example user agent
Mozilla/5.0 (compatible; Linux x86_64; Mail.RU_Bot/Fast/2.0; +http://go.mail.ru/help/robots) 3x detected (2x unique)
Perl,libwww
Example user agent
libwww-perl/5.800 (+http://passoire.afraid.org/mylittlewebsurvey/index.html) 3x detected (2x unique)
com crawler,semantic-visions.com crawler
Example user agent
Mozilla/5.0 (X11; compatible; semantic-visions.com crawler; HTTPClient 3.1) 5x detected (2x unique)
Python,Python-urllib
Example user agent
Python-urllib/2.7 2x detected (1x unique)
DomainScan Server Monitoring
Example user agent
Mozilla/5.0 (compatible; DomainSCAN/2.0; +http://domainscan.info/robot/) 4x detected (1x unique)
Scrubby
Example user agent
Mozilla/5.0 (compatible; Scrubby/3.2; +http://seotools.scrubtheweb.com/webpage-analyzer.html) 3x detected (1x unique)
SoftListBot
Example user agent
Mozilla/5.0 (compatible; SoftListBot/2.2; +http://www.softlist.us/) 7x detected (4x unique)
Bing,MsnBot,msnbot-media,BingBot
Example user agent
Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/534+ (KHTML, like Gecko) MsnBot-Media /1.0b 4x detected (2x unique)
JamesBOT,WebCrawler
Example user agent
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6 - James BOT - WebCrawler http://cognitiveseo.com/bot.html 2x detected (1x unique)
Alexa Site Audit
Example user agent
Mozilla/5.0 (compatible; alexa site audit/1.0; +http://www.alexa.com/help/webmasters; ) 4x detected (3x unique)
OOZBOT,veoozbot,Veooz Bot
Example user agent
Mozilla/5.0 ( compatible ; Veooz/1.0 ; +http://www.veooz.com/veoozbot.html ) 4x detected (1x unique)
GurujiBot
Example user agent
GurujiBot/1.0 (+http://www.guruji.com/en/WebmasterFAQ.html) 3x detected (1x unique)
AntBot
Example user agent
AntBot/1.0 (http://www.ant.com) 3x detected (2x unique)
Perl,libwww
Example user agent
libwww-perl/5.821 2x detected (1x unique)
Qwantify
Example user agent
Mozilla/5.0 (compatible; Qwantify/2.0; +https://www.qwant.com/) 2x detected (1x unique)
Lightspeed Systems Crawler
Example user agent
Mozilla/4.0 (compatible; MSIE 7.0;Windows NT 5.1;.NET CLR 1.1.4322;.NET CLR 2.0.50727;.NET CLR 3.0.04506.30) Lightspeedsystems 4x detected (3x unique)
Baiduspider,Baidu Mobile Search,Baidu Spider
Example user agent
Mozilla/5.0 (Linux;u;Android 2.3.7;zh-cn;) AppleWebKit/533.1 (KHTML,like Gecko) Version/4.0 Mobile Safari/533.1 (compatible; +http://www.baidu.com/search/spider.html) 2x detected (1x unique)
Proxy Gear Pro
Example user agent
Proxy Gear Pro/2.1 (Windows; N; Windows NT 6.1; en) 3x detected (1x unique)
oBot
Example user agent
Mozilla/5.0 (compatible; oBot/2.3.1; http://filterdb.iss.net/crawler/) 2x detected (1x unique)
Werbefreie Deutsche Suchmaschine
Example user agent
Mozilla/5.0 (compatible; DeuSu/0.1.0; +https://deusu.de/robot.html) 7x detected (4x unique)
Googlebot,Locubot,BingBot,Google
Example user agent
Locubot (compatible; Googlebot; msnbot) 3x detected (2x unique)
Java,Java Standard Library
Example user agent
Java1.3.1_06 2x detected (1x unique)
YOURLS
Example user agent
YOURLS v1.6 +http://yourls.org/ (running on http://vaiqui.com) 3x detected (2x unique)
Perl,libwww
Example user agent
libwww-perl/5.816 3x detected (1x unique)
fr-crawler
Example user agent
Mozilla/5.0 (compatible; fr-crawler/1.1) 3x detected (1x unique)
AjaxSnapBot
Example user agent
Mozilla/5.0 (compatible; AjaxSnapBot; +http://ajaxsnapshots.com/bot.html) 2x detected (1x unique)
JobBoerse Bot
Example user agent
Mozilla/5.0 (X11; U; Linux Core i7-4980HQ; de; rv:32.0; compatible; Jobboerse.com; http://www.xn--jobbrse-d1a.com) Gecko/20100401 Firefox/24.0 3x detected (2x unique)
fastbot,fastbot crawler
Example user agent
fastbot crawler beta 4.0 (+http://www.fastbot.de) 2x detected (1x unique)
Google Structured-Data TestingTool
Example user agent
Mozilla/5.0 (compatible; Google-Structured-Data-Testing-Tool +http://developers.google.com/structured-data/testing-tool/) 3x detected (1x unique)
Gregarius
Example user agent
Gregarius/0.5.2 (+http://devlog.gregarius.net/docs/ua) 2x detected (1x unique)
Shrook
Example user agent
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10) AppleWebKit/600.1.25 (KHTML, like Gecko) Shrook/1290y 6x detected (4x unique)
Bing,msnbot-media,BingBot,MSNBot
Example user agent
Mozilla/5.0 (iPhone; CPU iPhone OS 7_0 like Mac OS X; en-us) AppleWebKit/534+ (KHTML, like Gecko) Version/7.0 Mobile/11A465 Safari/9537.53 MsnBot-Media /1.0b 4x detected (1x unique)
Applebot
Example user agent
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/600.2.5 (KHTML, like Gecko) Version/8.0.2 Safari/600.2.5 (Applebot/0.1; +http://www.apple.com/go/applebot) 3x detected (1x unique)
SMTBot
Example user agent
Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A5376e Safari/8536.25 (compatible; SMTBot/1.0; +http://www.similartech.com/smtbot) 4x detected (1x unique)
GurujiBot
Example user agent
GurujiBot/1.0 (+http://www.guruji.com/WebmasterFAQ.html) 5x detected (2x unique)
Python,Python-urllib
Example user agent
Python-urllib/2.6 2x detected (1x unique)
Squzer
Example user agent
Mozilla/5.0 (compatible; Squzer/0.0.1; +http://www.declum.com/) 3x detected (2x unique)
Perl,libwww
Example user agent
libwww-perl/5.64 2x detected (1x unique)
HTTrack
Example user agent
Mozilla/4.5 (compatible; HTTrack 3.0x; Windows 98) 3x detected (1x unique)
MetaGeneratorCrawler
Example user agent
MetaGeneratorCrawler/1.3.4 (www.metagenerator.info) 3x detected (2x unique)
pagefreezer crawler,PageFreezer
Example user agent
PageFreezer (pagefreezer crawler; http://pagefreezer.com/pagefreezer-crawler/; info@pagefreezer.com) 4x detected (1x unique)
Scrubby
Example user agent
Mozilla/5.0 (compatible; Scrubby/2.2; +http://www.scrubtheweb.com/) 2x detected (1x unique)
akregator
Example user agent
Mozilla/5.0 (X11;) AppleWebKit/534.34 (KHTML, like Gecko) akregator/4.14.1 Safari/534.34 3x detected (2x unique)
imrbot,Mignify Bot
Example user agent
Mozilla/5.0 (compatible; imrbot/1.10.8 +http://www.mignify.com) 2x detected (1x unique)
Crawler
Example user agent
Mozilla/5.0 (compatible; Crawler/0.9; http://linkfluence.net/) 2x detected (1x unique)
Dolphin smalltalk http client
Example user agent
Dolphin http client/10.3.0(280) (Android) 5x detected (2x unique)
TweetmemeBot,Tweetmeme Bot
Example user agent
Mozilla/5.0 (compatible; TweetmemeBot/3.0; +http://tweetmeme.com/) 3x detected (1x unique)
SMTBot
Example user agent
Mozilla/5.0 (compatible; SMTBot/1.0; +http://www.similartech.com/smtbot) 5x detected (1x unique)
Gluten Free Crawler
Example user agent
Mozilla/5.0 (compatible; Gluten Free Crawler/1.0; +http://glutenfreepleasure.com/) 3x detected (2x unique)
PHP,Zend_Http_Client
Example user agent
Zend_Http_Client 5x detected (1x unique)
360spider
Example user agent
Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/21.0.1180.89 Safari/537.1; 360Spider 3x detected (2x unique)
Java,Java Standard Library
Example user agent
Java1.4.2_03 2x detected (1x unique)
YOURLS
Example user agent
YOURLS v1.6 +http://yourls.org/ (running on http://xurl.co) 4x detected (2x unique)
linkdexbot,Linkdex Bot
Example user agent
Mozilla/5.0 (compatible; linkdexbot/2.0; +http://www.linkdex.com/bots/) 4x detected (1x unique)
FyberSpider
Example user agent
FyberSpider 3x detected (1x unique)
TestCrawler
Example user agent
Mozilla/5.0 (compatible; TestCrawler) 3x detected (2x unique)
online-webceo-bot,webceo Bot
Example user agent
Mozilla/5.0 (compatible; online-webceo-bot/1.0; +http://online.webceo.com) 5x detected (2x unique)
Magpie RSS,MagpieRSS
Example user agent
MagpieRSS/0.7 ( http://magpierss.sf.net) 3x detected (1x unique)
AdvBot
Example user agent
Mozilla/5.0 (compatible; AdvBot/2.0; +http://advbot.net/bot.html) 5x detected (2x unique)
Google Web Preview,Googlebot
Example user agent
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko; Google Web Preview) Chrome/27.0.1453 Safari/537.36 3x detected (1x unique)
DBot
Example user agent
Mozilla/5.1 (compatible; DBot/7.0; +http://getdownload.ws) 2x detected (1x unique)
WinHttp
Example user agent
Mozilla/4.0 (compatible; Win32; WinHttp.WinHttpRequest.5) 2x detected (1x unique)
Baidu Mobile Search
Example user agent
Mozilla/5.0 (Linux;u;Android 2.3.7;zh-cn;) AppleWebKit/533.1 (KHTML,like Gecko) Version/4.0 Mobile Safari/533.1 (compatible; +http://www.baidu.com/search/spi_der.html) 3x detected (1x unique)
GarlikCrawler
Example user agent
GarlikCrawler/1.2 (http://garlik.com/, crawler@garlik.com) 4x detected (3x unique)
spbot,SEOprofiler,OpenLinkProfiler
Example user agent
Mozilla/5.0 (compatible; spbot/4.0.9; +http://OpenLinkProfiler.org/bot ) 4x detected (3x unique)
Perl,lwp-trivial,lwp
Example user agent
lwp-trivial/1.41 3x detected (1x unique)
JobdiggerSpider
Example user agent
Mozilla/5.0 (compatible; JobdiggerSpider +http://www.jobdigger.nl/spider) 3x detected (2x unique)
UA crawler,Internet Archive
Example user agent
Mozilla/2.0 (compatible; ARCHIVE.ORG.UA crawler) 5x detected (1x unique)
360spider
Example user agent
Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0); 360Spider 2x detected (1x unique)
Android Download Manager
Example user agent
AndroidDownloadManager 3x detected (1x unique)
IstellaBot
Example user agent
Mozilla/5.0 (compatible; IstellaBot/1.18.81 +http://www.tiscali.it/) 4x detected (3x unique)
Bing Preview,BingPreview,BingBot
Example user agent
Mozilla/5.0 (Windows NT 6.3; WOW64; Trident/7.0; rv:11.0; BingPreview/1.0b) like Gecko 3x detected (2x unique)
Web Crawler,Arachnida Web Crawler
Example user agent
Arachnida Web Crawler 4x detected (1x unique)
IRLbot
Example user agent
IRLbot/2.0 ( http://irl.cs.tamu.edu/crawler) 5x detected (2x unique)
Python,Python-urllib
Example user agent
Python-urllib/2.1 3x detected (1x unique)
NetzCheckBot
Example user agent
NetzCheckBot/1.0 (security seal; private-krankenversicherung.de.netzcheck.com; bot@netzcheck.com) 4x detected (2x unique)
SurveyBot,Survey Bot
Example user agent
Mozilla/5.0 (Windows; U; Windows NT 5.1; en; rv:1.9.0.13) Gecko/2009073022 Firefox/3.5.2 (.NET CLR 3.5.30729) SurveyBot/2.3 (DomainTools) 3x detected (2x unique)
Apple-PubSub,Apple iCloud
Example user agent
Apple-PubSub/65.28 2x detected (1x unique)
Google Toolbar
Example user agent
Mozilla/4.0 (compatible; GoogleToolbar 7.5.5111.1712; Windows XP 5.1; MSIE 8.0.6001.18702) 2x detected (1x unique)
FlipboardProxy
Example user agent
Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:28.0) Gecko/20100101 Firefox/28.0 (FlipboardProxy/1.1; +http://flipboard.com/browserproxy) 2x detected (1x unique)
bot
Example user agent
bot 3x detected (2x unique)
Perl,libwww
Example user agent
libwww-perl/5.65 3x detected (1x unique)
nbot
Example user agent
Mozilla/5.0 (compatible; nbot/2.0) 2x detected (1x unique)
YOURLS
Example user agent
YOURLS v1.7 +http://yourls.org/ (running on http://rutlink.info) 2x detected (1x unique)
YOURLS
Example user agent
YOURLS v1.7 +http://yourls.org/ (running on http://ecra.se) 2x detected (1x unique)
Android Download Manager
Example user agent
AndroidDownloadManager/4.3 (Linux; U; Android 4.3; ME302C Build/JSS15Q) 3x detected (1x unique)
HyperCrawl
Example user agent
Mozilla/5.0 (compatible; HyperCrawl/0.2; +http://www.seograph.net/bot.html) 4x detected (2x unique)
Orangebot,Orange Bot
Example user agent
Mozilla/5.0 (compatible; Orangebot/2.0; support.orangebot@orange.com) 3x detected (2x unique)
NewsBlur,NewsBlur Feed Fetcher
Example user agent
NewsBlur Feed Fetcher - 0 subscribers - http://www.newsblur.com/site/1719739/nouvelles-cinema (Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_1) AppleWebKit/534.48.3 (KHTML, like Gecko) Version/5.1 Safari/534.48.3) 3x detected (2x unique)
Perl,libwww
Example user agent
libwww-perl/5.800 4x detected (2x unique)
linkdexbot,Linkdex Bot
Example user agent
Mozilla/5.0 (compatible; linkdexbot/2.0; +http://www.linkdex.com/about/bots/) 4x detected (2x unique)
Java,Java Standard Library
Example user agent
Java/1.7 3x detected (1x unique)
JoobleBot
Example user agent
JoobleBot (compatible; http://jooble.org/jooble-bot) 2x detected (1x unique)
YOURLS
Example user agent
YOURLS v1.7 +http://yourls.org/ (running on http://www.bv-xa.com) 3x detected (1x unique)
MerchantCentricBot
Example user agent
Mozilla/5.0 (compatible; MerchantCentricBot/1.0) 7x detected (4x unique)
Goo,ichiro,ichiro/mobile,Ichiro Mobile Bot
Example user agent
DoCoMo/2.0 P900i(c100;TB;W24H11) (compatible; ichiro/mobile goo;+http://search.goo.ne.jp/option/use/sub4/sub4-1/) 4x detected (2x unique)
Linkdex Bot,linkdexbot
Example user agent
Mozilla/5.0 (iPhone; U; CPU iPhone OS 4_1 like Mac OS X; en-us) AppleWebKit/532.9 (KHTML, like Gecko) Version/4.0.5 Mobile/8B117 Safari/6531.22.7 (compatible; linkdexbot-mobile/2.1; +http://www.linkdex.com/about/bots/) 1x detected (1x unique)
PHP Server Monitor
Example user agent
Mozilla/5.0 (compatible; phpservermon/3.1.1; +http://www.phpservermonitor.org) 6x detected (1x unique)
TurnitinBot
Example user agent
TurnitinBot/3.0 (http://www.turnitin.com/robot/crawlerinfo.html) 7x detected (4x unique)
Goo,ichiro,ichiro/mobile,Ichiro Mobile Bot
Example user agent
DoCoMo/2.0 P900i(c100;TB;W24H11) (compatible; ichiro/mobile goo; +http://search.goo.ne.jp/option/use/sub4/sub4-1/) 4x detected (1x unique)
Screaming Frog SEO Spider
Example user agent
Screaming Frog SEO Spider/2.22 5x detected (1x unique)
Netvibes
Example user agent
Netvibes (http://www.netvibes.com/; 8 subscribers; feedID: 2244192) 1x detected (1x unique)
Daum
Example user agent
Mozilla/5.0 (compatible; MSIE or Firefox mutant;) Daum 4.1 2x detected (2x unique)
BountiiBot,Bountii Bot
Example user agent
Mozilla/5.0 (compatible; BountiiBot/1.1; +http://bountii.com/contact.php) 4x detected (2x unique)
Daumoa,Daum
Example user agent
Mozilla/5.0 (compatible; MSIE or Firefox mutant; not on Windows server;) Daumoa/4.0; 2x detected (2x unique)
UptimeRobot,Uptime Robot
Example user agent
Mozilla/5.0+(compatible; UptimeRobot/2.0; http://www.uptimerobot.com/) 4x detected (3x unique)
NewsBlur,Safari,NewsBlur Favicon Fetcher
Example user agent
NewsBlur Favicon Fetcher - 7 subscribers - http://www.newsblur.com/site/1948420/analytics-piwik (Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_1) AppleWebKit/534.48.3 (KHTML, like Gecko) Version/5.1 Safari/534.48.3) 1x detected (1x unique)
Baidu Spider
Example user agent
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; baidu Transcoder;) 5x detected (1x unique)
MojeekBot
Example user agent
Mozilla/5.0 (compatible; MojeekBot/0.6; http://www.mojeek.com/bot.html) 5x detected (2x unique)
Voilabot,Orange Bot
Example user agent
Mozilla/5.0 (Windows; U; Windows NT 5.1;fr;rv:1.8.1) VoilaBotCollector BETA 0.1 (http://www.voila.com/) 7x detected (4x unique)
Google Feedfetcher,FeedFetcher-Google,Googlebot,Feedfetcher Google
Example user agent
Feedfetcher-Google; (+http://www.google.com/feedfetcher.html; 2 subscribers; feed-id=17860707833818568603) 5x detected (2x unique)
Magpie RSS,MagpieRSS
Example user agent
MagpieRSS/0.72 (+http://magpierss.sf.net) 5x detected (1x unique)
Netvibes
Example user agent
Netvibes (http://www.netvibes.com) 3x detected (1x unique)
DotBot
Example user agent
dotbot 1x detected (1x unique)
Googlebot
Example user agent
Mozilla/5.0 (Linux; Android 4.0.4; Galaxy Nexus Build/IMM76B) AppleWebKit/537.36 (KHTML, like Gecko; Google-Publisher-Plugin) Chrome/27.0.1453 Mobile Safari/537.36 3x detected (2x unique)
NewsBlur,NewsBlur Feed Fetcher
Example user agent
NewsBlur Feed Fetcher - 7 subscribers - http://www.newsblur.com/site/1948420/analytics-piwik (Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_1) AppleWebKit/534.48.3 (KHTML, like Gecko) Version/5.1 Safari/534.48.3) 5x detected (3x unique)
Apache Nutch,Nutch,Nutch-based Bot
Example user agent
Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-CN; rv:1.9.2.8) Firefox/3.6.8/Nutch-1.7 4x detected (2x unique)
Google Page Speed,Google PageSpeed Insights
Example user agent
Mozilla/5.0 (iPhone; U; CPU iPhone OS 4_0_1 like Mac OS X; en-us) AppleWebKit/537.4 (KHTML, like Gecko; Google Page Speed Insights) Version/4.0.5 Mobile/8A306 Safari/6531.22.7 1x detected (1x unique)
Google Partner Monitoring
Example user agent
Mozilla/5.0 (Windows NT 5.1) AppleWebKit/535.1 (KHTML, like Gecko) Chrome/14.0.835.202 Safari/535.1 google_partner_monitoring FWSzVTDDBz14547302713138T 3x detected (2x unique)
NewsBlur,NewsBlur Favicon Fetcher
Example user agent
NewsBlur Page Fetcher - 7 subscribers - http://www.newsblur.com/site/3966817/analytics-piwik (Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_1) AppleWebKit/534.48.3 (KHTML, like Gecko) Version/5.1 Safari/534.48.3) 2x detected (2x unique)
EasouSpider,Easou Spider
Example user agent
Mozilla/5.0 (iPhone; U; CPU iPhone OS 3_0 like Mac OS X; en-us; EasouSpider; +http://www.easou.com/search/spider.html) 2x detected (2x unique)
SputnikImageBot,Sputnik Bot
Example user agent
Mozilla/5.0 (compatible; SputnikImageBot/2.2) 7x detected (4x unique)
Google Feedfetcher,FeedFetcher-Google,Googlebot,Feedfetcher Google
Example user agent
Feedfetcher-Google; (+http://www.google.com/feedfetcher.html; 375 subscribers; feed-id=15381863289700640853) 4x detected (2x unique)
OrangeBot,Orange Bot
Example user agent
Mozilla/5.0 (compatible; OrangeBot/2.0; support.orangebot@orange.com) 4x detected (1x unique)
rogerbot
Example user agent
rogerbot/1.0 (http://www.moz.com/dp/rogerbot, rogerbot-crawler@moz.com) 2x detected (1x unique)
NewsBlur
Example user agent
NewsBlur Feed Finder (Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_1) AppleWebKit/534.48.3 (KHTML, like Gecko) Version/5.1 Safari/534.48.3) 5x detected (3x unique)
sogou spider,Sogou Pic Spider,Sogou Image Crawler
Example user agent
Sogou Pic Spider/3.0(+http://www.sogou.com/docs/help/webmasters.htm 4x detected (1x unique)
ICC-Crawler
Example user agent
ICC-Crawler/2.0 (Mozilla-compatible; ; http://www.nict.go.jp/en/univ-com/plan/crawl.html) 2x detected (2x unique)
YYSpider,Yunyun Bot
Example user agent
Mozilla/5.0 (compatible; YYSpider; +http://www.yunyun.com/spider.html) 5x detected (2x unique)
Linkedin Bot,LinkedInBot
Example user agent
LinkedInBot/1.0 (compatible; Mozilla/5.0; Jakarta Commons-HttpClient/3.1 +http://www.linkedin.com) 3x detected (2x unique)
yrspider,Yunyun Bot
Example user agent
Mozilla/5.0 (compatible; YRSpider; +http://www.yunrang.com/yrspider.html) 2x detected (2x unique)
SputnikBot,Sputnik Bot
Example user agent
Mozilla/5.0 (compatible; SputnikBot/2.2) 4x detected (1x unique)
Applebot
Example user agent
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10 _1) AppleWebKit/600.2.5 (KHTML, like Gecko) Version/8.0.2 Safari/600.2.5 (Applebot/0.1; +http://www.apple.com/go/applebot) 4x detected (1x unique)
SimplePie
Example user agent
SimplePie/1.2.1-dev (Feed Parser; http://simplepie.org; Allow like Gecko) Build/20130514092120 3x detected (3x unique)
Gazeta Bot,Yandex Bot,YandexBot
Example user agent
Mozilla/5.0 (compatible; Linux i686; Yandex.Gazeta Bot/1.0; +http://gazeta.yandex.ru) 2x detected (2x unique)
YRSpider,Yunyun Bot
Example user agent
Mozilla/5.0 (compatible; YRSpider; +http://www.yunyun.com/SiteInfo.php?r=about) 7x detected (4x unique)
Google Feedfetcher,FeedFetcher-Google,Googlebot,Feedfetcher Google
Example user agent
Feedfetcher-Google; (+http://www.google.com/feedfetcher.html; 19 subscribers; feed-id=13965549748850348809) 5x detected (1x unique)
360spider
Example user agent
Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; MDDR; .NET4.0C; .NET4.0E; .NET CLR 1.1.4322; Tablet PC 2.0); 360Spider 3x detected (1x unique)
TweetedTimes Bot
Example user agent
TweetedTimes Bot/1.0 (Mozilla/5.0 Compatible, +http://tweetedtimes.com) 2x detected (1x unique)
Dazoobot
Example user agent
Mozilla/5.0 (compatible; Dazoobot/0.1; +http://dazoo.fr) 2x detected (1x unique)
NalezenCzBot
Example user agent
NalezenCzBot/1.0 (http://www.nalezen.cz/about-crawler) 3x detected (2x unique)
omgilibot,Omgili bot
Example user agent
omgilibot/0.3 +http://www.omgili.com/Crawler.html 2x detected (1x unique)
nlcrawler
Example user agent
nlcrawler/1.0 (+http://northernlight.com/) 4x detected (3x unique)
redditbot,Reddit,Reddit Bot
Example user agent
Mozilla/5.0 (compatible; redditbot/1.0; +http://www.reddit.com/feedback) 1x detected (1x unique)
Baidu Spider
Example user agent
Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-CN; rv:1.9.2.8;baidu Transcoder) Gecko/20100722 Firefox/3.6.8 ( .NET CLR 3.5.30729) 2x detected (2x unique)
VSMCrawler,Visual Site Mapper Crawler
Example user agent
Mozilla/5.0 (compatible; VSMCrawler; http://www.visualsitemapper.com/crawler/) 4x detected (1x unique)
SimplePie
Example user agent
SimplePie/1.3.1 (Feed Parser; http://simplepie.org; Allow like Gecko) Build/20121030175911 2x detected (2x unique)
SensikaBot,Sensika Bot
Example user agent
SensikaBot/x.33 (+http://sensika.com) 8x detected (3x unique)
Googlebot,Google Bot,Google
Example user agent
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Googlebot/2.1; +http://www.google.com/bot.html) Safari/537.36 4x detected (1x unique)
Superfeedr bot
Example user agent
Superfeedr bot/2.0 http://superfeedr.com - Make your feeds realtime: get in touch! 6x detected (2x unique)
NewsGator,NewsGatorOnline
Example user agent
NewsGatorOnline/2.0 (http://www.newsgator.com; 2 subscribers) 8x detected (2x unique)
Bing,bingbot
Example user agent
Mozilla/5.0 (iPhone; CPU iPhone OS 7_0 like Mac OS X) AppleWebKit/537.51.1 (KHTML, like Gecko) Version/7.0 Mobile/11A465 Safari/9537.53 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm) 4x detected (3x unique)
Borland Delphi,Rippers,Indy Library
Example user agent
Mozilla/3.0 (compatible; Indy Library) 3x detected (3x unique)
Google App Engine,AppEngine-Google,Google AppEngine
Example user agent
Mozilla/5.0 (Windows NT 6.3; WOW64; rv:27.0) Gecko/20100101 Firefox/27.0 AppEngine-Google; (+http://code.google.com/appengine; appid: s~chengzhoukun) 3x detected (3x unique)
Google App Engine,AppEngine-Google,Google AppEngine
Example user agent
Mozilla/5.0 (Windows NT 6.3; Trident/7.0; rv:11.0) like Gecko AppEngine-Google; (+http://code.google.com/appengine; appid: s~oowsxs) 7x detected (5x unique)
Google Ads Bot,AdsBot-Google,AdsBot Google-Mobile,Googlebot,AdsBot Google
Example user agent
AdsBot-Google-Mobile (+http://www.google.com/mobile/adsbot.html) Mozilla (iPhone; U; CPU iPhone OS 3 0 like Mac OS X) AppleWebKit/537.36 (KHTML, like Gecko) Mobile Safari/537.36 2x detected (2x unique)
Google App Engine,Google AppEngine
Example user agent
Mozilla/5.0 (Android; Mobile; rv:29.0) Gecko/29.0 Firefox/29.0 AppEngine-Google; (+http://code.google.com/appengine; appid: s~oldecreator) 3x detected (3x unique)
Google App Engine,AppEngine-Google,Google AppEngine
Example user agent
Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1626.1 Safari/537.36 AppEngine-Google; (+http://code.google.com/appengine; appid: s~hex-plus) 8x detected (3x unique)
Googlebot,Google Bot,Google
Example user agent
Mozilla/5.0 AppleWebKit/537.4 (KHTML, like Gecko; compatible; Googlebot/2.1; +http://www.google.com/bot.html) Safari/537.4 3x detected (2x unique)
OrangeBot,Orange Bot
Example user agent
Mozilla/4.0 (compatible; MSIE 4.01; Windows CE; PPC; 240x320; SPV M700; OpVer 19.123.2.733) OrangeBot-Mobile 2008.0 (mobilesearch.support@orange-ftgroup.com) 2x detected (2x unique)
PowerMapper,CrawlerProcess
Example user agent
Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.2; WOW64; Trident/6.0) CrawlerProcess (http://www.PowerMapper.com) /5.10.736.0 2x detected (2x unique)
Bing Preview,BingBot
Example user agent
Mozilla/5.0 (Windows Phone 10.0; Android 4.2.1; Nokia; Lumia 520) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.71 Mobile Safari/537.36 Edge/12.0 BingPreview/1.0b 7x detected (3x unique)
Googlebot,Google PageSpeed Insights,Google
Example user agent
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko; Google Page Speed Insights) Chrome/27.0.1453 Safari/537.36 GoogleBot/2.1 8x detected (3x unique)
Googlebot,Google Web Preview,Google
Example user agent
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; Google Web Preview Analytics) Chrome/27.0.1453 Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) 2x detected (2x unique)
FAST Web Crawler,FAST-WebCrawler
Example user agent
Opera/30.0.1835.59 (X11; Linux i686; Ubuntu/14.10) Presto/2.12.388 Version/29.16 AppleWebKit/599.0+ (KHTML, like Gecko) Chrome/41.0.2272.118 Safari/537.36 FAST-WebCrawler/3.8 3x detected (2x unique)
Google Web Preview,Googlebot
Example user agent
Mozilla/5.0 (Linux; U; Android 2.3.4; generic) AppleWebKit/537.4 (KHTML, like Gecko; Google Web Preview) Version/4.0 Mobile Safari/537.4 2x detected (2x unique)
Fastladder,livedoor FeedFetcher
Example user agent
Fastladder FeedFetcher/0.01 (http://fastladder.com/; 27 subscribers) 2x detected (2x unique)
Bing Preview,BingBot
Example user agent
Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.2; Trident/6.0; WOW64; Trident/6.0; BingPreview/1.0b) 7x detected (4x unique)
Goo,ichiro,ichiro/mobile,Ichiro Mobile Bot
Example user agent
DoCoMo/2.0 P900i(c100;TB;W24H11) (compatible; ichiro/mobile goo; +http://help.goo.ne.jp/help/article/1142/) 3x detected (1x unique)
Curl
Example user agent
curl/7.16.0 (i386-unknown-openbsd4.1) libcurl/7.16.0 OpenSSL/0.9.7j zlib/1.2.3 2x detected (2x unique)
Bing Preview,BingBot
Example user agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2486.0 Safari/537.36 Edge/13.11067 BingPreview/1.0b 2x detected (1x unique)
GomezAgent
Example user agent
Mozilla/4.0 (compatible; MSIE 5.01; GomezAgent 2.0; Windows NT) 3x detected (1x unique)
Curl
Example user agent
curl/7.14.1 (i386-portbld-freebsd4.7) libcurl/7.14.1 OpenSSL/0.9.8 zlib/1.1.3 6x detected (4x unique)
Google Feedfetcher,FeedFetcher-Google,Googlebot,Feedfetcher Google
Example user agent
FeedFetcher-Google; ( http://www.google.com/feedfetcher.html) 4x detected (2x unique)
ArchiveBot,ArchiveTeam ArchiveBot
Example user agent
ArchiveTeam ArchiveBot/20140807.01 (wpull 0.37a1) and not Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.125 Safari/537.36 6x detected (4x unique)
Google Mediapartners,Mediapartners-Google,Googlebot,MediaPartners Google
Example user agent
DoCoMo/2.0 SH905i(c100;TB;W24H16) (compatible; Mediapartners-Google/2.1; +http://www.google.com/bot.html) 1x detected (1x unique)
HeartRails Capture
Example user agent
Mozilla/5.0 (X11; U; Linux i686; xx; rv:1.9.2.18) Gecko/20110622 HeartRails_Capture/1.0.5 (+http://capture.heartrails.com/) Namoroka/3.6.18 1x detected (1x unique)
ADJUSTbot
Example user agent
Mozilla/5.0 (compatible; ADJUSTbot/2.0; +http://www.ad-just.jp/) 0x detected (0x unique)
Example user agent
SearQuBot/SearQuBot v1.0 3x detected (2x unique)
Yahoo,Yahoo! Japan
Example user agent
Mozilla/5.0 (Linux; U; Android 2.3.3; ja-jp; sdk Build/GRI34; Y!J-BRZ/YATSHA crawler; http://help.yahoo.co.jp/help/jp/search/indexing/indexing-15.html) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1 2x detected (2x unique)
mixi-crawler,mixi
Example user agent
mixi-crawler/2.00 (http://mixi.jp/) 4x detected (1x unique)
SimplePie
Example user agent
SimplePie/1.3-dev (Feed Parser; http://simplepie.org; Allow like Gecko) Build/20111118194710 3x detected (2x unique)
Yahoo,Yahoo! Japan
Example user agent
Mozilla/5.0 (iPhone; Y!J-BRY/YATSH crawler; http://help.yahoo.co.jp/help/jp/search/indexing/indexing-15.html) 1x detected (1x unique)
Rakutenbot
Example user agent
Mozilla/5.0 (compatible; Rakutenbot/1.0; +http://dynamic.rakuten.co.jp/bot.html) 7x detected (4x unique)
Google Feedfetcher,FeedFetcher-Google,Googlebot,Feedfetcher Google
Example user agent
Feedfetcher-Google; (+http://www.google.com/feedfetcher.html; feed-id=000000000000000000) 5x detected (2x unique)
TweetmemeBot,Tweetmeme Bot
Example user agent
Mozilla/5.0 (compatible; TweetmemeBot/2.11; +http://tweetmeme.com/) 1x detected (1x unique)
kizasi-spider
Example user agent
kizasi-spider/1.0 (+http://kizasi.jp/) 2x detected (2x unique)
mixi-news-crawler,mixi
Example user agent
mixi-news-crawler/1.00 (http://mixi.jp/) 1x detected (1x unique)
Feed Crawler
Example user agent
www2.apserver.net ASP-Ranker Feed Crawler 1x detected (1x unique)
cloudforecastbot
Example user agent
cloudforecastbot
Comparison created
2016-05-17 07:26:46 | by
ThaDafinser