Detected as bot - UserAgentStringCom

• Nymesis

  Example user agent
  Nymesis/1.0 (http://nymesis.com)

• lwp-trivial

  Example user agent
  lwp-trivial/1.41

• msnbot

  Example user agent
  msnbot-UDiscovery/2.0b (+http://search.msn.com/msnbot.htm)

• Exabot

  Example user agent
  Mozilla/5.0 (compatible; Exabot/3.0 (BiggerBetter); +http://www.exabot.com/go/robot)

• Baiduspider

  Example user agent
  BaiDuSpider

• omgilibot

  Example user agent
  omgilibot/0.3 +http://www.omgili.com/Crawler.html

• Shim-Crawler

  Example user agent
  Shim-Crawler

• YodaoBot

  Example user agent
  Mozilla/5.0 (compatible;YodaoBot-Image/1.0;http://www.youdao.com/help/webmaster/spider/;)

• W3C-checklink

  Example user agent
  W3C-checklink/4.81 libwww-perl/5.836

• Java

  Example user agent
  LG/KC910i/V10a Browser/Obigo-Q7.1 MMS/LG-MMS-V1.0/1.2 MediaPlayer/LGPlayer/1.0 Java/ASVM/1.1 Profile/MIDP-2.1 Configuration/CLDC-1.1

• Baiduspider

  Example user agent
  'Mozilla/5.0 (compatible; Baiduspider/2.0; +ht'

• NewsGator

  Example user agent
  NewsGator/3.0,gzip(gfe),gzip(gfe)

• Java

  Example user agent
  LG-A290/V100 Obigo/Q05A MMS/LG-MMS-V1.0/1.2 Java/ASVM/1.1 Profile/MIDP-2.1 Configuration/CLDC-1.1

• LDSpider

  Example user agent
  lworldspider

• Googlebot

  Example user agent
  Googlebot/Nutch-1.7

• Googlebot

  Example user agent
  Googlebot-Mobile (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html)

• Mnogosearch

  Example user agent
  MnoGoSearch/3.3.15

• sogou spider

  Example user agent
  Sogou develop spider/4.0( http://www.sogou.com/docs/help/webmasters.htm#07)

• Java

  Example user agent
  LG/KU990i/v10a Browser/Obigo-Q05A/3.6 MMS/LG-MMS-V1.0/1.2 Java/ASVM/1.0 Profile/MIDP-2.0 Configuration/CLDC-1.1

• msnbot

  Example user agent
  msnbot/1.1 (+http://search.msn.com/msnbot.htm)

• iaskspider

  Example user agent
  iaskspider/2.0(+http://iask.com/help/help_index.html)

• Googlebot

  Example user agent
  Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) ASProxy/5.5b5

• iaskspider

  Example user agent
  iaskspider2 (iask@staff.sina.com.cn)

• Scrubby

  Example user agent
  Scrubby/2.1 (http://www.scrubtheweb.com/)

• magpie-crawler

  Example user agent
  magpie-crawler/1.1 (U; Linux amd64; en-GB; http://www.brandwatch.net)

• CatchBot

  Example user agent
  CatchBot/1.0; http://www.catchbot.com

• Java

  Example user agent
  CTC/HUAWEIC5720 Java/HWJa/1.0 Profile/MIDP-2.0 Configuration/CLDC-1.1 UNTRUSTED/1.0

• EmeraldShield.com WebBot

  Example user agent
  EmeraldShield.com WebBot (http://www.emeraldshield.com/webbot.aspx)

• htdig

  Example user agent
  htdig/3.1.5 (infosys@storm.rmi.org)

• Speedy Spider

  Example user agent
  Speedy Spider (Entireweb; Beta/1.2; http://www.entireweb.com/about/search_tech/speedyspider/)

• Bingbot

  Example user agent
  Mozilla/5.0 (compatible; bingbot/2.0 http://www.bing.com/bingbot.htm)

• Gigabot

  Example user agent
  Gigabot/2.0/gigablast.com/spider.html

• msnbot

  Example user agent
  msnbot-NewsBlogs/2.0b (+http://search.msn.com/msnbot.htm)

• Googlebot

  Example user agent
  Googlebot/2.1 (http://www.googlebot.com/bot.html)

• Java

  Example user agent
  Java/1.6.0_43

• Java

  Example user agent
  LG-C320/V100 Obigo/Q7.3 MMS/LG-MMS-V1.0/1.2 MediaPlayer/LGPlayer/1.0 Java/ASVM/1.1 Profile/MIDP-2.1 Configuration/CLDC-1.1

• Baiduspider

  Example user agent
  Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search

• lmspider

  Example user agent
  lmspider (lmspider@scansoft.com)

• ZyBorg

  Example user agent
  Mozilla/4.0 compatible ZyBorg/1.0 (wn-14.zyborg@looksmart.net; http://www.WISEnutbot.com)

• semanticdiscovery

  Example user agent
  semanticdiscovery/2.0(http://www.semanticdiscovery.com/robot.html)

• Java

  Example user agent
  LG-BL40g/v1.0 Browser/Obigo-Q7.3 MMS/LG-MMS-V1.0/1.2 MediaPlayer/LGPlayer/1.0 Java/ASVM/1.1 Profile/MIDP-2.1 Configuration/CLDC-1.1

• Gregarius

  Example user agent
  Gregarius/0.5.2 (+http://devlog.gregarius.net/docs/ua)

• OmniExplorer_Bot

  Example user agent
  OmniExplorer_Bot/6.70 ( http://www.omni-explorer.com) WorldIndexer

• sogou spider

  Example user agent
  Sogou Mobile Spider1.0 (http://wap.sogou.com)

• Ezooms

  Example user agent
  Mozilla/5.0 (compatible; Ezooms/1.0; ezooms.bot@gmail.com)

• DotBot

  Example user agent
  dotbot

• GurujiBot

  Example user agent
  GurujiBot/1.0 (+http://www.guruji.com/en/WebmasterFAQ.html)

• Sosospider

  Example user agent
  Mozilla/5.0 (compatible; Sosospider/2.0; +http://help.soso.com/webspider.htm)

• W3C_Validator

  Example user agent
  W3C_Validator/1.767

• boitho.com-dc

  Example user agent
  boitho.com-dc/0.86 ( http://www.boitho.com/dcbot.html )

• sogou spider

  Example user agent
  Sogou Orion spider/4.0( http://www.sogou.com/docs/help/webmasters.htm#07)

• Googlebot

  Example user agent
  Mozilla/5.0 (Windows; Mozilla 4.0(compatible; Googlebot/5.0; http://www.google.com/googlebot); en-US;)

• Bloglines

  Example user agent
  Bloglines/3.0-rho (http://www.bloglines.com; 3 subscribers)

• Java

  Example user agent
  Apache-HttpClient/4.0-beta2 (java 1.5)

• Java

  Example user agent
  LG-T325/V100 Obigo/Q7.3 MMS/LG-MMS-V1.1/1.2 MediaPlayer/LGPlayer/1.0 Java/ASVM/1.1 Profile/MIDP-2.1 Configuration/CLDC-1.1

• sogou spider

  Example user agent
  Sogou Pic Spider/3.0(+http://www.sogou.com/docs/help/webmasters.htm

• Java

  Example user agent
  ProjectWF-java-test-crawler

• Radian6

  Example user agent
  radian6_default_(www.radian6.com/crawler)

• NewsGator

  Example user agent
  NewsGator FetchLinks extension/0.2.0 (http://graemef.com)

• LDSpider

  Example user agent
  ldspider (BTC 2011 crawl, harth@kit.edu, http://code.google.com/p/ldspider/wiki/Robots)

• genieBot

  Example user agent
  genieBot (http://64.5.245.11/faq/faq.html)

• ichiro

  Example user agent
  DoCoMo/2.0 P900i(c100;TB;W24H11)(compatible; ichiro/mobile goo; http://help.goo.ne.jp/door/crawler.html)

• Scrubby

  Example user agent
  Mozilla/5.0 (compatible; Scrubby/3.2; +http://seotools.scrubtheweb.com/webpage-analyzer.html)

• Vagabondo

  Example user agent
  Mozilla/5.0 (compatible; Vagabondo/Wapspider; webcrawler at wise-guys dot nl; http://webagent.wise-guys.nl/)

• NewsGator

  Example user agent
  NewsGatorOnline/2.0 (http://www.newsgator.com)

• Java

  Example user agent
  Mozilla/5.0 (LG-T500 AppleWebkit/531 Browser/Phantom/V2.0 Widget/LGMW/3.0 MMS/LG-MMS-V1.0/1.2 Java/ASVM/1.1 Profile/MIDP-2.1 Configuration/CLDC-1.1)

• iaskspider

  Example user agent
  iaskspider/2.0( http://iask.com/help/help_index.html)

• GSiteCrawler

  Example user agent
  GSiteCrawler/v1.12 rev. 260 (http://gsitecrawler.com/)

• FeedFetcher-Google

  Example user agent
  Feedfetcher-Google; (+http://www.google.com/feedfetcher.html; 19 subscribers; feed-id=13965549748850348809)

• DotBot

  Example user agent
  Mozilla/5.0 (compatible; DotBot/1.1; http://www.opensiteexplorer.org/dotbot, help@moz.com)

• OmniExplorer_Bot

  Example user agent
  OmniExplorer_Bot/4.32 (+http://www.omni-explorer.com) WorldIndexer

• Vivante Link Checker

  Example user agent
  Vivante Link Checker (http://www.vivante.com)

• Xenu Link Sleuth

  Example user agent
  Xenu Link Sleuth 1.2c

• Java

  Example user agent
  Java/1.5.0_17

• psbot

  Example user agent
  psbot-page (+http://www.picsearch.com/bot.html)

• lwp-trivial

  Example user agent
  lwp-trivial/1.35

• Java

  Example user agent
  LG-KF700Q Browser/Obigo-Q05A/3.12 MMS/LG-MMS-V1.0/1.2 Java/ASVM/1.1 Profile/MIDP-2.1 Configuration/CLDC-1.1

• Larbin

  Example user agent
  larbin_2.6.3 larbin2.6.3@unspecified.mail

• Speedy Spider

  Example user agent
  Speedy Spider (http://www.entireweb.com)

• GalaxyBot

  Example user agent
  GalaxyBot/1.0 (http://www.galaxy.com/galaxybot.html)

• Java

  Example user agent
  LG-A155/V100 Obigo/Q05A MMS/LG-MMS-V1.0/1.2 Java/ASVM/1.1 Profile/MIDP-2.1 Configuration/CLDC-1.1

• OmniExplorer_Bot

  Example user agent
  OmniExplorer_Bot/6.63b (+http://www.omni-explorer.com) WorldIndexer

• Moreoverbot

  Example user agent
  Moreoverbot/5.1 ( http://w.moreover.com; webmaster@moreover.com) Mozilla/5.0

• Googlebot

  Example user agent
  SAMSUNG-SGH-E250/1.0 Profile/MIDP-2.0 Configuration/CLDC-1.1 UP.Browser/6.2.3.3.c.1.101 (GUI) MMP/2.0 (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html)

• Xenu Link Sleuth

  Example user agent
  Xenu Link Sleuth 1.2d

• Java

  Example user agent
  Mozilla/5.0 (Java 1.6.0_01; Windows XP 5.1 x86; en) ICEbrowser/v6_1_2

• Java

  Example user agent
  Squrl Java/1.6.0_22

• AbiLogicBot

  Example user agent
  Mozilla/5.0 (compatible; AbiLogicBot/1.0; +http://www.abilogic.com/bot.html)

• Java

  Example user agent
  IE6.0,Java(TM) 2 Runtime Environment, Standard Edition

• WomlpeFactory

  Example user agent
  WomlpeFactory/0.1 (+http://www.Womple.com/bot.html)

• Baiduspider

  Example user agent
  Mozilla/5.0 (compatible; Baiduspider-cpro; +http://www.baidu.com/search/spider.html)

• Googlebot

  Example user agent
  Googlebot (gocrawl v0.4)

• Java

  Example user agent
  Mozilla/5.0 (Vodafone/1.0/LG-BL20/V10b Browser/Obigo-Q7.3 MMS/LG-MMS-V1.0/1.2 MediaPlayer/LGPlayer/1.0 Java/ASVM/1.1 Profile/MIDP-2.1 Configuration/CLDC-1.1)

• Java

  Example user agent
  Mozilla/5.0 (LG-C375 AppleWebkit/531 Browser/Phantom/V2.0 Widget/LGMW/3.0 MMS/LG-MMS-V1.0/1.2 Java/ASVM/1.1 Profile/MIDP-2.1 Configuration/CLDC-1.1)

• Java

  Example user agent
  Mozilla/5.0 (LG-T580 AppleWebkit/531 Browser/Phantom/V2.0 Widget/LGMW/3.0 MMS/LG-MMS-V1.0/1.2 Java/ASVM/1.1 Profile/MIDP-2.1 Configuration/CLDC-1.1)

• OOZBOT

  Example user agent
  Mozilla/5.0 (compatible; yoozBot-2.2; http://yooz.ir; info@yooz.ir)

• Baiduspider

  Example user agent
  Baiduspider+(+http://www.baidu.com/search/spider_jp.html)

• sogou spider

  Example user agent
  sogou spider

• msnbot

  Example user agent
  msnbot-media/1.1 (+http://search.msn.com/msnbot.htm)

• Java

  Example user agent
  LG-GU285g Browser/Obigo-Q7.3 MMS/LG-MMS-V1.0/1.2 MediaPlayer/LGPlayer/1.0 Java/ASVM/1.1 Profile/MIDP-2.1 Configuratbile-video=d6sdv39lierm72vfe4ucsfjo61

• Java

  Example user agent
  Huawei/1.0/HuaweiT7200/B100 Browser/Obigo-Browser/Q05A MMS/Obigo-MMS/Q05A SyncML/HW-SyncML/1.0 Java/HWJa/1.0 Profile/MIDP-2.0 Configuration/CLDC-1.1 Player/QTV-Player/5.3

• Jyxobot

  Example user agent
  Jyxobot/1

• Java

  Example user agent
  Epic Empire/1.80 pgid=454af7cb350c2a1b tag=454af7cb350c2a1b cv=180011 hw=ASUS ME371MG; ME371MG; gzip; JavaHttp

• Java

  Example user agent
  Mozilla/5.0 (LG-T385b-Bouygues AppleWebkit/531 Browser/Phantom/V2.0 Widget/LGMW/3.0 MMS/LG-MMS-V1.0/1.2 Java/ASVM/1.1 Profile/MIDP-2.1 Configuration/CLDC-1.1)

• AppEngine-Google

  Example user agent
  AppEngine-Google; ( http://code.google.com/appengine)

• Googlebot

  Example user agent
  Googlebot/1.3

• OOZBOT

  Example user agent
  Mozilla/5.0 ( compatible ; Veooz/1.0 ; +http://www.veooz.com/veoozbot.html )

• Vagabondo

  Example user agent
  Mozilla/4.0 (compatible; Vagabondo/4.0/EU; http://www.wise-guys.nl/)

• Yahoo! Slurp

  Example user agent
  Mozilla/5.0 (compatible; Yahoo! Slurp/3.0; http://help.yahoo.com/help/us/ysearch/slurp)

• Java

  Example user agent
  Mozilla/5.0 (LG-C330 AppleWebkit/531 Browser/Phantom/V2.0 Widget/LGMW/3.0 MMS/LG-MMS-V1.0/1.2 Java/ASVM/1.1 Profile/MIDP-2.1 Configuration/CLDC-1.1)

• IRLbot

  Example user agent
  IRLbot/3.0 (compatible; MSIE 6.0; http://irl.cs.tamu.edu/crawler)

• Bingbot

  Example user agent
  Mozilla/5.0 (compatible;bingbot/2.0;+http://www.bing.com/bingbot.htm)

• IRLbot

  Example user agent
  IRLbot/3.0 (compatible; MSIE 6.0; http://irl.cs.tamu.edu/crawler/)

• Mediapartners-Google

  Example user agent
  Mediapartners-Google

• FeedFetcher-Google

  Example user agent
  Mozilla/5.0 (compatible) Feedfetcher-Google;(+http://www.google.com/feedfetcher.html)

• Moreoverbot

  Example user agent
  Moreoverbot/5.00 (+http://www.moreover.com)

• L.webis

  Example user agent
  L.webis/0.87 (http://webalgo.iit.cnr.it/index.php?pg=lwebis)

• OmniExplorer_Bot

  Example user agent
  OmniExplorer_Bot/6.65a (+http://www.omni-explorer.com) WorldIndexer

• IssueCrawler

  Example user agent
  IssueCrawler

• Java

  Example user agent
  Huawei/1.0/0HuaweiU3300/B100 Browser/Obigo-Browser/Q05A MMS/Obigo-MMS/Q05A SyncML/HW-SyncML/1.0 Java/HWJa/1.0 Profile/MIDP-2.0 Configuration/CLDC-1.1 Player/QTV-Player/5.3

• SBIder

  Example user agent
  SBIder/0.8-dev (SBIder; http://www.sitesell.com/sbider.html; http://support.sitesell.com/contact-support.html)

• Notifixious

  Example user agent
  Notifixious/LinkChecker (http://notifixio.us)

• OOZBOT

  Example user agent
  OOZBOT/0.20 ( -- ; http://www.setooz.com/oozbot.html ; agentname at setooz dot_com )

• KoepaBot

  Example user agent
  Mozilla/5.0 (compatible; KoepaBot BETA; http://www.koepa.nl/bot.html)

• Java

  Example user agent
  LG/KF690/v10a Browser/Obigo-Q05A/3.12 MMS/LG-MMS-V1.0/1.2 Java/ASVM/1.1 Profile/MIDP-2.1 Configuration/CLDC-1.1 UNTRUSTED/1.0

• genieBot

  Example user agent
  genieBot ((http://64.5.245.11/faq/faq.html))

• g2Crawler

  Example user agent
  g2Crawler nobody@airmail.net

• Java

  Example user agent
  LG-BL42k/v10b Browser/Obigo-Q7.3 MMS/LG-MMS-V1.0/1.2 MediaPlayer/LGPlayer/1.0 Java/ASVM/1.1 Profile/MIDP-2.1 Configuration/CLDC-1.1

• BinGet

  Example user agent
  BinGet/1.00.A (http://www.bin-co.com/php/scripts/load/)

• FAST Enterprise Crawler

  Example user agent
  FAST Enterprise Crawler/6 (www.fastsearch.com)

• Java

  Example user agent
  Mozilla/5.0 (LG-C510 AppleWebKit/531 MMS/LG-MMS-V1.0/1.2 MediaPlayer/LGPlayer/1.0 Java/ASVM/1.1 Profile/MIDP-2.1),gzip(gfe),gzip(gfe)

• Vagabondo

  Example user agent
  Mozilla/4.0 (compatible; Vagabondo/4.0Beta; webcrawler at wise-guys dot nl; http://webagent.wise-guys.nl/)

• Java

  Example user agent
  SonyEricssonw580i/R4EA Java/SEMC-Java/2.0 Profile/MIDP-2.0 Configuration/CLDC-1.1 sonyericssonw580i

• Xaldon_WebSpider

  Example user agent
  Xaldon_WebSpider/2.0.b1

• Baiduspider

  Example user agent
  Baiduspider

• Sosospider

  Example user agent
  Sosospider+(+help.soso.com/webspider.htm)

• TurnitinBot

  Example user agent
  TurnitinBot/1.5 (http://www.turnitin.com/robot/crawlerinfo.html)

• PageBitesHyperBot

  Example user agent
  PageBitesHyperBot/600 (http://www.pagebites.com/)

• Teoma

  Example user agent
  Mozilla/2.0 (compatible; Ask Jeeves/Teoma)

• Java

  Example user agent
  Mozilla/5.0 (LG-T375 AppleWebkit/531 Browser/Phantom/V2.0 Widget/LGMW/3.0 MMS/LG-MMS-V1.0/1.2 Java/ASVM/1.1 Profile/MIDP-2.1

• Java

  Example user agent
  SEC-SGHE900/10 SAMSUNG JAVA CLIENT

• Java

  Example user agent
  Mozilla/5.0 (LG-T565b AppleWebkit/531 Browser/Phantom/V2.0 Widget/LGMW/3.0 MMS/LG-MMS-V1.0/1.2 Java/ASVM/1.1 Profile/MIDP-2.1 Configuration/CLDC-1.1)

• 008

  Example user agent
  Mozilla/5.0 (compatible; 008/0.85; http://www.80legs.com/webcrawler.html) Gecko/2008032620

• NewsGator

  Example user agent
  NewsGator/2.0 Bot (http://www.newsgator.com)

• Java

  Example user agent
  LG-P520/V100 Obigo/Q7.3 MMS/LG-MMS-V1.1/1.2 MediaPlayer/LGPlayer/1.0 Java/ASVM/1.1 Profile/MIDP-2.1 Configuration/CLDC-1.1

• Java

  Example user agent
  Mozilla/5.0 (LG-C365 AppleWebkit/531 Browser/Phantom/V2.0 Widget/LGMW/3.0 MMS/LG-MMS-V1.0/1.2 Java/ASVM/1.1 Profile/MIDP-2.1 Configuration/CLDC-1.1)

• silk

  Example user agent
  silk/1.0 (+http://www.slider.com/silk.htm)/3.7

• Baiduspider

  Example user agent
  Baiduspider+(+http://www.baidu.jp/spider/)

• htdig

  Example user agent
  htdig/3.1.5 (webmaster@online-medien.de)

• Java

  Example user agent
  SmartAndSimpleWebCrawler/1.3 (https://crawler.dev.java.net),gzip(gfe),gzip(gfe)

• Xenu Link Sleuth

  Example user agent
  Xenu Link Sleuth 1.2h

• Googlebot

  Example user agent
  Googlebot-News (2.3.3, ruby 1.9.3 (2013-11-22))

• AppEngine-Google

  Example user agent
  AppEngine-Google; (+http://code.google.com/appengine; appid: xbmacx)

• TweetedTimes Bot

  Example user agent
  Mozilla/5.0 (compatible; TweetedTimes Bot/1.0 (resolver); +http://tweetedtimes.com)

• Exabot

  Example user agent
  Mozilla/5.0 (compatible; Exabot/3.0; +http://www.exabot.com/go/robot)

• OmniExplorer_Bot

  Example user agent
  OmniExplorer_Bot/6.70 (+http://www.omni-explorer.com) WorldIndexer

• Python-urllib

  Example user agent
  Python-urllib/2.4

• OmniExplorer_Bot

  Example user agent
  OmniExplorer_Bot/5.25 (+http://www.omni-explorer.com) WorldIndexer

• OmniExplorer_Bot

  Example user agent
  OmniExplorer_Bot/5.28 (+http://www.omni-explorer.com) WorldIndexer

• 008

  Example user agent
  Mozilla/5.0 (compatible; 008/0.83; http://www.80legs.com/webcrawler.html;) Gecko/2008032620

• sogou spider

  Example user agent
  Sogou web spider/3.0( http://www.sogou.com/docs/help/webmasters.htm#07)

• Scrubby

  Example user agent
  Mozilla/5.0 (compatible; Scrubby/2.2; http://www.scrubtheweb.com/)

• Java

  Example user agent
  Mozilla/5.0 (LG-C207 AppleWebkit/531 Browser/Phantom/V2.0 Widget/LGMW/3.0 MMS/LG-MMS-V1.0/1.2 Java/ASVM/1.1 Profile/MIDP-2.1 Configuration/CLDC-1.1)

• Java

  Example user agent
  LG-T310-Orange/V10a Obigo/Q7.3 MMS/LG-MMS-V1.1/1.2 MediaPlayer/LGPlayer/1.0 Java/ASVM/1.1 Profile/MIDP-2.1 Configuration/CLDC-1.1

• Googlebot

  Example user agent
  Mozilla/5.0 (compatible; Googlebot/2.1; http://www.google.com/bot.html)

• updated

  Example user agent
  updated/0.1-alpha (updated crawler; http://www.updated.com; crawler@updated.com)

• Bingbot

  Example user agent
  Mozilla/5.0 (compatible; bingbot/2.0; )"

• ichiro

  Example user agent
  ichiro/1.0 (ichiro@nttr.co.jp)

• NetSeer Crawler

  Example user agent
  Mozilla/5.0 (compatible; NetSeer crawler/2.0; +http://www.netseer.com/crawler.html; crawler@netseer.com)

• Java

  Example user agent
  Mozilla/5.0 (LG-T505 AppleWebkit/531 Browser/Phantom/V2.0 Widget/LGMW/3.0 MMS/LG-MMS-V1.0/1.2 Java/A

• Speedy Spider

  Example user agent
  Speedy Spider (Entireweb; Beta/1.1; http://www.entireweb.com/about/search_tech/speedyspider/)

• IRLbot

  Example user agent
  IRLbot/2.0 (+http://irl.cs.tamu.edu/crawler)

• VoilaBot

  Example user agent
  Mozilla/4.0 (compatible; MSIE 5.0; Windows 95) VoilaBot BETA 1.2 (http://www.voila.com/)

• Java

  Example user agent
  Java1.2.2-JDeveloper

• MVAClient

  Example user agent
  MVAClient

• ScoutJet

  Example user agent
  Mozilla/5.0 (compatible; ScoutJet; +http://www.scoutjet.com/)

• Java

  Example user agent
  Mozilla/5.0 (LG-GD880-Orange/V09d AppleWebKit/531 MMS/LG-MMS-V1.0/1.2 MediaPlayer/LGPlayer/1.0 Java/ASVM/1.1 Profile/MIDP-2.1 Configuration/CLDC-1.1)

• ia_archiver

  Example user agent
  ia_archiver/8.7 (Windows NT 5.0; )

• zspider

  Example user agent
  zspider/0.9-dev http://feedback.redkolibri.com/

• Java

  Example user agent
  Huawei/1.0/0HuaweiU3315/B100 Browser/Obigo-Browser/Q05A MMS/Obigo-MMS/Q05A SyncML/HW-SyncML/1.0 Java/HWJa/1.0 Profile/MIDP-2.0 Configuration/CLDC-1.1 Player/QTV-Player/5.3

• Java

  Example user agent
  Java1.3.1_04

• FAST Enterprise Crawler

  Example user agent
  FAST Enterprise Crawler/5.3.4 (crawler@fast.no)

• Yahoo! Slurp China

  Example user agent
  Mozilla/5.0 (compatible; Yahoo! Slurp China; http://misc.yahoo.com.cn/help.html)

• Jaxified Bot

  Example user agent
  Jaxified Bot (+http://www.jaxified.com/crawler/)

• Vagabondo

  Example user agent
  Mozilla/5.0 (compatible; Vagabondo/2.1; webcrawler at wise-guys dot nl; http://webagent.wise-guys.nl/)

• Python-urllib

  Example user agent
  Python-urllib/2.6

• 008

  Example user agent
  Mozilla/5.0 (Linux; U; Android 2.3.4; ja-jp; 008Z Build/GINGERBREAD) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1

• WDG_Validator

  Example user agent
  WDG_Validator/1.6.5

• msnbot

  Example user agent
  msnbot/1.1 ( http://search.msn.com/msnbot.htm)

• Java

  Example user agent
  HuaweiU120/1.0/B127 Browser/Obigo-Browser/Q04A MMS/Obigo-MMS/Q04A SyncML/HW-SyncML/1.0 Java/QVM/4.1 Profile/MIDP-2.0 Configuration/CLDC-1.1

• FAST-WebCrawler

  Example user agent
  FAST-WebCrawler/3.8 (atw-crawler at fast dot no; http://fast.no/support/crawler.asp)

• Amaya

  Example user agent
  amaya/10 libwww/5.4.0

• ichiro

  Example user agent
  DoCoMo/2.0 P900i(c100;TB;W24H11) (compatible; ichiro/mobile goo; +http://help.goo.ne.jp/help/article/1142/)

• Java

  Example user agent
  SonyEricssonw760i/R4EA Java/SEMC-Java/2.0 Profile/MIDP-2.0 Configuration/CLDC-1.1 UNTRUSTED/1.0, sonyericssonw760i

• FurlBot

  Example user agent
  Mozilla/4.0 compatible FurlBot/Furl Search 2.0 (FurlBot; http://www.furl.net; wn.furlbot@looksmart.net)

• msnbot

  Example user agent
  "msnbot-NewsBlogs/2.0b (+http://search.msn.com/msnbot.htm)

• GreatNews

  Example user agent
  GreatNews/1.0

• Java

  Example user agent
  Mozilla/5.0 (LG-C195 AppleWebkit/531 Browser/Phantom/V2.0 Widget/LGMW/3.0 MMS/LG-MMS-V1.0/1.2 Java/ASVM/1.1 Profile/MIDP-2.1 Configuration/CLDC-1.1)

• ZyBorg

  Example user agent
  Mozilla/4.0 compatible ZyBorg/1.0 (wn-16.zyborg@looksmart.net; http://www.WISEnutbot.com)

• mxbot

  Example user agent
  Mozilla/5.0 (compatible; mxbot/1.0; http://www.chainn.com/mxbot.html)

• NewsGator

  Example user agent
  NewsGatorOnline/2.0 (http://www.newsgator.com; 2 subscribers)

• SeznamBot

  Example user agent
  Mozilla/5.0 (compatible; SeznamBot/3.2-test1; +http://fulltext.sblog.cz/)

• Java

  Example user agent
  Mozilla/5.0 (Vodafone/1.0/LG-GC900/V10b Browser/Obigo-Q7.1 MMS/LG-MMS-V1.0/1.2 MediaPlayer/LGPlayer/1.0 Java/ASVM/1.1 Profile/MIDP-2.1 Configuration/CLDC-1.1)

• FAST-WebCrawler

  Example user agent
  FAST-WebCrawler/3.7 (atw-crawler at fast dot no; http://fast.no/support/crawler.asp)

• findlinks

  Example user agent
  findlinks/2.6 ( http://wortschatz.uni-leipzig.de/findlinks/)

• yacy

  Example user agent
  yacybot (freeworld/global; amd64 Linux 3.2.0-4-amd64; java 1.7.0_25; Europe/en) http://yacy.net/bot.html

• SiteBar

  Example user agent
  SiteBar/3.3.5 (Bookmark Server; http://sitebar.org/)

• BeslistBot

  Example user agent
  Mozilla/5.0 (compatible; BeslistBot; nl; BeslistBot 1.0; http://www.beslist.nl/

• HappyFunBot

  Example user agent
  HappyFunBot/1.1 ( http://www.happyfunsearch.com/bot.html)

• Java

  Example user agent
  Huawei/1.0/0HuaweiU3311/B100 Browser/Obigo-Browser/Q05A MMS/Obigo-MMS/Q05A SyncML/HW-SyncML/1.0 Java/HWJa/1.0 Profile/MIDP-2.0 Configuration/CLDC-1.1 Player/QTV-Player/5.3

• Peew

  Example user agent
  Mozilla/5.0 (compatible; Peew/1.0; http://www.peew.de/crawler/)

• Java

  Example user agent
  Huawei/1.0/0HuaweiU5900/B000 Browser/Obigo-Browser/Q05A HUAWEI-U5900 HuaweiJava/1.0 Profile/MIDP-2.0 Configuration/CLDC-1.1

• Python-urllib

  Example user agent
  Python-urllib/2.5

• YahooSeeker

  Example user agent
  YahooSeeker/1.0 (compatible; Mozilla 4.0; MSIE 5.5; http://help.yahoo.com/help/us/shop/merchant/)

• MagpieRSS

  Example user agent
  MagpieRSS/0.7 ( http://magpierss.sf.net)

• OmniExplorer_Bot

  Example user agent
  OmniExplorer_Bot/5.01 (+http://www.omni-explorer.com) WorldIndexer

• Moreoverbot

  Example user agent
  Moreoverbot/5.00 (+http://www.moreover.com; webmaster@moreover.com)

• YandexBot

  Example user agent
  Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)

• cosmos

  Example user agent
  cosmos/0.9_(robot@xyleme.com)

• msnbot

  Example user agent
  msnbot/2.0b (+http://search.msn.com/msnbot.htm)._

• psbot

  Example user agent
  psbot-image (+http://www.picsearch.com/bot.html)

• Googlebot

  Example user agent
  Googlebot v2.1 (+http://www.google.com/bot.html) (http://www.openwebspider.org/)

• ia_archiver

  Example user agent
  ia_archiver (+http://www.alexa.com/site/help/webmasters; crawler@alexa.com)

• Java

  Example user agent
  0HuaweiU5705/Huawei/1.0/B100 Browser/Obigo-Browser/Q05A MMS/Obigo-MMS/Q05A SyncML/HW-SyncML/1.0 Java/HWJa/1.0 Profile/MIDP-2.0 Configuration/CLDC-1.1 Player/QTV-Player/5.3

• Zao

  Example user agent
  Zao/0.1 (http://www.kototoi.org/zao/)

• Java

  Example user agent
  LG-E500/V100 Obigo/Q7.3 MMS/LG-MMS-V1.1/1.2 MediaPlayer/LGPlayer/1.0 Java/ASVM/1.1 Profile/MIDP-2.1 Configuration/CLDC-1.1,gzip(gfe)

• SeznamBot

  Example user agent
  Mozilla/5.0 (compatible; SeznamBot/3.1-test1; +http://fulltext.sblog.cz/)

• Uzard Web

  Example user agent
  Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; WOW64; SV1; uZardWeb/1.0; Server_CN)

• ZyBorg

  Example user agent
  Mozilla/4.0 compatible ZyBorg/1.0 (wn.zyborg@looksmart.net; http://www.WISEnutbot.com)

• Java

  Example user agent
  Java1.3.1_06

• Java

  Example user agent
  Mozilla/5.0 (compatible; Ex-Crawler/v0.1.5ALPHA SVN rev58; powered by ex-crawler; +http://www.ex-crawler.de/) Java/1.6.0_20,gzip(gfe),gzip(gfe)

• Java

  Example user agent
  Mozilla/5.0 (LG-T395 AppleWebkit/531 Browser/Phantom/V2.0 Widget/LGMW/3.0 MMS/LG-MMS-V1.0/1.2 Java/ASVM/1.1 Profile/MIDP-2.1 Configuration/CLDC-1.1) UNTRUSTED/1.0

• Googlebot

  Example user agent
  Googlebot/2.1 ( http://www.googlebot.com/bot.html)

• Baiduspider

  Example user agent
  Mozilla/5.0+(compatible;+Baiduspider/2.0;++http://www.baidu.com/search/spider.html)

• oegp

  Example user agent
  oegp v. 1.3.0

• Bingbot

  Example user agent
  Mozilla/5.0 (compatible; bingbot/2.0 +http://www.bing.com/bingbot.htm)

• Googlebot-Image

  Example user agent
  Googlebot-Image/1.0

• Java

  Example user agent
  Huawei/1.0/0HuaweiU2800/B000 Browser/Obigo-Browser/Q05A HUAWEI-U2800 HuaweiJava/1.0 Profile/MIDP-2.0 Configuration/CLDC-1.1

• voyager

  Example user agent
  voyager/2.0 (http://www.kosmix.com/crawler.html)

• ZyBorg

  Example user agent
  Mozilla/4.0 compatible ZyBorg/1.0 DLC (wn.zyborg@looksmart.net; http://www.WISEnutbot.com)

• ZyBorg

  Example user agent
  Mozilla/4.0 compatible ZyBorg/1.0

• msnbot

  Example user agent
  MSNBot/Nutch-1.5.1

• Googlebot

  Example user agent
  Googlebot/2.1(+http://www.googlebot.com/bot.html)

• Java

  Example user agent
  Mozilla/5.0 (LG-T385 AppleWebkit/531 Browser/Phantom/V2.0 Widget/LGMW/3.0 MMS/LG-MMS-V1.0/1.2 Java/ASVM/1.1 Profile/MIDP-2.1 Configuration/CLDC-1.1)

• Googlebot

  Example user agent
  Mozilla/5.0 (compatible;acapbot/0.1.;treat like Googlebot)

• Java

  Example user agent
  Mozilla/5.0 (LG-T515 AppleWebkit/531 Browser/Phantom/V2.0 Widget/LGMW/3.0 MMS/LG-MMS-V1.0/1.2 Java/ASVM/1.1 Profile/MIDP-2.1 Configuration/CLDC-1.1)

• GurujiBot

  Example user agent
  Mozilla/5.0 GurujiBot/1.0 ( http://www.guruji.com/en/WebmasterFAQ.html)

• FeedFetcher-Google

  Example user agent
  FeedFetcher-Google; ( http://www.google.com/feedfetcher.html)

• Linguee Bot

  Example user agent
  Linguee Bot (bot@linguee.com)

• ZyBorg

  Example user agent
  Mozilla/4.0 compatible ZyBorg/1.0 Dead Link Checker (wn.zyborg@looksmart.net; http://www.WISEnutbot.com)

• Baiduspider

  Example user agent
  Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)

• FAST-WebCrawler

  Example user agent
  FAST-WebCrawler/2.2.11 (crawler@fast.no; http://www.fast.no/faq/faqfastwebsearch/faqfastwebcrawler.html)

• Java

  Example user agent
  ICE Browser/5.05 (Java 1.4.0; Windows 2000 5.0 x86)

• Xenu Link Sleuth

  Example user agent
  Xenu Link Sleuth/1.3.8

• DataparkSearch

  Example user agent
  DataparkSearch/4.37-23012006 ( http://www.dataparksearch.org/)

• AppEngine-Google

  Example user agent
  AppEngine-Google

• Yeti

  Example user agent
  Yeti/1.0 (NHN Corp.; http://help.naver.com/robots/)

• Teoma

  Example user agent
  Inne: Mozilla/2.0 (compatible; Ask Jeeves/Teoma; http://sp.ask.com/docs/about/tech_crawling.html)

• MSRBOT

  Example user agent
  MSRBOT (http://research.microsoft.com/research/sv/msrbot)

• Baiduspider

  Example user agent
  Baiduspider+(+http://www.baidu.com/search/spider.htm)

• ichiro

  Example user agent
  ichiro/3.0 (http://help.goo.ne.jp/door/crawler.html)

• Java

  Example user agent
  HUAWEI-U2800/001.00 Browser/Obigo-Browser/Q05A MMS/UNIBOX/V1.2 HuaweiJava/1.0 Profile/MIDP-2.0 Configuration/CLDC-1.1

• Java

  Example user agent
  Huawei/1.0/0HuaweiU3315h/B100 Browser/Obigo-Browser/Q05A MMS/Obigo-MMS/Q05A SyncML/HW-SyncML/1.0 Java/HWJa/1.0 Profile/MIDP-2.0 Configuration/CLDC-1.1 Player/QTV-Player/5.3

• Java

  Example user agent
  Java1.4.0_03

• IRLbot

  Example user agent
  IRLbot/2.0 (compatible; MSIE 6.0; http://irl.cs.tamu.edu/crawler)

• IRLbot

  Example user agent
  IRLbot/1.0 ( http://irl.cs.tamu.edu/crawler)

• Java

  Example user agent
  JNLP/1.7.0 javaws/10.17.2.02 (<internal>) Java/1.7.0_17

• msnbot

  Example user agent
  msnbot/2.0 (+http://search.msn.com/msnbot.htm)

• Java

  Example user agent
  Mozilla/5.0 (LG-T530-Orange/V10b AppleWebkit/531 Browser/Phantom/V2.0 Widget/LGMW/3.0 MMS/LG-MMS-V1.0/1.2 Java/ASVM/1.1 Profile/MIDP-2.1 Configuration/CLDC-1.1)

• Mozilla

  Example user agent
  Hybridcast/1.0 (;008045;PANATV1308;2;2;)

• Sosospider

  Example user agent
  Sosospider+(+http://help.soso.com/webspider.htm)

• Nusearch Spider

  Example user agent
  NuSearch Spider www.nusearch.com

• Urlfilebot

  Example user agent
  Mozilla/5.0 (compatible; Urlfilebot/2.2; +http://urlfile.com/bot.html)

• Teoma

  Example user agent
  Mozilla/2.0 (compatible; Ask Jeeves/Teoma; +http://sp.ask.com/docs/about/tech_crawling.html)

• Snappy

  Example user agent
  Snappy/1.1 ( http://www.urltrends.com/ )

• HuaweiSymantecSpider

  Example user agent
  Huaweisymantecspider (compatible; MSIE 7.0; Windows NT 5.1; Trident/4.0; .NET CLR 2.0.50727),gzip(gfe)

• Java

  Example user agent
  Mozilla/5.0 (Vodafone/1.0/LG-BL40/V10a Browser/Obigo-Q7.3 MMS/LG-MMS-V1.0/1.2 MediaPlayer/LGPlayer/1.0 Java/ASVM/1.1 Profile/MIDP-2.1 Configuration/CLDC-1.1)

• Scrubby

  Example user agent
  Mozilla/5.0 (compatible; Scrubby/2.2; +http://www.scrubtheweb.com/)

• Speedy Spider

  Example user agent
  Mozilla/5.0 (compatible; Speedy Spider; http://www.entireweb.com/about/search_tech/speedy_spider/)

• Java

  Example user agent
  Huawei/1.0/0HuaweiU1305/B100 Browser/Obigo-Browser/Q05A MMS/Obigo-MMS/Q05A SyncML/HW-SyncML/1.0 Java/HWJa/1.0 Profile/MIDP-2.0 Configuration/CLDC-1.1 Player/QTV-Player/5.3

• Sensis Web Crawler

  Example user agent
  Sensis Web Crawler (search_comments\at\sensis\dot\com\dot\au)

• igdeSpyder

  Example user agent
  igdeSpyder (compatible; igde.ru; +http://igde.ru/doc/tech.html)

• Baiduspider

  Example user agent
  Baiduspider ( http://www.baidu.com/search/spider.htm)

• Java

  Example user agent
  Huawei/1.0/0HuaweiU5700/B100 Browser/Obigo-Browser/Q05A MMS/Obigo-MMS/Q05A SyncML/HW-SyncML/1.0 Java/HWJa/1.0 Profile/MIDP-2.

• Java

  Example user agent
  LG/KC910e/V10c Browser/Obigo-Q7.1 MMS/LG-MMS-V1.0/1.2 MediaPlayer/LGPlayer/1.0 Java/ASVM/1.1 Profile/MIDP-2.1 Configuration/CLDC-1.1

• Java

  Example user agent
  LG-C299/V100 Obigo/Q05A MMS/LG-MMS-V1.0/1.2 Java/ASVM/1.1 Profile/MIDP-2.1 Configuration/CLDC-1.1

• Vagabondo

  Example user agent
  Mozilla/4.0 (compatible; Vagabondo/2.2; webcrawler at wise-guys dot nl; http://webagent.wise-guys.nl/)

• Xenu Link Sleuth

  Example user agent
  Xenu Link Sleuth/1.3.7

• TwengaBot

  Example user agent
  TwengaBot

• Java

  Example user agent
  Huawei/1.0/0HuaweiU1315/B100 Browser/Obigo-Browser/Q05A MMS/Obigo-MMS/Q05A SyncML/HW-SyncML/1.0 Java/HWJa/1.0 Profile/MIDP-2.0 Configuration/CLDC-1.1 Player/QTV-Player/5.3

• Java

  Example user agent
  Huawei/1.0/0MTC-752/B000 Browser/Obigo-Browser/Q05A MTC-752 HuaweiJava/1.0 Profile/MIDP-2.0 Configuration/CLDC-1.1

• Java

  Example user agent
  Java1.4.0_01

• REL Link Checker Lite

  Example user agent
  REL Link Checker Lite 1.0

• ScoutJet

  Example user agent
  Mozilla/5.0 (compatible; ScoutJet; http://www.scoutjet.com/)

• WWW-Mechanize

  Example user agent
  WWW-Mechanize/1.64

• ShopWiki

  Example user agent
  ShopWiki/1.0 ( +http://www.shopwiki.com/wiki/Help:Bot)

• Googlebot

  Example user agent
  Googlebot-News

• Java

  Example user agent
  HuaweiC6100 Java/HWJa/1.0 Profile/MIDP-2.0 Configuration/CLDC-1.1 UNTRUSTED/1.0

• Java

  Example user agent
  Apache-HttpClient/UNAVAILABLE (java 1.4)

• HuaweiSymantecSpider

  Example user agent
  Huaweisymantecspider (compatible; MSIE 8.0; DSE-support@huaweisymantec.com)

• Java

  Example user agent
  Java/1.17

• yoogliFetchAgent

  Example user agent
  yoogliFetchAgent/0.1

• TinEye

  Example user agent
  TinEye/1.1 (http://tineye.com/crawler.html)

• Speedy Spider

  Example user agent
  Speedy Spider (Entireweb; Beta/1.0; http://www.entireweb.com/about/search_tech/speedyspider/)

• Nusearch Spider

  Example user agent
  Nusearch Spider (compatible; MSIE 6.0)

• online link validator

  Example user agent
  online link validator (http://www.dead-links.com/)

• silk

  Example user agent
  Silk/1.0

• Java

  Example user agent
  LG-GU285f Browser/Obigo-Q7.3 MMS/LG-MMS-V1.0/1.2 MediaPlayer/LGPlayer/1.0 Java/ASVM/1.1 Profile/MIDP-2.1 Configuration/CLDC-1.1

• yrspider

  Example user agent
  Mozilla/5.0 (Android 3.0; YRSpider; +http://www.yunrang.com/yrspider.html)

• Java

  Example user agent
  LG-A390/V100 Obigo/Q05A MMS/LG-MMS-V1.0/1.2 Java/ASVM/1.1 Profile/MIDP-2.1 Configuration/CLDC-1.1

• lwp-trivial

  Example user agent
  lwp-trivial/1.38

• LexxeBot

  Example user agent
  LexxeBot/1.0 (lexxebot@lexxe.com)

• FeedFetcher-Google

  Example user agent
  Feedfetcher-Google; (+http://www.google.com/feedfetcher.html; 2 subscribers; feed-id=17860707833818568603)

• SeznamBot

  Example user agent
  SeznamBot/3.0 (+http://fulltext.sblog.cz/)

• OmniExplorer_Bot

  Example user agent
  OmniExplorer_Bot/6.60 (+http://www.omni-explorer.com) WorldIndexer

• SiteBar

  Example user agent
  SiteBar/3.3.8 (Bookmark Server; http://sitebar.org/)

• Larbin

  Example user agent
  LarbinWebCrawler (internet@bredband.net)

• Baiduspider

  Example user agent
  baiduspider-mobile-gate

• Bingbot

  Example user agent
  Mozilla/5.0 (compatible; bingbot/2.1; +http://www.bing.com/bingbot.htm)

• Java

  Example user agent
  Vodafone/1.0/0vodafone810/B626 Browser/Obigo-browser/Q05A MMS/Obigo-MMS/Q05A SyncML/HW-SyncML/1.0 Java/HWJA/1.0 Profile/MIDP-2.0 Configuration/CLDC-1.1

• SearchSight

  Example user agent
  SearchSight/2.0 (http://SearchSight.com/)

• Googlebot

  Example user agent
  Locubot (compatible; Googlebot; msnbot)

• Charlotte

  Example user agent
  Mozilla/5.0 (compatible; Charlotte/1.0b; charlotte@beta.spider.com)

• sogou spider

  Example user agent
  Sogou inst spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm

• ZyBorg

  Example user agent
  Mozilla/4.0 compatible ZyBorg/1.0 Dead Link Checker (wn.dlc@looksmart.net; http://www.WISEnutbot.com)

• sogou spider

  Example user agent
  Sogou Pic Spider/3.0(+http://www.sogou.com/docs/help/webmasters.htm#07)

• Java

  Example user agent
  Mozilla/5.0 (LG-T510 AppleWebkit/531 Browser/Phantom/V2.0 Widget/LGMW/3.0 MMS/LG-MMS-V1.0/1.2 Java/ASVM/1.1 Profile/MIDP-2.1 Configuration/CLDC-1.1)

• GurujiBot

  Example user agent
  Mozilla/5.0 GurujiBot/1.0 (+http://www.guruji.com/en/WebmasterFAQ.html)

• Baiduspider

  Example user agent
  Baiduspider-image+(+http://www.baidu.com/search/spider.htm)

• Java

  Example user agent
  Mozilla/5.0 (LG-C205 AppleWebkit/531 Browser/Phantom/V2.0 Widget/LGMW/3.0 MMS/LG-MMS-V1.0/1.2 Java/ASVM/1.1 Profile/MIDP-2.1 Configuration/CLDC-1.1)

• truwoGPS

  Example user agent
  truwoGPS/1.0 (GNU/Linux; U; i686; en-US; +http://www.lan4lano.net/browser.html )

• Exabot

  Example user agent
  Mozilla/5.0 (compatible; Alexabot/1.0; +http://www.alexa.com/help/certifyscan; certifyscan@alexa.com)

• Python-urllib

  Example user agent
  Python-urllib/3.1

• Java

  Example user agent
  Mozilla/5.0 (Vodafone/1.0/LG-GS290/V10b Browser/Obigo-Q7.3 MMS/LG-MMS-V1.0/1.2 MediaPlayer/LGPlayer/1.0 Java/ASVM/1.1 Profile/MIDP-2.1 Configuration/CLDC-1.1)

• msnbot

  Example user agent
  msnbot/0.9 (+http://search.msn.com/msnbot.htm)

• Vagabondo

  Example user agent
  Mozilla/3.0 (Vagabondo/2.0 MT; webcrawler@NOSPAMexperimental.net; http://aanmelden.ilse.nl/?aanmeld_mode=webhints)

• FDSE

  Example user agent
  Mozilla/4.0 (compatible: FDSE robot)

• msnbot

  Example user agent
  msnbot/2.1

• Bimbot

  Example user agent
  Bimbot/1.0

• suggybot

  Example user agent
  Mozilla/5.0 (compatible; suggybot v0.01a, http://blog.suggy.com/was-ist-suggy/suggy-webcrawler/)

• AbiLogicBot

  Example user agent
  Mozilla/5.0 (compatible; AbiLogicBot/1.0; +http://www.abilogic.com)

• msnbot

  Example user agent
  msnbot/2.0b ( http://search.msn.com/msnbot.htm)

• B-l-i-t-z-B-O-T

  Example user agent
  Mozilla/4.0 (compatible; B-l-i-t-z-B-O-T)

• htdig

  Example user agent
  htdig/3.1.5 (root@localhost)

• DotBot

  Example user agent
  Mozilla/5.0 (compatible; DotBot/1.1; http://www.dotnetdotcom.org/, crawler@dotnetdotcom.org)

• Xenu Link Sleuth

  Example user agent
  Xenu Link Sleuth 1.2i

• Reciprocal Link System PRO

  Example user agent
  InfoWizards Reciprocal Link System PRO - (http://www.infowizards.com)

• Python-urllib

  Example user agent
  Python-urllib/1.16

• Googlebot

  Example user agent
  Mozilla (Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html))

• PostPost

  Example user agent
  PostPost/1.0 (+http://postpo.st/crawlers)

• NutchCVS

  Example user agent
  NutchCVS/0.06-dev (Nutch running at UW; http://www.nutch.org/docs/en/bot.html; sycrawl@cs.washington.edu)

• Xenu Link Sleuth

  Example user agent
  Xenu Link Sleuth 1.2e

• iaskspider

  Example user agent
  iaskspider

• ichiro

  Example user agent
  DoCoMo/2.0 P900i(c100;TB;W24H11) (compatible; ichiro/mobile goo;+http://search.goo.ne.jp/option/use/sub4/sub4-1/)

• Speedy Spider

  Example user agent
  Speedy Spider (Entireweb; Beta/1.3; http://www.entireweb.com/about/search_tech/speedyspider/)

• Java

  Example user agent
  Mozilla/5.0 (LG-T395 AppleWebkit/531 Browser/Phantom/V2.0 Widget/LGMW/3.0 MMS/LG-MMS-V1.0/1.2 Java/ASVM/1.1 Profile/MIDP-2.1 Configuration/CLDC-1.1)

• Pompos

  Example user agent
  Pompos/1.2 http://pompos.iliad.fr

• msnbot

  Example user agent
  msnbot-media/2.0b (+http://search.msn.com/msnbot.htm)

• NetSeer Crawler

  Example user agent
  compatible; Netseer crawler/2.0; +http://www.netseer.com/crawler.html; crawler@netseer.com

• everyfeed-spider

  Example user agent
  everyfeed-spider/2.0 (http://www.everyfeed.com)

• TweetedTimes Bot

  Example user agent
  Mozilla/5.0 (compatible; TweetedTimes Bot/1.0; +http://tweetedtimes.com)

• htdig

  Example user agent
  htdig/3.1.6 (unconfigured@htdig.searchengine.maintainer)

• Scrubby

  Example user agent
  Scrubby/2.2 (http://www.scrubtheweb.com/)

• Java

  Example user agent
  MQQBrowser/Mini2.2 (Nokia5230/50.0.001/sw_platform=S60;sw_platform_version=5.0;java_build_version=1.4.43)

• msnbot

  Example user agent
  msnbot-media/1.0 ( http://search.msn.com/msnbot.htm)

• Orbiter

  Example user agent
  Orbiter (+http://www.dailyorbit.com/bot.htm)

• ScoutJet

  Example user agent
  Mozilla/5.0 (compatible; Blekkobot; ScoutJet; +http://blekko.com/about/blekkobot)

• Java

  Example user agent
  Java1.2.1

• Girafabot

  Example user agent
  Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; SV1; .NET CLR 1.1.4322; Girafabot [girafa.com])

• Python-urllib

  Example user agent
  Python-urllib/2.0a1

• Java

  Example user agent
  Java/1.7

• Speedy Spider

  Example user agent
  Speedy Spider (http://www.entireweb.com/about/search_tech/speedyspider/)

• IRLbot

  Example user agent
  IRLbot/2.0 ( http://irl.cs.tamu.edu/crawler)

• Vagabondo

  Example user agent
  Mozilla/4.0 (compatible; Vagabondo/2.3; webcrawler at wise-guys dot nl; http://webagent.wise-guys.nl/)

• Googlebot

  Example user agent
  googlebot

• UniversalFeedParser

  Example user agent
  UniversalFeedParser/3.3 +http://feedparser.org/

• YandexImages

  Example user agent
  Mozilla/5.0 (compatible; YandexImages/3.0; +http://yandex.com/bots)

• Zao

  Example user agent
  Zao-Crawler

• sogou spider

  Example user agent
  Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)

• Girafabot

  Example user agent
  Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 4.0; Girafabot; girafabot at girafa dot com; http://www.girafa.com)

• YandexMetrika

  Example user agent
  Mozilla/5.0 (compatible; YandexMetrika/2.0; +http://yandex.com/bots)

• Accoona-AI-Agent

  Example user agent
  Accoona-AI-Agent/1.1.1 (crawler at accoona dot com)

• LinkExaminer

  Example user agent
  LinkExaminer/1.01 (Windows)

• SeznamBot

  Example user agent
  Mozilla/5.0 (compatible; SeznamBot/3.2; +http://fulltext.sblog.cz/)

• hl_ftien_spider

  Example user agent
  hl_ftien_spider_v1.1

• MSRBOT

  Example user agent
  MSRBOT (http://research.microsoft.com/research/sv/msrbot/)

• Websquash.com

  Example user agent
  Websquash.com (Add url robot)

• YodaoBot

  Example user agent
  Mozilla/5.0 (compatible; YodaoBot/1.0; http://www.yodao.com/help/webmaster/spider/; )

• SandCrawler

  Example user agent
  SandCrawler - Compatibility Testing

• MojeekBot

  Example user agent
  Mozilla/5.0 (compatible; MojeekBot/0.6; http://www.mojeek.com/bot.html)

• Java

  Example user agent
  Java1.4.0

• msnbot

  Example user agent
  MSNBOT/0.1 (http://search.msn.com/msnbot.htm)

• Larbin

  Example user agent
  08_800w_web1 (larbin2.6.3@unspecified.mail)

• SynooBot

  Example user agent
  SynooBot/0.7.1 (SynooBot; http://www.synoo.de/bot.html; webmaster@synoo.com)

• Java

  Example user agent
  Nokia5228/40.1.003/sw_platform=S60;sw_platform_version=5.0;java_build_version=1.4.48

• MojeekBot

  Example user agent
  Mozilla/5.0 (compatible; MojeekBot/0.6; +https://www.mojeek.com/bot.html)

• ichiro

  Example user agent
  ichiro/5.0 (http://help.goo.ne.jp/door/crawler.html)

• Java

  Example user agent
  HuaweiU3100/B000 Browser/Obigo-Browser/Q05A MMS/Obigo-MMS/Q05A SyncML/HW-SyncML/1.0 Java/HWJa/2.0 Profile/MIDP-2.0 Configurat

• NFReader

  Example user agent
  NFReader/1.4.1.0 (http://www.gaijin.at/)

• MagpieRSS

  Example user agent
  MagpieRSS/0.72 \(+http://magpierss.sf.net\)

• Jaxified

  Example user agent
  Jaxified Crawler 1.0a (+http://www.jaxified.com/),gzip(gfe),gzip(gfe)

• Jaxified

  Example user agent
  Jaxified Public RSS Crawler ( http://www.jaxified.com/)

• SiteBot

  Example user agent
  Mozilla/5.0 (compatible; SiteBot/0.1; http://www.sitebot.org/robot/)

• yrspider

  Example user agent
  Mozilla/5.0 (compatible; YRSpider; +http://www.yunrang.com/yrspider.html)

• OmniExplorer_Bot

  Example user agent
  OmniExplorer_Bot/5.20 (+http://www.omni-explorer.com) WorldIndexer

• Java

  Example user agent
  HuaweiU7510/B000 Browser/Obigo-Browser/Q05A MMS/Obigo-MMS/Q05A SyncML/HW-SyncML/1.0 Java/HWJa/2.0 Profile/MIDP-2.0 Configuration/CLDC-1.1 Player/QTV-Player/5.3

• Yeti

  Example user agent
  Yeti/1.0 (+http://help.naver.com/robots/)

• Bingbot

  Example user agent
  Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)

• TinEye

  Example user agent
  TinEye

• Python-urllib

  Example user agent
  Python-urllib/1.17

• BecomeBot

  Example user agent
  Mozilla/5.0 (compatible; BecomeBot/3.0; MSIE 6.0 compatible; +http://www.become.com/site_owners.html)

• Java

  Example user agent
  Java1.3.1

• AddSugarSpiderBot

  Example user agent
  Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0) AddSugarSpiderBot www.idealobserver.com

• RAMPyBot

  Example user agent
  RAMPyBot - www.giveRAMP.com/0.1 (RAMPyBot - www.giveRAMP.com; http://www.giveramp.com/bot.html; support@giveRAMP.com)

• sogou spider

  Example user agent
  Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm

• yacy

  Example user agent
  yacybot (freeworld/global; x86 Windows XP 5.1; java 1.7.0_21; GMT+04:00/ru) http://yacy.net/bot.html

• FAST Enterprise Crawler

  Example user agent
  FAST Enteprise Crawler/6 (www dot fastsearch dot com)

• AppEngine-Google

  Example user agent
  Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots),gzip(gfe) AppEngine-Google; (+http://code.google.com/appengine; appid: twitter-mirror),gzip(gfe),gzip(gfe),gzip(gfe)

• Yeti

  Example user agent
  Yeti-FeedItemCrawler/1.0 (NHN Corp.;+http://help.naver.com/robots/)

• MojeekBot

  Example user agent
  MojeekBot/0.2 (archi; http://www.mojeek.com/bot.html)

• Googlebot

  Example user agent
  DoCoMo//2.0 N905i(c100;TB;W24H16) (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html)

• wf84

  Example user agent
  http://www.almaden.ibm.com/cs/crawler [wf84]

• OmniExplorer_Bot

  Example user agent
  OmniExplorer_Bot/4.80 (+http://www.omni-explorer.com) WorldIndexer

• Java

  Example user agent
  Mozilla/5.0 (Vodafone/1.0/LG-GT350/V10a Browser/Obigo-Q7.3 MMS/LG-MMS-V1.0/1.2 MediaPlayer/LGPlayer/1.0 Java/ASVM/1.1 Profile/MIDP-2.1 Configuration/CLDC-1.1

• CatchBot

  Example user agent
  CatchBot/1.0; +http://www.catchbot.com

• sogou spider

  Example user agent
  Sogou Push Spider/3.0( http://www.sogou.com/docs/help/webmasters.htm#07)

• Googlebot

  Example user agent
  Googlebot/2.1 (+http://www.googlebot.com/bot.html)

• msnbot

  Example user agent
  msnbot/0.11 ( http://search.msn.com/msnbot.htm)

• Java

  Example user agent
  Mozilla/5.0 (LG-T370 AppleWebkit/531 Browser/Phantom/V2.0 Widget/LGMW/3.0 MMS/LG-MMS-V1.0/1.2 Java/ASVM/1.1 Profile/MIDP-2.1 Configuration/CLDC-1.1)

• MQBOT

  Example user agent
  MQbot http://metaquerier.cs.uiuc.edu/crawler

• Googlebot

  Example user agent
  Googlebot/2.2

• Java

  Example user agent
  Huawei/1.0/0HuaweiU1310/B123 Browser/Obigo-Browser/Q05A MMS/Obigo-MMS/Q05A SyncML/HW-SyncML/1.0 Java/HWJa/1.0 Profile/MIDP-2.0 Configuration/CLDC-1.1 Player/QTV-Player/5.3

• Googlebot

  Example user agent
  KDDI-CA34 UP.Browser/6.2.0.10.2.2(GUI)MMP/2.0 (compatible; KDDI-Googlebot-Mobile/2.1; http://www.google.com/bot.html)

• DomainsDB.net MetaCrawler

  Example user agent
  DomainsDB.net MetaCrawler v.0.9.7b (http://domainsdb.net/)

• Googlebot

  Example user agent
  Googlebot/2.1 (+http://www.google.com/bot.html)

• W3C-checklink

  Example user agent
  W3C-checklink/4.0 [4.4] libwww-perl/5.803

• OmniExplorer_Bot

  Example user agent
  OmniExplorer_Bot/6.62 (+http://www.omni-explorer.com) WorldIndexer

• Speedy Spider

  Example user agent
  Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) Speedy Spider (http://www.entireweb.com/about/search_tech/speedy_spider/)

• SiteBot

  Example user agent
  Mozilla/5.0 (compatible; SiteBot/0.1; +http://www.sitebot.org/robot/)

• envolk[ITS]spider

  Example user agent
  envolk[ITS]spider/1.6 ( http://www.envolk.com/envolkspider.html)

• DataparkSearch

  Example user agent
  DataparkSearch/4.35 ( http://www.dataparksearch.org/)

• Nusearch Spider

  Example user agent
  nuSearch Spider <a href='http://www.nusearch.com'>www.nusearch.com</a> (compatible; MSIE 4.01; Windows NT)

• Java

  Example user agent
  Huawei/1.0/0HuaweiU1300/B123 Browser/Obigo-Browser/Q05A MMS/Obigo-MMS/Q05A SyncML/HW-SyncML/1.0 Java/HWJa/1.0 Profile/MIDP-2.0 Configuration/CLDC-1.1 Player/QTV-Player/5.3

• msnbot

  Example user agent
  adidxbot/2.0 (+http://search.msn.com/msnbot.htm)

• VYU2

  Example user agent
  VYU2 (GNU; OpenRISC)

• Googlebot

  Example user agent
  DoCoMo/2.0 N905i(c100;TB;W24H16) (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html)

• Baiduspider

  Example user agent
  Baiduspider-image ( http://www.baidu.com/search/spider.htm)

• iCCrawler

  Example user agent
  iCCrawler (http://www.iccenter.net/bot.htm)

• DataparkSearch

  Example user agent
  DataparkSearch/4.36 ( http://www.dataparksearch.org/)

• CSE HTML Validator

  Example user agent
  CSE HTML Validator Online (http://www.OnlineWebCheck.com) via 23.19.188.134

• CSE HTML Validator

  Example user agent
  CSE HTML Validator Online (http://www.OnlineWebCheck.com) via 108.62.151.31

• RufusBot

  Example user agent
  RufusBot (Rufus Web Miner; http://64.124.122.252/feedback.html)

• noxtrumbot

  Example user agent
  noxtrumbot/1.0 (crawler@noxtrum.com)

• Orbiter

  Example user agent
  Orbiter ( http://www.dailyorbit.com/bot.htm)

• FeedFetcher-Google

  Example user agent
  Feedly/1.0 (+http://www.feedly.com/fetcher.html; like FeedFetcher-Google)

• Java

  Example user agent
  Vodafone/1.0/0Vodafone715/B116 Browser/Obigo-Browser/Q04A MMS/Obigo-MMS/Q04A SyncML/HW-SyncML/1.0 Java/QVM/4.1 Profile/MIDP-2.0 Configuration/CLDC-1.1

• hl_ftien_spider

  Example user agent
  hl_ftien_spider

• LinkWalker

  Example user agent
  LinkWalker/2.0

• GurujiBot

  Example user agent
  Mozilla/5.0 (compatible; GurujiBot/1.0; +http://www.guruji.com/en/WebmasterFAQ.html)

• OOZBOT

  Example user agent
  OOZBOT/0.17 (--; http://www.setooz.com/oozbot.html; pvvpr at iiit dot ac dot in)

• GurujiBot

  Example user agent
  GurujiBot/1.0 (+http://www.guruji.com/WebmasterFAQ.html)

• Python-urllib

  Example user agent
  IMVU Client/498.0 Python-urllib/2.7

• Xenu Link Sleuth

  Example user agent
  Xenu Link Sleuth 1.1f

• TinEye

  Example user agent
  TinEye-bot/0.02 (see http://www.tineye.com/crawler.html)

• Python-urllib

  Example user agent
  Python-urllib/3.4

• ssearch_bot

  Example user agent
  ssearch_bot (sSearch Crawler; http://www.semantissimo.de)

• Speedy Spider

  Example user agent
  Speedy Spider (http://www.entireweb.com/about/search_tech/speedy_spider/)

• Java

  Example user agent
  Mozilla/5.0 (LG-T500 AppleWebkit/531 Browser/Phantom/V2.0 Widget/LGMW/3.0 MMS/LG-MMS-V1.0/1.2 Java/ASVM/1.1 Profile/MIDP-2.1

• Python-urllib

  Example user agent
  Python-urllib/1.15

• magpie-crawler

  Example user agent
  magpie-crawler/1.1 (U; Linux amd64; en-GB; +http://www.brandwatch.net)

• Shoula robot

  Example user agent
  Mozilla/4.0 (compatible: Shoula robot)

• semanticdiscovery

  Example user agent
  semanticdiscovery/0.2(http://www.semanticdiscovery.com/sd/robot.html)

• Bingbot

  Example user agent
  bingbot

• BillyBobBot

  Example user agent
  BillyBobBot/1.0 (+http://www.billybobbot.com/crawler/)

• Java

  Example user agent
  JNLP/6.0 javaws/1.6.0_14 (b08) Java/1.6.0_14

• Java

  Example user agent
  LG-A310-Orange/V10c Obigo/Q7.3 MMS/LG-MMS-V1.0/1.2 MediaPlayer/LGPlayer/1.0 Java/ASVM/1.1 Profile/MIDP-2.1 Configuration/CLDC-1.1

• CSE HTML Validator

  Example user agent
  CSE HTML Validator Online (http://www.OnlineWebCheck.com) via 108.62.195.142

• YoudaoBot

  Example user agent
  Mozilla/5.0 (compatible; YoudaoBot/1.0; http://www.youdao.com/help/webmaster/spider/; )

• Speedy Spider

  Example user agent
  Speedy Spider (Beta/1.0; www.entireweb.com)

• Baiduspider

  Example user agent
  Baiduspider-testbranch

• Gigabot

  Example user agent
  Gigabot/3.0 (http://www.gigablast.com/spider.html)

• Googlebot

  Example user agent
  Googlebot-Video/1.0

• Yeti

  Example user agent
  Yeti/1.1 (Naver Corp.; http://help.naver.com/robots/)

• Java

  Example user agent
  Mozilla/5.0 (LG-C199 AppleWebkit/531 Browser/Phantom/V2.0 Widget/LGMW/3.0 MMS/LG-MMS-V1.0/1.2 Java/ASVM/1.1 Profile/MIDP-2.1 Configuration/CLDC-1.1)

• Java

  Example user agent
  Nokia6650d-1bh/03.25/sw_platform=S60;sw_platform_version=3.2;java_build_version=74701 Profile/MIDP-2.1 Configuration/CLDC-1.1

• Java

  Example user agent
  Mozilla/5.0 (Java) outbrain

• Python-urllib

  Example user agent
  Mozilla/5.0 (Python-urllib2)

• sogou spider

  Example user agent
  Sogou Pic Spider/3.0( http://www.sogou.com/docs/help/webmasters.htm#07)

• msnbot

  Example user agent
  msnbot/2.0b v1394720947.6703

• Java

  Example user agent
  HuaweiU1250/B002 Browser/Access-Browser/Q05A MMS/Obigo-MMS/Q05A SyncML/HW-SyncML/1.0 Java/HWJa/1.0 Profile/MIDP-2.0 Configuration/CLDC-1.1 Player/QTV-Player/5.3

• Qseero

  Example user agent
  Qseero v1.0.0

• Java

  Example user agent
  LG-T300/V100 Obigo/Q7.3 MMS/LG-MMS-V1.1/1.2 MediaPlayer/LGPlayer/1.0 Java/ASVM/1.1 Profile/MIDP-2.1 Configuration/CLDC-1.1

• Java

  Example user agent
  Java/1.8.0_25

• DataparkSearch

  Example user agent
  DataparkSearch/4.35-02122005 ( http://www.dataparksearch.org/)

• Java

  Example user agent
  Java1.3.0

• Ezooms

  Example user agent
  Mozilla/5.0 (compatible; Ezooms/1.0; help@moz.com)

• Uzard Web

  Example user agent
  Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.2; WOW64; Trident/4.0; uZard/1.0; Server_KO_SKT)

• Scrubby

  Example user agent
  Mozilla/5.0 (compatible; Scrubby/2.1; +http://www.scrubtheweb.com/abs/meta-check.html)

• TweetedTimes Bot

  Example user agent
  TweetedTimes Bot/1.0 (Mozilla/5.0 Compatible, +http://tweetedtimes.com)

• Googlebot

  Example user agent
  Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

• psbot

  Example user agent
  psbot/0.1 (+http://www.picsearch.com/bot.html)

• htdig

  Example user agent
  htdig/3.1.5

• Yahoo! Slurp

  Example user agent
  Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp

• Java

  Example user agent
  Huawei/1.0/0HuaweiU1307/B100 Browser/Obigo-Browser/Q05A MMS/Obigo-MMS/Q05A SyncML/HW-SyncML/1.0 Java/HWJa/1.0 Profile/MIDP-2.0 Configuration/CLDC-1.1 Player/QTV-Player/5.3

• Sqworm

  Example user agent
  Sqworm/2.9.85-BETA (beta_release; 20011115-775; i686-pc-linux-gnu)

• TurnitinBot

  Example user agent
  TurnitinBot/3.0 (http://www.turnitin.com/robot/crawlerinfo.html)

• Mojoo Robot

  Example user agent
  Mojoo Robot (http://www.mojoo.com/)

• Teoma

  Example user agent
  Mozilla/2.0 (compatible; Ask Jeeves/Teoma; +http://about.ask.com/en/docs/about/webmasters.shtml)

• thumbshots-de-bot

  Example user agent
  thumbshots-de-bot (+http://www.thumbshots.de/)

• sogou spider

  Example user agent
  Sogou head spider/3.0( http://www.sogou.com/docs/help/webmasters.htm#07)

• Java

  Example user agent
  Java/1.6.0_22

• Java

  Example user agent
  Huawei/1.0/0HuaweiU3317/B100 Browser/Obigo-Browser/Q05A MMS/Obigo-MMS/Q05A SyncML/HW-SyncML/1.0 Java/HWJa/1.0 Profile/MIDP-2.0 Configuration/CLDC-1.1 Player/QTV-Player/5.3

• MagpieRSS

  Example user agent
  MagpieRSS/0.61 (+http://magpierss.sf.net)

• Xenu Link Sleuth

  Example user agent
  Xenu Link Sleuth 1.2b

• omgilibot

  Example user agent
  omgilibot/0.4 +http://omgili.com

• Link Valet

  Example user agent
  Link Valet Online 1.1

• Java

  Example user agent
  Mozilla/5.0 (compatible; Ex-Crawler/0.1.5a; powered by ex-crawler; +http://www.ex-crawler.de/) Java/1.6.0_20,gzip(gfe),gzip(gfe)

• mxbot

  Example user agent
  Mozilla/5.0 (compatible; mxbot/1.0; +http://www.chainn.com/mxbot.html)

• 008

  Example user agent
  Mozilla/5.0 (compatible; 008/0.83; http://www.80legs.com/webcrawler.html) Gecko/2008032620

• EsperanzaBot

  Example user agent
  EsperanzaBot(+http://www.esperanza.to/bot/)

• OmniExplorer_Bot

  Example user agent
  OmniExplorer_Bot/6.47 (+http://www.omni-explorer.com) WorldIndexer

• EmeraldShield.com WebBot

  Example user agent
  EmeraldShield.com WebBot

• Java

  Example user agent
  LG/KF750/v10a Browser/Obigo-Q05A/3.12 MMS/LG-MMS-V1.0/1.2 Java/ASVM/1.1 Profile/MIDP-2.1 Configuration/CLDC-1.1

• Python-urllib

  Example user agent
  Python-urllib/3.0

• Java

  Example user agent
  Java1.4.0_02

• ConveraCrawler

  Example user agent
  ConveraCrawler/0.9c ( http://www.authoritativeweb.com/crawl)

• Python-urllib

  Example user agent
  Python-urllib/2.7

• Googlebot

  Example user agent
  Googlebot-richsnippets

• CatchBot

  Example user agent
  CatchBot/2.0; +http://www.catchbot.com

• boitho.com-dc

  Example user agent
  boitho.com-dc/0.4 ( http://www.boitho.com/dcbot.html )

• lwp-trivial

  Example user agent
  lwp-trivial/1.36

• Java

  Example user agent
  OneBrowser/3.1 (Nokia501.2/14.0.4/java_runtime_version=Nokia_Asha_1_2)

• Java

  Example user agent
  Samsung/910XXIG2/sw_platform=S60;sw_platform_version=5.0;java_build_version=1.3.4 Profile/MIDP-2.1 Configuration/CLDC-1.1 UNTRUSTED/1.0

• Yeti

  Example user agent
  Mozilla/5.0 (compatible; Yeti/1.1; +http://help.naver.com/robots/)

• HuaweiSymantecSpider

  Example user agent
  HuaweiSymantecSpider/1.0 DSE-support@huaweisymantec.com (compatible; MSIE 7.0; Windows NT 5.1; Trident/4.0; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR ; http://www.huaweisymantec.com/cn/IRL/spider)

• Accoona-AI-Agent

  Example user agent
  Accoona-AI-Agent/1.1.2 (aicrawler at accoonabot dot com)

• Mediapartners-Google

  Example user agent
  DoCoMo/2.0 SH905i(c100;TB;W24H16) (compatible; Mediapartners-Google/2.1; +http://www.google.com/bot.html)

• PostPost

  Example user agent
  PostPost/1.0 ( http://postpo.st/crawlers)

• lmspider

  Example user agent
  lmspider lmspider@scansoft.com

• YoudaoBot

  Example user agent
  Mozilla/5.0 (compatible; YoudaoBot-rts/1.0; http://www.youdao.com/help/webmaster/spider/; )

• Xenu Link Sleuth

  Example user agent
  Xenu Link Sleuth 1.2g

• Java

  Example user agent
  Java1.4.2_03

• FyberSpider

  Example user agent
  FyberSpider

• BecomeBot

  Example user agent
  Mozilla/5.0 (compatible; BecomeBot/2.3; MSIE 6.0 compatible; +http://www.become.com/site_owners.html)

• Googlebot

  Example user agent
  Googlebot

• FeedFetcher-Google

  Example user agent
  Feedfetcher-Google; (+http://www.google.com/feedfetcher.html; 375 subscribers; feed-id=15381863289700640853)

• OmniExplorer_Bot

  Example user agent
  OmniExplorer_Bot/5.91c (+http://www.omni-explorer.com) WorldIndexer

• Yahoo! Slurp

  Example user agent
  Mozilla/5.0 (compatible; Yahoo! Slurp/3.0 ; http://help.yahoo.com/help/us/ysearch/slurp)

• Python-urllib

  Example user agent
  Python-urllib/2.1

• Holmes

  Example user agent
  holmes/3.12.4 (http://morfeo.centrum.cz/bot)

• Java

  Example user agent
  LG-BL20f Browser/Obigo-Q7.3 MMS/LG-MMS-V1.0/1.2 MediaPlayer/LGPlayer/1.0 Java/ASVM/1.1 Profile/MIDP-2.1 Configuration/CLDC-1.1

• Xenu Link Sleuth

  Example user agent
  Xenu Link Sleuth 1.2f

• omgilibot

  Example user agent
  omgilibot/0.3 http://www.omgili.com/Crawler.html

• yrspider

  Example user agent
  yrspider (Mozilla/5.0 (compatible; YRSpider; http://www.yunrang.com/yrspider.html))

• Java

  Example user agent
  Java/1.7.0_71

• Java

  Example user agent
  Java1.2.2

• Java

  Example user agent
  Mozilla/5.0 (LG-C360 AppleWebkit/531 Browser/Phantom/V2.0 Widget/LGMW/3.0 MMS/LG-MMS-V1.0/1.2 Java/ASVM/1.1 Profile/MIDP-2.1 Configuration/CLDC-1.1)

• UniversalFeedParser

  Example user agent
  UniversalFeedParser/5.1.3 +https://code.google.com/p/feedparser/

• W3C_Validator

  Example user agent
  W3C_Validator/1.3 http://validator.w3.org/services

• ZyBorg

  Example user agent
  Mozilla/4.0 compatible ZyBorg/1.0 (ZyBorg@WISEnutbot.com; http://www.WISEnutbot.com)

• AppEngine-Google

  Example user agent
  Mozilla/5.0 (compatible; AhrefsBot/5.0; +http://ahrefs.com/robot/) AppEngine-Google; (+http://code.google.com/appengine; appid: s~proxyfile1-hrd)

• W3C_CSS_Validator_JFouffa

  Example user agent
  Jigsaw/2.3.0 W3C_CSS_Validator_JFouffa/2.0 (See <http://validator.w3.org/services>)

• BlitzBot

  Example user agent
  Mozilla/4.0 (compatible; BlitzBot)

• mogimogi

  Example user agent
  mogimogi/1.0

• Holmes

  Example user agent
  holmes/2.3

• Accoona-AI-Agent

  Example user agent
  Accoona-AI-Agent/1.1.2

• MQBOT

  Example user agent
  CazoodleBot/0.1 (CazoodleBot Crawler; http://www.cazoodle.com; mqbot@cazoodle.com)

• AnyApexBot

  Example user agent
  Mozilla/5.0 (compatible; AnyApexBot/1.0; +http://www.anyapex.com/bot.html)

• Bloglines

  Example user agent
  Bloglines/3.1 (http://www.bloglines.com)

• FeedFetcher-Google

  Example user agent
  Feedfetcher-Google; (+http://www.google.com/feedfetcher.html; feed-id=000000000000000000)

• zspider

  Example user agent
  Zspider (+http://www.zhanzhangsou.com/index.htm)

• DotBot

  Example user agent
  DotBot/1.0.1 (http://www.dotnetdotcom.org/, crawler@dotnetdotcom.org)

• Java

  Example user agent
  LG-HB620T/v10a Browser/Obigo-Q05A/3.12 MMS/LG-MMS-V1.0/1.2 Java/ASVM/1.1 Profile/MIDP-2.1 Configuration/CLDC-1.1

• msnbot

  Example user agent
  msnbot/2.0b (+http://search.msn.com/msnbot.htm)

• Java

  Example user agent
  Mozilla/5.0 (LG-T585 AppleWebkit/531 Browser/Phantom/V2.0 Widget/LGMW/3.0 MMS/LG-MMS-V1.0/1.2 Java/ASVM/1.1 Profile/MIDP-2.1 Configuration/CLDC-1.1)

• TheSuBot

  Example user agent
  TheSuBot/0.2 (www.thesubot.de)

• Amaya

  Example user agent
  amaya/9.51 libwww/5.4.0

• discobot

  Example user agent
  Mozilla/5.0 (compatible; discobot/1.0; +http://discoveryengine.com/discobot.html)

• Java

  Example user agent
  Mozilla/5.0 (LG-C333 AppleWebkit/531 Browser/Phantom/V2.0 Widget/LGMW/3.0 MMS/LG-MMS-V1.0/1.2 Java/ASVM/1.1 Profile/MIDP-2.1 Configuration/CLDC-1.1)

• ichiro

  Example user agent
  DoCoMo/2.0 P900i(c100;TB;W24H11) (compatible; ichiro/mobile goo; +http://search.goo.ne.jp/option/use/sub4/sub4-1/)

• Zealbot

  Example user agent
  Mozilla/4.0 (compatible; Zealbot 1.0)

• Java

  Example user agent
  SonyEricssonW666i/R2BA Java/SEMC-Java/2.0 Profile/MIDP-2.0 Configuration/CLDC-1.1 UNTRUSTED/1.0

• Yahoo! Slurp

  Example user agent
  Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)

• MagpieRSS

  Example user agent
  MagpieRSS/0.72 (+http://magpierss.sf.net)

• Java

  Example user agent
  Mozilla/5.0 (Vodafone/1.0/LG-T385b/V10i AppleWebkit/531 Browser/Phantom/V2.0 Widget/LGMW/3.0 MMS/LG-MMS-V1.0/1.2 Java/ASVM/1.1 Profile/MIDP-2.1 Configuration/CLDC-1.1)

• htdig

  Example user agent
  htdig/3.1.6 (romieu@bastide-medical.fr)

• Java

  Example user agent
  Mozilla/5.0 (LG-T565 AppleWebkit/531 Browser/Phantom/V2.0 Widget/LGMW/3.0 MMS/LG-MMS-V1.0/1.2 Java/ASVM/1.1 Profile/MIDP-2.1 Configuration/CLDC-1.1)

• Java

  Example user agent
  Mozilla/5.0 (LG-C370 AppleWebkit/531 Browser/Phantom/V2.0 Widget/LGMW/3.0 MMS/LG-MMS-V1.0/1.2 Java/ASVM/1.1 Profile/MIDP-2.1 Configuration/CLDC-1.1)

• Java

  Example user agent
  LG-T310i/V100 Obigo/Q7.3 MMS/LG-MMS-V1.1/1.2 MediaPlayer/LGPlayer/1.0 Java/ASVM/1.1 Profile/MIDP-2.1 Configuration/CLDC-1.1

• htdig

  Example user agent
  htdig/3.1.6 (mathieu.peltier@inrialpes.fr)

• voyager

  Example user agent
  voyager/1.0

• TheSuBot

  Example user agent
  TheSuBot/0.1 (www.thesubot.de)

About this comparison