Dschini Bot Checker Plugin

After playing a bit with the Wordpress Plugin API I thought about developing a plugin which detects the search engine bots that visit my page. And now I am proud to anounce the release of the Dschini Bot Checker Plugin V1.0 for Wordpress. The plugin synchronizes your local bots database with the Dschini BotService via SOAP. So you need at least Wordpress and PHP 5.

After installation you will find the Dschini Bot Checker within Options where you can define the bots to track. You can choose between "High Popular" bots like Googlebot and "Medium"- and "Low"-Popular Bots. Whenever your update your options the Dschini BotService refreshes your bot database but will keep an eye on existing trackings.

The Bot Checker Management Panel analyses the bots and gives you some nice informations about the pages the bot visited. It also ranks up the most active bots and the most tracked pages.


You can download the release here. Then copy and unpack the file into your Wordpress plugin folder. I am looking for your feedback and I hope there´re no bugs. Have fun!

[Update] Here is also a non Soap Version: Bot Checker NON-Soap! But this version is ofcourse limited.

So if you have SOAP then take this version.

26 Comments

  1. flavien 2007-01-5, 12:45 pm

    Gravatar

    Hi Manfred, I installed the plugin but every time I update its options I got this :

    Fatal error: Class ‘SoapClient’ not found in ../blogurl/wp-content/plugins/dschini/botchecker.php on line 27

    and the two tables in database are always blank.

    My blog is hosted on DreamHost, php5 with WordPress 2.0.5.

    Can you advise why it goes that?

  2. manfred.weber 2007-01-5, 1:22 pm

    Gravatar

    Hi flavien,
    it seems your PHP Installtion is not configured with SOAP. You can easily check this with a phpinfo() page. However, I have created a Non-SOAP-Version which you can download at:

    http://manfred.dschini.org/wp-content/uploads/2007/01/dschini_botchecker-1-non-soap.zip
    You will also find a link [UPDATE] in this post.

    Extract the zip file and copy the botchecker.php into your plugin/dschini/ folder. It will overwrite the soap version.

    Then you should delete the 2 tables that got created from previous installation.

    Once this is done you have to deactivate and active this plugin again and it should work.

    Best Regards

  3. flavien 2007-01-5, 3:16 pm

    Gravatar

    thanks Manfred, it works now with no SOAP:)

    and I also find it says “x from 9 available bots” in Manage column. does it mean the non-SOAP version can only recognize 9 kinds of bots?

    if so, how many bots does the SOAP version include?

    Working on configuring my php with SOAP..;-)

  4. Manfred Weber 2007-01-5, 3:25 pm

    Gravatar

    Well, there exists the Dschini BotService which holds about 300 different bot data that a categorized into high popular, medium popular and low popular. I am always working to keep these bots up to date, thats why the plugin synchronizes with the Service. The Non-Soap Version ships with all bots which are currently marked as High Popular and Medium Popular and which track your site very often. Some of the bots marked as Low Popular are old and do not even exists anymore.

  5. au8ust 2007-01-6, 5:04 pm

    Gravatar

    SOAP version not working with my blog either, it returns Fatal error: Cannot instantiate non-existent class: soapclient in /home/watpitor/public_html/au8ust/wp-content/plugins/botchecker.php on line 27

    I’ll try non-SOAP one

  6. au8ust 2007-01-6, 5:29 pm

    Gravatar

    Non-SOAP version returns…

    WordPress database error: [Table 'watpitor_wordpress.wp_dschini_botchecker_bots' doesn't exist]
    SELECT COUNT(DISTINCT(bot_id)) FROM wp_dschini_botchecker_bots

    WordPress database error: [Table 'watpitor_wordpress.wp_dschini_botchecker_tracker' doesn't exist]
    SELECT COUNT(DISTINCT(bot_id)) FROM wp_dschini_botchecker_tracker

    WordPress database error: [Table 'watpitor_wordpress.wp_dschini_botchecker_tracker' doesn't exist]
    SELECT COUNT(bot_id) FROM wp_dschini_botchecker_tracker

    WordPress database error: [Table 'watpitor_wordpress.wp_dschini_botchecker_tracker' doesn't exist]
    SELECT a.bot_id, count(*) as c, b.name, b.detailsurl, b.description, b.history FROM wp_dschini_botchecker_tracker a, wp_dschini_botchecker_bots b WHERE a.bot_id=b.bot_id GROUP BY a.bot_id ORDER BY c DESC LIMIT 0,4

    WordPress database error: [Table 'watpitor_wordpress.wp_dschini_botchecker_tracker' doesn't exist]
    SELECT url, count(*) as c FROM wp_dschini_botchecker_tracker GROUP BY url ORDER BY c DESC LIMIT 0,4

    There are entries tracked! different from available bots are tracking your blog.

    Most active bots:

    Most tracked URL’s:

    WordPress database error: [Table 'watpitor_wordpress.wp_dschini_botchecker_bots' doesn't exist]
    SELECT b.id AS id, a.bot_id AS bot_id, a.description AS description, a.purpose AS purpose, a.history AS history, a.license AS license, a.exclusionuseragent AS exclusionuseragent, a.ownername AS ownername, a.ownerurl AS ownerurl, a.owneremail AS owneremail, a.popularity AS popularity, a.name AS name, a.detailsurl AS detailsurl, b.url AS url, b.created AS created FROM wp_dschini_botchecker_bots a,wp_dschini_botchecker_tracker b WHERE a.bot_id=b.bot_id ORDER by b.id DESC LIMIT 0,50

  7. MCB 2007-01-7, 12:42 am

    Gravatar

    au8ust: You get that error because dbDelta which Manfred uses to create the databases doesn’t always work correctly on many installations. I’ve never figured out why. (Maybe its a PHP4 thing.) I’ve had the same problem with other plugins too. Just create the tables manually and resave your options.

  8. Manfred Weber 2007-01-7, 11:13 am

    Gravatar

    via MCB
    —————————-

    SOAP is not available with PHP4 but you can use the following modification to make the plug-in PHP4 compatible.

    1. Install NuSOAP in your include path or in the same directory as botchecker.php. NuSOAP can be downloaded for SourceForge.

    2. Make the following modifications to botchecker.php;

    [FIND]
    ini_set(”soap.wsdl_cache_enabled”,”0″);

    [REPLACE]
    include(”nusoap.php”);

    [FIND]
    $client = new SoapClient(”http://services.dschini.org/bot.php?WSDL”);

    [REPLACE]
    $soapclient = new SoapClient(”http://services.dschini.org/bot.php?WSDL”, “wsdl”);
    $client = $soapclient->getProxy();

    [FIND]
    $client = new SoapClient(”http://services.dschini.org/bot.php?WSDL”);

    [REPLACE]
    $soapclient = new SoapClient(”http://services.dschini.org/bot.php?WSDL”, “wsdl”);
    $client = $soapclient->getProxy();

    3. My version of PHP had trouble with “->” for an array of arrays so I changed it to use basic array notation for named keys.

  9. Yamboo 2007-01-9, 8:46 pm

    Gravatar

    Wordpress BotChecker Plugin

    Ständig sind Suchmaschinenbots im Internet unterwegs und besuchen unsere Webseiten. Wäre es nicht interessant zu wissen, welche Seiten wann und wie lange besucht werden? Welche Bots besonders schnell sind und welche auf sich warten lassen?
    Al…

  10. Martin Daugaard Stevnhoved 2007-01-10, 8:49 pm

    Gravatar

    Hi,

    The synchronisation doesn’t work for me. The install hasn’t created the tables, so I created them manually as au8ust suggested. But it looks like the plugin don’t synchronise with the bot-database. The tables keeps staying empty.

    Best Regards,
    Martin

  11. SmileSquare 2007-01-11, 10:26 am

    Gravatar

    SOAP version working well for me, but it not record Technorati bot…

  12. Manfred Weber 2007-01-11, 11:32 am

    Gravatar

    Hmm, I added the TechnoratiBot (Click update under Options). However I noticed there´re plenty of existing bots not yet in the database. Here is a list of bots I want to add:


    - Netcraft Web Server Survey
    - Ask Jeeves/Teoma
    - grub
    - Wget
    - Feedster Crawler
    - BlogSearch
    - Syndic8
    - Cerberian
    - WISEnutbot
    - BlogPulse
    - A2B Location-Based Search Engine
    - BlogsNowBot
    - Blogslive
    - Blogshares
    - UniversalFeedParser
    - ping.blo.gs
    - PageBitesHyperBot
    - PubSub-RSS
    - SurveyBot
    - walhello
    - Mirar
    - OmniExplorer
    - W3C_Validator
    - IconSurf
    - TurnitinBot
    - psbot
    - aipbot
    - StumbleUpon
    - Gigabot
    - LinkWalker
    - rojo.com
    - ConveraCrawler
    - DiamondBot
    - HenryTheMiragoRobot
    - Baiduspider
    - WebFilter Robot
    - SURF
    - topicblogs
    - BecomeBot


    For each of this bot I need as much information as possible. Maybe you can help me with 1 or 2 of them by providing following information (example Googlebot):


    name:Googlebot
    detailsurl:http://www.googlebot.com/bot.html
    ownername:Google Inc.
    ownerurl:http://www.google.com/
    owneremail:googlebot@google.com
    state:active
    purpose:indexing
    application:standalone
    platform:Linux
    availability:none
    exclusion:yes
    exclusionuseragent:googlebot
    noindex:yes
    host:googlebot.com
    useragent:Googlebot/2.X (+http://www.googlebot.com/bot.html)
    language:c++
    description:Googles crawler
    history:Developed by Google Inc
    license:commercial
    author:googlebot@google.com


    send it to my email address and I will update the list of missing bots here.

    Thanks in advance

  13. Leigh 2007-01-14, 6:07 am

    Gravatar

    I’ve installled the non-soap version, but now I need to manually create the tables.

    It looks like I need to add 2 tables to the database:
    .wp_dschini_botchecker_bots
    .wp_dschini_botchecker_tracker

    Is that all, or are there other tables to add?

    How many fields should I create for each table?

  14. Manfred Weber 2007-01-14, 4:56 pm

    Gravatar

    Leigh, the tables should get creating automatically for you when you hit enable the plugin.

  15. Leigh 2007-01-26, 2:32 am

    Gravatar

    Hi Manfred,

    Unfortunately, enabling the plugin didn’t create the tables for me - same as for Martin, above.

    The difference is, he seems to know what tables to create, I don’t!

  16. moby 2007-02-7, 6:27 am

    Gravatar

    I got the same error (9bot…) until it had a chance to update and then the error went away. It seems to work perfectly now.

  17. mustaq 2007-02-7, 7:46 am

    Gravatar

    good plugin but have a problem . no new data base created after i activated can you give sql code so i can add manually .

  18. Scott McAndrew 2007-07-19, 6:15 am

    Gravatar

    I’m running PHP version 4.3.1. When I run phpinfo() it looks like SOAP is enabled (’enable-soap’). The result I get for Configure Command is below:

    ‘./configure’ ‘–with-cgi’ ‘–enable-fastcgi’ ‘–with-config-file-path=/web/conf’ ‘–with-gd’ ‘–with-xml’ ‘–with-gettext’ ‘–with-zlib-dir=/usr/src/zlib’ ‘–disable-posix’ ‘–with-jpeg-dir=../jpeg-6b’ ‘–enable-gd-native-ttf’ ‘–enable-ftp’ ‘–with-freetype-dir=/usr’ ‘–with-freetype’ ‘–with-sybase=/usr/src/freetds’ ‘–with-curl=/usr/bin/curl’ ‘–with-dom’ ‘–enable-calendar’ ‘–enable-soap’ ‘–enable-bcmath’ ‘–with-zip’ ‘–with-openssl’ ‘–with-mcrypt’ ‘–with-mysql=/usr/local/mysql-5.0′
    –
    However, here is what I get after installation when I configure the plug-in for which bots I would like it to track:

    WordPress database error: [Table 'smcandrewWP.wp_dschini_botchecker_bots' doesn't exist]
    DELETE FROM wp_dschini_botchecker_bots

    Fatal error: Cannot instantiate non-existent class: soapclient in /home/content/s/d/m/sdmcandrew/html/wp-content/plugins/botchecker.php on line 27
    –

    Can you please advise on what to do to get the plug-in to work. In theory, this sounds like a great plug-in!

  19. Manfred Weber 2007-07-19, 10:08 am

    Gravatar

    Hi Scott,
    hmm that could be a lot. I am looking at the sources again right now.
    First of all it seems the database table is not installed, (dont know why, this should have been done automatically for you actually).

    Table 1:
    CREATE TABLE smcandrewWP.p_dschini_botchecker_bots (
    id mediumint(9) NOT NULL AUTO_INCREMENT,
    bot_id mediumint(9) NOT NULL,
    description VARCHAR(255) NOT NULL,
    purpose VARCHAR(255) NOT NULL,
    history VARCHAR(255) NOT NULL,
    license VARCHAR(255) NOT NULL,
    exclusionuseragent VARCHAR(255) NOT NULL,
    ownername VARCHAR(255) NOT NULL,
    ownerurl VARCHAR(255) NOT NULL,
    owneremail VARCHAR(255) NOT NULL,
    popularity VARCHAR(255) NOT NULL,
    name VARCHAR(255) NOT NULL,
    detailsurl VARCHAR(255) NOT NULL,
    UNIQUE KEY id (id)

    Table 2:
    CREATE TABLE smcandrewWP.p_dschini_botchecker_tracker (
    id mediumint(9) NOT NULL AUTO_INCREMENT,
    bot_id mediumint(9) NOT NULL,
    url VARCHAR(255) NOT NULL,
    created DATETIME NOT NULL,
    UNIQUE KEY id (id)

    After this, tell me if the SOAP Problem still exists.

    If you cant get SOAP to work, there alöso exists a non-soap version:
    http://manfred.dschini.org/wp-content/uploads/2007/01/dschini_botchecker-1-non-soap.zip

    The non-soap version only checks for most important bots. Still good enough.

    Regards
    Manfred

  20. cross 2008-03-5, 9:57 pm

    Gravatar

    Thanks, nice plugin.

    Soon, I`ll try to use it on my own wordpress blog.

  21. (很好很强大,收藏备用)wordpress插件集锦 | 睡到自然醒blog 2008-05-31, 1:04 pm

    Gravatar

    [...] DownloadCounter - 统计附件被下载次数。 Dschini Bot Checker - 获得日志被搜索引擎检索的情况。 FirStats - [...]

  22. 知更鸟 » 30+WordPress统计插件 2008-05-31, 1:55 pm

    Gravatar

    [...] Dschini Bot Checker - 把机器人访问的页面及它们的访问频率告诉你。 [...]

  23. wordpress Plug-in Category collect | 王标 2008-06-2, 11:52 am

    Gravatar

    [...] DownloadCounter - 统计附件被下载次数。 Dschini Bot Checker - 获得日志被搜索引擎检索的情况。 FirStats - [...]

  24. 30个与统计相关的Wordpress插件 | Awesomer 2008-06-6, 1:16 pm

    Gravatar

    [...] Dschini Bot Checker - 获得日志被搜索引擎检索的情况。 [...]

  25. 30多个Wordpress统计插件 | 无限轮回-天仓零&完蛋了的国王 2008-06-11, 7:33 am

    Gravatar

    [...] Dschini Bot Checker - 反映搜索引擎机器访问情况 。 [...]

  26. wordpress插件集锦（不断添加中…） | Sharing 2008-06-20, 12:16 pm

    Gravatar

    [...] DownloadCounter - 统计附件被下载次数。 Dschini Bot Checker - 获得日志被搜索引擎检索的情况。 FirStats - [...]

Add a Comment