Monitoring Apache Page-Load Times With Cricket
Pages: 1, 2, 3

Installing Cricket

For our basic Cricket setup, let's assume you're going to install it on a Linux server that already has an Apache server running on it. Also you'll need to have installed the basic development tools: make, GCC compiler, and the Linux kernel headers.

I found these instructions in the beginner.txt file in the cricket/doc directory. My instructions are more explicitly geared for a Linux system; if you are running something else, you'll probably want to refer to the beginner.txt file as well. So, let's get started.

You will need a recent version of Perl -- version 5.004 or newer. To check the version number, use the perl -v command.

You will also need these packages from CPAN (Comprehensive Perl Archive Network); you may have some of them already.

If you have the CPAN module installed and configured, you can issue the following commands while running as "root." If you have the CPAN module but have not configured it, the first time you run it, it will ask some questions. Go ahead and give it a shot; it's not that hard. Otherwise you can go to CPAN.org web site. Here you can find the modules, download them, unpack them, and build and install each one by following the ReadMe file. This is known as doing it the hard way.

Each of these CPAN module commands will install the latest version of each package. It is safe to run the command; if the latest version is already installed, it will just tell you that and stop.

 perl -MCPAN -eshell  
 cpan> install MD5  
 cpan> install LWP  
 cpan> install DB_File  
 cpan> install Date::Parse  
 cpan> install Time::HiRes  
 cpan> quit

There are two more packages that are not in the CPAN archives, so you have to fetch and install them separately. Use your web browser to find the latest version of each and download them to a spot on your system where you can unpack and build the package. Then use your rootly powers to install it.

First the SNMP_Session package: For this HTTP tracking project we actually don't need to use any SNMP services, but Cricket requires the package so you have to install it anyway.

The SNMP_Session web site is:

http://www.switch.ch/misc/leinen/snmp/perl

You can download the latest version from:

ftp://ftp.switch.ch/software/sources/network/snmp/perl

Here are the abbreviated instructions on building version 0.76:

    % tar xzf SNMP_Session-0.76.tar.gz
    % cd SNMP_Session-0.76
    % perl Makefile.PL
    % make
    % su root
    Password:
    # make install
    # exit
    % cd ..

The RRD package is the heart of Cricket. The main RRD site is:

http://ee-staff.ethz.ch/~oetiker/webtools/rrdtool/.

You can download the package from:

http://ee-staff.ethz.ch/~oetiker/webtools/rrdtool/pub/.

Here are the abbreviated instructions on building version 1.0.11:

    % tar xzf rrdtool-1.0.11.tar.gz
    % cd rrdtool-1.0.11

I had to do this to get configure to work on one of my systems:

    % unset noclobber

    % ./configure
    % make
    % su root
    Password:

This next line will install the RRD Perl modules in your system's standard site-perl directory tree instead of putting them in a separate location (which is what make install does). This is necessary for the Cricket scripts to find the RRD modules.

    # make site-perl-install

Now, as root, create a user account that will run Cricket.

These commands work on a Linux system; use your own preference on your system to create a Cricket user. You don't strictly need a separate Cricket account, but I find it is a lot easier this way.

    # groupadd cricket
    # useradd -g cricket -c 'Cricket Traffic Grapher' cricket
    # passwd cricket
    # chmod 755 ~cricket

    # echo "cricket: root" >> /etc/aliases
    # newaliases
    # exit

Download and install Cricket.

    % su - cricket
    Password:

Now that you're running as Cricket, use a browser to download the Cricket source archive from here.

Here are the abbreviated instructions on installing version 0.72:

    % tar xzf cricket-0.72.tar.gz

Using this symbolic link will allow you to upgrade easily later:

    % ln -s cricket-0.72 cricket
    % cd cricket

Running "configure" will fix up the first line of each Cricket script so they can find Perl on your system.

    % ./configure

Now we get into the configuration; this is the most complicated step. Copy the sample configuration tree to the ~cricket/cricket-config directory, which is where the installed Cricket will look for it.

    % cd ..
    % cp -r cricket/sample-config cricket-config

There are lots of setup files in there that won't be used but maybe you'll want them later; they won't hurt anything for now. For now, we are only interested in cricket-config/http-performance. Edit the URLs file.

    % cd cricket-config/http-performance
    % ls
    Defaults  urls
    % emacs urls

The sample file looks like this. Remove those entries and put in entries for whatever you would like to monitor.

    target  cricket-home
        short-desc = "The Cricket Homepage"
        url = "http://www.munitions.com/~jra/test-file.txt"

    target  www.cnn.com
        url = "http://www.cnn.com"

• The target name (for example, cricket-home) will be used to name the database table, and as the default name in the web interface.
• The short-desc text will override the target name in the web interface.
• The URL can be any standard HTTP statement.

You can have as many targets as you like; each one will cause a database table of approximately 60 kilobytes to be created. If it takes more than five minutes to collect a set of data, you will receive warning messages telling you that the collection subtree is locked. You can ignore these messages or change the collection interval.

The defaults file in this directory contains settings to control how the data is displayed. For a while, one of my servers was consistently delivering pages in times greater than five seconds. I had to change the setting for y-max so that I could see more data on the graphs. I won't tell you how slow the server was -- too embarrassing. Every page on that site was generated by Perl scripts.

Each time you make changes to the cricket-config files, you have to recompile them. There will be error messages if you entered anything incorrectly.

   % cd ~
   % cricket/compile

To test the data collector, run it now manually.

   % cricket/collector /http-performance

If it works, you'll see a lot of messages like this indicating Cricket is testing each target in your configuration file:

   [09-Feb-2000 22:26:55 ] Retrieving data 
   (EXEC: /home/cricket/cricket-config/../cricket/util/test-url 
   http://www.xml.com) for XML

The collector will also create the data tables in the ~cricket/cricket-data/http-performance directory the first time you run it.

The collector is run from a script called collect-subtrees. You can set up Cron to run different collection sets at different intervals. The file cricket/subtrees-sets defines what is in each set. For our example, you will have to edit that file to change the lines.

    set normal:
  /routers
  /router-interfaces

to

    set normal:
  /http-performance

Now, to test collection of the set, run the wrapper script:

   % cricket/collect-subtrees normal
   % exit

You won't see any output from this script, but the wrapper will create another directory, cricket-logs, and log its output to file in it, normal.0.

Making a Cron Entry

Now you are ready to set up a Cron job to run the collection script. Make a Cron entry to run Cricket once every five minutes. I run Cricket from the /etc/cron.d directory. You could run it directly under the Cricket account (using the crontab -e command to edit the file), but I find it easier to keep track of what administrative Cron jobs are installed by putting them all in the /etc/cron.d directory.

   % su root

[Next command all on one line]

   # echo "*/5 * * * * cricket 
   /home/cricket/cricket/collect-subtrees normal" > 
   /etc/cron.d/cricket
   
   # exit

Now wait until the next 5-minute increment rolls around and watch to see if the data collection happens. Once Cricket has been running for awhile, you will see a series of files from normal.0 to normal.20; each time collect-subtrees file runs, it renumbers the files so the newest one is always normal.0.

Cricket is now logging data. If you modify the files in cricket-config, remember to re-run compile to update the configuration.

On page 3, we'll cover how to set things up for web browsing.