AddThis Social Bookmark Button

Listen Print Discuss

Visualizing Network Traffic with Netflow and FlowScan
Pages: 1, 2, 3

Saving Netflow Records from FlowScan

By default, FlowScan deletes flow files that it has processed. I suggest you retain those files for a few months, or as long as disk space allows. Create a saved subdirectory under your Netflow log directory and FlowScan will automatically move processed logs to this subdirectory.



Even if you don't want to retain records as ongoing practice, I recommend keeping them until you know FlowScan is working correctly. If your FlowScan configuration is broken, it will destroy the data you've already gathered without recording it properly. This is vastly annoying when troubleshooting.

Starting FlowScan

In theory, you have everything configured now. Cross your fingers and start FlowScan.

# /usr/local/var/db/flows/bin/flowscan

FlowScan will start spewing out all sorts of messages.

2004/09/02 11:35:17 working on file /var/log/netflows/ft-v01.2004-08-31.142629-0400...
2004/09/02 11:35:18 flowscan-1.020 CUFlow: Cflow::find took  1 wallclock secs ( 0.60 usr +  0.02 sys =  0.62 CPU) for 43011 flow file bytes, flow hit ratio: 2759/2760
2004/09/02 11:35:18 flowscan-1.020 CUFlow: report took  0 wallclock secs ( 0.15 usr  0.19 sys +  0.02 cusr  0.09 csys =  0.44 CPU)

FlowScan is parsing all the old flow files. This can take quite a while, depending on how many flows you've accumulated between implementing your collector and starting FlowScan. One interesting thing to look for here is the "flow hit ratio," or how many flows FlowScan found described in the configuration file. This particular flow file had a hit ratio of 2759 out of 2760; one flow out of 2760 didn't fit FlowScan's expectations. That's pretty good. If you have a hit ratio of 0, you probably messed up your FlowScan install or your Subnet statement.

If FlowScan complains about an "Invalid index in cflowd flow file," you probably didn't install the newest Flowscan.pm module. This is perhaps the most common error people make with FlowScan. If you have this problem, go get the appropriate version of the module as described earlier.

When FlowScan finishes parsing all your old flow files, it will print out "sleep 300...", wait for five minutes, and check your log directory for new flow files. You can Ctrl-C out of FlowScan.

You probably want FlowScan to start automatically at boot rather than taking over your terminal, so go under /usr/local/etc/rc.d and copy the sample startup script to flowscan.sh. This file works unedited, but I usually change the logfile to /var/log/flowscan.log simply because I like all my logs in one place.

Generating Graphs

"This is nice, but where are my graphs? You promised me pretty pictures!"

Fortunately, getting graphs out of the RRD files is trivial. CUFlow includes a CGI script, CUGrapher.pl. Copy this to your web server's cgi-bin directory. You only need to set two variables: $rrddir and $organization.

The $rrddir variable contains the directory where CUFlow stores the RRD files.

my $rrddir = "/var/log/cuflow";

To print your company's name at the top of the page, be sure to set the $organization variable.

my $organization = "LogicaCMG US IDT development area";

Now browse to the URL for this script and select, say, a network. You'll see an array of drop-down menus. Choose some item--say, a network, or a protocol--and hit "Generate graph."

Congratulations! You have better bandwidth graphs than MRTG alone provides.

One drawback with CUFlow is that it doesn't break down traffic by network and service. For example, if you choose "Dev network" and "http," you'll get entries for the amount of traffic to and from the dev network added to the amount of HTTP traffic the whole network sees. This isn't exactly useful. To generate more fine-grained reports than this, you'll have to write some custom Netflow reports. I'll explain that in a future article.

Michael W. Lucas

Return to the Sysadmin DevCenter