Monitoring Network Traffic with Netflow
Pages: 1, 2, 3, 4
On some recent versions of FreeBSD, /usr/ports/net-mgmt/p5-Cflow
automatically detects the presence of the flow-tools libraries. Cflow links
this library as -lnsl, and if the build process doesn't find it
during the configure process you'll see a warning like:
Note (probably harmless): No library found for -lnsl
This warning is not harmless; it means that this Cflow build will not work
for you. If you don't see this line, just install Cflow and see if it works.
Cflow includes flowdumper(1), a program to read flow files on the
command line. Check the largest flow file you have, so that you can be sure
the record includes something to view.
#flowdumper -s ft-v05.2005-04-28.201501-0400 | more
2005/04/28 19:14:01 172.16.30.247.80 -> 216.98.200.250.63647 6(SYN|ACK) 3 144
2005/04/28 19:14:01 216.98.200.250.63647 -> 172.16.30.247.80 6(SYN) 1 48
2005/04/28 19:14:01 172.16.30.247.80 -> 216.98.200.250.63648 6(SYN|ACK) 3 144
2005/04/28 19:14:01 216.98.200.250.63648 -> 172.16.30.247.80 6(SYN) 1 48
...
Each line is a flow. This records the source and destination IPs of assorted TCP/IP transactions. You might notice that this particular snippet of four lines is actually only two TCP/IP sessions. The first line indicates that traffic is coming from 172.16.30.247, port 80, to the host 216.98.200.25. The next line shows traffic from the second host going to the first.
If your Cflow install is faulty, flowdumper will return either silence or an error. You cannot proceed until you resolve this error--at least, you can't proceed if you want your reporting tools to work! Uninstall your current p5-Cflow package and build it another way.
Remember how I said to not clean the flow-tools port? Go back to the port directory, cd to the work subdirectory, and go to the source code directory. There is another Cflow tarball in a subdirectory named contrib. Extract it.
# cd /usr/ports/net-mgmt/flow-tools/work/flow-tools-0.67/contrib
# tar -xzvf Cflow-1.051.tar.gz
#
Cflow frequently picks up the proper library when installed from this location under a compiled flow-tools package. (This means that you have to have a built flow-tools in the directory above you; this is why I told you not to do a make clean.) Just follow the usual Perl module building process.
# perl Makefile.PL
# make
# make install
Try flowdumper again, and it should work.
On occasion, I've had even this fail. In that case, use brute force. Flow-tools installs libft.a under /usr/local/lib. Edit the section of Cflow.pm's Makefile.PL where it checks for the flow-tools library:
sub find_flow_tools {
my($ver, $dir);
my($libdir, $incdir);
if (-f '../../lib/libft.a') {
$dir = '../../lib';
$incdir = "-I$dir -I$dir/..";
$libdir = "-L$dir";
}
Edit the line that reads
if (-f '../../lib/libft.a') {
to read
if (-f '/usr/local/lib/libft.a') {
If this fails, there's something seriously wrong with your Perl install.
Now, run make and make install. You now have a
flow-tools aware flowdumper, which indicates that the Cflow.pm Perl module
underlying it works correctly with your collector.
You can probably easily imagine a whole slew of Perl scripts that would take this data and generate pretty graphs and reports on usage, or identify peak bandwidth consumers. Other people have already done the heavy lifting on this one, however. My next article will look at creating pretty pictures from Netflow data.
Return to the Sysadmin DevCenter
You must be logged in to the O'Reilly Network to post a talkback.
Showing messages 1 through 12 of 12.
-
Caligare NetFlow Monitoring
2005-09-22 06:24:19 Caligare [Reply | View]
Hello,
did you try Caligare Flow Inspector tool? Author develop netflow monitoring software for 5 years. If you are interest in check Caligare home page: http://www.caligare.com/
Thanx, Jan
-
libnsl on FreeBSD
2005-09-16 14:57:44 lukasz@bromirski.net [Reply | View]
It's interesting to note, that unless You tried to install Linux emulation package, You won't have libnsl on Your system - it's integrated into libc and calling it out via -lnsl only breaks functionality (at least it's that way on 5.x).
So, You need to remove all references to -lnsl in Makefile.PL, and then it'll work out of the ports (make fetch, make extract, cd to work/etc, edit Makefile.PL removing references to -lnsl, save, go to main directory of the port and finish building with make all install). -
libnsl on FreeBSD
2005-09-17 18:10:52 jasontaylor1 [Reply | View]
Hi,
Any chance of posting a patch to Makefile.PL?
Thanks
Jason -
libnsl on FreeBSD
2005-09-17 23:17:39 lukasz@bromirski.net [Reply | View]
--- Makefile.PL Fri Jan 11 23:23:52 2002
+++ Makefile-wo-lnsl.PL Fri Sep 16 23:47:10 2005
@@ -35,7 +35,7 @@
} else {
# neither found... cflowd support only.
return { INC => join(' ', $incdir),
- LIBS => [ join(' ', $libdir, '-lnsl') ] }
+ LIBS => [ join(' ', $libdir ) ] }
}
}
@@ -51,7 +51,7 @@
print "Found flow-tools... using \"-DOSU $incdir $libdir -lft -lz\".\n";
return { CCFLAGS => '-DOSU',
INC => join(' ', $incdir),
- LIBS => [ join(' ', $libdir, '-lnsl -lft -lz') ] }
+ LIBS => [ join(' ', $libdir, '-lft -lz') ] }
}
return undef
}
@@ -68,7 +68,7 @@
print "Found argus... using \"-DARGUS $incdir $dir/argus_common.a $dir/argus_parse.a\".\n";
return { CCFLAGS => '-DARGUS',
INC => join(' ', $incdir),
- LIBS => [ join(' ', $libdir, '-lnsl', '-lm') ],
+ LIBS => [ join(' ', $libdir, '-lm') ],
LDFROM => "\$(OBJECT) $dir/argus_common.a $dir/argus_parse.a" }
}
return undef
-
libnsl on FreeBSD
2005-09-18 12:22:12 jasontaylor1 [Reply | View]
Thanks for that. It now compiles and installs without complaining.
However I get no output from flowdumper.
eg:
[jasont@sechost]/var/log/netflows: ls -l
total 1154
-rw-r--r-- 1 root wheel 8070 Sep 18 18:40 ft-v05.2005-09-18.183947+0100
-rw-r--r-- 1 root wheel 50481 Sep 18 18:45 ft-v05.2005-09-18.184001+0100
-rw-r--r-- 1 root wheel 60604 Sep 18 18:50 ft-v05.2005-09-18.184501+0100
-rw-r--r-- 1 root wheel 76010 Sep 18 18:55 ft-v05.2005-09-18.185001+0100
-rw-r--r-- 1 root wheel 56227 Sep 18 19:00 ft-v05.2005-09-18.185501+0100
-rw-r--r-- 1 root wheel 57349 Sep 18 19:05 ft-v05.2005-09-18.190001+0100
-rw-r--r-- 1 root wheel 44022 Sep 18 19:10 ft-v05.2005-09-18.190501+0100
-rw-r--r-- 1 root wheel 48894 Sep 18 19:15 ft-v05.2005-09-18.191001+0100
-rw-r--r-- 1 root wheel 62026 Sep 18 19:20 ft-v05.2005-09-18.191501+0100
-rw-r--r-- 1 root wheel 61056 Sep 18 19:25 ft-v05.2005-09-18.192001+0100
-rw-r--r-- 1 root wheel 44053 Sep 18 19:30 ft-v05.2005-09-18.192501+0100
-rw-r--r-- 1 root wheel 57780 Sep 18 19:35 ft-v05.2005-09-18.193001+0100
-rw-r--r-- 1 root wheel 44531 Sep 18 19:40 ft-v05.2005-09-18.193501+0100
-rw-r--r-- 1 root wheel 46751 Sep 18 19:45 ft-v05.2005-09-18.194001+0100
-rw-r--r-- 1 root wheel 58878 Sep 18 19:50 ft-v05.2005-09-18.194502+0100
-rw-r--r-- 1 root wheel 74163 Sep 18 19:55 ft-v05.2005-09-18.195001+0100
-rw-r--r-- 1 root wheel 73601 Sep 18 20:00 ft-v05.2005-09-18.195501+0100
-rw-r--r-- 1 root wheel 71073 Sep 18 20:05 ft-v05.2005-09-18.200001+0100
-rw-r--r-- 1 root wheel 60490 Sep 18 20:10 ft-v05.2005-09-18.200501+0100
-rw-r--r-- 1 root wheel 58576 Sep 18 20:15 ft-v05.2005-09-18.201001+0100
-rw-r--r-- 1 root wheel 45904 Sep 18 20:20 ft-v05.2005-09-18.201501+0100
-rw-r--r-- 1 root wheel 100 Sep 18 20:20 tmp-v05.2005-09-18.202001+0100
[jasont@sechost]/var/log/netflows: ls ft-* | xargs -n 1 flowdumper -s
[jasont@sechost]/var/log/netflows:
Any ideas? I've tried Cflow on a few different boxes now (Redhat 9, FreeBSD 4.8 and 5.4) with similar results.
Thanks
Jason -
libnsl on FreeBSD
2005-09-20 05:11:36 lukasz@bromirski.net [Reply | View]
All I can say is that with the warning Michael Lucas is warning us, flowdumper won't show me nothing on three separate machines (4.x, 5.x and 6.x). With the nsl removed from Makefile it goes well and flowdumper works - I'm using it currently and plan to deploy more installations.
This is propably my error as Michael can't be wrong, but the fact is - flow-tools are real nightmare to succesfully set up and a typical case how to not write and maintain software (IMHO of course). -
libnsl on FreeBSD
2005-09-20 05:25:32 agshekeloh [Reply | View]
I can be very wrong. Extremely wrong. Utterly, thoroughly, completely wrong. It seems that the more certain I am that I'm right, the more likely that I am wrong.
I can say that before I submitted this article, I sent it to two different people on two different networks to review and follow. It worked for them.
Netflow setup is a complete nightmare. Perhaps it worked more smoothly back when these tools were new, I don't know. Getting cflow.pm working is the worst part by far. Flowdumper is just the simplest front end and test for cflow.pm.
I know of people working on new flow capturing tools, but they're not ready yet. :-( -
libnsl on FreeBSD
2005-09-19 18:18:45 agshekeloh [Reply | View]
I've installed the tools in exactly this manner on eight different machines. Following the instructions *exactly* will make flowdumper work.
When you say "it doesn't complain," are you talking about the line that says "no library found for -libnls"? If so, did you make exactly the edit in the article, or did you use the patch in the posted followup comment?
I can't comment on the patch in the comment, but I can say that the edit in the article has always worked, both for me and a variety of other people. I have lost no functionality from the tools, on 4.x and 5.x. -
libnsl on FreeBSD
2006-08-07 04:04:21 rihad [Reply | View]
It's a pity all the needed software has to be set up in a generic Unix way: its authors are not adhering to FreeBSD standards (rcNG, etc). Every locally maintained ad-hoc startup script makes system upgrades a bit more painful and is just plain ugly.
After almost one year, thing have not changed.
Thanks for the HOWTO, though. Everything worked for me. -
libnsl on FreeBSD
2005-09-20 15:07:18 jasontaylor1 [Reply | View]
I tried both methods and played around a bit myself. Flowdumper simply produced no output when run against my collected flow files.
BUT.. I've just tried again using method on last page of article and it's now working...
Thanks for the encouragements to retry. Now to get the other tools in part 2 of the article working, which are also not working right.
Jason.
-
libnsl on FreeBSD
2008-03-04 10:23:34 Joao Ceron [Reply | View]
You must ensure that the file referenced by perl is the same as that compiled.
cd flow-tools/work/flow-tools-0.68/contrib/Cflow-1.051
agrius# md5 Cflow.pm /usr/local/lib/perl5/site_perl/5.8.8/mach/Cflow.pm
MD5 (Cflow.pm) = 19e30f1834c7f81c6aeddeb605993757
MD5 (/usr/local/lib/perl5/site_perl/5.8.8/mach/Cflow.pm) = 19e30f1834c7f81c6aeddeb605993757






http://www.akmalabs.com/downloads_flowmatrix.aspx
Thank you,
A.A