lahondes at sibio
Jun 14, 2011, 10:50 AM
Post #1 of 1
(268 views)
Permalink
|
|
stability of RRD and ntop (and nProbe)
|
|
Hi,
I have some trouble making ntop (RRD in fact) stable. I have 8 probes (all some pfflowd daemon on 7 different pfsense (freebsd) firewalls) that send some NetFlow informations. Ntop works quite well for a few days but then some trouble arises...
I first noticed that there is a huge number (more than 100,000) of directories under /var/lib/ntop/rrd/<name of interface>/NetFlow. Curiously it happens only with some pfsenses.
I've tried 4 different installs (under FreeBSD, 2 different installs, with UFS then ZFS / DragonFlyBSD (Hammer) / Linux (tuned ext3)). Each time, there were issues with huge number of files (notably under UFS), either the kernel would crash (ZFS/Hammer) or ntop/RRD would crash (UFS/Ext3). Finally, under Linux/Debian, I have followed all recommandation here about filesystem:
http://oss.oetiker.ch/rrdtool-trac/wiki/TuningRRD"]http://oss.oetiker.ch/rrdtool-trac/wiki/TuningRRD
That is:
Files System Level Optimization - Set the noatime and nodiratime option when mounting the ext3 file system. This will save updates to the directory blocks. (DavePlonka??)
- Mount ext3 with data=writeback. This gives some performance boost at the expense of a possibility for old data to appear after a crash. Check man mount for details. (RyanKubica??)
- Use a file system that indexes directories (ext3 with dir_index) and keep all rrd files in a single directory (DavePlonka??)
Yet, after a few days, my RRD graphs just disappeared (they do not update). When I relaunch ntop, they come back again.
I have checked in FAQ and forums and it seems that I am the only one with this problem.
I have two questions :
- Have I done something wrong? Is it not some kind of well known problem with Ntop/NetFlow/RRD? (because I am nearly using it in a "out of the box" mode, just that I have 7 or 8 pfflowd probes, but I must not be the only one in this case).
- Would nProbe, which has database backends, help me get rid of that huge number of files or just stabilize ntop in this case?
Many thanks !
Raynald
PS: I use ntop version v4.0.3
|
