Mailing List Archive: NTop: Users

Netflow interfaces

Index | Next | Previous | View Threaded

gurtej2005 at gmail

Nov 23, 2005, 6:26 AM

Post #1 of 9 (1457 views)
Permalink

Netflow interfaces

Would anyone know how many router interface ntop can monitor for netflow
analysis.

thanks.

rlaager at wiktel

Jan 29, 2008, 3:14 PM

Post #2 of 9 (1080 views)
Permalink

Re: NetFlow Interfaces [In reply to]

On Tue, 2008-01-29 at 15:46 -0600, Gary Gatten wrote:
> Can you post the Netflow configs from your router(s)?

See the attached files.

> I too had issues with CPU on 3.3.3 on FreeBSD. I'm running 3.2.1 OK -
> but it segfaults and dies more than I'd like. I will tell you from what
> I've seen if there are a bunch of flows arriving to nTop when the
> netflow interfaces are activated it does take a LONG time to catch up
> and does consume 100% CPU until it does - even on 3.2.1. If you still
> have 3.3.x maybe turn it up when flow exports are light and see what
> happens.

I don't think this is it. I can try tonight, though, I suppose.

As another point of reference... I just disabled router2 (the one with
thousands of subinterfaces). To be clear, that left just router1 doing
netflows (and eth0, but the only traffic there is my browsing the ntop
HTTP interface); session tracking was off for all of this. Then, I
removed ntop 3.2 and installed ntop 3.3, which immediately maxed out the
CPU. I removed it and put 3.2 back and the CPU load is at most 5%.

I don't really know what it means, but in case this helps: According to
the traffic page, the total packets processed for router1 is 265,680 in
about 5 minutes.

Adding router2 back, still with ntop 3.2 and session tracking disabled,
brings me to about 10-15% CPU usage.

Richard

Attachments:

router1.cfg (1.23 KB)

router2.cfg (0.40 KB)

Ggatten at waddell

Jan 29, 2008, 3:26 PM

Post #3 of 9 (1074 views)
Permalink

Re: NetFlow Interfaces [In reply to]

I'm heading home soon so I'll take a look at this tomorrow. Just for
kicks, did you compile with the defaults? There are a TON of things to
tweak in some of the source files - globaldefines.h or something like
that is one of them. With as many hosts and flows as you may have it
might be worth looking into these even if it doesn't fix your immediate
problem.

FYI: On newer IOS's they support the command "ip flow ingress | egress"
within an interface. So, if the traffic you're wanting to monitor
eventually flows through one interface, you only have to configure "ip
flow ingress and egress" there - not on all 1000 interfaces - handy.

G

-----Original Message-----
From: ntop-bounces [at] unipi [mailto:ntop-bounces [at] unipi] On Behalf Of
Richard Laager
Sent: Tuesday, January 29, 2008 5:14 PM
To: ntop [at] unipi
Subject: Re: [Ntop] NetFlow Interfaces

On Tue, 2008-01-29 at 15:46 -0600, Gary Gatten wrote:
> Can you post the Netflow configs from your router(s)?

See the attached files.

> I too had issues with CPU on 3.3.3 on FreeBSD. I'm running 3.2.1 OK -
> but it segfaults and dies more than I'd like. I will tell you from
what
> I've seen if there are a bunch of flows arriving to nTop when the
> netflow interfaces are activated it does take a LONG time to catch up
> and does consume 100% CPU until it does - even on 3.2.1. If you still
> have 3.3.x maybe turn it up when flow exports are light and see what
> happens.

I don't think this is it. I can try tonight, though, I suppose.

As another point of reference... I just disabled router2 (the one with
thousands of subinterfaces). To be clear, that left just router1 doing
netflows (and eth0, but the only traffic there is my browsing the ntop
HTTP interface); session tracking was off for all of this. Then, I
removed ntop 3.2 and installed ntop 3.3, which immediately maxed out the
CPU. I removed it and put 3.2 back and the CPU load is at most 5%.

I don't really know what it means, but in case this helps: According to
the traffic page, the total packets processed for router1 is 265,680 in
about 5 minutes.

Adding router2 back, still with ntop 3.2 and session tracking disabled,
brings me to about 10-15% CPU usage.

Richard

<font size="1">
<div style='border:none;border-bottom:double windowtext 2.25pt;padding:0in 0in 1.0pt 0in'>
</div>
"This email is intended to be reviewed by only the intended recipient
and may contain information that is privileged and/or confidential.
If you are not the intended recipient, you are hereby notified that
any review, use, dissemination, disclosure or copying of this email
and its attachments, if any, is strictly prohibited. If you have
received this email in error, please immediately notify the sender by
return email and delete this email from your system."
</font>

_______________________________________________
Ntop mailing list
Ntop [at] unipi
http://listgateway.unipi.it/mailman/listinfo/ntop

rlaager at wiktel

Jan 29, 2008, 3:38 PM

Post #4 of 9 (1088 views)
Permalink

Re: NetFlow Interfaces [In reply to]

On Tue, 2008-01-29 at 17:26 -0600, Gary Gatten wrote:
> I'm heading home soon so I'll take a look at this tomorrow. Just for
> kicks, did you compile with the defaults?

I'm using the Ubuntu Dapper package for 3.2 and I build a 3.3 with the
3.3 tarball and the 3.2 packaging.

> FYI: On newer IOS's they support the command "ip flow ingress | egress"
> within an interface. So, if the traffic you're wanting to monitor
> eventually flows through one interface, you only have to configure "ip
> flow ingress and egress" there - not on all 1000 interfaces - handy.

Right. That would mean we'd miss out on local traffic between one
customer (subinterface) and another. Given my testing today, I doubt
that would help. We only have a handful of interfaces on the other
router and it can max out 3.3 by itself.

Thanks for your help on this. I look forward to hearing from you
tomorrow.

Richard

Attachments:

signature.asc (0.18 KB)

rlaager at wiktel

Jan 30, 2008, 2:14 PM

Post #5 of 9 (1074 views)
Permalink

Re: NetFlow Interfaces [In reply to]

On Tue, 2008-01-29 at 17:38 -0600, Richard Laager wrote:
> On Tue, 2008-01-29 at 17:26 -0600, Gary Gatten wrote:
> > I'm heading home soon so I'll take a look at this tomorrow. Just for
> > kicks, did you compile with the defaults?
>
> I'm using the Ubuntu Dapper package for 3.2 and I build a 3.3 with the
> 3.3 tarball and the 3.2 packaging.

I did some more testing today, with SVN revision 3415. Both that and the
version 3.3 from the tarball I had show the following issues:

With version 1: I get a HUGE explosion in the number of interfaces. I
have a handful of interfaces on the one router and I saw interface
numbers in the 50,000s. I wasn't able to display the netflow page with
all the graphs on it.

With version 5: Everything works for a bit, but the CPU maxes out and I
get the backlog. (This is the original issue in this thread.)

With version 9: All of the flows are shown as having an "unknown
template". I tried setting the template timeout to 1 minute and giving
ntop 5 minutes to ensure it got whatever templates the router needed to
send (note, I have no idea how this works; I'm using default templates
AFAIK). That didn't change anything.

Richard

P.S. Speaking of SVN... The etter.finger.os.gz file seems to be corrupt:

$ svn update
Restored 'etter.finger.os.gz'
At revision 3415.

$ gunzip etter.finger.os.gz

gunzip: etter.finger.os.gz: invalid compressed data--crc error

gunzip: etter.finger.os.gz: invalid compressed data--length error

Attachments:

signature.asc (0.18 KB)

Ggatten at waddell

Jan 31, 2010, 2:58 PM

Post #6 of 9 (758 views)
Permalink

Re: Netflow Interfaces [In reply to]

I'm guessing 32 is a max in globals-defines.h - hopefully only there. Browse around that file, increase to whatever your requirements are, and recompile.

----- Original Message -----
From: ntop-bounces [at] listgateway <ntop-bounces [at] listgateway>
To: ntop [at] listgateway <ntop [at] listgateway>
Sent: Sun Jan 31 15:53:50 2010
Subject: [Ntop] Netflow Interfaces

I added 32 interfaces, everything was fine until #32. After adding it
and attempting to set the interface name I get the following from the
web interface:

"Netflow configuration error. Unable to locate the specified device.
Please activate the plugin first."

And then ntop segfaulted and I got a kernel panic message in syslog.
I restarted ntop (started fine) and now when attempting to delete the
interfaces anytime I delete an interface with an ID >19 I get the same
error message.
_______________________________________________
Ntop mailing list
Ntop [at] listgateway
http://listgateway.unipi.it/mailman/listinfo/ntop

<font size="1">
<div style='border:none;border-bottom:double windowtext 2.25pt;padding:0in 0in 1.0pt 0in'>
</div>
"This email is intended to be reviewed by only the intended recipient
and may contain information that is privileged and/or confidential.
If you are not the intended recipient, you are hereby notified that
any review, use, dissemination, disclosure or copying of this email
and its attachments, if any, is strictly prohibited. If you have
received this email in error, please immediately notify the sender by
return email and delete this email from your system."
</font>

cmdrlinux at gmail

Jan 31, 2010, 3:39 PM

Post #7 of 9 (759 views)
Permalink

Re: Netflow Interfaces [In reply to]

I installed it from the EPEL repo trying to avoid maintaining it from
source if I can. Is there any way I can remove these additional
interfaces manually?

On Sun, Jan 31, 2010 at 4:58 PM, Gary Gatten <Ggatten [at] waddell> wrote:
> I'm guessing 32 is a max in globals-defines.h - hopefully only there.
> Browse around that file, increase to whatever your requirements are, and
> recompile.
>
> ----- Original Message -----
> From: ntop-bounces [at] listgateway <ntop-bounces [at] listgateway>
> To: ntop [at] listgateway <ntop [at] listgateway>
> Sent: Sun Jan 31 15:53:50 2010
> Subject: [Ntop] Netflow Interfaces
>
> I added 32 interfaces, everything was fine until #32.� After adding it
> and attempting to set the interface name I get the following from the
> web interface:
>
> "Netflow configuration error.� Unable to locate the specified device.
> Please activate the plugin first."
>
> And then ntop segfaulted and I got a kernel panic message in syslog.
> I restarted ntop (started fine) and now when attempting to delete the
> interfaces anytime I delete an interface with an ID >19 I get the same
> error message.
> _______________________________________________
> Ntop mailing list
> Ntop [at] listgateway
> http://listgateway.unipi.it/mailman/listinfo/ntop
>
> "This email is intended to be reviewed by only the intended recipient and
> may contain information that is privileged and/or confidential. If you are
> not the intended recipient, you are hereby notified that any review, use,
> dissemination, disclosure or copying of this email and its attachments, if
> any, is strictly prohibited. If you have received this email in error,
> please immediately notify the sender by return email and delete this email
> from your system."
> _______________________________________________
> Ntop mailing list
> Ntop [at] listgateway
> http://listgateway.unipi.it/mailman/listinfo/ntop
>
>
_______________________________________________
Ntop mailing list
Ntop [at] listgateway
http://listgateway.unipi.it/mailman/listinfo/ntop

Ggatten at waddell

Jan 31, 2010, 4:33 PM

Post #8 of 9 (755 views)
Permalink

Re: Netflow Interfaces [In reply to]

I think everything netflow is still stored in the prefscache.db file, along with many others customizations. Delete that file and start over.

----- Original Message -----
From: ntop-bounces [at] listgateway <ntop-bounces [at] listgateway>
To: ntop [at] unipi <ntop [at] unipi>
Sent: Sun Jan 31 17:39:04 2010
Subject: Re: [Ntop] Netflow Interfaces

I installed it from the EPEL repo trying to avoid maintaining it from
source if I can. Is there any way I can remove these additional
interfaces manually?

On Sun, Jan 31, 2010 at 4:58 PM, Gary Gatten <Ggatten [at] waddell> wrote:
> I'm guessing 32 is a max in globals-defines.h - hopefully only there.
> Browse around that file, increase to whatever your requirements are, and
> recompile.
>
> ----- Original Message -----
> From: ntop-bounces [at] listgateway <ntop-bounces [at] listgateway>
> To: ntop [at] listgateway <ntop [at] listgateway>
> Sent: Sun Jan 31 15:53:50 2010
> Subject: [Ntop] Netflow Interfaces
>
> I added 32 interfaces, everything was fine until #32. After adding it
> and attempting to set the interface name I get the following from the
> web interface:
>
> "Netflow configuration error. Unable to locate the specified device.
> Please activate the plugin first."
>
> And then ntop segfaulted and I got a kernel panic message in syslog.
> I restarted ntop (started fine) and now when attempting to delete the
> interfaces anytime I delete an interface with an ID >19 I get the same
> error message.
> _______________________________________________
> Ntop mailing list
> Ntop [at] listgateway
> http://listgateway.unipi.it/mailman/listinfo/ntop
>
> "This email is intended to be reviewed by only the intended recipient and
> may contain information that is privileged and/or confidential. If you are
> not the intended recipient, you are hereby notified that any review, use,
> dissemination, disclosure or copying of this email and its attachments, if
> any, is strictly prohibited. If you have received this email in error,
> please immediately notify the sender by return email and delete this email
> from your system."
> _______________________________________________
> Ntop mailing list
> Ntop [at] listgateway
> http://listgateway.unipi.it/mailman/listinfo/ntop
>
>
_______________________________________________
Ntop mailing list
Ntop [at] listgateway
http://listgateway.unipi.it/mailman/listinfo/ntop

<font size="1">
<div style='border:none;border-bottom:double windowtext 2.25pt;padding:0in 0in 1.0pt 0in'>
</div>
"This email is intended to be reviewed by only the intended recipient
and may contain information that is privileged and/or confidential.
If you are not the intended recipient, you are hereby notified that
any review, use, dissemination, disclosure or copying of this email
and its attachments, if any, is strictly prohibited. If you have
received this email in error, please immediately notify the sender by
return email and delete this email from your system."
</font>

cmdrlinux at gmail

Jan 31, 2010, 5:59 PM

Post #9 of 9 (761 views)
Permalink

Re: Netflow Interfaces [In reply to]

perfect, thank you for your help!

On Sun, Jan 31, 2010 at 6:33 PM, Gary Gatten <Ggatten [at] waddell> wrote:
> I think everything netflow is still stored in the prefscache.db file, along
> with many others customizations. Delete that file and start over.
>
> ----- Original Message -----
> From: ntop-bounces [at] listgateway <ntop-bounces [at] listgateway>
> To: ntop [at] unipi <ntop [at] unipi>
> Sent: Sun Jan 31 17:39:04 2010
> Subject: Re: [Ntop] Netflow Interfaces
>
> I installed it from the EPEL repo trying to avoid maintaining it from
> source if I can.� Is there any way I can remove these additional
> interfaces manually?
>
> On Sun, Jan 31, 2010 at 4:58 PM, Gary Gatten <Ggatten [at] waddell> wrote:
>> I'm guessing 32 is a max in globals-defines.h - hopefully only there.
>> Browse around that file, increase to whatever your requirements are, and
>> recompile.
>>
>> ----- Original Message -----
>> From: ntop-bounces [at] listgateway
>> <ntop-bounces [at] listgateway>
>> To: ntop [at] listgateway <ntop [at] listgateway>
>> Sent: Sun Jan 31 15:53:50 2010
>> Subject: [Ntop] Netflow Interfaces
>>
>> I added 32 interfaces, everything was fine until #32.� After adding it
>> and attempting to set the interface name I get the following from the
>> web interface:
>>
>> "Netflow configuration error.� Unable to locate the specified device.
>> Please activate the plugin first."
>>
>> And then ntop segfaulted and I got a kernel panic message in syslog.
>> I restarted ntop (started fine) and now when attempting to delete the
>> interfaces anytime I delete an interface with an ID >19 I get the same
>> error message.
>> _______________________________________________
>> Ntop mailing list
>> Ntop [at] listgateway
>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>
>> "This email is intended to be reviewed by only the intended recipient and
>> may contain information that is privileged and/or confidential. If you are
>> not the intended recipient, you are hereby notified that any review, use,
>> dissemination, disclosure or copying of this email and its attachments, if
>> any, is strictly prohibited. If you have received this email in error,
>> please immediately notify the sender by return email and delete this email
>> from your system."
>> _______________________________________________
>> Ntop mailing list
>> Ntop [at] listgateway
>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>
>>
> _______________________________________________
> Ntop mailing list
> Ntop [at] listgateway
> http://listgateway.unipi.it/mailman/listinfo/ntop
>
> "This email is intended to be reviewed by only the intended recipient and
> may contain information that is privileged and/or confidential. If you are
> not the intended recipient, you are hereby notified that any review, use,
> dissemination, disclosure or copying of this email and its attachments, if
> any, is strictly prohibited. If you have received this email in error,
> please immediately notify the sender by return email and delete this email
> from your system."
> _______________________________________________
> Ntop mailing list
> Ntop [at] listgateway
> http://listgateway.unipi.it/mailman/listinfo/ntop
>
>
_______________________________________________
Ntop mailing list
Ntop [at] listgateway
http://listgateway.unipi.it/mailman/listinfo/ntop

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads