Mailing List Archive: NTop: Users

ntop requeriments for high usage

Index | Next | Previous | View Threaded

tiago at symetric

Jun 27, 2012, 4:58 AM

Post #1 of 10 (707 views)
Permalink

ntop requeriments for high usage

Hello,
We have a link of 500Mbps+ that are being collected using fprobe for test purposes. Its going fine.

However, as expected we're having issues with ntop processing that data and I've read a lot of threads since then.

I'm wondering, what kind of hardware specs would be fine to get ntop + fprobe flow working well? Or for my scenario ntop doesn't fit it at all?

Just for information, I'm getting the epipe errors / max hash entries / web blank pages on a intel xeon server X3430+ with 1GB RAM running ntop only.

I'm using ntop 3.3 based on ubuntu.

Thanks very much.

Tiago

Burton at ntopSupport

Jun 27, 2012, 5:36 AM

Post #2 of 10 (674 views)
Permalink

Re: ntop requeriments for high usage [In reply to]

1 GB is not a lot of memory – ntop’s database is purely in-memory.

ntop is threaded, but it doesn’t scale above a modest #. Still you will see
benefits up to 3 or 4 cores – 1 per network interface ntop is monitoring, 1
or 2 for packet decoding and 1 for the web interface, dns resolution, etc.

The info page has statistics about queue sizes – as long as ntop is keeping
up with them it’s not processor.

It’s been a while, but I remember it being about 30 – 40 KB per active (non
purged) host. 8 years ago I setup a client who monitored a 25GB/day traffic
flow with – for then – a huge 8GB of memory and idle purge turned off (it
was a limited non-public network).

-----Burton

From: ntop-bounces [at] listgateway
[mailto:ntop-bounces [at] listgateway] On Behalf Of Tiago A. Peçanha
Sent: Wednesday, June 27, 2012 4:58 AM
To: ntop [at] listgateway
Subject: [Ntop] ntop requeriments for high usage

Hello,

We have a link of 500Mbps+ that are being collected using fprobe for test
purposes. Its going fine.

However, as expected we're having issues with ntop processing that data and
I've read a lot of threads since then.

I'm wondering, what kind of hardware specs would be fine to get ntop +
fprobe flow working well? Or for my scenario ntop doesn't fit it at all?

Just for information, I'm getting the epipe errors / max hash entries / web
blank pages on a intel xeon server X3430+ with 1GB RAM running ntop only.

I'm using ntop 3.3 based on ubuntu.

Thanks very much.

Tiago

Burton at ntopSupport

Jun 27, 2012, 5:36 AM

Post #3 of 10 (673 views)
Permalink

Re: ntop requeriments for high usage [In reply to]

tiago at symetric

Jun 27, 2012, 1:49 PM

Post #4 of 10 (673 views)
Permalink

Re: ntop requeriments for high usage [In reply to]

Hello Burton,
In my case we use almost 3TB / day... and considering what you mention, I think I would need an hyper hyper hyper server =)

Maybe its time to look for other solutions?

----- Original Message -----
From: Burton Strauss III
To: ntop [at] unipi ; ntop [at] listgateway
Sent: Wednesday, June 27, 2012 9:36 AM
Subject: Re: [Ntop] ntop requeriments for high usage

1 GB is not a lot of memory - ntop's database is purely in-memory.

ntop is threaded, but it doesn't scale above a modest #. Still you will see benefits up to 3 or 4 cores - 1 per network interface ntop is monitoring, 1 or 2 for packet decoding and 1 for the web interface, dns resolution, etc.

The info page has statistics about queue sizes - as long as ntop is keeping up with them it's not processor.

It's been a while, but I remember it being about 30 - 40 KB per active (non purged) host. 8 years ago I setup a client who monitored a 25GB/day traffic flow with - for then - a huge 8GB of memory and idle purge turned off (it was a limited non-public network).

-----Burton

From: ntop-bounces [at] listgateway [mailto:ntop-bounces [at] listgateway] On Behalf Of Tiago A. Peçanha
Sent: Wednesday, June 27, 2012 4:58 AM
To: ntop [at] listgateway
Subject: [Ntop] ntop requeriments for high usage

Hello,

We have a link of 500Mbps+ that are being collected using fprobe for test purposes. Its going fine.

However, as expected we're having issues with ntop processing that data and I've read a lot of threads since then.

I'm wondering, what kind of hardware specs would be fine to get ntop + fprobe flow working well? Or for my scenario ntop doesn't fit it at all?

Just for information, I'm getting the epipe errors / max hash entries / web blank pages on a intel xeon server X3430+ with 1GB RAM running ntop only.

I'm using ntop 3.3 based on ubuntu.

Thanks very much.

Tiago

------------------------------------------------------------------------------

_______________________________________________
Ntop mailing list
Ntop [at] listgateway
http://listgateway.unipi.it/mailman/listinfo/ntop

Burton at ntopSupport

Jun 27, 2012, 8:11 PM

Post #5 of 10 (659 views)
Permalink

Re: ntop requeriments for high usage [In reply to]

Netflow and such drastically reduce the workload since you are dealing with
flows not individual packets.

But no tool is going to work at 3TB/s on such puny hardware - basically, you
either spend some real time for your own and $ on hardware, OR $ for an
integrator to buy and integrate a solution OR $$$ for a commercial class
solution.

Your choice J

-----Burton

From: ntop-bounces [at] listgateway
[mailto:ntop-bounces [at] listgateway] On Behalf Of Tiago A. Peçanha
Sent: Wednesday, June 27, 2012 1:50 PM
To: ntop [at] unipi
Subject: Re: [Ntop] ntop requeriments for high usage

Hello Burton,

In my case we use almost 3TB / day... and considering what you mention, I
think I would need an hyper hyper hyper server =)

Maybe its time to look for other solutions?

----- Original Message -----

From: Burton Strauss III <mailto:Burton [at] ntopSupport>

To: ntop [at] unipi ; ntop [at] listgateway

Sent: Wednesday, June 27, 2012 9:36 AM

Subject: Re: [Ntop] ntop requeriments for high usage

1 GB is not a lot of memory – ntop’s database is purely in-memory.

ntop is threaded, but it doesn’t scale above a modest #. Still you will see
benefits up to 3 or 4 cores – 1 per network interface ntop is monitoring, 1
or 2 for packet decoding and 1 for the web interface, dns resolution, etc.

The info page has statistics about queue sizes – as long as ntop is keeping
up with them it’s not processor.

It’s been a while, but I remember it being about 30 – 40 KB per active (non
purged) host. 8 years ago I setup a client who monitored a 25GB/day traffic
flow with – for then – a huge 8GB of memory and idle purge turned off (it
was a limited non-public network).

-----Burton

From: ntop-bounces [at] listgateway
[mailto:ntop-bounces [at] listgateway] On Behalf Of Tiago A. Peçanha
Sent: Wednesday, June 27, 2012 4:58 AM
To: ntop [at] listgateway
Subject: [Ntop] ntop requeriments for high usage

Hello,

We have a link of 500Mbps+ that are being collected using fprobe for test
purposes. Its going fine.

However, as expected we're having issues with ntop processing that data and
I've read a lot of threads since then.

I'm wondering, what kind of hardware specs would be fine to get ntop +
fprobe flow working well? Or for my scenario ntop doesn't fit it at all?

Just for information, I'm getting the epipe errors / max hash entries / web
blank pages on a intel xeon server X3430+ with 1GB RAM running ntop only.

I'm using ntop 3.3 based on ubuntu.

Thanks very much.

Tiago

_____

_______________________________________________
Ntop mailing list
Ntop [at] listgateway
http://listgateway.unipi.it/mailman/listinfo/ntop

tiago at symetric

Jun 28, 2012, 4:10 AM

Post #6 of 10 (661 views)
Permalink

Re: ntop requeriments for high usage [In reply to]

I'm using fprobe for flows. As I said my main problem is analysis and the question...:

Would it work on quadcore / 4GB RAM? 8GB RAM? Because if it work I'm willing to buy/test hardware that could run. But I need to know that will run heheh..

Thanks
----- Original Message -----
From: Burton Strauss III
To: ntop [at] unipi
Sent: Thursday, June 28, 2012 12:11 AM
Subject: Re: [Ntop] ntop requeriments for high usage

Netflow and such drastically reduce the workload since you are dealing with flows not individual packets.

But no tool is going to work at 3TB/s on such puny hardware - basically, you either spend some real time for your own and $ on hardware, OR $ for an integrator to buy and integrate a solution OR $$$ for a commercial class solution.

Your choice J

-----Burton

From: ntop-bounces [at] listgateway [mailto:ntop-bounces [at] listgateway] On Behalf Of Tiago A. Peçanha
Sent: Wednesday, June 27, 2012 1:50 PM
To: ntop [at] unipi
Subject: Re: [Ntop] ntop requeriments for high usage

Hello Burton,

In my case we use almost 3TB / day... and considering what you mention, I think I would need an hyper hyper hyper server =)

Maybe its time to look for other solutions?

----- Original Message -----

From: Burton Strauss III

To: ntop [at] unipi ; ntop [at] listgateway

Sent: Wednesday, June 27, 2012 9:36 AM

Subject: Re: [Ntop] ntop requeriments for high usage

1 GB is not a lot of memory - ntop's database is purely in-memory.

ntop is threaded, but it doesn't scale above a modest #. Still you will see benefits up to 3 or 4 cores - 1 per network interface ntop is monitoring, 1 or 2 for packet decoding and 1 for the web interface, dns resolution, etc.

The info page has statistics about queue sizes - as long as ntop is keeping up with them it's not processor.

It's been a while, but I remember it being about 30 - 40 KB per active (non purged) host. 8 years ago I setup a client who monitored a 25GB/day traffic flow with - for then - a huge 8GB of memory and idle purge turned off (it was a limited non-public network).

-----Burton

From: ntop-bounces [at] listgateway [mailto:ntop-bounces [at] listgateway] On Behalf Of Tiago A. Peçanha
Sent: Wednesday, June 27, 2012 4:58 AM
To: ntop [at] listgateway
Subject: [Ntop] ntop requeriments for high usage

Hello,

We have a link of 500Mbps+ that are being collected using fprobe for test purposes. Its going fine.

However, as expected we're having issues with ntop processing that data and I've read a lot of threads since then.

I'm wondering, what kind of hardware specs would be fine to get ntop + fprobe flow working well? Or for my scenario ntop doesn't fit it at all?

Just for information, I'm getting the epipe errors / max hash entries / web blank pages on a intel xeon server X3430+ with 1GB RAM running ntop only.

I'm using ntop 3.3 based on ubuntu.

Thanks very much.

Tiago

----------------------------------------------------------------------------

_______________________________________________
Ntop mailing list
Ntop [at] listgateway
http://listgateway.unipi.it/mailman/listinfo/ntop

------------------------------------------------------------------------------

_______________________________________________
Ntop mailing list
Ntop [at] listgateway
http://listgateway.unipi.it/mailman/listinfo/ntop

Burton at ntopSupport

Jun 28, 2012, 5:49 AM

Post #7 of 10 (660 views)
Permalink

Re: ntop requeriments for high usage [In reply to]

I don’t think anybody can tell you – that’s what I was saying, either act as
your own integrator or pay somebody to do so. You are asking a high-end
support question on a list basically populated by other users who show up,
ask a question or two and rarely stick around to answer any more questions.
Most of whom are running on-the-cheap trying desperately to get SOME
information from within their organization to understand even a little of
what is going on.

If you want to find out whether it will work, try it. The nice thing about
nTop is that it doesn’t take long to install Linux on an OS, install nTop
and try it. If you have the hardware on your floor, it’s an hour’s
investment.

But seriously-

Luca answers some questions; he has an employee who answers some more. A
couple of the other committers will answer something that falls into an area
they know.

I was the generalist. When I closed ntopSupport.com in 2006 – in part
because nobody would pay for support for ‘free’ software and in part because
I got a real job (driven by the former), one of the other users stepped up
and took over the role of answering questions for quite a number of years.
He was one of the largest users of nTop and while not a code-monkey himself,
he was a great coordinator and passer-along of hard won knowledge. When he
lost that job, he disappeared.

I haven’t touched the code since 2006, except for a very brief period where
I was working on trying to figure out the DNS name resolution problem (so
only one tiny bit of the code) since. And I’m the best you’ve got.

Does that help to level-set?

-----Burton

From: ntop-bounces [at] listgateway
[mailto:ntop-bounces [at] listgateway] On Behalf Of Tiago A. Peçanha
Sent: Thursday, June 28, 2012 4:11 AM
To: ntop [at] unipi
Subject: Re: [Ntop] ntop requeriments for high usage

I'm using fprobe for flows. As I said my main problem is analysis and the
question...:

Would it work on quadcore / 4GB RAM? 8GB RAM? Because if it work I'm willing
to buy/test hardware that could run. But I need to know that will run
heheh..

Thanks

----- Original Message -----

From: Burton Strauss III <mailto:Burton [at] ntopSupport>

To: ntop [at] unipi

Sent: Thursday, June 28, 2012 12:11 AM

Subject: Re: [Ntop] ntop requeriments for high usage

Netflow and such drastically reduce the workload since you are dealing with
flows not individual packets.

But no tool is going to work at 3TB/s on such puny hardware - basically, you
either spend some real time for your own and $ on hardware, OR $ for an
integrator to buy and integrate a solution OR $$$ for a commercial class
solution.

Your choice J

-----Burton

From: ntop-bounces [at] listgateway
[mailto:ntop-bounces [at] listgateway] On Behalf Of Tiago A. Peçanha
Sent: Wednesday, June 27, 2012 1:50 PM
To: ntop [at] unipi
Subject: Re: [Ntop] ntop requeriments for high usage

Hello Burton,

In my case we use almost 3TB / day... and considering what you mention, I
think I would need an hyper hyper hyper server =)

Maybe its time to look for other solutions?

----- Original Message -----

From: Burton Strauss III <mailto:Burton [at] ntopSupport>

To: ntop [at] unipi ; ntop [at] listgateway

Sent: Wednesday, June 27, 2012 9:36 AM

Subject: Re: [Ntop] ntop requeriments for high usage

1 GB is not a lot of memory – ntop’s database is purely in-memory.

ntop is threaded, but it doesn’t scale above a modest #. Still you will see
benefits up to 3 or 4 cores – 1 per network interface ntop is monitoring, 1
or 2 for packet decoding and 1 for the web interface, dns resolution, etc.

The info page has statistics about queue sizes – as long as ntop is keeping
up with them it’s not processor.

It’s been a while, but I remember it being about 30 – 40 KB per active (non
purged) host. 8 years ago I setup a client who monitored a 25GB/day traffic
flow with – for then – a huge 8GB of memory and idle purge turned off (it
was a limited non-public network).

-----Burton

From: ntop-bounces [at] listgateway
[mailto:ntop-bounces [at] listgateway] On Behalf Of Tiago A. Peçanha
Sent: Wednesday, June 27, 2012 4:58 AM
To: ntop [at] listgateway
Subject: [Ntop] ntop requeriments for high usage

Hello,

We have a link of 500Mbps+ that are being collected using fprobe for test
purposes. Its going fine.

However, as expected we're having issues with ntop processing that data and
I've read a lot of threads since then.

I'm wondering, what kind of hardware specs would be fine to get ntop +
fprobe flow working well? Or for my scenario ntop doesn't fit it at all?

Just for information, I'm getting the epipe errors / max hash entries / web
blank pages on a intel xeon server X3430+ with 1GB RAM running ntop only.

I'm using ntop 3.3 based on ubuntu.

Thanks very much.

Tiago

_____

_______________________________________________
Ntop mailing list
Ntop [at] listgateway
http://listgateway.unipi.it/mailman/listinfo/ntop

_____

_______________________________________________
Ntop mailing list
Ntop [at] listgateway
http://listgateway.unipi.it/mailman/listinfo/ntop

tiago at symetric

Jun 28, 2012, 8:52 AM

Post #8 of 10 (660 views)
Permalink

Re: ntop requeriments for high usage [In reply to]

Yes, it helped :)

Thanks very much!
----- Original Message -----
From: Burton Strauss III
To: ntop [at] unipi
Sent: Thursday, June 28, 2012 9:49 AM
Subject: Re: [Ntop] ntop requeriments for high usage

I don't think anybody can tell you - that's what I was saying, either act as your own integrator or pay somebody to do so. You are asking a high-end support question on a list basically populated by other users who show up, ask a question or two and rarely stick around to answer any more questions. Most of whom are running on-the-cheap trying desperately to get SOME information from within their organization to understand even a little of what is going on.

If you want to find out whether it will work, try it. The nice thing about nTop is that it doesn't take long to install Linux on an OS, install nTop and try it. If you have the hardware on your floor, it's an hour's investment.

But seriously-

Luca answers some questions; he has an employee who answers some more. A couple of the other committers will answer something that falls into an area they know.

I was the generalist. When I closed ntopSupport.com in 2006 - in part because nobody would pay for support for 'free' software and in part because I got a real job (driven by the former), one of the other users stepped up and took over the role of answering questions for quite a number of years. He was one of the largest users of nTop and while not a code-monkey himself, he was a great coordinator and passer-along of hard won knowledge. When he lost that job, he disappeared.

I haven't touched the code since 2006, except for a very brief period where I was working on trying to figure out the DNS name resolution problem (so only one tiny bit of the code) since. And I'm the best you've got.

Does that help to level-set?

-----Burton

From: ntop-bounces [at] listgateway [mailto:ntop-bounces [at] listgateway] On Behalf Of Tiago A. Peçanha
Sent: Thursday, June 28, 2012 4:11 AM
To: ntop [at] unipi
Subject: Re: [Ntop] ntop requeriments for high usage

I'm using fprobe for flows. As I said my main problem is analysis and the question...:

Would it work on quadcore / 4GB RAM? 8GB RAM? Because if it work I'm willing to buy/test hardware that could run. But I need to know that will run heheh..

Thanks

----- Original Message -----

From: Burton Strauss III

To: ntop [at] unipi

Sent: Thursday, June 28, 2012 12:11 AM

Subject: Re: [Ntop] ntop requeriments for high usage

Netflow and such drastically reduce the workload since you are dealing with flows not individual packets.

But no tool is going to work at 3TB/s on such puny hardware - basically, you either spend some real time for your own and $ on hardware, OR $ for an integrator to buy and integrate a solution OR $$$ for a commercial class solution.

Your choice J

-----Burton

From: ntop-bounces [at] listgateway [mailto:ntop-bounces [at] listgateway] On Behalf Of Tiago A. Peçanha
Sent: Wednesday, June 27, 2012 1:50 PM
To: ntop [at] unipi
Subject: Re: [Ntop] ntop requeriments for high usage

Hello Burton,

In my case we use almost 3TB / day... and considering what you mention, I think I would need an hyper hyper hyper server =)

Maybe its time to look for other solutions?

----- Original Message -----

From: Burton Strauss III

To: ntop [at] unipi ; ntop [at] listgateway

Sent: Wednesday, June 27, 2012 9:36 AM

Subject: Re: [Ntop] ntop requeriments for high usage

1 GB is not a lot of memory - ntop's database is purely in-memory.

ntop is threaded, but it doesn't scale above a modest #. Still you will see benefits up to 3 or 4 cores - 1 per network interface ntop is monitoring, 1 or 2 for packet decoding and 1 for the web interface, dns resolution, etc.

The info page has statistics about queue sizes - as long as ntop is keeping up with them it's not processor.

It's been a while, but I remember it being about 30 - 40 KB per active (non purged) host. 8 years ago I setup a client who monitored a 25GB/day traffic flow with - for then - a huge 8GB of memory and idle purge turned off (it was a limited non-public network).

-----Burton

From: ntop-bounces [at] listgateway [mailto:ntop-bounces [at] listgateway] On Behalf Of Tiago A. Peçanha
Sent: Wednesday, June 27, 2012 4:58 AM
To: ntop [at] listgateway
Subject: [Ntop] ntop requeriments for high usage

Hello,

We have a link of 500Mbps+ that are being collected using fprobe for test purposes. Its going fine.

However, as expected we're having issues with ntop processing that data and I've read a lot of threads since then.

I'm wondering, what kind of hardware specs would be fine to get ntop + fprobe flow working well? Or for my scenario ntop doesn't fit it at all?

Just for information, I'm getting the epipe errors / max hash entries / web blank pages on a intel xeon server X3430+ with 1GB RAM running ntop only.

I'm using ntop 3.3 based on ubuntu.

Thanks very much.

Tiago

--------------------------------------------------------------------------

_______________________________________________
Ntop mailing list
Ntop [at] listgateway
http://listgateway.unipi.it/mailman/listinfo/ntop

----------------------------------------------------------------------------

_______________________________________________
Ntop mailing list
Ntop [at] listgateway
http://listgateway.unipi.it/mailman/listinfo/ntop

------------------------------------------------------------------------------

_______________________________________________
Ntop mailing list
Ntop [at] listgateway
http://listgateway.unipi.it/mailman/listinfo/ntop

r.bortolameotti at gmail

Jul 16, 2012, 1:18 AM

Post #9 of 10 (514 views)
Permalink

Re: ntop requeriments for high usage [In reply to]

Hi guys,

i've had the same problem. I'm doing a project for my University so
i've no money to spend. Probably it is possible to monitor your
network not directly on your last network node, which is linked with
the Internet, but configuring ntop on several machines within your
network. On this way, you don't need expensive hardware. It's a kind
of "distributed system". I don't know yet if is possibile to set up
ntop on several different machines and use a "main machine" where ntop
can shows all data elaborated by other ntops.

2012/6/28 Tiago A. Peçanha <tiago [at] symetric>:
> Yes, it helped :)
>
> Thanks very much!
>
> ----- Original Message -----
> From: Burton Strauss III
> To: ntop [at] unipi
> Sent: Thursday, June 28, 2012 9:49 AM
> Subject: Re: [Ntop] ntop requeriments for high usage
>
> I don’t think anybody can tell you – that’s what I was saying, either act as
> your own integrator or pay somebody to do so. You are asking a high-end
> support question on a list basically populated by other users who show up,
> ask a question or two and rarely stick around to answer any more questions.
> Most of whom are running on-the-cheap trying desperately to get SOME
> information from within their organization to understand even a little of
> what is going on.
>
>
>
> If you want to find out whether it will work, try it. The nice thing about
> nTop is that it doesn’t take long to install Linux on an OS, install nTop
> and try it. If you have the hardware on your floor, it’s an hour’s
> investment.
>
>
>
>
>
> But seriously-
>
>
>
> Luca answers some questions; he has an employee who answers some more. A
> couple of the other committers will answer something that falls into an area
> they know.
>
>
>
> I was the generalist. When I closed ntopSupport.com in 2006 – in part
> because nobody would pay for support for ‘free’ software and in part because
> I got a real job (driven by the former), one of the other users stepped up
> and took over the role of answering questions for quite a number of years.
> He was one of the largest users of nTop and while not a code-monkey himself,
> he was a great coordinator and passer-along of hard won knowledge. When he
> lost that job, he disappeared.
>
>
>
> I haven’t touched the code since 2006, except for a very brief period where
> I was working on trying to figure out the DNS name resolution problem (so
> only one tiny bit of the code) since. And I’m the best you’ve got.
>
>
>
>
>
> Does that help to level-set?
>
>
>
> -----Burton
>
>
>
>
>
> From: ntop-bounces [at] listgateway
> [mailto:ntop-bounces [at] listgateway] On Behalf Of Tiago A. Peçanha
> Sent: Thursday, June 28, 2012 4:11 AM
> To: ntop [at] unipi
> Subject: Re: [Ntop] ntop requeriments for high usage
>
>
>
> I'm using fprobe for flows. As I said my main problem is analysis and the
> question...:
>
>
>
> Would it work on quadcore / 4GB RAM? 8GB RAM? Because if it work I'm willing
> to buy/test hardware that could run. But I need to know that will run
> heheh..
>
>
>
> Thanks
>
> ----- Original Message -----
>
> From: Burton Strauss III
>
> To: ntop [at] unipi
>
> Sent: Thursday, June 28, 2012 12:11 AM
>
> Subject: Re: [Ntop] ntop requeriments for high usage
>
>
>
> Netflow and such drastically reduce the workload since you are dealing with
> flows not individual packets.
>
>
>
> But no tool is going to work at 3TB/s on such puny hardware - basically, you
> either spend some real time for your own and $ on hardware, OR $ for an
> integrator to buy and integrate a solution OR $$$ for a commercial class
> solution.
>
>
>
> Your choice J
>
>
>
> -----Burton
>
>
>
> From: ntop-bounces [at] listgateway
> [mailto:ntop-bounces [at] listgateway] On Behalf Of Tiago A. Peçanha
> Sent: Wednesday, June 27, 2012 1:50 PM
> To: ntop [at] unipi
> Subject: Re: [Ntop] ntop requeriments for high usage
>
>
>
> Hello Burton,
>
> In my case we use almost 3TB / day... and considering what you mention, I
> think I would need an hyper hyper hyper server =)
>
>
>
> Maybe its time to look for other solutions?
>
>
>
>
>
> ----- Original Message -----
>
> From: Burton Strauss III
>
> To: ntop [at] unipi ; ntop [at] listgateway
>
> Sent: Wednesday, June 27, 2012 9:36 AM
>
> Subject: Re: [Ntop] ntop requeriments for high usage
>
>
>
> 1 GB is not a lot of memory – ntop’s database is purely in-memory.
>
>
>
> ntop is threaded, but it doesn’t scale above a modest #. Still you will see
> benefits up to 3 or 4 cores – 1 per network interface ntop is monitoring, 1
> or 2 for packet decoding and 1 for the web interface, dns resolution, etc.
>
>
>
> The info page has statistics about queue sizes – as long as ntop is keeping
> up with them it’s not processor.
>
>
>
>
>
> It’s been a while, but I remember it being about 30 – 40 KB per active (non
> purged) host. 8 years ago I setup a client who monitored a 25GB/day traffic
> flow with – for then – a huge 8GB of memory and idle purge turned off (it
> was a limited non-public network).
>
>
>
> -----Burton
>
>
>
> From: ntop-bounces [at] listgateway
> [mailto:ntop-bounces [at] listgateway] On Behalf Of Tiago A. Peçanha
> Sent: Wednesday, June 27, 2012 4:58 AM
> To: ntop [at] listgateway
> Subject: [Ntop] ntop requeriments for high usage
>
>
>
> Hello,
>
> We have a link of 500Mbps+ that are being collected using fprobe for test
> purposes. Its going fine.
>
>
>
> However, as expected we're having issues with ntop processing that data and
> I've read a lot of threads since then.
>
>
>
> I'm wondering, what kind of hardware specs would be fine to get ntop +
> fprobe flow working well? Or for my scenario ntop doesn't fit it at all?
>
>
>
> Just for information, I'm getting the epipe errors / max hash entries / web
> blank pages on a intel xeon server X3430+ with 1GB RAM running ntop only.
>
>
>
> I'm using ntop 3.3 based on ubuntu.
>
>
>
> Thanks very much.
>
>
>
> Tiago
>
> ________________________________
>
> _______________________________________________
> Ntop mailing list
> Ntop [at] listgateway
> http://listgateway.unipi.it/mailman/listinfo/ntop
>
> ________________________________
>
> _______________________________________________
> Ntop mailing list
> Ntop [at] listgateway
> http://listgateway.unipi.it/mailman/listinfo/ntop
>
> ________________________________
>
> _______________________________________________
> Ntop mailing list
> Ntop [at] listgateway
> http://listgateway.unipi.it/mailman/listinfo/ntop
>
>
> _______________________________________________
> Ntop mailing list
> Ntop [at] listgateway
> http://listgateway.unipi.it/mailman/listinfo/ntop
>

--
R.
_______________________________________________
Ntop mailing list
Ntop [at] listgateway
http://listgateway.unipi.it/mailman/listinfo/ntop

r.bortolameotti at gmail

Jul 16, 2012, 1:25 AM

Post #10 of 10 (511 views)
Permalink

Re: ntop requeriments for high usage [In reply to]

Oh of course another possible improvement could be a script which
restart ntop everyday (i.e. 2 am), consequently your memory is not
overloaded. (Dumping your data before reset, otherwise your data will
be lost)

2012/7/16 Riccardo Bortolameotti <r.bortolameotti [at] gmail>:
> Hi guys,
>
> i've had the same problem. I'm doing a project for my University so
> i've no money to spend. Probably it is possible to monitor your
> network not directly on your last network node, which is linked with
> the Internet, but configuring ntop on several machines within your
> network. On this way, you don't need expensive hardware. It's a kind
> of "distributed system". I don't know yet if is possibile to set up
> ntop on several different machines and use a "main machine" where ntop
> can shows all data elaborated by other ntops.
>
>
> 2012/6/28 Tiago A. Peçanha <tiago [at] symetric>:
>> Yes, it helped :)
>>
>> Thanks very much!
>>
>> ----- Original Message -----
>> From: Burton Strauss III
>> To: ntop [at] unipi
>> Sent: Thursday, June 28, 2012 9:49 AM
>> Subject: Re: [Ntop] ntop requeriments for high usage
>>
>> I don’t think anybody can tell you – that’s what I was saying, either act as
>> your own integrator or pay somebody to do so. You are asking a high-end
>> support question on a list basically populated by other users who show up,
>> ask a question or two and rarely stick around to answer any more questions.
>> Most of whom are running on-the-cheap trying desperately to get SOME
>> information from within their organization to understand even a little of
>> what is going on.
>>
>>
>>
>> If you want to find out whether it will work, try it. The nice thing about
>> nTop is that it doesn’t take long to install Linux on an OS, install nTop
>> and try it. If you have the hardware on your floor, it’s an hour’s
>> investment.
>>
>>
>>
>>
>>
>> But seriously-
>>
>>
>>
>> Luca answers some questions; he has an employee who answers some more. A
>> couple of the other committers will answer something that falls into an area
>> they know.
>>
>>
>>
>> I was the generalist. When I closed ntopSupport.com in 2006 – in part
>> because nobody would pay for support for ‘free’ software and in part because
>> I got a real job (driven by the former), one of the other users stepped up
>> and took over the role of answering questions for quite a number of years.
>> He was one of the largest users of nTop and while not a code-monkey himself,
>> he was a great coordinator and passer-along of hard won knowledge. When he
>> lost that job, he disappeared.
>>
>>
>>
>> I haven’t touched the code since 2006, except for a very brief period where
>> I was working on trying to figure out the DNS name resolution problem (so
>> only one tiny bit of the code) since. And I’m the best you’ve got.
>>
>>
>>
>>
>>
>> Does that help to level-set?
>>
>>
>>
>> -----Burton
>>
>>
>>
>>
>>
>> From: ntop-bounces [at] listgateway
>> [mailto:ntop-bounces [at] listgateway] On Behalf Of Tiago A. Peçanha
>> Sent: Thursday, June 28, 2012 4:11 AM
>> To: ntop [at] unipi
>> Subject: Re: [Ntop] ntop requeriments for high usage
>>
>>
>>
>> I'm using fprobe for flows. As I said my main problem is analysis and the
>> question...:
>>
>>
>>
>> Would it work on quadcore / 4GB RAM? 8GB RAM? Because if it work I'm willing
>> to buy/test hardware that could run. But I need to know that will run
>> heheh..
>>
>>
>>
>> Thanks
>>
>> ----- Original Message -----
>>
>> From: Burton Strauss III
>>
>> To: ntop [at] unipi
>>
>> Sent: Thursday, June 28, 2012 12:11 AM
>>
>> Subject: Re: [Ntop] ntop requeriments for high usage
>>
>>
>>
>> Netflow and such drastically reduce the workload since you are dealing with
>> flows not individual packets.
>>
>>
>>
>> But no tool is going to work at 3TB/s on such puny hardware - basically, you
>> either spend some real time for your own and $ on hardware, OR $ for an
>> integrator to buy and integrate a solution OR $$$ for a commercial class
>> solution.
>>
>>
>>
>> Your choice J
>>
>>
>>
>> -----Burton
>>
>>
>>
>> From: ntop-bounces [at] listgateway
>> [mailto:ntop-bounces [at] listgateway] On Behalf Of Tiago A. Peçanha
>> Sent: Wednesday, June 27, 2012 1:50 PM
>> To: ntop [at] unipi
>> Subject: Re: [Ntop] ntop requeriments for high usage
>>
>>
>>
>> Hello Burton,
>>
>> In my case we use almost 3TB / day... and considering what you mention, I
>> think I would need an hyper hyper hyper server =)
>>
>>
>>
>> Maybe its time to look for other solutions?
>>
>>
>>
>>
>>
>> ----- Original Message -----
>>
>> From: Burton Strauss III
>>
>> To: ntop [at] unipi ; ntop [at] listgateway
>>
>> Sent: Wednesday, June 27, 2012 9:36 AM
>>
>> Subject: Re: [Ntop] ntop requeriments for high usage
>>
>>
>>
>> 1 GB is not a lot of memory – ntop’s database is purely in-memory.
>>
>>
>>
>> ntop is threaded, but it doesn’t scale above a modest #. Still you will see
>> benefits up to 3 or 4 cores – 1 per network interface ntop is monitoring, 1
>> or 2 for packet decoding and 1 for the web interface, dns resolution, etc.
>>
>>
>>
>> The info page has statistics about queue sizes – as long as ntop is keeping
>> up with them it’s not processor.
>>
>>
>>
>>
>>
>> It’s been a while, but I remember it being about 30 – 40 KB per active (non
>> purged) host. 8 years ago I setup a client who monitored a 25GB/day traffic
>> flow with – for then – a huge 8GB of memory and idle purge turned off (it
>> was a limited non-public network).
>>
>>
>>
>> -----Burton
>>
>>
>>
>> From: ntop-bounces [at] listgateway
>> [mailto:ntop-bounces [at] listgateway] On Behalf Of Tiago A. Peçanha
>> Sent: Wednesday, June 27, 2012 4:58 AM
>> To: ntop [at] listgateway
>> Subject: [Ntop] ntop requeriments for high usage
>>
>>
>>
>> Hello,
>>
>> We have a link of 500Mbps+ that are being collected using fprobe for test
>> purposes. Its going fine.
>>
>>
>>
>> However, as expected we're having issues with ntop processing that data and
>> I've read a lot of threads since then.
>>
>>
>>
>> I'm wondering, what kind of hardware specs would be fine to get ntop +
>> fprobe flow working well? Or for my scenario ntop doesn't fit it at all?
>>
>>
>>
>> Just for information, I'm getting the epipe errors / max hash entries / web
>> blank pages on a intel xeon server X3430+ with 1GB RAM running ntop only.
>>
>>
>>
>> I'm using ntop 3.3 based on ubuntu.
>>
>>
>>
>> Thanks very much.
>>
>>
>>
>> Tiago
>>
>> ________________________________
>>
>> _______________________________________________
>> Ntop mailing list
>> Ntop [at] listgateway
>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>
>> ________________________________
>>
>> _______________________________________________
>> Ntop mailing list
>> Ntop [at] listgateway
>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>
>> ________________________________
>>
>> _______________________________________________
>> Ntop mailing list
>> Ntop [at] listgateway
>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>
>>
>> _______________________________________________
>> Ntop mailing list
>> Ntop [at] listgateway
>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>
>
>
>
> --
> R.

--
R.
_______________________________________________
Ntop mailing list
Ntop [at] listgateway
http://listgateway.unipi.it/mailman/listinfo/ntop

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads