Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: NTop: Users

Fwd: Introducing netmap: line-rate packet send/receive at 10Gbit/s

 

 

NTop users RSS feed   Index | Next | Previous | View Threaded


kurt.buff at gmail

Jun 2, 2011, 4:30 PM

Post #1 of 6 (1062 views)
Permalink
Fwd: Introducing netmap: line-rate packet send/receive at 10Gbit/s

Is FreeBSD catching up with Linux? Can ntop on FreeBSD be tweaked to
use this facility?

Not that *my* infrastructure needs it, but inquiring minds want to know...

Kurt

---------- Forwarded message ----------
From: Luigi Rizzo <rizzo [at] iet>
Date: Thu, Jun 2, 2011 at 15:31
Subject: Introducing netmap: line-rate packet send/receive at 10Gbit/s
To: net [at] freebsd
Cc: current [at] freebsd


Hi,
we have recently worked on a project, called netmap, which lets
FreeBSD send/receive packets at line rate even at 10 Gbit/s with
very low CPU overhead: one core at 1.33 GHz does 14.88 Mpps with a
modified ixgbe driver, which gives plenty of CPU cycles to handle
multiple interface and/or do useful work (packet forwarding, analysis, etc.)

You can find full documentation and source code and even a picobsd image at

       http://info.iet.unipi.it/~luigi/netmap/

The system uses memory mapped packet buffers to reduce the cost of
data movements, but this would not be enough to make it useful or
novel.  Netmap uses many other small but important tricks to make
the system fast, safe and easy to use, and support transmission,
reception, and communication with the host stack.

You can see full details in  documentation at the above link.

Feedback welcome.

       cheers
       luigi
-----------------------------------------+-------------------------------
 Prof. Luigi RIZZO, rizzo [at] iet  . Dip. di Ing. dell'Informazione
 http://www.iet.unipi.it/~luigi/        . Universita` di Pisa
 TEL      +39-050-2211611               . via Diotisalvi 2
 Mobile   +39-338-6809875               . 56122 PISA (Italy)
-----------------------------------------+-------------------------------
_______________________________________________
Ntop mailing list
Ntop [at] listgateway
http://listgateway.unipi.it/mailman/listinfo/ntop


deri at ntop

Jun 3, 2011, 11:03 PM

Post #2 of 6 (1014 views)
Permalink
Re: Fwd: Introducing netmap: line-rate packet send/receive at 10Gbit/s [In reply to]

Kurt
if you want an answer to your question, you can probably read http://luca.ntop.org/10g.pdf

Luca

On Jun 3, 2011, at 1:30 AM, Kurt Buff wrote:

> Is FreeBSD catching up with Linux? Can ntop on FreeBSD be tweaked to
> use this facility?
>
> Not that *my* infrastructure needs it, but inquiring minds want to know...
>
> Kurt
>
> ---------- Forwarded message ----------
> From: Luigi Rizzo <rizzo [at] iet>
> Date: Thu, Jun 2, 2011 at 15:31
> Subject: Introducing netmap: line-rate packet send/receive at 10Gbit/s
> To: net [at] freebsd
> Cc: current [at] freebsd
>
>
> Hi,
> we have recently worked on a project, called netmap, which lets
> FreeBSD send/receive packets at line rate even at 10 Gbit/s with
> very low CPU overhead: one core at 1.33 GHz does 14.88 Mpps with a
> modified ixgbe driver, which gives plenty of CPU cycles to handle
> multiple interface and/or do useful work (packet forwarding, analysis, etc.)
>
> You can find full documentation and source code and even a picobsd image at
>
> http://info.iet.unipi.it/~luigi/netmap/
>
> The system uses memory mapped packet buffers to reduce the cost of
> data movements, but this would not be enough to make it useful or
> novel. Netmap uses many other small but important tricks to make
> the system fast, safe and easy to use, and support transmission,
> reception, and communication with the host stack.
>
> You can see full details in documentation at the above link.
>
> Feedback welcome.
>
> cheers
> luigi
> -----------------------------------------+-------------------------------
> Prof. Luigi RIZZO, rizzo [at] iet . Dip. di Ing. dell'Informazione
> http://www.iet.unipi.it/~luigi/ . Universita` di Pisa
> TEL +39-050-2211611 . via Diotisalvi 2
> Mobile +39-338-6809875 . 56122 PISA (Italy)
> -----------------------------------------+-------------------------------
> _______________________________________________
> Ntop mailing list
> Ntop [at] listgateway
> http://listgateway.unipi.it/mailman/listinfo/ntop

---
Bildung ist kein Verbrechen




_______________________________________________
Ntop mailing list
Ntop [at] listgateway
http://listgateway.unipi.it/mailman/listinfo/ntop


kurt.buff at gmail

Jun 4, 2011, 10:40 AM

Post #3 of 6 (1008 views)
Permalink
Re: Fwd: Introducing netmap: line-rate packet send/receive at 10Gbit/s [In reply to]

Thank you for that. I'm reading it now.

Kurt

On Fri, Jun 3, 2011 at 23:03, Luca Deri <deri [at] ntop> wrote:
> Kurt
> if you want an answer to your question, you can probably read http://luca.ntop.org/10g.pdf
>
> Luca
>
> On Jun 3, 2011, at 1:30 AM, Kurt Buff wrote:
>
>> Is FreeBSD catching up with Linux? Can ntop on FreeBSD be tweaked to
>> use this facility?
>>
>> Not that *my* infrastructure needs it, but inquiring minds want to know...
>>
>> Kurt
>>
>> ---------- Forwarded message ----------
>> From: Luigi Rizzo <rizzo [at] iet>
>> Date: Thu, Jun 2, 2011 at 15:31
>> Subject: Introducing netmap: line-rate packet send/receive at 10Gbit/s
>> To: net [at] freebsd
>> Cc: current [at] freebsd
>>
>>
>> Hi,
>> we have recently worked on a project, called netmap, which lets
>> FreeBSD send/receive packets at line rate even at 10 Gbit/s with
>> very low CPU overhead: one core at 1.33 GHz does 14.88 Mpps with a
>> modified ixgbe driver, which gives plenty of CPU cycles to handle
>> multiple interface and/or do useful work (packet forwarding, analysis, etc.)
>>
>> You can find full documentation and source code and even a picobsd image at
>>
>>        http://info.iet.unipi.it/~luigi/netmap/
>>
>> The system uses memory mapped packet buffers to reduce the cost of
>> data movements, but this would not be enough to make it useful or
>> novel.  Netmap uses many other small but important tricks to make
>> the system fast, safe and easy to use, and support transmission,
>> reception, and communication with the host stack.
>>
>> You can see full details in  documentation at the above link.
>>
>> Feedback welcome.
>>
>>        cheers
>>        luigi
>> -----------------------------------------+-------------------------------
>>  Prof. Luigi RIZZO, rizzo [at] iet  . Dip. di Ing. dell'Informazione
>>  http://www.iet.unipi.it/~luigi/        . Universita` di Pisa
>>  TEL      +39-050-2211611               . via Diotisalvi 2
>>  Mobile   +39-338-6809875               . 56122 PISA (Italy)
>> -----------------------------------------+-------------------------------
>> _______________________________________________
>> Ntop mailing list
>> Ntop [at] listgateway
>> http://listgateway.unipi.it/mailman/listinfo/ntop
>
> ---
> Bildung ist kein Verbrechen
>
>
>
>
> _______________________________________________
> Ntop mailing list
> Ntop [at] listgateway
> http://listgateway.unipi.it/mailman/listinfo/ntop
>
_______________________________________________
Ntop mailing list
Ntop [at] listgateway
http://listgateway.unipi.it/mailman/listinfo/ntop


deri at ntop

Jun 4, 2011, 2:48 PM

Post #4 of 6 (1006 views)
Permalink
Re: Fwd: Introducing netmap: line-rate packet send/receive at 10Gbit/s [In reply to]

Kurt
I jave forgot to say that ntop (like snort, wireshark and many other apps) are CPU rather than network bound. THis menas that even if you use a FPGA-based card such as Napatech/Endace you won't improve much the speed overall speed as the CPU is the bottleneck. If you want to speed-up ntop (as of today) you need to use nprobe as pre-processor

Cheers Luca

On Jun 4, 2011, at 7:40 PM, Kurt Buff wrote:

> Thank you for that. I'm reading it now.
>
> Kurt
>
> On Fri, Jun 3, 2011 at 23:03, Luca Deri <deri [at] ntop> wrote:
>> Kurt
>> if you want an answer to your question, you can probably read http://luca.ntop.org/10g.pdf
>>
>> Luca
>>
>> On Jun 3, 2011, at 1:30 AM, Kurt Buff wrote:
>>
>>> Is FreeBSD catching up with Linux? Can ntop on FreeBSD be tweaked to
>>> use this facility?
>>>
>>> Not that *my* infrastructure needs it, but inquiring minds want to know...
>>>
>>> Kurt
>>>
>>> ---------- Forwarded message ----------
>>> From: Luigi Rizzo <rizzo [at] iet>
>>> Date: Thu, Jun 2, 2011 at 15:31
>>> Subject: Introducing netmap: line-rate packet send/receive at 10Gbit/s
>>> To: net [at] freebsd
>>> Cc: current [at] freebsd
>>>
>>>
>>> Hi,
>>> we have recently worked on a project, called netmap, which lets
>>> FreeBSD send/receive packets at line rate even at 10 Gbit/s with
>>> very low CPU overhead: one core at 1.33 GHz does 14.88 Mpps with a
>>> modified ixgbe driver, which gives plenty of CPU cycles to handle
>>> multiple interface and/or do useful work (packet forwarding, analysis, etc.)
>>>
>>> You can find full documentation and source code and even a picobsd image at
>>>
>>> http://info.iet.unipi.it/~luigi/netmap/
>>>
>>> The system uses memory mapped packet buffers to reduce the cost of
>>> data movements, but this would not be enough to make it useful or
>>> novel. Netmap uses many other small but important tricks to make
>>> the system fast, safe and easy to use, and support transmission,
>>> reception, and communication with the host stack.
>>>
>>> You can see full details in documentation at the above link.
>>>
>>> Feedback welcome.
>>>
>>> cheers
>>> luigi
>>> -----------------------------------------+-------------------------------
>>> Prof. Luigi RIZZO, rizzo [at] iet . Dip. di Ing. dell'Informazione
>>> http://www.iet.unipi.it/~luigi/ . Universita` di Pisa
>>> TEL +39-050-2211611 . via Diotisalvi 2
>>> Mobile +39-338-6809875 . 56122 PISA (Italy)
>>> -----------------------------------------+-------------------------------
>>> _______________________________________________
>>> Ntop mailing list
>>> Ntop [at] listgateway
>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>
>> ---
>> Bildung ist kein Verbrechen
>>
>>
>>
>>
>> _______________________________________________
>> Ntop mailing list
>> Ntop [at] listgateway
>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>
> _______________________________________________
> Ntop mailing list
> Ntop [at] listgateway
> http://listgateway.unipi.it/mailman/listinfo/ntop

---
Due to lack of interest, tomorrow is cancelled - Kaiser Chiefs


_______________________________________________
Ntop mailing list
Ntop [at] listgateway
http://listgateway.unipi.it/mailman/listinfo/ntop


kurt.buff at gmail

Jun 4, 2011, 4:32 PM

Post #5 of 6 (1008 views)
Permalink
Re: Fwd: Introducing netmap: line-rate packet send/receive at 10Gbit/s [In reply to]

Understood. I believe that memory also becomes an issue under certain
conditions as well, especially if using things like sticky hosts
and/or name resolution.

Kurt

On Sat, Jun 4, 2011 at 14:48, Luca Deri <deri [at] ntop> wrote:
> Kurt
> I jave forgot to say that ntop (like snort, wireshark and many other apps) are CPU rather than network bound. THis menas that even if you use a FPGA-based card such as Napatech/Endace you won't improve much the speed overall speed as the CPU is the bottleneck. If you want to speed-up ntop (as of today) you need to use nprobe as pre-processor
>
> Cheers Luca
>
> On Jun 4, 2011, at 7:40 PM, Kurt Buff wrote:
>
>> Thank you for that. I'm reading it now.
>>
>> Kurt
>>
>> On Fri, Jun 3, 2011 at 23:03, Luca Deri <deri [at] ntop> wrote:
>>> Kurt
>>> if you want an answer to your question, you can probably read http://luca.ntop.org/10g.pdf
>>>
>>> Luca
>>>
>>> On Jun 3, 2011, at 1:30 AM, Kurt Buff wrote:
>>>
>>>> Is FreeBSD catching up with Linux? Can ntop on FreeBSD be tweaked to
>>>> use this facility?
>>>>
>>>> Not that *my* infrastructure needs it, but inquiring minds want to know...
>>>>
>>>> Kurt
>>>>
>>>> ---------- Forwarded message ----------
>>>> From: Luigi Rizzo <rizzo [at] iet>
>>>> Date: Thu, Jun 2, 2011 at 15:31
>>>> Subject: Introducing netmap: line-rate packet send/receive at 10Gbit/s
>>>> To: net [at] freebsd
>>>> Cc: current [at] freebsd
>>>>
>>>>
>>>> Hi,
>>>> we have recently worked on a project, called netmap, which lets
>>>> FreeBSD send/receive packets at line rate even at 10 Gbit/s with
>>>> very low CPU overhead: one core at 1.33 GHz does 14.88 Mpps with a
>>>> modified ixgbe driver, which gives plenty of CPU cycles to handle
>>>> multiple interface and/or do useful work (packet forwarding, analysis, etc.)
>>>>
>>>> You can find full documentation and source code and even a picobsd image at
>>>>
>>>>        http://info.iet.unipi.it/~luigi/netmap/
>>>>
>>>> The system uses memory mapped packet buffers to reduce the cost of
>>>> data movements, but this would not be enough to make it useful or
>>>> novel.  Netmap uses many other small but important tricks to make
>>>> the system fast, safe and easy to use, and support transmission,
>>>> reception, and communication with the host stack.
>>>>
>>>> You can see full details in  documentation at the above link.
>>>>
>>>> Feedback welcome.
>>>>
>>>>        cheers
>>>>        luigi
>>>> -----------------------------------------+-------------------------------
>>>>  Prof. Luigi RIZZO, rizzo [at] iet  . Dip. di Ing. dell'Informazione
>>>>  http://www.iet.unipi.it/~luigi/        . Universita` di Pisa
>>>>  TEL      +39-050-2211611               . via Diotisalvi 2
>>>>  Mobile   +39-338-6809875               . 56122 PISA (Italy)
>>>> -----------------------------------------+-------------------------------
>>>> _______________________________________________
>>>> Ntop mailing list
>>>> Ntop [at] listgateway
>>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>>
>>> ---
>>> Bildung ist kein Verbrechen
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> Ntop mailing list
>>> Ntop [at] listgateway
>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>>
>> _______________________________________________
>> Ntop mailing list
>> Ntop [at] listgateway
>> http://listgateway.unipi.it/mailman/listinfo/ntop
>
> ---
> Due to lack of interest, tomorrow is cancelled - Kaiser Chiefs
>
>
> _______________________________________________
> Ntop mailing list
> Ntop [at] listgateway
> http://listgateway.unipi.it/mailman/listinfo/ntop
>
_______________________________________________
Ntop mailing list
Ntop [at] listgateway
http://listgateway.unipi.it/mailman/listinfo/ntop


deri at ntop

Jun 5, 2011, 1:36 AM

Post #6 of 6 (1004 views)
Permalink
Re: Fwd: Introducing netmap: line-rate packet send/receive at 10Gbit/s [In reply to]

Kurt
this is also true. Nevertheless the main problem of ntop is that for each packet too many counters are incremented. THis causes the issue. I now want to release the new PF_RING, nProbe is already in good shape, and this summer to focus on ntop again to see if we can address these limitations

Luca


On Jun 5, 2011, at 1:32 AM, Kurt Buff wrote:

> Understood. I believe that memory also becomes an issue under certain
> conditions as well, especially if using things like sticky hosts
> and/or name resolution.
>
> Kurt
>
> On Sat, Jun 4, 2011 at 14:48, Luca Deri <deri [at] ntop> wrote:
>> Kurt
>> I jave forgot to say that ntop (like snort, wireshark and many other apps) are CPU rather than network bound. THis menas that even if you use a FPGA-based card such as Napatech/Endace you won't improve much the speed overall speed as the CPU is the bottleneck. If you want to speed-up ntop (as of today) you need to use nprobe as pre-processor
>>
>> Cheers Luca
>>
>> On Jun 4, 2011, at 7:40 PM, Kurt Buff wrote:
>>
>>> Thank you for that. I'm reading it now.
>>>
>>> Kurt
>>>
>>> On Fri, Jun 3, 2011 at 23:03, Luca Deri <deri [at] ntop> wrote:
>>>> Kurt
>>>> if you want an answer to your question, you can probably read http://luca.ntop.org/10g.pdf
>>>>
>>>> Luca
>>>>
>>>> On Jun 3, 2011, at 1:30 AM, Kurt Buff wrote:
>>>>
>>>>> Is FreeBSD catching up with Linux? Can ntop on FreeBSD be tweaked to
>>>>> use this facility?
>>>>>
>>>>> Not that *my* infrastructure needs it, but inquiring minds want to know...
>>>>>
>>>>> Kurt
>>>>>
>>>>> ---------- Forwarded message ----------
>>>>> From: Luigi Rizzo <rizzo [at] iet>
>>>>> Date: Thu, Jun 2, 2011 at 15:31
>>>>> Subject: Introducing netmap: line-rate packet send/receive at 10Gbit/s
>>>>> To: net [at] freebsd
>>>>> Cc: current [at] freebsd
>>>>>
>>>>>
>>>>> Hi,
>>>>> we have recently worked on a project, called netmap, which lets
>>>>> FreeBSD send/receive packets at line rate even at 10 Gbit/s with
>>>>> very low CPU overhead: one core at 1.33 GHz does 14.88 Mpps with a
>>>>> modified ixgbe driver, which gives plenty of CPU cycles to handle
>>>>> multiple interface and/or do useful work (packet forwarding, analysis, etc.)
>>>>>
>>>>> You can find full documentation and source code and even a picobsd image at
>>>>>
>>>>> http://info.iet.unipi.it/~luigi/netmap/
>>>>>
>>>>> The system uses memory mapped packet buffers to reduce the cost of
>>>>> data movements, but this would not be enough to make it useful or
>>>>> novel. Netmap uses many other small but important tricks to make
>>>>> the system fast, safe and easy to use, and support transmission,
>>>>> reception, and communication with the host stack.
>>>>>
>>>>> You can see full details in documentation at the above link.
>>>>>
>>>>> Feedback welcome.
>>>>>
>>>>> cheers
>>>>> luigi
>>>>> -----------------------------------------+-------------------------------
>>>>> Prof. Luigi RIZZO, rizzo [at] iet . Dip. di Ing. dell'Informazione
>>>>> http://www.iet.unipi.it/~luigi/ . Universita` di Pisa
>>>>> TEL +39-050-2211611 . via Diotisalvi 2
>>>>> Mobile +39-338-6809875 . 56122 PISA (Italy)
>>>>> -----------------------------------------+-------------------------------
>>>>> _______________________________________________
>>>>> Ntop mailing list
>>>>> Ntop [at] listgateway
>>>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>>>
>>>> ---
>>>> Bildung ist kein Verbrechen
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Ntop mailing list
>>>> Ntop [at] listgateway
>>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>>>
>>> _______________________________________________
>>> Ntop mailing list
>>> Ntop [at] listgateway
>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>
>> ---
>> Due to lack of interest, tomorrow is cancelled - Kaiser Chiefs
>>
>>
>> _______________________________________________
>> Ntop mailing list
>> Ntop [at] listgateway
>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>
> _______________________________________________
> Ntop mailing list
> Ntop [at] listgateway
> http://listgateway.unipi.it/mailman/listinfo/ntop

---
Due to lack of interest, tomorrow is cancelled - Kaiser Chiefs


_______________________________________________
Ntop mailing list
Ntop [at] listgateway
http://listgateway.unipi.it/mailman/listinfo/ntop

NTop users RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.