Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: NTop: Misc

ixgbe RSS hashing

 

 

NTop misc RSS feed   Index | Next | Previous | View Threaded


c.d.wakelin at reading

May 28, 2012, 10:23 AM

Post #1 of 4 (965 views)
Permalink
ixgbe RSS hashing

I've been looking at the issue of multiqueue RSS hashing in ixgbe (and
presumably igb), which is one of the things libzero is supposed to help
with.

The problem is that (at least in some cases) traffic from X->Y may end
up in a different queue, and therefore different CPU core, to that from
Y->X.

I found some patches that are supposed to help

http://www.mail-archive.com/e1000-devel [at] lists/msg04364.html

and

http://kerneltrap.org/mailarchive/linux-netdev/2010/12/18/6292110

So it appears that the hashing algorithm can be modified in the driver!

In the ensuing discussion on kerneltrap.org

http://kerneltrap.org/mailarchive/linux-netdev/2011/1/3/6292774

the author says that when flow-director is enabled, the patch isn't needed.

So if I turn on "ntuple" with "ethtool -K eth1 ntuple: on", with no
filtering rules, would that mean the queues now match?

My testing has been pretty inconclusive. Certainly, even in the default
modes, I get *some* traffic where X->Y and Y->X are in the same queue,
but of course you'd probably expect that to happen in 1/N cases for N
queues.

Best Wishes,
Chris

--
--+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+-
Christopher Wakelin, c.d.wakelin [at] reading
IT Services Centre, The University of Reading, Tel: +44 (0)118 378 2908
Whiteknights, Reading, RG6 6AF, UK Fax: +44 (0)118 975 3094
_______________________________________________
Ntop-misc mailing list
Ntop-misc [at] listgateway
http://listgateway.unipi.it/mailman/listinfo/ntop-misc


cardigliano at ntop

May 28, 2012, 3:35 PM

Post #2 of 4 (931 views)
Permalink
Re: ixgbe RSS hashing [In reply to]

Chris
thank you for the pointers!
Some preliminary tests reprogramming rss key and redirection table are promising.

Regards
Alfredo

On May 28, 2012, at 7:23 PM, Chris Wakelin wrote:

> I've been looking at the issue of multiqueue RSS hashing in ixgbe (and
> presumably igb), which is one of the things libzero is supposed to help
> with.
>
> The problem is that (at least in some cases) traffic from X->Y may end
> up in a different queue, and therefore different CPU core, to that from
> Y->X.
>
> I found some patches that are supposed to help
>
> http://www.mail-archive.com/e1000-devel [at] lists/msg04364.html
>
> and
>
> http://kerneltrap.org/mailarchive/linux-netdev/2010/12/18/6292110
>
> So it appears that the hashing algorithm can be modified in the driver!
>
> In the ensuing discussion on kerneltrap.org
>
> http://kerneltrap.org/mailarchive/linux-netdev/2011/1/3/6292774
>
> the author says that when flow-director is enabled, the patch isn't needed.
>
> So if I turn on "ntuple" with "ethtool -K eth1 ntuple: on", with no
> filtering rules, would that mean the queues now match?
>
> My testing has been pretty inconclusive. Certainly, even in the default
> modes, I get *some* traffic where X->Y and Y->X are in the same queue,
> but of course you'd probably expect that to happen in 1/N cases for N
> queues.
>
> Best Wishes,
> Chris
>
> --
> --+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+-
> Christopher Wakelin, c.d.wakelin [at] reading
> IT Services Centre, The University of Reading, Tel: +44 (0)118 378 2908
> Whiteknights, Reading, RG6 6AF, UK Fax: +44 (0)118 975 3094
> _______________________________________________
> Ntop-misc mailing list
> Ntop-misc [at] listgateway
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc

_______________________________________________
Ntop-misc mailing list
Ntop-misc [at] listgateway
http://listgateway.unipi.it/mailman/listinfo/ntop-misc


c.d.wakelin at reading

May 30, 2012, 5:49 AM

Post #3 of 4 (928 views)
Permalink
Re: ixgbe RSS hashing [In reply to]

I noticed you modified the rss ixgbe DNA driver yesterday, so I gave it
a try :-)

It seemed to do the right thing with the flows, though my CPU usage was
near 99% on some threads.

You then reversed the changes. Did you find it caused problems, or is it
just too experimental?

Best Wishes,
Chris

On 28/05/12 23:35, Alfredo Cardigliano wrote:
> Chris
> thank you for the pointers!
> Some preliminary tests reprogramming rss key and redirection table are promising.
>
> Regards
> Alfredo
>
> On May 28, 2012, at 7:23 PM, Chris Wakelin wrote:
>
>> I've been looking at the issue of multiqueue RSS hashing in ixgbe (and
>> presumably igb), which is one of the things libzero is supposed to help
>> with.
>>
>> The problem is that (at least in some cases) traffic from X->Y may end
>> up in a different queue, and therefore different CPU core, to that from
>> Y->X.
>>
>> I found some patches that are supposed to help
>>
>> http://www.mail-archive.com/e1000-devel [at] lists/msg04364.html
>>
>> and
>>
>> http://kerneltrap.org/mailarchive/linux-netdev/2010/12/18/6292110
>>
>> So it appears that the hashing algorithm can be modified in the driver!
>>
>> In the ensuing discussion on kerneltrap.org
>>
>> http://kerneltrap.org/mailarchive/linux-netdev/2011/1/3/6292774
>>
>> the author says that when flow-director is enabled, the patch isn't needed.
>>
>> So if I turn on "ntuple" with "ethtool -K eth1 ntuple: on", with no
>> filtering rules, would that mean the queues now match?
>>
>> My testing has been pretty inconclusive. Certainly, even in the default
>> modes, I get *some* traffic where X->Y and Y->X are in the same queue,
>> but of course you'd probably expect that to happen in 1/N cases for N
>> queues.
>>
>> Best Wishes,
>> Chris
>>
>> --
>> --+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+-
>> Christopher Wakelin, c.d.wakelin [at] reading
>> IT Services Centre, The University of Reading, Tel: +44 (0)118 378 2908
>> Whiteknights, Reading, RG6 6AF, UK Fax: +44 (0)118 975 3094
>> _______________________________________________
>> Ntop-misc mailing list
>> Ntop-misc [at] listgateway
>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>
> _______________________________________________
> Ntop-misc mailing list
> Ntop-misc [at] listgateway
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc


--
--+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+-
Christopher Wakelin, c.d.wakelin [at] reading
IT Services Centre, The University of Reading, Tel: +44 (0)118 378 2908
Whiteknights, Reading, RG6 6AF, UK Fax: +44 (0)118 975 3094
_______________________________________________
Ntop-misc mailing list
Ntop-misc [at] listgateway
http://listgateway.unipi.it/mailman/listinfo/ntop-misc


cardigliano at ntop

May 30, 2012, 6:49 AM

Post #4 of 4 (937 views)
Permalink
Re: ixgbe RSS hashing [In reply to]

Chris
we are still running some experiments, as the change was not satisfactory with all packet types so we reverted it.
Stay tuned.

Alfredo

On May 30, 2012, at 2:49 PM, Chris Wakelin wrote:

> I noticed you modified the rss ixgbe DNA driver yesterday, so I gave it
> a try :-)
>
> It seemed to do the right thing with the flows, though my CPU usage was
> near 99% on some threads.
>
> You then reversed the changes. Did you find it caused problems, or is it
> just too experimental?
>
> Best Wishes,
> Chris
>
> On 28/05/12 23:35, Alfredo Cardigliano wrote:
>> Chris
>> thank you for the pointers!
>> Some preliminary tests reprogramming rss key and redirection table are promising.
>>
>> Regards
>> Alfredo
>>
>> On May 28, 2012, at 7:23 PM, Chris Wakelin wrote:
>>
>>> I've been looking at the issue of multiqueue RSS hashing in ixgbe (and
>>> presumably igb), which is one of the things libzero is supposed to help
>>> with.
>>>
>>> The problem is that (at least in some cases) traffic from X->Y may end
>>> up in a different queue, and therefore different CPU core, to that from
>>> Y->X.
>>>
>>> I found some patches that are supposed to help
>>>
>>> http://www.mail-archive.com/e1000-devel [at] lists/msg04364.html
>>>
>>> and
>>>
>>> http://kerneltrap.org/mailarchive/linux-netdev/2010/12/18/6292110
>>>
>>> So it appears that the hashing algorithm can be modified in the driver!
>>>
>>> In the ensuing discussion on kerneltrap.org
>>>
>>> http://kerneltrap.org/mailarchive/linux-netdev/2011/1/3/6292774
>>>
>>> the author says that when flow-director is enabled, the patch isn't needed.
>>>
>>> So if I turn on "ntuple" with "ethtool -K eth1 ntuple: on", with no
>>> filtering rules, would that mean the queues now match?
>>>
>>> My testing has been pretty inconclusive. Certainly, even in the default
>>> modes, I get *some* traffic where X->Y and Y->X are in the same queue,
>>> but of course you'd probably expect that to happen in 1/N cases for N
>>> queues.
>>>
>>> Best Wishes,
>>> Chris
>>>
>>> --
>>> --+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+-
>>> Christopher Wakelin, c.d.wakelin [at] reading
>>> IT Services Centre, The University of Reading, Tel: +44 (0)118 378 2908
>>> Whiteknights, Reading, RG6 6AF, UK Fax: +44 (0)118 975 3094
>>> _______________________________________________
>>> Ntop-misc mailing list
>>> Ntop-misc [at] listgateway
>>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>>
>> _______________________________________________
>> Ntop-misc mailing list
>> Ntop-misc [at] listgateway
>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>
>
> --
> --+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+-
> Christopher Wakelin, c.d.wakelin [at] reading
> IT Services Centre, The University of Reading, Tel: +44 (0)118 378 2908
> Whiteknights, Reading, RG6 6AF, UK Fax: +44 (0)118 975 3094
> _______________________________________________
> Ntop-misc mailing list
> Ntop-misc [at] listgateway
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc

_______________________________________________
Ntop-misc mailing list
Ntop-misc [at] listgateway
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

NTop misc RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.