Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Linux Virtual Server: Users

[lvs-users] LVS ixgbe rx_queue

 

 

Linux Virtual Server users RSS feed   Index | Next | Previous | View Threaded


aleksey at bb

Nov 21, 2011, 12:26 AM

Post #1 of 2 (247 views)
Permalink
[lvs-users] LVS ixgbe rx_queue

Hello!



I have LVS system with two Six-Core CPUs and 2x 10G Intel NICs. Intel ixgbe
driver creates 24 tx/rx queues.

While testing with iperf tool traffic was distributing among 24 RX queues.

But with LVS traffic distributed only among 16 RX queues.



How to direct traffic to all 24 rx queues?



HW configuration:

HP ProLiant DL360 G7, 2x Intel Six-Core E5645, 16GB RAM, 2x Intel
Corporation 82599EB 10-Gigabit NICs

Linux Kernel 2.6.38.8



Some ethtool output



# ethtool -i eth0

driver: ixgbe

version: 3.2.9-k2

firmware-version: 0.9-3

bus-info: 0000:09:00.0



# ethtool -k eth0

Offload parameters for eth0:

rx-checksumming: on

tx-checksumming: on

scatter-gather: on

tcp-segmentation-offload: on

udp-fragmentation-offload: off

generic-segmentation-offload: on

generic-receive-offload: off

large-receive-offload: off

rx-vlan-offload: on

tx-vlan-offload: on

ntuple-filters: off

receive-hashing: off



# ethtool -S eth0

NIC statistics:

rx_packets: 2353149

tx_packets: 2710579

rx_bytes: 651217098

tx_bytes: 659738990

rx_pkts_nic: 2353149

tx_pkts_nic: 2710579

rx_bytes_nic: 660629694

tx_bytes_nic: 670597164

lsc_int: 2

tx_busy: 0

non_eop_descs: 0

rx_errors: 0

tx_errors: 0

rx_dropped: 0

tx_dropped: 0

multicast: 99

broadcast: 1281

rx_no_buffer_count: 0

collisions: 0

rx_over_errors: 0

rx_crc_errors: 0

rx_frame_errors: 0

hw_rsc_aggregated: 0

hw_rsc_flushed: 0

fdir_match: 291115

fdir_miss: 2031557

rx_fifo_errors: 0

rx_missed_errors: 0

tx_aborted_errors: 0

tx_carrier_errors: 0

tx_fifo_errors: 0

tx_heartbeat_errors: 0

tx_timeout_count: 0

tx_restart_queue: 20

rx_long_length_errors: 0

rx_short_length_errors: 0

tx_flow_control_xon: 0

rx_flow_control_xon: 0

tx_flow_control_xoff: 0

rx_flow_control_xoff: 0

rx_csum_offload_errors: 49

alloc_rx_page_failed: 0

alloc_rx_buff_failed: 0

rx_no_dma_resources: 0

tx_queue_0_packets: 165043

tx_queue_0_bytes: 38401840

tx_queue_1_packets: 162906

tx_queue_1_bytes: 38138855

tx_queue_2_packets: 168570

tx_queue_2_bytes: 41790265

tx_queue_3_packets: 167731

tx_queue_3_bytes: 40535275

tx_queue_4_packets: 168591

tx_queue_4_bytes: 42697032

tx_queue_5_packets: 170165

tx_queue_5_bytes: 43270191

tx_queue_6_packets: 173063

tx_queue_6_bytes: 40746228

tx_queue_7_packets: 171329

tx_queue_7_bytes: 42974539

tx_queue_8_packets: 171401

tx_queue_8_bytes: 40370468

tx_queue_9_packets: 174085

tx_queue_9_bytes: 43069158

tx_queue_10_packets: 165402

tx_queue_10_bytes: 40150477

tx_queue_11_packets: 165939

tx_queue_11_bytes: 38720432

tx_queue_12_packets: 174055

tx_queue_12_bytes: 42915187

tx_queue_13_packets: 169259

tx_queue_13_bytes: 40416664

tx_queue_14_packets: 167273

tx_queue_14_bytes: 39214682

tx_queue_15_packets: 175688

tx_queue_15_bytes: 46319151

tx_queue_16_packets: 2

tx_queue_16_bytes: 248

tx_queue_17_packets: 28

tx_queue_17_bytes: 2222

tx_queue_18_packets: 5

tx_queue_18_bytes: 620

tx_queue_19_packets: 5

tx_queue_19_bytes: 620

tx_queue_20_packets: 5

tx_queue_20_bytes: 620

tx_queue_21_packets: 1

tx_queue_21_bytes: 124

tx_queue_22_packets: 0

tx_queue_22_bytes: 0

tx_queue_23_packets: 33

tx_queue_23_bytes: 4092

rx_queue_0_packets: 143415

rx_queue_0_bytes: 37851657

rx_queue_1_packets: 139924

rx_queue_1_bytes: 39637036

rx_queue_2_packets: 147665

rx_queue_2_bytes: 41246049

rx_queue_3_packets: 147788

rx_queue_3_bytes: 40565392

rx_queue_4_packets: 146184

rx_queue_4_bytes: 41169068

rx_queue_5_packets: 149493

rx_queue_5_bytes: 42221674

rx_queue_6_packets: 143145

rx_queue_6_bytes: 37153935

rx_queue_7_packets: 148494

rx_queue_7_bytes: 40966091

rx_queue_8_packets: 148678

rx_queue_8_bytes: 39129607

rx_queue_9_packets: 146972

rx_queue_9_bytes: 39589930

rx_queue_10_packets: 146489

rx_queue_10_bytes: 42729748

rx_queue_11_packets: 142334

rx_queue_11_bytes: 37601609

rx_queue_12_packets: 150304

rx_queue_12_bytes: 42118352

rx_queue_13_packets: 153071

rx_queue_13_bytes: 43629946

rx_queue_14_packets: 142691

rx_queue_14_bytes: 37838601

rx_queue_15_packets: 156491

rx_queue_15_bytes: 47767637

rx_queue_16_packets: 0

rx_queue_16_bytes: 0

rx_queue_17_packets: 11

rx_queue_17_bytes: 766

rx_queue_18_packets: 0

rx_queue_18_bytes: 0

rx_queue_19_packets: 0

rx_queue_19_bytes: 0

rx_queue_20_packets: 0

rx_queue_20_bytes: 0

rx_queue_21_packets: 0

rx_queue_21_bytes: 0

rx_queue_22_packets: 0

rx_queue_22_bytes: 0

rx_queue_23_packets: 0

rx_queue_23_bytes: 0







Best regards,

Aleksey



_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users


aleksey.chudov at gmail

Nov 21, 2011, 12:27 AM

Post #2 of 2 (240 views)
Permalink
[lvs-users] LVS ixgbe rx_queue [In reply to]

Hello!



I have LVS system with two Six-Core CPUs and 2x 10G Intel NICs. Intel ixgbe
driver creates 24 tx/rx queues.

While testing with iperf tool traffic was distributing among 24 RX queues.

But with LVS traffic distributed only among 16 RX queues.



How to direct traffic to all 24 rx queues?



HW configuration:

HP ProLiant DL360 G7, 2x Intel Six-Core E5645, 16GB RAM, 2x Intel
Corporation 82599EB 10-Gigabit NICs

Linux Kernel 2.6.38.8



Some ethtool output



# ethtool -i eth0

driver: ixgbe

version: 3.2.9-k2

firmware-version: 0.9-3

bus-info: 0000:09:00.0



# ethtool -k eth0

Offload parameters for eth0:

rx-checksumming: on

tx-checksumming: on

scatter-gather: on

tcp-segmentation-offload: on

udp-fragmentation-offload: off

generic-segmentation-offload: on

generic-receive-offload: off

large-receive-offload: off

rx-vlan-offload: on

tx-vlan-offload: on

ntuple-filters: off

receive-hashing: off



# ethtool -S eth0

NIC statistics:

rx_packets: 2353149

tx_packets: 2710579

rx_bytes: 651217098

tx_bytes: 659738990

rx_pkts_nic: 2353149

tx_pkts_nic: 2710579

rx_bytes_nic: 660629694

tx_bytes_nic: 670597164

lsc_int: 2

tx_busy: 0

non_eop_descs: 0

rx_errors: 0

tx_errors: 0

rx_dropped: 0

tx_dropped: 0

multicast: 99

broadcast: 1281

rx_no_buffer_count: 0

collisions: 0

rx_over_errors: 0

rx_crc_errors: 0

rx_frame_errors: 0

hw_rsc_aggregated: 0

hw_rsc_flushed: 0

fdir_match: 291115

fdir_miss: 2031557

rx_fifo_errors: 0

rx_missed_errors: 0

tx_aborted_errors: 0

tx_carrier_errors: 0

tx_fifo_errors: 0

tx_heartbeat_errors: 0

tx_timeout_count: 0

tx_restart_queue: 20

rx_long_length_errors: 0

rx_short_length_errors: 0

tx_flow_control_xon: 0

rx_flow_control_xon: 0

tx_flow_control_xoff: 0

rx_flow_control_xoff: 0

rx_csum_offload_errors: 49

alloc_rx_page_failed: 0

alloc_rx_buff_failed: 0

rx_no_dma_resources: 0

tx_queue_0_packets: 165043

tx_queue_0_bytes: 38401840

tx_queue_1_packets: 162906

tx_queue_1_bytes: 38138855

tx_queue_2_packets: 168570

tx_queue_2_bytes: 41790265

tx_queue_3_packets: 167731

tx_queue_3_bytes: 40535275

tx_queue_4_packets: 168591

tx_queue_4_bytes: 42697032

tx_queue_5_packets: 170165

tx_queue_5_bytes: 43270191

tx_queue_6_packets: 173063

tx_queue_6_bytes: 40746228

tx_queue_7_packets: 171329

tx_queue_7_bytes: 42974539

tx_queue_8_packets: 171401

tx_queue_8_bytes: 40370468

tx_queue_9_packets: 174085

tx_queue_9_bytes: 43069158

tx_queue_10_packets: 165402

tx_queue_10_bytes: 40150477

tx_queue_11_packets: 165939

tx_queue_11_bytes: 38720432

tx_queue_12_packets: 174055

tx_queue_12_bytes: 42915187

tx_queue_13_packets: 169259

tx_queue_13_bytes: 40416664

tx_queue_14_packets: 167273

tx_queue_14_bytes: 39214682

tx_queue_15_packets: 175688

tx_queue_15_bytes: 46319151

tx_queue_16_packets: 2

tx_queue_16_bytes: 248

tx_queue_17_packets: 28

tx_queue_17_bytes: 2222

tx_queue_18_packets: 5

tx_queue_18_bytes: 620

tx_queue_19_packets: 5

tx_queue_19_bytes: 620

tx_queue_20_packets: 5

tx_queue_20_bytes: 620

tx_queue_21_packets: 1

tx_queue_21_bytes: 124

tx_queue_22_packets: 0

tx_queue_22_bytes: 0

tx_queue_23_packets: 33

tx_queue_23_bytes: 4092

rx_queue_0_packets: 143415

rx_queue_0_bytes: 37851657

rx_queue_1_packets: 139924

rx_queue_1_bytes: 39637036

rx_queue_2_packets: 147665

rx_queue_2_bytes: 41246049

rx_queue_3_packets: 147788

rx_queue_3_bytes: 40565392

rx_queue_4_packets: 146184

rx_queue_4_bytes: 41169068

rx_queue_5_packets: 149493

rx_queue_5_bytes: 42221674

rx_queue_6_packets: 143145

rx_queue_6_bytes: 37153935

rx_queue_7_packets: 148494

rx_queue_7_bytes: 40966091

rx_queue_8_packets: 148678

rx_queue_8_bytes: 39129607

rx_queue_9_packets: 146972

rx_queue_9_bytes: 39589930

rx_queue_10_packets: 146489

rx_queue_10_bytes: 42729748

rx_queue_11_packets: 142334

rx_queue_11_bytes: 37601609

rx_queue_12_packets: 150304

rx_queue_12_bytes: 42118352

rx_queue_13_packets: 153071

rx_queue_13_bytes: 43629946

rx_queue_14_packets: 142691

rx_queue_14_bytes: 37838601

rx_queue_15_packets: 156491

rx_queue_15_bytes: 47767637

rx_queue_16_packets: 0

rx_queue_16_bytes: 0

rx_queue_17_packets: 11

rx_queue_17_bytes: 766

rx_queue_18_packets: 0

rx_queue_18_bytes: 0

rx_queue_19_packets: 0

rx_queue_19_bytes: 0

rx_queue_20_packets: 0

rx_queue_20_bytes: 0

rx_queue_21_packets: 0

rx_queue_21_bytes: 0

rx_queue_22_packets: 0

rx_queue_22_bytes: 0

rx_queue_23_packets: 0

rx_queue_23_bytes: 0







Best regards,

Aleksey

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

Linux Virtual Server users RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.