tmacmd at gmail
May 15, 2008, 1:15 PM
Post #5 of 6
(326 views)
Permalink
|
Notes aboutof 10GbE TOE NIC limitations : The 10GbE TOE NIC cards
have a number of limitations. They include:
$B"!(B Multimode vif limited to two (2) 10GbE TOE NICs
$B"!(B LACP not supported with 10GbE TOE NICs
$B"!(B TOE functionality disabled on 10GbE NIC in vif
Do you need a
switchport access vlan ###
in the interface lines?
Here is the config I use for vifs on my switch:
interface Port-Channel7
description netapp-vif
switchport
switchport access vlan 239
switchport mode access
no ip address
flowcontrol receive on
flowcontrol send on
spanning-tree portfast
!
interface GigabitEthernet7/42
description filer-e9
switchport
switchport access vlan 239
switchport mode access
no ip address
flowcontrol receive on
flowcontrol send on
spanning-tree portfast
channel-group 7 mode active
!
In your case, with 10-gig, the only change should be:
channel-group 7 mode on
(which you are already doing)
Also, make sure your filer is setup correctly (not using LACP for 10Gig)
vif create multi tgif -b mac e2a r3b
On Thu, May 15, 2008 at 3:55 PM, Page, Jeremy <jeremy.page[at]gilbarco.com> wrote:
> I *think* the problem may be isolated to array1. I've tried setting them
> to single mode instead of multi (LACP is not supposed to work on a
> stacked switch).
>
>
> array01> sysconfig -v 2
> slot 2: Dual TOE-10G Ethernet Controller (T320E-XFP)
> Device Type: CT-31-1
> Version Number:
> T3-SRAM1.1.0-BR1040-20-C0-FW4.6.0-DR03
> Serial Number: PT4107009
> e2a MAC Address: 00:07:43:05:16:98
> (auto-10g_sr-fd-up)
> e2b MAC Address: 00:07:43:05:16:99
> (auto-10g_sr-fd-up)
>
>
> **switch config (short form)**
> version 12.2
> no aaa new-model
> switch 1 provision ws-c3750e-24td
> switch 2 provision ws-c3750e-24td
> interface TenGigabitEthernet1/0/1
> switchport mode access
> channel-group 1 mode on
> !
> interface TenGigabitEthernet1/0/2
> switchport mode access
> channel-group 2 mode on
> interface TenGigabitEthernet2/0/1
> switchport mode access
> channel-group 1 mode on
> !
> interface TenGigabitEthernet2/0/2
> switchport mode access
> channel-group 2 mode on
> !
> interface Vlan1
> ip address 172.1.0.250 255.255.0.0
>
> -----Original Message-----
> From: tmac [mailto:tmacmd[at]gmail.com]
> Sent: Thursday, May 15, 2008 3:46 PM
> To: Page, Jeremy
> Cc: toasters[at]mathworks.com
> Subject: Re: VIF weirdness
>
> How about a "sysconfig -v 2" from the filers
> and a "show running-config " that shows at a minimum:
> the port config of the 10-gig stuff and the port-channel definitions.
>
> --tmac
>
> On Thu, May 15, 2008 at 2:19 PM, Page, Jeremy <jeremy.page[at]gilbarco.com>
> wrote:
>> They are 10 gig cards, they are only in a VIF with themselves (no
> other
>> cards). It's a 3070 with 2 copper 100/1000 2 FC gig and 2 FC 10g
>> connections. The 2 1000 are on our production network as lan0 VIF for
>> CIFS/NFS, the fibre gig is not used and the two 10g FC are VIF tgif on
> the
>> storage only network (no routing etc, private for storage devices and
> the
>> hosts using them only).
>>
>>
>>
>> They are plugged into stacked 3750e boxes, I tried putting them on the
> same
>> switch just to make sure it was not a stacking thing with no real
> change.
>>
>>
>>
>> The oddness is that occasionally a single packet will get through, and
> if I
>> bring the interfaces up and down it may change who can get to what.
>>
>>
>>
>> ________________________________
>>
>> From: Webster, Stetson [mailto:Stetson.Webster[at]netapp.com]
>> Sent: Thursday, May 15, 2008 2:13 PM
>> To: Page, Jeremy; toasters[at]mathworks.com
>> Subject: RE: VIF weirdness
>>
>>
>>
>> Are these 10g TOE cards? Are they in a VIF with other 'non-TOE'
> cards?
>>
>>
>>
>> Stetson M. Webster
>> Onsite Professional Services Engineer
>> PS - North Amer. - East
>>
>> NetApp
>> 919.250.0052 Mobile
>> Stetson.Webster[at]netapp.com
>> www.netapp.com
>>
>>
>>
>>
>>
>> ________________________________
>>
>> From: Page, Jeremy [mailto:jeremy.page[at]gilbarco.com]
>> Sent: Thursday, May 15, 2008 1:32 PM
>> To: toasters[at]mathworks.com
>> Subject: VIF weirdness
>>
>> I am having a great deal of difficulty getting my filer's 10g
> interfaces
>> connected to a pair of 3750E switches I am trying to use for my
> storage
>> network's backbone. I can inconsistently ping other objects on the
> network
>> (there really is nothing there yet, we are just getting started, but
> I do
>> have a host and an RLM card just for troubleshooting purposes), but
> they
>> cannot ping each other. >From a remote host (172.1.2.5) I can ping
>> everything but array1. If I bring down either of the two stacked
> switches
>> (which brings down one port of each VIF member pairs) everything
> works.
>>
>>
>>
>> array01> ping 172.1.0.1
>>
>> 172.1.0.1 is alive
>>
>> array01> ping 172.1.0.2
>>
>> array01> ping 172.1.0.4
>>
>> ping: wrote 172.1.0.4 64 chars, error=Host is down
>>
>> ping: wrote 172.1.0.4 64 chars, error=Host is down
>>
>> array01> Thu May 15 13:28:33 EDT [gvr-array01: nis_worker_0:info]:
> Local NIS
>> group update successful.
>>
>> array01> ping 172.1.2.5
>>
>> 172.1.2.5 is alive
>>
>>
>>
>> array02> ping 172.1.0.1
>>
>> array02> ping 172.1.0.4
>>
>> 172.1.0.4 is alive
>>
>> array02> ping 172.1.0.4
>>
>> 172.1.0.4 is alive
>>
>> array02> ping 172.1.0.250
>>
>> 172.1.0.250 is alive
>>
>> array02> ping 172.1.2.5
>>
>> no answer from 172.1.2.5
>>
>>
>>
>>
>>
>> array01> ifconfig -a
>>
>> e0a: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>>
>> ether 02:a0:98:08:22:b7 (auto-1000t-fd-up) flowcontrol full
>>
>> trunked lan0
>>
>> e0b: flags=108042<BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>>
>> ether 00:a0:98:08:22:b6 (auto-unknown-cfg_down) flowcontrol
> full
>>
>> e0c: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>>
>> ether 02:a0:98:08:22:b7 (auto-1000t-fd-up) flowcontrol full
>>
>> trunked lan0
>>
>> e0d: flags=108042<BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>>
>> ether 00:a0:98:08:22:b4 (auto-unknown-cfg_down) flowcontrol
> full
>>
>> e2a: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>>
>> ether 02:a0:98:08:22:b6 (auto-10g_sr-fd-up) flowcontrol full
>>
>> trunked tgif
>>
>> e2b: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>>
>> ether 02:a0:98:08:22:b6 (auto-10g_sr-fd-up) flowcontrol full
>>
>> trunked tgif
>>
>> lo: flags=1948049<UP,LOOPBACK,RUNNING,MULTICAST,TCPCKSUM> mtu 8160
>>
>> inet 127.0.0.1 netmask 0xff000000 broadcast 127.0.0.1
>>
>> ether 00:00:00:00:00:00 (VIA Provider)
>>
>> lan0: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>>
>> inet 10.28.17.213 netmask 0xffffff00 broadcast 10.28.17.255
>>
>> partner 10.28.17.214 (not in use)
>>
>> ether 02:a0:98:08:22:b7 (Enabled virtual interface)
>>
>> tgif: flags=4948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM,NOWINS>
> mtu 1500
>>
>> inet 172.1.0.1 netmask 0xffff0000 broadcast 172.1.255.255
>>
>> partner 172.1.0.2 (not in use)
>>
>> ether 02:a0:98:08:22:b6 (Enabled virtual interface)
>>
>> nfo enabled
>>
>> array02>ifconfig -a
>>
>> e0a: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>>
>> ether 02:a0:98:0c:1a:37 (auto-1000t-fd-up) flowcontrol full
>>
>> trunked lan0
>>
>> e0b: flags=108042<BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>>
>> ether 00:a0:98:0c:1a:36 (auto-unknown-cfg_down) flowcontrol
> full
>>
>> e0c: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>>
>> ether 02:a0:98:0c:1a:37 (auto-1000t-fd-up) flowcontrol full
>>
>> trunked lan0
>>
>> e0d: flags=108042<BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>>
>> ether 00:a0:98:0c:1a:34 (auto-unknown-cfg_down) flowcontrol
> full
>>
>> e2a: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>>
>> ether 02:a0:98:0c:1a:36 (auto-10g_sr-fd-up) flowcontrol full
>>
>> trunked tgif
>>
>> e2b: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>>
>> ether 02:a0:98:0c:1a:36 (auto-10g_sr-fd-up) flowcontrol full
>>
>> trunked tgif
>>
>> lo: flags=1948049<UP,LOOPBACK,RUNNING,MULTICAST,TCPCKSUM> mtu 8160
>>
>> inet 127.0.0.1 netmask 0xff000000 broadcast 127.0.0.1
>>
>> ether 00:00:00:00:00:00 (VIA Provider)
>>
>> lan0: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>>
>> inet 10.28.17.214 netmask 0xffffff00 broadcast 10.28.17.255
>>
>> partner 10.28.17.213 (not in use)
>>
>> ether 02:a0:98:0c:1a:37 (Enabled virtual interface)
>>
>> tgif: flags=4948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM,NOWINS>
> mtu 1500
>>
>> inet 172.1.0.2 netmask 0xffff0000 broadcast 172.1.255.255
>>
>> partner 172.1.0.1 (not in use)
>>
>> ether 02:a0:98:0c:1a:36 (Enabled virtual interface)
>>
>> nfo enabled
>>
>> This message (including any attachments) contains confidential
>> and/or proprietary information intended only for the addressee.
>> Any unauthorized disclosure, copying, distribution or reliance on
>> the contents of this information is strictly prohibited and may
>> constitute a violation of law. If you are not the intended
>> recipient, please notify the sender immediately by responding to
>> this e-mail, and delete the message from your system. If you
>> have any questions about this e-mail please notify the sender
>> immediately.
>>
>> This message (including any attachments) contains confidential
>> and/or proprietary information intended only for the addressee.
>> Any unauthorized disclosure, copying, distribution or reliance on
>> the contents of this information is strictly prohibited and may
>> constitute a violation of law. If you are not the intended
>> recipient, please notify the sender immediately by responding to
>> this e-mail, and delete the message from your system. If you
>> have any questions about this e-mail please notify the sender
>> immediately.
>>
>
>
>
> --
> --tmac
>
> RedHat Certified Engineer #804006984323821 (RHEL4)
> RedHat Certified Engineer #805007643429572 (RHEL5)
>
> Principal Consultant
>
>
> This message (including any attachments) contains confidential
> and/or proprietary information intended only for the addressee.
> Any unauthorized disclosure, copying, distribution or reliance on
> the contents of this information is strictly prohibited and may
> constitute a violation of law. If you are not the intended
> recipient, please notify the sender immediately by responding to
> this e-mail, and delete the message from your system. If you
> have any questions about this e-mail please notify the sender
> immediately.
>
--
--tmac
RedHat Certified Engineer #804006984323821 (RHEL4)
RedHat Certified Engineer #805007643429572 (RHEL5)
Principal Consultant
|
image002.jpg
