Stetson.Webster at netapp
Sep 25, 2012, 8:02 AM
Post #6 of 6
Don't forget to update your /etc/rc file as well. That's a VERY common oversight seen in Support. It sounds like this:
Hello NetApp Support,
Our cluster failed over (or the head rebooted after an upgrade, etc) and now it won't come back up. It's not accessible, only via console or RLM. The system looks fine, but we can't access it.
This can be several months (or years) later and it's not always obvious what the problem is. So just beware ;-)
Lead Storage Architect – NetApp IT
NCIE-SAN, NCIE-BR, SCSN-E, VCP
stetson [at] netapp<mailto:name [at] netapp>
[Businesses built on NetApp go further, faster.]<http://www.netapp.com/us/?REF_SOURCE=emmbonrunnersignaturehp-us>
Follow us: [Facebook] <http://www.facebook.com/NetApp?REF_SOURCE=ems-facebook> [Twitter] <http://twitter.com/#netapp?REF_SOURCE=ems-twitter> [Linked In] <http://www.linkedin.com/groups/NetApp-111681/about?REF_SOURCE=ems-linkedin> [YouTube] <http://www.youtube.com/user/NetAppTV?REF_SOURCE=ems-youtube> [Slideshare] <http://www.slideshare.net/NetApp?REF_SOURCE=ems-slideshare> [Community] <https://communities.netapp.com/welcome?REF_SOURCE=ems-cty>
Tweet us: #netapp
From: Mark Flint <mf1 [at] sanger<mailto:mf1 [at] sanger>>
Date: Tuesday, September 25, 2012 6:34 AM
To: Ray Van Dolson <rvandolson [at] esri<mailto:rvandolson [at] esri>>
Cc: "toasters [at] teaparty<mailto:toasters [at] teaparty>" <toasters [at] teaparty<mailto:toasters [at] teaparty>>
Subject: Re: e0M port as default gateway?
You'll need a 'route delete default', a 'route -f' to flush the table, then a 'route add default 10.44.X.X Z (the Z represents the metric, either 0 or 1, depending on hop count, I think in most cases it's 0, but someone more knowledgable will be along shortly to correct me :)
Principle Systems Administrator
Storage & Backups Team
System Support Group
Wellcome Trust Sanger Institute
Wellcome Trust Genome Campus
email: mark.flint [at] sanger<mailto:mark.flint [at] sanger>
On 11 Aug 2012, at 00:17, Ray Van Dolson wrote:
On Fri, Aug 10, 2012 at 05:40:04PM +0000, Jayanathan, David wrote:
How are your routing groups setup, from the "network routing-groups
show" command, and what metric is assigned to each? Do you have your
vserver configured to allow protocol services to use Data LIFs? You
can verify by running a "vserver show -vserver <vservername>".
These commands don't appear to be valid on our version of Data ONTAP.
Here is the top half of our routing tables (the routable stuff):
Destination Gateway Flags Refs Use Interface
default 10.44.15.254 UGS 18 12334464694 e0M
10.44.12/22 link#7 UC 0 0 e0M
10.44.12.6 0:a0:98:1a:bd:c6 UHL 0 0 e0M
10.44.12.7 0:a0:98:1a:bc:c4 UHL 0 12 lo
10.44.15.254 0:0:c:7:ac:64 UHL 1 0 e0M
10.44.15.255 ff:ff:ff:ff:ff:ff UHL 0 32258 e0M
10.49/16 link#14 UC 0 0 ifgrp0-49
10.49.2.69 0:50:56:88:0:32 UHL 0 6 ifgrp0-49
10.49.4.7 0:1c:23:e2:40:c7 UHL 0 6 ifgrp0-49
10.49.4.43 14:fe:b5:c9:39:8e UHL 0 0 ifgrp0-49
10.49.4.68 d0:67:e5:f8:86:68 UHL 0 3 ifgrp0-49
10.49.4.76 0:15:17:45:b5:41 UHL 0 9 ifgrp0-49
10.49.5.106 0:15:17:60:e4:c UHL 0 3 ifgrp0-49
10.49.6.49 0:25:90:3d:47:6e UHL 0 28662 ifgrp0-49
10.49.6.156 0:25:90:63:88:24 UHL 0 7 ifgrp0-49
10.49.6.161 90:e2:ba:13:3:68 UHL 0 12718 ifgrp0-49
10.49.6.162 90:e2:ba:12:fb:3c UHL 0 8764 ifgrp0-49
10.49.6.167 0:25:90:b:15:da UHL 0 4 ifgrp0-49
10.49.11.72 0:15:c5:e1:41:58 UHL 0 18 ifgrp0-49
10.49.13.10 0:21:9b:9d:6:6f UHL 0 18 ifgrp0-49
10.49.13.53 7a:2b:cb:16:7f:a0 UHL 0 12 ifgrp0-49
10.49.51.84 0:50:56:ab:47:9a UHL 1 1742 ifgrp0-49
10.49.51.95 0:50:56:ab:48:a6 UHL 0 1907277490 ifgrp0-49
10.49.51.249 0:15:17:45:66:ec UHL 0 21 ifgrp0-49
10.49.53.11 0:21:9b:ca:96:5 UHL 0 18 ifgrp0-49
10.49.104.131 84:2b:2b:17:12:c5 UHL 0 91806 ifgrp0-49
10.49.104.132 0:26:b9:28:b1:9c UHL 0 0 ifgrp0-49
10.49.104.207 0:50:56:88:5:97 UHL 0 2 ifgrp0-49
10.49.106.139 0:26:b9:28:b1:a9 UHL 0 161 ifgrp0-49
10.49.106.145 0:50:56:88:1:53 UHL 0 37 ifgrp0-49
10.49.107.101 0:21:9b:8f:b5:38 UHL 0 3 ifgrp0-49
10.49.110.207 0:18:8b:3f:38:d3 UHL 0 0 ifgrp0-49
10.49.110.239 0:50:56:ab:47:96 UHL 0 0 ifgrp0-49
10.49.254.252 0:c:31:46:4c:a UHL 0 0 ifgrp0-49
10.49.254.253 0:b:bf:c9:e4:8a UHL 0 0 ifgrp0-49
10.49.255.255 ff:ff:ff:ff:ff:ff UHL 0 32258 ifgrp0-49
We just need to move the default gateway to one of the ifgrp interfaces
isntead of e0M is my impression.
From: Ray Van Dolson [mailto:rvandolson [at] esri]
Sent: Thursday, August 09, 2012 4:40 PM
To: Jayanathan, David
Cc: toasters [at] teaparty<mailto:toasters [at] teaparty>
Subject: Re: e0M port as default gateway?
On Thu, Aug 09, 2012 at 10:54:57PM +0000, Jayanathan, David wrote:
We allow all management activity to go through the same interfaces
that data goes through. Only on our Cluster Mode implementations do we
use e0M for management ports, otherwise we only plug in e0M in order
to have the RLM/SP connections on the network.
Hope this helps!
We are running cluster mode on these systems (two controllers).
Key thing we're seeing is that NFS/CIFS file serving activity grinds
to a halt when we lose connectivity to AD/NIS servers (the
communication for which currently goes through e0M due to default
I presume this is expected behavior, but am a bit surprised that NFS
file serving would hang for activity not involving an NIS UID.
Presumably any NFS requests gets mapped to a UID and would require an
Anyone run into this before?
From: toasters-bounces [at] teaparty<mailto:toasters-bounces [at] teaparty>
[mailto:toasters-bounces [at] teaparty] On Behalf Of Ray Van Dolson
Sent: Thursday, August 09, 2012 2:26 PM
To: toasters [at] teaparty<mailto:toasters [at] teaparty>
Subject: e0M port as default gateway?
We have some N6240's (rebadged FAS3xxx series) on which the default
gateway is set to an IP that routes out the e0M interface.
As such, if the e0M port (wrench port) goes offline, essentially
functionality on that controller ceases as communication with AD, NIS,
etc. stops working.
We have 10GbE ports on these devices -- we could easily configure the
default gateways to use one of those instead, but I am unclear as to
what best practice is here. We access management functionality
through the IP bound on the e0M port now.
How do the rest of you set things up?
Toasters mailing list
Toasters [at] teaparty<mailto:Toasters [at] teaparty>
-- The Wellcome Trust Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE.