Mailing List Archive: Netapp: toasters

FCP Partner Path Misconfigured - Host I/O access through a non-primary and non-optimal path was detected.

Index | Next | Previous | View Threaded

sysadmin.linux at gmail

Mar 4, 2009, 11:38 AM

Post #1 of 13 (7799 views)
Permalink

FCP Partner Path Misconfigured - Host I/O access through a non-primary and non-optimal path was detected.

Hello Toasters.

I am receiving the following message from auto support: FCP Partner Path
Misconfigured - Host I/O access through a non-primary and non-optimal path
was detected.

I have a single cluster acting in single fabric mode and it is wired up in
the following way.
Each host has an i-group on each node of the cluster for. Each of the netapp
node is connected to sw1 and sw2 and hosts also has a path to sw1 and sw2.
Why would the host access luns on N2 via partner node N1

--------------- ---------------
{ n1 } { n2 }

| \ / |
| \ / |
| \ / |
| \ / |

--------------- ---------------
{ s1 } { s2 }

\ /

\ /

---------------
{ host }

Peter.Learmonth at netapp

Mar 4, 2009, 11:59 AM

Post #2 of 13 (7579 views)
Permalink

RE: FCP Partner Path Misconfigured - Host I/O access through a non-primary and non-optimal path was detected. [In reply to]

Hi!
By default, ESX accesses LUNs via the first path discovered. In half of
the cases, this will be through the partner head, which is non-optimal.
The ESX Host Utilities have a script called config_mpath, which is
mainly a wrapper for esxcfg-mpath, that sets paths correctly. I
recommend you download, read the docs, and install the EHU on each of
your ESX hosts.
http://now.netapp.com/NOW/download/software/sanhost_esx/ESX/

Share and enjoy!

Peter

________________________________

From: Linux Admin [mailto:sysadmin.linux [at] gmail]
Sent: Wednesday, March 04, 2009 11:38 AM
To: NetApp Toasters List
Subject: FCP Partner Path Misconfigured - Host I/O access through a
non-primary and non-optimal path was detected.

Hello Toasters.

I am receiving the following message from auto support: FCP Partner Path
Misconfigured - Host I/O access through a non-primary and non-optimal
path was detected.

I have a single cluster acting in single fabric mode and it is wired up
in the following way.
Each host has an i-group on each node of the cluster for. Each of the
netapp node is connected to sw1 and sw2 and hosts also has a path to sw1
and sw2.
Why would the host access luns on N2 via partner node N1

--------------- ---------------
{ n1 } { n2 }

| \ / |
| \ / |
| \ / |
| \ / |

--------------- ---------------
{ s1 } { s2 }

\ /

\ /

---------------
{ host }

fredgrieco at yahoo

Mar 4, 2009, 12:20 PM

Post #3 of 13 (7563 views)
Permalink

Re: FCP Partner Path Misconfigured - Host I/O access through a non-primary and non-optimal path was detected. [In reply to]

I'm a little foggy on the details... it's been a while since I read them or thought about it. But, assuming you mean single_image mode, host can see the luns on all four ports, and data can be accessed through all four ports. Now N1 may own the lun, but it can be accessed "through the interconnect" through N2.

Multipathing on the host determines which path will be used to access data, and whether or not the an active/passive or round robin (or whatever). If you are using Netapp multipathing, it knows to use only the N1 path. But other multipathing doesn't understand this, and will pick the primary path at random.

I have tons of VMWare hosts that have seemless esx multipathing, but pick the wrong paths by themselves. Then you get the error. I get this error all the time and really haven't had a problem with it... but it's the first thing support will bring up when you call about anything. I have noticed that it seems to run the CPU a little high (since it has to swap things in and out of memory through the interconnect).

Fred

________________________________
From: Linux Admin <sysadmin.linux [at] gmail>
To: NetApp Toasters List <toasters [at] mathworks>
Sent: Wednesday, March 4, 2009 2:38:19 PM
Subject: FCP Partner Path Misconfigured - Host I/O access through a non-primary and non-optimal path was detected.

Hello Toasters.

I am receiving the following message from auto support: FCP Partner Path Misconfigured - Host I/O access through a non-primary and non-optimal path was detected.

I have a single cluster acting in single fabric mode and it is wired up in the following way.
Each host has an i-group on each node of the cluster for. Each of the netapp node is connected to sw1 and sw2 and hosts also has a path to sw1 and sw2.
Why would the host access luns on N2 via partner node N1

--------------- ---------------
{ n1 } { n2 }

| \ / |
| \ / |
| \ / |
| \ / |

--------------- ---------------
{ s1 } { s2 }

\ /

\ /

---------------
{ host }

Errol.Fouquet at netapp

Mar 4, 2009, 12:52 PM

Post #4 of 13 (7571 views)
Permalink

RE: FCP Partner Path Misconfigured - Host I/O access through a non-primary and non-optimal path was detected. [In reply to]

Accessing a LUN across a partner path will function fine ... and in some
cases, there may be no problems perceived.
However, IOs across the the partner path do not perform as well under
load than access to the primary paths.
For ESX, proper path prioritization is very easy to accomplish with the
scripts provided in the Host Utilities Kit.

As for the original issue on this thread ... assuming you have
multipathing set up correctly, the warning may be associated with normal
path checking activity.
Check your lun stats data and divide the "partner kb" by "partner ops",
if the answer there is ~512b ... then these warnings could simply be
path checking done by multipathd.
If the average IO size seems significant, open a case with support and
get some assistance in determining why IO is going over the partner
path.

--
errol

________________________________

From: Fred Grieco [mailto:fredgrieco [at] yahoo]
Sent: Wednesday, March 04, 2009 2:21 PM
To: Linux Admin; NetApp Toasters List
Subject: Re: FCP Partner Path Misconfigured - Host I/O access through a
non-primary and non-optimal path was detected.

I'm a little foggy on the details... it's been a while since I read them
or thought about it. But, assuming you mean single_image mode, host can
see the luns on all four ports, and data can be accessed through all
four ports. Now N1 may own the lun, but it can be accessed "through the
interconnect" through N2.

Multipathing on the host determines which path will be used to access
data, and whether or not the an active/passive or round robin (or
whatever). If you are using Netapp multipathing, it knows to use only
the N1 path. But other multipathing doesn't understand this, and will
pick the primary path at random.

I have tons of VMWare hosts that have seemless esx multipathing, but
pick the wrong paths by themselves. Then you get the error. I get this
error all the time and really haven't had a problem with it... but it's
the first thing support will bring up when you call about anything. I
have noticed that it seems to run the CPU a little high (since it has to
swap things in and out of memory through the interconnect).

Fred

________________________________

From: Linux Admin <sysadmin.linux [at] gmail>
To: NetApp Toasters List <toasters [at] mathworks>
Sent: Wednesday, March 4, 2009 2:38:19 PM
Subject: FCP Partner Path Misconfigured - Host I/O access through a
non-primary and non-optimal path was detected.

Hello Toasters.

I am receiving the following message from auto support: FCP Partner Path
Misconfigured - Host I/O access through a non-primary and non-optimal
path was detected.

I have a single cluster acting in single fabric mode and it is wired up
in the following way.
Each host has an i-group on each node of the cluster for. Each of the
netapp node is connected to sw1 and sw2 and hosts also has a path to sw1
and sw2.
Why would the host access luns on N2 via partner node N1

--------------- ---------------
{ n1 } { n2 }

| \ / |
| \ / |
| \ / |
| \ / |

--------------- ---------------
{ s1 } { s2 }

\ /

\ /

---------------
{ host }

sysadmin.linux at gmail

Mar 4, 2009, 12:55 PM

Post #5 of 13 (7574 views)
Permalink

Re: FCP Partner Path Misconfigured - Host I/O access through a non-primary and non-optimal path was detected. [In reply to]

Thank you all for your help!

On Wed, Mar 4, 2009 at 2:52 PM, Fouquet, Errol <Errol.Fouquet [at] netapp>wrote:

> Accessing a LUN across a partner path will function fine ... and in some
> cases, there may be no problems perceived.
> However, IOs across the the partner path do not perform as well under load
> than access to the primary paths.
> For ESX, proper path prioritization is very easy to accomplish with the
> scripts provided in the Host Utilities Kit.
>
> As for the original issue on this thread ... assuming you have multipathing
> set up correctly, the warning may be associated with normal path checking
> activity.
> Check your lun stats data and divide the "partner kb" by "partner ops", if
> the answer there is ~512b ... then these warnings could simply be path
> checking done by multipathd.
> If the average IO size seems significant, open a case with support and get
> some assistance in determining why IO is going over the partner path.
>
> --
> errol
>
>
> ------------------------------
> *From:* Fred Grieco [mailto:fredgrieco [at] yahoo]
> *Sent:* Wednesday, March 04, 2009 2:21 PM
> *To:* Linux Admin; NetApp Toasters List
> *Subject:* Re: FCP Partner Path Misconfigured - Host I/O access through a
> non-primary and non-optimal path was detected.
>
> I'm a little foggy on the details... it's been a while since I read them
> or thought about it. But, assuming you mean single_image mode, host can see
> the luns on all four ports, and data can be accessed through all four
> ports. Now N1 may own the lun, but it can be accessed "through the
> interconnect" through N2.
>
> Multipathing on the host determines which path will be used to access data,
> and whether or not the an active/passive or round robin (or whatever). If
> you are using Netapp multipathing, it knows to use only the N1 path. But
> other multipathing doesn't understand this, and will pick the primary path
> at random.
>
> I have tons of VMWare hosts that have seemless esx multipathing, but pick
> the wrong paths by themselves. Then you get the error. I get this error
> all the time and really haven't had a problem with it... but it's the first
> thing support will bring up when you call about anything. I have noticed
> that it seems to run the CPU a little high (since it has to swap things in
> and out of memory through the interconnect).
>
> Fred
>
> ------------------------------
> *From:* Linux Admin <sysadmin.linux [at] gmail>
> *To:* NetApp Toasters List <toasters [at] mathworks>
> *Sent:* Wednesday, March 4, 2009 2:38:19 PM
> *Subject:* FCP Partner Path Misconfigured - Host I/O access through a
> non-primary and non-optimal path was detected.
>
> Hello Toasters.
>
> I am receiving the following message from auto support: FCP Partner Path
> Misconfigured - Host I/O access through a non-primary and non-optimal path
> was detected.
>
> I have a single cluster acting in single fabric mode and it is wired up in
> the following way.
> Each host has an i-group on each node of the cluster for. Each of the
> netapp node is connected to sw1 and sw2 and hosts also has a path to sw1 and
> sw2.
> Why would the host access luns on N2 via partner node N1
>
>
>
> --------------- ---------------
> { n1 } { n2 }
>
>
> | \ / |
> | \ / |
> | \ / |
> | \ / |
>
>
> --------------- ---------------
> { s1 } { s2 }
>
> \ /
>
> \ /
>
> ---------------
> { host }
>

jack1729 at gmail

Mar 4, 2009, 4:33 PM

Post #6 of 13 (7566 views)
Permalink

Re: FCP Partner Path Misconfigured - Host I/O access through a non-primary and non-optimal path was detected. [In reply to]

We wrote our own script that will fix the optimal path across all 12 esx
servers, but over time they seem to switch but aren't sure what is
causing the path to switch. We know the paths aren't going down and we
are not exceeding the capacity of the ports?

Thanks.
Jack

Linux Admin wrote:
> Thank you all for your help!
>
> On Wed, Mar 4, 2009 at 2:52 PM, Fouquet, Errol
> <Errol.Fouquet [at] netapp <mailto:Errol.Fouquet [at] netapp>> wrote:
>
> Accessing a LUN across a partner path will function fine ... and
> in some cases, there may be no problems perceived.
> However, IOs across the the partner path do not perform as well
> under load than access to the primary paths.
> For ESX, proper path prioritization is very easy to accomplish
> with the scripts provided in the Host Utilities Kit.
>
> As for the original issue on this thread ... assuming you have
> multipathing set up correctly, the warning may be associated with
> normal path checking activity.
> Check your lun stats data and divide the "partner kb" by "partner
> ops", if the answer there is ~512b ... then these warnings could
> simply be path checking done by multipathd.
> If the average IO size seems significant, open a case with support
> and get some assistance in determining why IO is going over the
> partner path.
>
> --
> errol
>
>
> ------------------------------------------------------------------------
> *From:* Fred Grieco [mailto:fredgrieco [at] yahoo
> <mailto:fredgrieco [at] yahoo>]
> *Sent:* Wednesday, March 04, 2009 2:21 PM
>
> *To:* Linux Admin; NetApp Toasters List
> *Subject:* Re: FCP Partner Path Misconfigured - Host I/O access
> through a non-primary and non-optimal path was detected.
>
> I'm a little foggy on the details... it's been a while since I
> read them or thought about it. But, assuming you mean
> single_image mode, host can see the luns on all four ports, and
> data can be accessed through all four ports. Now N1 may own the
> lun, but it can be accessed "through the interconnect" through N2.
>
> Multipathing on the host determines which path will be used to
> access data, and whether or not the an active/passive or round
> robin (or whatever). If you are using Netapp multipathing, it
> knows to use only the N1 path. But other multipathing doesn't
> understand this, and will pick the primary path at random.
>
> I have tons of VMWare hosts that have seemless esx multipathing,
> but pick the wrong paths by themselves. Then you get the error.
> I get this error all the time and really haven't had a problem
> with it... but it's the first thing support will bring up when you
> call about anything. I have noticed that it seems to run the CPU
> a little high (since it has to swap things in and out of memory
> through the interconnect).
>
> Fred
>
> ------------------------------------------------------------------------
> *From:* Linux Admin <sysadmin.linux [at] gmail
> <mailto:sysadmin.linux [at] gmail>>
> *To:* NetApp Toasters List <toasters [at] mathworks
> <mailto:toasters [at] mathworks>>
> *Sent:* Wednesday, March 4, 2009 2:38:19 PM
> *Subject:* FCP Partner Path Misconfigured - Host I/O access
> through a non-primary and non-optimal path was detected.
>
> Hello Toasters.
>
> I am receiving the following message from auto support: FCP
> Partner Path Misconfigured - Host I/O access through a non-primary
> and non-optimal path was detected.
>
> I have a single cluster acting in single fabric mode and it is
> wired up in the following way.
> Each host has an i-group on each node of the cluster for. Each of
> the netapp node is connected to sw1 and sw2 and hosts also has a
> path to sw1 and sw2.
> Why would the host access luns on N2 via partner node N1
>
>
>
> --------------- ---------------
> { n1 } { n2 }
>
>
> | \ / |
> | \ / |
> | \ / |
> | \ / |
>
>
> --------------- ---------------
> { s1 } { s2 }
>
> \ /
>
> \ /
>
> ---------------
> { host }
>
>

sysadmin.linux at gmail

Mar 5, 2009, 1:41 PM

Post #7 of 13 (7541 views)
Permalink

Re: FCP Partner Path Misconfigured - Host I/O access through a non-primary and non-optimal path was detected. [In reply to]

I solaris 10 as bad about this as ESX server?
I also see the same issue with Solaris server

On Wed, Mar 4, 2009 at 6:33 PM, Jack Lyons <jack1729 [at] gmail> wrote:

> We wrote our own script that will fix the optimal path across all 12 esx
> servers, but over time they seem to switch but aren't sure what is causing
> the path to switch. We know the paths aren't going down and we are not
> exceeding the capacity of the ports?
>
> Thanks.
> Jack
>
>
> Linux Admin wrote:
>
>> Thank you all for your help!
>>
>> On Wed, Mar 4, 2009 at 2:52 PM, Fouquet, Errol <Errol.Fouquet [at] netapp<mailto:
>> Errol.Fouquet [at] netapp>> wrote:
>>
>> Accessing a LUN across a partner path will function fine ... and
>> in some cases, there may be no problems perceived.
>> However, IOs across the the partner path do not perform as well
>> under load than access to the primary paths.
>> For ESX, proper path prioritization is very easy to accomplish
>> with the scripts provided in the Host Utilities Kit.
>> As for the original issue on this thread ... assuming you have
>> multipathing set up correctly, the warning may be associated with
>> normal path checking activity.
>> Check your lun stats data and divide the "partner kb" by "partner
>> ops", if the answer there is ~512b ... then these warnings could
>> simply be path checking done by multipathd.
>> If the average IO size seems significant, open a case with support
>> and get some assistance in determining why IO is going over the
>> partner path.
>> --
>> errol
>>
>>
>> ------------------------------------------------------------------------
>> *From:* Fred Grieco [mailto:fredgrieco [at] yahoo
>> <mailto:fredgrieco [at] yahoo>]
>> *Sent:* Wednesday, March 04, 2009 2:21 PM
>>
>> *To:* Linux Admin; NetApp Toasters List
>> *Subject:* Re: FCP Partner Path Misconfigured - Host I/O access
>> through a non-primary and non-optimal path was detected.
>>
>> I'm a little foggy on the details... it's been a while since I
>> read them or thought about it. But, assuming you mean
>> single_image mode, host can see the luns on all four ports, and
>> data can be accessed through all four ports. Now N1 may own the
>> lun, but it can be accessed "through the interconnect" through N2.
>>
>> Multipathing on the host determines which path will be used to
>> access data, and whether or not the an active/passive or round
>> robin (or whatever). If you are using Netapp multipathing, it
>> knows to use only the N1 path. But other multipathing doesn't
>> understand this, and will pick the primary path at random.
>>
>> I have tons of VMWare hosts that have seemless esx multipathing,
>> but pick the wrong paths by themselves. Then you get the error. I
>> get this error all the time and really haven't had a problem
>> with it... but it's the first thing support will bring up when you
>> call about anything. I have noticed that it seems to run the CPU
>> a little high (since it has to swap things in and out of memory
>> through the interconnect).
>>
>> Fred
>>
>>
>> ------------------------------------------------------------------------
>> *From:* Linux Admin <sysadmin.linux [at] gmail
>> <mailto:sysadmin.linux [at] gmail>>
>> *To:* NetApp Toasters List <toasters [at] mathworks
>> <mailto:toasters [at] mathworks>>
>> *Sent:* Wednesday, March 4, 2009 2:38:19 PM
>> *Subject:* FCP Partner Path Misconfigured - Host I/O access
>> through a non-primary and non-optimal path was detected.
>>
>> Hello Toasters.
>>
>> I am receiving the following message from auto support: FCP
>> Partner Path Misconfigured - Host I/O access through a non-primary
>> and non-optimal path was detected.
>>
>> I have a single cluster acting in single fabric mode and it is
>> wired up in the following way.
>> Each host has an i-group on each node of the cluster for. Each of
>> the netapp node is connected to sw1 and sw2 and hosts also has a
>> path to sw1 and sw2.
>> Why would the host access luns on N2 via partner node N1
>>
>>
>>
>> --------------- ---------------
>> { n1 } { n2 }
>>
>> | \ / |
>> | \ / |
>> | \ / |
>> | \ / |
>>
>>
>> --------------- ---------------
>> { s1 } { s2 }
>>
>> \ /
>>
>> \ /
>>
>> ---------------
>> { host }
>>
>>
>>
>

Silviu.Angelescu at netapp

Mar 5, 2009, 1:58 PM

Post #8 of 13 (7664 views)
Permalink

Re: FCP Partner Path Misconfigured - Host I/O access through a non-primary and non-optimal path was detected. [In reply to]

Solaris 10 U2 and later supports ALUA. NetApp also supports ALUA. Bingo!
Enable ALUA on Solaris. Enable ALUA on the igroup on the NetApp side, set
cfmode to single image and off you go.

Silviu

On 3/5/09 4:41 PM, "Linux Admin" <sysadmin.linux [at] gmail> wrote:

> I solaris 10 as bad about this as ESX server?
> I also see the same issue with Solaris server
>
>
> On Wed, Mar 4, 2009 at 6:33 PM, Jack Lyons <jack1729 [at] gmail> wrote:
>> We wrote our own script that will fix the optimal path across all 12 esx
>> servers, but over time they seem to switch but aren't sure what is causing
>> the path to switch. �We know the paths aren't going down and we are not
>> exceeding the capacity of the ports?
>> da totaul
>> Thanks.
>> Jack
>>
>>
>> Linux Admin wrote:
>>> Thank you all for your help!
>>>
>>> On Wed, Mar 4, 2009 at 2:52 PM, Fouquet, Errol <Errol.Fouquet [at] netapp
>>> <mailto:Errol.Fouquet [at] netapp>> wrote:
>>>
>>> � �Accessing a LUN across a partner path will function fine ... and
>>> � �in some cases, there may be no problems perceived.
>>> � �However, IOs across the the partner path do not perform as well
>>> � �under load than access to the primary paths.
>>> � �For ESX, proper path prioritization is very easy to accomplish
>>> � �with the scripts provided in the Host Utilities Kit.
>>> � � � �As for the original issue on this thread ... assuming you have
>>> � �multipathing set up correctly, the warning may be associated with
>>> � �normal path checking activity.
>>> � �Check your lun stats data and divide the "partner kb" by "partner
>>> � �ops", if the answer there is ~512b ... then these warnings could
>>> � �simply be path checking done by multipathd.
>>> � �If the average IO size seems significant, open a case with support
>>> � �and get some assistance in determining why IO is going over the
>>> � �partner path.
>>> � � � �--
>>> � �errol
>>> � �
>>> � �------------------------------------------------------------------------
>>> � �*From:* Fred Grieco [mailto:fredgrieco [at] yahoo
>>> � �<mailto:fredgrieco [at] yahoo>]
>>> � �*Sent:* Wednesday, March 04, 2009 2:21 PM
>>>
>>> � �*To:* Linux Admin; NetApp Toasters List
>>> � �*Subject:* Re: FCP Partner Path Misconfigured - Host I/O access
>>> � �through a non-primary and non-optimal path was detected.
>>>
>>> � �I'm a little foggy on the details... it's been a while since I
>>> � �read them or thought about it. �But, assuming you mean
>>> � �single_image mode, host can see the luns on all four ports, and
>>> � �data can be accessed through all four ports. �Now N1 may own the
>>> � �lun, but it can be accessed "through the interconnect" through N2.
>>>
>>> � �Multipathing on the host determines which path will be used to
>>> � �access data, and whether or not the an active/passive or round
>>> � �robin (or whatever). �If you are using Netapp multipathing, it
>>> � �knows to use only the N1 path. �But other multipathing doesn't
>>> � �understand this, and will pick the primary path at random.
>>>
>>> � �I have tons of VMWare hosts that have seemless esx multipathing,
>>> � �but pick the wrong paths by themselves. �Then you get the error. � �I
>>> get this error all the time and really haven't had a problem
>>> � �with it... but it's the first thing support will bring up when you
>>> � �call about anything. �I have noticed that it seems to run the CPU
>>> � �a little high (since it has to swap things in and out of memory
>>> � �through the interconnect).
>>>
>>> � �Fred
>>>
>>> � �------------------------------------------------------------------------
>>> � �*From:* Linux Admin <sysadmin.linux [at] gmail
>>> � �<mailto:sysadmin.linux [at] gmail>>
>>>
>>> � �*To:* NetApp Toasters List <toasters [at] mathworks
>>> � �<mailto:toasters [at] mathworks>>
>>>
>>> � �*Sent:* Wednesday, March 4, 2009 2:38:19 PM
>>> � �*Subject:* FCP Partner Path Misconfigured - Host I/O access
>>> � �through a non-primary and non-optimal path was detected.
>>>
>>> � �Hello Toasters.
>>>
>>> � �I am receiving the following message from auto support: FCP
>>> � �Partner Path Misconfigured - Host I/O access through a non-primary
>>> � �and non-optimal path was detected.
>>>
>>> � �I have a single cluster acting in single fabric mode and it is
>>> � �wired up in the following way.
>>> � �Each host has an i-group on each node of the cluster for. Each of
>>> � �the netapp node is connected to sw1 and sw2 and hosts also has a
>>> � �path to sw1 and sw2.
>>> � �Why would the host access luns on N2 via partner node N1
>>>
>>>
>>>
>>> � � --------------- � � � � � � ---------------
>>> � �{ � �n1 � �} � � � � � �{ � �n2 � �}
>>>
>>> � � � � � �| � �\ � � � � � � � � �/ � �|
>>> � � � �| � � \ � � � � � � � �/ � � |
>>> � � � �| � � �\ � � � � � � �/ � � �|
>>> � � � �| � � � \ � � � � � �/ � � � �|
>>>
>>>
>>> � � --------------- � � � � � � ---------------
>>> � �{ � �s1 � �} � � � � � �{ � �s2 � �}
>>>
>>> � � � � � � � \ � � � � � � �/
>>>
>>> � � � � � � � �\ � � � � � �/
>>>
>>> � � � � � � � � ---------------
>>> � � � � � � � �{ � �host � �}
>>>
>>>
>>
>
>

sysadmin.linux at gmail

Mar 5, 2009, 2:15 PM

Post #9 of 13 (7546 views)
Permalink

Re: FCP Partner Path Misconfigured - Host I/O access through a non-primary and non-optimal path was detected. [In reply to]

I am bit confused, even Solaris 10 with MPIO has problme figure out what
correct path to take

It has 2 path on san (sw1 and sw2) and efectivly 4 path to lun (netapp1 and
netapp2 are connected each into sw1 and sw2)

Why would both Solaris and ESX server access netapp1 luns via netapp2 for
most parts?

Should I just kill fabric mash and have netap1-sw1 and netap2-sw2 to
simplify things?

Do host utulities for Solaris help with that at all?

On Thu, Mar 5, 2009 at 3:59 PM, Brad Knowles <brad [at] shub-internet> wrote:

> Linux Admin wrote:
>
> I solaris 10 as bad about this as ESX server?
>> I also see the same issue with Solaris server
>>
>
> Ironically, I overheard my co-workers talking about this very subject
> outside my office just a few minutes ago.
>
> There are apparently still problems with ESX with Linux (and maybe Solaris)
> VMs where there is a minor hiccup and the pre-configured primary path is
> replaced by ESX with the alternate path. Unlike what happens when you do a
> VMotion, the OS is not stopped by ESX during this process -- all the VM sees
> is that the disks have gone away (albeit for a brief period of time), so
> they mark the disks as read-only and then you're hosed until someone logs in
> and reboots the VM or otherwise fixes it.
>
> I think this is caused by a disconnect with the multipathing done at the
> ESX level and no multipathing done at the VM level. I strongly suspect
> you'll need to install multipathing software at the VM level that is
> compatible with the multipathing used by your storage -- so, EMC PowerPath,
> if the underlying storage is on EMC, etc....
>
> Even if ESX is doing multipathing just fine, that doesn't necessarily mean
> that the hosted VMs can also automatically do multipathing in a way that
> will work with what the storage vendor supplies, what ESX does, and so on.
>
>
> Our solution is that the storage is simply never allowed to go down, and we
> don't touch anything that is served to the ESX machines without first
> getting prior approval and coordination with the ESX administrators, as well
> as with the appropriate VM administrators.
>
> And when you have hundreds of VMs on a set of ESX servers in a single
> infrastructure which could potentially be VMotion'ed at any time from one
> node to another, that is a hell of a lot of pre-coordination.
>
> Even the slightest bump of a cable could do something that the storage or
> ESX doesn't like, they do their multipath thing, and then all the affected
> VMs on the box are screwed. Oops.
>
> --
> Brad Knowles
> <brad [at] shub-internet> If you like Jazz/R&B guitar, check out
> LinkedIn Profile: my friend bigsbytracks on YouTube at
> <http://tinyurl.com/y8kpxu> http://preview.tinyurl.com/bigsbytracks
>

Silviu.Angelescu at netapp

Mar 5, 2009, 3:04 PM

Post #10 of 13 (7541 views)
Permalink

Re: FCP Partner Path Misconfigured - Host I/O access through a non-primary and non-optimal path was detected. [In reply to]

Make sure ALUA is enabled on Solaris as well as on NetApp and set the cfmode to single image on NetApp.

I suspect ALUA may be off on Solaris. If you installed the host utilities for solaris run "basic_config -ssd_set",
"reboot -- -r", "stmsboot -e" (do not let stmsboot to reboot; use reboot instead in next step), "reboot -- -r". This should work.

Silviu
.

________________________________

From: Linux Admin
To: Brad Knowles
Cc: Jack Lyons ; NetApp Toasters List
Sent: Thu Mar 05 17:15:45 2009
Subject: Re: FCP Partner Path Misconfigured - Host I/O access through a non-primary and non-optimal path was detected.

I am bit confused, even Solaris 10 with MPIO has problme figure out what correct path to take

It has 2 path on san (sw1 and sw2) and efectivly 4 path to lun (netapp1 and netapp2 are connected each into sw1 and sw2)

Why would both Solaris and ESX server access netapp1 luns via netapp2 for most parts?

Should I just kill fabric mash and have netap1-sw1 and netap2-sw2 to simplify things?

Do host utulities for Solaris help with that at all?

On Thu, Mar 5, 2009 at 3:59 PM, Brad Knowles <brad [at] shub-internet> wrote:

Linux Admin wrote:

I solaris 10 as bad about this as ESX server?
I also see the same issue with Solaris server

Ironically, I overheard my co-workers talking about this very subject outside my office just a few minutes ago.

There are apparently still problems with ESX with Linux (and maybe Solaris) VMs where there is a minor hiccup and the pre-configured primary path is replaced by ESX with the alternate path. Unlike what happens when you do a VMotion, the OS is not stopped by ESX during this process -- all the VM sees is that the disks have gone away (albeit for a brief period of time), so they mark the disks as read-only and then you're hosed until someone logs in and reboots the VM or otherwise fixes it.

I think this is caused by a disconnect with the multipathing done at the ESX level and no multipathing done at the VM level. I strongly suspect you'll need to install multipathing software at the VM level that is compatible with the multipathing used by your storage -- so, EMC PowerPath, if the underlying storage is on EMC, etc....

Even if ESX is doing multipathing just fine, that doesn't necessarily mean that the hosted VMs can also automatically do multipathing in a way that will work with what the storage vendor supplies, what ESX does, and so on.

Our solution is that the storage is simply never allowed to go down, and we don't touch anything that is served to the ESX machines without first getting prior approval and coordination with the ESX administrators, as well as with the appropriate VM administrators.

And when you have hundreds of VMs on a set of ESX servers in a single infrastructure which could potentially be VMotion'ed at any time from one node to another, that is a hell of a lot of pre-coordination.

Even the slightest bump of a cable could do something that the storage or ESX doesn't like, they do their multipath thing, and then all the affected VMs on the box are screwed. Oops.

--
Brad Knowles
<brad [at] shub-internet> If you like Jazz/R&B guitar, check out
LinkedIn Profile: my friend bigsbytracks on YouTube at
<http://tinyurl.com/y8kpxu> http://preview.tinyurl.com/bigsbytracks

Errol.Fouquet at netapp

Mar 6, 2009, 5:05 AM

Post #11 of 13 (7701 views)
Permalink

RE: FCP Partner Path Misconfigured - Host I/O access through a non-primary and non-optimal path was detected. [In reply to]

Another common mistake made on Solaris/FCP configurations is the
unintentional use of the mpxio_set script. I've seen some FCP customers
run "mpxio_set -e" by mistake. The end result of doing this is that all
paths appear to be primary paths. Running "mpxio_set -e" is only done
for iSCSI configurations.

Here are some notes about this:

* If ALUA is enabled on the filer, you must not use the "mpxio_set
-e" command.
* If you had previously used the mpxio_set script to add Netapp
settings to the /kernel/drv/scsi_vhci.conf file, ALUA will not work. You
MUST remove those using the "mpxio_set -d" command and then reboot to
enable ALUA.

________________________________

From: Angelescu, Silviu
Sent: Thursday, March 05, 2009 5:05 PM
To: sysadmin.linux [at] gmail; brad [at] shub-internet
Cc: jack1729 [at] gmail; toasters [at] mathworks
Subject: Re: FCP Partner Path Misconfigured - Host I/O access through a
non-primary and non-optimal path was detected.

Make sure ALUA is enabled on Solaris as well as on NetApp and set the
cfmode to single image on NetApp.

I suspect ALUA may be off on Solaris. If you installed the host
utilities for solaris run "basic_config -ssd_set",
"reboot -- -r", "stmsboot -e" (do not let stmsboot to reboot; use reboot
instead in next step), "reboot -- -r". This should work.

Silviu
.

________________________________

From: Linux Admin
To: Brad Knowles
Cc: Jack Lyons ; NetApp Toasters List
Sent: Thu Mar 05 17:15:45 2009
Subject: Re: FCP Partner Path Misconfigured - Host I/O access through a
non-primary and non-optimal path was detected.

I am bit confused, even Solaris 10 with MPIO has problme figure out what
correct path to take

It has 2 path on san (sw1 and sw2) and efectivly 4 path to lun (netapp1
and netapp2 are connected each into sw1 and sw2)

Why would both Solaris and ESX server access netapp1 luns via netapp2
for most parts?

Should I just kill fabric mash and have netap1-sw1 and netap2-sw2 to
simplify things?

Do host utulities for Solaris help with that at all?

On Thu, Mar 5, 2009 at 3:59 PM, Brad Knowles <brad [at] shub-internet>
wrote:

Linux Admin wrote:

I solaris 10 as bad about this as ESX server?
I also see the same issue with Solaris server

Ironically, I overheard my co-workers talking about this very
subject outside my office just a few minutes ago.

There are apparently still problems with ESX with Linux (and
maybe Solaris) VMs where there is a minor hiccup and the pre-configured
primary path is replaced by ESX with the alternate path. Unlike what
happens when you do a VMotion, the OS is not stopped by ESX during this
process -- all the VM sees is that the disks have gone away (albeit for
a brief period of time), so they mark the disks as read-only and then
you're hosed until someone logs in and reboots the VM or otherwise fixes
it.

I think this is caused by a disconnect with the multipathing
done at the ESX level and no multipathing done at the VM level. I
strongly suspect you'll need to install multipathing software at the VM
level that is compatible with the multipathing used by your storage --
so, EMC PowerPath, if the underlying storage is on EMC, etc....

Even if ESX is doing multipathing just fine, that doesn't
necessarily mean that the hosted VMs can also automatically do
multipathing in a way that will work with what the storage vendor
supplies, what ESX does, and so on.

Our solution is that the storage is simply never allowed to go
down, and we don't touch anything that is served to the ESX machines
without first getting prior approval and coordination with the ESX
administrators, as well as with the appropriate VM administrators.

And when you have hundreds of VMs on a set of ESX servers in a
single infrastructure which could potentially be VMotion'ed at any time
from one node to another, that is a hell of a lot of pre-coordination.

Even the slightest bump of a cable could do something that the
storage or ESX doesn't like, they do their multipath thing, and then all
the affected VMs on the box are screwed. Oops.

--
Brad Knowles
<brad [at] shub-internet> If you like Jazz/R&B guitar,
check out
LinkedIn Profile: my friend bigsbytracks on
YouTube at
<http://tinyurl.com/y8kpxu>
http://preview.tinyurl.com/bigsbytracks

sysadmin.linux at gmail

Mar 18, 2009, 12:34 PM

Post #12 of 13 (7352 views)
Permalink

Re: FCP Partner Path Misconfigured - Host I/O access through a non-primary and non-optimal path was detected. [In reply to]

I wanted to thank everyone for their great suggestion on vmware and solaris.
My only other question to the group is in regard to esx 3i.
What to do with ESX 3i?

On Wed, Mar 4, 2009 at 2:59 PM, Learmonth, Peter <Peter.Learmonth [at] netapp
> wrote:

> Hi!
> By default, ESX accesses LUNs via the first path discovered. In half of
> the cases, this will be through the partner head, which is non-optimal. The
> ESX Host Utilities have a script called config_mpath, which is mainly a
> wrapper for esxcfg-mpath, that sets paths correctly. I recommend you
> download, read the docs, and install the EHU on each of your ESX hosts.
> http://now.netapp.com/NOW/download/software/sanhost_esx/ESX/
>
> Share and enjoy!
>
> Peter
>
> ------------------------------
> *From:* Linux Admin [mailto:sysadmin.linux [at] gmail]
> *Sent:* Wednesday, March 04, 2009 11:38 AM
> *To:* NetApp Toasters List
> *Subject:* FCP Partner Path Misconfigured - Host I/O access through a
> non-primary and non-optimal path was detected.
>
> Hello Toasters.
>
> I am receiving the following message from auto support: FCP Partner Path
> Misconfigured - Host I/O access through a non-primary and non-optimal path
> was detected.
>
> I have a single cluster acting in single fabric mode and it is wired up in
> the following way.
> Each host has an i-group on each node of the cluster for. Each of the
> netapp node is connected to sw1 and sw2 and hosts also has a path to sw1 and
> sw2.
> Why would the host access luns on N2 via partner node N1
>
>
>
> --------------- ---------------
> { n1 } { n2 }
>
>
> | \ / |
> | \ / |
> | \ / |
> | \ / |
>
>
> --------------- ---------------
> { s1 } { s2 }
>
> \ /
>
> \ /
>
> ---------------
> { host }
>

sysadmin.linux at gmail

Mar 18, 2009, 12:58 PM

Post #13 of 13 (7327 views)
Permalink

Re: FCP Partner Path Misconfigured - Host I/O access through a non-primary and non-optimal path was detected. [In reply to]

Found the KB:
https://now.netapp.com/Knowledgebase/solutionarea.asp?id=kb44784

On Wed, Mar 18, 2009 at 2:34 PM, Linux Admin <sysadmin.linux [at] gmail>wrote:

> I wanted to thank everyone for their great suggestion on vmware and
> solaris.
> My only other question to the group is in regard to esx 3i.
> What to do with ESX 3i?
>
>
> On Wed, Mar 4, 2009 at 2:59 PM, Learmonth, Peter <
> Peter.Learmonth [at] netapp> wrote:
>
>> Hi!
>> By default, ESX accesses LUNs via the first path discovered. In half of
>> the cases, this will be through the partner head, which is non-optimal. The
>> ESX Host Utilities have a script called config_mpath, which is mainly a
>> wrapper for esxcfg-mpath, that sets paths correctly. I recommend you
>> download, read the docs, and install the EHU on each of your ESX hosts.
>> http://now.netapp.com/NOW/download/software/sanhost_esx/ESX/
>>
>> Share and enjoy!
>>
>> Peter
>>
>> ------------------------------
>> *From:* Linux Admin [mailto:sysadmin.linux [at] gmail]
>> *Sent:* Wednesday, March 04, 2009 11:38 AM
>> *To:* NetApp Toasters List
>> *Subject:* FCP Partner Path Misconfigured - Host I/O access through a
>> non-primary and non-optimal path was detected.
>>
>> Hello Toasters.
>>
>> I am receiving the following message from auto support: FCP Partner Path
>> Misconfigured - Host I/O access through a non-primary and non-optimal path
>> was detected.
>>
>> I have a single cluster acting in single fabric mode and it is wired up in
>> the following way.
>> Each host has an i-group on each node of the cluster for. Each of the
>> netapp node is connected to sw1 and sw2 and hosts also has a path to sw1 and
>> sw2.
>> Why would the host access luns on N2 via partner node N1
>>
>>
>>
>> --------------- ---------------
>> { n1 } { n2 }
>>
>>
>> | \ / |
>> | \ / |
>> | \ / |
>> | \ / |
>>
>>
>> --------------- ---------------
>> { s1 } { s2 }
>>
>> \ /
>>
>> \ /
>>
>> ---------------
>> { host }
>>
>
>

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads