Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Netapp: toasters

limit i/o for lun or volume

 

 

Netapp toasters RSS feed   Index | Next | Previous | View Threaded


alonz at emet

Dec 6, 2011, 12:20 AM

Post #1 of 7 (1425 views)
Permalink
limit i/o for lun or volume

Hi all
i have a situation of a virtual machine running on esx 4.0 over iscsi
this machine (while the user is running his stuff) is using a lot of
i/o on a very small netapp controller (2020) with only 5 data disks aggr
and getting very bad latency and eventually getting a lun reset errors
and the machine freeze for a few minutes
i have increased the timeouts from the windows side but it doesn't seems
to help
this application don't need so much i/o and can be run a 5.4k disk on an
old laptop but given the resources of netapp it uses them to the fullest
it is important to mention that this application is running over MSSQL
db and the i/o problem goes to rdm lun
my question: is there a way to limit the i/o of this machine from either
netapp side /vmware side / windows side / network side
i'm aweare of vmware storage i/o control but this is only supported in
vsphere 4.1 and an upgrade is not an options right now
i'm also aware of flexshare from the netapp side but i don't think
giving low priority to this volume will help in this case
is there other way you can think of to limit the i/o goes from vmware
virtual machine through iscsi to an rdm lun or his hosting volume?

thank you

_______________________________________________
Toasters mailing list
Toasters [at] teaparty
http://www.teaparty.net/mailman/listinfo/toasters


speedtoys.racing at gmail

Dec 6, 2011, 12:17 AM

Post #2 of 7 (1369 views)
Permalink
Re: limit i/o for lun or volume [In reply to]

Is the LUN aligned properly?

On Tue, Dec 6, 2011 at 3:00 AM, Alon Zeltser <alonz [at] emet> wrote:

> Hi all
> i have a situation of a virtual machine running on esx 4.0 over iscsi
> this machine (while the user is running his stuff) is using a lot of
> i/o on a very small netapp controller (2020) with only 5 data disks aggr
> and getting very bad latency and eventually getting a lun reset errors
> and the machine freeze for a few minutes
> i have increased the timeouts from the windows side but it doesn't seems
> to help
> this application don't need so much i/o and can be run a 5.4k disk on an
> old laptop but given the resources of netapp it uses them to the fullest
> it is important to mention that this application is running over MSSQL
> db and the i/o problem goes to rdm lun
> my question: is there a way to limit the i/o of this machine from either
> netapp side /vmware side / windows side / network side
> i'm aweare of vmware storage i/o control but this is only supported in
> vsphere 4.1 and an upgrade is not an options right now
> i'm also aware of flexshare from the netapp side but i don't think
> giving low priority to this volume will help in this case
> is there other way you can think of to limit the i/o goes from vmware
> virtual machine through iscsi to an rdm lun or his hosting volume?
>
> thank you
>
> _______________________________________________
> Toasters mailing list
> Toasters [at] teaparty
> http://www.teaparty.net/mailman/listinfo/toasters
>



--
---
Gustatus Similis Pullus


alonz at emet

Dec 6, 2011, 1:05 AM

Post #3 of 7 (1370 views)
Permalink
Re: limit i/o for lun or volume [In reply to]

i believe so
since we are talking about an rdm with an ntfs file system (created with
snapdrive ) there isn't usually an alignment issue like on vmfs luns


On 12/06/2011 10:17 AM, Jeff Mohler wrote:
> Is the LUN aligned properly?
>
> On Tue, Dec 6, 2011 at 3:00 AM, Alon Zeltser <alonz [at] emet
> <mailto:alonz [at] emet>> wrote:
>
> Hi all
> i have a situation of a virtual machine running on esx 4.0 over iscsi
> this machine (while the user is running his stuff) is using a lot of
> i/o on a very small netapp controller (2020) with only 5 data
> disks aggr
> and getting very bad latency and eventually getting a lun reset errors
> and the machine freeze for a few minutes
> i have increased the timeouts from the windows side but it doesn't
> seems
> to help
> this application don't need so much i/o and can be run a 5.4k disk
> on an
> old laptop but given the resources of netapp it uses them to the
> fullest
> it is important to mention that this application is running over MSSQL
> db and the i/o problem goes to rdm lun
> my question: is there a way to limit the i/o of this machine from
> either
> netapp side /vmware side / windows side / network side
> i'm aweare of vmware storage i/o control but this is only supported in
> vsphere 4.1 and an upgrade is not an options right now
> i'm also aware of flexshare from the netapp side but i don't think
> giving low priority to this volume will help in this case
> is there other way you can think of to limit the i/o goes from vmware
> virtual machine through iscsi to an rdm lun or his hosting volume?
>
> thank you
>
> _______________________________________________
> Toasters mailing list
> Toasters [at] teaparty <mailto:Toasters [at] teaparty>
> http://www.teaparty.net/mailman/listinfo/toasters
>
>
>
>
> --
> ---
> Gustatus Similis Pullus


fredgrieco at yahoo

Dec 6, 2011, 6:32 AM

Post #4 of 7 (1364 views)
Permalink
Re: limit i/o for lun or volume [In reply to]

Do you mean that there is a performance issue with that particular VM, or with other systems on the filer that are caused by I/O on that VM?

You should be able to use flex share to set a lower priority for a "cache-hogging" and reduce its impact on the other volumes.  On the command line, run "priority on."  Then run "priority set volume $volumename level=Low" for the problem volume.  This will give less priority to that volume in the write cache, free up the cache for other things, and (hopefully) reduce latency on those other volumes.


In your case, you'll need to have that RDM in its own volume.



________________________________
From: Alon Zeltser <alonz [at] emet>
To: toasters [at] teaparty
Sent: Tuesday, December 6, 2011 6:00 AM
Subject: limit i/o for lun or volume

Hi all
i have a situation of a virtual machine running on esx 4.0 over iscsi
this machine  (while the user is running his stuff)  is using a lot of
i/o on a very small netapp controller (2020) with only 5 data disks aggr
and getting very bad latency and eventually getting a lun reset errors
and the machine freeze for a few minutes
i have increased the timeouts from the windows side but it doesn't seems
to help
this application don't need so much i/o and can be run a 5.4k disk on an
old laptop but given the resources of netapp it uses them to the fullest
it is important to mention that this application is running over MSSQL
db and the i/o problem goes to rdm lun
my question: is there a way to limit the i/o of this machine from either
netapp side /vmware side / windows side / network side
i'm aweare of vmware storage i/o control but this is only supported in
vsphere 4.1 and an upgrade is not an options right now
i'm also aware of flexshare from the netapp side but i don't think
giving low priority to this volume will help in this case
is there other way you can think of to limit the i/o goes from vmware
virtual machine through iscsi to an rdm lun or his hosting volume?

thank you

_______________________________________________
Toasters mailing list
Toasters [at] teaparty
http://www.teaparty.net/mailman/listinfo/toasters


alonz at emet

Dec 6, 2011, 6:43 AM

Post #5 of 7 (1365 views)
Permalink
Re: limit i/o for lun or volume [In reply to]

Thank you for replay but as I wrote im aware of flexshare but I don't think it will help since this system is not impacting other systems and not compeating with other vms on resources
Its only hurting itself it doing so much iops and getting such bad latency that after few minutes its doing lun reset and the vm freezes and the user procces exit abnormally
I'm trying to limit the iops from this system so it wont load the Netapp so much and get better latency I don't care if it will take much longer to run

Thanks again

Fred Grieco <fredgrieco [at] yahoo> wrote:

Do you mean that there is a performance issue with that particular VM, or with other systems on the filer that are caused by I/O on that VM?

You should be able to use flex share to set a lower priority for a "cache-hogging" and reduce its impact on the other volumes.  On the command line, run "priority on."  Then run "priority set volume $volumename level=Low" for the problem volume.  This will give less priority to that volume in the write cache, free up the cache for other things, and (hopefully) reduce latency on those other volumes.


In your case, you'll need to have that RDM in its own volume.



________________________________
From: Alon Zeltser <alonz [at] emet>
To: toasters [at] teaparty
Sent: Tuesday, December 6, 2011 6:00 AM
Subject: limit i/o for lun or volume

Hi all
i have a situation of a virtual machine running on esx 4.0 over iscsi
this machine  (while the user is running his stuff)  is using a lot of
i/o on a very small netapp controller (2020) with only 5 data disks aggr
and getting very bad latency and eventually getting a lun reset errors
and the machine freeze for a few minutes
i have increased the timeouts from the windows side but it doesn't seems
to help
this application don't need so much i/o and can be run a 5.4k disk on an
old laptop but given the resources of netapp it uses them to the fullest
it is important to mention that this application is running over MSSQL
db and the i/o problem goes to rdm lun
my question: is there a way to limit the i/o of this machine from either
netapp side /vmware side / windows side / network side
i'm aweare of vmware storage i/o control but this is only supported in
vsphere 4.1 and an upgrade is not an options right now
i'm also aware of flexshare from the netapp side but i don't think
giving low priority to this volume will help in this case
is there other way you can think of to limit the i/o goes from vmware
virtual machine through iscsi to an rdm lun or his hosting volume?

thank you

_______________________________________________
Toasters mailing list
Toasters [at] teaparty
http://www.teaparty.net/mailman/listinfo/toasters

_______________________________________________
Toasters mailing list
Toasters [at] teaparty
http://www.teaparty.net/mailman/listinfo/toasters


fredgrieco at yahoo

Dec 6, 2011, 7:01 AM

Post #6 of 7 (1364 views)
Permalink
Re: limit i/o for lun or volume [In reply to]

Have you run sysstat on the command line of the filer?  (sysstat -x 1)  Are you getting a lot of "b" deferred back to back type CPs?  My understanding is that the filer starts rejecting new scsi requests when there are back-to-back deferred CPs (because there's no where to put them in memory locally).  That's when you get resets and errors on the hosts.


Flexshare will limit the IOs to that volume and will reduce the deferred CPs, which may prevent the resets and errors.  The host itself may not like that though-- there may be a timeout or some other tuning in iscsi that will help with this.



________________________________
From: Alon Zeltser <alonz [at] emet>
To: Fred Grieco <fredgrieco [at] yahoo>; "toasters [at] teaparty" <toasters [at] teaparty>
Sent: Tuesday, December 6, 2011 9:43 AM
Subject: Re: limit i/o for lun or volume

Thank you for replay but as I wrote im aware of flexshare but I don't think it will help since this system is not impacting other systems and not compeating with other vms on resources
Its only hurting itself it doing so much iops and getting such bad latency that after few minutes its doing lun reset and the vm freezes and the user procces exit abnormally
I'm trying to limit the iops from this system so it wont load the Netapp so much and get better latency I don't care if it will take much longer to run

Thanks again

Fred Grieco <fredgrieco [at] yahoo> wrote:

Do you mean that there is a performance issue with that particular VM, or with other systems on the filer that are caused by I/O on that VM?

You should be able to use flex share to set a lower priority for a "cache-hogging" and reduce its impact on the other volumes.  On the
command line, run "priority on."  Then run "priority set volume $volumename level=Low" for the problem volume.  This will give less priority to that volume in the write cache, free up the cache for other things, and (hopefully) reduce latency on those other volumes.


In your case, you'll need to have that RDM in its own volume.



________________________________
From: Alon Zeltser <alonz [at] emet>
To: toasters [at] teaparty
Sent: Tuesday, December 6, 2011 6:00 AM
Subject: limit i/o for lun or volume

Hi all
i have a situation of a virtual machine running on esx 4.0 over iscsi
this machine  (while the user is running his stuff)  is using a lot of
i/o on a very small netapp controller (2020) with only 5 data disks
aggr
and getting very bad latency and eventually getting a lun reset errors
and the machine freeze for a few minutes
i have increased the timeouts from the windows side but it doesn't seems
to help
this application don't need so much i/o and can be run a 5.4k disk on an
old laptop but given the resources of netapp it uses them to the fullest
it is important to mention that this application is running over MSSQL
db and the i/o problem goes to rdm lun
my question: is there a way to limit the i/o of this machine from either
netapp side /vmware side / windows side / network side
i'm aweare of vmware storage i/o control but this is only supported in
vsphere 4.1 and an upgrade is not an options right now
i'm also aware of flexshare from the netapp side but i don't think
giving low priority to this volume will help in this case
is there other way you can think of to limit the i/o goes from vmware
virtual
machine through iscsi to an rdm lun or his hosting volume?

thank you

_______________________________________________
Toasters mailing list
Toasters [at] teaparty
http://www.teaparty.net/mailman/listinfo/toasters


alonz at emet

Dec 6, 2011, 7:12 AM

Post #7 of 7 (1387 views)
Permalink
Re: limit i/o for lun or volume [In reply to]

I run sysstat perfstat cmpg and other performance tools the problem here is not in the writes but in the reads
while the user run his stuff I get 1500 iops most of them read which is kind of awsome for only 5 data disks and 2 parity disks so back to back cp is not the issue here
But I will try to set the priority for this volume to low maybe it will help
Any other ideas on how to limit io ?

Fred Grieco <fredgrieco [at] yahoo> wrote:

Have you run sysstat on the command line of the filer?  (sysstat -x 1)  Are you getting a lot of "b" deferred back to back type CPs?  My understanding is that the filer starts rejecting new scsi requests when there are back-to-back deferred CPs (because there's no where to put them in memory locally).  That's when you get resets and errors on the hosts.


Flexshare will limit the IOs to that volume and will reduce the deferred CPs, which may prevent the resets and errors.  The host itself may not like that though-- there may be a timeout or some other tuning in iscsi that will help with this.



________________________________
From: Alon Zeltser <alonz [at] emet>
To: Fred Grieco <fredgrieco [at] yahoo>; "toasters [at] teaparty" <toasters [at] teaparty>
Sent: Tuesday, December 6, 2011 9:43 AM
Subject: Re: limit i/o for lun or volume

Thank you for replay but as I wrote im aware of flexshare but I don't think it will help since this system is not impacting other systems and not compeating with other vms on resources
Its only hurting itself it doing so much iops and getting such bad latency that after few minutes its doing lun reset and the vm freezes and the user procces exit abnormally
I'm trying to limit the iops from this system so it wont load the Netapp so much and get better latency I don't care if it will take much longer to run

Thanks again

Fred Grieco <fredgrieco [at] yahoo> wrote:

Do you mean that there is a performance issue with that particular VM, or with other systems on the filer that are caused by I/O on that VM?

You should be able to use flex share to set a lower priority for a "cache-hogging" and reduce its impact on the other volumes.  On the
command line, run "priority on."  Then run "priority set volume $volumename level=Low" for the problem volume.  This will give less priority to that volume in the write cache, free up the cache for other things, and (hopefully) reduce latency on those other volumes.


In your case, you'll need to have that RDM in its own volume.



________________________________
From: Alon Zeltser <alonz [at] emet>
To: toasters [at] teaparty
Sent: Tuesday, December 6, 2011 6:00 AM
Subject: limit i/o for lun or volume

Hi all
i have a situation of a virtual machine running on esx 4.0 over iscsi
this machine  (while the user is running his stuff)  is using a lot of
i/o on a very small netapp controller (2020) with only 5 data disks
aggr
and getting very bad latency and eventually getting a lun reset errors
and the machine freeze for a few minutes
i have increased the timeouts from the windows side but it doesn't seems
to help
this application don't need so much i/o and can be run a 5.4k disk on an
old laptop but given the resources of netapp it uses them to the fullest
it is important to mention that this application is running over MSSQL
db and the i/o problem goes to rdm lun
my question: is there a way to limit the i/o of this machine from either
netapp side /vmware side / windows side / network side
i'm aweare of vmware storage i/o control but this is only supported in
vsphere 4.1 and an upgrade is not an options right now
i'm also aware of flexshare from the netapp side but i don't think
giving low priority to this volume will help in this case
is there other way you can think of to limit the i/o goes from vmware
virtual
machine through iscsi to an rdm lun or his hosting volume?

thank you

_______________________________________________
Toasters mailing list
Toasters [at] teaparty
http://www.teaparty.net/mailman/listinfo/toasters

_______________________________________________
Toasters mailing list
Toasters [at] teaparty
http://www.teaparty.net/mailman/listinfo/toasters

Netapp toasters RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.