fcocquyt at stanford
Apr 26, 2012, 11:46 PM
Post #8 of 8
Peta - we were dealing with this very issue (unexplained latency spikes Netapp blamed on VM misalignment)
back in 2010 - I wrote up how we deconstructed the IOPs after many wasted perfstat iterations
to solve it pretty much on our own:
It was maddening to me back in 2010 how netapp support could blockade support cases with a
blanket "must align VMs first" without a real quantification of the impact of misalignment - see
We ended up taking the downtime back then to align all VMs
But now, I would be one to encourage your making the leap to 8.x - we are on 8.1GA and we are not looking back.
The data motion of vFilers is allowing us to upgrade clusters with no downtime
They have me almost believing in cluster mode for scale out...
On Apr 25, 2012, at 11:40 PM, Peta Thames wrote:
> Hi Jack,
> You're right, and I should have mentioned it before. Large numbers of
> the VMDKs are misaligned. I'd estimate about 33%, but I don't know
> exactly how many as the shiny new VSC scanner got stuck halfway
> through the scan I ran, leaving several VMs in a "being scanned"
> state. I have a case open with Netapp to find out how to get those
> VMs out of that state so I can a) continue the scan b) schedule fixing
> the misaligned luns.
> Not all the luns that have large latency spikes are misaligned
> however. Mind you, by the same token, not all of them are fragmented,
> although so far (I'm still getting through measuring them all) there's
> definitely a strong correlation.
> I also have to admit that I read the scale wrong in perf advisor, and
> the numbers I'm seeing are in microseconds, not milliseconds. Still
> way more than the 10ms I would like, but an order of magnitude better!
> On 26 April 2012 15:52, Jack Lyons <jack1729 [at] gmail> wrote:
>> Have you checked the alignment of the VMDK's?
>> Sent from my Verizon Wireless BlackBerry
>> -----Original Message-----
>> From: Peta Thames <petathames [at] gmail>
>> Sender: toasters-bounces [at] teaparty
>> Date: Thu, 26 Apr 2012 14:49:43
>> To: <Toasters [at] teaparty>
>> Subject: Reallocation and VMware
>> Hi all,
>> I'd like to pick your collective brains about your experiences with
>> reallocate, specifically when reallocating luns under VMware.
>> For background, we're running ONTAP 8.0.1 on a 3170 that's over three
>> years old. I've been going through measuring reallocation, and most
>> of the volumes are over 3. We have no snapshots, and only a
>> relatively small number of volumes are de-duplicated. All our volumes
>> and luns are thin-provisioned, and no aggregate is more than 76% full
>> (most are ~65%). We regularly have huge latency spikes (worst I've
>> seen so far is 5000000ms, and there are far too many to even track
>> over 50000ms daily), and on one filer head, but not its partner, I
>> regularly see disk utilisation go to 100% or more. I'm hoping
>> reallocate will help here.
>> I have a brief note from a NetApp support person who says "It’s very
>> important that you complete the reallocation in the following order:
>> 1:OS 2:LUN 3: Volume".
>> I have two questions about this:
>> - is it absolutely necessary to defrag the OS before you reallocate
>> the lun? I'm sure I've run reallocate without defraging the OS and
>> still seen performance improvements. I'm also assuming that this is
>> only relevant to Windows VMs, not Linux (in our case, Red Hat/CentOS)
>> - if you only have one lun per volume, do you still need to run
>> reallocate on both the lun and the volume? If only one, which is
>> All advice appreciated.
>> Toasters mailing list
>> Toasters [at] teaparty
> Toasters mailing list
> Toasters [at] teaparty