tmerrill at mathworks
Nov 13, 2008, 8:51 AM
Post #2 of 2
(10304 views)
Permalink
|
On Tue, 30 Sep 2008, Todd C. Merrill wrote:
> OK, recent migration from 980's to 6070's via volume snapmirror...
>
> Volume deswizzling has been going on for over a week, pounding
> the disks at 90-100% utilization (via `sysstat` and `statit`).
>
> WHEN WILL IT END?
>
> How does one read the `wafl scan status` line for deswizzling?
> What are the stages, in what order do they run (if any), do
> they wrap around/interleave/etc., do they restart, at what
> priority do they run, etc.
>
> The performance degradation is noticed by our users and their
> patience is growing thin with us (toaster admins).
A follow-up... I received a handful of very useful private replies
to this thread. A big thanks to those who replied!
The main point from most of the replies was to stop snapshot
activity and allow the deswizzling to complete. We are running
7.2.4, which doesn't have the apparent deswizzle re-start capabilities
in later versions.
We stopped all regular snapshots, all snapmirrors, and all snapvaults
to the volumes in question, and let the deswizzling run from Friday
night through the weekend. By Tue morning, it wasn't finished. :(
So, the next weekend, we stopped all snapshot activity *and* deleted
all but two snapshots on the affected volumes (we had 30-40 prior).
That was barely enough...the deswizzle scans finished by 7 AM the
following Tuesday.
These weren't extraordinary volumes...just user data (home
directories) with volume sizes of about 500 GB with a couple
million files.
So, we are back to normal operating load levels now. Yea!
(I think a knowledge base article on NOW is in order for this
consequence of running volume snapmirrors. It took us by surprise, and
it took the collective knowledge of this mailing list to nail down the
answer.)
Until next time...
The MathWorks, Inc. 508-647-7000 x7792
3 Apple Hill Drive, Natick, MA 01760-2098 508-647-7001 FAX
tmerrill [at] mathworks http://www.mathworks.com
---
|
