Mailing List Archive: MythTV: Users

Seagate Lockups (was libraries over 8TB)

Index | Next | Previous | View Threaded

jra at baylink

Jul 7, 2012, 10:28 AM

Post #1 of 10 (621 views)
Permalink

Seagate Lockups (was libraries over 8TB)

----- Original Message -----
> From: "Alexander Fisher" <alex [at] alexfisher>

> I've also had some odd behavior from one of my non-enterprise Seagate
> drives. It's firmware locked up the other day, the drive not coming
> back without a power cycle.

I believe you had the same problem that I've had on 5 different Seagates
over the last 4 years:

There's a black hole somewhere on the drive. If you never touch it, the drive
works just fine. But the second you even read one of those sectors, the
drive adapter falls off line, and Linux can't touch it anymore, and it doesn't
recover until you power the drive off. Once you do, everything's fine until
you hit that spot again.

Sound familiar?

Cheers,
-- jra
--
Jay R. Ashworth Baylink jra [at] baylink
Designer The Things I Think RFC 2100
Ashworth & Associates http://baylink.pitas.com 2000 Land Rover DII
St Petersburg FL USA http://photo.imageinc.us +1 727 647 1274
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users

travis at tabbal

Jul 7, 2012, 10:33 AM

Post #2 of 10 (590 views)
Permalink

Re: Seagate Lockups (was libraries over 8TB) [In reply to]

On Sat, Jul 7, 2012 at 11:28 AM, Jay Ashworth <jra [at] baylink> wrote:

> ----- Original Message -----
> > From: "Alexander Fisher" <alex [at] alexfisher>
>
> > I've also had some odd behavior from one of my non-enterprise Seagate
> > drives. It's firmware locked up the other day, the drive not coming
> > back without a power cycle.
>
> I believe you had the same problem that I've had on 5 different Seagates
> over the last 4 years:
>
> There's a black hole somewhere on the drive. If you never touch it, the
> drive
> works just fine. But the second you even read one of those sectors, the
> drive adapter falls off line, and Linux can't touch it anymore, and it
> doesn't
> recover until you power the drive off. Once you do, everything's fine
> until
> you hit that spot again.
>

My test for new drives is now to put them in a ZFS mirror, and fill them to
100% capacity, then to a zpool scrub on them to make sure they are holding
the data properly. I've found a few bad drives this way that wouldn't have
shown up with normal testing unless I happened to hit that bad block. I
usually repeat the test a few times, by destroying the pool, then starting
over. It takes a long time, but it beats finding out the hard way months
later.

alex at alexfisher

Jul 7, 2012, 10:53 AM

Post #3 of 10 (594 views)
Permalink

Re: Seagate Lockups (was libraries over 8TB) [In reply to]

On 7 July 2012 18:28, Jay Ashworth <jra [at] baylink> wrote:
> ----- Original Message -----
>> From: "Alexander Fisher" <alex [at] alexfisher>
>
>> I've also had some odd behavior from one of my non-enterprise Seagate
>> drives. It's firmware locked up the other day, the drive not coming
>> back without a power cycle.
>
> I believe you had the same problem that I've had on 5 different Seagates
> over the last 4 years:
>
> There's a black hole somewhere on the drive. If you never touch it, the drive
> works just fine. But the second you even read one of those sectors, the
> drive adapter falls off line, and Linux can't touch it anymore, and it doesn't
> recover until you power the drive off. Once you do, everything's fine until
> you hit that spot again.
>
> Sound familiar?

My issue was more like...
http://www.readynas.com/forum/viewtopic.php?f=20&t=62740

I was running a defrag (xfs_fsr) when the bug triggered for me.

I contacted seagate support, hoping to get hold of firmware that fixes
the issue.
No such luck. I was told I had the latest firmware (even though it
didn't match the firmware you can download from their website).
They said I could RMA the drive, but I'm not sure what good that would
do. Wouldn't they just give me another one with exactly the same
issue?

Fortunately I've only got one of these drives. It's also one of the
reasons I'm running a 3 disk RAID 5 with a hot spare instead of a 4
disk RAID 6. I fully expect the seagate to get booted from the array
before too long at which point the WD spare can take over. I guess
I'll then use the seagate in a non raid environment, probably as some
backup storage. Seagate don't release details of what goes into
firmware updates, so it's going to be hard to know if the problem is
fixed.

It's really quite poor especially as in the datasheet they recommended
the drive for use in NASes.

Kind Regards,
Alex
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users

linux at thehobsons

Jul 7, 2012, 11:05 AM

Post #4 of 10 (587 views)
Permalink

Re: Seagate Lockups (was libraries over 8TB) [In reply to]

Jay Ashworth wrote:

>There's a black hole somewhere on the drive. If you never touch it, the drive
>works just fine. But the second you even read one of those sectors, the
>drive adapter falls off line, and Linux can't touch it anymore, and it doesn't
>recover until you power the drive off. Once you do, everything's fine until
>you hit that spot again.

I've got one at work like that. I thought I'd try recovering it with
dd rescue - partly for the experience. Didn't take long to give up on
it !

Also, I'm not 100% certain, but I think it also managed to lock up
the host controller as well. I found that pulling the power cable to
the drive and reconnecting it didn't bring it back to life - while
power cycling the whole machine did. So I put it on an external USB
adapter so I could power cycle the drive & controller without power
cycling the machine as well.

Enough to put you off a Seagate drive. Any half decent firmware would
report an error and carry on - leaving the OS to decide how to deal
with it.

--
Simon Hobson

Visit http://www.magpiesnestpublishing.co.uk/ for books by acclaimed
author Gladys Hobson. Novels - poetry - short stories - ideal as
Christmas stocking fillers. Some available as e-books.
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users

dl-mythtv at catspoiler

Jul 8, 2012, 3:11 PM

Post #5 of 10 (577 views)
Permalink

Re: Seagate Lockups (was libraries over 8TB) [In reply to]

On 7 Jul, Travis Tabbal wrote:

> My test for new drives is now to put them in a ZFS mirror, and fill them to
> 100% capacity, then to a zpool scrub on them to make sure they are holding
> the data properly. I've found a few bad drives this way that wouldn't have
> shown up with normal testing unless I happened to hit that bad block. I
> usually repeat the test a few times, by destroying the pool, then starting
> over. It takes a long time, but it beats finding out the hard way months
> later.

I've got a drive burnin machine that I use to test all of my new drives.
I boot it using the Ultimate Boot CD and fire up Darik's Boot and Nuke,
which I configure to do four passes of random data overwrite, with
verification of all passes. This can take several days to run on a
large drive. This seems to catch all of the early drive failures,
usually within the first pass or two. So far I haven't had any drives
that pass this test fail within their next couple of years of runtime.

_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users

acstadt at stadt

Jul 9, 2012, 4:23 PM

Post #6 of 10 (551 views)
Permalink

Re: Seagate Lockups (was libraries over 8TB) [In reply to]

On 07/07/2012 1:28 PM, Jay Ashworth wrote:
> ----- Original Message -----
>> From: "Alexander Fisher" <alex [at] alexfisher>
>> I've also had some odd behavior from one of my non-enterprise Seagate
>> drives. It's firmware locked up the other day, the drive not coming
>> back without a power cycle.
> I believe you had the same problem that I've had on 5 different Seagates
> over the last 4 years:
>
> There's a black hole somewhere on the drive. If you never touch it, the drive
> works just fine. But the second you even read one of those sectors, the
> drive adapter falls off line, and Linux can't touch it anymore, and it doesn't
> recover until you power the drive off. Once you do, everything's fine until
> you hit that spot again.
>
> Sound familiar?
>
> Cheers,
> -- jra
Funny thing is I had exactly this problem with a Seagate drive this
weekend, its not in one of my arrays, just a plain old disk in a myth
storage group, so I'll just copy the data off it and retire it from
active use (probably through it in a test build box or something).

Just thought it was odd that if happened in the middle of this
discussion. Have to make a note of that zfs mirror trick for future
drives though.

_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users

raymond at wagnerrp

Jul 9, 2012, 5:10 PM

Post #7 of 10 (544 views)
Permalink

Re: Seagate Lockups (was libraries over 8TB) [In reply to]

On 7/7/2012 13:33, Travis Tabbal wrote:
>
>
> On Sat, Jul 7, 2012 at 11:28 AM, Jay Ashworth <jra [at] baylink
> <mailto:jra [at] baylink>> wrote:
>
> ----- Original Message -----
> > From: "Alexander Fisher" <alex [at] alexfisher
> <mailto:alex [at] alexfisher>>
>
> > I've also had some odd behavior from one of my non-enterprise Seagate
> > drives. It's firmware locked up the other day, the drive not coming
> > back without a power cycle.
>
> I believe you had the same problem that I've had on 5 different Seagates
> over the last 4 years:
>
> There's a black hole somewhere on the drive. If you never touch it,
> the drive
> works just fine. But the second you even read one of those sectors, the
> drive adapter falls off line, and Linux can't touch it anymore, and
> it doesn't
> recover until you power the drive off. Once you do, everything's
> fine until
> you hit that spot again.
>
>
>
> My test for new drives is now to put them in a ZFS mirror, and fill them
> to 100% capacity, then to a zpool scrub on them to make sure they are
> holding the data properly.

No need to use a mirror, just do single drives. ZFS checksums
everything and will detect an error on a scrub even if you don't have
another copy of the data to compare against.
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users

travis at tabbal

Jul 10, 2012, 7:38 AM

Post #8 of 10 (541 views)
Permalink

Re: Seagate Lockups (was libraries over 8TB) [In reply to]

On Mon, Jul 9, 2012 at 6:10 PM, Raymond Wagner <raymond [at] wagnerrp> wrote:

>
> No need to use a mirror, just do single drives. ZFS checksums everything
> and will detect an error on a scrub even if you don't have another copy of
> the data to compare against.
>

True. I use mirrors as I'm usually testing more than one drive. It makes it
faster to fill them up, I only need to copy 1 drive worth of data. If
you're only using one drive, it will work fine as well. The other test
utility mentioned on the Ultimate Boot CD also sounds interesting,
particularly if it can do multiple drives in parallel like ZFS can. It
stinks that you can't trust even brand new hardware. HDDs have been
particularly bad the past few years, even before the flood.

dl-mythtv at catspoiler

Jul 10, 2012, 7:55 AM

Post #9 of 10 (538 views)
Permalink

Re: Seagate Lockups (was libraries over 8TB) [In reply to]

On 10 Jul, Travis Tabbal wrote:

> True. I use mirrors as I'm usually testing more than one drive. It makes it
> faster to fill them up, I only need to copy 1 drive worth of data. If
> you're only using one drive, it will work fine as well. The other test
> utility mentioned on the Ultimate Boot CD also sounds interesting,
> particularly if it can do multiple drives in parallel like ZFS can. It
> stinks that you can't trust even brand new hardware. HDDs have been
> particularly bad the past few years, even before the flood.

Darik's Boot and Nuke does handle multiple drives in parallel, though
that slows it down a bit. It also shows its progress and an estimate of
the completion time, which isn't totally accurate because of the
non-uniform data rate between the drive's inner and outer cylinders.

_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users

mythlist at assursys

Jul 14, 2012, 12:31 AM

Post #10 of 10 (509 views)
Permalink

Re: Seagate Lockups (was libraries over 8TB) [In reply to]

On Sat, 7 Jul 2012, Jay Ashworth wrote:

> ----- Original Message -----
>> From: "Alexander Fisher" <alex [at] alexfisher>
>
>> I've also had some odd behavior from one of my non-enterprise Seagate
>> drives. It's firmware locked up the other day, the drive not coming
>> back without a power cycle.
>
> I believe you had the same problem that I've had on 5 different Seagates
> over the last 4 years:
>
> There's a black hole somewhere on the drive. If you never touch it, the drive
> works just fine. But the second you even read one of those sectors, the
> drive adapter falls off line, and Linux can't touch it anymore, and it doesn't
> recover until you power the drive off. Once you do, everything's fine until
> you hit that spot again.
>
> Sound familiar?

Probably the drive has developed some unreadable sector(s). Have a look at
the output of 'smartctl -a /dev/sdX' (specifically "Current Pending Sector"
and "Offline Uncorrectable" attributes) and the kernel error log, which
might give you some clue as to where the problem is.

Once you know where the problematic sectors are, find what's occupying them,
delete it (it's unreadable now, anyway; you've got a backup, right?!), then
*write* to those sectors. If the write fails, the drive is toast. If it
succeeds, the drive will probably be good as new (and the "Reallocated
Sector C[oun]t" attribute will be higher, if they were hard - rather than
soft - errors).

> Cheers,
> -- jra

Best Regards,
Alex
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads