Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: MythTV: Users

sluggish system, potential HD failure, and how best to get back up and runnin

 

 

MythTV users RSS feed   Index | Next | Previous | View Threaded


gaberubin at gmail

Mar 4, 2014, 8:01 AM

Post #1 of 15 (461 views)
Permalink
sluggish system, potential HD failure, and how best to get back up and runnin

My system was extremely non-responsive last night. I saw these messages
spammed to the console:
systemd-journald failed to write entry ignoring read-only

And when I tried to start up certain services like httpd, I saw these
messages:
Warning: Unit file of httpd.service changed on disk, 'systemctl --system
daemon-reload' recommended.

From some basic google searching, it appears these messages are symptoms of
a hard drive going bad. This is not that surprising that this HD would go
bad because it is fairly old.

So I am thinking of replacing it (unless anyone believes this indicates
something that is not HD failing). This HD contains the OS (Fedora) and
database and all non-media files (I have all my recordings on a newer 2TB
drive segregating the DB off that). I had two questions:

1) Is getting a SSD worth it? Will it speed up my performance? Is there
an issue with storing the DB and having the multiple writes that occur
daily on the DB be on a SSD?


2) Assuming the dying drive is not dead yet, what is the easiest way to
mirror the drive over to a new drive so I don't have to do a full
re-install and set up (and remember) all my configurations. Just setting
up lirc can take a few hours, so I would prefer to avoid doing a reinstall
of myth from scratch.

Thanks in advance


hikavdh at gmail

Mar 4, 2014, 8:08 AM

Post #2 of 15 (452 views)
Permalink
Re: sluggish system, potential HD failure, and how best to get back up and runnin [In reply to]

Hoi Gabe,

Tuesday, March 4, 2014, 5:01:58 PM, you wrote:

> My system was extremely non-responsive last night.  I saw these messages spammed to the console:
> systemd-journald failed to write entry ignoring read-only


> And when I tried to start up certain services like httpd, I saw these messages:
> Warning: Unit file of httpd.service changed on disk, 'systemctl
> --system daemon-reload' recommended.


> From some basic google searching, it appears these messages are
> symptoms of a hard drive going bad.  This is not that surprising
> that this HD would go bad because it is fairly old.


> So I am thinking of replacing it (unless anyone believes this
> indicates something that is not HD failing).  This HD contains the
> OS (Fedora) and database and all non-media files (I have all my
> recordings on a newer 2TB drive segregating the DB off that).  I had two questions:


> 1) Is getting a SSD worth it?  Will it speed up my performance?  Is
> there an issue with storing the DB and having the multiple writes
> that occur daily on the DB be on a SSD?



> 2) Assuming the dying drive is not dead yet, what is the easiest
> way to mirror the drive over to a new drive so I don't have to do a
> full re-install and set up (and remember) all my configurations. 
> Just setting up lirc can take a few hours, so I would prefer to
> avoid doing a reinstall of myth from scratch.

> Thanks in advance


Download the systemrescue image from http://www.sysresccd.org/SystemRescueCd_Homepage
It contains all tools you need. There are also instructions to put it
on a usb stick




Tot mails,
Hika mailto:hikavdh [at] gmail

"Zonder hoop kun je niet leven
Zonder leven is er geen hoop
Het eeuwige dilemma
Zeker als je hoop moet vernietigen om te kunnen overleven!"

De lerende Mens
--

_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org


wdormann at gmail

Mar 4, 2014, 8:27 AM

Post #3 of 15 (445 views)
Permalink
Re: sluggish system, potential HD failure, and how best to get back up and runnin [In reply to]

On 3/4/14, 11:01 AM, Gabe Rubin wrote:
> 2) Assuming the dying drive is not dead yet, what is the easiest way to
> mirror the drive over to a new drive so I don't have to do a full
> re-install and set up (and remember) all my configurations. Just
> setting up lirc can take a few hours, so I would prefer to avoid doing a
> reinstall of myth from scratch.

A few tips to prevent this sort of situation in the future. (tl;dr:
always have backups for important things; always monitor system health)

1) Make sure that your mythtv database is backed up regularly. Mine is
set to run daily via cron. The database is the most important part of
MythTV.
http://www.mythtv.org/wiki/Database_Backup_and_Restore#Database_Backup

2) Back up your entire system at least once (to another
device/drive/etc), and then on demand before making any major changes.
An OS restore plus a database restore from 1) will mean that you'll
never lose more than 24 hours. My recordings are not on the local FS, so
those aren't backed up. Here's my script. Depending on your partition
layout and filesystems used, you'll want to modify to suit your system.

---- begin backup.sh ----

#!/bin/sh

backupdir=[PATH_TO_BACKUP_DIRECTORY]

mkdir -p $backupdir
echo Copying documentation...
cp mbr.txt $backupdir
cp xfsrestore.txt $backupdir
echo Backing up MBR...
dd if=/dev/sda of=$backupdir/mythbuntu-mbr.bin bs=512 count=1
echo Dumping boot filesystem...
dump -0uf $backupdir/boot.dump /boot
echo Dumping root xfs filesystem...
xfsdump -e -l0 - / | gzip > $backupdir/mythbuntu.dgz

---- end backup.sh ------


I also occasionally burn a bootable DVD copy of the backup, padded with
dvdisaster ECC info, just to be extra paranoid:

---- begin burnbackup.sh ----

#!/bin/sh

backupdir=[PATH_TO_BACKUP_DIRECTORY]
cd /mnt/store/rescue
# Copy systemrescuecd iso
ionice -c 3 cp sysrec.iso recovery.iso
# Add mythbuntu backup files
ionice -c 3 growisofs -M recovery.iso -J -R $backupdir
# Create dvdisaster redundancy
ionice -c 3 dvdisaster -c -mRS02 -i recovery.iso
# Burn to disc
growisofs -dvd-compat -Z /dev/sr0=/mnt/store/rescue/recovery.iso

---- end burnbackup.sh ------


3) Use smartd from smartmontools to monitor your drive health.
Configure it to both do scheduled self-test scans and also email you in
the case of pending failure.
<https://help.ubuntu.com/community/Smartmontools>


You can also run smartctl from smartmontools now to confirm that your
drive is dying. e.g.:
sudo smartctl -a /dev/sda


As for recovering your data, here are the most important parts to get
off of your drive asap:
1) The database (see above)
2) /etc
3) /home

If you're able to back those up, you should be able to get your system
back to normal with a moderate amount of effort.


-WD



_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org


gaberubin at gmail

Mar 4, 2014, 8:47 AM

Post #4 of 15 (442 views)
Permalink
Re: sluggish system, potential HD failure, and how best to get back up and runnin [In reply to]

Thanks, these are all great tips and I will use them in the future. I
think I can get to the data on the OS disc, it is just slow as heck to boot
up from that and I believe it starts up in read only mode.

I do a DB backup every night. I am pretty sure I have the backup go to my
other drive, because that would be the smart thing to do, but I will have
to check. Obviously my first issue will be locating that backup and
putting it on a safe location.

I have wanted to do a backup of the full disc as a cron job for a while and
back it up to my windows box. I just did not know what the best way to do
that is, so the script will be implemented when I get my system back up and
running.

The key questions I have are, is a SSD a good option considering the DB
gets many writess a day and I thought SSDs fail after a certain amount of
writes. Even if that is not an issue, is the cost worth it? I don't
neccesarily want to replace this drive with a HDD because I only need about
50-100 gigs for the system drive (maybe more, I am forgetting how large the
db can get).

Once I replace the drive, what is the easiest way to mirror the old drive
onto the new one and any gotchas (for example, I think I can't just my old
fstab file because it has entries specific to the actual drives and not
just their labels). I realize there will be some work migrating, but
hoping not a ton.

Thanks again!


hikavdh at gmail

Mar 4, 2014, 9:32 AM

Post #5 of 15 (441 views)
Permalink
Re: sluggish system, potential HD failure, and how best to get back up and runnin [In reply to]

Hoi Gabe,

Tuesday, March 4, 2014, 5:47:02 PM, you wrote:

> Thanks, these are all great tips and I will use them in the future.
> I think I can get to the data on the OS disc, it is just slow as
> heck to boot up from that and I believe it starts up in read only mode.


> I do a DB backup every night.  I am pretty sure I have the backup
> go to my other drive, because that would be the smart thing to do,
> but I will have to check.  Obviously my first issue will be locating
> that backup and putting it on a safe location.


> I have wanted to do a backup of the full disc as a cron job for a
> while and back it up to my windows box.  I just did not know what
> the best way to do that is, so the script will be implemented when I
> get my system back up and running.

> The key questions I have are, is a SSD a good option considering
> the DB gets many writess a day and I thought SSDs fail after a
> certain amount of writes.  Even if that is not an issue, is the cost
> worth it?  I don't neccesarily want to replace this drive with a HDD
> because I only need about 50-100 gigs for the system drive (maybe
> more, I am forgetting how large the db can get).

> Once I replace the drive, what is the easiest way to mirror the old
> drive onto the new one and any gotchas (for example, I think I can't
> just my old fstab file because it has entries specific to the actual
> drives and not just their labels).  I realize there will be some
> work migrating, but hoping not a ton.

> Thanks again!


I don't know about the average livecycle of HDD drives. But if you
boot your system from cd or usbstick you can use cp -rpd to copy the
partition to a new one r for subdirectories p for preserve ownwership
etc. and d for not following simlinks. You can add -v for verbose.
After that you have to chroot in to your new disk: chroot /mnt/newdisk
/bin/bash and configure your boorloader.

Tot mails,
Hika mailto:hikavdh [at] gmail

"Zonder hoop kun je niet leven
Zonder leven is er geen hoop
Het eeuwige dilemma
Zeker als je hoop moet vernietigen om te kunnen overleven!"

De lerende Mens
--

_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org


hikavdh at gmail

Mar 4, 2014, 10:48 AM

Post #6 of 15 (435 views)
Permalink
Re: sluggish system, potential HD failure, and how best to get back up and runnin [In reply to]

Hoi Hika,

Tuesday, March 4, 2014, 6:32:25 PM, you wrote:

> Hoi Gabe,

> Tuesday, March 4, 2014, 5:47:02 PM, you wrote:

>> Thanks, these are all great tips and I will use them in the future.
>> I think I can get to the data on the OS disc, it is just slow as
>> heck to boot up from that and I believe it starts up in read only mode.


>> I do a DB backup every night.  I am pretty sure I have the backup
>> go to my other drive, because that would be the smart thing to do,
>> but I will have to check.  Obviously my first issue will be locating
>> that backup and putting it on a safe location.


>> I have wanted to do a backup of the full disc as a cron job for a
>> while and back it up to my windows box.  I just did not know what
>> the best way to do that is, so the script will be implemented when I
>> get my system back up and running.

>> The key questions I have are, is a SSD a good option considering
>> the DB gets many writess a day and I thought SSDs fail after a
>> certain amount of writes.  Even if that is not an issue, is the cost
>> worth it?  I don't neccesarily want to replace this drive with a HDD
>> because I only need about 50-100 gigs for the system drive (maybe
>> more, I am forgetting how large the db can get).

>> Once I replace the drive, what is the easiest way to mirror the old
>> drive onto the new one and any gotchas (for example, I think I can't
>> just my old fstab file because it has entries specific to the actual
>> drives and not just their labels).  I realize there will be some
>> work migrating, but hoping not a ton.

>> Thanks again!


> I don't know about the average livecycle of HDD drives. But if you
> boot your system from cd or usbstick you can use cp -rpd to copy the
> partition to a new one r for subdirectories p for preserve ownwership
> etc. and d for not following simlinks. You can add -v for verbose.
> After that you have to chroot in to your new disk: chroot /mnt/newdisk
> /bin/bash and configure your boorloader.

> Tot mails,
> Hika mailto:hikavdh [at] gmail

> "Zonder hoop kun je niet leven
> Zonder leven is er geen hoop
> Het eeuwige dilemma
> Zeker als je hoop moet vernietigen om te kunnen overleven!"

> De lerende Mens
> --


Oh, I forgot. before you chroot into your new disk you have to mount
your sys, proc and dev partition ito it
mount -t proc none /mnt/newdisk/proc
mount --rbind /sys /mnt/newdisk/sys
mount --rbind /dev /mnt/newdisk/dev

also if your boot partition is separate you have to copy and mount it.

Tot mails,
Hika mailto:hikavdh [at] gmail

"Zonder hoop kun je niet leven
Zonder leven is er geen hoop
Het eeuwige dilemma
Zeker als je hoop moet vernietigen om te kunnen overleven!"

De lerende Mens
--

_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org


monkeypet at gmail

Mar 4, 2014, 10:56 AM

Post #7 of 15 (439 views)
Permalink
Re: sluggish system, potential HD failure, and how best to get back up and runnin [In reply to]

On Tue, Mar 4, 2014 at 8:01 AM, Gabe Rubin <gaberubin [at] gmail> wrote:

> My system was extremely non-responsive last night. I saw these messages
> spammed to the console:
> systemd-journald failed to write entry ignoring read-only
>
> And when I tried to start up certain services like httpd, I saw these
> messages:
> Warning: Unit file of httpd.service changed on disk, 'systemctl --system
> daemon-reload' recommended.
>
> From some basic google searching, it appears these messages are symptoms
> of a hard drive going bad. This is not that surprising that this HD would
> go bad because it is fairly old.
>
> So I am thinking of replacing it (unless anyone believes this indicates
> something that is not HD failing). This HD contains the OS (Fedora) and
> database and all non-media files (I have all my recordings on a newer 2TB
> drive segregating the DB off that). I had two questions:
>
> 1) Is getting a SSD worth it? Will it speed up my performance? Is there
> an issue with storing the DB and having the multiple writes that occur
> daily on the DB be on a SSD?
>
>
> 2) Assuming the dying drive is not dead yet, what is the easiest way to
> mirror the drive over to a new drive so I don't have to do a full
> re-install and set up (and remember) all my configurations. Just setting
> up lirc can take a few hours, so I would prefer to avoid doing a reinstall
> of myth from scratch.
>

Last time my boot drive had bad sectors but still functioned, I booted from
a rescue image, used ddrescue to mirror the drive, then used fsck to check
the filesystem on the mirrored drive.



>
> Thanks in advance
>
>
> _______________________________________________
> mythtv-users mailing list
> mythtv-users [at] mythtv
> http://www.mythtv.org/mailman/listinfo/mythtv-users
> http://wiki.mythtv.org/Mailing_List_etiquette
> MythTV Forums: https://forum.mythtv.org
>
>


linux at thehobsons

Mar 4, 2014, 11:05 AM

Post #8 of 15 (436 views)
Permalink
Re: sluggish system, potential HD failure, and how best to get back up and runnin [In reply to]

On 4 Mar 2014, at 16:47, Gabe Rubin <gaberubin [at] gmail> wrote:

> The key questions I have are, is a SSD a good option considering the DB gets many writess a day and I thought SSDs fail after a certain amount of writes. Even if that is not an issue, is the cost worth it? I don't neccesarily want to replace this drive with a HDD because I only need about 50-100 gigs for the system drive (maybe more, I am forgetting how large the db can get).

You can do a **LOT** to reduce the drive I/O load by tuning the DB. It also makes a vast difference to performance - reducing reschedule times (which happens any time a recording starts/ends or you add/change/delete a rule).

But in any case, SSDs include wear levelling algorithms so they don't keep writing to the same place all the time. The emptier the drive, the more (and more easily) it can spread the wear around. The DB doesn't get all that big anyway, my root system is only 1.2G used, and /var (which includes the database) has only 2G used - on a system that's recorded 7022 episodes of 670 shows over a little over 5 years.

I added an SSD not long ago (when eBuyer has a good deal on a 64G one). That and some DB tuning made a big difference to the system - reschedules normally happen before the "upcoming recordings" screen refreshes, while before, it would refresh and not show the last change (eg I could click "Don't record" and the program would still show as to be recorded until the next page refresh).

_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org


gthompson20 at gmail

Mar 4, 2014, 11:49 AM

Post #9 of 15 (438 views)
Permalink
Re: sluggish system, potential HD failure, and how best to get back up and runnin [In reply to]

On Mar 4, 2014, at 2:05 PM, Simon Hobson <linux [at] thehobsons> wrote:

> On 4 Mar 2014, at 16:47, Gabe Rubin <gaberubin [at] gmail> wrote:
>
>> The key questions I have are, is a SSD a good option considering the DB gets many writess a day and I thought SSDs fail after a certain amount of writes. Even if that is not an issue, is the cost worth it? I don't neccesarily want to replace this drive with a HDD because I only need about 50-100 gigs for the system drive (maybe more, I am forgetting how large the db can get).
>
> You can do a **LOT** to reduce the drive I/O load by tuning the DB. It also makes a vast difference to performance - reducing reschedule times (which happens any time a recording starts/ends or you add/change/delete a rule).
>
> But in any case, SSDs include wear levelling algorithms so they don't keep writing to the same place all the time. The emptier the drive, the more (and more easily) it can spread the wear around. The DB doesn't get all that big anyway, my root system is only 1.2G used, and /var (which includes the database) has only 2G used - on a system that's recorded 7022 episodes of 670 shows over a little over 5 years.
>
> I added an SSD not long ago (when eBuyer has a good deal on a 64G one). That and some DB tuning made a big difference to the system - reschedules normally happen before the "upcoming recordings" screen refreshes, while before, it would refresh and not show the last change (eg I could click "Don't record" and the program would still show as to be recorded until the next page refresh).
>
> _______________________________________________
> mythtv-users mailing list
> mythtv-users [at] mythtv
> http://www.mythtv.org/mailman/listinfo/mythtv-users
> http://wiki.mythtv.org/Mailing_List_etiquette
> MythTV Forums: https://forum.mythtv.org


Mind sharing your DB Tuning Tips?

Greg
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org


linux at thehobsons

Mar 5, 2014, 3:20 AM

Post #10 of 15 (411 views)
Permalink
Re: sluggish system, potential HD failure, and how best to get back up and runnin [In reply to]

Greg Thompson <gthompson20 [at] gmail> wrote:

> Mind sharing your DB Tuning Tips?


With the disclaimer that I'm no expert on DB tuning ...

A good starting point is mysqltuner.pl, look at the results and twiddle settings.

However, even after reading the docs, some of the settings are a tad obscure. You also start getting into tradeoffs - some memory allocation are global, while some are per-connection. If you keep the allowable connections up then you find the total possible memory usage also goes up.

Extract from my current results :

[--] Data in MyISAM tables: 268M (Tables: 96)
[--] Data in InnoDB tables: 48K (Tables: 3)
Gives a total data size - ideally you want the memory allocation to be larger than this so all the tables can stay resident.

[!!] Total fragmented tables: 27
You can optimise the tables - but they'll fragment again sooner or later. Don't worry too much about this.

[--] Up for: 142d 21h 7m 22s (66M q [5.404 qps], 22K conn, TX: 244B, RX: 46B)
This is good, it means you get good statistics. Just after a restart you don't get meaningful numbers or recommendations.

[--] Total buffers: 930.0M global + 4.6M per thread (20 max threads)
[OK] Maximum possible memory usage: 1021.2M (51% of installed RAM)
Start to worry if this gets to "most of your RAM", although it's unlikely that every connection will use all it's possible memory.

[!!] Highest connection usage: 100% (20/20)
Hmm, looks like I need to tweak it up a bit, but then I've not seen any odd effects. Not sure when all the connections get used, because I've tried "making the system busy" and never got that high during testing.

[OK] Key buffer size / total MyISAM indexes: 384.0M/323.6M
IIRC this is one of the biggies - the buffer is big enough to keep all indexes in memory. That means any indexed operation (select, join, etc) shouldn't need to hit the disk.

[OK] Key buffer hit rate: 99.9% (1B cached / 1M reads)
And the result is a good figure

[OK] Sorts requiring temporary tables: 0% (288 temp sorts / 445K sorts)
That's good, and suggests some good use of indexes etc for sorts.

[!!] Joins performed without indexes: 94335
Ideally all joins would be via indexes, but largely that's out of the user's control as it's controlled by the programming (how it does joins) and the database schema (what indexes there are). I suspect "bad" custom criteria in recording rules could contribute to this figure.

[OK] Temporary tables created on disk: 7% (11K on disk / 152K total)
That's good, most of the temporary tables created were in memory.

[!!] Table cache hit rate: 0% (236 open / 361K opened)
Dunno about this one !

Then at the end if gives a list of recommendations :

-------- Recommendations -----------------------------------------------------
General recommendations:
Run OPTIMIZE TABLE to defragment tables for better performance
Enable the slow query log to troubleshoot bad queries
Reduce or eliminate persistent connections to reduce connection usage
Adjust your join queries to always utilize indexes
Increase table_cache gradually to avoid file descriptor limits
Variables to adjust:
max_connections (> 20)
wait_timeout (< 28800)
interactive_timeout (< 28800)
query_cache_size (> 24M)
join_buffer_size (> 2.0M, or always use indexes with joins)
table_cache (> 8192)

I suggest that you don't just blindly follow the instructions, but go and read the MySQL docs and see what each setting does first. If nothing else, it'll make you realise just what a complex area it is :-O
Also, after making a change, MySQL tuner won't give meaningful results for some aspects. So it can be a lengthy process of making adjustments, leaving the system to run for a few days, and then looking again. Where you can get immediate feedback is the total possible memory consumption and how your changes have affected it.


From memory, the defaults are "fairly conservative - with the result that a lot of disk I/O is created which is not necessary. On a system doing nothing but running a DB it wouldn't be as bad as OS caching would hide the problem, but with a Myth system, there's so much other I/O that you're guaranteed to flush DB data from the OS cache and so it's a lot more important to tune the DB itself to keep it's own cache. More memory is often a good investment - and if there isn't enough memory to tune out virtually all disk access, then adding more memory will probably be a better investment than an SSD (or faster drive).


Hope that's some help.


_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org


hikavdh at gmail

Mar 5, 2014, 3:11 PM

Post #11 of 15 (392 views)
Permalink
Re: sluggish system, potential HD failure, and how best to get back up and runnin [In reply to]

Hoi Simon,

Wednesday, March 5, 2014, 12:20:34 PM, you wrote:

> Greg Thompson <gthompson20 [at] gmail> wrote:

>> Mind sharing your DB Tuning Tips?


> With the disclaimer that I'm no expert on DB tuning ...

> A good starting point is mysqltuner.pl, look at the results and twiddle settings.

> However, even after reading the docs, some of the settings are a
> tad obscure. You also start getting into tradeoffs - some memory
> allocation are global, while some are per-connection. If you keep
> the allowable connections up then you find the total possible memory usage also goes up.

> Extract from my current results :

> [--] Data in MyISAM tables: 268M (Tables: 96)
> [--] Data in InnoDB tables: 48K (Tables: 3)
> Gives a total data size - ideally you want the memory allocation to
> be larger than this so all the tables can stay resident.

> [!!] Total fragmented tables: 27
> You can optimise the tables - but they'll fragment again sooner or
> later. Don't worry too much about this.

> [--] Up for: 142d 21h 7m 22s (66M q [5.404 qps], 22K conn, TX: 244B, RX: 46B)
> This is good, it means you get good statistics. Just after a
> restart you don't get meaningful numbers or recommendations.

> [--] Total buffers: 930.0M global + 4.6M per thread (20 max threads)
> [OK] Maximum possible memory usage: 1021.2M (51% of installed RAM)
> Start to worry if this gets to "most of your RAM", although it's
> unlikely that every connection will use all it's possible memory.

> [!!] Highest connection usage: 100% (20/20)
> Hmm, looks like I need to tweak it up a bit, but then I've not seen
> any odd effects. Not sure when all the connections get used, because
> I've tried "making the system busy" and never got that high during testing.

> [OK] Key buffer size / total MyISAM indexes: 384.0M/323.6M
> IIRC this is one of the biggies - the buffer is big enough to keep
> all indexes in memory. That means any indexed operation (select,
> join, etc) shouldn't need to hit the disk.

> [OK] Key buffer hit rate: 99.9% (1B cached / 1M reads)
> And the result is a good figure

> [OK] Sorts requiring temporary tables: 0% (288 temp sorts / 445K sorts)
> That's good, and suggests some good use of indexes etc for sorts.

> [!!] Joins performed without indexes: 94335
> Ideally all joins would be via indexes, but largely that's out of
> the user's control as it's controlled by the programming (how it
> does joins) and the database schema (what indexes there are). I
> suspect "bad" custom criteria in recording rules could contribute to this figure.

> [OK] Temporary tables created on disk: 7% (11K on disk / 152K total)
> That's good, most of the temporary tables created were in memory.

> [!!] Table cache hit rate: 0% (236 open / 361K opened)
> Dunno about this one !

> Then at the end if gives a list of recommendations :

> -------- Recommendations
> -----------------------------------------------------
> General recommendations:
> Run OPTIMIZE TABLE to defragment tables for better performance
> Enable the slow query log to troubleshoot bad queries
> Reduce or eliminate persistent connections to reduce connection usage
> Adjust your join queries to always utilize indexes
> Increase table_cache gradually to avoid file descriptor limits
> Variables to adjust:
> max_connections (> 20)
> wait_timeout (< 28800)
> interactive_timeout (< 28800)
> query_cache_size (> 24M)
> join_buffer_size (> 2.0M, or always use indexes with joins)
> table_cache (> 8192)

> I suggest that you don't just blindly follow the instructions, but
> go and read the MySQL docs and see what each setting does first. If
> nothing else, it'll make you realise just what a complex area it is :-O
> Also, after making a change, MySQL tuner won't give meaningful
> results for some aspects. So it can be a lengthy process of making
> adjustments, leaving the system to run for a few days, and then
> looking again. Where you can get immediate feedback is the total
> possible memory consumption and how your changes have affected it.


> From memory, the defaults are "fairly conservative - with the
> result that a lot of disk I/O is created which is not necessary. On
> a system doing nothing but running a DB it wouldn't be as bad as OS
> caching would hide the problem, but with a Myth system, there's so
> much other I/O that you're guaranteed to flush DB data from the OS
> cache and so it's a lot more important to tune the DB itself to keep
> it's own cache. More memory is often a good investment - and if
> there isn't enough memory to tune out virtually all disk access,
> then adding more memory will probably be a better investment than an SSD (or faster drive).


> Hope that's some help.


> _______________________________________________
> mythtv-users mailing list
> mythtv-users [at] mythtv
> http://www.mythtv.org/mailman/listinfo/mythtv-users
> http://wiki.mythtv.org/Mailing_List_etiquette
> MythTV Forums: https://forum.mythtv.org

Another way to reduce disk access and increase speed is putting /tmp
on ramfs. Of cause you need to have enough ram or your system starts
swapping. I do it for /tmp but not for /var/tmp because the latter
gets often used for bigger files. Note the difference between ramfs
and tmpfs. ramfs assigns space at need and tmpfs assigns an in kernel
predefined size. A size you can change with kernel options.
I also use it for my constant rrd sensor measuring and commit it to
disk on shutdown and every once in a while.

Tot mails,
Hika mailto:hikavdh [at] gmail

"Zonder hoop kun je niet leven
Zonder leven is er geen hoop
Het eeuwige dilemma
Zeker als je hoop moet vernietigen om te kunnen overleven!"

De lerende Mens
--

_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org


linux at thehobsons

Mar 6, 2014, 3:06 AM

Post #12 of 15 (387 views)
Permalink
Re: sluggish system, potential HD failure, and how best to get back up and runnin [In reply to]

Hika van den Hoven <hikavdh [at] gmail> wrote:

> Note the difference between ramfs
> and tmpfs. ramfs assigns space at need and tmpfs assigns an in kernel
> predefined size. A size you can change with kernel options.

As I read things, that's not correct.
ramfs doesn't limit the size - so you can keep writing until memory is all used up and OOM killer will crash the system. tmpfs has a fixed upper size which you can set at mount time.
I'm pretty certain that neither allocates the memory until it is actually used. I have a system (sort of semi-embedded and running from a CF card) where I've put several volatile filesystems on tmpfs - and until I realised the presence of the size option, I had the tmpfs volumes adding up to twice the available ram (and no swap on the system).

https://wiki.debian.org/ramfs

https://www.kernel.org/doc/Documentation/filesystems/tmpfs.txt
> tmpfs puts everything into the kernel internal caches and grows and
> shrinks to accommodate the files it contains ...

http://www.thegeekstuff.com/2008/11/overview-of-ramfs-and-tmpfs-on-linux/

_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org


hikavdh at gmail

Mar 6, 2014, 5:14 AM

Post #13 of 15 (384 views)
Permalink
Re: sluggish system, potential HD failure, and how best to get back up and runnin [In reply to]

Hoi Simon,

Thursday, March 6, 2014, 12:06:35 PM, you wrote:

> Hika van den Hoven <hikavdh [at] gmail> wrote:

>> Note the difference between ramfs
>> and tmpfs. ramfs assigns space at need and tmpfs assigns an in kernel
>> predefined size. A size you can change with kernel options.

> As I read things, that's not correct.
> ramfs doesn't limit the size - so you can keep writing until memory
> is all used up and OOM killer will crash the system. tmpfs has a
> fixed upper size which you can set at mount time.
> I'm pretty certain that neither allocates the memory until it is
> actually used. I have a system (sort of semi-embedded and running
> from a CF card) where I've put several volatile filesystems on tmpfs
> - and until I realised the presence of the size option, I had the
> tmpfs volumes adding up to twice the available ram (and no swap on the system).

> https://wiki.debian.org/ramfs

> https://www.kernel.org/doc/Documentation/filesystems/tmpfs.txt
>> tmpfs puts everything into the kernel internal caches and grows and
>> shrinks to accommodate the files it contains ...

> http://www.thegeekstuff.com/2008/11/overview-of-ramfs-and-tmpfs-on-linux/

> _______________________________________________
> mythtv-users mailing list
> mythtv-users [at] mythtv
> http://www.mythtv.org/mailman/listinfo/mythtv-users
> http://wiki.mythtv.org/Mailing_List_etiquette
> MythTV Forums: https://forum.mythtv.org

I might have misinterpreted the preallocation part. But the default
size for tmpfs is quite small. I believe 8 Mb. If you set it higher
with ramdisk_size=xxx (in kb) it goes for all tmpfses.
If you use ramfs you have to be aware of its maximum size to prevent
swapping. Therefore I say only use it for /tmp and not for /var/tmp.

Tot mails,
Hika mailto:hikavdh [at] gmail

"Zonder hoop kun je niet leven
Zonder leven is er geen hoop
Het eeuwige dilemma
Zeker als je hoop moet vernietigen om te kunnen overleven!"

De lerende Mens
--

_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org


linux at thehobsons

Mar 6, 2014, 5:27 AM

Post #14 of 15 (383 views)
Permalink
Re: sluggish system, potential HD failure, and how best to get back up and runnin [In reply to]

Hika van den Hoven <hikavdh [at] gmail> wrote:

> But the default
> size for tmpfs is quite small. I believe 8 Mb. If you set it higher
> with ramdisk_size=xxx (in kb) it goes for all tmpfses.

Actually, and I only know this because I was digging up information yesterday for my little box, the default is half the system memory - so 125M on my box with 1/4G RAM. You can specify on a per-filesystem basis with a "size=..." option to mount.
From my little box's fstab :
tmpfs /tmp tmpfs defaults,size=25M 0 0
tmpfs /var/tmp tmpfs defaults,size=25M 0 0
tmpfs /run tmpfs defaults,size=5M,mode=0755 0 0
tmpfs /var/log tmpfs defaults,size=5M,mode=0755 0 0
tmpfs /run/lock tmpfs defaults,size=5M 0 0

Top reports (shortly after clearing the caches) :
top - 13:20:52 up 2 days, 3:20, 1 user, load average: 0.08, 0.03, 0.05
Tasks: 52 total, 1 running, 51 sleeping, 0 stopped, 0 zombie
%Cpu(s): 15.6 us, 3.1 sy, 0.0 ni, 75.0 id, 0.0 wa, 0.0 hi, 6.2 si, 0.0 st
KiB Mem: 254976 total, 64892 used, 190084 free, 172 buffers
KiB Swap: 0 total, 0 used, 0 free, 7920 cached

And df reports :
Filesystem Size Used Avail Use% Mounted on
tmpfs 5.0M 112K 4.9M 3% /run
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 50M 0 50M 0% /run/shm
tmpfs 25M 0 25M 0% /tmp
tmpfs 25M 0 25M 0% /var/tmp
tmpfs 5.0M 24K 5.0M 1% /var/log

I'm fairly confident that I've got things about right for this box :)

_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org


hikavdh at gmail

Mar 6, 2014, 5:36 AM

Post #15 of 15 (383 views)
Permalink
Re: sluggish system, potential HD failure, and how best to get back up and runnin [In reply to]

Hoi Simon,

Thursday, March 6, 2014, 2:27:27 PM, you wrote:

> Hika van den Hoven <hikavdh [at] gmail> wrote:

>> But the default
>> size for tmpfs is quite small. I believe 8 Mb. If you set it higher
>> with ramdisk_size=xxx (in kb) it goes for all tmpfses.

> Actually, and I only know this because I was digging up information
> yesterday for my little box, the default is half the system memory -
> so 125M on my box with 1/4G RAM. You can specify on a per-filesystem
> basis with a "size=..." option to mount.
> From my little box's fstab :
> tmpfs /tmp tmpfs defaults,size=25M 0 0
> tmpfs /var/tmp tmpfs defaults,size=25M 0 0
> tmpfs /run tmpfs defaults,size=5M,mode=0755 0 0
> tmpfs /var/log tmpfs defaults,size=5M,mode=0755 0 0
> tmpfs /run/lock tmpfs defaults,size=5M 0 0

> Top reports (shortly after clearing the caches) :
> top - 13:20:52 up 2 days, 3:20, 1 user, load average: 0.08, 0.03, 0.05
> Tasks: 52 total, 1 running, 51 sleeping, 0 stopped, 0 zombie
> %Cpu(s): 15.6 us, 3.1 sy, 0.0 ni, 75.0 id, 0.0 wa, 0.0 hi, 6.2 si, 0.0 st
> KiB Mem: 254976 total, 64892 used, 190084 free, 172 buffers
> KiB Swap: 0 total, 0 used, 0 free, 7920 cached

> And df reports :
> Filesystem Size Used Avail Use% Mounted on
> tmpfs 5.0M 112K 4.9M 3% /run
> tmpfs 5.0M 0 5.0M 0% /run/lock
> tmpfs 50M 0 50M 0% /run/shm
> tmpfs 25M 0 25M 0% /tmp
> tmpfs 25M 0 25M 0% /var/tmp
> tmpfs 5.0M 24K 5.0M 1% /var/log

> I'm fairly confident that I've got things about right for this box :)

> _______________________________________________
> mythtv-users mailing list
> mythtv-users [at] mythtv
> http://www.mythtv.org/mailman/listinfo/mythtv-users
> http://wiki.mythtv.org/Mailing_List_etiquette
> MythTV Forums: https://forum.mythtv.org


You're right, for me it was quit some time ago I looked into it.
And looking now into my system they are half the system memory. Probably
some change along the kernel defaults since I looked at it.
I stand corrected!

Tot mails,
Hika mailto:hikavdh [at] gmail

"Zonder hoop kun je niet leven
Zonder leven is er geen hoop
Het eeuwige dilemma
Zeker als je hoop moet vernietigen om te kunnen overleven!"

De lerende Mens
--

_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

MythTV users RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.