mythtv at cvs
Nov 29, 2006, 9:13 PM
Post #1 of 1
mythtv commit: r12151 - in trunk/mythtv by cpinkham
Date: 2006-11-30 05:13:44 +0000 (Thu, 30 Nov 2006)
New Revision: 12151
Storage Groups (a novel by Chris Pinkham)
This commit upgrades MythTV from using a single recording directory to allow
multiple directories in what are called "Storage Groups." A Storage Group
is just a list of directories, it has no other attributes. The only
required Storage Group is the 'Default' group. Storage Groups are created
on the master backend via mythtv-setup, but can be configured and overridden
on a slave backend by running mythtv-setup on the slave.
The 'Default' Storage Group is where recordings are stored by default.
LiveTV recordings may be put into their own directory or directories by
creating a special 'LiveTV' Storage Group. If the LiveTV Storage Group does
not exist, LiveTV recordings will be placed in the directories used
for the 'Default' Storage Group. A scheduled recording is assigned to a
particular Storage Group in the scheduled recording editor.
Storage Groups are global unless overridden on a slave via mythtv-setup.
If a directory in a Storage Group does not exist, it will be skipped over
in the load balancing determination code when a new recording is starting.
If you want to record to directories /a, /b, and /c on the master, but
only /a and /b on the slave even though /c does exist, then you need to
override the master Storage Group on the slave and only add /a and /b to
the slave's Storage Group.
The existing recording directory is copied over from the old
RecordFilePrefix setting to the new Default Storage. If you have
multiple backends, it is best to mount their recording directories in the
same location on each backend to make the Storage Groups cleaner, but this
is not required and current systems should have no problems continuing to
work. You can have one Storage Group with a directory list where one
directory is only visible on one server and another directory is only
visible on a different server. The StorageGroup class should work
correctly with this because it detects what directories are available on
Free space on the mythfrontend status page and the mythbackend status
webpage is now grouped by filesystem. The Storage Group code is able
to detect if you are using multiple directories on the same filesystem
and will not double, triple, etc. the available and free space counts
if the same filesystem is used for multiple directories. This applies
whether the directories are in the same Storage Group or different
Storage Groups, and no matter on which backend they are found.
The AutoExpirer now only runs on the master backend. The master is
in charge of all expiration. The AutoExpirer will determine what
filesystems are shared and what directories are on those filesystems.
This may now cause recordings to be expired out of order because the first
recordings in the expire list could be on a filesystem with lots of space,
but another recording further up the list could reside on a filesystem
that needs space to be freed.
<spoiler, ie. not in the current code>
It is _planned_ to eventually allow the AutoExpirer to move recordings
around via a planned builtin JobQueue job so that the expire list can
still be processed more in the desired order as well as allowing actions
such as archiving of recordings instead of expiring them.
The Storage Group Editor is in mythtv-setup. There should be no limitations
on the number of directories that can be in a group. Instead of striping
across 5 drives, you could create 10 filesystems, each with its own
recording directory and put each of these directories in the same Storage
The current load-balancing preferences in order are as follows:
- Local filesystems over remote
- Less-busy (less weight) over more-busy (more weight)
- More Free Space over Less Free Space
The 'business' of a filesystem is determined by weights. The following
weights are added to a filesystem if it is in use for the following things:
- recording = +10
- playback = +5 (mythfrontend)
- comm flagging = +5 (mythcommflag)
- transcoding = +5 (mythtranscode)
If a recording is due to end within 3 minutes, it is not counted against
the weight of a filesystem. This is done to account for the pre/post-roll
and start-early/end-late settings.
Some StorageGroup debugging info is printed out when "-v file" is used on the
command line, but other more verbose parts are only printed when
"-v file,schedule" is used since the code is in the scheduler and we are
The Storage Groups code does not do anything special to handle symlinks when
you have two directories in the same Storage Group and one directory contains
links pointed to files in the other directory. If you are using Storage Groups
on a system where you previously used symbolic links and the "delete follows
symlinks" setting, then your best action is to delete all symlinks and add
the secondary directories to the Default Storage Group. You should no longer
need to have the "delete follows symlinks" setting turned ON, but it doesn't
Thanks to Bruce and Janne for testing along with some bug reports and fixes.
NOTE: I need some input from someone with an OSX backend. Currently the
code assumes all filesystems are local under OSX. This code is in
programs/mythbackend/backendutil.cpp inside BackendQueryDiskSpace() if
someone wants to take a look at modifying the 'if' statement for OSX.
For Linux, we can look at the f_type field from statfs, but OSX does not
fill that field in, and we instead have to do a string compare on
f_fstypename, but I don't know what to compare to right now.
mythtv-commits mailing list
mythtv-commits [at] mythtv