Mailing List Archive: MythTV: Users

HLS file expiration

Index | Next | Previous | View Threaded

gnassas at mac

Apr 11, 2012, 5:36 AM

Post #1 of 3 (214 views)
Permalink

HLS file expiration

Hi, I notice that eventually the backend removes the files associated with an http live stream. Problem is it doesn't remove the database entry for the stream itself so it continues to appear in the list returned by GetLiveStreamList but when you try to use the associated FullURL it doesn't regenerate the stream on the fly. On my system (Mac & iPad) the player just hangs. That sounds like a bug, either the stream entry should be removed or its files regenerated on demand. Probably the first is the better option.

Also related is when you delete a TV recording its HLS files continue to exist. That sounds wrong since, without the recording's metadata, there's no way to know what a stream refers to. Probably the same problem exists with videos. I guess the right behaviour is to do a broad delete when source material disappears.

Personally I'd like control over expiration policy. I'm using streams as a way to have an iPad-compatible copy on tap while still keeping a full res version around for my main rig & it's not amusing when the iPad version gets yanked out from under me. It's not like I need the disk space.

I can open bugs but maybe there needs to be a brief discussion on what is the good behaviour?

Oh, also, the RemoveRecorded http api doesn't take a parameter for allow rerecord. The code seems to do a permanent delete. Is this an oversight or intentional? I'd like to have the option & might even be able to supply a patch if people agree.

- George
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users

cpinkham at bc2va

Apr 11, 2012, 9:06 AM

Post #2 of 3 (199 views)
Permalink

Re: HLS file expiration [In reply to]

* On Wed Apr 11, 2012 at 08:36:39AM -0400, George Nassas wrote:
> Hi, I notice that eventually the backend removes the files associated
> with an http live stream. Problem is it doesn't remove the database entry
> for the stream itself so it continues to appear in the list returned

There is no code to auto-expire HLS stream files. The only cleanup done
on these files is when you delete a stream which should delete the files
along with the DB entry. Perhaps you have something else cleaning up
~/.mythtv/tmp/hls. Is your home directory persistent?

> Also related is when you delete a TV recording its HLS files continue
> to exist. That sounds wrong since, without the recording's metadata,
> there's no way to know what a stream refers to. Probably the same problem
> exists with videos. I guess the right behaviour is to do a broad delete
> when source material disappears.

HLS supprt is a work in progress, and currently there is nothing to tie
together a recording or video and it's HLS stream. This glue will come
later and be made easier by the planned file table changes.

> Personally I'd like control over expiration policy. I'm using streams
> as a way to have an iPad-compatible copy on tap while still keeping
> a full res version around for my main rig & it's not amusing when the
> iPad version gets yanked out from under me. It's not like I need the
> disk space.

Once the file table schema changes are in place, you'll actually be able
to have a second copy of a recording in .mp4 format and MythTV will
track it right along with the original .mpg file. Tracking a HLS
stream will also be possible once we get the glue in place.

> I can open bugs but maybe there needs to be a brief discussion on what
> is the good behaviour?

The issue isn't ideas, it's time. I haven't had time leading up to 0.25
to add even 1/4 of the items on my HLS TODO list, but I didn't want to
rip out the existing HLS support prior to release either. I figured
the existing support is usable as-is and I'll add more glue post 0.25.

> Oh, also, the RemoveRecorded http api doesn't take a parameter for
> allow rerecord. The code seems to do a permanent delete. Is this an
> oversight or intentional? I'd like to have the option & might even be
> able to supply a patch if people agree.

I think this was probably an oversight. A patch to allow this should
be welcome.

--
Chris
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users

gnassas at mac

Apr 11, 2012, 2:36 PM

Post #3 of 3 (193 views)
Permalink

Re: HLS file expiration [In reply to]

On 2012-04-11, at 12:06 PM, Chris Pinkham <cpinkham [at] bc2va> wrote:

> There is no code to auto-expire HLS stream files. The only cleanup done
> on these files is when you delete a stream which should delete the files
> along with the DB entry. Perhaps you have something else cleaning up
> ~/.mythtv/tmp/hls.

Hmph, interesting. My "Streaming" storage group has folders on 4 of my 5 HDs and I do see stream files being written there. I don't manage the contents in any way other than HLS create/delete calls. Since I started playing with streaming I've noticed maybe a half dozen streams disappearing in this way but today was the first time I looked into it. The logs show the stream was created March 15 or 13 but nothing about deleting it. I know I played that stream in the last 4 to 7 days so the remove was recent.

The reason I suspect a reaper process is it's always the oldest or second oldest stream which disappears. Can you say for sure you've had streams survive on your disks for a month or more? Maybe I'll create a new stream, backdate the files six months, and see how long they last. Could a preview image pruner be running amok?

> The issue isn't ideas, it's time. I haven't had time leading up to 0.25
> to add even 1/4 of the items on my HLS TODO list

Of course and it's understandable. Could you share some sort of roadmap? I mean unexpected bon bons are very romantic but I'd rather have a general idea of what sort of things are likely for this release, the next, etc.

BTW, is there some kind of web sockets event feed on this todo list? It would be good to be told when a new recording has started or one gets deleted. My recorded list is huge and constant polling would not be pretty.

>> Oh, also, the RemoveRecorded http api doesn't take a parameter for
> I think this was probably an oversight. A patch to allow this should
> be welcome.

I'll see what I can come up with.

- George
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads