Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: MythTV: Users

0.25: this is nice

 

 

MythTV users RSS feed   Index | Next | Previous | View Threaded


warpme at o2

Mar 14, 2012, 3:35 AM

Post #1 of 21 (3150 views)
Permalink
0.25: this is nice

Hi *

Seeing some issues with CPU load (#10414) I decided to look how it goes i my case (2xDVB-S2, 1xDVB-T), AMD235e, mysql5.5.17, 64bit ArchLinux derivate, OS on 2T green Caviar, recordings on another 2T green Caviar.
Concurrent recording of 8xHD + 4xSD (all what I can launch) gives 18-22% CPU load. For more stress I launch also 4x comm flagging tasks & test system look & feel.
Heh - interesting: MythWEB was flashy as in Idle. Loading 450+ rec list on ION1 FE was little slower - but for person who not knows about BE is so stressed, FE responsiveness was perceived as "normal" (user words).
Nice !


_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users


stichnot at gmail

Mar 14, 2012, 10:12 AM

Post #2 of 21 (3093 views)
Permalink
Re: 0.25: this is nice [In reply to]

On Wed, Mar 14, 2012 at 3:35 AM, warpme <warpme [at] o2> wrote:
> Loading 450+ rec list on ION1 FE was little slower - but for person who not knows about BE is so stressed, FE responsiveness was perceived as "normal" (user words).

I feel your pain, I have over 2000 recordings, and my frontends are
ION1 machines. You may be interested in the patch in
http://code.mythtv.org/trac/ticket/10161 . That particular approach
probably won't make it into the code base, but something in that
direction might.

Jim
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users


ylee at pobox

Mar 14, 2012, 3:11 PM

Post #3 of 21 (3086 views)
Permalink
Re: 0.25: this is nice [In reply to]

Jim Stichnoth <stichnot [at] gmail> says:
> You may be interested in the patch in
> http://code.mythtv.org/trac/ticket/10161 .

Jim, does any version of your patch apply to 0.24?

--
MythTV FAQ Q: "Cheap frontend/backend?" A: Revo, $200-300 @ Newegg
Q: "Record HD cable/satellite?" A: Hauppauge HD-PVR, $200 @ Newegg
Q: "Can't change Live TV channels w/multirec!" A: Hit NEXTCARD key
More answers @ <URL:http://www.gossamer-threads.com/lists/mythtv/>
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users


stichnot at gmail

Mar 14, 2012, 4:15 PM

Post #4 of 21 (3086 views)
Permalink
Re: 0.25: this is nice [In reply to]

On Wed, Mar 14, 2012 at 3:11 PM, Yeechang Lee <ylee [at] pobox> wrote:
> Jim Stichnoth <stichnot [at] gmail> says:
>> You may be interested in the patch in
>> http://code.mythtv.org/trac/ticket/10161 .
>
> Jim, does any version of your patch apply to 0.24?

I don't have anything running 0.24. I uploaded a version of the patch
that at least compiles against 0.24, but I can't run it so no
guarantees.

Jim
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users


jyavenard at gmail

Mar 14, 2012, 8:50 PM

Post #5 of 21 (3078 views)
Permalink
Re: 0.25: this is nice [In reply to]

On 15 March 2012 04:12, Jim Stichnoth <stichnot [at] gmail> wrote:
> I feel your pain, I have over 2000 recordings, and my frontends are
> ION1 machines.  You may be interested in the patch in
> http://code.mythtv.org/trac/ticket/10161 .  That particular approach
> probably won't make it into the code base, but something in that
> direction might.

That patch made an amazing difference.

on my iMac i7 3.4GHz quad-core, it takes 6.7s to load the recording
screen, 0.9s with the patch
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users


warpme at o2

Mar 15, 2012, 1:51 AM

Post #6 of 21 (3076 views)
Permalink
Re: 0.25: this is nice [In reply to]

On 3/14/12 6:12 PM, Jim Stichnoth wrote:
> On Wed, Mar 14, 2012 at 3:35 AM, warpme<warpme [at] o2> wrote:
>> Loading 450+ rec list on ION1 FE was little slower - but for person who not knows about BE is so stressed, FE responsiveness was perceived as "normal" (user words).
> I feel your pain, I have over 2000 recordings, and my frontends are
> ION1 machines. You may be interested in the patch in
> http://code.mythtv.org/trac/ticket/10161 . That particular approach
> probably won't make it into the code base, but something in that
> direction might.
>
> Jim
> _______________________________________________
> mythtv-users mailing list
> mythtv-users [at] mythtv
> http://www.mythtv.org/mailman/listinfo/mythtv-users
>
Jim,
Thx for this.
I will apply it during my weekly builds (next days).

BTW: speaking about ION1, I must say it running....surprisingly well
with 0.25.
When I moved from 0.24->master (half 9 months) I moved also one frontend
from AMD X2 4400 to ION1. At that time myth FE slowdown was really
noticeable.
Over last 6 months I saw continuous improvement of myth and currently
myth on ION1 is almost comparable with 0.24 on AMD X2 4400.
Sure - some actions are slight faster on X2 4400 that on ION but who
cares is window opening in 100ms or 200ms.
My overall impression is that this tiny ION1 is perfectly suitable as
myth FE - and this shows that current mythv code is pretty good one.

-br
Attachments: warpme.vcf (83 B)


briandlong at gmail

Mar 15, 2012, 3:56 AM

Post #7 of 21 (3070 views)
Permalink
Re: 0.25: this is nice [In reply to]

On Wed, Mar 14, 2012 at 6:11 PM, Yeechang Lee <ylee [at] pobox> wrote:

> Jim Stichnoth <stichnot [at] gmail> says:
> > You may be interested in the patch in
> > http://code.mythtv.org/trac/ticket/10161 .
>
> Jim, does any version of your patch apply to 0.24?


Yeechang, if you add this patch to the "bijou" 0.24.2 packages on AtRPMS,
I'll be the first to test it. :-)

/Brian/


mythtv-users2 at dwilga-linux1

Mar 15, 2012, 7:59 AM

Post #8 of 21 (3058 views)
Permalink
Re: 0.25: this is nice [In reply to]

On 3/14/12 7:15 PM, Jim Stichnoth wrote:
> On Wed, Mar 14, 2012 at 3:11 PM, Yeechang Lee<ylee [at] pobox> wrote:
>
>> Jim Stichnoth<stichnot [at] gmail> says:
>>
>>> You may be interested in the patch in
>>> http://code.mythtv.org/trac/ticket/10161 .
>>>
>> Jim, does any version of your patch apply to 0.24?
>>
> I don't have anything running 0.24. I uploaded a version of the patch
> that at least compiles against 0.24, but I can't run it so no
> guarantees.
>
I'm compiling Jim's patch against 0.24-fixes right now, and will test
tonight on my ION1 boxes, with before and after timings.

--
Dan Wilga "Ook."

_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users


adeffs.mythtv at gmail

Mar 15, 2012, 10:20 AM

Post #9 of 21 (3047 views)
Permalink
Re: 0.25: this is nice [In reply to]

On Thu, Mar 15, 2012 at 10:59 AM, Dan Wilga
<mythtv-users2 [at] dwilga-linux1> wrote:
> On 3/14/12 7:15 PM, Jim Stichnoth wrote:
>> On Wed, Mar 14, 2012 at 3:11 PM, Yeechang Lee<ylee [at] pobox>  wrote:
>>
>>> Jim Stichnoth<stichnot [at] gmail>  says:
>>>
>>>> You may be interested in the patch in
>>>> http://code.mythtv.org/trac/ticket/10161 .
>>>>
>>> Jim, does any version of your patch apply to 0.24?
>>>
>> I don't have anything running 0.24.  I uploaded a version of the patch
>> that at least compiles against 0.24, but I can't run it so no
>> guarantees.
>>
> I'm compiling Jim's patch against 0.24-fixes right now, and will test
> tonight on my ION1 boxes, with before and after timings.
>
> --
> Dan Wilga                                                        "Ook."

I've got two ION1 systems and we're running 0.25, I get fairly quick
load times (never worse than 0.24), but about 1 in 5 times it just
won't load the data and sits there, I have to kill the frontend
process.

Is this along the lines of what this patch is attempting to resolve?


--
Steve
http://www.mythtv.org/wiki/User:Steveadeff
Before you ask, read the FAQ!
http://www.mythtv.org/wiki/Frequently_Asked_Questions
then search the Wiki, and this list,
http://www.gossamer-threads.com/lists/mythtv/
Mailinglist etiquette - http://www.mythtv.org/wiki/Mailing_List_etiquette
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users


stichnot at gmail

Mar 15, 2012, 10:32 AM

Post #10 of 21 (3055 views)
Permalink
Re: 0.25: this is nice [In reply to]

On Thu, Mar 15, 2012 at 10:20 AM, Steven Adeff <adeffs.mythtv [at] gmail> wrote:
> I've got two ION1 systems and we're running 0.25, I get fairly quick
> load times (never worse than 0.24), but about 1 in 5 times it just
> won't load the data and sits there, I have to kill the frontend
> process.
>
> Is this along the lines of what this patch is attempting to resolve?

Not at all. The patch only improves performance by deferring many
string creations until they are actually needed. The improvement
won't be noticeable until you have on the order of thousands of
recordings.

What you're describing is a mostly likely a bug. If you can catch it,
please try to attach gdb to the mythfrontend process to get a
backtrace, and open a new ticket with frontend logs and the backtrace.
Even better if you can get a backtrace from a Debug or Profile build.

Jim
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users


ylee at pobox

Mar 15, 2012, 2:29 PM

Post #11 of 21 (3049 views)
Permalink
Re: 0.25: this is nice [In reply to]

Jean-Yves Avenard <jyavenard [at] gmail> says:
> That patch made an amazing difference.
>
> on my iMac i7 3.4GHz quad-core, it takes 6.7s to load the recording
> screen, 0.9s with the patch

Yes, after brief testing I can also report a substantial improvement
with the 0.24 version of the patch. A list of ~1500 recordings appears
in under three seconds, about one third to one half of the time needed
without the patch.

--
MythTV FAQ Q: "Cheap frontend/backend?" A: Revo, $200-300 @ Newegg
Q: "Record HD cable/satellite?" A: Hauppauge HD-PVR, $200 @ Newegg
Q: "Can't change Live TV channels w/multirec!" A: Hit NEXTCARD key
More answers @ <URL:http://www.gossamer-threads.com/lists/mythtv/>
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users


mtdean at thirdcontact

Mar 15, 2012, 3:17 PM

Post #12 of 21 (3044 views)
Permalink
Re: 0.25: this is nice [In reply to]

On 03/15/2012 05:29 PM, Yeechang Lee wrote:
> Jean-Yves Avenard says:
>> That patch made an amazing difference.
>>
>> on my iMac i7 3.4GHz quad-core, it takes 6.7s to load the recording
>> screen, 0.9s with the patch
> Yes, after brief testing I can also report a substantial improvement
> with the 0.24 version of the patch. A list of ~1500 recordings appears
> in under three seconds, about one third to one half of the time needed
> without the patch.

Strange. On my 0.24-fixes (remote) frontend without the patch, I can
load ~1900 recordings in well under a second (like it's on screen before
my finger releases the remote button). It's fast enough I often use the
Watch Recordings jump point to go back to the "All Programs" at the top
of the list rather than scrolling.

Are you guys who have extremely slow load times using some non-default
options, perhaps? Or do you have poor-performing MySQL servers or
something?

Have you tried running optimize_mythdb.pl?

I can't believe it's down to CPU or RAM, since JYA's 3.4GHz Core i7 quad
is much more powerful than my AMD Athlon II X2 250 (3.0GHz dual-core)
with 2GB RAM.

Mike
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users


jyavenard at gmail

Mar 15, 2012, 3:42 PM

Post #13 of 21 (3047 views)
Permalink
Re: 0.25: this is nice [In reply to]

Hi

On 16 March 2012 09:17, Michael T. Dean <mtdean [at] thirdcontact> wrote:
> Are you guys who have extremely slow load times using some non-default
> options, perhaps?  Or do you have poor-performing MySQL servers or
> something?

I should state that my backend is now using a SATA3 SSD (Intel 520) as
boot disk where the mysql DB

My mac mini 2.4 Ghz C2D (ubuntu 11.10) loads the same screen in less
than 2s where my iMac (a *significantly* faster machine takes almost
7s).
This excludes the CPU and database for sure...

Jim mentioned that it could be related to a slower implementation of
QDateTime on the mac platform.
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users


stichnot at gmail

Mar 15, 2012, 3:44 PM

Post #14 of 21 (3046 views)
Permalink
Re: 0.25: this is nice [In reply to]

On Thu, Mar 15, 2012 at 3:17 PM, Michael T. Dean
<mtdean [at] thirdcontact> wrote:
> On 03/15/2012 05:29 PM, Yeechang Lee wrote:
>> Jean-Yves Avenard says:
>>> That patch made an amazing difference.
>>>
>>> on my iMac i7 3.4GHz quad-core, it takes 6.7s to load the recording
>>> screen, 0.9s with the patch
>> Yes, after brief testing I can also report a substantial improvement
>> with the 0.24 version of the patch. A list of ~1500 recordings appears
>> in under three seconds, about one third to one half of the time needed
>> without the patch.
>
> Strange.  On my 0.24-fixes (remote) frontend without the patch, I can
> load ~1900 recordings in well under a second (like it's on screen before
> my finger releases the remote button).  It's fast enough I often use the
> Watch Recordings jump point to go back to the "All Programs" at the top
> of the list rather than scrolling.
>
> Are you guys who have extremely slow load times using some non-default
> options, perhaps?  Or do you have poor-performing MySQL servers or
> something?
>
> Have you tried running optimize_mythdb.pl?
>
> I can't believe it's down to CPU or RAM, since JYA's 3.4GHz Core i7 quad
> is much more powerful than my AMD Athlon II X2 250 (3.0GHz dual-core)
> with 2GB RAM.

This is unlikely to be a DB issue. The best test of that is to go
into the Watch Recordings screen, then switch in and out of the All
Recording group via the "Change Group Filter" menu. This operation
reuses cached data from the backend. On a slow frontend with lots of
recordings, it may take several seconds without the patch but is
virtually instantaneous with the patch.

The performance killer is almost certainly the various QDateTime
calculations in ProgramInfo::ToMap(). Most of these involve calls to
MythDateTimeToString(). For people like Jean-Yves who suffer from
poor performance despite a powerful frontend, my bet is either a poor
implementation of QDateTime on the platform, or that their DB Settings
values of ShortDateFormat, DateFormat, and TimeFormat are pushing the
QDateTime code into some bad-performance region.

Jim
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users


jyavenard at gmail

Mar 15, 2012, 7:15 PM

Post #15 of 21 (3036 views)
Permalink
Re: 0.25: this is nice [In reply to]

Changing in the display settings the format of the date (long one)
from Friday March 16 2012 to 16/030/2012
changed the time it took to load the recording screen from 6.7s to 1.7s..
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users


mtdean at thirdcontact

Mar 15, 2012, 8:25 PM

Post #16 of 21 (3031 views)
Permalink
Re: 0.25: this is nice [In reply to]

On 03/15/2012 10:15 PM, Jean-Yves Avenard wrote:
> Changing in the display settings the format of the date (long one)
> from Friday March 16 2012 to 16/030/2012
> changed the time it took to load the recording screen from 6.7s to 1.7s..

For others who might be reading/following this thread, there's an
outstanding question about whether running mythfrontend with the time
zone specified in the TZ variable would also make speed up the loading.
So, for example, with a "slow" format (like "Friday March 16 2012"
chosen), try restarting mythfrontend using:

TZ='America/New_York' mythfrontend

(change the time zone identifier, as appropriate, based on what your
master backend uses), then test loading the Watch Recordings screen.

Note, also, that on my system--where I see Watch Recordings loaded in
well under a second with ~1900 recordings--I use the same "Friday March
16 2012" format for my long date format, so this issue seems to only
affect some systems.

Mike
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users


jyavenard at gmail

Mar 15, 2012, 8:28 PM

Post #17 of 21 (3029 views)
Permalink
Re: 0.25: this is nice [In reply to]

On 16 March 2012 14:25, Michael T. Dean <mtdean [at] thirdcontact> wrote:
> TZ='America/New_York' mythfrontend

No difference in time whatsoever...
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users


ylee at pobox

Mar 16, 2012, 3:00 AM

Post #18 of 21 (3016 views)
Permalink
Re: 0.25: this is nice [In reply to]

Jean-Yves Avenard <jyavenard [at] gmail> says:
> Changing in the display settings the format of the date (long one)
> from Friday March 16 2012 to 16/030/2012 changed the time it took to
> load the recording screen from 6.7s to 1.7s..

!!! I can confirm this. Using your newest g910385c 0.24 OS X client,
without Jim Stitchnoth's optimization patch, switching the date format
from Fri 16 March 2012 to 03/16 halves the Watch Recordings loading
time over 802.11n from 20 to 10 seconds.

I will be curious to see whether this makes a difference on my Linux
frontend/backend, which uses the same date format. I'd always ascribed
the relatively slow loading time there (6 seconds without Jim's patch;
<3 with) to the combination of age (a seven years-old Pentium 4) and
size (almost 1600 recordings in the Default view), but perhaps
RHEL/CentOS has a non-optimal Qt implementation too?

--
MythTV FAQ Q: "Cheap frontend/backend?" A: Revo, $200-300 @ Newegg
Q: "Record HD cable/satellite?" A: Hauppauge HD-PVR, $200 @ Newegg
Q: "Can't change Live TV channels w/multirec!" A: Hit NEXTCARD key
More answers @ <URL:http://www.gossamer-threads.com/lists/mythtv/>
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users


mythtv-users2 at dwilga-linux1

Mar 16, 2012, 5:33 AM

Post #19 of 21 (3012 views)
Permalink
Re: 0.25: this is nice [In reply to]

On 3/15/12 10:59 AM, Dan Wilga wrote:
> I'm compiling Jim's patch against 0.24-fixes right now, and will test
> tonight on my ION1 boxes, with before and after timings.
>
I ran my test on 0.24-fixes (JYA), and 7.5 seconds went down to 5
seconds. That's with 1353 recordings, on an ION1 330.

Based on my, and others', success, I'd say the patch is RTBC and WTBC
(Welcome) too.

--
Dan Wilga "Ook."

_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users


stichnot at gmail

Mar 16, 2012, 10:15 AM

Post #20 of 21 (3004 views)
Permalink
Re: 0.25: this is nice [In reply to]

On Fri, Mar 16, 2012 at 5:33 AM, Dan Wilga
<mythtv-users2 [at] dwilga-linux1> wrote:
> On 3/15/12 10:59 AM, Dan Wilga wrote:
> I ran my test on 0.24-fixes (JYA), and 7.5 seconds went down to 5
> seconds. That's with 1353 recordings, on an ION1 330.
>
> Based on my, and others', success, I'd say the patch is RTBC and WTBC
> (Welcome) too.

It's good to know that I wasn't the only one affected by the
performance. Let me try to summarize and set expectations.

Bringing up the Watch Recordings screen involves several steps: (1)
load up the recordings-ui.xml theme file, (2) load and cache
ProgramInfo objects from the backend, (3) generate display strings
from the ProgramInfo objects, and (4) build the screen layout. I
think (1) and (4) are pretty fast. (2) involves the frontend waiting
for data from the backend, which is probably dominated by the sql
query. (3) is all on the frontend, and can take a while because all
possible strings (total of 68 last time I counted) are generated for
every ProgramInfo object in the desired recording group, and a bunch
of those strings are dates and times which can be notoriously
expensive to generate.

The patch works by deferring the work in (3) until it is actually
needed. So instead of waiting to generate 68 strings times 2000 (or
whatever) recordings, it generates 68 strings times 10 items on the
page, or however many items per page the theme specifies, which is of
course far less than 2000. So you pay a tiny performance penalty when
you page through the listings for the first time, but it won't be
noticeable, and the strings are cached so that paging through a second
time has no penalty.

The actual time component of (3) can be best evaluated through the
"Change Group Filter" menu item, which basically only involves step
(3) and not the other steps. (You may have to press MENU twice to see
the Change Group Filter option.)

On a modern and reasonable-performance CPU without the patch,
processing even 2000 recordings shouldn't really tax the CPU, but it
can be very noticeable on an ION-based system. In addition, for
reasons we don't yet fully understand, there's something something odd
about Qt on the Mac OS that makes some date formatting unbelievably
expensive, as Jean-Yves has shown, and the patch happens to benefit
that setup as well.

The patch, as written, isn't necessarily as clean and maintainable as
we'd like; think of it as a proof of concept that minimizes the size
of the patch. As such, don't count on seeing it in 0.25.
Nonetheless, I use the patch on my own production system so I'll
definitely keep it up to date in
http://code.mythtv.org/trac/ticket/10161 for anyone else who wants it.

Jim
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users


ylee at pobox

Mar 16, 2012, 2:44 PM

Post #21 of 21 (2994 views)
Permalink
Re: 0.25: this is nice [In reply to]

I wrote:
> I will be curious to see whether [changing the long date format] on
> my Linux frontend/backend, which uses the same date format. [...]
> perhaps RHEL/CentOS has a non-optimal Qt implementation too?

After testing, it does not have the issue the OS X Qt
exhibits. Whether the long or short date format is used, on my Linux
0.24 frontend/backend switching from the Default to All Programs takes
~3 seconds without Jim's patch and is instantaneous with it.

--
MythTV FAQ Q: "Cheap frontend/backend?" A: Revo, $200-300 @ Newegg
Q: "Record HD cable/satellite?" A: Hauppauge HD-PVR, $200 @ Newegg
Q: "Can't change Live TV channels w/multirec!" A: Hit NEXTCARD key
More answers @ <URL:http://www.gossamer-threads.com/lists/mythtv/>
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users

MythTV users RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.