Mailing List Archive: Request Tracker: Users

RT3 Performance Issues Update

Index | Next | Previous | View Threaded

jonathan at jonmasters

Aug 11, 2004, 4:12 AM

Post #1 of 6 (1195 views)
Permalink

RT3 Performance Issues Update

Hi all,

I have talked to a bunch of people about RT performance and at a recent
conference discovered that I am certainly not alone. The MySQL logs show
that queries are completing normally and the slow logs set to 1 second
are completely empty. It is not MySQL.

However I have forced myself to play around more with perl and have been
using an interactive mod_perl debugging session to determine that there
is potentially a problem in the DBIx::SearchBuilder::Record::Cacheable
code as it likes to go and sit in big long loops looking for stuff that
has expired from its object cache. Can I turn off the cacheing and just
use the database directly?

It likes running the expire code path hundreds of times, e.g.:

DB<21> T
. = DBIx::SearchBuilder::Record::Cachable::_expire(ref(RT::Principal),
'Transactions:id=742') called from file
`/usr/local/share/perl/5.8.4/DBIx/SearchBuilder/Record/Cachable.pm' line 174
. =
DBIx::SearchBuilder::Record::Cachable::_gc_expired(ref(RT::Principal))
called from file
`/usr/local/share/perl/5.8.4/DBIx/SearchBuilder/Record/Cachable.pm' line 328
$ =
DBIx::SearchBuilder::Record::Cachable::_lookup_primary_cache_key(ref(RT::Principal),
'Principals:PrincipalType=User,ObjectId=22') called from file
`/usr/local/share/perl/5.8.4/DBIx/SearchBuilder/Record/Cachable.pm' line 92
. =
DBIx::SearchBuilder::Record::Cachable::LoadByCols(ref(RT::Principal),
'PrincipalType', 'User', 'ObjectId', 22) called from file
`/opt/rt3/lib/RT/Record.pm' line 272
. = RT::Record::LoadByCols(ref(RT::Principal), 'ObjectId', 22,
'PrincipalType', 'User') called from file
`/opt/rt3/lib/RT/User_Overlay.pm' line 1126
$ = RT::User::PrincipalObj(ref(RT::User)) called from file
`/opt/rt3/lib/RT/User_Overlay.pm' line 1454
$ = RT::User::HasRight(ref(RT::User), 'Object', ref(RT::Ticket),
'Right', 'ShowTicket') called from file
`/opt/rt3/lib/RT/Ticket_Overlay.pm' line 3949
$ = RT::Ticket::HasRight(ref(RT::Ticket), 'Principal', ref(RT::User),
'Right', 'ShowTicket') called from file
`/opt/rt3/lib/RT/Ticket_Overlay.pm' line 3913
$ = RT::Ticket::CurrentUserHasRight(ref(RT::Ticket), 'ShowTicket')
called from file `/opt/rt3/lib/RT/Ticket_Overlay.pm' line 3860
$ = RT::Ticket::_Value(ref(RT::Ticket), 'EffectiveId') called from file
`/usr/local/share/perl/5.8.4/DBIx/SearchBuilder/Record.pm' line 409
$ =
DBIx::SearchBuilder::Record::__ANON__[/usr/local/share/perl/5.8.4/DBIx/SearchBuilder/Record.pm:409](ref(RT::Ticket))
called from file `/opt/rt3/lib/RT/Ticket_Overlay.pm' line 238
. = RT::Ticket::Load(ref(RT::Ticket), 1065) called from file
`/opt/rt3/lib/RT/Interface/Web.pm' line 414
$ = HTML::Mason::Commands::LoadTicket(1065) called from file
`/opt/rt3/share/html/Ticket/Display.html' line 87
. =
HTML::Mason::Commands::__ANON__[/opt/rt3/share/html/Ticket/Display.html:59]('id',
1065, 'id', 1065) called from file
`/usr/local/share/perl/5.8.3/HTML/Mason/Component.pm' line 134
. = HTML::Mason::Component::run(ref(HTML::Mason::Component::FileBased),
'id', 1065, 'id', 1065) called from file
`/usr/local/share/perl/5.8.3/HTML/Mason/Request.pm' line 1074
. = eval {...} called from file
`/usr/local/share/perl/5.8.3/HTML/Mason/Request.pm' line 1068
. = HTML::Mason::Request::comp(undef, undef, 'id', 1065, 'id', 1065)
called from file `/usr/local/share/perl/5.8.3/HTML/Mason/Request.pm'
line 760
. =
HTML::Mason::Request::call_next(ref(HTML::Mason::Request::ApacheHandler),
'id', 1065) called from file `/opt/rt3/share/html/autohandler' line 199
$ =
HTML::Mason::Commands::__ANON__[/opt/rt3/share/html/autohandler:212]('id',
1065) called from file
`/usr/local/share/perl/5.8.3/HTML/Mason/Component.pm' line 134
$ = HTML::Mason::Component::run(ref(HTML::Mason::Component::FileBased),
'id', 1065) called from file
`/usr/local/share/perl/5.8.3/HTML/Mason/Request.pm' line 1072
. = eval {...} called from file
`/usr/local/share/perl/5.8.3/HTML/Mason/Request.pm' line 1068
$ = HTML::Mason::Request::comp(undef, undef, undef, 'id', 1065) called
from file `/usr/local/share/perl/5.8.3/HTML/Mason/Request.pm' line 338
$ = eval {...} called from file
`/usr/local/share/perl/5.8.3/HTML/Mason/Request.pm' line 338
. = eval {...} called from file
`/usr/local/share/perl/5.8.3/HTML/Mason/Request.pm' line 297
$ = HTML::Mason::Request::exec(ref(HTML::Mason::Request::ApacheHandler))
called from file
`/usr/local/share/perl/5.8.3/HTML/Mason/ApacheHandler.pm' line 134
. = eval {...} called from file
`/usr/local/share/perl/5.8.3/HTML/Mason/ApacheHandler.pm' line 134
$ =
HTML::Mason::Request::ApacheHandler::exec(ref(HTML::Mason::Request::ApacheHandler))
called from file
`/usr/local/share/perl/5.8.3/HTML/Mason/ApacheHandler.pm' line 792
$ =
HTML::Mason::ApacheHandler::handle_request(ref(HTML::Mason::ApacheHandler),
ref(Apache)) called from file `/opt/rt3/bin/webmux.pl' line 145
. = eval {...} called from file `/opt/rt3/bin/webmux.pl' line 145
$ = RT::Mason::handler(ref(Apache)) called from file `/dev/null' line 0
$ = eval {...} called from file `/dev/null' line 0

Jon.

_______________________________________________
http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users

Be sure to check out the RT wiki at http://wiki.bestpractical.com

jesse at bestpractical

Aug 11, 2004, 7:04 AM

Post #2 of 6 (1171 views)
Permalink

Re: RT3 Performance Issues Update [In reply to]

On Aug 11, 2004, at 7:12 AM, Jon Masters wrote:

> Hi all,
>
> However I have forced myself to play around more with perl and have
> been using an interactive mod_perl debugging session to determine that
> there is potentially a problem in the
> DBIx::SearchBuilder::Record::Cacheable code as it likes to go and sit
> in big long loops looking for stuff that has expired from its object
> cache. Can I turn off the cacheing and just use the database directly?

Sure, but that will hurt in other ways.

> It likes running the expire code path hundreds of times, e.g.:

How long does the cache expirey take when run outside the debugger?
Small tight loops have this habit of becoming big, heavy loops when a
debugger touches them.
But you have hit on one of the places I've had flagged in the back of
my head as a possible point of pain for a while. I'd be quite happy to
see a patch for a more efficient cache expiry mechanism.

Jesse

_______________________________________________
http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users

Be sure to check out the RT wiki at http://wiki.bestpractical.com

jonathan at jonmasters

Aug 12, 2004, 5:10 AM

Post #3 of 6 (1154 views)
Permalink

Re: RT3 Performance Issues Update [In reply to]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Jesse Vincent wrote:

| On Aug 11, 2004, at 7:12 AM, Jon Masters wrote:
|
|> Hi all,
|>
|> However I have forced myself to play around more with perl and have
|> been using an interactive mod_perl debugging session to determine that
|> there is potentially a problem in the
|> DBIx::SearchBuilder::Record::Cacheable code as it likes to go and sit
|> in big long loops looking for stuff that has expired from its object
|> cache. Can I turn off the cacheing and just use the database directly?
|
|
| Sure, but that will hurt in other ways.

How would you recommend that I do this - I would like to rule out issues
with that part of SearchBuilder - documentation is somewhat light.

|> It likes running the expire code path hundreds of times, e.g.:

| How long does the cache expirey take when run outside the debugger?

I'll run this through the profiler again.

| But you have hit on one of the places I've had flagged in the back of my
| head as a possible point of pain for a while.

I think it is the cause of most of my grief.

| I'd be quite happy to see a patch for a more
| efficient cache expiry mechanism.

I'll take a look however I rarely write much perl these days and hate
object orientation in general, so it might be quite fun.

Jon.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.2 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFBG14qeTyyexZHHxERAk0VAJ9ZShX9kKA91zghWrV6tEsEXNxG4wCgkiER
asKhDUYuZQbjWDGixVZA+Zs=
=+Qx7
-----END PGP SIGNATURE-----

_______________________________________________
http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users

Be sure to check out the RT wiki at http://wiki.bestpractical.com

dimitry.faynerman at hypermediasystems

Aug 17, 2004, 7:35 AM

Post #4 of 6 (1142 views)
Permalink

RE: RT3 Performance Issues Update [In reply to]

>>However I have forced myself to play around more with perl and have been
>>using an interactive mod_perl debugging session to determine that there
>>is potentially a problem in the DBIx::SearchBuilder::Record::Cacheable
>>code as it likes to go and sit in big long loops looking for stuff that
>>has expired from its object cache. Can I turn off the cacheing and just
>>use the database directly?

So how can I turn off the cacheing?
I want to play with it and see if the performance improves

-Dimitry

_______________________________________________
http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users

Be sure to check out the RT wiki at http://wiki.bestpractical.com

jesse at bestpractical

Aug 17, 2004, 7:40 AM

Post #5 of 6 (1163 views)
Permalink

Re: RT3 Performance Issues Update [In reply to]

Have a look in RT::Record.

On Aug 17, 2004, at 3:35 PM, Dimitry Faynerman wrote:

>
>
>>> However I have forced myself to play around more with perl and have
>>> been
>>> using an interactive mod_perl debugging session to determine that
>>> there
>>> is potentially a problem in the
>>> DBIx::SearchBuilder::Record::Cacheable
>>> code as it likes to go and sit in big long loops looking for stuff
>>> that
>>> has expired from its object cache. Can I turn off the cacheing and
>>> just
>>> use the database directly?
>
>
> So how can I turn off the cacheing?
> I want to play with it and see if the performance improves
>
> -Dimitry
>
>
>
> _______________________________________________
> http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users
>
> Be sure to check out the RT wiki at http://wiki.bestpractical.com
>

_______________________________________________
http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users

Be sure to check out the RT wiki at http://wiki.bestpractical.com

dimitry.faynerman at hypermediasystems

Aug 17, 2004, 9:31 AM

Post #6 of 6 (1150 views)
Permalink

RE: RT3 Performance Issues Update [In reply to]

Yes, It helped a lot, thanks

-----Original Message-----
From: rt-users-bounces [at] lists
[mailto:rt-users-bounces [at] lists] On Behalf Of Jesse Vincent
Sent: Tuesday, August 17, 2004 7:40 AM
To: Dimitry Faynerman
Cc: rt-users [at] lists
Subject: Re: [rt-users] RT3 Performance Issues Update

Have a look in RT::Record.

_______________________________________________
http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users

Be sure to check out the RT wiki at http://wiki.bestpractical.com

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads