curtisb at vianet
Jul 23, 2009, 6:37 AM
Post #3 of 3
(1082 views)
Permalink
|
I also have this issue but have worked around it by keeping the searches
small. We have a lot of tickets and large searches will basically have
apache grow until it runs out of memory after you click display. I have
reported it before but wasn't able to detail it like this, hopefully
there is a workable solution for it.
Bram wrote:
> Hi,
>
>
> When a large search is done and the user clicks the link to display
> the ticket then apache starts using a lot more memory.
> This is due to a memory 'leak' in share/html/Ticket/Display.html:
>
> http://github.com/bestpractical/rt/blob/90546414c8887e344f995da5f20f53a58939bbd8/share/html/Ticket/Display.html
> lines 200-206:
>
> if (defined $session{'tickets'} and ($ARGS{'Query'} or
> $session{'CurrentSearchHash'}->{'Query'})) {
> my $item_map = $session{'tickets'}->ItemMap;
> $link_rel{first} = "Ticket/Display.html?id=" . $item_map->{first}
> if $item_map->{$TicketObj->Id}{prev};
> $link_rel{prev} = "Ticket/Display.html?id=" .
> $item_map->{$TicketObj->Id}{prev} if $item_map->{$TicketObj->Id}{prev};
> $link_rel{next} = "Ticket/Display.html?id=" .
> $item_map->{$TicketObj->Id}{next} if $item_map->{$TicketObj->Id}{next};
> $link_rel{last} = "Ticket/Display.html?id=" . $item_map->{last}
> if $item_map->{$TicketObj->Id}{next};
> }
>
>
> This code will execute the query in $session{'tickets'} and store the
> results of the search in $session{'tickets'}->....
> Meaning: the session object now contains the query + the results of
> the search. Which obviously results in a lot higher memory usage when
> the search is large.
>
> (When I run a search which returns 13000 tickets apache jumps from
> 40/60 mb of memory to 200mb of memory.
> Since it is in the session it also means every later request from the
> same session will result in another process using 200mb of memory.)
>
>
> I modified the code a bit and added Dumper($session{tickets});
> I then did a search for 2 tickets. (truncated Dumper output at the end).
>
> A (truncated) diff from the dumper output:
>
> (- is before, + is after)
> - 'must_redo_search' => 1,
> + 'must_redo_search' => 0,
>
> + 'items_array' => [],
>
> + 'dbix_sb_unique_cache' => {
> + '37737' => 1,
> + '39447' => 1
> + },
>
> + 'item_map' => {
> + '37737' => {
> + 'next' => '39447',
> + 'prev' => 0,
> + 'defined' => 1
> + },
> + 'first' => '37737',
> + '39447' => {
> + 'prev' => '37737',
> + 'defined' => 1
> + },
> + 'last' => '39447'
> + },
>
> + 'items' => [.
> + bless( {
> + '_Class' => 'RT::Ticket',
> + 'original_user' => undef,
> +
> '_SB_Record_Primary_RecordCache_key' => 'id=37737',
> + 'user' => $VAR1->{'user'},
> + 'table' => 'Tickets',
> + 'values' => {
> + # [snipped]
> + },
> + 'fetched' => {
> + # [snipped]
> + }
> + }, 'RT::Ticket' ),
> + bless( {
> + '_Class' => 'RT::Ticket',
> + 'original_user' => undef,
> +
> '_SB_Record_Primary_RecordCache_key' => 'id=39447',
> + 'user' => $VAR1->{'user'},
> + 'table' => 'Tickets',
> + 'values' => {
> + # [snipped]
> + },
> + 'fetched' => {
> + # [snipped]
> + }, 'RT::Ticket' )
> + ]
>
>
> This means that after the display it contains extra data in items,
> item_map and dbix_sb_unique_cache.
> Clearing items is simple and can be done by adding:
> $session{'tickets'}->PrepForSerialization();
>
> in the code (which is what Search/Results.html calls).
>
>
> But that still leaves item_map and dbix_sb_unique_cache in the session.
> Any advice on clearing those?
>
>
> Best regards,
>
> Bram
>
>
>
> (Truncated) Dumper output:
>
> [Thu Jul 23 12:49:59 2009] [info]: BEFORE
> (/opt/rt3/local/html/Ticket/Display.html:223)
> [Thu Jul 23 12:49:59 2009] [info]: $VAR1 = bless( {
> '_open_parens' => {},
> 'alias_count' => 0,
> 'table' => 'Tickets',
> 'where_clause' => '',
> 'order_by' => [.
> {
> 'FIELD' => 'id',
> 'ORDER' => 'ASC'
> }
> ],
> 'RecalcTicketLimits' => 0,
> '_sql_trattachalias' => undef,
> '_sql_object_cfv_alias' => undef,
> 'is_limited' => 1,
> 'order' => '',
> '_sql_watcher_join_users_alias' => undef,
> 'user' => bless( {
> # [ snipped ]
> }, 'RT::CurrentUser' ),
> 'restriction_index' => 1,
> '_sql_query' => 'id = 37737 OR id = 39447',
> '_sql_transalias' => undef,
> 'DBIxHandle' => bless( {
> 'dsn' =>
> 'dbi:Pg:dbname=ayuda;host=db',
> 'StatementLog' => [],
> 'DisconnectHandleOnDestroy' => undef
> }, 'RT::Handle' ),
> 'limit_clause' => '',
> 'restrictions' => {
> 'main.Status' => [.
> {
> 'value' =>
> '\'deleted\'',
> 'field' =>
> 'main.Status',
> 'op' => '!='
> }
> ],
> 'ticketsql' => [.
> {
> 'value' =>
> '\'37737\'',
> 'field' => 'main.id',
> 'op' => '='
> },
> 'OR',
> {
> 'value' =>
> '\'39447\'',
> 'field' => 'main.id',
> 'op' => '='
> }
> ],
> 'main.EffectiveId' => [.
> {
>
> 'value' => 'main.id',
>
> 'field' => 'main.EffectiveId',
> 'op' => '='
> }
> ],
> 'main.Type' => [.
> {
> 'value' =>
> '\'ticket\'',
> 'field' =>
> 'main.Type',
> 'op' => '='
> }
> ]
> },
> 'primary_key' => 'id',
> 'count_all' => '2',
> 'must_redo_search' => 1,
> 'looking_at_effective_id' => 0,
> 'itemscount' => 0,
> 'show_rows' => 0,
> '_sql_cf_alias' => undef,
> 'left_joins' => {},
> 'aliases' => [],
> 'subclauses' => {
> 'generic_restrictions' =>
> '(main.Status != \'deleted\') AND (main.id = \'37737\' OR main.id =
> \'39447\') AND (main.Type = \'ticket\') AND (main.EffectiveId =
> main.id)'
> },
> 'first_row' => 0,
> 'looking_at_type' => 0,
> '_sql_looking_at' => {
> 'id' => 1
> }
> }, 'RT::Tickets' );
> (/opt/rt3/local/html/Ticket/Display.html:224)
>
>
> [Thu Jul 23 12:50:00 2009] [info]: AFTER
> (/opt/rt3/local/html/Ticket/Display.html:233)
> [Thu Jul 23 12:50:00 2009] [info]: $VAR1 = bless( {
> '_open_parens' => {},
> 'table' => 'Tickets',
> '_sql_trattachalias' => undef,
> '_sql_object_cfv_alias' => undef,
> 'is_limited' => 1,
> 'order' => '',
> '_sql_watcher_join_users_alias' => undef,
> 'user' => bless( {
> # [ snipped ]
> }, 'RT::CurrentUser' ),
> '_sql_query' => 'id = 37737 OR id = 39447',
> 'restrictions' => {
> 'main.Status' => [.
> {
> 'value' =>
> '\'deleted\'',
> 'field' =>
> 'main.Status',
> 'op' => '!='
> }
> ],
> 'ticketsql' => [.
> {
> 'value' =>
> '\'37737\'',
> 'field' => 'main.id',
> 'op' => '='
> },
> 'OR',
> {
> 'value' =>
> '\'39447\'',
> 'field' => 'main.id',
> 'op' => '='
> }
> ],
> 'main.EffectiveId' => [.
> {
>
> 'value' => 'main.id',
>
> 'field' => 'main.EffectiveId',
> 'op' => '='
> }
> ],
> 'main.Type' => [.
> {
> 'value' =>
> '\'ticket\'',
> 'field' =>
> 'main.Type',
> 'op' => '='
> }
> ]
> },
> 'primary_key' => 'id',
> 'count_all' => '2',
> 'items_array' => [],
> 'must_redo_search' => 0,
> 'looking_at_effective_id' => 0,
> 'subclauses' => {
> 'generic_restrictions' =>
> '(main.Status != \'deleted\') AND (main.id = \'37737\' OR main.id =
> \'39447\') AND (main.EffectiveId = main.id) AND (main.Type =
> \'ticket\')'
> },
> '_sql_looking_at' => {
> 'id' => 1
> },
> 'alias_count' => 0,
> 'dbix_sb_unique_cache' => {
> '37737' => 1,
> '39447' => 1
> },
> 'order_by' => [.
> {
> 'FIELD' => 'id',
> 'ORDER' => 'ASC'
> }
> ],
> 'where_clause' => '',
> 'RecalcTicketLimits' => 0,
> '_sql_transalias' => undef,
> 'restriction_index' => 1,
> 'item_map' => {
> '37737' => {
> 'next' => '39447',
> 'prev' => 0,
> 'defined' => 1
> },
> 'first' => '37737',
> '39447' => {
> 'prev' => '37737',
> 'defined' => 1
> },
> 'last' => '39447'
> },
> 'limit_clause' => '',
> 'DBIxHandle' => bless( {
> 'dsn' =>
> 'dbi:Pg:dbname=ayuda;host=db',
> 'StatementLog' => [],
> 'DisconnectHandleOnDestroy' => undef
> }, 'RT::Handle' ),
> 'itemscount' => 0,
> '_sql_cf_alias' => undef,
> 'show_rows' => 0,
> 'left_joins' => {},
> 'aliases' => [],
> 'first_row' => 0,
> 'looking_at_type' => 0,
> 'items' => [.
> bless( {
> '_Class' => 'RT::Ticket',
> 'original_user' => undef,
>
> '_SB_Record_Primary_RecordCache_key' => 'id=37737',
> 'user' => $VAR1->{'user'},
> 'table' => 'Tickets',
> 'values' => {
> # [snipped]
> },
> 'fetched' => {
> # [snipped]
> }
> }, 'RT::Ticket' ),
> bless( {
> '_Class' => 'RT::Ticket',
> 'original_user' => undef,
>
> '_SB_Record_Primary_RecordCache_key' => 'id=39447',
> 'user' => $VAR1->{'user'},
> 'table' => 'Tickets',
> 'values' => {
> # [snipped]
> },
> 'fetched' => {
> # [snipped]
> }
> }, 'RT::Ticket' )
> ]
> }, 'RT::Tickets' );
> (/opt/rt3/local/html/Ticket/Display.html:234)
>
>
>
> _______________________________________________
> List info: http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-devel
>
>
>
_______________________________________________
List info: http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-devel
|
