Mailing List Archive: Interchange: users

Is the more tag very inefficient?

Index | Next | Previous | View Threaded

ic at tvcables

Jul 31, 2013, 2:41 AM

Post #1 of 9 (52 views)
Permalink

Is the more tag very inefficient?

Hi folks,

I am trying to sort out server load problems and I have found the more pages
show approx a 10-20 fold increase in page benchmark time, this also ties up
with what I see in terms of increased server loads, is the more tag not very
efficient, I mainly use it in a query tag, I've tried permanent more as well
which seems equally as bad.

The initial page running the query has an average benchmark of 0.2-0.3
seconds, subsequent /SCAN/MM= pages benchmark at 2-4 seconds.

I am guessing the more tag is reverting to doing something like returning
every row in the products database?

Andy

_______________________________________________
interchange-users mailing list
interchange-users [at] icdevgroup
http://www.icdevgroup.org/mailman/listinfo/interchange-users

racke at linuxia

Jul 31, 2013, 2:51 AM

Post #2 of 9 (50 views)
Permalink

Re: Is the more tag very inefficient? [In reply to]

On 07/31/2013 11:41 AM, IC wrote:
> Hi folks,
>
> I am trying to sort out server load problems and I have found the more pages
> show approx a 10-20 fold increase in page benchmark time, this also ties up
> with what I see in terms of increased server loads, is the more tag not very
> efficient, I mainly use it in a query tag, I've tried permanent more as well
> which seems equally as bad.
>
> The initial page running the query has an average benchmark of 0.2-0.3
> seconds, subsequent /SCAN/MM= pages benchmark at 2-4 seconds.
>
> I am guessing the more tag is reverting to doing something like returning
> every row in the products database?
>

The problem is that the complete search result is stored in a file in tmp/.
If you don't restrict the fields with rf=, it can become large and cause
bad performance.

Also /SCAN/ is really ugly in general and not friendly to search engines.

Regards
Racke

--
LinuXia Systems => http://www.linuxia.de/
Expert Interchange Consulting and System Administration
ICDEVGROUP => http://www.icdevgroup.org/
Interchange Development Team

_______________________________________________
interchange-users mailing list
interchange-users [at] icdevgroup
http://www.icdevgroup.org/mailman/listinfo/interchange-users

ic at tvcables

Jul 31, 2013, 3:23 AM

Post #3 of 9 (50 views)
Permalink

Re: Is the more tag very inefficient? [In reply to]

> The problem is that the complete search result is stored in a file in
> tmp/.
> If you don't restrict the fields with rf=, it can become large and cause
> bad performance.
>
> Also /SCAN/ is really ugly in general and not friendly to search engines.
>
> Regards
> Racke

Hi Racke,

I added this to the query tag and it made little difference, still 10-20x
slower on the benchmark speed, in the query tag I have:-

type=list
more=1
ml="[scratch numreturn]"
pm=1
rf=sku description comment size thumb sale_price

I get the impression its returning every row though which is causing the
time delay.

Regards,
Andy

_______________________________________________
interchange-users mailing list
interchange-users [at] icdevgroup
http://www.icdevgroup.org/mailman/listinfo/interchange-users

racke at linuxia

Jul 31, 2013, 3:27 AM

Post #4 of 9 (49 views)
Permalink

Re: Is the more tag very inefficient? [In reply to]

On 07/31/2013 12:23 PM, IC wrote:
>
>> The problem is that the complete search result is stored in a file in
>> tmp/.
>> If you don't restrict the fields with rf=, it can become large and cause
>> bad performance.
>>
>> Also /SCAN/ is really ugly in general and not friendly to search engines.
>>
>> Regards
>> Racke
>
> Hi Racke,
>
> I added this to the query tag and it made little difference, still 10-20x
> slower on the benchmark speed, in the query tag I have:-
>
> type=list
> more=1
> ml="[scratch numreturn]"
> pm=1
> rf=sku description comment size thumb sale_price
>
> I get the impression its returning every row though which is causing the
> time delay.

How many matches does this search have?

Regards
Racke

--
LinuXia Systems => http://www.linuxia.de/
Expert Interchange Consulting and System Administration
ICDEVGROUP => http://www.icdevgroup.org/
Interchange Development Team

_______________________________________________
interchange-users mailing list
interchange-users [at] icdevgroup
http://www.icdevgroup.org/mailman/listinfo/interchange-users

ic at tvcables

Jul 31, 2013, 3:41 AM

Post #5 of 9 (49 views)
Permalink

Re: Is the more tag very inefficient? [In reply to]

> > I added this to the query tag and it made little difference, still 10-
> 20x
> > slower on the benchmark speed, in the query tag I have:-
> >
> > type=list
> > more=1
> > ml="[scratch numreturn]"
> > pm=1
> > rf=sku description comment size thumb sale_price
> >
> > I get the impression its returning every row though which is causing the
> > time delay.
>
> How many matches does this search have?
>
> Regards
> Racke

The query returns 209 products, more paginates them in pages of 20.

Benchmarking around the query tag alone is showing first page query/list
finishes in 0.3s, the more pages take 4s.

Regards,
Andy

_______________________________________________
interchange-users mailing list
interchange-users [at] icdevgroup
http://www.icdevgroup.org/mailman/listinfo/interchange-users

racke at linuxia

Jul 31, 2013, 3:45 AM

Post #6 of 9 (50 views)
Permalink

Re: Is the more tag very inefficient? [In reply to]

On 07/31/2013 12:41 PM, IC wrote:
>>> I added this to the query tag and it made little difference, still 10-
>> 20x
>>> slower on the benchmark speed, in the query tag I have:-
>>>
>>> type=list
>>> more=1
>>> ml="[scratch numreturn]"
>>> pm=1
>>> rf=sku description comment size thumb sale_price
>>>
>>> I get the impression its returning every row though which is causing the
>>> time delay.
>>
>> How many matches does this search have?
>>
>> Regards
>> Racke
>
> The query returns 209 products, more paginates them in pages of 20.
>
> Benchmarking around the query tag alone is showing first page query/list
> finishes in 0.3s, the more pages take 4s.
>

Yeah, 209 products shouldn't hurt the performance at all. What about the
ITL inside query? There is usually a lot of things that can be improved.

Regards
Racke

--
LinuXia Systems => http://www.linuxia.de/
Expert Interchange Consulting and System Administration
ICDEVGROUP => http://www.icdevgroup.org/
Interchange Development Team

_______________________________________________
interchange-users mailing list
interchange-users [at] icdevgroup
http://www.icdevgroup.org/mailman/listinfo/interchange-users

ic at tvcables

Jul 31, 2013, 6:46 AM

Post #7 of 9 (50 views)
Permalink

Re: Is the more tag very inefficient? [In reply to]

> Yeah, 209 products shouldn't hurt the performance at all. What about the
> ITL inside query? There is usually a lot of things that can be improved.
>
> Regards
> Racke
>

Hi Racke,

Stripping the code in the query tag does speed things up but I still see the
same 10-20x processing speed difference between the first page and scan
pages, to test it I just put this in the query tag:-

[list]
[sql-param description]<br>
[/list]
[more_list]
Matches [matches] of [match-count] shown.<BR>
[more]
[/more_list]

Query benchmark times are 0.01 to 0.02 seconds on the first page, 0.22-0.3
seconds on the scan pages.

I haven't tried to work out exactly how the more tag works, maybe Mike could
comment, I wonder if it is returning every table row rather than just those
needed?

It would seem more efficient to try and use the sql limit if this is the
case?

Any help appreciated.

Regards,
Andy

_______________________________________________
interchange-users mailing list
interchange-users [at] icdevgroup
http://www.icdevgroup.org/mailman/listinfo/interchange-users

mike at perusion

Jul 31, 2013, 6:55 AM

Post #8 of 9 (50 views)
Permalink

Re: Is the more tag very inefficient? [In reply to]

Quoting IC (ic [at] tvcables):
> > Yeah, 209 products shouldn't hurt the performance at all. What about the
> > ITL inside query? There is usually a lot of things that can be improved.
> >
> > Regards
> > Racke
> >
>
> Hi Racke,
>
> Stripping the code in the query tag does speed things up but I still see the
> same 10-20x processing speed difference between the first page and scan
> pages, to test it I just put this in the query tag:-
>
> [list]
> [sql-param description]<br>
> [/list]
> [more_list]
> Matches [matches] of [match-count] shown.<BR>
> [more]
> [/more_list]
>
> Query benchmark times are 0.01 to 0.02 seconds on the first page, 0.22-0.3
> seconds on the scan pages.
>
> I haven't tried to work out exactly how the more tag works, maybe Mike could
> comment, I wonder if it is returning every table row rather than just those
> needed?
>
> It would seem more efficient to try and use the sql limit if this is the
> case?

Yes, it would be more efficient. The code was written in 1997, before
standard, reliable, low-cost SQL was readily available for Linux. Many
of us have done SQL-based versions of such searches, but no one has
put in the effort to integrate it into the core.

It would be pretty easy to produce a Vend::MySQLSearch module that did that
type of thing. The hard part, of course, is making it production quality.

--
Mike Heins
Perusion -- Expert Interchange Consulting http://www.perusion.com/
phone +1.765.253.4194 ... Ask me about jobs ...

Nothing is foolproof to a sufficiently equipped fool. -- unknown

_______________________________________________
interchange-users mailing list
interchange-users [at] icdevgroup
http://www.icdevgroup.org/mailman/listinfo/interchange-users

ic at tvcables

Jul 31, 2013, 7:05 AM

Post #9 of 9 (50 views)
Permalink

Re: Is the more tag very inefficient? [In reply to]

> Yes, it would be more efficient. The code was written in 1997, before
> standard, reliable, low-cost SQL was readily available for Linux. Many
> of us have done SQL-based versions of such searches, but no one has
> put in the effort to integrate it into the core.
>
> It would be pretty easy to produce a Vend::MySQLSearch module that did
> that
> type of thing. The hard part, of course, is making it production quality.

Hi Mike,

It wouldn't be so easy for me to do though :-(

Do you have any on-page or usertag code you could share that would paginate
a query with an sql limit?

I could cobble something together but it will take me ages...

Regards,
Andy.

_______________________________________________
interchange-users mailing list
interchange-users [at] icdevgroup
http://www.icdevgroup.org/mailman/listinfo/interchange-users

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads