I've been doing some benchmarking and the latest results I have show Links SQL capable of doing 16 queries per second, in a small database (1000 records) using a MySQL 4 DB, and a single CPU 1ghz machine. is this good compared to what other people have? I've benchmarking ASPSeek and it can do 50 queries per second in a database several times bigger (100.000) records. It is not perl, so I guess that's why it can handle loads better, but I would like Links SQL to have comparable performance to that of ASPSeeks' s 50 Queries per second. Can someone give me hints as of what else to do to enhance performance?

I've a dual Apache reverse proxy configuration, mod_perl and optimized http.conf files also the my.cnf MySQL file is optimized for high traffic, so I've already convered the 'obvious' stuff, is there any other tips on how to enhance performance? aditional fields to index? or 16 queries per second is simply the max I can achieve with my current hardware?

Thanks,

Links SQL under mod_perl should be able to do a uqery every few milliseconds depending on the type of query.

They are standard, random queries for 1 term searches, I'm able to max at 16 per second, which is not bad, but I was wondering how other Links SQL instalations perform under heavy traffic. I don't have a reference as to max capacity because I don't have numbers for other sites benchmarking results. Basically, I just want to ask for some other tips to improve performance and/or hear from other people regarding how their LSQL instalations perform.

With fast_cgi I was able to max at 20 queries per second and memory utlization was considerably lower. Can somebody comment on any tunning tips? I need to achieve 50 queries per second with LSQL but it seems nobody is interested in high performance LSQL instalations, or maybe no one has experience with high traffic sites and LSQL?

Links SQL is only as fast as your server setup. If you have mysql on a dual 2ghz athlon with 4GB RAM then I expect the number of queries Links SQL can execute will be significantly higher =)

If you have the GT libraries loaded into mod_perl (hence using Apache::DBI) then that's about as much as you can do regarding tuning Links SQL. The rest is down to the server.

Thanks Paul,

I just want to make sure I get the most out of my setup, of course that a bigger server will allow me to do more, but I'm trying to reduce the exponential growth of costs in relation to traffic.

...oh also remember to index your database.

Hi,

Is this Apache on a windows box (I'm assuming so as you are using ASP on it). If so, Apache doesn't work as well under heavy loads on Windows due to the fork model it uses. Apache 2 improves this greatly. We've benchmarked a Links SQL install that got ~ 100 hits/s. This was on a dual p3 800 machine with 1 gig of ram.

What are you using to do your benchmarks?

Cheers,

Alex
--
Gossamer Threads Inc.

Hi Alex,

No it is linux. ASPSeek it's a search engine software (A Google clone) well, maybe Google is a clone of this software don't know. Point is that I'm trying to get the most out of my current setup and was thinking on getting any aditional tips on enhancing performance. THe tool I use for benchmarking is called siege.

Ah, it's C++, I had assumed ASP. =)

One other thing to look at, is make sure you turn off KeepAlive on your backend server. That can hurt performance.

I'm suprised at the numbers though, as it looks like aspseek is still running under normal cgi (which means each request requires a fork from apache, a connect to mysql, and an execute of the program). Under a persistent environment like mod_perl, the program stays in memory all the time, and the connections to the database are reused.

Cheers,

Alex
--
Gossamer Threads Inc.

100 hits/s means to you hits as in log file lines? you mean requests like GET / blah blah blah or you mean hits as pageviews? because I can currently do 27 pageviews per second with Links SQL (thats aproximately 189 hits/sec assuming one pageview is equivalent to 7 hits.

Hi,

This was requests to just search.cgi, and did not simulate an actual user which may also load images or banners. So the numbers are a bit different.

Cheers,

Alex
--
Gossamer Threads Inc.

Thanks alex, I also checked that. keepalives are Off, I know that in a mod_perl enviroment you want to connect and discconect from the server as fastest as you can, that's why I use the reverse proxy configuration. I'm sorry to be such a pain ;) I just want to make sure I don't miss one tip... apparently I've already covered all known tips... I was expecting for insider no body knows kind of tips to enhance the peformance.

Hi,

One last tip, do you have mysql query cache enabled and check that it is working:

[root@gossamer root]# mysqladmin ext | grep Qca
| Qcache_queries_in_cache | 17828 |
| Qcache_inserts | 95381929 |
| Qcache_hits | 81775943 |
| Qcache_lowmem_prunes | 137779 |
| Qcache_not_cached | 7490457 |
| Qcache_free_memory | 113830360 |
| Qcache_free_blocks | 12157 |
| Qcache_total_blocks | 47966 |
[root@gossamer root]#

(You can see 81 million cached hits, 7 million not cached).

Cheers,

Alex
--
Gossamer Threads Inc.

Ahhh... finally ;)

Got zeros, So I guess I don't have that enabled.

Is there a document on how to optimize MySQL for LSQL? tips like the ones you are proposing here... can you share some more?

Thanks,