Mmmm...

I've just uploaded a new database which uses the flat text file. It contains around 6000 records, each with 12 fields and weighs in at over 7 megs. I've tried it and, considering I have a pretty ropey ISP which is badly overloaded, it is not too bad.

Searched for a keyword: 10 secs to come back with first records. Around 40 secs to finish displaying the first 10 records out of 166 found.

Anyone any feelings about this?

------------------
Regards,

AJ

Yea...switch to the SQL version, which will handle a lot of the processing overhead. In terms of flat text files, large files (1 MG or higher) are cumbersome to search.

Or divide up your database file into separate database files.

Regards,

------------------
Eliot Lee
Founder and Editor
Anthro TECH, L.L.C
http://www.anthrotech.com/
info@anthrotech.com
==========================
Coconino Community College
http://www.coco.cc.az.us/
Web Technology
Coordinator
elee@coco.cc.az.us

Why though? My site is non profit making and just a bit of fun. I'm already paying for the domain name and commercial hosting. I have a couple of sections that are revenue generating: Amazon and the Original art section. The first is making me nothing, the second is selling my mum's paintings.

At the moment I can't afford to buy the SQL version. Also, I haven't the first idea about SQL. I thought you had to have special software installed by your hosting company to support it. Something else that I'd probably have to pay for.

Whilst I'm providing this information for free at my own expense and the speeds are not too bad then I'll stick with what I know.

Cheers for the comment. If you tell me that I don't need anything special on my server and that a trained monkey could set up the SQK side of things then I *might* just go for it



------------------
Regards,

AJ

Well, the only solution is to use SQL. Most ISPs do have at least MYSQL installed on their servers, which allows you to run SQL scripts on it. To be honest, this is the only solution (other than breaking up your database files into smaller files) that will make your searches more efficient.

Unfortunately, there is nothing that can be done within the db.cgi to decrease search times with large files.



Regards,

------------------
Eliot Lee
Founder and Editor
Anthro TECH, L.L.C
http://www.anthrotech.com/
info@anthrotech.com
==========================
Coconino Community College
http://www.coco.cc.az.us/
Web Technology
Coordinator
elee@coco.cc.az.us

I've done the breaking the databases up with another area on the site as it lends itself to that. The new area is a bit tricky as there are no obvious categories to split the biggy up into. It's a gardening site and the first section can be split into perennials, shrubs, bulbs etc. The new one is all about plants that are edible, have medicinal uses etc. It could be broken up many ways.

The speed at the moment is pretty good and would possibly be compromised, i.e. I could split it up in the same way but then someone wanting, say, aphrodisiacs would have to go through all the categories. If I split it up by use then people wanting all annuals with any use would have the same problem.

I think I'll ask my hosting company whether they support MySQL or anything. I'm starting to think it might be a good learning process for me, if nothing else.

------------------
Regards,

AJ

You can check out the following web sites to learn more about MySQL and MSQL and other SQL programming aspects:

www.mysql.com
www.hughes.com.au/
www.sql.com
www.inquiry.com/techtips/thesqlpro/
www.sqlcourse.com

Also, at some point you might want to consider using PHP3 to connect your databases in your site. You can find out about PHP3 at the official site:

www.php3.com
www.php.net

Hope this helps.

Regards,


------------------
Eliot Lee
Founder and Editor
Anthro TECH, L.L.C
http://www.anthrotech.com/
info@anthrotech.com
==========================
Coconino Community College
http://www.coco.cc.az.us/
Web Technology
Coordinator
elee@coco.cc.az.us