Gossamer Forum
Home : Products : Links 2.0 : Discussions :

Would this help large databases?

Quote Reply
Would this help large databases?
I had a thought the other day that might possibly be an improvement when databases get too big (as opposed to switching to SQL). Let's say 2,000 links is a rough cutoff for where performance drops (just a guess). Is there any reason the program couldn't be set up to cap individual databases at 2,000 (could be configurable to users' preferences) and then start on b,c,d,etc. databases?

So, instead of a link being #102, it might be a102 or b1345, or which ever subset it falls in. It might complicate the admin processes, but I can't think of any reason it would add difficulties for surfers. Seems like it would speed things up significantly.

Of course, at close to 1,300 links my installation has shown no signs of slowing down...

Dan

------------------
My links are at: http://www.dankaplan.com/runninglinks
Quote Reply
Re: Would this help large databases? In reply to
Dan,

Unfortunately, this would not speed up searches. If you have 10,000 records in a flat file database, when you do a search, you still have to search 10,000 lines of text, whether they are in one text file or five text files. Breaking them up may even slow it down a tad by having to open and close the extra files.

I'm no expert, but the way I understand it, SQL is much faster because when you search in a flat file DB, in order to find the field to search, each character in each line has to be checked to see if it is a delimiter. If it is, then it is counted until the right number of delimiters is reached that matches the field to be searched. This is done on every line of the file.

With SQL, the fields are indexed, so it doesn't have to be found each and every time. Technically, I have no idea how this is done, but it is.

Hope this answers your question,

Joe
Quote Reply
Re: Would this help large databases? In reply to
Hmm, I can see how it would slow down searches. I was thinking more along the lines of less time for looking up jump URL's, adding/modifying/deleting records, etc. But since the pages are static HTML, there's no speed to be gained there and the searching probably becomes the main factor. Oh well, I guess not all my ideas can be brilliant. Smile

Dan