Gossamer Forum
Home : Products : Gossamer Links : Version 1.x :

How big can my Links 2.0 DB be until I have to upgrade to SQL

Quote Reply
How big can my Links 2.0 DB be until I have to upgrade to SQL
Hello,

Well the subject says it all. How big can my links 2.0 DB be before I have to upgrade to Links SQL?

Thanks.

------------------
James L. Murray
VirtueTech, Inc.
www.virtuetech.com


Quote Reply
Re: How big can my Links 2.0 DB be until I have to upgrade to SQL In reply to
That depends on your specific application.

I needed to upgrade with only 400 links, other people can get by with with many, many more.

If your links are short, and you have little activity, and you use direct linking rather than jump.cgi you can grow a pretty large database.

I think the question is more not 'how big' can it be before you upgrade, but really 'how long can I afford NOT to.'

If you have an active, growing site, with a lot of visitors and additions or changes then SQL will make a big difference to you. If you are just compiling a list of URL's and not much is going on, then Links 2.0 is probably fine for the long run.

What will probably happen is some benchmarks will become obvious between the versions, and people will start pointing out other reasons to upgrade.

FWIW ... I still have my Links 2.0 site as the main, but I haven't added to it in over 3 weeks because it's really plumped to capacity for performance with 400 links IMHO. I've split the site, and have LinkSQL running for the newer area, since it has less modifications, and the difference in performance is striking. My beta site for the old area is going to go live any day now, and the performance difference there is striking too ---

But, the most amazing difference is how easy it is to affect mods and additions once you get the hang of it. There are no file-handling routines to worry about since everything is just another call to the SQL database with a different table name. Data from one area of the program -- or even other databases (programs, applications, etc) are available in a single call (well, 2 lines).

The amount of data a MySQL table can hold is at _least_ 1 million lines, 40 fields and I forget how many bytes. That's only ONE table, not the whole database. And that is only what has been known to work solidly for 2+ years. Compare that to the data corruption people have had with various flat-file add ons with only a few hundred records!

The access to the MySQL database is much faster, and it handles the queuing of the requests. All of that is optimized in a very fast C code. The more requests you have for information, the more speed difference you will see between Link 2.0 and LinkSQL in terms of number of requests per unit time handled -- and thus, better response on your server over all.

50 to 100 requests per minute to the SQL engine is a reasonable load on a good machine -- how well do you think your server would handle 50-100 requests per minute for links on Links 2.0 (I'm not talking about the HTTPD serving PAGES, I'm talking about CGI requests for a link -- search, etc) if your database was more than 150 or 200k in size?? Just making those requests would kill your server even if it had 500 Meg of RAM.

With SQL, those requests are queued, and handled sequentially (perhaps simultaneously depending on the engine and the access) but you are not slurping the database into memory each time, reading through it and parsing out the data. You are requesting it from the SQL engine, which efficiently handles it, and passes it back.

The MySQL site has some really good performance data.

Moving from Link 2.0 to LinkSQL will shock you.

Granted, Link 2.0 is a more mature product, but it's also limited.

I'll NEVER (well, never say never) write another non-SQL database type application again. There is no reason to. The suffering you go through trying to make flat-files behave is eliminated with SQL.

NOTE: I'm using SQL to refer to both the query and the engine, and specifically in this case to MySQL... I wouldn't touch M$-SQL and ODBC. Why? How many sites do you visit that give you "ODBC error xx in module xxx.asp" or "Configuration error" ?? How many times do you see something like that from a Unix based system? I've started to collect M$ error messages, just to tally them. I've gotten one Unix message, and that was "Connection refused, over 300 connections already" -- which I think is legitimate, since they were probably running on an Intel box Wink


Quote Reply
Re: How big can my Links 2.0 DB be until I have to upgrade to SQL In reply to
Thanks for your response. Smile

What modifications are available for SQL that are available in Links 2.0?

I need the following:
1. Password to modify
2. Upload a file
3. Review.cgi 2 (Which Widgetz has coming)
4. Field Sorting
5. 3 types of New Graphics
6. I need to be able to choose which link.html, modify.html, or category.html template the system uses depending on the category.
7. Link Expirator
8. Recip Link Checker
9. Email Password
10. Add Confirmation
11. Report Dead Link Mod
12. Recommend It Mod
13. SubCategories Like Yahoo

The rest of the changes are simple. Is SQL have its own lanuage, or is everything still written n PERL. (Sorry if that question is a dumb one.) Smile


------------------
James L. Murray
VirtueTech, Inc.
www.virtuetech.com


Quote Reply
Re: How big can my Links 2.0 DB be until I have to upgrade to SQL In reply to
Hello,

I just purchased Links SQL! Wink

I'm excited to say the least. I hope I can mod it so that its works like my links 2.0.

I wonder If I can use Widgetz's Sort mod to that I can dynamically sort the DB.

------------------
James L. Murray
VirtueTech, Inc.
www.virtuetech.com


Quote Reply
Re: How big can my Links 2.0 DB be until I have to upgrade to SQL In reply to
What might be useful is a help module that links into Links. (no pun intended)

I've been meaning to write an add-on object that provides context help, provides a centralized interface to plunder the forum & resource section..., and also downloads update packages (so you can search the basic help files locally).

Problem is, I don't have the time. If anyone wants to write such a thing, I can provide as much support as I can with time constraints.

kitsune@canada.com
Quote Reply
Re: How big can my Links 2.0 DB be until I have to upgrade to SQL In reply to
Hello..

Many of the are built in, or the hooks are there to make it easy to do.

I've also volunteered to run an FAQ here for Link SQL, so that I'll be compiling the best help, hints and tricks into an easy to use and find format.
Quote Reply
Re: How big can my Links 2.0 DB be until I have to upgrade to SQL In reply to
There is a subcategories like yahoo for Links SQL.. except it's almost the same as Links 2.0.. i am gonna change it so that it uses a different field.. but that is not neccessary right now..

review.cgi v2 will be out a day or 2 after 1.1 is released..

i also have another mod coming.. i dunno what it's gonna be called.. but i imagine something like 'My Links'.. it's already been made twice for links 2.0 (both by me..) but those are using cookies.. i plan to make this one using user.cgi allowing the user to have a dynamically personalized main page.. with bookmarks and some other stuff.. sorta based on how your database structure is..

for instance.. my site has it's own seperate DB for software.. there are 4 different types of software and 2 different types of CPU.. the user can specify that they only want these 2 types and 1 CPU listed so they can download software that only they want..

just think of it that way.. also.. by adding fields to the User Database.. you can do weather and stuff.. but that's too easy.. so i won't go through how to implement that Smile

jerry
Quote Reply
Re: How big can my Links 2.0 DB be until I have to upgrade to SQL In reply to
pugdog.. hehe.. you will never write another flatfile eh? Smile

i hate writing anything not using

Code:
use strict;
use CGI ();
use CGI::Carp qw/fatalsToBrowser/;

there is an exception though.. when i write something for links 2.0.. i always have to take off use strict.. cause there are like a million global variables in that script Wink

but use CGI is useful.. cause you no longer have to parse forms and stuff..

jerry
Quote Reply
Re: How big can my Links 2.0 DB be until I have to upgrade to SQL In reply to
I can add to that "hate" list ---

I hate writing anything for MySQL that doesn't begin with:


use Links: BSQL;

You get attached to things pretty quickly
Quote Reply
Re: How big can my Links 2.0 DB be until I have to upgrade to SQL In reply to
My understanding is that there is no point in starting with SQL if one has a really small database. Say a couple of hundred links. I would rather buy SQL, if it is not true as eventually I have to have one. There is no point in modding Links 2.0 and then upgrade and do the whole thing.

Please give some feedback.

How difficult is it to convert the current perl mods to work with SQL or does one have to write completely new ones?

[This message has been edited by socrates (edited November 26, 1999).]
Quote Reply
Re: How big can my Links 2.0 DB be until I have to upgrade to SQL In reply to
i rewrite mods all the time.. even from 1.1b1 to 1.1b2 i rewrote them.. i haven't put many mods into 1.1b3 cause i rather wait..

links 2.0 mods are won't work if you plugged it into any of the scripts in links sql.. EVEN IF YOU PLUGGED IT IN TO SOMEWHERE THAT IT WON'T BE USED.. it just gives you a software error because of use strict..

modding a links 2.0 mod to work with links sql isn't that hard.. it depends on the size of the mod actually.. if it goes through many files then it will be harder.. but if it's only in one little section of nph-build.cgi it will be much easier..

most of the links sql code is links 2.0.. just there are so many different variables and with strict it is a little different.. especially with nph-build.cgi sending variables to HTML_Templates.pm.. but it's not hard.. you just put everything into the $OUT hash Smile

jerry