Gossamer Forum: Products: Gossamer Links: Pre Sales: How many static detailed pages?

May 18, 2003, 1:13 AM

mrrob

Novice (32 posts)

May 18, 2003, 1:13 AM

Post #1 of 12

Shortcut

How many static detailed pages?

Hi,

I'm planning on building a directory with 50000 - 200000 links. I want to use static pages for better search engine placement, including detailed pages. Given that the detailed pages appear to be created in a single directory, is there a maximum number of links/detailed pages? (either a Links SQL or Linux limit?) Also, is there a point at which too many detailed pages in one directory might start to degrade performance?

Kind Regards

Rob

May 19, 2003, 1:50 AM

Andy

Veteran / Moderator (18441 posts)

May 19, 2003, 1:50 AM

Post #2 of 12

Shortcut

Re: [mrrob] How many static detailed pages? In reply to

The only limitation really is your server. If you have a very high spec machine, with mod_perl and all the other extras that can go on servers; then 50,000 to 200,000 static pages will not be a problem. The building time may be quite long with that many pages, so you may also not want to build the pages too often, or you will see quite a performance drop.

Cheers

Andy (mod)
andy@ultranerds.co.uk

May 19, 2003, 1:54 AM

Paul

Veteran (19537 posts)

May 19, 2003, 1:54 AM

Post #3 of 12

Shortcut

Re: [Andy] How many static detailed pages? In reply to

Quote:

If you have a very high spec machine, with mod_perl and all the other extras that can go on servers; then 50,000 to 200,000 static pages will not be a problem.

How is mod_perl going to affect static pages? Wink

May 19, 2003, 2:06 AM

Andy

Veteran / Moderator (18441 posts)

May 19, 2003, 2:06 AM

Post #4 of 12

Shortcut

Re: [Paul] How many static detailed pages? In reply to

good point :p It wouldn't really speed up the build process, but I suppose it would take some of the strain off other things, like search.cgi, jump.cgi etc.

Cheers

Andy (mod)
andy@ultranerds.co.uk

May 20, 2003, 8:19 AM

mrrob

Novice (32 posts)

May 20, 2003, 8:19 AM

Post #5 of 12

Shortcut

Re: [Andy] How many static detailed pages? In reply to

Thanks for the advice chaps.

Cheers

Rob

May 21, 2003, 8:50 PM

klauslovgreen

Enthusiast (671 posts)

May 21, 2003, 8:50 PM

Post #6 of 12

Shortcut

Re: [mrrob] How many static detailed pages? In reply to

Hi Rob,

I have about 200,000 links in my installation and run the whole thing in dynamic mode - and with re-write rules you can get the same SE effect as static pages..

Just a thought

Cheers
Klaus

http://www.ameinfo.com

Jun 6, 2003, 5:32 PM

brakkar

User (103 posts)

Jun 6, 2003, 5:32 PM

Post #7 of 12

Shortcut

Re: [mrrob] How many static detailed pages? In reply to

I have suggested to gossamer that the statc page build process be staggered, lets say every 300-500 pages.

Jun 7, 2003, 2:43 AM

Andy

Veteran / Moderator (18441 posts)

Jun 7, 2003, 2:43 AM

Post #8 of 12

Shortcut

Re: [brakkar] How many static detailed pages? In reply to

Have you tried 'Staggered Build' ? Wink

Cheers

Andy (mod)
andy@ultranerds.co.uk

Jun 7, 2003, 4:41 AM

brakkar

User (103 posts)

Jun 7, 2003, 4:41 AM

Post #9 of 12

Shortcut

Re: [Andy] How many static detailed pages? In reply to

Yes, but rigt now, it doesn't split the build process of tha detaied page, thus my suggestion....

Brakkar

Jun 7, 2003, 4:47 AM

Andy

Veteran / Moderator (18441 posts)

Jun 7, 2003, 4:47 AM

Post #10 of 12

Shortcut

Re: [brakkar] How many static detailed pages? In reply to

Sorry, didn't realise you were refering to details builds being split Blush

Cheers

Andy (mod)
andy@ultranerds.co.uk

Jun 16, 2003, 11:53 AM

webslicer

User (236 posts)

Jun 16, 2003, 11:53 AM

Post #11 of 12

Shortcut

Re: [mrrob] How many static detailed pages? In reply to

There is a solution to the problem of building large amounts of static pages in a single directory.

The problem occurs in most operating systems when the number of files ()pages) per directory exceed what the os can handle without slowing down progressively.

Why the slow down? The slow down happens because the directory file structure holds only a set number of files, and then the os has to make another piggybacked structure, linked from the previous and so on. When a file is looked up, each structure in turn is checked to locate the file...

The solution that I use, is to buy the Pagebuilder Plugin, and use it to build your detailed pages, you can specify the directory to build them in by setting conditions, or program Pagebuilder to use the whole links directory path...

There have been other posts on this, I posted a solution requiring a PageBuilder hack to use the Title in the build directory field with spaces turned into underlins... look up some Pagebuilder posts!

Here's one "dumb" way you could specify Pagebuilder to build to a separate directory per 10,000 links. Lets say you have 100,000 links.
You would make 10 PageBuilder templates/pages.
First one would have a build condition of ID<10,001
the 2nd would build between 10,001 and 20,000, etc.

I do NOT know the SQL command to select a range,
where build is true between x and y but expect that something would work...

(ID > x) AND (ID < y)

If someone could help out there, I'd like to know myself!!

Good Luck!

I think there is another way, using hashed indexes possibly, but am not sure if it works for detailed pages!

Jun 17, 2003, 8:22 AM

cwi

User (56 posts)

Jun 17, 2003, 8:22 AM

Post #12 of 12

Shortcut

Re: [webslicer] How many static detailed pages? In reply to

There might perhaps be an easier way than the one suggested:

http://www.gossamer-threads.com/t/232420/