Gossamer Forum
Home : Products : Gossamer Links : Version 1.x :

Build problems - nph-build.cgi

Quote Reply
Build problems - nph-build.cgi
Hello Alex!

I am not able to build with 300 categories and 9000 links. Earlier I could not build with 4600 with 10 links!!!

I commented out all but &build_category_pages. Playing with the routine in there, I could simply change the select category from range and build only that perticular category. So I could acheive build the category.

Problem : There is foreach loop of $category_r that will go around each category and make one index.html! If I can stop this than I can make few copies of the subroutines and builds staggered and category only. I would like to know how to stop this foreach loop and change it to $subcategory_r instaed.

If this is done then one can modify with a foreach of the $subcategory_r instead of looping through the main category. Thereafter, it would be no problem to build the mainpages instead of Homepage. Ofcourse the problem there would be, ofcourse the links on the index.html.

Currently it gives me timeout problems. When I run nph-verify.cgi, it did not give me timeout problem and went through all the links check. But nph-build.cgi does not work at all. All above is ofcourse regardless of versions. I tried all of them.

An interesting thing I noted that nph-builds 1.1b1/2 make wrong calculation and 1.1b3 did correct. This was a splendid surprise indeed how different they worked and that there were bugs in there speaking volumes of improvements.

Above, to make it clear, I am trying to look for a solution of timeout problems and without this I am not able to do anything. Thanks..........

Quote Reply
Re: Build problems - nph-build.cgi In reply to
The only solution would be to run this from telnet, a cron job, or to see how far Alex has come on the staggered build and/or build changed routines.

The time out is set by your server for cgi processes. If you trip it by telnet, or cron you will not time out (or shouldn't) until the job is done.

Then, again, some ISP's terminate any job running for more than x-minutes. There may be no way around that, except to find a new ISP.

My feeling, and it's a very, very valid one, is that if you are trying to run a business, you need to find -- AND PAY FOR -- the hosting services you need. I am against free hosts for any sort of business process, and usually do not visit, or back out of immediately any links to hypermart, geocities, etc. This is very common.

I'm not saying that applies to you, but rather than try to get the program to over come short comings in hosting, the solution is better hosting.

LinkSQL is not for the casual user. Many might want to use it that way, but when you bump into limitations in hosting or server performance don't blame LinkSQL.

I made my business plan, bought my server, my bandwidth and the support I needed to make it run. I never considered free hosting -- my only decision was co-locate or lease.

I realize there are problems in individual situations, but some of them YOU are obligated to try to overcome, and not expect Alex to do it for you.

He is working on some solutions. But the REAL solution for you may be better hosting, and a couple of perl/cgi programmers on your staff.

Quote Reply
Re: Build problems - nph-build.cgi In reply to
Hello Mr. Pataki!

Thanks for your response.

1.
I would appreciate to know how to build by telnet. May be I could try.

2.
For your information, I have paid more than thousand dollars (Stupid of me) to my provider for hosting under a special offer a year ago for some services and the contract expires next summer. Only after this I plan to have my own server. At the moment I do not have any extra expectation from the server nor plan to make Links SQL my main source or even a part of my income, nor ever in the future!!! So lets not get in to such personal discussions. Lets be constructive and help each other, if possible. No one stops someone NOT answering a message!

It is the script that has problems and not the server. Server ofcourse has timeout limitations after x-minites as you correctly pointed out. Links SQL will build one thousand pages in one minite!!! At the moment it goes though the entire database for everthing, for even one new links, and it is that which has given timeout problems.

Currently, I am only looking for a rescue and temporarly solution, if possible. If it is not possible, I may have to put this entire Links SQL project on ice and in connection to the problems I think I have had enough. Alex mentioned somewhere, that he intends to work on "New only build" and this will come some day, anyway and it is in the list of things to do, I beleive.

In my opinion a developer, regardless of Alex -but saying in general, cannot see his development of the product limited to constratints like - Only working on a dedicated machines, Only for advanced users, etc. Further what is custom and what not is only a different view point and will differ with different user, which is very subjective. So Pugdog, lets never get into discussions of what is not or is custom requirements or subjective discussions.
Quote Reply
Re: Build problems - nph-build.cgi In reply to
I would suggest running the script from telnet as Pugdog suggested. That is as easy as telneting into the server, and typing:

./nph-build.cgi

at the prompt (you must be in the admin directory). One word of caution though, the script will now be run as your own userid, not the web server's userid. You may run into permission problems. If this is the case, you should erase the pages directory, and then run nph-build.cgi.

Hope this helps,

Alex
Quote Reply
Re: Build problems - nph-build.cgi In reply to
rajani,

$1000 for a year of hosting is nothing. PER-MONTH is a good starting point for a working business with the CPU, bandwith and CGI access you need.

In order to overcome many of the problems of free-hosting, and low-power hosting, you end up bloating the code. Every program is going to have minimum requirements, and different capabilities on each different platform level.

One thing is time of a running process. The more power a machine has, the faster it will accomplish the task. What takes an hour on a 486 with 32 meg of ram running NT, may take 5 minutes on a Sparc or Pentium Unix box with 256 meg of RAM.

Optimized systems could see striking performance differences.

Dedicated systems running under mod_perl would be significantly faster than shared or virtual systems running without it.

To say a program should run under every platform, and in every situation is unrealistic. Your situation seems to be one of those that is very problematic.

I've done extensive looking, testing, and
evaluating. There simply isn't anything out there of the quality or flexibility of LinkSQL (or even links 2.0). Everything else has been designed with a single idea in mind, or has tried to be everything. Also the support on this program and the number of people offering solutions cannot be equaled.

As early as June, it was obvious LinkSQL was not working for you. You've probably hung with it for the reasons I've stated above -- There IS NOTHING else out there.

The advantage of buying-in early is that you get to put your input in for additions, changes and features that are missing or need improvement or changes. Many of the things we have ALL suggested have been incorporated, and others are being worked on.

What you need because of the client-server nature of the web is not a trivial thing to work into the whole of LinkSQL. There are problems with the category selection process, but much of that is due to the client-server stateless nature of the web. There are quite a few different solutions, but anything that is not "pure perl" will not run for everyone -- Netscape tends to crash on certain java applications, and MSIE behaves strangely on others.

As the program matures, some things will become more obvious, and certain solutions will make themselves more of a contender than others.

Quote Reply
Re: Build problems - nph-build.cgi In reply to
Hello Alex!

Yes it seems to work more than before. I will let you know. The process gets killed after lets say five to seven minites.

Now you were correct, I had permission problems. So once the pages are created by the telnet, I am not able to even try through netscape. Permission problems. Just for the test matter, I changed build_dir_per to different values but no change. Pages build with help of tenlet cannot be accessed by the admin.cgi for nph-build.cgi! Can this be true or have I to try something more?

[This message has been edited by rajani (edited November 29, 1999).]