Hi,

I've mentioned about nph-build.cgi 'timing out' a few times and I know there may be something coming in the upgrade (soon I hope), but this is more to satisfy my curiosity than anything else.

I wondered if it would be possible to get nph-build.cgi to count the number of categories its built, and when it hits a certain number 'sleep' for x no. of seconds, then carry on building until x number, sleep again and loop in this fashion until all the categories were built?

If so, would the 'sleep' time be considered by the server as a break from the total CPU time used, or would it still accumulate and time-out in the usual fashion?

Just curious


All the best
Shaun

No, that wouldn't affect how Unix coun'ts the process time.

The time is counted from the time the process is started or spawned, until it terminates. So, if you ran for 3 minutes, then slept for 3 minutes, then ran for 3 more minutes, your total run time would be "9" minutes, and your CPU time would be about 6 minutes (assuming full cpu usage during the 3 minute rn times).

In order to do what you want, you'd have to do what Links 2.0 does. Does part of the build, then sends a page back to the browser, which using a browser pull, initiates the next phase of the program, etc. Each of those calls to the server from the browser is a new process.

With the SQL version, you'd probably have to store status information in a temporary table, then wait for the next call, read that information, and do the next chunk of work, etc.

But, as far as running processes, there are two measures of "time". CPU time and run time. Some ISP's limit cpu to 5 minutes or less, and run time to 10-30 minutes. If you exceed either of those, your process is killed.


http://www.postcards.com
FAQ: http://www.postcards.com/FAQ/LinkSQL/

Hello!

This is not possible. We have done extensive research here on the script of Gossamer Threads. It is not a joke to buy a script and let is remain not using it.

The Provider CERTIFIED and JUSTIFIED the reason why. Alex can play with you guys trying to make his product better with cosmetic add-ons but in the fundament it still REMAINS a Developing product however a good and excellent design it may be.

Here is what it happens.

Once you start the nph-build.cgi process, you can stop it only if you have an administrator rights i.e. root. Then you can kill the process. If you start the process twice, the build, it will continue more in the process as two seperate processes. Thats what it is doing it now.

The manner it has been designed, it makes a query of the database and does a lot of things. More the categories you have more the time it needs to get into those loops. Mind you I have not seen this observation, but my provider and other programmers.

So actually it depends on the manner it is going to build, i.e. the queries of what and how and the logic of it rather than anything else. I tried to suggest in November last year before giving up the project that editor.cgi makes the primary directions and then nph-build.cgi builds it as it gets the parameters from there.

Alex wrote me in the email before a couple of months:

Sorry, I have no good news for you. There has been no other user who has build problems, who is on a shared server.

That means either he cannot make the code or is giving less importance to tthe problems, continuing more add-ons.

When I started the script, it drained 92% of the resources i.e. three nph-build.cgi were invoked. All the cgi where to do different function calls, for e.g. one would only count stats, other only pages, other only detailed. Only Build routine took about 43% resources. The ISP clearly mentioned that how can they allow a script running like this when the CPU is really draining the power and there are 100 other users. Fair enough and true.

Alex knows this and keeps the eyes shut. You buy a script and realize that you cannot use itt because it needs hundreds of dollars of server power where it could be designed the other way. It is nott only the server CPU power. It is also an intellectual logic. If you have 25 links NEW and have 150,000 links in tthe database with 4000 categories, my statistics shows thatt you would need about ONE HOUR at leats to build 25 links without the detailed pages.

The users did not object to it. Whenever I bring such a topic to this forum, Pugdog need to publish a big list of things to be done. The big list is really nice and would help me but I ask, what is the most important thing?

Check Links SQL 2.0beta topic in the Discussion forum too.

Hi!

That's pretty unfair. You either haven't been reading what we are working on, or you don't care. The new version is a complete redesign, with a new category structure and a new SQL engine. But yes, it is a developing product, we will continue to enhance and make it better, but this can be said about a lot of things: Linux, Word, Netscape Navigator, etc.

That's not true. As with any unix process, the owner of the process can kill it. If you start it from telnet, you can kill it. If you start it from the web, the web server user can kill it (which means you'd need another cgi script to do the killing of it). I'm not sure why that matters though.

Yes, since 1 category = 1 html page, the more categories you have, the longer it is going to take. Also, the fact that the program is based off of a static model (i.e. using static html pages), makes the build process much more complex. I didn't want a search result to have different rating then a static html page, so when building it also needs to update the links information which is quite involved.

No! What this means is that I didn't believe you were with an adequate ISP. Having seen the program run on hundreds of different systems, the load you were seeing indicated that your ISP had too many other users on that machine, or not enough system resources. To use the program with that size directory you needed to either move to a new ISP, or upgrade to a shared server with less account or a dedicated server. The reason none of the other users said anything is because they didn't have the problems you did. I tried to convince you of this last year.

You are thinking too simple. Consider what has to happen when you build:

1. 25 new detailed pages need to get created.
2. 25 new links need to get added to let's say 10 new categories. That's 10 changed pages.
3. For each of the 10 changed categories, we need to change every category above that and update the number of links counter.
4. We need to change new links that are no longer to new and rebuild the category pages they are in.
5. We need to rebuild the category pages above them.
6. Same goes for the popular.
7. We need to update the Rating and Votes for links that have been clicked on and rated/voted.
8. We need to rebuild category pages that contain changed links.
9. We need to rebuild category pages that contain links the admin deleted (which we also need to track).
etc, etc.

There is a lot of overhead involved here, and it may end up taking longer then to just do a clean build. All that said, there is room for improvement on how it handles things, and that's what we have been working on.

Cheers,

Alex

--
Gossamer Threads Inc.