Gossamer Forum
Home : Products : Gossamer Links : Discussions :

SSH Build commands

Quote Reply
SSH Build commands
I looked in the manual and searched the forum first, but didn't find the answer. This will sound dumb and basic for some of you BUT I haven't tried doing things via ssh befroe and needs some help. What are the commands for build changed and build staggered via ssh? Probably wouldn't be a bad idea to let me know the verify and reindex commands either... I was hoping for a page in the manual, like an appendix page, that would list the commonly used ssh commands. I was told doing builds and indexing via ssh rather than the browser would be less server intensive.....
Quote Reply
Re: [loxly] SSH Build commands In reply to
Hi,

These should help :)

Quote:
cd /path/to/admin/folder

perl nph-build.cgi --all
perl nph-build.cgi --changed
perl nph-build.cgi --repair

"Repair" recounts the category starts for you

To be more advanced, you could also use;

nohup perl nph-build.cgi --all > log.txt &
tail -n20 -f log.txt

..which will keep the process running, even if your SSH window shuts for some reason :)

Cheers

Andy (mod)
andy@ultranerds.co.uk
Want to give me something back for my help? Please see my Amazon Wish List
GLinks ULTRA Package | GLinks ULTRA Package PRO
Links SQL Plugins | Website Design and SEO | UltraNerds | ULTRAGLobals Plugin | Pre-Made Template Sets | FREE GLinks Plugins!
Quote Reply
Re: [Andy] SSH Build commands In reply to
Thanks Andy! I did actually find the commands through the your cron plug in after I posted, but I am surprized there isn't a command for build staggered. I guess I need to investigate mod rewrite on one site in particular sooner rather than later! I can't get through a full build...
Quote Reply
Re: [loxly] SSH Build commands In reply to
Hi,

No worries =)

Quote:
but I am surprized there isn't a command for build staggered

Staggered = xx per page, thus running via SSH wouldn't really work, as it doesn't need to be staggered =) Staggered is designed specifically for use in the browser, as this is the only place you would really get timeouts.

Quote:
I can't get through a full build..

Do you get an error? If running via SSH, the only timeout you may find, is the Perl timeout itself, which I've very rarely ever encountered (unless it was running for days-and-days).

Cheers

Andy (mod)
andy@ultranerds.co.uk
Want to give me something back for my help? Please see my Amazon Wish List
GLinks ULTRA Package | GLinks ULTRA Package PRO
Links SQL Plugins | Website Design and SEO | UltraNerds | ULTRAGLobals Plugin | Pre-Made Template Sets | FREE GLinks Plugins!
Quote Reply
Re: [Andy] SSH Build commands In reply to
No error, after a few hours of running it just said "Terminated" and ended.
Quote Reply
Re: [loxly] SSH Build commands In reply to
Sounds like a timeout error :( Do you get this with --staggered ?

Cheers

Andy (mod)
andy@ultranerds.co.uk
Want to give me something back for my help? Please see my Amazon Wish List
GLinks ULTRA Package | GLinks ULTRA Package PRO
Links SQL Plugins | Website Design and SEO | UltraNerds | ULTRAGLobals Plugin | Pre-Made Template Sets | FREE GLinks Plugins!
Quote Reply
Re: [Andy] SSH Build commands In reply to
In Reply To:
Sounds like a timeout error :( Do you get this with --staggered ?

Cheers

-- staggered didn't work, it gave me a list of the other options instead.

Since this is a new unspidered site, I would like to learn how to implement mod rewrite :) Check your email :)
Quote Reply
Re: [loxly] SSH Build commands In reply to
Hi,

Sorry, I mean't --changed ... *not* --staggered *blush*

Cheers

Andy (mod)
andy@ultranerds.co.uk
Want to give me something back for my help? Please see my Amazon Wish List
GLinks ULTRA Package | GLinks ULTRA Package PRO
Links SQL Plugins | Website Design and SEO | UltraNerds | ULTRAGLobals Plugin | Pre-Made Template Sets | FREE GLinks Plugins!
Quote Reply
Re: [Andy] SSH Build commands In reply to
lol, time for someone to go to bed I think.... will try changed and see what happens, but since I added 40,000 links and moved around a bunch of categories, I figured build all would be the smarter thing to do.
Quote Reply
Re: [loxly] SSH Build commands In reply to
Hi Andy, or anyone...
Before I do this, I just wanted to make sure I got it right.
If I go into SSH and type:

nohup perl nph-build.cgi --all > log.txt & <return>
tail -n20 -f log.txt

my directory will build all until the entire series is done - EVEN - if I close the terminal window. Right?

Also, on a dual 2.4 Xeon server, does anyone have a ballpark of how long it would take to build about 15,000 pages?
Quote Reply
Re: [Giray] SSH Build commands In reply to
Hi,

Quote:
nohup perl nph-build.cgi --all > log.txt & <return>
tail -n20 -f log.txt

my directory will build all until the entire series is done - EVEN - if I close the terminal window. Right?

Indeed. You will obviously need to 'cd' first though <G> The only time it will stop running, is if a problem is encountered in the build process (i.e something couldn't be written/a database error/plugin error etc).

Quote:
Also, on a dual 2.4 Xeon server, does anyone have a ballpark of how long it would take to build about 15,000 pages?

Thats a little like trying to ask the length of a piece of string Tongue

Personally, I'd probably say about 20-45 mins. It all depends on page sizes, and server load. I can normally run a build on 30,000+ links, in 10,000~ categories, in less than 20 mins (even at busy server times).

Hope that helps.

Cheers

Andy (mod)
andy@ultranerds.co.uk
Want to give me something back for my help? Please see my Amazon Wish List
GLinks ULTRA Package | GLinks ULTRA Package PRO
Links SQL Plugins | Website Design and SEO | UltraNerds | ULTRAGLobals Plugin | Pre-Made Template Sets | FREE GLinks Plugins!
Quote Reply
Re: [Andy] SSH Build commands In reply to
Thanks Andy.
A follow up that actually concerns your DMOZ plugin (but it's an ssh question, so I'll slip it into this post in case it helps anyone else).
On the DMOZ plugin, when you launch an ssh, it's also terminal dependent. Is there a command to shift it all over to the server?
If you want this dicussion taken over to ultranerds, let me now Blush
Quote Reply
Re: [Giray] SSH Build commands In reply to
Hi,

No worries =)

Regarding your other question ... I'm not quite sure what you mean? If the import is run as root (or su), then it should, in theory, run at its optimum speed (some users have restrictions in place, which can slow it down). Hopefully thats what your asking <G>

Cheers

Andy (mod)
andy@ultranerds.co.uk
Want to give me something back for my help? Please see my Amazon Wish List
GLinks ULTRA Package | GLinks ULTRA Package PRO
Links SQL Plugins | Website Design and SEO | UltraNerds | ULTRAGLobals Plugin | Pre-Made Template Sets | FREE GLinks Plugins!
Quote Reply
Re: [Andy] SSH Build commands In reply to
We launched a:
perl nph-build.cgi --all

On 1.7 million links, with detailed pages, it looks like it'll take about ten days at our current rate (we have detailed pages activated).

Question, is there a way to take advantage of two or three of our processors with the script? We're on our own linux box. If we can do it in five days, instead of ten, that would be sweet.
Quote Reply
Re: [Giray] SSH Build commands In reply to
Hi,

Have you considered using mod_rewrite, alongside mod_perl? This way, you get a static feel of the directory (i.e they look like normal pages), but they are instead passed through page.cgi. The advantage of this, is that you don't have to run a build at all.

We do this on most of our site, and its prooven to work very well, especially with the ol' search engines <G>

Hope that helps.

Cheers

Andy (mod)
andy@ultranerds.co.uk
Want to give me something back for my help? Please see my Amazon Wish List
GLinks ULTRA Package | GLinks ULTRA Package PRO
Links SQL Plugins | Website Design and SEO | UltraNerds | ULTRAGLobals Plugin | Pre-Made Template Sets | FREE GLinks Plugins!