I ran into a problem today in that I was doing a build staggered and my isp dropped (despite the fact I pay a lot of money for a cable modem but anyway....). Now this is not necessarliy a problem with a small directory, but in mine I have over 1.4m links and creating over 12,400 pages. It stopped at 11,400 ($%#@#) after running for about 10 hours.

Anyway a suggestion unless someone already has a solution, but is it possible to start the build all at a certain id number or category or in some way so that I do not have to build all just to update the last 1000 pages?

If it can't be done, maybe it could be added to future versions of Links sql.
Webmaster
http://www.e-bannerx.com

Hi,

You can use build staggered to start at a particular spot.

Step 5 is building categories, so you could do:

nph-build.cgi?do=staggered&s=5&p=PAGE&o=OFFSET

where page is the page number, and offset is the offset. i.e. to build categories 14,000 on, 1,000 a time, you would want p=14 and o=1000.

Hope that helps,

Alex
--
Gossamer Threads Inc.

Alex thanks it works great.

Also I have noticed that build changed tends to fall over very early (only after say 60 pages). Is that a common problem?
Webmaster
http://www.e-bannerx.com

What do you mean by "fall over"?

Cheers,

Alex
--
Gossamer Threads Inc.

With that many links, I'm assuming you are on a dedicated server...correct? If so, then it may be nicer (and faster) to set the script to run on a cron job. When you run it via browser/telnet some restrictions are there (i.e. your download rate, the limit on your users CPU/memory allowances etc). When I was doing my DMOZ dumps I found that doing it via cron took 2 mins for the adults category, and via Telnet took over 1 hour! It may be something to look into.

Andy (mod)
andy@ultranerds.co.uk

---

Want to give me something back for my help? Please see my Amazon Wish List
GLinks ULTRA Package | GLinks ULTRA Package PRO
Links SQL Plugins | Website Design and SEO | UltraNerds | ULTRAGLobals Plugin | Pre-Made Template Sets | FREE GLinks Plugins!

Alex - what I mean by fall over is that it just stops. It happens every time, therefore that is why I use build staggered almost exclusively. Also in the last few days I have had the same problem with backups. A set the backup running and it always stops at about 50,000 to 60,000k (backup is just over 400MB). I am about to give it another go.

Andy - Thanks for that. Sounds like a very good idea. I guess using what Alex posted previously I could use a cron job to run overnight when server load is lower. On the dedicated I have just purchased so I hope to try and move over the next few weeks.

Thanks
Webmaster
http://www.e-bannerx.com

Alex - Just tried a backup and it again fell over at 50,000k. Any thoughts what might cause these problems where I assume the script just stops?
Webmaster
http://www.e-bannerx.com

Hi all,

I am in the process of moving servers and the problem is my database will not back up. I run the create backip function under import/export and it starts but always stops when the file is around 50,000k.

Would anyone be able to offer some advice on how I might go about the backup another way?

Thanks in advance.
Webmaster
http://www.e-bannerx.com

Is it possible to backup the databse in parts? ie that is via tables? Its just that I am trying to find a solution to me problem?



Thanks
Webmaster
http://www.e-bannerx.com

You could try `mysqldump` or use our MySQLMan tool to backup your database.

Cheers,

Alex
--
Gossamer Threads Inc.

Thanks Alex, I tried mysqlman and had no luck. Also tried phpmyadmin and had no luck. I am about to try mysqldump and see if it will work.
Webmaster
http://www.e-bannerx.com

Alex - I was able to create a backup via mysqldump and I have transferred it to my new server.

However, when I tried to use the import function in links sql it went through the script and said it completed but when I repaired tables and then went to browse the database, nothing had been imported?

I have therefore updated my database by using ssh and mysql.

My question is - it has created all new tables which are links_Category rather than lsql_ (obvioulsy new table prefix because thats how they are named in my current database). Should I just change the extension in SQL server for the table prefix to be links_. Also how do I then get rid of all the old tables with the table prefix of lsql_. Or am I going about all this totally wrong?
Webmaster
http://www.e-bannerx.com

Hi,

Links SQL can not import a backup made via mysqldump, you must import it using command line:

mysql -uUSERNAME -pPASSWORD DATABASE < backup.txt

or via MySQLMan SQL Monitor feature.

What I recommend is:

1. On the new server, goto Setup->SQL and create the tables with the same prefix as your old server.

2. On your old server run:

mysqldump --add-drop-tables -uUSER -pPASS DATABASE > backup.txt

3. Copy that file to your newserver and run:

mysql -uUSER -pPASS DATABASE < backup.txt

and you should have all your data restored.

Cheers,

Alex
--
Gossamer Threads Inc.

Alex - Thanks once again. All has been downloaded and transferred. Your help is greatly appreciated.

One question however, when I try to backup in either the build process of through the create backup under import/export the backup always falls over (stops) around 50,000k. Do you have a feel for what might be causing it. My old server used 2.05 and I have taken the opportunity to upgrade to 2.1 on the new server. Also I have not tried it on the new server as yet (building pages and it takes 14 hours) but I will when it has finished. My guess is due to the size of the database as the backup in phpmyadmin didn't work either. Only the shell backup worked. My database is 1.4m links and about 124,000 categories. Would this be a reason for it not to work?
Webmaster
http://www.e-bannerx.com