Mailing List Archive: Wikipedia: Wikitech

Re: orphans page takes too long (PS)

Index | Next | Previous | View Threaded

hidders at uia

Feb 27, 2002, 3:42 AM

Post #1 of 9 (254 views)
Permalink

Re: orphans page takes too long (PS)

From: "Jan Hidders" <hidders [at] uia>
>
> Oh dear. Sorry that my instructions were not clear. You are supposed to
run
> the script, wait for a link to appear and then click this link for the
next
> batch. It seems you did only one batch. You need to repeat this until all
> the records in table cur have been processed. This means you have to click
> about 37 or 38 times, or you have to increase $size to do bigger batches.
> (But avoid the 30 s. time-out. If you get a time-out you have the clear
the
> table "DELETE FROM linked WHERE 1=1;" and start from the beginning again.)

Because you already did a batch you need to empty the tables first. The
fastest way to do this is simply to drop the tables altogether. After that
you can recreate them without the indexes (as in updSchema.sql) and fill
them with the script. Considering the speed of your system you might simply
try setting $size to 40000 or so, and attempt to do everything in one go. If
that times-out, clear the tables 'linked' and 'unlinked' with DELETE and
try again with half the value. Et cetera. After this you can add the indexes
as in updSchema.sql.

-- Jan Hidders

brion at pobox

Feb 27, 2002, 9:24 AM

Post #2 of 9 (246 views)
Permalink

Re: orphans page takes too long (PS) [In reply to]

On mer, 2002-02-27 at 02:42, Jan Hidders wrote:
> Because you already did a batch you need to empty the tables first. The
> fastest way to do this is simply to drop the tables altogether. After that
> you can recreate them without the indexes (as in updSchema.sql) and fill
> them with the script. Considering the speed of your system you might simply
> try setting $size to 40000 or so, and attempt to do everything in one go. If
> that times-out, clear the tables 'linked' and 'unlinked' with DELETE and
> try again with half the value. Et cetera. After this you can add the indexes
> as in updSchema.sql.

You can also set set_time_limit(9999) or some such. No nasty timeouts!

-- brion vibber (brion @ pobox.com)

hidders at uia

Feb 27, 2002, 9:43 AM

Post #3 of 9 (246 views)
Permalink

Re: orphans page takes too long (PS) [In reply to]

From: "Brion L. VIBBER" <brion [at] pobox>
>
> You can also set set_time_limit(9999) or some such. No nasty timeouts!

Ah, great!! That really supersedes the Apache setting? You learn something
new every day. :-) Or rather, :-/. I should have known this earlier on. I
read that this only works if PHP does not run in safe mode, but I suppose
that since the default is off, Jimbo probably has it not running in safe
mode anyway.

-- Jan Hidders

jwales at bomis

Feb 27, 2002, 10:00 AM

Post #4 of 9 (246 views)
Permalink

Re: orphans page takes too long (PS) [In reply to]

Right, I did drop cur_linked_links and cur_unlinked_links.

> No panic, the linking information is still there except we now need a
> different script because we need to get it out of the cur_text field. This
> is a bit of a problem because I really have to prepare some lessons for
> tomorrow. Any of the other programmers feel able to do this?

Well, there's no special hurry is there?

The site looks fine, this is just a problem for the orphan pages script? And that's
not super-duper important.

hidders at uia

Feb 27, 2002, 10:13 AM

Post #5 of 9 (246 views)
Permalink

Re: orphans page takes too long (PS) [In reply to]

From: "Jimmy Wales" <jwales [at] bomis>
>
> This answered my question.
>
> When I ran this by hand, it took a long time, so I better just stick with
1000 and
> click it a bunch of times. That's no bother to me.
>
> If I drop the tables will the site break until I create them again?

WAIT!! It just occurred to me that there is a problem because you probably
by now have dropped the cur_linked_links and cur_unlinked_links columns,
right? The script updLinks.php assumes they are still there and gets its
information for the new tables from them. So just running it again won't
work.

No panic, the linking information is still there except we now need a
different script because we need to get it out of the cur_text field. This
is a bit of a problem because I really have to prepare some lessons for
tomorrow. Any of the other programmers feel able to do this?

-- Jan Hidders

hidders at uia

Feb 27, 2002, 10:34 AM

Post #6 of 9 (246 views)
Permalink

Re: orphans page takes too long (PS) [In reply to]

From: "Jimmy Wales" <jwales [at] bomis>
>
> Well, there's no special hurry is there?
>
> The site looks fine, this is just a problem for the orphan pages script?
And that's
> not super-duper important.

.. and the mostWanted page, but that is also not really fatal. Ok. I'll try
to have the new script ready by tomorrow.

-- Jan Hidders

brion at pobox

Feb 27, 2002, 10:56 AM

Post #7 of 9 (247 views)
Permalink

Re: orphans page takes too long (PS) [In reply to]

On mer, 2002-02-27 at 09:34, Jan Hidders wrote:
> From: "Jimmy Wales" <jwales [at] bomis>
> >
> > Well, there's no special hurry is there?
> >
> > The site looks fine, this is just a problem for the orphan pages script?
> And that's
> > not super-duper important.
>
> .. and the mostWanted page, but that is also not really fatal.

And "pages that link to this page".

> Ok. I'll try
> to have the new script ready by tomorrow.

I just committed a rebuildLinks.php, does it look like it's doing the
right things?

-- brion vibber (brion @ pobox.com)

hidders at uia

Feb 28, 2002, 8:44 AM

Post #8 of 9 (246 views)
Permalink

Re: orphans page takes too long (PS) [In reply to]

First my apologies for replying this late, but my students go first.

From: "Brion L. VIBBER" <brion [at] pobox>
>
> I just committed a rebuildLinks.php, does it look like it's doing the
> right things?

I think it doesn't work correctly for links between <pre> tags. Anyway, I've
committed a new version that uses the same code as that which is run when a
page is updated. This script is more robust than my previous one (you can
run it as many times as you like for the same records) and tells you a bit
more about what is going on. It also tells you when all records have been
processed and doesn't present you another link then.

So Jimbo, can I ask you to try the new script rebuildLinks.php?

-- Jan Hidders

hidders at uia

Mar 1, 2002, 1:08 AM

Post #9 of 9 (246 views)
Permalink

Re: orphans page takes too long (PS) [In reply to]

From: "Jan Hidders" <hidders [at] uia>
>
> So Jimbo, can I ask you to try the new script rebuildLinks.php?

Hm, I forgot to explain the procedure. Here it is:

Step 1: run the script in your browser and keep on clicking the presented
link until it says it's ready.

That's it. So you don't have to empty the tables, recreate the indexes, or
anything. It also clears up any wrong links that might have been inserted
before.

-- Jan Hidders

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads