Mailing List Archive: Wikipedia: Wikitech

Identifying pages that are slow to render

Index | Next | Previous | View Threaded

rarohde at gmail

Mar 6, 2013, 10:05 PM

Post #1 of 13 (7730 views)
Permalink

Identifying pages that are slow to render

Is there some list or tool that identifies Wikipedia pages that are
slow to parse?

My interest is mainly in the context of Lua deployment. I would like
to identify non-obvious templates that may be having an appreciable
impact on performance. (As opposed to things like {{cite}}, where the
performance problems are already well-known.)

I don't suppose the database stores a "time for last parse" somewhere?

On enwiki we've already made Lua conversions with most of the string
templates, several formatting templates (e.g. {{rnd}}, {{precision}}),
{{coord}}, and a number of others. And there is work underway on a
number of the more complex overhauls (e.g. {{cite}}, {{convert}}).
However, it would be nice to identify problematic templates that may
be less obvious.

-Robert Rohde
aka Dragons_flight

_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

wikiposta at gmail

Mar 6, 2013, 10:09 PM

Post #2 of 13 (7573 views)
Permalink

Re: Identifying pages that are slow to render [In reply to]

A partial answer:
I wrote a script under Pywikipedia framework to search templates with many
parser functions that are worth to convert to Lua now:
[[mw:Special:Code/pywikipedia/11099]]. I hope that it helps you to find
good targets.

The result looks like
http://hu.wikipedia.org/wiki/Wikip%C3%A9dia:Sablonm%C5%B1hely/Lua_k%C3%ADv%C3%A1ns%C3%A1glista/Sok_parserf%C3%BCggv%C3%A9nyt_tartalmaz%C3%B3_sablonok
.

2013/3/7 Robert Rohde <rarohde [at] gmail>

> Is there some list or tool that identifies Wikipedia pages that are
> slow to parse?
>
> My interest is mainly in the context of Lua deployment. I would like
> to identify non-obvious templates that may be having an appreciable
> impact on performance. (As opposed to things like {{cite}}, where the
> performance problems are already well-known.)
>
> I don't suppose the database stores a "time for last parse" somewhere?
>
> On enwiki we've already made Lua conversions with most of the string
> templates, several formatting templates (e.g. {{rnd}}, {{precision}}),
> {{coord}}, and a number of others. And there is work underway on a
> number of the more complex overhauls (e.g. {{cite}}, {{convert}}).
> However, it would be nice to identify problematic templates that may
> be less obvious.
>
> -Robert Rohde
> aka Dragons_flight
>
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-l [at] lists
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l

--
Bináris
_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

jeremy at tuxmachine

Mar 6, 2013, 10:21 PM

Post #3 of 13 (7582 views)
Permalink

Re: Identifying pages that are slow to render [In reply to]

On Mar 7, 2013 1:06 AM, "Robert Rohde" <rarohde [at] gmail> wrote:
> Is there some list or tool that identifies Wikipedia pages that are
> slow to parse?
>
> My interest is mainly in the context of Lua deployment. I would like
> to identify non-obvious templates that may be having an appreciable
> impact on performance. (As opposed to things like {{cite}}, where the
> performance problems are already well-known.)
>
> I don't suppose the database stores a "time for last parse" somewhere?

Not sure. Maybe try one of these cats:
https://gerrit.wikimedia.org/r/gitweb?p=mediawiki/core.git;a=blob;f=includes/parser/Parser.php;hb=3d0f6878f355ff38f5822e37121661352d272fe8#l3107
_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

nemowiki at gmail

Mar 6, 2013, 11:58 PM

Post #4 of 13 (7572 views)
Permalink

Re: Identifying pages that are slow to render [In reply to]

There's slow-parse.log, but it's private unless a solution is found for
https://gerrit.wikimedia.org/r/#/c/49678/
https://wikitech.wikimedia.org/wiki/Logs

Nemo

_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

hashar+wmf at free

Mar 7, 2013, 12:00 PM

Post #5 of 13 (7572 views)
Permalink

Re: Identifying pages that are slow to render [In reply to]

Le 06/03/13 23:58, Federico Leva (Nemo) a écrit :
> There's slow-parse.log, but it's private unless a solution is found for
> https://gerrit.wikimedia.org/r/#/c/49678/
> https://wikitech.wikimedia.org/wiki/Logs

And slow-parse.log is probably going to be kept private unless proven it
is not harmful =)

--
Antoine "hashar" Musso

_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

hashar+wmf at free

Mar 7, 2013, 12:00 PM

Post #6 of 13 (7568 views)
Permalink

Re: Identifying pages that are slow to render [In reply to]

hashar+wmf at free

Mar 7, 2013, 12:02 PM

Post #7 of 13 (7571 views)
Permalink

Re: Identifying pages that are slow to render [In reply to]

Le 06/03/13 22:05, Robert Rohde a écrit :
> On enwiki we've already made Lua conversions with most of the string
> templates, several formatting templates (e.g. {{rnd}}, {{precision}}),
> {{coord}}, and a number of others. And there is work underway on a
> number of the more complex overhauls (e.g. {{cite}}, {{convert}}).
> However, it would be nice to identify problematic templates that may
> be less obvious.

You can get in touch with Brad Jorsch and Tim Starling. They most
probably have a list of templates that should quickly converted to LUA
modules.

If we got {{cite}} out, that will be already a nice improvement :-]

--
Antoine "hashar" Musso

_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

mflaschen at wikimedia

Mar 7, 2013, 12:06 PM

Post #8 of 13 (7568 views)
Permalink

Re: Identifying pages that are slow to render [In reply to]

On 03/07/2013 12:00 PM, Antoine Musso wrote:
> Le 06/03/13 23:58, Federico Leva (Nemo) a écrit :
>> There's slow-parse.log, but it's private unless a solution is found for
>> https://gerrit.wikimedia.org/r/#/c/49678/
>> https://wikitech.wikimedia.org/wiki/Logs
>
> And slow-parse.log is probably going to be kept private unless proven it
> is not harmful =)

Why would it be harmful for public wikis? Anyone can do this on an
article-by-article basis by copying the source their own MediaWiki
instances.

But it ends up being repeated work.

Matt

_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

jeremy at tuxmachine

Mar 7, 2013, 12:32 PM

Post #9 of 13 (7574 views)
Permalink

Re: Identifying pages that are slow to render [In reply to]

On Thu, Mar 7, 2013 at 8:06 PM, Matthew Flaschen
<mflaschen [at] wikimedia> wrote:
> Why would it be harmful for public wikis? Anyone can do this on an
> article-by-article basis by copying the source their own MediaWiki
> instances.

That user would have to pick which articles to copy and test (or test them all).

The log doesn't contain (I guess?) all articles. Only slow articles.

-Jeremy

_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

bawolff+wn at gmail

Mar 7, 2013, 5:12 PM

Post #10 of 13 (7549 views)
Permalink

Re: Identifying pages that are slow to render [In reply to]

On 2013-03-07 4:06 PM, "Matthew Flaschen" <mflaschen [at] wikimedia> wrote:
>
> On 03/07/2013 12:00 PM, Antoine Musso wrote:
> > Le 06/03/13 23:58, Federico Leva (Nemo) a écrit :
> >> There's slow-parse.log, but it's private unless a solution is found for
> >> https://gerrit.wikimedia.org/r/#/c/49678/
> >> https://wikitech.wikimedia.org/wiki/Logs
> >
> > And slow-parse.log is probably going to be kept private unless proven it
> > is not harmful =)
>
> Why would it be harmful for public wikis? Anyone can do this on an
> article-by-article basis by copying the source their own MediaWiki
> instances.
>
> But it ends up being repeated work.
>
> Matt
>
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-l [at] lists
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l

+1 . I have trouble imagining how making this public could be harmful.
There are plenty of well known slow to parse pages already. There's also
more than a couple of ways to convince mw to make slow queries (longer than
the php time limit), we publically release detailed profiling data, etc.
Well that sort of thing isnt exactly proclaimed to the world, its also not
a secret. If someone wanted to find slow points on mediawiki, theres a lot
worse things just floating around the internet than a slow to parse page
list.

-bawolff
_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

z at mzmcbride

Mar 8, 2013, 1:17 PM

Post #11 of 13 (7580 views)
Permalink

Re: Identifying pages that are slow to render [In reply to]

Federico Leva (Nemo) wrote:
>There's slow-parse.log, but it's private unless a solution is found for
>https://gerrit.wikimedia.org/r/#/c/49678/
>https://wikitech.wikimedia.org/wiki/Logs

"Separate slow-parse into public and private files"
<https://bugzilla.wikimedia.org/show_bug.cgi?id=45830>

<https://gerrit.wikimedia.org/r/49678> was abandoned; it looks like
<https://gerrit.wikimedia.org/r/52608> is now the relevant Gerrit
changeset.

MZMcBride

_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

saper at saper

Mar 8, 2013, 3:43 PM

Post #12 of 13 (7564 views)
Permalink

Re: Identifying pages that are slow to render [In reply to]

>> Antoine Musso <hashar+wmf [at] free> wrote:
> Le 06/03/13 22:05, Robert Rohde a écrit :
>> On enwiki we've already made Lua conversions with most of the string
>> templates, several formatting templates (e.g. {{rnd}}, {{precision}}),
>> {{coord}}, and a number of others. And there is work underway on a
>> number of the more complex overhauls (e.g. {{cite}}, {{convert}}).
>> However, it would be nice to identify problematic templates that may
>> be less obvious.
>
> You can get in touch with Brad Jorsch and Tim Starling. They most
> probably have a list of templates that should quickly converted to LUA
> modules.
>
> If we got {{cite}} out, that will be already a nice improvement :-]

Not really, given https://bugzilla.wikimedia.org/show_bug.cgi?id=45861

//Saper

_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

ariel at wikimedia

Mar 11, 2013, 7:44 AM

Post #13 of 13 (7570 views)
Permalink

Re: Identifying pages that are slow to render [In reply to]

Στις 07-03-2013, ημέρα Πεμ, και ώρα 21:12 -0400, ο/η bawolff έγραψε:
> On 2013-03-07 4:06 PM, "Matthew Flaschen" <mflaschen [at] wikimedia> wrote:
> >
> > On 03/07/2013 12:00 PM, Antoine Musso wrote:
> > > Le 06/03/13 23:58, Federico Leva (Nemo) a écrit :
> > >> There's slow-parse.log, but it's private unless a solution is found for
> > >> https://gerrit.wikimedia.org/r/#/c/49678/
> > >> https://wikitech.wikimedia.org/wiki/Logs
> > >
> > > And slow-parse.log is probably going to be kept private unless proven it
> > > is not harmful =)
> >
> > Why would it be harmful for public wikis? Anyone can do this on an
> > article-by-article basis by copying the source their own MediaWiki
> > instances.
> >
> > But it ends up being repeated work.
> >
> > Matt
> >
> > _______________________________________________
> > Wikitech-l mailing list
> > Wikitech-l [at] lists
> > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
> +1 . I have trouble imagining how making this public could be harmful.
> There are plenty of well known slow to parse pages already. There's also
> more than a couple of ways to convince mw to make slow queries (longer than
> the php time limit), we publically release detailed profiling data, etc.
> Well that sort of thing isnt exactly proclaimed to the world, its also not
> a secret. If someone wanted to find slow points on mediawiki, theres a lot
> worse things just floating around the internet than a slow to parse page
> list.
>

The log in its current form is not just a list of publically viewable
pages with parse times. The extraneous information would need to be
removed before it could be made public.

Ariel

_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads