Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Wikipedia: Foundation

Continued concern with performance

 

 

Wikipedia foundation RSS feed   Index | Next | Previous | View Threaded


rich_holton at yahoo

Jan 17, 2005, 11:48 AM

Post #1 of 15 (424 views)
Permalink
Continued concern with performance

We are now well into the fourth day since I originally expressed my
concern about the performance issues on en.wikipedia. While the
situation may have improved a bit, and some of the ugliest error
messages seem to be reduced, the situation is still far from being
resolved.

Attempting to do any useful work on Wikipedia has become futile.
Article view time has become sporadic, and frequently very bad to
intolerable. For any site other than Wikipedia, I would never know that
the site was working at all, since I would fairly quickly write it off
as "down".

I am committed to Wikipedia and its objectives. I will tolerate these
performance delays. But I believe that there must be many who will not
tolerate them, and will write-off Wikipedia. Is this what we want?

While it appears that the developers are very busy attempting to solve
the technical issues, there seems to be little actual progress. How
long will we allow this situation to continue without attempting
solutions of a different sort? A good operational definition of
insanity is "continuing the same actions expecting different results."

If our developers lack time, resources, or skill needed to resolve the
issues, then let us immediately hire a temporary consultant to focus on
this problem and resolve it.

We owe our developers great thanks, appreciation, and respect. But a
big part of that lies in recognizing when we are expecting them to
accomplish what is impossible. Given their restrictions on time,
resources, and skill, are we asking them to do the impossible?

With respect, concern, and frustration

Rich Holton
(en.Wikipedia:User:Rholton)



__________________________________
Do you Yahoo!?
The all-new My Yahoo! - Get yours free!
http://my.yahoo.com


pfayolle at gmail

Jan 17, 2005, 12:32 PM

Post #2 of 15 (420 views)
Permalink
Re: Continued concern with performance [In reply to]

I don't know if anyone has ever suggested this before, but couldn't
the software be adapted to lay a larger responsability on the client?
I'm not saying this would be a trivial instant solution, but as far as
I can tell it could make a huge difference in the long run.

Just for a start, imagine the possibilities of having a JavaScript
wikicode parser. Preview pages could be created on the fly without the
need for a page reload. Pages could be sent to the client in raw
wikicode state (processing only message inclusions) and parsed
afterwards by the client, they could even be switched to edit mode
without sending a new request. Having such a feature implemented would
for sure reduce the load on the servers.

I don't have much experience with JavaScript and the DOM, but I'll
begin doing some experiments this week on my free time. I have seen
XML parsers written in JS, so a wikicode parser shouldn't be that hard
to make, or it should be at least possible.

I know the common objections to an idea like this: it depends on
JS-enabled browsers and also some JS interpreters might not work the
same as others. This should not be seen as a replacement for the
server-side processor, just an off-by-default alternative. If it were
there and it was proven to be useful and to relieve the servers I bet
most caring users would give it a shot.

Just my 2 cents. By the way, forgive my broken English.

-Pedro Fayolle

On Mon, 17 Jan 2005 10:48:01 -0800 (PST), Rich Holton
<rich_holton [at] yahoo> wrote:
> We are now well into the fourth day since I originally expressed my
> concern about the performance issues on en.wikipedia. While the
> situation may have improved a bit, and some of the ugliest error
> messages seem to be reduced, the situation is still far from being
> resolved.
>
> Attempting to do any useful work on Wikipedia has become futile.
> Article view time has become sporadic, and frequently very bad to
> intolerable. For any site other than Wikipedia, I would never know that
> the site was working at all, since I would fairly quickly write it off
> as "down".
>
> I am committed to Wikipedia and its objectives. I will tolerate these
> performance delays. But I believe that there must be many who will not
> tolerate them, and will write-off Wikipedia. Is this what we want?
>
> While it appears that the developers are very busy attempting to solve
> the technical issues, there seems to be little actual progress. How
> long will we allow this situation to continue without attempting
> solutions of a different sort? A good operational definition of
> insanity is "continuing the same actions expecting different results."
>
> If our developers lack time, resources, or skill needed to resolve the
> issues, then let us immediately hire a temporary consultant to focus on
> this problem and resolve it.
>
> We owe our developers great thanks, appreciation, and respect. But a
> big part of that lies in recognizing when we are expecting them to
> accomplish what is impossible. Given their restrictions on time,
> resources, and skill, are we asking them to do the impossible?
>
> With respect, concern, and frustration
>
> Rich Holton
> (en.Wikipedia:User:Rholton)
>
> __________________________________
> Do you Yahoo!?
> The all-new My Yahoo! - Get yours free!
> http://my.yahoo.com
>
> _______________________________________________
> foundation-l mailing list
> foundation-l [at] wikimedia
> http://mail.wikipedia.org/mailman/listinfo/foundation-l
>


t.starling at physics

Jan 17, 2005, 8:08 PM

Post #3 of 15 (422 views)
Permalink
Re: Continued concern with performance [In reply to]

Rich Holton wrote:
> While it appears that the developers are very busy attempting to
> solve the technical issues, there seems to be little actual progress.
> How long will we allow this situation to continue without attempting
> solutions of a different sort? A good operational definition of
> insanity is "continuing the same actions expecting different
> results."

At this stage, I think we just need more hardware. I think we've solved
the major system administration problems. Software development will
continue with optimisation work but that's a long term goal even with
hired help. You could try using the site in off-peak times, it works
very well then. More hardware is on order.

> If our developers lack time, resources, or skill needed to resolve
> the issues, then let us immediately hire a temporary consultant to
> focus on this problem and resolve it.

What problem? You don't even understand what the problem is. I've
repeatedly given my informed opinion on what is needed, and it doesn't
involve hiring consultants at a rate of $100/hr. Long term work on
optimisation is needed. JamesDay, our database optimisation specialist,
can give you a list of features as long as your arm that would improve
performance. I've tried and mostly failed to get people interested in
writing such things. Hardware is needed, and rapid hardware repair is
needed.

More system administrators are needed. My interest is in programming,
but I end up spending most of my time on Linux system administration,
because there's nobody else around to do it. Understand that Linux is an
operating system I had *never even used* before I started on Wikipedia.
I had to pick it up as I went along. It's a bizarre, flawed, buggy
operating system which I only use because I'm forced to.

Paid system administration help may be an option, but the first priority
for paid help should be hardware maintenance, since there are no
volunteers in that field, and a small amount of time spent will make a
big difference.

-- Tim Starling


t.starling at physics

Jan 17, 2005, 8:10 PM

Post #4 of 15 (425 views)
Permalink
Re: Continued concern with performance [In reply to]

Pedro Fayolle wrote:
> I don't know if anyone has ever suggested this before, but couldn't
> the software be adapted to lay a larger responsability on the client?
> I'm not saying this would be a trivial instant solution, but as far as
> I can tell it could make a huge difference in the long run.
>
> Just for a start, imagine the possibilities of having a JavaScript
> wikicode parser.

Yes, it's been suggested before. Read the wikitech-l archives for
information about this approach.

-- Tim Starling


rich_holton at yahoo

Jan 17, 2005, 10:58 PM

Post #5 of 15 (420 views)
Permalink
Re: Re: Continued concern with performance [In reply to]

--- Tim Starling <t.starling [at] physics> wrote:

> Rich Holton wrote:
> > While it appears that the developers are very busy attempting to
> > solve the technical issues, there seems to be little actual
> progress.
> > How long will we allow this situation to continue without
> attempting
> > solutions of a different sort? A good operational definition of
> > insanity is "continuing the same actions expecting different
> > results."
>
> At this stage, I think we just need more hardware. I think we've
> solved
> the major system administration problems. Software development will
> continue with optimisation work but that's a long term goal even with
> hired help. You could try using the site in off-peak times, it works
> very well then. More hardware is on order.
>
I have and will continue to use Wikipedia at off-peak hours. Of course,
this is at best a work-around, not a solution.

> > If our developers lack time, resources, or skill needed to resolve
> > the issues, then let us immediately hire a temporary consultant to
> > focus on this problem and resolve it.
>
> What problem? You don't even understand what the problem is. I've
> repeatedly given my informed opinion on what is needed, and it
> doesn't
> involve hiring consultants at a rate of $100/hr. Long term work on
> optimisation is needed. JamesDay, our database optimisation
> specialist,
> can give you a list of features as long as your arm that would
> improve
> performance. I've tried and mostly failed to get people interested in
>
> writing such things. Hardware is needed, and rapid hardware repair is
>
> needed.
>
Thanks for setting me straight on where the problem lies. One of the
real challenges is staying current with what progress has been made. In
my own defense, there is a prominent link at the top of the technical
section of the Village Pump that leads to a Live Journal article
stating emphatically that the problem is not hardware. Apparently this
is out of date. Is there a better way to stay informed that I should
know about?

> More system administrators are needed. My interest is in programming,
>
> but I end up spending most of my time on Linux system administration,
>
> because there's nobody else around to do it. Understand that Linux is
> an
> operating system I had *never even used* before I started on
> Wikipedia.
> I had to pick it up as I went along. It's a bizarre, flawed, buggy
> operating system which I only use because I'm forced to.
>

Knowing your path to becoming a competent system administrator gives me
some hope for myself, as I would have to go through a learning curve
similar to yours. Of course, even if I and a hundred others were to
begin that learning curve now, it would not solve any immediate
problems.

> Paid system administration help may be an option, but the first
> priority
> for paid help should be hardware maintenance, since there are no
> volunteers in that field, and a small amount of time spent will make
> a
> big difference.
>
> -- Tim Starling
>

My greatest concern is that there are continuing performance issues
with little apparent progress over an extended period. Without
hesitation I presume that we have good people volunteering their time
and working hard to resolve these issues.

Sometimes it is necessary for an organization to consider the
possibility that internal resources are not sufficient to address an
immediate need. Wikipedia has come a long way relying almost entirely
on volunteer workers. I do not believe this can continue indefinitely.

With humble respect,

Rich Holton
(en.Wikipedia:User:Rholton)




__________________________________
Do you Yahoo!?
Yahoo! Mail - 250MB free storage. Do more. Manage less.
http://info.mail.yahoo.com/mail_250


yann at forget-me

Jan 18, 2005, 1:07 AM

Post #6 of 15 (424 views)
Permalink
Re: Re: Continued concern with performance [In reply to]

Hi,

Le Tuesday 18 January 2005 04:08, Tim Starling a écrit :
(...)
> More system administrators are needed. My interest is in programming,
> but I end up spending most of my time on Linux system administration,
> because there's nobody else around to do it. Understand that Linux is an
> operating system I had *never even used* before I started on Wikipedia.
> I had to pick it up as I went along. It's a bizarre, flawed, buggy
> operating system which I only use because I'm forced to.

I am volunteering to help here.
I am not a developer but a Linux system administrator for the last 6 years.
See http://www.forget-me.net/pro/ (sorry, only in French).
First I would like to help installing a basic antispam filter, as managing the
mailing lists is becoming a burden due to spams.
Now I am off upto Saturday due to RL work ;o) but I will read the archives.

> Paid system administration help may be an option, but the first priority
> for paid help should be hardware maintenance, since there are no
> volunteers in that field, and a small amount of time spent will make a
> big difference.
>
> -- Tim Starling

Regards,
Yann
--
http://www.non-violence.org/ | Site collaboratif sur la non-violence
http://www.forget-me.net/ | Alternatives sur le Net
http://fr.wikipedia.org/ | Encyclopédie libre
http://www.forget-me.net/pro/ | Formations et services Linux


nicolas.weeger at laposte

Jan 18, 2005, 1:29 AM

Post #7 of 15 (421 views)
Permalink
Re: Re: Continued concern with performance [In reply to]

> I am volunteering to help here.

I'm volunteering too, if i can be of any help.
I'm no linux sysadmin (ok, i can install a system & solve issues now and
then, having picked stuff managing the small LAN my company uses,
including dns, web site, and so on... but i'm at core a software
developer ;p). But if there are routine tasks, or simple maintenance
stuff i could do, i'll gladly help.

[[fr:User:Ryo|Nicolas Weeger]]


fun at thingy

Jan 18, 2005, 3:01 AM

Post #8 of 15 (422 views)
Permalink
Re: Re: Continued concern with performance [In reply to]

Tim Starling (t.starling [at] physics) [050118 14:08]:

> More system administrators are needed. My interest is in programming,
> but I end up spending most of my time on Linux system administration,
> because there's nobody else around to do it. Understand that Linux is an
> operating system I had *never even used* before I started on Wikipedia.
> I had to pick it up as I went along. It's a bizarre, flawed, buggy
> operating system which I only use because I'm forced to.


Obviously Kate needs to port MediaWiki to VMS ;-)

(I feel your pain. I used to dislike running Linux, now I despise it.)


> Paid system administration help may be an option, but the first priority
> for paid help should be hardware maintenance, since there are no
> volunteers in that field, and a small amount of time spent will make a
> big difference.


A good Linux admin with a screwdriver?


- d.


maveric149 at yahoo

Jan 18, 2005, 9:26 AM

Post #9 of 15 (423 views)
Permalink
Re: Re: Continued concern with performance [In reply to]

--- Rich Holton <rich_holton [at] yahoo> wrote:
> My greatest concern is that there are continuing performance issues
> with little apparent progress over an extended period. Without
> hesitation I presume that we have good people volunteering their time
> and working hard to resolve these issues.

There has been great progress and the number of servers has grown dramatically.
The trouble with exponential growth in traffic is that any set-back in trying
to keep up quickly becomes a major issue.

-- mav



__________________________________
Do you Yahoo!?
Yahoo! Mail - Find what you need with new enhanced search.
http://info.mail.yahoo.com/mail_250


chris_mahan at yahoo

Jan 18, 2005, 10:19 AM

Post #10 of 15 (423 views)
Permalink
Re: Continued concern with performance [In reply to]

> If our developers lack time, resources, or skill needed to resolve
> the
> issues, then let us immediately hire a temporary consultant to
> focus on
> this problem and resolve it.

Just a tidbit:

You do not speed up the pregnancy by assigning more women to the
project.



=====
Chris Mahan
818.943.1850 cell
chris_mahan [at] yahoo
chris.mahan [at] gmail
http://www.christophermahan.com/




__________________________________
Do you Yahoo!?
Yahoo! Mail - You care about security. So do we.
http://promotions.yahoo.com/new_mail


saintonge at telus

Jan 18, 2005, 1:34 PM

Post #11 of 15 (425 views)
Permalink
Re: Continued concern with performance [In reply to]

Christopher Mahan wrote:

>>If our developers lack time, resources, or skill needed to resolve
>>the issues, then let us immediately hire a temporary consultant to
>>focus on this problem and resolve it.
>>
>>
>Just a tidbit:
>
>You do not speed up the pregnancy by assigning more women to the
>project.
>
>
If the technology were there we might find support for changing that
task into a shared labour. :-)

Ec


chris_mahan at yahoo

Jan 18, 2005, 1:57 PM

Post #12 of 15 (422 views)
Permalink
Re: Continued concern with performance [In reply to]

--- Ray Saintonge <saintonge [at] telus> wrote:

> >Just a tidbit:
> >
> >You do not speed up the pregnancy by assigning more women to the
> >project.
> >
> >
> If the technology were there we might find support for changing
> that
> task into a shared labour. :-)

I have actually been thinking about that concept, that of core
decentralization of the system, to allow for better separation of
the... Ooops, wrong list :)

But, nice pun and double-entendre.

=====
Chris Mahan
818.943.1850 cell
chris_mahan [at] yahoo
chris.mahan [at] gmail
http://www.christophermahan.com/



__________________________________
Do you Yahoo!?
Read only the mail you want - Yahoo! Mail SpamGuard.
http://promotions.yahoo.com/new_mail


jwales at wikia

Jan 19, 2005, 3:12 AM

Post #13 of 15 (422 views)
Permalink
Re: Re: Continued concern with performance [In reply to]

Rich Holton wrote:
> My greatest concern is that there are continuing performance issues
> with little apparent progress over an extended period. Without
> hesitation I presume that we have good people volunteering their time
> and working hard to resolve these issues.

I think you're overlooking our staggering growth rate, though. There
has been enormous progress on a great many fronts, but we continue to
grow grow grow at an astonishing rate.

It isn't as if we have steady traffic and the devs are working this
hard and achieving nothing. It's that we have remarkable progress on
all fronts and enough growth to make it not seem very impressive if
you don't recognize what's going on.

> Sometimes it is necessary for an organization to consider the
> possibility that internal resources are not sufficient to address an
> immediate need. Wikipedia has come a long way relying almost entirely
> on volunteer workers. I do not believe this can continue indefinitely.

I would tend to agree with you on this, but we should not expect paid
work to be a cure-all. In some well-defined areas, yes. But in
general, it is not so clear.

--Jimbo


rich_holton at yahoo

Jan 19, 2005, 9:07 AM

Post #14 of 15 (422 views)
Permalink
Re: Re: Continued concern with performance [In reply to]

--- "Jimmy (JiJimboWales" <jwjwalesiwikiaom> wrote:

> Rich HoHoltonrote:
> > My greatest concern is that there are continuing performance issues
> > with little apparent progress over an extended period. Without
> > hesitation I presume that we have good people volunteering their
> time
> > and working hard to resolve these issues.
>
> I think you're overlooking our staggering growth rate, though. There
> has been enormous progress on a great many fronts, but we continue to
> grow grow grow at an astonishing rate.
>
> It isn't as if we have steady traffic and the dedevsre working this
> hard and achieving nothing. It's that we have remarkable progress on
> all fronts and enough growth to make it not seem very impressive if
> you don't recognize what's going on.

There are continuing performance issues without _apparent_ progress
over an extended period. It's not a question of whether the dedevsre
achieving nothing, it a question of whether WiWikipediaWiWikiMediais
able to handle it own growth. Do we have the organization and structure
necessary to be a top 100 site? Can we effectively respond to spurts in
usage, hardware failures, or the loss of key developers?

I am not bitching about the developers. I've tried to make that clear,
though I may not have always succeeded. We have volunteers who are
making extraordinary efforts, and achieving extraordinary things. But
have we reached a point where that is not enough?

According to Alexa, our traffic rank has dropped significantly over the
past two weeks -- to the point where we are now at a level comparable
to our lolowpointsver Mid November to Mid December.

To look at this as purely a tetechnicalssue is, in my opinion, a huge
mistake. This is a challenge to the entire organization, and
particularly to "the management". That's why I'm bringing this up on
the Foundation mailing list.

If I am missing something here, it's safe to assume that I am not the
only one. There are limited sources of information for the typical
editor:

* The tech section of VP (on en) still points to this LiLiveJournalage:
hthttp/wwwwwilivejournalom/community/wiwikitech432.hthtml* Error
messages point to: hthttp/wwwwwilivejournalom/community/wiwikitechand
also to:
hthttp/opopenfactseberliosedendex-en.phphtmlitle=WiWikipediatatus,
where I see that JaJamesDayas finally given some response to people's
questioning and complaining. Check out the archive for the first part
of January though:
hthttp/opopenfactseberliosedendex-en.phphtmlitle=WiWikipediatatus/archive2005

As far as I can tell, none of these give an adequate indication of what
the issues are and what progress is being made. I would try to add
something somewhere to point to more up-to-date and reliable
information, but I don't know where to find such information.

>
> > Sometimes it is necessary for an organization to consider the
> > possibility that internal resources are not sufficient to address
> an
> > immediate need. WiWikipediaas come a long way relying almost
> entirely
> > on volunteer workers. I do not believe this can continue
> indefinitely.
>
> I would tend to agree with you on this, but we should not expect paid
> work to be a cure-all. In some well-defined areas, yes. But in
> general, it is not so clear.
>

No, there is no panacea. But there are times when a small but focused
action can achieve sisignificantesults.

I very much value what WiWikipedias and stands for. That's exactly why
I'm concerned.

-Rich HoHolton



__________________________________
Do you Yahoo!?
Yahoo! Mail - Helps protect you from nasty viruses.
http://promotions.yahoo.com/new_mail


rich_holton at yahoo

Jan 19, 2005, 9:35 AM

Post #15 of 15 (422 views)
Permalink
Re: Re: Continued concern with performance [In reply to]

--- Rich Holton <rich_holton [at] yahoo> wrote:

> --- "Jimmy (JiJimboWales" <jwjwalesiwikiaom> wrote:
>
> > Rich HoHoltonrote:
> > > My greatest concern is that there are continuing performance
> issues
> > > with little apparent progress over an extended period. Without
> > > hesitation I presume that we have good people volunteering their
> > time
> > > and working hard to resolve these issues.
> >
> > I think you're overlooking our staggering growth rate, though.
> There
> > has been enormous progress on a great many fronts, but we continue
> to
> > grow grow grow at an astonishing rate.
> >
> > It isn't as if we have steady traffic and the dedevsre working this
> > hard and achieving nothing. It's that we have remarkable progress
> on
> > all fronts and enough growth to make it not seem very impressive if
> > you don't recognize what's going on.
>

There are continuing performance issues without _apparent_ progress
over an extended period. It's not a question of whether the dedevsre
achieving nothing, it a question of whether WiWikipediaWiWikiMediais
able to handle it own growth. Do we have the organization and structure
necessary to be a top 100 site? Can we effectively respond to spurts
in usage, hardware failures, or the loss of key developers?

I am not bitching about the developers. I've tried to make that
clear,
though I may not have always succeeded. We have volunteers who are
making extraordinary efforts, and achieving extraordinary things. But
have we reached a point where that is not enough?

According to Alexa, our traffic rank has dropped significantly over
the
past two weeks -- to the point where we are now at a level comparable
to our lolowpointsver Mid November to Mid December.

To look at this as purely a tetechnicalssue is, in my opinion, a huge
mistake. This is a challenge to the entire organization, and
particularly to "the management". That's why I'm bringing this up on
the Foundation mailing list.

If I am missing something here, it's safe to assume that I am not the
only one. There are limited sources of information for the typical
editor:

The tech section of VP (on en) still points to this LiveJournal page:
http://www.livejournal.com/community/wikitech/1432.html

Error messages point to:
http://www.livejournal.com/community/wikitech/

and also to:
http://openfacts.berlios.de/index-en.phtml?title=Wikipedia_status

where I see that JamesDay has finally given some response to people's
questioning and complainings. Check out the archive for the first part
of January though:
http://openfacts.berlios.de/index-en.phtml?title=Wikipedia_status/archive2005

As far as I can tell, none of these give an adequate indication of what
the issues are and what progress is being made. I would try to add
something somewhere to point to more up-to-date and reliable
information, but I don't know where to find such information.

>
> > > Sometimes it is necessary for an organization to consider the
> > > possibility that internal resources are not sufficient to address
> > an
> > > immediate need. WiWikipediaas come a long way relying almost
> > entirely
> > > on volunteer workers. I do not believe this can continue
> > indefinitely.
> >
> > I would tend to agree with you on this, but we should not expect
> paid
> > work to be a cure-all. In some well-defined areas, yes. But in
> > general, it is not so clear.
> >
>
No, there is no panacea. But there are times when a small but focused
action can achieve sisignificantesults.

I very much value what WiWikipedias and stands for. That's exactly why
I'm concerned.

-Rich HoHolton


>
> __________________________________
> Do you Yahoo!?
> Yahoo! Mail - Helps protect you from nasty viruses.
> http://promotions.yahoo.com/new_mail
> _______________________________________________
> foundation-l mailing list
> foundation-l [at] wikimedia
> http://mail.wikipedia.org/mailman/listinfo/foundation-l
>




__________________________________
Do you Yahoo!?
Yahoo! Mail - Helps protect you from nasty viruses.
http://promotions.yahoo.com/new_mail

Wikipedia foundation RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.