Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Wikipedia: Wikitech

Balancing MediaWiki Core/Extensions

 

 

Wikipedia wikitech RSS feed   Index | Next | Previous | View Threaded


tparscal at wikimedia

Sep 21, 2010, 8:34 PM

Post #1 of 22 (2105 views)
Permalink
Balancing MediaWiki Core/Extensions

In response to recent comments in our code review tool about whether
some extensions should be merged into core MediaWiki, or not. I would
like to try and initiate a productive conversation about this topic in
hopes that we can collaboratively define a set of guidelines for
evaluating when to move features out of core and into an extension or
out of an extension and into core.

<unintended bias>

Arguments I have made/observed *against* merging things into core include:

1. Fewer developers have commit access to core, which pushes people
off into branches who would have otherwise been able to contribute
directly to trunk, inhibiting entry-level contribution.
2. Extensions encourage modularity and are easier to learn and work
on because they are smaller sets of code organized in discreet
bundles.
3. We should be looking to make core less inclusive, not more. The
line between Wikipedia and MediaWiki is already blurry enough as
it is.

Arguments I have made/observed *for* merging things into core include:

1. MediaWiki should be awesome out-of-the-box, so extensions that
would be good for virtually everyone seem silly to bury deep
within the poorly organized depths of the extensions folder.
2. When an extension is unable to do what it needs to do because it's
dependent on a limited set of hooks, none of which quite do what
it needs.
3. Because someone said so.

</unintended bias>

<obvious bias>

I will respond to these three pro-integration points; mostly because I
am generally biased against integration and would like to state why. I
realize that there are probably additional pro-integration points that
are far less biased than the three I've listed, but I am basing these on
arguments I've actually seen presented.

1. This is a very valid and important goal, but am unconvinced and
merging extensions into core is the only way to achieve it. We
can, for instance, take advantage the new installer that demon is
working on which has the ability to automate the installation of
extensions at setup-time.
2. This seems like a call for better APIs/a more robust set of hooks.
Integration for this sort of reason is more likely to introduce
cruft and noise than improve the software in any way.
3. Noting that "so-and-so said I should integrate this into core" is
not going to magically absolve anyone of having to stand behind
their decision to proceed with such action and support it with
logic and reason.

</obvious bias>

If we are to develop guidelines for when to push things in/pull things
out of core, it's going to be important that we reach some general
consensus on the merits of these points. This mailing list is not
historically known for its efficacy in consensus building, but I still
feel like this conversation is going to be better off here than in a
series of disjointed code review comments.

- Trevor
_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


niklas.laxstrom at gmail

Sep 21, 2010, 10:58 PM

Post #2 of 22 (2064 views)
Permalink
Re: Balancing MediaWiki Core/Extensions [In reply to]

On 22 September 2010 06:34, Trevor Parscal <tparscal [at] wikimedia> wrote:
> †In response to recent comments in our code review tool about whether
> some extensions should be merged into core MediaWiki, or not. I would
> like to try and initiate a productive conversation about this topic in
> hopes that we can collaboratively define a set of guidelines for
> evaluating when to move features out of core and into an extension or
> out of an extension and into core.
>

I'm generally against adding random extensions to the core, and there
are many things I would like to rip of the core code if possible.

I don't think we can come up with any clear rules. I'd imagine the
rule would just be:
* if it's useful to most users (=installations) of mediawiki, add it to the core
With the "most users" getting a slightly less important if the feature
is small but useful.

I think that for example the new edit toolbar or one-button search are
clearly such usability improvements, that only enhance existing core
features and thus should be in the core, replacing the poor original
ones where possible.

Maybe also code quality and stability plays a role, since making an
extension is kind of like making a branch - you can develop mostly
freely and nobody cares. On the other hand the extension code might
also need extensive rewriting if merged to the core.

We have a quite free access policy even to the trunk code, so I do not
see it as a limitation for including code in the core. On the other
hand putting code in the trunk seems to be the only sure way to get
your code reviewed even a little and deployed into WMF (it may still
take a year).

In short: I agree keeping the core slick and clean, but I think the
case is clear for (at least part of) the usability enhancements.

-Niklas

--
Niklas LaxstrŲm

_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


hashar+wmf at free

Sep 21, 2010, 11:49 PM

Post #3 of 22 (2059 views)
Permalink
Re: Balancing MediaWiki Core/Extensions [In reply to]

On 22/09/10 05:34, Trevor Parscal wrote:
<argument #2 against merging>
> 2. Extensions encourage modularity and are easier to learn and work
> on because they are smaller sets of code organized in discreet
> bundles.

That is probably the most important point. It ease code review, any new
comer can easily understand code.

> Arguments I have made/observed *for* merging things into core include:
>
> 1. MediaWiki should be awesome out-of-the-box, so extensions that
> would be good for virtually everyone seem silly to bury deep
> within the poorly organized depths of the extensions folder.

What about shipping the Mediawiki release with some of the best
extensions ? You do not need to actually merge them with trunk, just
chip mediawiki + some extensions preconfigured. Hooks in the installer
would let user tweak the configuration.
I had a little script which, given a revision number would :
- installed trunk and stable branch
- copy my favorite extensions in the extension folders
- append to local settings my extension configuration files

<snip>
> If we are to develop guidelines for when to push things in/pull things
> out of core, ...

Would you move as extensions : squid support, math, languages, external
stores, external auth, API, chinese support, Tidy, Antivirus ... They
are all awesome features but not necessarily needed in core.

--
Ashar Voultoiz % mv hashar /mw/extensions/


_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


rlane32 at gmail

Sep 22, 2010, 12:11 AM

Post #4 of 22 (2066 views)
Permalink
Re: Balancing MediaWiki Core/Extensions [In reply to]

On Tue, Sep 21, 2010 at 8:34 PM, Trevor Parscal <tparscal [at] wikimedia> wrote:
> †In response to recent comments in our code review tool about whether
> some extensions should be merged into core MediaWiki, or not. I would
> like to try and initiate a productive conversation about this topic in
> hopes that we can collaboratively define a set of guidelines for
> evaluating when to move features out of core and into an extension or
> out of an extension and into core.
>
> <unintended bias>
>
> Arguments I have made/observed *against* merging things into core include:
>
> † 1. Fewer developers have commit access to core, which pushes people
> † † †off into branches who would have otherwise been able to contribute
> † † †directly to trunk, inhibiting entry-level contribution.

People can get access to core. This is a poor argument for not
integrating things into core. If this is a problem, then it is a
problem with us, and our processes, and should be fixed.

> † 2. Extensions encourage modularity and are easier to learn and work
> † † †on because they are smaller sets of code organized in discreet
> † † †bundles.
> † 3. We should be looking to make core less inclusive, not more. The
> † † †line between Wikipedia and MediaWiki is already blurry enough as
> † † †it is.
>

I think this should be the case, but there are certain sets of
functionality that people expect to be installed in a default
installation. A usable editing interface is definitely one of those
things.

> Arguments I have made/observed *for* merging things into core include:
>
> † 1. MediaWiki should be awesome out-of-the-box, so extensions that
> † † †would be good for virtually everyone seem silly to bury deep
> † † †within the poorly organized depths of the extensions folder.

I totally agree with this to an extent. I don't think we should be
including everything and the kitchen sink, but things that make
MediaWiki simply feel broken when they are missing should be
integrated. ParserFunctions is an amazing example of this. MediaWiki
simply doesn't work without ParserFunctions. You can tell me it does,
and that people can live without it, but I refuse to believe that. We
get support issues extremely frequently that end in "install
ParserFunctions".

I'd also like to mention that something I'm seeing in even greater
frequency now is "how do I get Vector to work like it does on
Wikipedia?" or "How do I make the sidebar collapse", or "why doesn't
my search bar work like it does on Wikipedia"?

Why did we expend all of this effort on usability if we are going to
make it a pain to use if you aren't Wikimedia? That is very
anti-usability.

> † 2. When an extension is unable to do what it needs to do because it's
> † † †dependent on a limited set of hooks, none of which quite do what
> † † †it needs.

This goes back to people needing core access to add hooks. In reality,
they can send patches, or ask a dev with core access to do so. I've
had a number of hooks added this way.

> † 3. Because someone said so.
>

Agreed. This is never a reasonable argument.

> </unintended bias>
>
> <obvious bias>
>
> I will respond to these three pro-integration points; mostly because I
> am generally biased against integration and would like to state why. I
> realize that there are probably additional pro-integration points that
> are far less biased than the three I've listed, but I am basing these on
> arguments I've actually seen presented.
>
> † 1. This is a very valid and important goal, but am unconvinced and
> † † †merging extensions into core is the only way to achieve it. We
> † † †can, for instance, take advantage the new installer that demon is
> † † †working on which has the ability to automate the installation of
> † † †extensions at setup-time.

One problem I have every time I upgrade MediaWiki is extension
compatibility. Extension management in MediaWiki is simply broken.
Until we have a reasonable way to handle extension compatibility and
some reasonable means for people to upgrade extensions, this is a
painful course of action.

> † 2. This seems like a call for better APIs/a more robust set of hooks.
> † † †Integration for this sort of reason is more likely to introduce
> † † †cruft and noise than improve the software in any way.

This would help.

> † 3. Noting that "so-and-so said I should integrate this into core" is
> † † †not going to magically absolve anyone of having to stand behind
> † † †their decision to proceed with such action and support it with
> † † †logic and reason.
>

Agreed. We need to be picky about what goes into core.

> </obvious bias>
>
> If we are to develop guidelines for when to push things in/pull things
> out of core, it's going to be important that we reach some general
> consensus on the merits of these points. This mailing list is not
> historically known for its efficacy in consensus building, but I still
> feel like this conversation is going to be better off here than in a
> series of disjointed code review comments.
>

Well, to restate my above opinions: I think in general we don't
integrate enough into core. We create way too many extensions, even
for things that are essentially required to properly run MediaWiki. I
totally understand the want to not bloat our core software, and we
should strive to keep it lean and fast, but we also need to take into
consideration our third party users. Requiring users to install and
manage a million extensions to get basic functionality is asking a
lot.

Respectfully,

Ryan Lane

_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


z at mzmcbride

Sep 22, 2010, 1:30 AM

Post #5 of 22 (2060 views)
Permalink
Re: Balancing MediaWiki Core/Extensions [In reply to]

Trevor Parscal wrote:
> In response to recent comments in our code review tool about whether
> some extensions should be merged into core MediaWiki, or not. I would
> like to try and initiate a productive conversation about this topic in
> hopes that we can collaboratively define a set of guidelines for
> evaluating when to move features out of core and into an extension or
> out of an extension and into core.

I know I mentioned this in a CodeReview comment, but I'll mention it again
here: http://mediawiki.org/wiki/List_of_extensions_to_be_merged_to_the_core

I think developing a set of guidelines for this issue might be nice, though
I'm not sure there's really all that much disagreement about what should and
shouldn't be in core currently.

MZMcBride



_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


tparscal at wikimedia

Sep 22, 2010, 1:33 AM

Post #6 of 22 (2061 views)
Permalink
Re: Balancing MediaWiki Core/Extensions [In reply to]

It seems like there are two dimensions to this problem.

X: Should it be part of stock MediaWiki or not
Y: Is it better to write features as discreet modules or spread them out
the way core is currently done

The problem is when you move across X, you are forced to cross Y; If we
were to start moving extensions into core, it would mean breaking up
otherwise well bundled units of code into lots of tiny parts and
spreading them out. But why must this be the case?

Modular feature development being unique to extensions points out a
significant flaw in the design of MediaWiki core. There's no reason we
can't convert existing core features to discreet components, much like
how extensions are written, while leaving them in stock MediaWiki. This
sort of design would also make light work of adding extensions to core.

- Trevor


On 9/22/10 12:11 AM, Ryan Lane wrote:
> On Tue, Sep 21, 2010 at 8:34 PM, Trevor Parscal<tparscal [at] wikimedia> wrote:
>> In response to recent comments in our code review tool about whether
>> some extensions should be merged into core MediaWiki, or not. I would
>> like to try and initiate a productive conversation about this topic in
>> hopes that we can collaboratively define a set of guidelines for
>> evaluating when to move features out of core and into an extension or
>> out of an extension and into core.
>>
>> <unintended bias>
>>
>> Arguments I have made/observed *against* merging things into core include:
>>
>> 1. Fewer developers have commit access to core, which pushes people
>> off into branches who would have otherwise been able to contribute
>> directly to trunk, inhibiting entry-level contribution.
> People can get access to core. This is a poor argument for not
> integrating things into core. If this is a problem, then it is a
> problem with us, and our processes, and should be fixed.
>
>> 2. Extensions encourage modularity and are easier to learn and work
>> on because they are smaller sets of code organized in discreet
>> bundles.
>> 3. We should be looking to make core less inclusive, not more. The
>> line between Wikipedia and MediaWiki is already blurry enough as
>> it is.
>>
> I think this should be the case, but there are certain sets of
> functionality that people expect to be installed in a default
> installation. A usable editing interface is definitely one of those
> things.
>
>> Arguments I have made/observed *for* merging things into core include:
>>
>> 1. MediaWiki should be awesome out-of-the-box, so extensions that
>> would be good for virtually everyone seem silly to bury deep
>> within the poorly organized depths of the extensions folder.
> I totally agree with this to an extent. I don't think we should be
> including everything and the kitchen sink, but things that make
> MediaWiki simply feel broken when they are missing should be
> integrated. ParserFunctions is an amazing example of this. MediaWiki
> simply doesn't work without ParserFunctions. You can tell me it does,
> and that people can live without it, but I refuse to believe that. We
> get support issues extremely frequently that end in "install
> ParserFunctions".
>
> I'd also like to mention that something I'm seeing in even greater
> frequency now is "how do I get Vector to work like it does on
> Wikipedia?" or "How do I make the sidebar collapse", or "why doesn't
> my search bar work like it does on Wikipedia"?
>
> Why did we expend all of this effort on usability if we are going to
> make it a pain to use if you aren't Wikimedia? That is very
> anti-usability.
>
>> 2. When an extension is unable to do what it needs to do because it's
>> dependent on a limited set of hooks, none of which quite do what
>> it needs.
> This goes back to people needing core access to add hooks. In reality,
> they can send patches, or ask a dev with core access to do so. I've
> had a number of hooks added this way.
>
>> 3. Because someone said so.
>>
> Agreed. This is never a reasonable argument.
>
>> </unintended bias>
>>
>> <obvious bias>
>>
>> I will respond to these three pro-integration points; mostly because I
>> am generally biased against integration and would like to state why. I
>> realize that there are probably additional pro-integration points that
>> are far less biased than the three I've listed, but I am basing these on
>> arguments I've actually seen presented.
>>
>> 1. This is a very valid and important goal, but am unconvinced and
>> merging extensions into core is the only way to achieve it. We
>> can, for instance, take advantage the new installer that demon is
>> working on which has the ability to automate the installation of
>> extensions at setup-time.
> One problem I have every time I upgrade MediaWiki is extension
> compatibility. Extension management in MediaWiki is simply broken.
> Until we have a reasonable way to handle extension compatibility and
> some reasonable means for people to upgrade extensions, this is a
> painful course of action.
>
>> 2. This seems like a call for better APIs/a more robust set of hooks.
>> Integration for this sort of reason is more likely to introduce
>> cruft and noise than improve the software in any way.
> This would help.
>
>> 3. Noting that "so-and-so said I should integrate this into core" is
>> not going to magically absolve anyone of having to stand behind
>> their decision to proceed with such action and support it with
>> logic and reason.
>>
> Agreed. We need to be picky about what goes into core.
>
>> </obvious bias>
>>
>> If we are to develop guidelines for when to push things in/pull things
>> out of core, it's going to be important that we reach some general
>> consensus on the merits of these points. This mailing list is not
>> historically known for its efficacy in consensus building, but I still
>> feel like this conversation is going to be better off here than in a
>> series of disjointed code review comments.
>>
> Well, to restate my above opinions: I think in general we don't
> integrate enough into core. We create way too many extensions, even
> for things that are essentially required to properly run MediaWiki. I
> totally understand the want to not bloat our core software, and we
> should strive to keep it lean and fast, but we also need to take into
> consideration our third party users. Requiring users to install and
> manage a million extensions to get basic functionality is asking a
> lot.
>
> Respectfully,
>
> Ryan Lane
>
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-l [at] lists
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l

_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


roan.kattouw at gmail

Sep 22, 2010, 5:40 AM

Post #7 of 22 (2055 views)
Permalink
Re: Balancing MediaWiki Core/Extensions [In reply to]

2010/9/22 Trevor Parscal <tparscal [at] wikimedia>:
> Modular feature development being unique to extensions points out a
> significant flaw in the design of MediaWiki core. There's no reason we
> can't convert existing core features to discreet components, much like
> how extensions are written, while leaving them in stock MediaWiki. This
> sort of design would also make light work of adding extensions to core.
>
Making MediaWiki more modular won't magically make it possible (or
even desirable) to write any old feature as an extension. Some things
will still have to go in core and some things we'll simply /want/ to
put in core because making them extensions would be ridiculous for
some reason. It's true that more modularity will lower the difference
in awkwardness between doing certain things as extensions vs. in core
(and is generally desirable), but I also don't see how it would make
(much) lighter work of adding extensions to core: most extensions
should either be easy to add to core (Vector and WikiEditor for
instance, any special page extension) or cause some bumps because they
were written in an awkward way to begin with, probably because it'd
have been better to put them in core.

Roan Kattouw (Catrope)

_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


innocentkiller at gmail

Sep 22, 2010, 5:52 AM

Post #8 of 22 (2055 views)
Permalink
Re: Balancing MediaWiki Core/Extensions [In reply to]

On Tue, Sep 21, 2010 at 11:34 PM, Trevor Parscal <tparscal [at] wikimedia> wrote:
> when to move features out of core and into an extension or
> out of an extension and into core.
>

I don't think anyone's commented on the former (everyone's been
talking about pushing in, not pulling out). IMO, the conditions for
splitting something into an extension

A) Not a lot of people use it anyway (hard to gauge)
B) It probably shouldn't have been in core in the first place (eg: AskSQL)

So far, the only successful case I can think of offhand for splitting
an extension out was AskSQL, but it's a perfect example of what
should happen.

>   2. When an extension is unable to do what it needs to do because it's
>      dependent on a limited set of hooks, none of which quite do what
>      it needs.
>

I think this also makes a good case for keeping something in core
and not removing it. If we need to add 6 new hooks to keep it working,
then it might be best to keep it in core (single purpose hooks are lame)

>   1. This is a very valid and important goal, but am unconvinced and
>      merging extensions into core is the only way to achieve it. We
>      can, for instance, take advantage the new installer that demon is
>      working on which has the ability to automate the installation of
>      extensions at setup-time.
>

Quick note on the installer. It only enables extensions that already
reside in your extensions folder. Since we don't distribute any with
the default package, this might not be terribly useful. More awesome
is Jeroen's GSoC work on extension management. Something to look
at post-1.17 branch though.

>   3. Because someone said so.
>

If enough people say so ;-)

-Chad

_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


dgerard at gmail

Sep 22, 2010, 6:55 AM

Post #9 of 22 (2063 views)
Permalink
Re: Balancing MediaWiki Core/Extensions [In reply to]

On 22 September 2010 13:52, Chad <innocentkiller [at] gmail> wrote:

> Quick note on the installer. It only enables extensions that already
> reside in your extensions folder. Since we don't distribute any with
> the default package, this might not be terribly useful. More awesome
> is Jeroen's GSoC work on extension management. Something to look
> at post-1.17 branch though.


Note from the perspective of a tarball user: yes please. Though code
management may be much more sensible with things organised into the
extensions folder, having a basic set of extensions included by
default in the tarball would be most helpful to the sysadmin whose
user just asked for a new wiki.

(I'm thinking by analogy with operating systems - a "base" set of
extensions which are a good working set to start with, but which you
can strip right down to "core" if you really want to.)


- d.

_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


agarrett at wikimedia

Sep 22, 2010, 7:17 AM

Post #10 of 22 (2058 views)
Permalink
Re: Balancing MediaWiki Core/Extensions [In reply to]

On Wed, Sep 22, 2010 at 6:33 PM, Trevor Parscal <tparscal [at] wikimedia> wrote:
> Modular feature development being unique to extensions points out a
> significant flaw in the design of MediaWiki core. There's no reason we
> can't convert existing core features to discreet components, much like
> how extensions are written, while leaving them in stock MediaWiki. This
> sort of design would also make light work of adding extensions to core.

I think that I would support allowing core features to use hooks
instead of writing the code in all over the place. All you would have
to do is initialise $wgHooks somewhere before LocalSettings.php is
run.

Giving structural proximity to functionally-related code would make
things much less confusing for new developers (and simpler for those
with experience). It would allow us to declutter our core code (things
like user permission checking come to mind), and it would make light
work of merging extensions into core.

--
Andrew Garrett
http://werdn.us/

_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


Simetrical+wikilist at gmail

Sep 22, 2010, 9:49 AM

Post #11 of 22 (2053 views)
Permalink
Re: Balancing MediaWiki Core/Extensions [In reply to]

On Tue, Sep 21, 2010 at 11:34 PM, Trevor Parscal <tparscal [at] wikimedia> wrote:
>  In response to recent comments in our code review tool about whether
> some extensions should be merged into core MediaWiki, or not. I would
> like to try and initiate a productive conversation about this topic in
> hopes that we can collaboratively define a set of guidelines for
> evaluating when to move features out of core and into an extension or
> out of an extension and into core.

Partly this is a matter of preference. I personally write things
almost entirely for core -- I've never written an extension, so my
only extension commits are to things that other people wrote as
extensions. Other people prefer to write extensions instead of core
features. As long as the code gets written and works either way, I
don't think it's usually a huge deal. A lot of stuff will work fine
either in core or as an extension -- this doesn't have to be made a
religious issue or one where we have detailed official policies.

What I think is clear is that anything that the large majority of
MediaWikis would benefit from should be enabled by default, which
given our current setup means it has to be in core. I haven't looked
closely, but I believe this includes a significant part of the
usability stuff (although not all). MediaWiki should work as well as
possible for as many users as possible out of the box, and whatever
that implies for how the code is structured is what we should do. I
cannot conceive of any reason why most of the features of the Vector
extension should not be in core, for instance -- why would you *not*
want them? Why would you ever want to disable them? ("Admins should
have choice" is not a good reason by itself. Cf.:
http://weblogs.mozillazine.org/bz/archives/020683.html)

Other than things you want to enable by default, here are some reasons
to put some features in core rather than extensions:

* Core features are much easier for site admins to enable than
extensions. It would be great if this were fixed, but currently, it's
not, so it can't be dismissed. You can dismiss this reason (basically
your #1) as soon as it's become irrelevant in practice, but not
before.
* For smaller features, it will take a lot more code to write them as
extensions than as part of core. You can often add a useful core
feature in less than ten lines of code -- extensions are much
heavier-weight. This is unavoidable, because extensions have to have
a whole bunch of metadata to specify where their code executes and so
on; with core code you can just put it in the right place.
* It's easier to understand how core code works in the absence of
extensions, because the code is explicitly written out where it
occurs. When extensions are involved, you have a wfRunHooks() line in
core, which basically means "absolutely anything could happen at this
point". When you run into a hook when reading code, it doesn't only
complicate understanding how the code will run in practice, it also
makes it much harder to change. If you want to refactor, you have to
write a lot of painstaking and ugly logic to preserve the semantics of
the hooks, or else just give up and get rid of them.

On Wed, Sep 22, 2010 at 4:33 AM, Trevor Parscal <tparscal [at] wikimedia> wrote:
> Modular feature development being unique to extensions points out a
> significant flaw in the design of MediaWiki core. There's no reason we
> can't convert existing core features to discreet components, much like
> how extensions are written, while leaving them in stock MediaWiki.

This can be accomplished by regular old object-oriented programming.
Keep all the related code in one class, and make as much possible
private so that classes can only call a limited number of
well-defined, well-documented, easily-understood entry points. I
don't think extensions are a good model for this -- to the contrary,
they're antithetical to modularity, since any extension can
drastically change how any part of the program behaves (given enough
hooks).

On Wed, Sep 22, 2010 at 10:17 AM, Andrew Garrett <agarrett [at] wikimedia> wrote:
> I think that I would support allowing core features to use hooks
> instead of writing the code in all over the place. All you would have
> to do is initialise $wgHooks somewhere before LocalSettings.php is
> run.
>
> Giving structural proximity to functionally-related code would make
> things much less confusing for new developers (and simpler for those
> with experience). It would allow us to declutter our core code (things
> like user permission checking come to mind), and it would make light
> work of merging extensions into core.

This is reminiscent of COMEFROM. It makes it really hard to
understand what's going on if you have the code scattered everywhere.
Modularity is great, but it needs to happen along clear boundaries to
keep the code comprehensible -- not "anything can happen at this
point".

_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


tparscal at wikimedia

Sep 22, 2010, 10:57 AM

Post #12 of 22 (2039 views)
Permalink
Re: Balancing MediaWiki Core/Extensions [In reply to]

On 9/22/10 9:49 AM, Aryeh Gregor wrote:
> Partly this is a matter of preference. I personally write things
> almost entirely for core -- I've never written an extension, so my
> only extension commits are to things that other people wrote as
> extensions. Other people prefer to write extensions instead of core
> features. As long as the code gets written and works either way, I
> don't think it's usually a huge deal. A lot of stuff will work fine
> either in core or as an extension -- this doesn't have to be made a
> religious issue or one where we have detailed official policies.
I certainly never mentioned anything about making this a religious issue
(metaphor understood) but I think being able to point to a simple
guideline on the wiki would be an improvement on the current situation.

> This can be accomplished by regular old object-oriented programming.
> Keep all the related code in one class, and make as much possible
> private so that classes can only call a limited number of
> well-defined, well-documented, easily-understood entry points. I
> don't think extensions are a good model for this -- to the contrary,
> they're antithetical to modularity, since any extension can
> drastically change how any part of the program behaves (given enough
> hooks).
>
And what about javascript and css files? What about messages? Generally
speaking lots of the MediaWiki PHP code is modular, but it's the rest of
the stuff that goes into making a feature work that is not. Extensions
bundle this all together, and that proximity makes maintenance and
review easier.

- Trevor

_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


tstarling at wikimedia

Sep 22, 2010, 10:02 PM

Post #13 of 22 (2042 views)
Permalink
Re: Balancing MediaWiki Core/Extensions [In reply to]

I suppose I have a foot in each camp here.

Ryan Lane wrote:
> ParserFunctions is an amazing example of this. MediaWiki
> simply doesn't work without ParserFunctions. You can tell me it does,
> and that people can live without it, but I refuse to believe that. We
> get support issues extremely frequently that end in "install
> ParserFunctions".

I wrote ParserFunctions under protest, and I hoped at the time that it
would not be essential or widely used. That's why it was an extension.
Maybe the time has come to merge it in.

I also wrote the ImageMap extension, which some might argue should be
integrated.

Aryeh Gregor wrote:
> Partly this is a matter of preference. I personally write things
> almost entirely for core -- I've never written an extension, so my
> only extension commits are to things that other people wrote as
> extensions.

I agree that there is an element of personal preference. Brion was
certainly in favour of putting lots of functionality in extensions,
and, like Trevor, used the argument that at some point in the future,
extensions will be easier to install and/or bundled with core
releases. Brion wrote the Poem extension, for instance, which is
easily as trivial as Trevor's ArticleEmblems extension.

My view is somewhat more moderate. I think that:

* Trivial, uncontroversial things like EditWarning should be in the core.
* MediaWiki should be usable without extensions, some basic set of
functionality is required.
* Complex functionality, large code size, complex installation and
external dependencies are all arguments for splitting a feature out to
an extension.

Trevor Parscal wrote:
> In response to recent comments in our code review tool about whether
> some extensions should be merged into core MediaWiki, or not. I would
> like to try and initiate a productive conversation about this topic in
> hopes that we can collaboratively define a set of guidelines for
> evaluating when to move features out of core and into an extension or
> out of an extension and into core.

I think there is a need for judgement, and a definitive set of
guidelines will be difficult to produce.

> 3. Noting that "so-and-so said I should integrate this into core" is
> not going to magically absolve anyone of having to stand behind
> their decision to proceed with such action and support it with
> logic and reason.

Yes, however, community consensus should be respected. The logic and
reason you've been expressing in this thread does not appear to be
particularly more convincing than anyone else's.

Aryeh Gregor wrote:
> * It's easier to understand how core code works in the absence of
> extensions, because the code is explicitly written out where it
> occurs. When extensions are involved, you have a wfRunHooks() line in
> core, which basically means "absolutely anything could happen at this
> point". When you run into a hook when reading code, it doesn't only
> complicate understanding how the code will run in practice, it also
> makes it much harder to change. If you want to refactor, you have to
> write a lot of painstaking and ugly logic to preserve the semantics of
> the hooks, or else just give up and get rid of them.

I agree with this. The point of modularity is to separate concerns,
and a hook-based model is opposed to this. A well-designed modular
architecture has a simple, well-documented interface between the
modules. Some of our extensions are a long way from this.

Nowadays I prefer adding object-oriented extension interfaces, which
work by having the extension subclass a core base class. See for
instance MediaHandler, FileRepo, LBFactory and PoolCounter.

Trevor Parscal wrote:
> And what about javascript and css files? What about messages? Generally
> speaking lots of the MediaWiki PHP code is modular, but it's the rest of
> the stuff that goes into making a feature work that is not. Extensions
> bundle this all together, and that proximity makes maintenance and
> review easier.

There are security problems with having web-accessible files like
JavaScript and CSS in the same directory as non-web-accessible files
like PHP includes and command-line scripts. Perhaps the resource
loader will fix this, and we'll be able to organise such files by
module in the future.

There is a significant performance advantage to splitting up messages
by language. However it is possible to have module-specific message
files in the core -- we have one now for the new installer.

-- Tim Starling


_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


z at mzmcbride

Sep 23, 2010, 10:11 AM

Post #14 of 22 (2046 views)
Permalink
Re: Balancing MediaWiki Core/Extensions [In reply to]

Tim Starling wrote:
> Ryan Lane wrote:
>> ParserFunctions is an amazing example of this. MediaWiki
>> simply doesn't work without ParserFunctions. You can tell me it does,
>> and that people can live without it, but I refuse to believe that. We
>> get support issues extremely frequently that end in "install
>> ParserFunctions".
>
> I wrote ParserFunctions under protest, and I hoped at the time that it
> would not be essential or widely used. That's why it was an extension.
> Maybe the time has come to merge it in.

To me, this has consistently seemed like a "perfect is the enemy of done"
issue. I understand your reluctance to merge ParserFunctions into core, but
there really doesn't appear to be any viable alternative on the horizon. For
years now, we've added extra hassle to nearly every MediaWiki installation
under the pretense that one day, there might be something better than
ParserFunctions.

Similar arguments apply to StringFunctions, which are intentionally disabled
on Wikimedia wikis, even though people have created far worse hacks and
workarounds using functions like {{padleft:}}.

If there's a realistic goal and timeline for an better template syntax
system (JavaScript, Lua, whatever), I think it would be fine to continue
putting this off. But if it really is a pipe dream, I don't think it's
sensible to continue shipping MediaWiki without at least ParserFunctions.

> I also wrote the ImageMap extension, which some might argue should be
> integrated.

I'm not sure who might be arguing for ImageMap to be in core. The largest
use-case of ImageMap I ever saw was as a hack to override the link
destination of images, which was finally properly integrated into image
syntax (by you, as it turns out). I don't think most wiki installations
would find any value in this extension.

MZMcBride



_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


saper at saper

Sep 24, 2010, 12:30 AM

Post #15 of 22 (2044 views)
Permalink
Re: Balancing MediaWiki Core/Extensions [In reply to]

>> Roan Kattouw <roan.kattouw [at] gmail> wrote:
> 2010/9/22 Trevor Parscal <tparscal [at] wikimedia>:
>> Modular feature development being unique to extensions points out a
>> significant flaw in the design of MediaWiki core. There's no reason we
>> can't convert existing core features to discreet components, much like
>> how extensions are written, while leaving them in stock MediaWiki. This
>> sort of design would also make light work of adding extensions to core.
>>
> Making MediaWiki more modular won't magically make it possible (or
> even desirable) to write any old feature as an extension. Some things
> will still have to go in core and some things we'll simply /want/ to
> put in core because making them extensions would be ridiculous for
> some reason.

I'd rather have MediaWiki build on some classes & object instances
with a clear responsibilities, Inversion of Control and possibility
to test each _object_ separately without causing interference
to other components. Discussion what's core/extension is to me
secondary. Maybe at some point "hooks" as we know them will
not be needed, we could be able to use interfaces as
provided by the programming language instead of hooks,
possibly (yes, I'm dreaming) tied together by something like
PicoContainer.

Even still, those interfaces *will* change and it would be
good if developers could refactor the code in the core
and extensions at the same time.

I don't why we really duplicate so much functions even
within core (API vs. standard special pages for example).
But that's probably the issue to be solved in phase4 :)

So before asking how much to add into core, maybe we should
first clean up some, and then possibly add. Or sometimes
adding something (like a proper multi-wiki configuration
management $wgConf++) may clean up and simplify some things
inside core.

//Marcin


_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


innocentkiller at gmail

Sep 24, 2010, 5:19 AM

Post #16 of 22 (2040 views)
Permalink
Re: Balancing MediaWiki Core/Extensions [In reply to]

On Fri, Sep 24, 2010 at 3:30 AM, Marcin Cieslak <saper [at] saper> wrote:
> So before asking how much to add into core, maybe we should
> first clean up some, and then possibly add. Or sometimes
> adding something (like a proper multi-wiki configuration
> management $wgConf++) may clean up and simplify some things
> inside core.
>

$wgConf sucks ;-)

But proper configuration management is on the horizon.

-Chad

_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


Platonides at gmail

Sep 28, 2010, 4:15 PM

Post #17 of 22 (2002 views)
Permalink
Re: Balancing MediaWiki Core/Extensions [In reply to]

Chad wrote:
> On Tue, Sep 21, 2010 at 11:34 PM, Trevor Parscal wrote:
>> when to move features out of core and into an extension or
>> out of an extension and into core.
>>
>
> I don't think anyone's commented on the former (everyone's been
> talking about pushing in, not pulling out). IMO, the conditions for
> splitting something into an extension
>
> A) Not a lot of people use it anyway (hard to gauge)
> B) It probably shouldn't have been in core in the first place (eg: AskSQL)
>
> So far, the only successful case I can think of offhand for splitting
> an extension out was AskSQL, but it's a perfect example of what
> should happen.

DumpHTML was also split from core. texvc should have been moved out of
core, but since that would change things set there from the mwbeginning,
nobody did it yet.



>> 1. This is a very valid and important goal, but am unconvinced and
>> merging extensions into core is the only way to achieve it. We
>> can, for instance, take advantage the new installer that demon is
>> working on which has the ability to automate the installation of
>> extensions at setup-time.
>>
>
> Quick note on the installer. It only enables extensions that already
> reside in your extensions folder. Since we don't distribute any with
> the default package, this might not be terribly useful. More awesome
> is Jeroen's GSoC work on extension management. Something to look
> at post-1.17 branch though.

I think the point is to start shipping mediawiki with common extensions
there.


_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


krinklemail at gmail

Sep 28, 2010, 5:11 PM

Post #18 of 22 (2000 views)
Permalink
Re: Balancing MediaWiki Core/Extensions [In reply to]

Perhaps the new installer could contain that as an option during the
inital setup.
Like a two or three-column thing with a bunch of checkboxes.

Language: English [\/]
Default theme (X) Vector (_) Monobook (_) Foobar
Common Extension: [X] ParserFunctions [X] SpecialInterwiki
[X] Cite.php [X] CharInsert
[X] CategoryTree

etc. you get the idea

Op 29 sep 2010, om 01:15 heeft Platonides het volgende geschreven:

> Chad wrote:
>> On Tue, Sep 21, 2010 at 11:34 PM, Trevor Parscal wrote:
>>> when to move features out of core and into an extension or
>>> out of an extension and into core.
>>>
>>
>> I don't think anyone's commented on the former (everyone's been
>> talking about pushing in, not pulling out). IMO, the conditions for
>> splitting something into an extension
>>
>> A) Not a lot of people use it anyway (hard to gauge)
>> B) It probably shouldn't have been in core in the first place (eg:
>> AskSQL)
>>
>> So far, the only successful case I can think of offhand for splitting
>> an extension out was AskSQL, but it's a perfect example of what
>> should happen.
>
> DumpHTML was also split from core. texvc should have been moved out of
> core, but since that would change things set there from the
> mwbeginning,
> nobody did it yet.
>
>
>
>>> 1. This is a very valid and important goal, but am unconvinced and
>>> merging extensions into core is the only way to achieve it. We
>>> can, for instance, take advantage the new installer that demon
>>> is
>>> working on which has the ability to automate the installation of
>>> extensions at setup-time.
>>>
>>
>> Quick note on the installer. It only enables extensions that already
>> reside in your extensions folder. Since we don't distribute any with
>> the default package, this might not be terribly useful. More awesome
>> is Jeroen's GSoC work on extension management. Something to look
>> at post-1.17 branch though.
>
> I think the point is to start shipping mediawiki with common
> extensions
> there.
>
>
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-l [at] lists
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l

_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


krinklemail at gmail

Sep 28, 2010, 5:13 PM

Post #19 of 22 (2003 views)
Permalink
Re: Balancing MediaWiki Core/Extensions [In reply to]

Op 29 sep 2010, om 01:15 heeft Platonides het volgende geschreven:
>
> I think the point is to start shipping mediawiki with common
> extensions
> there.
>
>
Yeah, that sounds good !
Perhaps the new installer could contain that as an option during the
inital setup.
Like a two or three-column thing with a bunch of checkboxes.

Language: English [\/]
Default theme (X) Vector (_) Monobook (_) Foobar
Common Extension: [X] ParserFunctions [X] SpecialInterwiki
[X] Cite.php [X] CharInsert
[X] CategoryTree

etc. you get the idea

--
Krinkle
_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


soxred93 at gmail

Sep 28, 2010, 5:37 PM

Post #20 of 22 (1995 views)
Permalink
Re: Balancing MediaWiki Core/Extensions [In reply to]

That seems like a good idea. I would love to see this in action.

-X!


Sent from my iPod

On Sep 28, 2010, at 8:11 PM, Krinkle <krinklemail [at] gmail> wrote:

> Perhaps the new installer could contain that as an option during the
> inital setup.
> Like a two or three-column thing with a bunch of checkboxes.
>
> Language: English [\/]
> Default theme (X) Vector (_) Monobook (_) Foobar
> Common Extension: [X] ParserFunctions [X] SpecialInterwiki
> [X] Cite.php [X] CharInsert
> [X] CategoryTree
>
> etc. you get the idea
>
> Op 29 sep 2010, om 01:15 heeft Platonides het volgende geschreven:
>
>> Chad wrote:
>>> On Tue, Sep 21, 2010 at 11:34 PM, Trevor Parscal wrote:
>>>> when to move features out of core and into an extension or
>>>> out of an extension and into core.
>>>>
>>>
>>> I don't think anyone's commented on the former (everyone's been
>>> talking about pushing in, not pulling out). IMO, the conditions for
>>> splitting something into an extension
>>>
>>> A) Not a lot of people use it anyway (hard to gauge)
>>> B) It probably shouldn't have been in core in the first place (eg:
>>> AskSQL)
>>>
>>> So far, the only successful case I can think of offhand for splitting
>>> an extension out was AskSQL, but it's a perfect example of what
>>> should happen.
>>
>> DumpHTML was also split from core. texvc should have been moved out of
>> core, but since that would change things set there from the
>> mwbeginning,
>> nobody did it yet.
>>
>>
>>
>>>> 1. This is a very valid and important goal, but am unconvinced and
>>>> merging extensions into core is the only way to achieve it. We
>>>> can, for instance, take advantage the new installer that demon
>>>> is
>>>> working on which has the ability to automate the installation of
>>>> extensions at setup-time.
>>>>
>>>
>>> Quick note on the installer. It only enables extensions that already
>>> reside in your extensions folder. Since we don't distribute any with
>>> the default package, this might not be terribly useful. More awesome
>>> is Jeroen's GSoC work on extension management. Something to look
>>> at post-1.17 branch though.
>>
>> I think the point is to start shipping mediawiki with common
>> extensions
>> there.
>>
>>
>> _______________________________________________
>> Wikitech-l mailing list
>> Wikitech-l [at] lists
>> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-l [at] lists
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l

_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


rlane32 at gmail

Sep 28, 2010, 6:01 PM

Post #21 of 22 (2002 views)
Permalink
Re: Balancing MediaWiki Core/Extensions [In reply to]

> I think the point is to start shipping mediawiki with common extensions
> there.
>

This is great if we are going to version the extensions with the
version of MediaWiki shipped, and backport changes to those extensions
to ensure they get security and bug fixes like core does.
Alternatively, we could provide a way for extensions to be easily
upgraded to compatible versions, similar to wordpress and other sane
applications.

If we don't plan on doing one of two things, then we are forcing our
users into the crapshoot that is finding the correct extension
version, or possibly running insecure or buggy extensions.

Respectfully,

Ryan Lane

_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


bryan.tongminh at gmail

Sep 29, 2010, 12:45 AM

Post #22 of 22 (1994 views)
Permalink
Re: Balancing MediaWiki Core/Extensions [In reply to]

On Wed, Sep 29, 2010 at 3:01 AM, Ryan Lane <rlane32 [at] gmail> wrote:
>
> If we don't plan on doing one of two things, then we are forcing our
> users into the crapshoot that is finding the correct extension
> version, or possibly running insecure or buggy extensions.
>
Like we are doing today. So yes, indeed something that we should fix.

_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Wikipedia wikitech RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.