Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Wikipedia: Wikitech

MediaWiki extensions as core-like libraries: MediaWiki's fun new landmine for admins

 

 

First page Previous page 1 2 Next page Last page  View All Wikipedia wikitech RSS feed   Index | Next | Previous | View Threaded


rlane32 at gmail

Jul 19, 2013, 11:10 AM

Post #1 of 50 (919 views)
Permalink
MediaWiki extensions as core-like libraries: MediaWiki's fun new landmine for admins

I attempted to install Wikibase the other day and made a fun discovery.
Installing it properly requires the following (12) extensions:

WikibaseClient
Wikibase DataModel
WikibaseLib
Wikibase Repository
DataValues
DataTypes
ValueParsers
ValueView
ValueValidators
ValueFormatters
Diff
Scribunto

And the following optional (2) extensions:

Universal Language Selector
Babel

Soon it will also require at least the following (4) extensions:

Ask
Wikibase Query
Wikibase Database
Wikibase QueryEngine

To fully deploy wikibase in a way that will work like wikidata, it will
take at least 18 extensions, all of which are versioned differently, and
are broken out in this manner to be used as libraries.

What this is subtly doing is adding another dependency chain into
MediaWiki: extension libraries. Since these extensions are meant to be used
as libraries, other extensions will eventually do so and admins will have
to worry about not only extension compatibility with MediaWiki (an already
nearly impossible task), but will also need to worry about extension
dependency with extension libraries. The compatibility matrix for this is
going to be terrible and exacerbates one of MediaWiki's biggest problems
for admins.

Quite a few of these should be core functionality or if they can't properly
pass review they should be removed. For legitimate library-like extensions
I have no constructive alternative, but there must be some sane alternative
to this.

- Ryan
_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


cananian at wikimedia

Jul 19, 2013, 11:20 AM

Post #2 of 50 (900 views)
Permalink
Re: MediaWiki extensions as core-like libraries: MediaWiki's fun new landmine for admins [In reply to]

It might also be worth going a "wikibase" release (on its own schedule or
else scheduled shortly after each MW release), that contains a particular
MW version along with version of the extensions that have been tested to
work with it. This is the usual way in which these sorts of dependency
chains are dealt with outside of WMF. (Ie, when RedHat Fedora is released,
it contains the latest GNOME at that time, which in turn contains a version
of GTK+ and various utilities all tested to work well together).

For the admin, it should just be a matter of using "wikibase 1.23" or
whatever; they shouldn't have to hunt down individual extensions and figure
out compatible versions for themselves.
--scott

--
(http://cscott.net)
_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


rlane32 at gmail

Jul 19, 2013, 11:27 AM

Post #3 of 50 (903 views)
Permalink
Re: MediaWiki extensions as core-like libraries: MediaWiki's fun new landmine for admins [In reply to]

On Fri, Jul 19, 2013 at 11:20 AM, C. Scott Ananian
<cananian [at] wikimedia>wrote:

> It might also be worth going a "wikibase" release (on its own schedule or
> else scheduled shortly after each MW release), that contains a particular
> MW version along with version of the extensions that have been tested to
> work with it. This is the usual way in which these sorts of dependency
> chains are dealt with outside of WMF. (Ie, when RedHat Fedora is released,
> it contains the latest GNOME at that time, which in turn contains a version
> of GTK+ and various utilities all tested to work well together).
>

For the admin, it should just be a matter of using "wikibase 1.23" or
> whatever; they shouldn't have to hunt down individual extensions and figure
> out compatible versions for themselves.
>


What if you want to use wikibase 1.23 and MyAbominationExtension 1.5 that
requires an incompatible version of DataValues and MyAwesomeExtension 1.0
that requires an incompatible version of ValueView?

- Ryan
_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


jeroendedauw at gmail

Jul 19, 2013, 11:44 AM

Post #4 of 50 (904 views)
Permalink
Re: MediaWiki extensions as core-like libraries: MediaWiki's fun new landmine for admins [In reply to]

Hey,

What if you want to use wikibase 1.23 and MyAbominationExtension 1.5 that
> requires an incompatible version of DataValues and MyAwesomeExtension 1.0
> that requires an incompatible version of ValueView?
>

If you have releases of certain software that have requirements that cannot
satisfied together, then you cannot install them together. That is a pretty
inherent property of incompatible software.

As a user, when I run into such a situation, what I want to know is which
versions of the software I am interested in I can install together. That
is, after being told the latest releases do not work together. Sounds like
we need some kind of package management :) In case of the components
created for Wikidata, we have been supporting Composer for a while now,
which is a great fit to our needs.

I attempted to install Wikibase the other day and made a fun discovery.
> Installing it properly requires the following (12) extensions:
>

That is somewhat inaccurate, and is misleading with regard to Wikibase
installation. Nevertheless, the concerns you bring up are certainly
relevant, and currently not really tackled well in the MediaWiki community.
That is to bad, as it encourages people to inappropriately bundle things
and throw re usability out of the window (plus causing a long list of other
problems).

Cheers

--
Jeroen De Dauw
http://www.bn2vs.com
Don't panic. Don't be evil. ~=[,,_,,]:3
--
_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


tylerromeo at gmail

Jul 19, 2013, 11:46 AM

Post #5 of 50 (903 views)
Permalink
Re: MediaWiki extensions as core-like libraries: MediaWiki's fun new landmine for admins [In reply to]

Man, if only PHP had some sort of dependency management system.....

*-- *
*Tyler Romeo*
Stevens Institute of Technology, Class of 2016
Major in Computer Science
www.whizkidztech.com | tylerromeo [at] gmail


On Fri, Jul 19, 2013 at 2:44 PM, Jeroen De Dauw <jeroendedauw [at] gmail>wrote:

> Hey,
>
> What if you want to use wikibase 1.23 and MyAbominationExtension 1.5 that
> > requires an incompatible version of DataValues and MyAwesomeExtension 1.0
> > that requires an incompatible version of ValueView?
> >
>
> If you have releases of certain software that have requirements that cannot
> satisfied together, then you cannot install them together. That is a pretty
> inherent property of incompatible software.
>
> As a user, when I run into such a situation, what I want to know is which
> versions of the software I am interested in I can install together. That
> is, after being told the latest releases do not work together. Sounds like
> we need some kind of package management :) In case of the components
> created for Wikidata, we have been supporting Composer for a while now,
> which is a great fit to our needs.
>
> I attempted to install Wikibase the other day and made a fun discovery.
> > Installing it properly requires the following (12) extensions:
> >
>
> That is somewhat inaccurate, and is misleading with regard to Wikibase
> installation. Nevertheless, the concerns you bring up are certainly
> relevant, and currently not really tackled well in the MediaWiki community.
> That is to bad, as it encourages people to inappropriately bundle things
> and throw re usability out of the window (plus causing a long list of other
> problems).
>
> Cheers
>
> --
> Jeroen De Dauw
> http://www.bn2vs.com
> Don't panic. Don't be evil. ~=[,,_,,]:3
> --
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-l [at] lists
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


cananian at wikimedia

Jul 19, 2013, 11:46 AM

Post #6 of 50 (901 views)
Permalink
Re: MediaWiki extensions as core-like libraries: MediaWiki's fun new landmine for admins [In reply to]

On Fri, Jul 19, 2013 at 2:27 PM, Ryan Lane <rlane32 [at] gmail> wrote:

> What if you want to use wikibase 1.23 and MyAbominationExtension 1.5 that
> requires an incompatible version of DataValues and MyAwesomeExtension 1.0
> that requires an incompatible version of ValueView?
>

You file a bug report against MyAbominationExtension and/or
MyAwesomeExtension, telling them they should update their extensions to be
compatible with the latest wikibase. Or wait for wikibase 1.24, with
updated DataValues and ValueViews. Again, the extension authors are
responsible to keep up-to-date.

Or the admin can try to upgrade/downgrade individual components themselves,
just like a Fedora developer is free to do so. But if that breaks you get
to keep both pieces.
--scott
--
(http://cscott.net)
_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


mah at everybody

Jul 19, 2013, 12:16 PM

Post #7 of 50 (905 views)
Permalink
Re: MediaWiki extensions as core-like libraries: MediaWiki's fun new landmine for admins [In reply to]

On 07/19/2013 02:46 PM, Tyler Romeo wrote:
> Man, if only PHP had some sort of dependency management system.

The real question is: why doesn't MediaWiki use Composer?
http://getcomposer.org/download/

We've discussed this here before, though, so it really isn't anything
more than a rhetorical question.

Ryan's post just shows us, once again, why there is a need for it.

--
http://hexmode.com/

Love alone reveals the true shape of the universe.
-- "Everywhere Present", Stephen Freeman

_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


tylerromeo at gmail

Jul 19, 2013, 1:28 PM

Post #8 of 50 (896 views)
Permalink
Re: MediaWiki extensions as core-like libraries: MediaWiki's fun new landmine for admins [In reply to]

On Fri, Jul 19, 2013 at 3:16 PM, Mark A. Hershberger <mah [at] everybody>wrote:

> The real question is: why doesn't MediaWiki use Composer?
> http://getcomposer.org/download/
>
> We've discussed this here before, though, so it really isn't anything
> more than a rhetorical question.
>
> Ryan's post just shows us, once again, why there is a need for it.
>

From what Antoine found out from the composer people, basically what we'll
have to do to make MW use Composer is that we need to separate the entry
points from the backend classes (in other words, put everything except
index.php, api.php, etc. in one repository, and then have the entry points
in another). That way the MW core itself becomes a library, which is how
Symfony does it. Then people just make a project from the entry point
project, and then they can use "composer require" to add more extensions.

*-- *
*Tyler Romeo*
Stevens Institute of Technology, Class of 2016
Major in Computer Science
www.whizkidztech.com | tylerromeo [at] gmail
_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


rlane32 at gmail

Jul 19, 2013, 1:35 PM

Post #9 of 50 (894 views)
Permalink
Re: MediaWiki extensions as core-like libraries: MediaWiki's fun new landmine for admins [In reply to]

On Fri, Jul 19, 2013 at 11:44 AM, Jeroen De Dauw <jeroendedauw [at] gmail>wrote:

> Hey,
>
> What if you want to use wikibase 1.23 and MyAbominationExtension 1.5 that
> > requires an incompatible version of DataValues and MyAwesomeExtension 1.0
> > that requires an incompatible version of ValueView?
> >
>
> If you have releases of certain software that have requirements that cannot
> satisfied together, then you cannot install them together. That is a pretty
> inherent property of incompatible software.
>
>
Yes, but the only requirement right now is "Am I using the correct version
of this extension with this version of MediaWiki?" and you're adding in a
whole new set of incompatibilities. You're really not thinking of this from
the perspective of the person using the software.


> As a user, when I run into such a situation, what I want to know is which
> versions of the software I am interested in I can install together. That
> is, after being told the latest releases do not work together. Sounds like
> we need some kind of package management :) In case of the components
> created for Wikidata, we have been supporting Composer for a while now,
> which is a great fit to our needs.
>
>
"We" in this situation is Wikidata and not the developer community. In
fact, there were a number of threads about composer with no consensus and a
number of objections. So, what you're saying is that the Wikidata team has
made a decision on behalf of the community?


> I attempted to install Wikibase the other day and made a fun discovery.
> > Installing it properly requires the following (12) extensions:
> >
>
> That is somewhat inaccurate, and is misleading with regard to Wikibase
> installation. Nevertheless, the concerns you bring up are certainly
> relevant, and currently not really tackled well in the MediaWiki community.
> That is to bad, as it encourages people to inappropriately bundle things
> and throw re usability out of the window (plus causing a long list of other
> problems).
>
>
The solution to this isn't to sneak a requirement in without consensus....

Also, it's poor architecture to split things into a large number of small
pieces in anticipation of reuse. It's better to split things apart when
there's a need.

Something that's being sidestepped here is that extensions are being used
as a means to avoid getting things reviewed for core. Quite a few of these
extensions should just be core functionality or they shouldn't exist.

- Ryan
_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


hashar+wmf at free

Jul 19, 2013, 2:26 PM

Post #10 of 50 (896 views)
Permalink
Re: MediaWiki extensions as core-like libraries: MediaWiki's fun new landmine for admins [In reply to]

Le 19/07/13 22:28, Tyler Romeo a écrit :
> From what Antoine found out from the composer people, basically what we'll
> have to do to make MW use Composer is that we need to separate the entry
> points from the backend classes (in other words, put everything except
> index.php, api.php, etc. in one repository, and then have the entry points
> in another). That way the MW core itself becomes a library, which is how
> Symfony does it. Then people just make a project from the entry point
> project, and then they can use "composer require" to add more extensions.

Exactly :-]

And installing yoursite would be something like:

mkdir mysite
cd mysite
composer require mediawiki/core
composer require wikibase/wikibase
# which installs data-values/data-values ask/ask as well

Your root directory ends up with a composer.json:

{ "require": {
"mediawiki/core": "*",
"diff/diff": "*",
"wikibase/wikibase": "*"
} }

And a vendor directory containing all dependencies. At the root of your
directory you would write some glue maybe looking like:

<?php
require('vendor/autoload.php');
require('vendor/mediawiki/core/index.php');


(that does not work, but you get the idea).


--
Antoine "hashar" Musso


_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


jeroendedauw at gmail

Jul 20, 2013, 4:34 AM

Post #11 of 50 (894 views)
Permalink
Re: MediaWiki extensions as core-like libraries: MediaWiki's fun new landmine for admins [In reply to]

Hey,

> you're adding in a whole new set of incompatibilities.

How so?

> You're really not thinking of this from the perspective of the person
using the software.

Oh, glad to know you understand what I am thinking, since clearly I do not.

> In case of the components
> > created for Wikidata, we have been supporting Composer for a while now,
> > which is a great fit to our needs.
> >
> >
> "We" in this situation is Wikidata and not the developer community. In
> fact, there were a number of threads about composer with no consensus and a
> number of objections.
>

Given my first sentence, I would think it is indeed abundantly clear this
is about the components created for Wikidata and not the whole community.
Thanks for making it even more obvious.

So, what you're saying is that the Wikidata team has
> made a decision on behalf of the community?
>

What I am saying is that I have played around with something that might be
of use to the community in general. Where did I imply I, or the WD team,
had made a decision for the whole community here? I don't see it, but since
you know better then me what I am thinking, please explain.

Something that's being sidestepped here is that extensions are being used
> as a means to avoid getting things reviewed for core. Quite a few of these
> extensions should just be core functionality or they shouldn't exist.
>

This is preposterous, and I find the accusation you made outrageous. I'd
love to have some constructive discussion here, though this is very
difficult if you come at it from the angle you are.

Cheers

--
Jeroen De Dauw
http://www.bn2vs.com
Don't panic. Don't be evil. ~=[,,_,,]:3
--
_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


rlane32 at gmail

Jul 20, 2013, 2:08 PM

Post #12 of 50 (888 views)
Permalink
Re: MediaWiki extensions as core-like libraries: MediaWiki's fun new landmine for admins [In reply to]

On Sat, Jul 20, 2013 at 4:34 AM, Jeroen De Dauw <jeroendedauw [at] gmail>wrote:

> Hey,
>
> > you're adding in a whole new set of incompatibilities.
>
> How so?
>
>
Extensions that use any of these extension libaries now depend on the
version of MediaWiki and the version of the extension. That's a new set of
dependencies that can be incompatible now.


> > You're really not thinking of this from the perspective of the person
> using the software.
>
> Oh, glad to know you understand what I am thinking, since clearly I do not.
>
>
Maintaining MediaWiki installs is already relatively difficult, due to a
lack of extension management. Many extensions are also poorly maintained,
and just getting an extension that works with your version of MediaWiki
reliably is hard. Upgrades are incredibly hard due to this. Adding in
another set of incompatibilities is going to make this much harder.

This is making something easier for developers at the expense of admins.


> > In case of the components
> > > created for Wikidata, we have been supporting Composer for a while now,
> > > which is a great fit to our needs.
> > >
> > >
> > "We" in this situation is Wikidata and not the developer community. In
> > fact, there were a number of threads about composer with no consensus
> and a
> > number of objections.
> >
>
> Given my first sentence, I would think it is indeed abundantly clear this
> is about the components created for Wikidata and not the whole community.
> Thanks for making it even more obvious.
>
>
Many of these components are generic enough to


> So, what you're saying is that the Wikidata team has
> > made a decision on behalf of the community?
> >
>
> What I am saying is that I have played around with something that might be
> of use to the community in general. Where did I imply I, or the WD team,
> had made a decision for the whole community here? I don't see it, but since
> you know better then me what I am thinking, please explain.
>
>
My concern is adoption of these extension libraries. Other extensions will
use these libraries, which is the point of them... If the only way to
sanely install them is via composer and a decent number of extensions now
relies on them, it's be come a de-facto requirement of MediaWiki, without
consensus.


> Something that's being sidestepped here is that extensions are being used
> > as a means to avoid getting things reviewed for core. Quite a few of
> these
> > extensions should just be core functionality or they shouldn't exist.
> >
>
> This is preposterous, and I find the accusation you made outrageous. I'd
> love to have some constructive discussion here, though this is very
> difficult if you come at it from the angle you are.
>
>
Sorry, I phrased that poorly. Here's my concern: If extension libraries are
generic enough that they could be considered core (Diff is a great
example), then other extensions will likely use it like core functionality.
Wikibase already is. These extensions won't get the same level of review as
they would if they were core functionality.

Is there any compelling reason that they are extensions, rather than being
added to core?

- Ryan
_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


rlane32 at gmail

Jul 20, 2013, 2:14 PM

Post #13 of 50 (886 views)
Permalink
Re: MediaWiki extensions as core-like libraries: MediaWiki's fun new landmine for admins [In reply to]

On Sat, Jul 20, 2013 at 2:08 PM, Ryan Lane <rlane32 [at] gmail> wrote:

> > In case of the components
>
>> > > created for Wikidata, we have been supporting Composer for a while
>> now,
>> > > which is a great fit to our needs.
>> > >
>> > >
>> > "We" in this situation is Wikidata and not the developer community. In
>> > fact, there were a number of threads about composer with no consensus
>> and a
>> > number of objections.
>> >
>>
>> Given my first sentence, I would think it is indeed abundantly clear this
>> is about the components created for Wikidata and not the whole community.
>> Thanks for making it even more obvious.
>>
>>
> Many of these components are generic enough to
>
>

That's what I get for writing that email out of order... I meant to remove
that as I made my point regarding generic components and composer becoming
a de-facto standard elsewhere.

- Ryan
_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


jeroendedauw at gmail

Jul 21, 2013, 3:30 AM

Post #14 of 50 (882 views)
Permalink
Re: MediaWiki extensions as core-like libraries: MediaWiki's fun new landmine for admins [In reply to]

Hey,

> > you're adding in a whole new set of incompatibilities.
> >
> > How so?
> >
> >
> Extensions that use any of these extension libaries now depend on the
> version of MediaWiki and the version of the extension. That's a new set of
> dependencies that can be incompatible now.
>

It adds a component as dependency yes. It however does not add code as
dependency. One of the main reasons for creating well designed components
rather then throwing everything into one bucket is that it minimizes source
code dependencies. If you minimize dependencies (as a good programmer
should do on all levels), you also minimize the things that can cause
compatibility conflicts. It thus certainly benefits the user.

Imagine you have a package P, which contains components A, B and C. Only
package P is versioned and released. That it could be split into 3 packages
is a detail not visible to the end user. A is stand alone, B depends on A
and C depends on A. You want to install extension E0 that depends on P v1.0
or later. In particular, it depends on B, though again, the end user does
not know this. You want to make this install on a wiki where you already
have E1 installed. E1 depends on P v0.5 to v0.8. Internally it only cares
about C. This situation means you can in fact not install E0 and E1 at the
same time. The only reason for this is because to many things are packaged
together. If B and C where in their own packages, there would be no issue.

So one can for instance go from depending on package ABC that contains sub
components A, B and C to depending on A and B. That's two packages instead
of one if you look at it naively, though two components instead of 3 on
closer investigation. It is quite bad to force users to care about C while
there is no reason for doing so. They'll be affected by any issues that
occur in C for no good reason. Every time ABC has a new release because
something in C had to change, users will have to deal with this release,
since to them it is not visible this is pointless.

These are simple examples, with one component that could be split into 3.
The problems for the user get significantly worse if one throws more things
together and as this problem is replicated in the dependency chain.

Maintaining MediaWiki installs is already relatively difficult, due to a
> lack of extension management. Many extensions are also poorly maintained,
> and just getting an extension that works with your version of MediaWiki
> reliably is hard. Upgrades are incredibly hard due to this. Adding in
> another set of incompatibilities is going to make this much harder.
>

Like noted above, it is simply not the case more things become dependent on
each other. Rather the opposite.

Another example. Take the Diff library. It does not depend on anything
(except of course PHP itself). You talk about putting it into core. Imagine
Diff 0.7 with MW 1.23 and Diff 1.0 gets bundled with MW 1.24. Now I have
extension E0 that needs MW 1.24 or later. I also have extension E1 that
needs Diff < 1.0. Again the user runs into problems. In case Diff is merely
bundled with core, the user can resolve this by changing either Diff or MW
version. In case they are solidly glued together, installation is again not
possible for no good reason. Like the one before, this really is a toy
example, that merely illustrates how problems arise. The problems users
encounter in the real world are more significant.

Now you might ask, where are those problems you talk about?

> just getting an extension that works with your version of MediaWiki
reliably is hard.

And finding a set of extensions that work together with that version is
even harder. And you'll have hassle whenever you upgrade. I've seen this
many many times, and have of course encountered it myself more then enough.
After years of listening to users reporting problems and clients running
into this, I have concluded that a lot of this could be avoided if people
did not introduce silly dependencies.

The Diff extension is a good example how how to get it right. Initially it
did not. It dependent on MediaWiki. It was throwing MWExeptions at places,
its tests derives from MediaWikiTestCase and whatnot. This bound it to MW,
putting in a requirement people have to deal with. This dependency has now
been removed, as there really was no reason for it, thus making usage of
Diff easier. Once again, this is the simplest example I could think of.

Sorry, I phrased that poorly. Here's my concern: If extension libraries are
> generic enough that they could be considered core (Diff is a great
> example), then other extensions will likely use it like core functionality.
> Wikibase already is.


I already outlined how this can be problematic, depending on what you mean.
If you are suggesting to merge it into core somehow, and just having one
release for the whole thing, then it will cause hassle. Bundling would be
better. We are already doing this with some extensions. That is something
to look at on a per extension or per library basis though. Clearly we do
not want to bundle every library needed for any extension with core.

In fact, we can't do this if you consider usage of third party libraries.
Imagine you have this app A that uses some common library such as Monolog,
and you have two delivery mechanism plugins for this app A. First you have
B that is an interface to MW, second you have C that is an interface to
some other system. An attempt to "assimilate" A into core is going very
likely to end badly. The MW community is not doing a fantastic job when it
comes to interoperability and sharing of components with the rest of the
PHP world as it is. To me this is a quite unfortunate miss of opportunities.


> These extensions won't get the same level of review as
> they would if they were core functionality.
>

This implies you are thinking of more then just bundling. Or not?

This is again something that will need judgement on case-by-case basis. In
case for Diff, the code quality requirements are significantly higher then
those of core, and these are not going to be lowered.

Is there any compelling reason that they are extensions, rather than being
> added to core?
>

Hell yeah! See above :)

Cheers

--
Jeroen De Dauw
http://www.bn2vs.com
Don't panic. Don't be evil. ~=[,,_,,]:3
--
_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


happy.melon.wiki at gmail

Jul 21, 2013, 8:24 AM

Post #15 of 50 (894 views)
Permalink
Re: MediaWiki extensions as core-like libraries: MediaWiki's fun new landmine for admins [In reply to]

On 21 July 2013 11:30, Jeroen De Dauw <jeroendedauw [at] gmail> wrote:

> Hey,
>
> > > you're adding in a whole new set of incompatibilities.
> > >
> > > How so?
> > >
> > >
> > Extensions that use any of these extension libaries now depend on the
> > version of MediaWiki and the version of the extension. That's a new set
> of
> > dependencies that can be incompatible now.
> >
>
> It adds a component as dependency yes. It however does not add code as
> dependency. One of the main reasons for creating well designed components
> rather then throwing everything into one bucket is that it minimizes source
> code dependencies. If you minimize dependencies (as a good programmer
> should do on all levels), you also minimize the things that can cause
> compatibility conflicts. It thus certainly benefits the user.
>
> Imagine you have a package P, which contains components A, B and C. Only
> package P is versioned and released. That it could be split into 3 packages
> is a detail not visible to the end user. A is stand alone, B depends on A
> and C depends on A. You want to install extension E0 that depends on P v1.0
> or later. In particular, it depends on B, though again, the end user does
> not know this. You want to make this install on a wiki where you already
> have E1 installed. E1 depends on P v0.5 to v0.8. Internally it only cares
> about C. This situation means you can in fact not install E0 and E1 at the
> same time. The only reason for this is because to many things are packaged
> together. If B and C where in their own packages, there would be no issue.
>
> So one can for instance go from depending on package ABC that contains sub
> components A, B and C to depending on A and B. That's two packages instead
> of one if you look at it naively, though two components instead of 3 on
> closer investigation. It is quite bad to force users to care about C while
> there is no reason for doing so. They'll be affected by any issues that
> occur in C for no good reason. Every time ABC has a new release because
> something in C had to change, users will have to deal with this release,
> since to them it is not visible this is pointless.
>
> These are simple examples, with one component that could be split into 3.
> The problems for the user get significantly worse if one throws more things
> together and as this problem is replicated in the dependency chain.
>

The fact that this thought experiment is almost impossible to visualise
without pen and paper is symptomatic of the problem. The end user of
MediaWiki doesn't care about arcane internal dependencies, or the rationale
for them, they want it to Just Work. So either the system has to actually *
be* simple (version X of an extension requires version Y of MediaWiki, so
if I update my MW install I need to also update extensions (or vice
versa)); or it needs to be *properly* abstracted behind a system which
makes it *appear* simple (you run some script and everything magically
works or gives you informative explanations of why it didn't).

The former is obviously preferable in terms of clarity and maintainability;
and that is realised immediately if all essential components are properly
backwards-compatible. There should never *be* a dependency of the form
"package X *no later than* vY". That is a fault of package X, not of the
extension which depends on it. Package X should have been properly managed
so that its public signature does not change incompatibly; and if a change
is necessary, the change is managed over several versions so that consumers
can safely update their use of the new signature and increase their
dependency version to the one which introduces the new behaviour. "I need
all my components to be sufficiently recent that they support all the
features I want" is a methodology that end users can understand (and also
one which means that they should *always* be able to get the feature set
they want). "I need to calculate the intersection of the dependencies of
all my features and resolve them manually or automatically" is not;
especially when there may not even *be* an intersection.

None of that is in any way groundbreaking, it's just basic design
principles which I think we all subscribe to, even if they don't always
materialise. The point is that MediaWiki is *much* more likely to be able
to police and maintain that proper signature management in core code, than
in extensions and obscure miscellaneous modules. It's great that modules
like Diff have good beady eyes on code quality and portability, long may
that continue. That is *definitely* the exception, not the rule, with
extensions. Our core code definitely isn't perfect in terms of signatures,
or even acceptable in many places. But it's getting better because that's
where the most attention is focused.

--HM
_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


jeroendedauw at gmail

Jul 21, 2013, 9:41 AM

Post #16 of 50 (883 views)
Permalink
Re: MediaWiki extensions as core-like libraries: MediaWiki's fun new landmine for admins [In reply to]

Hey,

It's great that modules
> like Diff have good beady eyes on code quality and portability, long may
> that continue. That is *definitely* the exception, not the rule, with
> extensions.
>

The point is that MediaWiki is *much* more likely to be able
> to police and maintain that proper signature management in core code, than
> in extensions and obscure miscellaneous modules.
>

If we are just talking about badly maintained and managed code, then how is
pushing this code into core going to help? You'll still need to do the
effort to maintain it before it improves. For one random unmaintained
extension this might work, but there are hundreds. If you make to many of
them part of core while not increasing maintenance capacity or without
making people care about the code in question, you just end up with the
same situation. Except that now the unmaintained code is causing problems
for everyone, rather then sitting on a side, not being maintained, which
perhaps is because no one cares about the thing any more anyway.

This seems to go into the direction of arguing against having real
extensions. I for one, think it is a bad idea to ditch extensions.

The fact that this thought experiment is almost impossible to visualise
> without pen and paper is symptomatic of the problem.
>

Not really. The problem is there now. That understanding of the system is
required to pinpoint what is causing the problem is why end users are not
flagging these points as being problematic. They just have hassle and know
_something_ is going wrong. Understanding all these internal dependencies
between components is going to be something non-trivial and something users
should not be bothered with in any situation. Lack of understanding is not
a symptom of the problem, this lack, at least on the dev side, is the cause
of it.

So either the system has to actually *
> be* simple (version X of an extension requires version Y of MediaWiki, so
> if I update my MW install I need to also update extensions (or vice
> versa)); or it needs to be *properly* abstracted behind a system which
> makes it *appear* simple (you run some script and everything magically
> works or gives you informative explanations of why it didn't).
>

Fully agree.

I attempted to install Wikibase the other day and made a fun discovery.
> Installing it properly requires the following (12) extensions
>

What started this thread was Ryan having problems with installing Wikibase.
And I can see why this would not be all that smooth. The components you
need is probably not the biggest hassle. After all, you just need to do 4
git clones instead of 3. Then you also need to do a bunch of config, and
figure out if you want to use additional functionality.

For instance, you need Scrubuntu if you want to have lua support on the
client. As a new user of the software, how will you know you need this or
not? Some very good documentation can help, but we don't have this. Putting
the lua stuff into its own extension would make this a lot more clear, and
would not bother users with the requirement of understanding what this is
while they don't even want to use lua. (This model of creating extensions
on top of your main extension is something done by SMW and is working very
well. Users generally do not get confused by it at all.)

Doing 4 git clones and having some basic understanding of the dependencies
is something quite reasonable to expect from developers. As everyone here,
including me, is saying, this is not acceptable for users. The problem in
case of Wikibase is that it is the only process we currently have, as no
one has bothered to do a proper release. You know, one of these things with
a version number, release notes, tarballs, etc. If we want this software to
be usable by non-devs, this needs to happen. And if it does, it is also
going to be useful to devs. So I'm actually quite disappointed we did not
have a release yet, and am regularly shouting to people about it.

So Ryan, I agree that currently this is not in good state. I however
disagree there is this dichotomy between developers and users, where we can
only have it work nicely for one. Attacking the developer process is thus
not a good option, as that will just end up hurting everyone, including the
end users.

Cheers

--
Jeroen De Dauw
http://www.bn2vs.com
Don't panic. Don't be evil. ~=[,,_,,]:3
--
_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


rlane32 at gmail

Jul 21, 2013, 1:58 PM

Post #17 of 50 (884 views)
Permalink
Re: MediaWiki extensions as core-like libraries: MediaWiki's fun new landmine for admins [In reply to]

On Sun, Jul 21, 2013 at 9:41 AM, Jeroen De Dauw <jeroendedauw [at] gmail>wrote:

> What started this thread was Ryan having problems with installing Wikibase.
> And I can see why this would not be all that smooth. The components you
> need is probably not the biggest hassle. After all, you just need to do 4
> git clones instead of 3. Then you also need to do a bunch of config, and
> figure out if you want to use additional functionality.
>
> For instance, you need Scrubuntu if you want to have lua support on the
> client. As a new user of the software, how will you know you need this or
> not? Some very good documentation can help, but we don't have this. Putting
> the lua stuff into its own extension would make this a lot more clear, and
> would not bother users with the requirement of understanding what this is
> while they don't even want to use lua. (This model of creating extensions
> on top of your main extension is something done by SMW and is working very
> well. Users generally do not get confused by it at all.)
>
>
Actually, funny enough, I think SMW is one of the more difficult extension
sets to use as an end-user. It was the first extension where I saw your
Validator code used as a dependency extension. Validator was rejected from
being included in core, but now basically every extension you maintain uses
it. I didn't bother to say anything about this when it was limited to SMW
and your extensions, but now Wikimedia deployed extensions are starting to
use them.


> Doing 4 git clones and having some basic understanding of the dependencies
> is something quite reasonable to expect from developers. As everyone here,
> including me, is saying, this is not acceptable for users. The problem in
> case of Wikibase is that it is the only process we currently have, as no
> one has bothered to do a proper release. You know, one of these things with
> a version number, release notes, tarballs, etc. If we want this software to
> be usable by non-devs, this needs to happen. And if it does, it is also
> going to be useful to devs. So I'm actually quite disappointed we did not
> have a release yet, and am regularly shouting to people about it.
>
>
The actual method of getting the extensions isn't that hard. Even if an
end-user can't do a git clone, they can have git.wm.o give them a tar or
zip. It's specifically the dependencies that are hard. When I attempt to
upgrade MediaWiki I currently have to write down all of the extensions, and
ensure all of them are compatible with MediaWiki. With some subsets, I need
to ensure they are compatible with each other (like SMW, SF, SRF). Now I'm
going to need to do that and track the compatibility between extensions and
dependency extensions. I'm actually going to have to write an upgrade
matrix to upgrade, and that's not OK.


> So Ryan, I agree that currently this is not in good state. I however
> disagree there is this dichotomy between developers and users, where we can
> only have it work nicely for one. Attacking the developer process is thus
> not a good option, as that will just end up hurting everyone, including the
> end users.
>
>
I didn't say there's a dichotomy. I believe that we can have both, but this
approach isn't it.

- Ryan
_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


jeroendedauw at gmail

Jul 22, 2013, 3:03 AM

Post #18 of 50 (876 views)
Permalink
Re: MediaWiki extensions as core-like libraries: MediaWiki's fun new landmine for admins [In reply to]

Hey,

> Validator was rejected from being included in core

I'm happy it did. The code was quite poor at that time (two years back?).
And it still is not a hallmark of great design, though certainly not below
average MW quality at this point. Regardless of that, I now think it is a
bad idea to have it in core and would not petition to have it there in the
future.

> but now basically every extension you maintain uses it.

Can you please look at objective reality first before making claims about
how you would like the world to be so you can bitch about it? Not only are
you overly simplifying things, you get simple facts plain wrong.

Cheers

--
Jeroen De Dauw
http://www.bn2vs.com
Don't panic. Don't be evil. ~=[,,_,,]:3
--
_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


denny.vrandecic at wikimedia

Jul 22, 2013, 4:29 AM

Post #19 of 50 (875 views)
Permalink
Re: MediaWiki extensions as core-like libraries: MediaWiki's fun new landmine for admins [In reply to]

2013/7/19 Antoine Musso <hashar+wmf [at] free>

> And installing yoursite would be something like:
>
> mkdir mysite
> cd mysite
> composer require mediawiki/core
> composer require wikibase/wikibase
> # which installs data-values/data-values ask/ask as well
>
>
Just curious, why is "composer require mediawiki/core" needed?
_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


jeroendedauw at gmail

Jul 22, 2013, 5:11 AM

Post #20 of 50 (875 views)
Permalink
Re: MediaWiki extensions as core-like libraries: MediaWiki's fun new landmine for admins [In reply to]

Hey,

> Just curious, why is "composer require mediawiki/core" needed?


Because Hashar is talking about having a "MediaWiki installation" package,
which effectively turns into "My MediaWiki installation" when you start
using it. This package specifies all the things you want to install (in its
composer.json). This will always include core, and also list the extensions
you want on your particular install.

The reason for having this seperate package is to get around the problem I
described earlier in this thread - if you just have core, and you install
other things into it by modifying the composer.josn of core, you'll end up
with a modified non-optional file in the core repo, which causes hassle
when you want to update.

Of course if we got with such a "MediaWiki installation" package, I'd be
natural to put mediawiki-core already in the composer.json file, so people
would not actually need to run this command.

Cheers

--
Jeroen De Dauw
http://www.bn2vs.com
Don't panic. Don't be evil. ~=[,,_,,]:3
--
_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


hashar+wmf at free

Jul 22, 2013, 5:13 AM

Post #21 of 50 (871 views)
Permalink
Re: MediaWiki extensions as core-like libraries: MediaWiki's fun new landmine for admins [In reply to]

Le 21/07/13 22:58, Ryan Lane a écrit :
> When I attempt to
> upgrade MediaWiki I currently have to write down all of the extensions, and
> ensure all of them are compatible with MediaWiki. With some subsets, I need
> to ensure they are compatible with each other (like SMW, SF, SRF). Now I'm
> going to need to do that and track the compatibility between extensions and
> dependency extensions. I'm actually going to have to write an upgrade
> matrix to upgrade, and that's not OK.

Why would you want to manually keep track of the dependencies when a
tool such as composer can handle it for you?

--
Antoine "hashar" Musso


_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


hashar+wmf at free

Jul 22, 2013, 5:17 AM

Post #22 of 50 (877 views)
Permalink
Re: MediaWiki extensions as core-like libraries: MediaWiki's fun new landmine for admins [In reply to]

Le 22/07/13 13:29, Denny Vrandečić a écrit :
>> > And installing yoursite would be something like:
>> >
>> > mkdir mysite
>> > cd mysite
>> > composer require mediawiki/core
>> > composer require wikibase/wikibase
>> > # which installs data-values/data-values ask/ask as well
>> >
>> >
> Just curious, why is "composer require mediawiki/core" needed?

wikibase/wikibase does not list mediawiki/core as a dependency :)


$ composer show wikibase/wikibase
...
requires:
php >=5.3.0
data-values/data-values *
wikibase/data-model *
diff/diff >=0.6
data-values/data-types *


--
Antoine "hashar" Musso


_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


jeroendedauw at gmail

Jul 22, 2013, 5:31 AM

Post #23 of 50 (877 views)
Permalink
Re: MediaWiki extensions as core-like libraries: MediaWiki's fun new landmine for admins [In reply to]

Hey,

> wikibase/wikibase does not list mediawiki/core as a dependency :)

Indeed. Right now this just allows you to install Wikibase into an existing
MW install. Before we can go all the way, we first need to be able to do a
MediaWiki (+extensions) install, which is something still under discussion.

Cheers

--
Jeroen De Dauw
http://www.bn2vs.com
Don't panic. Don't be evil. ~=[,,_,,]:3
--
_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


denny.vrandecic at wikimedia

Jul 22, 2013, 5:55 AM

Post #24 of 50 (873 views)
Permalink
Re: MediaWiki extensions as core-like libraries: MediaWiki's fun new landmine for admins [In reply to]

Ah, OK, understood. Thanks.


2013/7/22 Jeroen De Dauw <jeroendedauw [at] gmail>

> Hey,
>
> > wikibase/wikibase does not list mediawiki/core as a dependency :)
>
> Indeed. Right now this just allows you to install Wikibase into an existing
> MW install. Before we can go all the way, we first need to be able to do a
> MediaWiki (+extensions) install, which is something still under discussion.
>
> Cheers
>
> --
> Jeroen De Dauw
> http://www.bn2vs.com
> Don't panic. Don't be evil. ~=[,,_,,]:3
> --
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-l [at] lists
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>



--
Project director Wikidata
Wikimedia Deutschland e.V. | Obentrautstr. 72 | 10963 Berlin
Tel. +49-30-219 158 26-0 | http://wikimedia.de

Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e.V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter
der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für
Körperschaften I Berlin, Steuernummer 27/681/51985.
_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


maxsem.wiki at gmail

Jul 22, 2013, 6:18 AM

Post #25 of 50 (872 views)
Permalink
Re: MediaWiki extensions as core-like libraries: MediaWiki's fun new landmine for admins [In reply to]

On 22.07.2013, 16:11 Jeroen wrote:

> Hey,

>> Just curious, why is "composer require mediawiki/core" needed?


> Because Hashar is talking about having a "MediaWiki installation" package,
> which effectively turns into "My MediaWiki installation" when you start
> using it. This package specifies all the things you want to install (in its
> composer.json). This will always include core, and also list the extensions
> you want on your particular install.

> The reason for having this seperate package is to get around the problem I
> described earlier in this thread - if you just have core, and you install
> other things into it by modifying the composer.josn of core, you'll end up
> with a modified non-optional file in the core repo, which causes hassle
> when you want to update.

> Of course if we got with such a "MediaWiki installation" package, I'd be
> natural to put mediawiki-core already in the composer.json file, so people
> would not actually need to run this command.

So it's a Composer's limitation. It woud've been awesome if it was
possible to do something like

if exists "optional.json":
install "optional.json"

:)

--
Best regards,
Max Semenik ([[User:MaxSem]])


_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

First page Previous page 1 2 Next page Last page  View All Wikipedia wikitech RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.