dnessett at yahoo
Jul 9, 2009, 8:39 AM
Post #3 of 5
(496 views)
Permalink
|
|
Re: Defining a configuration for regression testing
[In reply to]
|
|
Hi Gerard,
I am very interested in the tool you mention. Let's keep the discussion on list, since I suspect there are others who might want to set up a regression test environment either now or later.
Can you provide some pointers how to use this tool? Is it described in the standard SVN documentation or is that located somewhere else? What is its name? Would its use allow testing against the most recent version in trunk?
My initial thoughts for such a regression test installation are:
I think some extensions involve database schema changes. My initial idea is to create a new installation, make all of the schema changes necessary for any extensions in the regression test set and then dump the database. This dump could then be used to set up new regression test installations (or reinitialize existing test installations). Perhaps the dump could be placed in Subversion in a "test" area.
Just loading a set of extensions in an installation doesn't really provide much in the way of verifying they work together. The test needs to use them together. One way to do this would be to create a set of pages that concurrently access the extensions when the pages are rendered. Knowing which extensions to use together is a key to this approach. Perhaps there is information in Bugzilla that would help determine that. Also, extension authors might provide some tests that could be incorporated into the test set. If you or others have some ideas on how to test extensions that would be very helpful.
When I worked on Solaris at Sun in the mid-90s, developers were required to regression test their changes before submitting them (through a gatekeeper) for inclusion in the nightly build. Those who failed to do so and broke the build had their hands slapped. Perhaps something similar might be established for the mediawiki development process. Extension authors might be required to: 1) provide some extension tests that could included in the regression test set (if their extensions ever become important enough to do that), and 2) run their extension tests and the standard tests against a standard regression test installation and provide evidence that there are no problems before their extensions are included in the mediawiki extensions matrix.
Dan
--- On Wed, 7/8/09, Gerard Meijssen <gerard.meijssen [at] gmail> wrote:
> From: Gerard Meijssen <gerard.meijssen [at] gmail>
> Subject: Re: [Wikitech-l] Defining a configuration for regression testing
> To: "Wikimedia developers" <wikitech-l [at] lists>
> Cc: "Kim Bruning" <kim [at] bruning>
> Date: Wednesday, July 8, 2009, 10:28 PM
> Hoi.
> In Subversion there is a tool created for the setup of
> environments. What it
> does well is setup an environment with a specific
> configuration. This
> configuration for a specific environment can be found in a
> file. In this way
> it is possible to define a specific revision or tag for
> either MediaWiki
> itself or for an extension. The software is such that you
> can specify
> multiple languages for an environment.. As there are
> important differences
> because of the language, the script you want to be able to
> test for a key
> subset of wikis. Duplication of an initial created wiki
> works well.
>
> When you look at ALL the extensions used whereever on WMF
> projects, you will
> find that they are not an homongous bunch; they are not
> used together. This
> means that you may want to have multiple environments
> configured. At this
> time there is a Wikipeida configuration and a Usability
> Initiative
> configuration. Given that the configuration is in a file,
> there is room to
> indicate a specific revision..
>
> As you can imagine, there are scripts to install particular
> extensions that
> can not be installed in a default way.
>
> When you have an interest, contact me or ask on this list.
> Thanks,
> GerardM
>
> 2009/7/9 dan nessett <dnessett [at] yahoo>
>
> >
> > I am setting up a testing environment for mediawiki
> and the first thing
> > that came to mind is testing new extensions against a
> "regression test
> > configuration". That raises the question of what
> should constitute such a
> > configuration. One issue is which extensions should be
> loaded.
> >
> > There are over 2000 extensions in the mediawiki
> extensions matrix and 512
> > stable extensions. It would be impractical to run a
> configuration with all
> > of either class. So, I asked around and received a
> suggestion that at the
> > very least the extensions on the wikimedia servers
> should be loaded. I went
> > to http://noc.wikimedia.org/conf/ and copied
> CommonSettings.php. From it
> > I extracted 75 extensions that are used on wikimedia's
> servers. I list these
> > below.
> >
> > A question for readers of this list is: should a
> regression test
> > configuration load only these extensions or should it
> load others? Another
> > question is: what other settings should define a
> regression test
> > configuration.
> >
> > Wikimedia installed extensions:
> >
> > Timeline, wikihiero, SiteMatrix, CharInsert,
> CheckUser,
> > SpecialMakesysop, Makebot, ParserFunctions, Cite,
> InputBox,
> > ExpandTemplates, ImageMap, SyntaxHighlight_GeSHi,
> DoubleWiki, Poem,
> > PovWatch, AjaxTest, UnicodeConverter, CategoryTree,
> ProofreadPage, lst,
> > SpamBlacklist, UploadBlacklist, TitleBlacklist, Quiz,
> Gadgets,
> > OggHandler, AssertEdit, FormPreloadPostCache,
> SkinPerPage, Schulenburg,
> > Tomas, ContributionReporting, ContributionTracking,
> ContactPage,
> > ExtensionDistributor, GlobalBlocking, TrustedXFF,
> ContactPage,
> > SecurePoll, OAIRepo, DynamicPageList, Nogomatch,
> > SpecialCrossNamespaceLinks, SpecialRenameuser,
> SpecialNuke, AntiBot,
> > TorBlock, CookieBlock, ScanSet, SpecialCite,
> FixedImage, UserThrottle,
> > ConfirmEdit, FancyCaptcha, HideRevision, AntiSpoof,
> CentralAuth,
> > DismissableSiteNotice, UsernameBlacklist,
> MiniDonation, CentralNotice,
> > TitleKey, WikimediaMessages, SimpleAntiSpam,
> Collection, NewUserMessage,
> > CodeReview, Drafts, Configure, AbuseFilter,
> ClientSide, CommunityVoice,
> > PdfHandler, UsabilityInitiative
> >
> > Regards,
> >
> > Dan Nessett
> >
> >
> >
> >
> >
> > _______________________________________________
> > Wikitech-l mailing list
> > Wikitech-l [at] lists
> > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> >
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-l [at] lists
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
|
