Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Wikipedia: Wikitech

Git migration - updated schedule & workflow descriptions

 

 

Wikipedia wikitech RSS feed   Index | Next | Previous | View Threaded


sumanah at wikimedia

Feb 1, 2012, 2:30 PM

Post #1 of 18 (1073 views)
Permalink
Git migration - updated schedule & workflow descriptions

I talked with Chad today and together we redid:

* https://www.mediawiki.org/wiki/Git
* https://www.mediawiki.org/wiki/Git/Conversion - the schedule for Git
conversion
* https://www.mediawiki.org/wiki/Git/Workflow - describing the workflow
for gerrit, git-review, and any other relevant tools we use
* https://www.mediawiki.org/wiki/Git/Guide - Basic guide to using git.
Trying not to duplicate docs elsewhere on the web -- keep this
appropriate for our users and link generously to outside resources

Hope this helps. You can expect more communication from Chad on this
next week, as well as a blog post.

--
Sumana Harihareswara
Volunteer Development Coordinator
Wikimedia Foundation

_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


arichards at wikimedia

Feb 1, 2012, 5:15 PM

Post #2 of 18 (1060 views)
Permalink
Re: Git migration - updated schedule & workflow descriptions [In reply to]

Thanks for sharing these Sumana. One thing I haven't really seen mentioned
anywhere is what's going to happen with the WIkimedia repository (rather
than the MediaWiki repository) and how that figures into the
roadmap/conversion schedule. Chad or Sumana, can you shed some light on
this?

On Wed, Feb 1, 2012 at 2:30 PM, Sumana Harihareswara
<sumanah [at] wikimedia>wrote:

> I talked with Chad today and together we redid:
>
> * https://www.mediawiki.org/wiki/Git
> * https://www.mediawiki.org/wiki/Git/Conversion - the schedule for Git
> conversion
> * https://www.mediawiki.org/wiki/Git/Workflow - describing the workflow
> for gerrit, git-review, and any other relevant tools we use
> * https://www.mediawiki.org/wiki/Git/Guide - Basic guide to using git.
> Trying not to duplicate docs elsewhere on the web -- keep this
> appropriate for our users and link generously to outside resources
>
> Hope this helps. You can expect more communication from Chad on this
> next week, as well as a blog post.
>
> --
> Sumana Harihareswara
> Volunteer Development Coordinator
> Wikimedia Foundation
>
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-l [at] lists
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>



--
Arthur Richards
Software Engineer, Mobile
[[User:Awjrichards]]
IRC: awjr
+1-415-839-6885 x6687
_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


innocentkiller at gmail

Feb 10, 2012, 5:10 AM

Post #3 of 18 (1026 views)
Permalink
Re: Git migration - updated schedule & workflow descriptions [In reply to]

On Wed, Feb 1, 2012 at 8:15 PM, Arthur Richards <arichards [at] wikimedia> wrote:
> Thanks for sharing these Sumana. One thing I haven't really seen mentioned
> anywhere is what's going to happen with the WIkimedia repository (rather
> than the MediaWiki repository) and how that figures into the
> roadmap/conversion schedule. Chad or Sumana, can you shed some light on
> this?
>

I totally want to apologize for delaying in responding on this. Somehow
it got past my inbox and I missed it until I was looking for another e-mail.

The main point I want to reiterate for everyone is that there is no deadline.
MediaWiki and its extensions are on a pretty firm migration path right now
and we've got dates pencilled in for when we'd like to finish converting.
Converting these parts of the repository are going to be the hardest anyway,
so once we've cleared that hump hopefully it'll be trivial to migrate the
remaining projects.

I know fundraising is on a different schedule than everyone else, so that's
perfectly ok and we can look to a time after the main migrations are done
that will work well for you guys too.

I've mentioned this number in a couple of places, but I think I'll say it here
too so everyone's clear: I'd like to have the svn repository completely
read-only by this time next year (very roughly). If you've got a project that
you're maintaining in svn (other than core or extensions), now's the time to
start thinking about where you want it to eventually end up. If you're
wanting to move to git with the rest of us, the WMF git repo will obviously
welcome any existing svn project. If you're wanting to stick with svn or go
another route--let me know, I'll be happy to work with you to figure
something out.

-Chad

_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


sumanah at wikimedia

Feb 10, 2012, 5:50 AM

Post #4 of 18 (1028 views)
Permalink
Re: Git migration - updated schedule & workflow descriptions [In reply to]

On 02/10/2012 08:10 AM, Chad wrote:
> On Wed, Feb 1, 2012 at 8:15 PM, Arthur Richards <arichards [at] wikimedia> wrote:
>> Thanks for sharing these Sumana. One thing I haven't really seen mentioned
>> anywhere is what's going to happen with the WIkimedia repository (rather
>> than the MediaWiki repository) and how that figures into the
>> roadmap/conversion schedule. Chad or Sumana, can you shed some light on
>> this?
>>
>
> I totally want to apologize for delaying in responding on this. Somehow
> it got past my inbox and I missed it until I was looking for another e-mail.
>
> The main point I want to reiterate for everyone is that there is no deadline.
> MediaWiki and its extensions are on a pretty firm migration path right now
> and we've got dates pencilled in for when we'd like to finish converting.
> Converting these parts of the repository are going to be the hardest anyway,
> so once we've cleared that hump hopefully it'll be trivial to migrate the
> remaining projects.
>
> I know fundraising is on a different schedule than everyone else, so that's
> perfectly ok and we can look to a time after the main migrations are done
> that will work well for you guys too.
>
> I've mentioned this number in a couple of places, but I think I'll say it here
> too so everyone's clear: I'd like to have the svn repository completely
> read-only by this time next year (very roughly). If you've got a project that
> you're maintaining in svn (other than core or extensions), now's the time to
> start thinking about where you want it to eventually end up. If you're
> wanting to move to git with the rest of us, the WMF git repo will obviously
> welcome any existing svn project. If you're wanting to stick with svn or go
> another route--let me know, I'll be happy to work with you to figure
> something out.
>
> -Chad

(CC'ing Merlijn as he's been my liaison to pywikipediabot on this topic.)

Thanks for the timeline on that, Chad. I've added that mid-2013 hope to

https://www.mediawiki.org/wiki/Git/Conversion#Affected_development_projects

I want that to be a list of all the projects that live on
svn.wikimedia.org (fundraising, Wikimedia analytics, mwdumper, etc.) so
we can systematically reach out to them and help them figure out what to do.

I'm also collecting open git migration questions today at

https://www.mediawiki.org/wiki/Talk:Git/Conversion/issues#Topics_for_Chad.27s_email_to_wikitech-l_this_week

so that Chad and I can work on an email and a blog post about 8 hours
from now. Feel free to add your questions. (Example: "what about
gerrit? when will it be not a UX nightmare?")

--
Sumana Harihareswara
Volunteer Development Coordinator
Wikimedia Foundation

_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


gregory.varnum at gmail

Feb 10, 2012, 3:12 PM

Post #5 of 18 (1030 views)
Permalink
Re: Git migration - updated schedule & workflow descriptions [In reply to]

I might be missing something obvious - but when should/can extension developers start working on git vs. svn?

Should we wait for the conversion of all extensions or will the extensions area of the repo be available for use beforehand for any new extensions?

-greg aka varnent

On Feb 10, 2012, at 8:50 AM, Sumana Harihareswara wrote:

> On 02/10/2012 08:10 AM, Chad wrote:
>> On Wed, Feb 1, 2012 at 8:15 PM, Arthur Richards <arichards [at] wikimedia> wrote:
>>> Thanks for sharing these Sumana. One thing I haven't really seen mentioned
>>> anywhere is what's going to happen with the WIkimedia repository (rather
>>> than the MediaWiki repository) and how that figures into the
>>> roadmap/conversion schedule. Chad or Sumana, can you shed some light on
>>> this?
>>>
>>
>> I totally want to apologize for delaying in responding on this. Somehow
>> it got past my inbox and I missed it until I was looking for another e-mail.
>>
>> The main point I want to reiterate for everyone is that there is no deadline.
>> MediaWiki and its extensions are on a pretty firm migration path right now
>> and we've got dates pencilled in for when we'd like to finish converting.
>> Converting these parts of the repository are going to be the hardest anyway,
>> so once we've cleared that hump hopefully it'll be trivial to migrate the
>> remaining projects.
>>
>> I know fundraising is on a different schedule than everyone else, so that's
>> perfectly ok and we can look to a time after the main migrations are done
>> that will work well for you guys too.
>>
>> I've mentioned this number in a couple of places, but I think I'll say it here
>> too so everyone's clear: I'd like to have the svn repository completely
>> read-only by this time next year (very roughly). If you've got a project that
>> you're maintaining in svn (other than core or extensions), now's the time to
>> start thinking about where you want it to eventually end up. If you're
>> wanting to move to git with the rest of us, the WMF git repo will obviously
>> welcome any existing svn project. If you're wanting to stick with svn or go
>> another route--let me know, I'll be happy to work with you to figure
>> something out.
>>
>> -Chad
>
> (CC'ing Merlijn as he's been my liaison to pywikipediabot on this topic.)
>
> Thanks for the timeline on that, Chad. I've added that mid-2013 hope to
>
> https://www.mediawiki.org/wiki/Git/Conversion#Affected_development_projects
>
> I want that to be a list of all the projects that live on
> svn.wikimedia.org (fundraising, Wikimedia analytics, mwdumper, etc.) so
> we can systematically reach out to them and help them figure out what to do.
>
> I'm also collecting open git migration questions today at
>
> https://www.mediawiki.org/wiki/Talk:Git/Conversion/issues#Topics_for_Chad.27s_email_to_wikitech-l_this_week
>
> so that Chad and I can work on an email and a blog post about 8 hours
> from now. Feel free to add your questions. (Example: "what about
> gerrit? when will it be not a UX nightmare?")
>
> --
> Sumana Harihareswara
> Volunteer Development Coordinator
> Wikimedia Foundation
>
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-l [at] lists
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Attachments: signature.asc (0.82 KB)


sumanah at wikimedia

Feb 10, 2012, 5:51 PM

Post #6 of 18 (1032 views)
Permalink
Re: Git migration - updated schedule & workflow descriptions [In reply to]

> On Feb 10, 2012, at 8:50 AM, Sumana Harihareswara wrote:
>
>> On 02/10/2012 08:10 AM, Chad wrote:
>>> On Wed, Feb 1, 2012 at 8:15 PM, Arthur Richards <arichards [at] wikimedia> wrote:
>>>> Thanks for sharing these Sumana. One thing I haven't really seen mentioned
>>>> anywhere is what's going to happen with the WIkimedia repository (rather
>>>> than the MediaWiki repository) and how that figures into the
>>>> roadmap/conversion schedule. Chad or Sumana, can you shed some light on
>>>> this?
>>>>
>>>
>>> I totally want to apologize for delaying in responding on this. Somehow
>>> it got past my inbox and I missed it until I was looking for another e-mail.
>>>
>>> The main point I want to reiterate for everyone is that there is no deadline.
>>> MediaWiki and its extensions are on a pretty firm migration path right now
>>> and we've got dates pencilled in for when we'd like to finish converting.
>>> Converting these parts of the repository are going to be the hardest anyway,
>>> so once we've cleared that hump hopefully it'll be trivial to migrate the
>>> remaining projects.
>>>
>>> I know fundraising is on a different schedule than everyone else, so that's
>>> perfectly ok and we can look to a time after the main migrations are done
>>> that will work well for you guys too.
>>>
>>> I've mentioned this number in a couple of places, but I think I'll say it here
>>> too so everyone's clear: I'd like to have the svn repository completely
>>> read-only by this time next year (very roughly). If you've got a project that
>>> you're maintaining in svn (other than core or extensions), now's the time to
>>> start thinking about where you want it to eventually end up. If you're
>>> wanting to move to git with the rest of us, the WMF git repo will obviously
>>> welcome any existing svn project. If you're wanting to stick with svn or go
>>> another route--let me know, I'll be happy to work with you to figure
>>> something out.
>>>
>>> -Chad
>>
>> (CC'ing Merlijn as he's been my liaison to pywikipediabot on this topic.)
>>
>> Thanks for the timeline on that, Chad. I've added that mid-2013 hope to
>>
>> https://www.mediawiki.org/wiki/Git/Conversion#Affected_development_projects
>>
>> I want that to be a list of all the projects that live on
>> svn.wikimedia.org (fundraising, Wikimedia analytics, mwdumper, etc.) so
>> we can systematically reach out to them and help them figure out what to do.
>>
>> I'm also collecting open git migration questions today at
>>
>> https://www.mediawiki.org/wiki/Talk:Git/Conversion/issues#Topics_for_Chad.27s_email_to_wikitech-l_this_week
>>
>> so that Chad and I can work on an email and a blog post about 8 hours
>> from now. Feel free to add your questions. (Example: "what about
>> gerrit? when will it be not a UX nightmare?")
>>
>> --
>> Sumana Harihareswara
>> Volunteer Development Coordinator
>> Wikimedia Foundation

On 02/10/2012 06:12 PM, Gregory Varnum wrote:
> I might be missing something obvious - but when should/can extension
developers start working on git vs. svn?
>
> Should we wait for the conversion of all extensions or will the
extensions area of the repo be available for use beforehand for any new
extensions?
>
> -greg aka varnent

Good question, Greg!

We had a meeting today to lay out some of the open questions and the
answers. Really raw notes are at

https://www.mediawiki.org/wiki/Talk:Git/Conversion/issues#Topics_for_Chad.27s_email_to_wikitech-l_this_week

It was so long that the email and blog post are going to happen early
next week instead. But here are some key points:

* Please stop creating new extensions in Subversion, right now. It
makes Chad's migration work more error-prone. If you want to create a
new extension in git, ask Chad and he'll help you.

* Extension developers can start learning git now and will have a choice
to move to git starting in March.

* As far as Chad knows, none of the gerrit bugs listed in
https://labsconsole.wikimedia.org/wiki/Gerrit_bugs_that_matter are bad
enough to block the migration. What, *specifically*, do people hate
about gerrit? He needs to know specifics to make decisions that might
delay the migration.

* In Chad's view, we can't use Phabricator as it is, mainly because it
doesn't have LDAP support (and possibly doesn't have per-project
permissions -- this needs investigating), and it is unlikely that our
community can get those back-end features added and tested, to a quality
comparable to gerrit's support for those features, by March. However,
three months after MW core switches to git-and-gerrit, we'll have a
serious tools overview and evaluate gerrit, git-review, gitweb, and all
the other parts of our code management infrastructure, and revisit the
gerrit decision. At that time we might find that Phabricator is a
better fit, and git repos are more portable so switching would be feasible.

* There will be gerrit training documents available before MW core
moves, and some kind of screen-sharing training by or before Monday,
Feb. 27th.

* Chad is trying to get the real git repo up and running ASAP so people
can start doing their real work there. He and RobLa believe we should
encourage people to consider SVN indefinitely slushed. If you are going
to do large things, they prefer you start doing them in git.

Please feel free to reply with questions in this thread, or start a new
topic on that talk page. Thanks.

--
Sumana Harihareswara
Volunteer Development Coordinator
Wikimedia Foundation

_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


arichards at wikimedia

Feb 10, 2012, 8:11 PM

Post #7 of 18 (1031 views)
Permalink
Re: Git migration - updated schedule & workflow descriptions [In reply to]

Thanks for the response, Chad.

I know fundraising is on a different schedule than everyone else, so that's
> perfectly ok and we can look to a time after the main migrations are done
> that will work well for you guys too.
>

The Wikimedia repository is used by others besides just fundraising,
although the fundraising team is probably its heaviest user. I know that
some folks from Wikimedia Sweden are using it, as is Ryan Faulkner for
community analytics, and I believe Nimish has been using at as well.

That said, I imagine timing of a switchover will be most important to
coordinate with the fundraising team. I'm no longer the engineering lead
for fundraising (I've moved to the mobile team), so I suggest coordinating
with Katie Horn.


read-only by this time next year (very roughly). If you've got a project
> that
> you're maintaining in svn (other than core or extensions), now's the time
> to
> start thinking about where you want it to eventually end up. If you're
> wanting to move to git with the rest of us, the WMF git repo will obviously
> welcome any existing svn project. If you're wanting to stick with svn or go
> another route--let me know, I'll be happy to work with you to figure
> something out.
>

IMO, it would be best to have the Wikimedia repo also completely converted
to Git. I imagine it will be a big logistical headache attempting to
maintain both SVN and Git repositories. But thank you for being willing to
be flexible :)

--
Arthur Richards
Software Engineer, Mobile
[[User:Awjrichards]]
IRC: awjr
+1-415-839-6885 x6687
_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


innocentkiller at gmail

Feb 10, 2012, 8:31 PM

Post #8 of 18 (1026 views)
Permalink
Re: Git migration - updated schedule & workflow descriptions [In reply to]

On Fri, Feb 10, 2012 at 8:51 PM, Sumana Harihareswara
<sumanah [at] wikimedia> wrote:
> * Chad is trying to get the real git repo up and running ASAP so people
> can start doing their real work there. He and RobLa believe we should
> encourage people to consider SVN indefinitely slushed.  If you are going
> to do large things, they prefer you start doing them in git.
>

Just a quick note before I head off to bed for the night. I've dropped
the latest conversion of core with all tags into gerrit. You can browse
it via gitweb at[0] and clone via [1] for LDAP users or [2] for anonymous
users.

-Chad

[0] https://gerrit.wikimedia.org/r/gitweb?p=test/mediawiki/core2.git;a=summary
[1] ssh://<user>@gerrit.wikimedia.org/test/mediawiki/core2.git
[2] https://gerrit.wikimedia.org/r/p/test/mediawiki/core2.git

_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


gicodewarrior at gmail

Feb 10, 2012, 10:52 PM

Post #9 of 18 (1026 views)
Permalink
Re: Git migration - updated schedule & workflow descriptions [In reply to]

On Fri, Feb 10, 2012 at 8:31 PM, Chad <innocentkiller [at] gmail> wrote:
> [1] ssh://<user>@gerrit.wikimedia.org/test/mediawiki/core2.git

I think this one is actually on port 29418.

This will work:
ssh://<user>@gerrit.wikimedia.org:29418/test/mediawiki/core2.git

Or, add the following to ~/.ssh/config (the "User" line is only
necessary if different from your local username)
Host gerrit.wikimedia.org
User <user>
Port 29418

and then this will work:
git clone gerrit.wikimedia.org:test/mediawiki/core2.git

~Rusty

_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


niklas.laxstrom at gmail

Feb 11, 2012, 1:20 AM

Post #10 of 18 (1028 views)
Permalink
Re: Git migration - updated schedule & workflow descriptions [In reply to]

On 11 February 2012 03:51, Sumana Harihareswara <sumanah [at] wikimedia> wrote:
> We had a meeting today to lay out some of the open questions and the
> answers.  Really raw notes are at
>
> https://www.mediawiki.org/wiki/Talk:Git/Conversion/issues#Topics_for_Chad.27s_email_to_wikitech-l_this_week
>
> It was so long that the email and blog post are going to happen early
> next week instead.  But here are some key points:
>
> * Please stop creating new extensions in Subversion, right now.  It
> makes Chad's migration work more error-prone.  If you want to create a
> new extension in git, ask Chad and he'll help you.

Since you already are suggesting people to start git repositories, we
should prepare the i18n support right now. Translatewiki.net is not
ready to handle random git repositories all over the world (nor it
necessarily wants to do that). MediaWIki has had very good i18n and
l10n including extensions - I don't want to lose that. So we need to
plan it now, not when we have hundreds of extensions migrated to git
and every one doing the things differently.

If our commit access is restricted (committing updated translations,
but also fixing i18n issues) we cannot do our job properly, which
means that we will start dropping support for stuff that doesn't
handle i18n adequately.

Which reminds me, does LocalisationUpdate support git? Or Extension Distributor?

> * As far as Chad knows, none of the gerrit bugs listed in
> https://labsconsole.wikimedia.org/wiki/Gerrit_bugs_that_matter are bad
> enough to block the migration.  What, *specifically*, do people hate
> about gerrit?  He needs to know specifics to make decisions that might
> delay the migration.

I read through almost all of the commit mails. If I need to spend any
more time on it or do more clicking around my head will probably
explode. This means that I want commit emails with links to review
tool, review tool should load fast, display all diffs inline (not
hidden behind links or anything) and I should be able to filter
commits by path or author. In addition I want to be able to tag
commits (for l10n team to review or keeping list of commits I want to
deploy to the live site). Any failure to meet these requirements is
highly likely to be a blocker for me. Oh and something that signs that
this commit has followups that fixes problems in it.

> * Chad is trying to get the real git repo up and running ASAP so people
> can start doing their real work there. He and RobLa believe we should
> encourage people to consider SVN indefinitely slushed.  If you are going
> to do large things, they prefer you start doing them in git.

We have been in a slush most of this year already and it has affected
the work of our team. We have already started to ignore the slush and
will probably do it again. On the other hand we probably will migrate
our extensions to git among the first ones.

Are extensions used by WMF forced to have the gated trunk model?

How much space do the git repos require? Right now we spend under 500M
to support the i18n of three branches of mediawiki and all extensions.

-Niklas

--
Niklas Laxström

_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


p858snake at gmail

Feb 11, 2012, 1:57 AM

Post #11 of 18 (1025 views)
Permalink
Re: Git migration - updated schedule & workflow descriptions [In reply to]

On Sat, Feb 11, 2012 at 7:20 PM, Niklas Laxström
<niklas.laxstrom [at] gmail> wrote:
> Which reminds me, does LocalisationUpdate support git? Or Extension Distributor?

AFAIK No and No, For the latter I believe Tim (As to my understanding
when I last saw that discussed) is going to set up a RO Git -> SVN
mirror/gateway for that functionality, And since I saw that discussed
there has also been several end users comment in the channel about
wanting a mirror of phase3 as well.


Has the work been done, so the cluster is ready to support the base
being managed in Git compared to SVN?

_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


robla at wikimedia

Feb 11, 2012, 10:40 PM

Post #12 of 18 (1028 views)
Permalink
Re: Git migration - updated schedule & workflow descriptions [In reply to]

Hi Niklas

Comments inline...

On Sat, Feb 11, 2012 at 1:20 AM, Niklas Laxstrm
<niklas.laxstrom [at] gmail> wrote:
> Since you already are suggesting people to start git repositories, we
> should prepare the i18n support right now. Translatewiki.net is not
> ready to handle random git repositories all over the world (nor it
> necessarily wants to do that). MediaWIki has had very good i18n and
> l10n including extensions - I don't want to lose that. So we need to
> plan it now, not when we have hundreds of extensions migrated to git
> and every one doing the things differently.

Fair enough. I think we should handle new extension directories on a
case-by-case basis.

> If our commit access is restricted (committing updated translations,
> but also fixing i18n issues) we cannot do our job properly, which
> means that we will start dropping support for stuff that doesn't
> handle i18n adequately.
>
> Which reminds me, does LocalisationUpdate support git?

Not yet:
https://bugzilla.wikimedia.org/show_bug.cgi?id=34137

> Or Extension Distributor?

Not yet:
https://bugzilla.wikimedia.org/show_bug.cgi?id=27812

>> * As far as Chad knows, none of the gerrit bugs listed in
>> https://labsconsole.wikimedia.org/wiki/Gerrit_bugs_that_matter are bad
>> enough to block the migration. What, *specifically*, do people hate
>> about gerrit? He needs to know specifics to make decisions that might
>> delay the migration.
>
> I read through almost all of the commit mails. If I need to spend any
> more time on it or do more clicking around my head will probably
> explode. This means that I want commit emails with links to review
> tool, review tool should load fast, display all diffs inline (not
> hidden behind links or anything) and I should be able to filter
> commits by path or author. In addition I want to be able to tag
> commits (for l10n team to review or keeping list of commits I want to
> deploy to the live site). Any failure to meet these requirements is
> highly likely to be a blocker for me. Oh and something that signs that
> this commit has followups that fixes problems in it.

I copied the issues you raised to the "Gerrit bugs that matter" page.
I also added a tracking bug to make sure we triage that list:
https://bugzilla.wikimedia.org/show_bug.cgi?id=34349

>> * Chad is trying to get the real git repo up and running ASAP so people
>> can start doing their real work there. He and RobLa believe we should
>> encourage people to consider SVN indefinitely slushed. If you are going
>> to do large things, they prefer you start doing them in git.
>
> We have been in a slush most of this year already and it has affected
> the work of our team. We have already started to ignore the slush and
> will probably do it again.

This comment really bears scrutiny. You realize that Chad isn't going
to be able to fix the Gerrit/Git issues above if we're mired in a sea
of code review, right? Or fixing bugs introduced because some
deprecated feature had to be completely removed *right now*, despite
the fact that it wasn't harming anyone to keep it around? It's not
like your code is actually going to get deployed if you slow down the
process this way.

I think rather than ignore Brion's email here:
http://thread.gmane.org/gmane.science.linguistics.wikipedia.technical/58622/focus=58630

...you should respond to it.

> On the other hand we probably will migrate
> our extensions to git among the first ones.
>
> Are extensions used by WMF forced to have the gated trunk model?

This is a good question. I don't see how we can avoid it. Gerrit
doesn't work with post-commit review, and we make a point of reviewing
the extensions that are going onto the cluster.

> How much space do the git repos require? Right now we spend under 500M
> to support the i18n of three branches of mediawiki and all extensions.

My understanding is that the core repo is about 100M with the full
revision history. I don't know what it looks like with all
extensions, but I have to believe it'll be manageable.

Rob

_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


innocentkiller at gmail

Feb 12, 2012, 5:29 AM

Post #13 of 18 (1025 views)
Permalink
Re: Git migration - updated schedule & workflow descriptions [In reply to]

On Sun, Feb 12, 2012 at 1:40 AM, Rob Lanphier <robla [at] wikimedia> wrote:
>> How much space do the git repos require? Right now we spend under 500M
>> to support the i18n of three branches of mediawiki and all extensions.
>
> My understanding is that the core repo is about 100M with the full
> revision history.  I don't know what it looks like with all
> extensions, but I have to believe it'll be manageable.
>

The reason the core repo is so large is because it's tracking
the history of 19 branches and 161 tags ;-)

-Chad

_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


hashar+wmf at free

Feb 12, 2012, 10:41 AM

Post #14 of 18 (1020 views)
Permalink
Re: Git migration - updated schedule & workflow descriptions [In reply to]

Chad wrote:
> The reason the core repo is so large is because it's tracking
> the history of 19 branches and 161 tags ;-)

And the daily i18n updates? :-) /troll

--
Antoine "hashar" Musso


_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


roan.kattouw at gmail

Feb 12, 2012, 2:12 PM

Post #15 of 18 (1023 views)
Permalink
Re: Git migration - updated schedule & workflow descriptions [In reply to]

On Sat, Feb 11, 2012 at 5:11 AM, Arthur Richards
<arichards [at] wikimedia> wrote:
> The Wikimedia repository is used by others besides just fundraising,
> although the fundraising team is probably its heaviest user. I know that
> some folks from Wikimedia Sweden are using it, as is Ryan Faulkner for
> community analytics, and I believe Nimish has been using at as well.
>
The proper course of action here is probably to split these things up
into multiple repositories. That way each of these parties could move
to git on their own terms.

Roan

_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


roan.kattouw at gmail

Feb 12, 2012, 2:36 PM

Post #16 of 18 (1029 views)
Permalink
Re: Git migration - updated schedule & workflow descriptions [In reply to]

On Sun, Feb 12, 2012 at 7:40 AM, Rob Lanphier <robla [at] wikimedia> wrote:
>> Which reminds me, does LocalisationUpdate support git?
>
> Not yet:
> https://bugzilla.wikimedia.org/show_bug.cgi?id=34137
>
That bug is not about LocalisationUpdate. LU is not a TranslateWiki
tool, it's a WMF-side extension for pulling in up-to-date translations
(which happen to come from TWN, but it doesn't care about that). See
https://www.mediawiki.org/wiki/Extension:LocalisationUpdate for
documentation (details are slightly outdated, but the main points are
accurate).

LocalisationUpdate has SVN support built-in, but WMF stopped using
that a long time ago due to performance issues. Instead, the wrapper
script that we use to run the LU update on the cluster updates a local
checkout of the SVN repository, then passes the path as a command-line
parameter to the LU update script. This wrapper script is managed by
puppet, and is in the puppet git repository [1].

>> I read through almost all of the commit mails. If I need to spend any
>> more time on it or do more clicking around my head will probably
>> explode. This means that I want commit emails with links to review
>> tool,
These exist. Gerrit sends notifications when an event occurs (creation
of a new change, comment/review on an existing change, new version of
an existing change, change merged), and it sends these notifications
to all users that have commented on the change, as well as the author.
I believe that Mark B gets e-mail notifications for all new changes in
the puppet repository (I think he said this once, not sure), so that
would seem to mean that setting up a mailing list that all
notifications go to is possible.

These notification e-mails have links to the change in Gerrit, and
also include a description of who did what (e.g. "Mark Bergsma
submitted this change and it was merged") and, for comment
notifications, the text of the comment.

>> review tool should load fast, display all diffs inline (not
>> hidden behind links or anything)
That's a problem. I also want a full diff (you can't get that with
Gerrit currently, only per-file diffs) that's displayed inline (can't
do this either, diffs are displayed in new tabs). The only workaround
that exists is that clicking the (kind of hidden) "Diff All Unified"
button will load all of the per-file diffs at once, i.e. it will open
a new tab for each file in the diff. I find this sort of bearable, but
it's really not nce.

This is the first thing I tried to fix once I got my hands on a
working dev install of Gerrit, but since I don't have that much Java
experience, I didn't really get anywhere.

>> and I should be able to filter
>> commits by path or author.
Mark does post-commit (after the fact) review of puppet changes
sometimes, and he's sort of indicated that that doesn't work very
nicely; Mark, could you pitch in here and tell us what works for that
and what's lacking?

Per-author filtering of commits is implemented for sure, and you can
also filter for unmerged commits by a given author. There are no path
filtering features that I know of, at least for unmerged commits. I'm
pretty sure Gitweb can do path filtering for commits that have already
been merged into the mainline.

>> In addition I want to be able to tag
>> commits (for l10n team to review or keeping list of commits I want to
>> deploy to the live site).
There is some notion of a "topic", which is a kind of tag that the
committer applies when they submit their commit into Gerrit. The topic
is usually the name of the branch the committer used locally, or a bug
number, and it's used to group related changes. But you can't change
the topic after submission AFAIK, and you can't have multiple topics
for one revision. So it's not real tagging as it exists in CodeReview.

>> Any failure to meet these requirements is
>> highly likely to be a blocker for me. Oh and something that signs that
>> this commit has followups that fixes problems in it.
>
Follow-up marking is also lacking, as far as I know. The way I've been
doing it is to include the change number or Gerrit URL of the
followed-up revision in the commit summary, but that doesn't list the
follow-up in the reverse direction (i.e. if I commit B as a follow-up
to A, B will say it's a follow-up to A, but A will not say it was
followed up on in B). It's true that follow-ups should be less common
because commits can be amended while they're being reviewed (so there
will be no follow-up commits for things that are caught during
review), but of course reviewers aren't perfect, so if something
passes review (and unit tests!) but is still broken in some way, it'll
still need a separate follow-up commit.

> I copied the issues you raised to the "Gerrit bugs that matter" page.
> I also added a tracking bug to make sure we triage that list:
> https://bugzilla.wikimedia.org/show_bug.cgi?id=34349
>
I've edited that page a bit to add the notes I added above.
Specifically, having notification e-mails is not a "Gerrit bug that
matters", it's a feature that has already been implemented. Please
only add things to the bugs list after you've verified they're not
actually already in Gerrit :)

Roan

_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


roan.kattouw at gmail

Feb 12, 2012, 2:41 PM

Post #17 of 18 (1027 views)
Permalink
Re: Git migration - updated schedule & workflow descriptions [In reply to]

On Sun, Feb 12, 2012 at 11:36 PM, Roan Kattouw <roan.kattouw [at] gmail> wrote:
> LocalisationUpdate has SVN support built-in, but WMF stopped using
> that a long time ago due to performance issues. Instead, the wrapper
> script that we use to run the LU update on the cluster updates a local
> checkout of the SVN repository, then passes the path as a command-line
> parameter to the LU update script. This wrapper script is managed by
> puppet, and is in the puppet git repository [1].
>
Now with URL:

[1] https://gerrit.wikimedia.org/r/gitweb?p=operations/puppet.git;a=blob_plain;f=files/misc/l10nupdate/l10nupdate-1;hb=HEAD

P.S.: Gitweb pages load slowly, and for some reason they seem to
disable the bfcache in Firefox. These issues should be looked into as
well.

_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


rlane32 at gmail

Feb 12, 2012, 2:48 PM

Post #18 of 18 (1020 views)
Permalink
Re: Git migration - updated schedule & workflow descriptions [In reply to]

> These exist. Gerrit sends notifications when an event occurs (creation
> of a new change, comment/review on an existing change, new version of
> an existing change, change merged), and it sends these notifications
> to all users that have commented on the change, as well as the author.
> I believe that Mark B gets e-mail notifications for all new changes in
> the puppet repository (I think he said this once, not sure), so that
> would seem to mean that setting up a mailing list that all
> notifications go to is possible.
>

Yep. This is possible. In fact, we do this with our internal
operations list for puppet. It also lets you subscribe to repositories
with a number of options. That said, the emails are both a little
spammy, and a little terse.

> These notification e-mails have links to the change in Gerrit, and
> also include a description of who did what (e.g. "Mark Bergsma
> submitted this change and it was merged") and, for comment
> notifications, the text of the comment.
>

Yeah, this is why it's a little terse. A full inline diff in the email
would rock. I checked, this is a code change :(.

> Mark does post-commit (after the fact) review of puppet changes
> sometimes, and he's sort of indicated that that doesn't work very
> nicely; Mark, could you pitch in here and tell us what works for that
> and what's lacking?
>

There's no tagging, like we have in CodeReview. You can only comment -
can't -1 an already merged change (which makes sense). We really need
to add freeform tagging support.

> Per-author filtering of commits is implemented for sure, and you can
> also filter for unmerged commits by a given author. There are no path
> filtering features that I know of, at least for unmerged commits. I'm
> pretty sure Gitweb can do path filtering for commits that have already
> been merged into the mainline.
>

Here's the full search capabilities. It looks like a lot of what we
need is there, but you need to know the search commands:

http://gerrit.googlecode.com/svn/documentation/2.1.6/user-search.html

- Ryan

_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Wikipedia wikitech RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.