Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Wikipedia: Foundation

Regarding Berkman/Sciences Po study

 

 

First page Previous page 1 2 Next page Last page  View All Wikipedia foundation RSS feed   Index | Next | Previous | View Threaded


dtaraborelli at wikimedia

Dec 9, 2011, 7:51 PM

Post #1 of 31 (1423 views)
Permalink
Regarding Berkman/Sciences Po study

I’d like to give everybody on this list some information on the Berkman/Sciences Po research project that many of you have been discussing here.

On Thursday the Wikimedia Foundation announced the launch of a banner to support a study led by a team at the Berkman Center/Sciences Po and recruiting participants from the English Wikipedia editor community [1]. The banner was taken down within hours of its launch after concerns raised in various community forums (the Admin Noticeboard [2], the Village Pump Tech [3], various IRC channels and mailing lists such as foundation-l [4] and internal-l [5]) that the design was confusing, that it was perceived as a commercial ad and that the community approval process and privacy terms were unclear and hardly visible.

Here’s what happened until the launch, what went wrong after the launch and what we are planning to do next.

==The prequel==
This proposal went through a long review process, involving community forums, the Research Committee and various WMF departments since early 2010.

The Berkman research team first approached WMF to discuss this study in January 2010. They suggested a protocol to recruit English Wikipedia contributors to participate in an early version of this study by March 2010 and posted a proposal to the Administrators’ noticeboard to get community feedback [6]. The community response at that time opposed the proposed recruitment protocol (posting individual invitation messages on user talk pages). It was suggested instead that the recruitment should be handled through a CentralNotice banner to be displayed to registered editors, but concerns were raised on how to minimize the disruption.

To address these concerns, the proposal went through a full review with the Wikimedia Research Committee, that was completed in July 2011. The RCom evaluated the methods, the recruitment strategy, the language used in the survey and approved the proposal pending a final solution for the recruitment taking into account the concerns expressed by the community [7].

Based on suggestions made by community members (e.g. [8]) the research team started to work on a technical solution to selectively display a banner to a subset of registered editors of the English Wikipedia meeting certain eligibility conditions. WMF agreed to invest engineering effort into a system that would allow CentralNotice to serve contents to a specific set of editors – functionality that would benefit future campaigns run by the community, chapters or the Foundation [9] [10].

A new CentralNotice backend was then designed to look up various editor metrics (i.e. number of contributions, account registration date and editor privileges) – all public information available from our database – and to perform a participant eligibility check against these metrics. A banner would then be displayed to eligible participants, posting the above data (user ID + editor metrics) along with a unique token to the server hosting the survey upon clicking. On the landing page of the survey, participants would have the possibility to read the privacy terms of the survey and decide whether to take it or not.

Throughout the review process of this recruitment protocol, the research team received constant feedback from the Foundation’s legal team, the community department, the tech department and the communication team before the campaign went live.

The campaign was announced in the CentralNotice calendar one month before its launch [11] and the launch was with a post on the Foundation’s blog. The banner was enabled on December 8 at 11:00pm UTC. 800+ participants completed the study within a few hours since its launch. The banner was then taken down by a meta-admin a few hours after the launch due to the concerns described above.

So what went wrong?

==A few explanations we owe you==

• Is the Foundation running ads?
No, this banner is a recruitment campaign for a research project that has been thoroughly reviewed by the Research Committee. We have a long tradition of supporting recruitment for research about our communities via various sitenotices. The methodology of this project is sound and the recruitment method less invasive than thousands of individual messages posted on user talk pages. We believe this research will help advance our understanding of the dynamics of participation in our projects. Receiving support by the Research Committee implies that all published output and anonymized data produced by this study will be made available under open licenses. [12] The banner also received full Wikimedia Foundation approval before its launch.

• Is this campaign conflicting with the fundraiser?
No, this banner is running only for a subset of logged-in editors for whom the main fundraiser campaign has already been taken down. We carefully timed this campaign to minimize the impact on the fundraiser and we scheduled it on the CentralNotice calendar with a month notice for this reason.

• Is this campaign running at 100% on the English Wikipedia?
No, the banner has been designed to target a subsample of the English Wikipedia registered editor population. Based on estimates by the research team, the eligibility criteria apply to about 10,000 very active contributors and about 30,000 new editors of the English Wikipedia. The target number of completed responses is 1500.

• Why does the banner include logos of organizations not affiliated with Wikimedia?
The design of the banner was based on the decision to give participants as much information as possible about the research team running the project and to set accurate expectations about the study.


==What we are doing now==

We realize that despite an extensive review, the launch of this project was not fully advertised on community forums. We plan to shortly resume the campaign (for the time needed by the researchers to complete their responses) after a full redesign of the recruitment protocol in order to address the concerns raised by many of you over the last 24 hours. Here’s what we are doing:

• Provide you with better information about the project
We asked the research team to promptly set up a FAQ section on the project page on Meta [13], and to be available to address any concern about the study on the discussion page of this project. The project page on Meta will be linked from the recruitment banner itself.

• Redesign the banner
We understand that the banner design has been interpreted by some as ad-like (even if the goal was to make clear that this study was not being run by WMF, as it implied a redirection to a third party website for performing the experiment). In coordination with the research team, we will come up with a banner design that will be more in line with the concerns expressed by the community (for instance by removing the logos from the banner).

• Make privacy terms as transparent as possible
Upon clicking on the banner, participants accept to share their username, edit count and user privileges with the research team. The previous version didn’t make it explicit and we are working to address this problem. To make the process totally transparent we will make the acceptance of these terms explicit in the banner itself.

Once redirected to the landing page, participants will have to accept the terms of participation in order to enter the study. The project is funded by the European Research Council: the data collected in this study is subject to strict European privacy protocols. The research team will use this data for research purposes only. The research team is not exposed to and does not record participants’ IP addresses.

==How you can help==

We would like to hear from you on the redesign of the banner to make sure it meets the expectations of the community and doesn’t lend itself to any kind of confusion. We will post the new banners to Meta and try to address all pending questions before we resume the campaign.

This is one of the first times we’re supporting a complex, important research initiative like this one, and I apologize for the bumps in the road. We believe that supporting research is part of our mission: it helps advance our understanding of ourselves. So thanks again for all support you can give in making this a success.


Dario Taraborelli
Senior Research Analyst, Wikimedia Foundation

[1] http://blog.wikimedia.org/2011/12/08/experiment-decision-making/
[2] http://en.wikipedia.org/wiki/Wikipedia:Administrators%27_noticeboard/Incidents#Harvard.2FScience_Po_Adverts
[3] http://en.wikipedia.org/wiki/Wikipedia:Village_pump_%28technical%29#Search_banner_Wikipedia_Research_Committee
[4] http://lists.wikimedia.org/pipermail/foundation-l/2011-December/070742.html
[5] https://lists.wikimedia.org/mailman/private/internal-l/2011-December/018842.html
[6] http://en.wikipedia.org/wiki/Wikipedia:Administrators%27_noticeboard/Archive222#Researchers_requesting_administrators.E2.80.99_advices_to_launch_a_study
[7] http://meta.wikimedia.org/wiki/Research_talk:Dynamics_of_Online_Interactions_and_Behavior#RCom_review
[8] http://lists.wikimedia.org/pipermail/foundation-l/2011-May/065580.html
[9] http://lists.wikimedia.org/pipermail/foundation-l/2011-May/065558.html
[10] http://meta.wikimedia.org/wiki/CentralNotice_banner_guidelines
[11] http://meta.wikimedia.org/w/index.php?title=CentralNotice/Calendar&oldid=3056067
[12] http://meta.wikimedia.org/wiki/Research:Subject_recruitment
[13] meta.wikimedia.org/wiki/Research:Dynamics_of_Online_Interactions_and_Behavior

_______________________________________________
foundation-l mailing list
foundation-l [at] lists
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l


risker.wp at gmail

Dec 9, 2011, 9:28 PM

Post #2 of 31 (1399 views)
Permalink
Re: Regarding Berkman/Sciences Po study [In reply to]

On 9 December 2011 22:51, Dario Taraborelli <dtaraborelli [at] wikimedia>wrote:

> I’d like to give everybody on this list some information on the
> Berkman/Sciences Po research project that many of you have been discussing
> here.
>
> On Thursday the Wikimedia Foundation announced the launch of a banner to
> support a study led by a team at the Berkman Center/Sciences Po and
> recruiting participants from the English Wikipedia editor community [1].
> The banner was taken down within hours of its launch after concerns raised
> in various community forums (the Admin Noticeboard [2], the Village Pump
> Tech [3], various IRC channels and mailing lists such as foundation-l [4]
> and internal-l [5]) that the design was confusing, that it was perceived as
> a commercial ad and that the community approval process and privacy terms
> were unclear and hardly visible.
>
> Here’s what happened until the launch, what went wrong after the launch
> and what we are planning to do next.
>
> ==The prequel==
> This proposal went through a long review process, involving community
> forums, the Research Committee and various WMF departments since early 2010.
>
> The Berkman research team first approached WMF to discuss this study in
> January 2010. They suggested a protocol to recruit English Wikipedia
> contributors to participate in an early version of this study by March 2010
> and posted a proposal to the Administrators’ noticeboard to get community
> feedback [6]. The community response at that time opposed the proposed
> recruitment protocol (posting individual invitation messages on user talk
> pages). It was suggested instead that the recruitment should be handled
> through a CentralNotice banner to be displayed to registered editors, but
> concerns were raised on how to minimize the disruption.
>
> To address these concerns, the proposal went through a full review with
> the Wikimedia Research Committee, that was completed in July 2011. The RCom
> evaluated the methods, the recruitment strategy, the language used in the
> survey and approved the proposal pending a final solution for the
> recruitment taking into account the concerns expressed by the community [7].
>
> Based on suggestions made by community members (e.g. [8]) the research
> team started to work on a technical solution to selectively display a
> banner to a subset of registered editors of the English Wikipedia meeting
> certain eligibility conditions. WMF agreed to invest engineering effort
> into a system that would allow CentralNotice to serve contents to a
> specific set of editors – functionality that would benefit future
> campaigns run by the community, chapters or the Foundation [9] [10].
>
> A new CentralNotice backend was then designed to look up various editor
> metrics (i.e. number of contributions, account registration date and editor
> privileges) – all public information available from our database – and to
> perform a participant eligibility check against these metrics. A banner
> would then be displayed to eligible participants, posting the above data
> (user ID + editor metrics) along with a unique token to the server hosting
> the survey upon clicking. On the landing page of the survey, participants
> would have the possibility to read the privacy terms of the survey and
> decide whether to take it or not.
>
> Throughout the review process of this recruitment protocol, the research
> team received constant feedback from the Foundation’s legal team, the
> community department, the tech department and the communication team before
> the campaign went live.
>
> The campaign was announced in the CentralNotice calendar one month before
> its launch [11] and the launch was with a post on the Foundation’s blog.
> The banner was enabled on December 8 at 11:00pm UTC. 800+ participants
> completed the study within a few hours since its launch. The banner was
> then taken down by a meta-admin a few hours after the launch due to the
> concerns described above.
>
> So what went wrong?
>
> ==A few explanations we owe you==
>
> • Is the Foundation running ads?
> No, this banner is a recruitment campaign for a research project that has
> been thoroughly reviewed by the Research Committee. We have a long
> tradition of supporting recruitment for research about our communities via
> various sitenotices. The methodology of this project is sound and the
> recruitment method less invasive than thousands of individual messages
> posted on user talk pages. We believe this research will help advance our
> understanding of the dynamics of participation in our projects. Receiving
> support by the Research Committee implies that all published output and
> anonymized data produced by this study will be made available under open
> licenses. [12] The banner also received full Wikimedia Foundation approval
> before its launch.
>
> • Is this campaign conflicting with the fundraiser?
> No, this banner is running only for a subset of logged-in editors for whom
> the main fundraiser campaign has already been taken down. We carefully
> timed this campaign to minimize the impact on the fundraiser and we
> scheduled it on the CentralNotice calendar with a month notice for this
> reason.
>
> • Is this campaign running at 100% on the English Wikipedia?
> No, the banner has been designed to target a subsample of the English
> Wikipedia registered editor population. Based on estimates by the research
> team, the eligibility criteria apply to about 10,000 very active
> contributors and about 30,000 new editors of the English Wikipedia. The
> target number of completed responses is 1500.
>
> • Why does the banner include logos of organizations not affiliated with
> Wikimedia?
> The design of the banner was based on the decision to give participants as
> much information as possible about the research team running the project
> and to set accurate expectations about the study.
>
>
> ==What we are doing now==
>
> We realize that despite an extensive review, the launch of this project
> was not fully advertised on community forums. We plan to shortly resume the
> campaign (for the time needed by the researchers to complete their
> responses) after a full redesign of the recruitment protocol in order to
> address the concerns raised by many of you over the last 24 hours. Here’s
> what we are doing:
>
> • Provide you with better information about the project
> We asked the research team to promptly set up a FAQ section on the project
> page on Meta [13], and to be available to address any concern about the
> study on the discussion page of this project. The project page on Meta will
> be linked from the recruitment banner itself.
>
> • Redesign the banner
> We understand that the banner design has been interpreted by some as
> ad-like (even if the goal was to make clear that this study was not being
> run by WMF, as it implied a redirection to a third party website for
> performing the experiment). In coordination with the research team, we will
> come up with a banner design that will be more in line with the concerns
> expressed by the community (for instance by removing the logos from the
> banner).
>
> • Make privacy terms as transparent as possible
> Upon clicking on the banner, participants accept to share their username,
> edit count and user privileges with the research team. The previous version
> didn’t make it explicit and we are working to address this problem. To make
> the process totally transparent we will make the acceptance of these terms
> explicit in the banner itself.
>
> Once redirected to the landing page, participants will have to accept the
> terms of participation in order to enter the study. The project is funded
> by the European Research Council: the data collected in this study is
> subject to strict European privacy protocols. The research team will use
> this data for research purposes only. The research team is not exposed to
> and does not record participants’ IP addresses.
>
> ==How you can help==
>
> We would like to hear from you on the redesign of the banner to make sure
> it meets the expectations of the community and doesn’t lend itself to any
> kind of confusion. We will post the new banners to Meta and try to address
> all pending questions before we resume the campaign.
>
> This is one of the first times we’re supporting a complex, important
> research initiative like this one, and I apologize for the bumps in the
> road. We believe that supporting research is part of our mission: it helps
> advance our understanding of ourselves. So thanks again for all support you
> can give in making this a success.
>
>
> Dario Taraborelli
> Senior Research Analyst, Wikimedia Foundation
>
> [1] http://blog.wikimedia.org/2011/12/08/experiment-decision-making/
> [2]
> http://en.wikipedia.org/wiki/Wikipedia:Administrators%27_noticeboard/Incidents#Harvard.2FScience_Po_Adverts
> [3]
> http://en.wikipedia.org/wiki/Wikipedia:Village_pump_%28technical%29#Search_banner_Wikipedia_Research_Committee
> [4]
> http://lists.wikimedia.org/pipermail/foundation-l/2011-December/070742.html
> [5]
> https://lists.wikimedia.org/mailman/private/internal-l/2011-December/018842.html
> [6]
> http://en.wikipedia.org/wiki/Wikipedia:Administrators%27_noticeboard/Archive222#Researchers_requesting_administrators.E2.80.99_advices_to_launch_a_study
> [7]
> http://meta.wikimedia.org/wiki/Research_talk:Dynamics_of_Online_Interactions_and_Behavior#RCom_review
> [8] http://lists.wikimedia.org/pipermail/foundation-l/2011-May/065580.html
> [9] http://lists.wikimedia.org/pipermail/foundation-l/2011-May/065558.html
> [10] http://meta.wikimedia.org/wiki/CentralNotice_banner_guidelines
> [11]
> http://meta.wikimedia.org/w/index.php?title=CentralNotice/Calendar&oldid=3056067
> [12] http://meta.wikimedia.org/wiki/Research:Subject_recruitment
> [13]
> meta.wikimedia.org/wiki/Research:Dynamics_of_Online_Interactions_and_Behavior
>
>

Dario, nobody in any of the discussions on the English Wikipedia (whose
editors are the target of this research project) suggested that a *central
site notice* be used for this or any other research project. The
discussion in April 2011 showed consensus opposition to bot-delivered talk
page notices. One editor involved in the discussion suggested "site
notices" (which I believe were interpreted by the participants to mean a
local site notice) and two others mentioned watchlist notices. The
subsequent discussion about central notices discussed the possibility of
developing a narrowcasting ability for such notices, and discussed
specifically notices directly related to WMF projects or activities. It
did not, in any way, address the concept of using a central notice to
promote a non-WMF activity (such as this research project). Indeed, this is
the first use of a central notice for anything not directly related to an
obviously WMF-related activity.

The ability to narrowcast central notices is a positive advancement;
however, the processes for proposing and determining the appropriateness of
a narrowcast are poorly publicized, and some of them don't appear to have
even existed until after this notice was taken down. There are still no
community-approved guidelines for the use of central notices, although a
draft one is currently up for comment.[1] An RFC initiated in August 2010
with respect to "global banners"/central notices, well in advance of the
development of the narrowcasting ability, strongly supported consensus
approval on Meta for non-fundraising global banners.[2] Now that there is
the ability to target central notices to only one project or community, it
is extremely important that that community be directly notified of such
discussion - a discussion that never took place in any public forum that I
can see in advance of this central notice being activated.

The links above include one to a private mailing list that the majority of
readers of this list have no access to. You may want to consider asking the
persons whose contributions are contained in that particular message to
grant permission for it to be reproduced here so that the rest of us aren't
left in the dark about who said what.

I don't begridge scholars carrying out approved research with Wiki?edians
who volunteer to do so; in fact, I've responded to several requests myself.
I do, however, have concerns about any research that expects to contact
40,000 editors and involve 1500 of them; that is a very significant portion
of our active editorship on the English Wikipedia project. I'm curious to
know if scholars have shown much interest in studying some of the other
projects as much as they've initiated studies on enwp.

Risker/Anne



[1] http://meta.wikimedia.org/wiki/CentralNotice_banner_guidelines
http://meta.wikimedia.org/wiki/Requests_for_comment/Global_banners
_______________________________________________
foundation-l mailing list
foundation-l [at] lists
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l


jerome.hergueux at gmail

Dec 9, 2011, 9:42 PM

Post #3 of 31 (1395 views)
Permalink
Re: Regarding Berkman/Sciences Po study [In reply to]

I do, however, have concerns about any research that expects to contact
40,000 editors and involve 1500 of them; that is a very significant portion
of our active editorship on the English Wikipedia project.

Commenting on this: out of those targeted 40,000 editors, 30,000 or so are
*newly registered users*, so that the sample remains somewhat
representative of the diversity we find on en:wp. The rest of it indeed are
active contributors.

Regards,

Jérôme.


2011/12/10 Risker <risker.wp [at] gmail>

> On 9 December 2011 22:51, Dario Taraborelli <dtaraborelli [at] wikimedia
> >wrote:
>
> > I’d like to give everybody on this list some information on the
> > Berkman/Sciences Po research project that many of you have been
> discussing
> > here.
> >
> > On Thursday the Wikimedia Foundation announced the launch of a banner to
> > support a study led by a team at the Berkman Center/Sciences Po and
> > recruiting participants from the English Wikipedia editor community [1].
> > The banner was taken down within hours of its launch after concerns
> raised
> > in various community forums (the Admin Noticeboard [2], the Village Pump
> > Tech [3], various IRC channels and mailing lists such as foundation-l [4]
> > and internal-l [5]) that the design was confusing, that it was perceived
> as
> > a commercial ad and that the community approval process and privacy terms
> > were unclear and hardly visible.
> >
> > Here’s what happened until the launch, what went wrong after the launch
> > and what we are planning to do next.
> >
> > ==The prequel==
> > This proposal went through a long review process, involving community
> > forums, the Research Committee and various WMF departments since early
> 2010.
> >
> > The Berkman research team first approached WMF to discuss this study in
> > January 2010. They suggested a protocol to recruit English Wikipedia
> > contributors to participate in an early version of this study by March
> 2010
> > and posted a proposal to the Administrators’ noticeboard to get community
> > feedback [6]. The community response at that time opposed the proposed
> > recruitment protocol (posting individual invitation messages on user talk
> > pages). It was suggested instead that the recruitment should be handled
> > through a CentralNotice banner to be displayed to registered editors, but
> > concerns were raised on how to minimize the disruption.
> >
> > To address these concerns, the proposal went through a full review with
> > the Wikimedia Research Committee, that was completed in July 2011. The
> RCom
> > evaluated the methods, the recruitment strategy, the language used in the
> > survey and approved the proposal pending a final solution for the
> > recruitment taking into account the concerns expressed by the community
> [7].
> >
> > Based on suggestions made by community members (e.g. [8]) the research
> > team started to work on a technical solution to selectively display a
> > banner to a subset of registered editors of the English Wikipedia meeting
> > certain eligibility conditions. WMF agreed to invest engineering effort
> > into a system that would allow CentralNotice to serve contents to a
> > specific set of editors – functionality that would benefit future
> > campaigns run by the community, chapters or the Foundation [9] [10].
> >
> > A new CentralNotice backend was then designed to look up various editor
> > metrics (i.e. number of contributions, account registration date and
> editor
> > privileges) – all public information available from our database – and to
> > perform a participant eligibility check against these metrics. A banner
> > would then be displayed to eligible participants, posting the above data
> > (user ID + editor metrics) along with a unique token to the server
> hosting
> > the survey upon clicking. On the landing page of the survey, participants
> > would have the possibility to read the privacy terms of the survey and
> > decide whether to take it or not.
> >
> > Throughout the review process of this recruitment protocol, the research
> > team received constant feedback from the Foundation’s legal team, the
> > community department, the tech department and the communication team
> before
> > the campaign went live.
> >
> > The campaign was announced in the CentralNotice calendar one month before
> > its launch [11] and the launch was with a post on the Foundation’s blog.
> > The banner was enabled on December 8 at 11:00pm UTC. 800+ participants
> > completed the study within a few hours since its launch. The banner was
> > then taken down by a meta-admin a few hours after the launch due to the
> > concerns described above.
> >
> > So what went wrong?
> >
> > ==A few explanations we owe you==
> >
> > • Is the Foundation running ads?
> > No, this banner is a recruitment campaign for a research project that has
> > been thoroughly reviewed by the Research Committee. We have a long
> > tradition of supporting recruitment for research about our communities
> via
> > various sitenotices. The methodology of this project is sound and the
> > recruitment method less invasive than thousands of individual messages
> > posted on user talk pages. We believe this research will help advance our
> > understanding of the dynamics of participation in our projects. Receiving
> > support by the Research Committee implies that all published output and
> > anonymized data produced by this study will be made available under open
> > licenses. [12] The banner also received full Wikimedia Foundation
> approval
> > before its launch.
> >
> > • Is this campaign conflicting with the fundraiser?
> > No, this banner is running only for a subset of logged-in editors for
> whom
> > the main fundraiser campaign has already been taken down. We carefully
> > timed this campaign to minimize the impact on the fundraiser and we
> > scheduled it on the CentralNotice calendar with a month notice for this
> > reason.
> >
> > • Is this campaign running at 100% on the English Wikipedia?
> > No, the banner has been designed to target a subsample of the English
> > Wikipedia registered editor population. Based on estimates by the
> research
> > team, the eligibility criteria apply to about 10,000 very active
> > contributors and about 30,000 new editors of the English Wikipedia. The
> > target number of completed responses is 1500.
> >
> > • Why does the banner include logos of organizations not affiliated with
> > Wikimedia?
> > The design of the banner was based on the decision to give participants
> as
> > much information as possible about the research team running the project
> > and to set accurate expectations about the study.
> >
> >
> > ==What we are doing now==
> >
> > We realize that despite an extensive review, the launch of this project
> > was not fully advertised on community forums. We plan to shortly resume
> the
> > campaign (for the time needed by the researchers to complete their
> > responses) after a full redesign of the recruitment protocol in order to
> > address the concerns raised by many of you over the last 24 hours. Here’s
> > what we are doing:
> >
> > • Provide you with better information about the project
> > We asked the research team to promptly set up a FAQ section on the
> project
> > page on Meta [13], and to be available to address any concern about the
> > study on the discussion page of this project. The project page on Meta
> will
> > be linked from the recruitment banner itself.
> >
> > • Redesign the banner
> > We understand that the banner design has been interpreted by some as
> > ad-like (even if the goal was to make clear that this study was not being
> > run by WMF, as it implied a redirection to a third party website for
> > performing the experiment). In coordination with the research team, we
> will
> > come up with a banner design that will be more in line with the concerns
> > expressed by the community (for instance by removing the logos from the
> > banner).
> >
> > • Make privacy terms as transparent as possible
> > Upon clicking on the banner, participants accept to share their username,
> > edit count and user privileges with the research team. The previous
> version
> > didn’t make it explicit and we are working to address this problem. To
> make
> > the process totally transparent we will make the acceptance of these
> terms
> > explicit in the banner itself.
> >
> > Once redirected to the landing page, participants will have to accept the
> > terms of participation in order to enter the study. The project is funded
> > by the European Research Council: the data collected in this study is
> > subject to strict European privacy protocols. The research team will use
> > this data for research purposes only. The research team is not exposed to
> > and does not record participants’ IP addresses.
> >
> > ==How you can help==
> >
> > We would like to hear from you on the redesign of the banner to make sure
> > it meets the expectations of the community and doesn’t lend itself to any
> > kind of confusion. We will post the new banners to Meta and try to
> address
> > all pending questions before we resume the campaign.
> >
> > This is one of the first times we’re supporting a complex, important
> > research initiative like this one, and I apologize for the bumps in the
> > road. We believe that supporting research is part of our mission: it
> helps
> > advance our understanding of ourselves. So thanks again for all support
> you
> > can give in making this a success.
> >
> >
> > Dario Taraborelli
> > Senior Research Analyst, Wikimedia Foundation
> >
> > [1] http://blog.wikimedia.org/2011/12/08/experiment-decision-making/
> > [2]
> >
> http://en.wikipedia.org/wiki/Wikipedia:Administrators%27_noticeboard/Incidents#Harvard.2FScience_Po_Adverts
> > [3]
> >
> http://en.wikipedia.org/wiki/Wikipedia:Village_pump_%28technical%29#Search_banner_Wikipedia_Research_Committee
> > [4]
> >
> http://lists.wikimedia.org/pipermail/foundation-l/2011-December/070742.html
> > [5]
> >
> https://lists.wikimedia.org/mailman/private/internal-l/2011-December/018842.html
> > [6]
> >
> http://en.wikipedia.org/wiki/Wikipedia:Administrators%27_noticeboard/Archive222#Researchers_requesting_administrators.E2.80.99_advices_to_launch_a_study
> > [7]
> >
> http://meta.wikimedia.org/wiki/Research_talk:Dynamics_of_Online_Interactions_and_Behavior#RCom_review
> > [8]
> http://lists.wikimedia.org/pipermail/foundation-l/2011-May/065580.html
> > [9]
> http://lists.wikimedia.org/pipermail/foundation-l/2011-May/065558.html
> > [10] http://meta.wikimedia.org/wiki/CentralNotice_banner_guidelines
> > [11]
> >
> http://meta.wikimedia.org/w/index.php?title=CentralNotice/Calendar&oldid=3056067
> > [12] http://meta.wikimedia.org/wiki/Research:Subject_recruitment
> > [13]
> >
> meta.wikimedia.org/wiki/Research:Dynamics_of_Online_Interactions_and_Behavior
> >
> >
>
> Dario, nobody in any of the discussions on the English Wikipedia (whose
> editors are the target of this research project) suggested that a *central
> site notice* be used for this or any other research project. The
> discussion in April 2011 showed consensus opposition to bot-delivered talk
> page notices. One editor involved in the discussion suggested "site
> notices" (which I believe were interpreted by the participants to mean a
> local site notice) and two others mentioned watchlist notices. The
> subsequent discussion about central notices discussed the possibility of
> developing a narrowcasting ability for such notices, and discussed
> specifically notices directly related to WMF projects or activities. It
> did not, in any way, address the concept of using a central notice to
> promote a non-WMF activity (such as this research project). Indeed, this is
> the first use of a central notice for anything not directly related to an
> obviously WMF-related activity.
>
> The ability to narrowcast central notices is a positive advancement;
> however, the processes for proposing and determining the appropriateness of
> a narrowcast are poorly publicized, and some of them don't appear to have
> even existed until after this notice was taken down. There are still no
> community-approved guidelines for the use of central notices, although a
> draft one is currently up for comment.[1] An RFC initiated in August 2010
> with respect to "global banners"/central notices, well in advance of the
> development of the narrowcasting ability, strongly supported consensus
> approval on Meta for non-fundraising global banners.[2] Now that there is
> the ability to target central notices to only one project or community, it
> is extremely important that that community be directly notified of such
> discussion - a discussion that never took place in any public forum that I
> can see in advance of this central notice being activated.
>
> The links above include one to a private mailing list that the majority of
> readers of this list have no access to. You may want to consider asking the
> persons whose contributions are contained in that particular message to
> grant permission for it to be reproduced here so that the rest of us aren't
> left in the dark about who said what.
>
> I don't begridge scholars carrying out approved research with Wiki?edians
> who volunteer to do so; in fact, I've responded to several requests myself.
> I do, however, have concerns about any research that expects to contact
> 40,000 editors and involve 1500 of them; that is a very significant portion
> of our active editorship on the English Wikipedia project. I'm curious to
> know if scholars have shown much interest in studying some of the other
> projects as much as they've initiated studies on enwp.
>
> Risker/Anne
>
>
>
> [1] http://meta.wikimedia.org/wiki/CentralNotice_banner_guidelines
> http://meta.wikimedia.org/wiki/Requests_for_comment/Global_banners
> _______________________________________________
> foundation-l mailing list
> foundation-l [at] lists
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
>
_______________________________________________
foundation-l mailing list
foundation-l [at] lists
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l


risker.wp at gmail

Dec 9, 2011, 9:54 PM

Post #4 of 31 (1396 views)
Permalink
Re: Regarding Berkman/Sciences Po study [In reply to]

Hi Jerome - please show me where it says that; I've not been able to verify
that interpretation at all. My understanding is that the 30,000 are users
with fewer than 100 edits per month on average, not that they are new users.

Risker/Anne

2011/12/10 Jérôme Hergueux <jerome.hergueux [at] gmail>

> I do, however, have concerns about any research that expects to contact
> 40,000 editors and involve 1500 of them; that is a very significant portion
> of our active editorship on the English Wikipedia project.
>
> Commenting on this: out of those targeted 40,000 editors, 30,000 or so are
> *newly registered users*, so that the sample remains somewhat
> representative of the diversity we find on en:wp. The rest of it indeed are
> active contributors.
>
> Regards,
>
> Jérôme.
>
>
> 2011/12/10 Risker <risker.wp [at] gmail>
>
> > On 9 December 2011 22:51, Dario Taraborelli <dtaraborelli [at] wikimedia
> > >wrote:
> >
> > > I’d like to give everybody on this list some information on the
> > > Berkman/Sciences Po research project that many of you have been
> > discussing
> > > here.
> > >
> > > On Thursday the Wikimedia Foundation announced the launch of a banner
> to
> > > support a study led by a team at the Berkman Center/Sciences Po and
> > > recruiting participants from the English Wikipedia editor community
> [1].
> > > The banner was taken down within hours of its launch after concerns
> > raised
> > > in various community forums (the Admin Noticeboard [2], the Village
> Pump
> > > Tech [3], various IRC channels and mailing lists such as foundation-l
> [4]
> > > and internal-l [5]) that the design was confusing, that it was
> perceived
> > as
> > > a commercial ad and that the community approval process and privacy
> terms
> > > were unclear and hardly visible.
> > >
> > > Here’s what happened until the launch, what went wrong after the launch
> > > and what we are planning to do next.
> > >
> > > ==The prequel==
> > > This proposal went through a long review process, involving community
> > > forums, the Research Committee and various WMF departments since early
> > 2010.
> > >
> > > The Berkman research team first approached WMF to discuss this study in
> > > January 2010. They suggested a protocol to recruit English Wikipedia
> > > contributors to participate in an early version of this study by March
> > 2010
> > > and posted a proposal to the Administrators’ noticeboard to get
> community
> > > feedback [6]. The community response at that time opposed the proposed
> > > recruitment protocol (posting individual invitation messages on user
> talk
> > > pages). It was suggested instead that the recruitment should be handled
> > > through a CentralNotice banner to be displayed to registered editors,
> but
> > > concerns were raised on how to minimize the disruption.
> > >
> > > To address these concerns, the proposal went through a full review with
> > > the Wikimedia Research Committee, that was completed in July 2011. The
> > RCom
> > > evaluated the methods, the recruitment strategy, the language used in
> the
> > > survey and approved the proposal pending a final solution for the
> > > recruitment taking into account the concerns expressed by the community
> > [7].
> > >
> > > Based on suggestions made by community members (e.g. [8]) the research
> > > team started to work on a technical solution to selectively display a
> > > banner to a subset of registered editors of the English Wikipedia
> meeting
> > > certain eligibility conditions. WMF agreed to invest engineering effort
> > > into a system that would allow CentralNotice to serve contents to a
> > > specific set of editors – functionality that would benefit future
> > > campaigns run by the community, chapters or the Foundation [9] [10].
> > >
> > > A new CentralNotice backend was then designed to look up various editor
> > > metrics (i.e. number of contributions, account registration date and
> > editor
> > > privileges) – all public information available from our database – and
> to
> > > perform a participant eligibility check against these metrics. A banner
> > > would then be displayed to eligible participants, posting the above
> data
> > > (user ID + editor metrics) along with a unique token to the server
> > hosting
> > > the survey upon clicking. On the landing page of the survey,
> participants
> > > would have the possibility to read the privacy terms of the survey and
> > > decide whether to take it or not.
> > >
> > > Throughout the review process of this recruitment protocol, the
> research
> > > team received constant feedback from the Foundation’s legal team, the
> > > community department, the tech department and the communication team
> > before
> > > the campaign went live.
> > >
> > > The campaign was announced in the CentralNotice calendar one month
> before
> > > its launch [11] and the launch was with a post on the Foundation’s
> blog.
> > > The banner was enabled on December 8 at 11:00pm UTC. 800+ participants
> > > completed the study within a few hours since its launch. The banner was
> > > then taken down by a meta-admin a few hours after the launch due to the
> > > concerns described above.
> > >
> > > So what went wrong?
> > >
> > > ==A few explanations we owe you==
> > >
> > > • Is the Foundation running ads?
> > > No, this banner is a recruitment campaign for a research project that
> has
> > > been thoroughly reviewed by the Research Committee. We have a long
> > > tradition of supporting recruitment for research about our communities
> > via
> > > various sitenotices. The methodology of this project is sound and the
> > > recruitment method less invasive than thousands of individual messages
> > > posted on user talk pages. We believe this research will help advance
> our
> > > understanding of the dynamics of participation in our projects.
> Receiving
> > > support by the Research Committee implies that all published output and
> > > anonymized data produced by this study will be made available under
> open
> > > licenses. [12] The banner also received full Wikimedia Foundation
> > approval
> > > before its launch.
> > >
> > > • Is this campaign conflicting with the fundraiser?
> > > No, this banner is running only for a subset of logged-in editors for
> > whom
> > > the main fundraiser campaign has already been taken down. We carefully
> > > timed this campaign to minimize the impact on the fundraiser and we
> > > scheduled it on the CentralNotice calendar with a month notice for this
> > > reason.
> > >
> > > • Is this campaign running at 100% on the English Wikipedia?
> > > No, the banner has been designed to target a subsample of the English
> > > Wikipedia registered editor population. Based on estimates by the
> > research
> > > team, the eligibility criteria apply to about 10,000 very active
> > > contributors and about 30,000 new editors of the English Wikipedia. The
> > > target number of completed responses is 1500.
> > >
> > > • Why does the banner include logos of organizations not affiliated
> with
> > > Wikimedia?
> > > The design of the banner was based on the decision to give participants
> > as
> > > much information as possible about the research team running the
> project
> > > and to set accurate expectations about the study.
> > >
> > >
> > > ==What we are doing now==
> > >
> > > We realize that despite an extensive review, the launch of this project
> > > was not fully advertised on community forums. We plan to shortly resume
> > the
> > > campaign (for the time needed by the researchers to complete their
> > > responses) after a full redesign of the recruitment protocol in order
> to
> > > address the concerns raised by many of you over the last 24 hours.
> Here’s
> > > what we are doing:
> > >
> > > • Provide you with better information about the project
> > > We asked the research team to promptly set up a FAQ section on the
> > project
> > > page on Meta [13], and to be available to address any concern about the
> > > study on the discussion page of this project. The project page on Meta
> > will
> > > be linked from the recruitment banner itself.
> > >
> > > • Redesign the banner
> > > We understand that the banner design has been interpreted by some as
> > > ad-like (even if the goal was to make clear that this study was not
> being
> > > run by WMF, as it implied a redirection to a third party website for
> > > performing the experiment). In coordination with the research team, we
> > will
> > > come up with a banner design that will be more in line with the
> concerns
> > > expressed by the community (for instance by removing the logos from the
> > > banner).
> > >
> > > • Make privacy terms as transparent as possible
> > > Upon clicking on the banner, participants accept to share their
> username,
> > > edit count and user privileges with the research team. The previous
> > version
> > > didn’t make it explicit and we are working to address this problem. To
> > make
> > > the process totally transparent we will make the acceptance of these
> > terms
> > > explicit in the banner itself.
> > >
> > > Once redirected to the landing page, participants will have to accept
> the
> > > terms of participation in order to enter the study. The project is
> funded
> > > by the European Research Council: the data collected in this study is
> > > subject to strict European privacy protocols. The research team will
> use
> > > this data for research purposes only. The research team is not exposed
> to
> > > and does not record participants’ IP addresses.
> > >
> > > ==How you can help==
> > >
> > > We would like to hear from you on the redesign of the banner to make
> sure
> > > it meets the expectations of the community and doesn’t lend itself to
> any
> > > kind of confusion. We will post the new banners to Meta and try to
> > address
> > > all pending questions before we resume the campaign.
> > >
> > > This is one of the first times we’re supporting a complex, important
> > > research initiative like this one, and I apologize for the bumps in the
> > > road. We believe that supporting research is part of our mission: it
> > helps
> > > advance our understanding of ourselves. So thanks again for all support
> > you
> > > can give in making this a success.
> > >
> > >
> > > Dario Taraborelli
> > > Senior Research Analyst, Wikimedia Foundation
> > >
> > > [1] http://blog.wikimedia.org/2011/12/08/experiment-decision-making/
> > > [2]
> > >
> >
> http://en.wikipedia.org/wiki/Wikipedia:Administrators%27_noticeboard/Incidents#Harvard.2FScience_Po_Adverts
> > > [3]
> > >
> >
> http://en.wikipedia.org/wiki/Wikipedia:Village_pump_%28technical%29#Search_banner_Wikipedia_Research_Committee
> > > [4]
> > >
> >
> http://lists.wikimedia.org/pipermail/foundation-l/2011-December/070742.html
> > > [5]
> > >
> >
> https://lists.wikimedia.org/mailman/private/internal-l/2011-December/018842.html
> > > [6]
> > >
> >
> http://en.wikipedia.org/wiki/Wikipedia:Administrators%27_noticeboard/Archive222#Researchers_requesting_administrators.E2.80.99_advices_to_launch_a_study
> > > [7]
> > >
> >
> http://meta.wikimedia.org/wiki/Research_talk:Dynamics_of_Online_Interactions_and_Behavior#RCom_review
> > > [8]
> > http://lists.wikimedia.org/pipermail/foundation-l/2011-May/065580.html
> > > [9]
> > http://lists.wikimedia.org/pipermail/foundation-l/2011-May/065558.html
> > > [10] http://meta.wikimedia.org/wiki/CentralNotice_banner_guidelines
> > > [11]
> > >
> >
> http://meta.wikimedia.org/w/index.php?title=CentralNotice/Calendar&oldid=3056067
> > > [12] http://meta.wikimedia.org/wiki/Research:Subject_recruitment
> > > [13]
> > >
> >
> meta.wikimedia.org/wiki/Research:Dynamics_of_Online_Interactions_and_Behavior
> > >
> > >
> >
> > Dario, nobody in any of the discussions on the English Wikipedia (whose
> > editors are the target of this research project) suggested that a
> *central
> > site notice* be used for this or any other research project. The
> > discussion in April 2011 showed consensus opposition to bot-delivered
> talk
> > page notices. One editor involved in the discussion suggested "site
> > notices" (which I believe were interpreted by the participants to mean a
> > local site notice) and two others mentioned watchlist notices. The
> > subsequent discussion about central notices discussed the possibility of
> > developing a narrowcasting ability for such notices, and discussed
> > specifically notices directly related to WMF projects or activities. It
> > did not, in any way, address the concept of using a central notice to
> > promote a non-WMF activity (such as this research project). Indeed, this
> is
> > the first use of a central notice for anything not directly related to an
> > obviously WMF-related activity.
> >
> > The ability to narrowcast central notices is a positive advancement;
> > however, the processes for proposing and determining the appropriateness
> of
> > a narrowcast are poorly publicized, and some of them don't appear to have
> > even existed until after this notice was taken down. There are still no
> > community-approved guidelines for the use of central notices, although a
> > draft one is currently up for comment.[1] An RFC initiated in August 2010
> > with respect to "global banners"/central notices, well in advance of the
> > development of the narrowcasting ability, strongly supported consensus
> > approval on Meta for non-fundraising global banners.[2] Now that there is
> > the ability to target central notices to only one project or community,
> it
> > is extremely important that that community be directly notified of such
> > discussion - a discussion that never took place in any public forum that
> I
> > can see in advance of this central notice being activated.
> >
> > The links above include one to a private mailing list that the majority
> of
> > readers of this list have no access to. You may want to consider asking
> the
> > persons whose contributions are contained in that particular message to
> > grant permission for it to be reproduced here so that the rest of us
> aren't
> > left in the dark about who said what.
> >
> > I don't begridge scholars carrying out approved research with Wiki?edians
> > who volunteer to do so; in fact, I've responded to several requests
> myself.
> > I do, however, have concerns about any research that expects to contact
> > 40,000 editors and involve 1500 of them; that is a very significant
> portion
> > of our active editorship on the English Wikipedia project. I'm curious
> to
> > know if scholars have shown much interest in studying some of the other
> > projects as much as they've initiated studies on enwp.
> >
> > Risker/Anne
> >
> >
> >
> > [1] http://meta.wikimedia.org/wiki/CentralNotice_banner_guidelines
> > http://meta.wikimedia.org/wiki/Requests_for_comment/Global_banners
> > _______________________________________________
> > foundation-l mailing list
> > foundation-l [at] lists
> > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
> >
> _______________________________________________
> foundation-l mailing list
> foundation-l [at] lists
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
>
_______________________________________________
foundation-l mailing list
foundation-l [at] lists
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l


jerome.hergueux at gmail

Dec 9, 2011, 10:10 PM

Post #5 of 31 (1393 views)
Permalink
Re: Regarding Berkman/Sciences Po study [In reply to]

This is actually not the case. Those 30,000 users or so are users who
registered their Wikipedia account 30 days prior to the launch of the
study. There are no other requirements for those users to be eligible to
participate. This is in line with Dario's previous message:

the banner has been designed to target a subsample of the English Wikipedia
registered editor population. Based on estimates by the research team, the
eligibility criteria apply to about 10,000 very active contributors and
about 30,000 new editors of the English Wikipedia.

Regards,

Jérôme.

2011/12/10 Risker <risker.wp [at] gmail>

> Hi Jerome - please show me where it says that; I've not been able to verify
> that interpretation at all. My understanding is that the 30,000 are users
> with fewer than 100 edits per month on average, not that they are new
> users.
>
> Risker/Anne
>
> 2011/12/10 Jérôme Hergueux <jerome.hergueux [at] gmail>
>
> > I do, however, have concerns about any research that expects to contact
> > 40,000 editors and involve 1500 of them; that is a very significant
> portion
> > of our active editorship on the English Wikipedia project.
> >
> > Commenting on this: out of those targeted 40,000 editors, 30,000 or so
> are
> > *newly registered users*, so that the sample remains somewhat
> > representative of the diversity we find on en:wp. The rest of it indeed
> are
> > active contributors.
> >
> > Regards,
> >
> > Jérôme.
> >
> >
> > 2011/12/10 Risker <risker.wp [at] gmail>
> >
> > > On 9 December 2011 22:51, Dario Taraborelli <
> dtaraborelli [at] wikimedia
> > > >wrote:
> > >
> > > > I’d like to give everybody on this list some information on the
> > > > Berkman/Sciences Po research project that many of you have been
> > > discussing
> > > > here.
> > > >
> > > > On Thursday the Wikimedia Foundation announced the launch of a banner
> > to
> > > > support a study led by a team at the Berkman Center/Sciences Po and
> > > > recruiting participants from the English Wikipedia editor community
> > [1].
> > > > The banner was taken down within hours of its launch after concerns
> > > raised
> > > > in various community forums (the Admin Noticeboard [2], the Village
> > Pump
> > > > Tech [3], various IRC channels and mailing lists such as foundation-l
> > [4]
> > > > and internal-l [5]) that the design was confusing, that it was
> > perceived
> > > as
> > > > a commercial ad and that the community approval process and privacy
> > terms
> > > > were unclear and hardly visible.
> > > >
> > > > Here’s what happened until the launch, what went wrong after the
> launch
> > > > and what we are planning to do next.
> > > >
> > > > ==The prequel==
> > > > This proposal went through a long review process, involving community
> > > > forums, the Research Committee and various WMF departments since
> early
> > > 2010.
> > > >
> > > > The Berkman research team first approached WMF to discuss this study
> in
> > > > January 2010. They suggested a protocol to recruit English Wikipedia
> > > > contributors to participate in an early version of this study by
> March
> > > 2010
> > > > and posted a proposal to the Administrators’ noticeboard to get
> > community
> > > > feedback [6]. The community response at that time opposed the
> proposed
> > > > recruitment protocol (posting individual invitation messages on user
> > talk
> > > > pages). It was suggested instead that the recruitment should be
> handled
> > > > through a CentralNotice banner to be displayed to registered editors,
> > but
> > > > concerns were raised on how to minimize the disruption.
> > > >
> > > > To address these concerns, the proposal went through a full review
> with
> > > > the Wikimedia Research Committee, that was completed in July 2011.
> The
> > > RCom
> > > > evaluated the methods, the recruitment strategy, the language used in
> > the
> > > > survey and approved the proposal pending a final solution for the
> > > > recruitment taking into account the concerns expressed by the
> community
> > > [7].
> > > >
> > > > Based on suggestions made by community members (e.g. [8]) the
> research
> > > > team started to work on a technical solution to selectively display a
> > > > banner to a subset of registered editors of the English Wikipedia
> > meeting
> > > > certain eligibility conditions. WMF agreed to invest engineering
> effort
> > > > into a system that would allow CentralNotice to serve contents to a
> > > > specific set of editors – functionality that would benefit future
> > > > campaigns run by the community, chapters or the Foundation [9] [10].
> > > >
> > > > A new CentralNotice backend was then designed to look up various
> editor
> > > > metrics (i.e. number of contributions, account registration date and
> > > editor
> > > > privileges) – all public information available from our database –
> and
> > to
> > > > perform a participant eligibility check against these metrics. A
> banner
> > > > would then be displayed to eligible participants, posting the above
> > data
> > > > (user ID + editor metrics) along with a unique token to the server
> > > hosting
> > > > the survey upon clicking. On the landing page of the survey,
> > participants
> > > > would have the possibility to read the privacy terms of the survey
> and
> > > > decide whether to take it or not.
> > > >
> > > > Throughout the review process of this recruitment protocol, the
> > research
> > > > team received constant feedback from the Foundation’s legal team, the
> > > > community department, the tech department and the communication team
> > > before
> > > > the campaign went live.
> > > >
> > > > The campaign was announced in the CentralNotice calendar one month
> > before
> > > > its launch [11] and the launch was with a post on the Foundation’s
> > blog.
> > > > The banner was enabled on December 8 at 11:00pm UTC. 800+
> participants
> > > > completed the study within a few hours since its launch. The banner
> was
> > > > then taken down by a meta-admin a few hours after the launch due to
> the
> > > > concerns described above.
> > > >
> > > > So what went wrong?
> > > >
> > > > ==A few explanations we owe you==
> > > >
> > > > • Is the Foundation running ads?
> > > > No, this banner is a recruitment campaign for a research project that
> > has
> > > > been thoroughly reviewed by the Research Committee. We have a long
> > > > tradition of supporting recruitment for research about our
> communities
> > > via
> > > > various sitenotices. The methodology of this project is sound and the
> > > > recruitment method less invasive than thousands of individual
> messages
> > > > posted on user talk pages. We believe this research will help advance
> > our
> > > > understanding of the dynamics of participation in our projects.
> > Receiving
> > > > support by the Research Committee implies that all published output
> and
> > > > anonymized data produced by this study will be made available under
> > open
> > > > licenses. [12] The banner also received full Wikimedia Foundation
> > > approval
> > > > before its launch.
> > > >
> > > > • Is this campaign conflicting with the fundraiser?
> > > > No, this banner is running only for a subset of logged-in editors for
> > > whom
> > > > the main fundraiser campaign has already been taken down. We
> carefully
> > > > timed this campaign to minimize the impact on the fundraiser and we
> > > > scheduled it on the CentralNotice calendar with a month notice for
> this
> > > > reason.
> > > >
> > > > • Is this campaign running at 100% on the English Wikipedia?
> > > > No, the banner has been designed to target a subsample of the English
> > > > Wikipedia registered editor population. Based on estimates by the
> > > research
> > > > team, the eligibility criteria apply to about 10,000 very active
> > > > contributors and about 30,000 new editors of the English Wikipedia.
> The
> > > > target number of completed responses is 1500.
> > > >
> > > > • Why does the banner include logos of organizations not affiliated
> > with
> > > > Wikimedia?
> > > > The design of the banner was based on the decision to give
> participants
> > > as
> > > > much information as possible about the research team running the
> > project
> > > > and to set accurate expectations about the study.
> > > >
> > > >
> > > > ==What we are doing now==
> > > >
> > > > We realize that despite an extensive review, the launch of this
> project
> > > > was not fully advertised on community forums. We plan to shortly
> resume
> > > the
> > > > campaign (for the time needed by the researchers to complete their
> > > > responses) after a full redesign of the recruitment protocol in order
> > to
> > > > address the concerns raised by many of you over the last 24 hours.
> > Here’s
> > > > what we are doing:
> > > >
> > > > • Provide you with better information about the project
> > > > We asked the research team to promptly set up a FAQ section on the
> > > project
> > > > page on Meta [13], and to be available to address any concern about
> the
> > > > study on the discussion page of this project. The project page on
> Meta
> > > will
> > > > be linked from the recruitment banner itself.
> > > >
> > > > • Redesign the banner
> > > > We understand that the banner design has been interpreted by some as
> > > > ad-like (even if the goal was to make clear that this study was not
> > being
> > > > run by WMF, as it implied a redirection to a third party website for
> > > > performing the experiment). In coordination with the research team,
> we
> > > will
> > > > come up with a banner design that will be more in line with the
> > concerns
> > > > expressed by the community (for instance by removing the logos from
> the
> > > > banner).
> > > >
> > > > • Make privacy terms as transparent as possible
> > > > Upon clicking on the banner, participants accept to share their
> > username,
> > > > edit count and user privileges with the research team. The previous
> > > version
> > > > didn’t make it explicit and we are working to address this problem.
> To
> > > make
> > > > the process totally transparent we will make the acceptance of these
> > > terms
> > > > explicit in the banner itself.
> > > >
> > > > Once redirected to the landing page, participants will have to accept
> > the
> > > > terms of participation in order to enter the study. The project is
> > funded
> > > > by the European Research Council: the data collected in this study is
> > > > subject to strict European privacy protocols. The research team will
> > use
> > > > this data for research purposes only. The research team is not
> exposed
> > to
> > > > and does not record participants’ IP addresses.
> > > >
> > > > ==How you can help==
> > > >
> > > > We would like to hear from you on the redesign of the banner to make
> > sure
> > > > it meets the expectations of the community and doesn’t lend itself to
> > any
> > > > kind of confusion. We will post the new banners to Meta and try to
> > > address
> > > > all pending questions before we resume the campaign.
> > > >
> > > > This is one of the first times we’re supporting a complex, important
> > > > research initiative like this one, and I apologize for the bumps in
> the
> > > > road. We believe that supporting research is part of our mission: it
> > > helps
> > > > advance our understanding of ourselves. So thanks again for all
> support
> > > you
> > > > can give in making this a success.
> > > >
> > > >
> > > > Dario Taraborelli
> > > > Senior Research Analyst, Wikimedia Foundation
> > > >
> > > > [1] http://blog.wikimedia.org/2011/12/08/experiment-decision-making/
> > > > [2]
> > > >
> > >
> >
> http://en.wikipedia.org/wiki/Wikipedia:Administrators%27_noticeboard/Incidents#Harvard.2FScience_Po_Adverts
> > > > [3]
> > > >
> > >
> >
> http://en.wikipedia.org/wiki/Wikipedia:Village_pump_%28technical%29#Search_banner_Wikipedia_Research_Committee
> > > > [4]
> > > >
> > >
> >
> http://lists.wikimedia.org/pipermail/foundation-l/2011-December/070742.html
> > > > [5]
> > > >
> > >
> >
> https://lists.wikimedia.org/mailman/private/internal-l/2011-December/018842.html
> > > > [6]
> > > >
> > >
> >
> http://en.wikipedia.org/wiki/Wikipedia:Administrators%27_noticeboard/Archive222#Researchers_requesting_administrators.E2.80.99_advices_to_launch_a_study
> > > > [7]
> > > >
> > >
> >
> http://meta.wikimedia.org/wiki/Research_talk:Dynamics_of_Online_Interactions_and_Behavior#RCom_review
> > > > [8]
> > > http://lists.wikimedia.org/pipermail/foundation-l/2011-May/065580.html
> > > > [9]
> > > http://lists.wikimedia.org/pipermail/foundation-l/2011-May/065558.html
> > > > [10] http://meta.wikimedia.org/wiki/CentralNotice_banner_guidelines
> > > > [11]
> > > >
> > >
> >
> http://meta.wikimedia.org/w/index.php?title=CentralNotice/Calendar&oldid=3056067
> > > > [12] http://meta.wikimedia.org/wiki/Research:Subject_recruitment
> > > > [13]
> > > >
> > >
> >
> meta.wikimedia.org/wiki/Research:Dynamics_of_Online_Interactions_and_Behavior
> > > >
> > > >
> > >
> > > Dario, nobody in any of the discussions on the English Wikipedia (whose
> > > editors are the target of this research project) suggested that a
> > *central
> > > site notice* be used for this or any other research project. The
> > > discussion in April 2011 showed consensus opposition to bot-delivered
> > talk
> > > page notices. One editor involved in the discussion suggested "site
> > > notices" (which I believe were interpreted by the participants to mean
> a
> > > local site notice) and two others mentioned watchlist notices. The
> > > subsequent discussion about central notices discussed the possibility
> of
> > > developing a narrowcasting ability for such notices, and discussed
> > > specifically notices directly related to WMF projects or activities.
> It
> > > did not, in any way, address the concept of using a central notice to
> > > promote a non-WMF activity (such as this research project). Indeed,
> this
> > is
> > > the first use of a central notice for anything not directly related to
> an
> > > obviously WMF-related activity.
> > >
> > > The ability to narrowcast central notices is a positive advancement;
> > > however, the processes for proposing and determining the
> appropriateness
> > of
> > > a narrowcast are poorly publicized, and some of them don't appear to
> have
> > > even existed until after this notice was taken down. There are still no
> > > community-approved guidelines for the use of central notices, although
> a
> > > draft one is currently up for comment.[1] An RFC initiated in August
> 2010
> > > with respect to "global banners"/central notices, well in advance of
> the
> > > development of the narrowcasting ability, strongly supported consensus
> > > approval on Meta for non-fundraising global banners.[2] Now that there
> is
> > > the ability to target central notices to only one project or community,
> > it
> > > is extremely important that that community be directly notified of such
> > > discussion - a discussion that never took place in any public forum
> that
> > I
> > > can see in advance of this central notice being activated.
> > >
> > > The links above include one to a private mailing list that the majority
> > of
> > > readers of this list have no access to. You may want to consider asking
> > the
> > > persons whose contributions are contained in that particular message to
> > > grant permission for it to be reproduced here so that the rest of us
> > aren't
> > > left in the dark about who said what.
> > >
> > > I don't begridge scholars carrying out approved research with
> Wiki?edians
> > > who volunteer to do so; in fact, I've responded to several requests
> > myself.
> > > I do, however, have concerns about any research that expects to contact
> > > 40,000 editors and involve 1500 of them; that is a very significant
> > portion
> > > of our active editorship on the English Wikipedia project. I'm curious
> > to
> > > know if scholars have shown much interest in studying some of the other
> > > projects as much as they've initiated studies on enwp.
> > >
> > > Risker/Anne
> > >
> > >
> > >
> > > [1] http://meta.wikimedia.org/wiki/CentralNotice_banner_guidelines
> > > http://meta.wikimedia.org/wiki/Requests_for_comment/Global_banners
> > > _______________________________________________
> > > foundation-l mailing list
> > > foundation-l [at] lists
> > > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
> > >
> > _______________________________________________
> > foundation-l mailing list
> > foundation-l [at] lists
> > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
> >
> _______________________________________________
> foundation-l mailing list
> foundation-l [at] lists
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
>
_______________________________________________
foundation-l mailing list
foundation-l [at] lists
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l


risker.wp at gmail

Dec 9, 2011, 10:27 PM

Post #6 of 31 (1410 views)
Permalink
Re: Regarding Berkman/Sciences Po study [In reply to]

Hi Jerome -

The only documentation from the research team that I have seen so far with
respect to the target participation is in the initial proposal on enwp back
in 2010, when it was proposed to leave 40,000 talk page messages; there was
no indication that 30,000 of them would be newly registered users at that
time. Not to criticize the genuine attempt at information sharing on
Dario's part - it is much appreciated - but there is so much change in what
was put forward from what we had initially been approached about that it's
preferable to hear it from the researcher's mouth, and to have it well
documented.

Something that has never been clear is the reason that English Wikipedia
editors were identified as the preferred target; there does not appear to
be anything in this study that is particularly oriented toward Wikipedia
activity.

Risker/Anne

2011/12/10 Jérôme Hergueux <jerome.hergueux [at] gmail>

> This is actually not the case. Those 30,000 users or so are users who
> registered their Wikipedia account 30 days prior to the launch of the
> study. There are no other requirements for those users to be eligible to
> participate. This is in line with Dario's previous message:
>
> the banner has been designed to target a subsample of the English Wikipedia
> registered editor population. Based on estimates by the research team, the
> eligibility criteria apply to about 10,000 very active contributors and
> about 30,000 new editors of the English Wikipedia.
>
> Regards,
>
> Jérôme.
>
> 2011/12/10 Risker <risker.wp [at] gmail>
>
> > Hi Jerome - please show me where it says that; I've not been able to
> verify
> > that interpretation at all. My understanding is that the 30,000 are
> users
> > with fewer than 100 edits per month on average, not that they are new
> > users.
> >
> > Risker/Anne
> >
> > 2011/12/10 Jérôme Hergueux <jerome.hergueux [at] gmail>
> >
> > > I do, however, have concerns about any research that expects to contact
> > > 40,000 editors and involve 1500 of them; that is a very significant
> > portion
> > > of our active editorship on the English Wikipedia project.
> > >
> > > Commenting on this: out of those targeted 40,000 editors, 30,000 or so
> > are
> > > *newly registered users*, so that the sample remains somewhat
> > > representative of the diversity we find on en:wp. The rest of it indeed
> > are
> > > active contributors.
> > >
> > > Regards,
> > >
> > > Jérôme.
> > >
> > >
> > > 2011/12/10 Risker <risker.wp [at] gmail>
> > >
> > > > On 9 December 2011 22:51, Dario Taraborelli <
> > dtaraborelli [at] wikimedia
> > > > >wrote:
> > > >
> > > > > I’d like to give everybody on this list some information on the
> > > > > Berkman/Sciences Po research project that many of you have been
> > > > discussing
> > > > > here.
> > > > >
> > > > > On Thursday the Wikimedia Foundation announced the launch of a
> banner
> > > to
> > > > > support a study led by a team at the Berkman Center/Sciences Po and
> > > > > recruiting participants from the English Wikipedia editor community
> > > [1].
> > > > > The banner was taken down within hours of its launch after concerns
> > > > raised
> > > > > in various community forums (the Admin Noticeboard [2], the Village
> > > Pump
> > > > > Tech [3], various IRC channels and mailing lists such as
> foundation-l
> > > [4]
> > > > > and internal-l [5]) that the design was confusing, that it was
> > > perceived
> > > > as
> > > > > a commercial ad and that the community approval process and privacy
> > > terms
> > > > > were unclear and hardly visible.
> > > > >
> > > > > Here’s what happened until the launch, what went wrong after the
> > launch
> > > > > and what we are planning to do next.
> > > > >
> > > > > ==The prequel==
> > > > > This proposal went through a long review process, involving
> community
> > > > > forums, the Research Committee and various WMF departments since
> > early
> > > > 2010.
> > > > >
> > > > > The Berkman research team first approached WMF to discuss this
> study
> > in
> > > > > January 2010. They suggested a protocol to recruit English
> Wikipedia
> > > > > contributors to participate in an early version of this study by
> > March
> > > > 2010
> > > > > and posted a proposal to the Administrators’ noticeboard to get
> > > community
> > > > > feedback [6]. The community response at that time opposed the
> > proposed
> > > > > recruitment protocol (posting individual invitation messages on
> user
> > > talk
> > > > > pages). It was suggested instead that the recruitment should be
> > handled
> > > > > through a CentralNotice banner to be displayed to registered
> editors,
> > > but
> > > > > concerns were raised on how to minimize the disruption.
> > > > >
> > > > > To address these concerns, the proposal went through a full review
> > with
> > > > > the Wikimedia Research Committee, that was completed in July 2011.
> > The
> > > > RCom
> > > > > evaluated the methods, the recruitment strategy, the language used
> in
> > > the
> > > > > survey and approved the proposal pending a final solution for the
> > > > > recruitment taking into account the concerns expressed by the
> > community
> > > > [7].
> > > > >
> > > > > Based on suggestions made by community members (e.g. [8]) the
> > research
> > > > > team started to work on a technical solution to selectively
> display a
> > > > > banner to a subset of registered editors of the English Wikipedia
> > > meeting
> > > > > certain eligibility conditions. WMF agreed to invest engineering
> > effort
> > > > > into a system that would allow CentralNotice to serve contents to a
> > > > > specific set of editors – functionality that would benefit future
> > > > > campaigns run by the community, chapters or the Foundation [9]
> [10].
> > > > >
> > > > > A new CentralNotice backend was then designed to look up various
> > editor
> > > > > metrics (i.e. number of contributions, account registration date
> and
> > > > editor
> > > > > privileges) – all public information available from our database –
> > and
> > > to
> > > > > perform a participant eligibility check against these metrics. A
> > banner
> > > > > would then be displayed to eligible participants, posting the above
> > > data
> > > > > (user ID + editor metrics) along with a unique token to the server
> > > > hosting
> > > > > the survey upon clicking. On the landing page of the survey,
> > > participants
> > > > > would have the possibility to read the privacy terms of the survey
> > and
> > > > > decide whether to take it or not.
> > > > >
> > > > > Throughout the review process of this recruitment protocol, the
> > > research
> > > > > team received constant feedback from the Foundation’s legal team,
> the
> > > > > community department, the tech department and the communication
> team
> > > > before
> > > > > the campaign went live.
> > > > >
> > > > > The campaign was announced in the CentralNotice calendar one month
> > > before
> > > > > its launch [11] and the launch was with a post on the Foundation’s
> > > blog.
> > > > > The banner was enabled on December 8 at 11:00pm UTC. 800+
> > participants
> > > > > completed the study within a few hours since its launch. The banner
> > was
> > > > > then taken down by a meta-admin a few hours after the launch due to
> > the
> > > > > concerns described above.
> > > > >
> > > > > So what went wrong?
> > > > >
> > > > > ==A few explanations we owe you==
> > > > >
> > > > > • Is the Foundation running ads?
> > > > > No, this banner is a recruitment campaign for a research project
> that
> > > has
> > > > > been thoroughly reviewed by the Research Committee. We have a long
> > > > > tradition of supporting recruitment for research about our
> > communities
> > > > via
> > > > > various sitenotices. The methodology of this project is sound and
> the
> > > > > recruitment method less invasive than thousands of individual
> > messages
> > > > > posted on user talk pages. We believe this research will help
> advance
> > > our
> > > > > understanding of the dynamics of participation in our projects.
> > > Receiving
> > > > > support by the Research Committee implies that all published output
> > and
> > > > > anonymized data produced by this study will be made available under
> > > open
> > > > > licenses. [12] The banner also received full Wikimedia Foundation
> > > > approval
> > > > > before its launch.
> > > > >
> > > > > • Is this campaign conflicting with the fundraiser?
> > > > > No, this banner is running only for a subset of logged-in editors
> for
> > > > whom
> > > > > the main fundraiser campaign has already been taken down. We
> > carefully
> > > > > timed this campaign to minimize the impact on the fundraiser and we
> > > > > scheduled it on the CentralNotice calendar with a month notice for
> > this
> > > > > reason.
> > > > >
> > > > > • Is this campaign running at 100% on the English Wikipedia?
> > > > > No, the banner has been designed to target a subsample of the
> English
> > > > > Wikipedia registered editor population. Based on estimates by the
> > > > research
> > > > > team, the eligibility criteria apply to about 10,000 very active
> > > > > contributors and about 30,000 new editors of the English Wikipedia.
> > The
> > > > > target number of completed responses is 1500.
> > > > >
> > > > > • Why does the banner include logos of organizations not affiliated
> > > with
> > > > > Wikimedia?
> > > > > The design of the banner was based on the decision to give
> > participants
> > > > as
> > > > > much information as possible about the research team running the
> > > project
> > > > > and to set accurate expectations about the study.
> > > > >
> > > > >
> > > > > ==What we are doing now==
> > > > >
> > > > > We realize that despite an extensive review, the launch of this
> > project
> > > > > was not fully advertised on community forums. We plan to shortly
> > resume
> > > > the
> > > > > campaign (for the time needed by the researchers to complete their
> > > > > responses) after a full redesign of the recruitment protocol in
> order
> > > to
> > > > > address the concerns raised by many of you over the last 24 hours.
> > > Here’s
> > > > > what we are doing:
> > > > >
> > > > > • Provide you with better information about the project
> > > > > We asked the research team to promptly set up a FAQ section on the
> > > > project
> > > > > page on Meta [13], and to be available to address any concern about
> > the
> > > > > study on the discussion page of this project. The project page on
> > Meta
> > > > will
> > > > > be linked from the recruitment banner itself.
> > > > >
> > > > > • Redesign the banner
> > > > > We understand that the banner design has been interpreted by some
> as
> > > > > ad-like (even if the goal was to make clear that this study was not
> > > being
> > > > > run by WMF, as it implied a redirection to a third party website
> for
> > > > > performing the experiment). In coordination with the research team,
> > we
> > > > will
> > > > > come up with a banner design that will be more in line with the
> > > concerns
> > > > > expressed by the community (for instance by removing the logos from
> > the
> > > > > banner).
> > > > >
> > > > > • Make privacy terms as transparent as possible
> > > > > Upon clicking on the banner, participants accept to share their
> > > username,
> > > > > edit count and user privileges with the research team. The previous
> > > > version
> > > > > didn’t make it explicit and we are working to address this problem.
> > To
> > > > make
> > > > > the process totally transparent we will make the acceptance of
> these
> > > > terms
> > > > > explicit in the banner itself.
> > > > >
> > > > > Once redirected to the landing page, participants will have to
> accept
> > > the
> > > > > terms of participation in order to enter the study. The project is
> > > funded
> > > > > by the European Research Council: the data collected in this study
> is
> > > > > subject to strict European privacy protocols. The research team
> will
> > > use
> > > > > this data for research purposes only. The research team is not
> > exposed
> > > to
> > > > > and does not record participants’ IP addresses.
> > > > >
> > > > > ==How you can help==
> > > > >
> > > > > We would like to hear from you on the redesign of the banner to
> make
> > > sure
> > > > > it meets the expectations of the community and doesn’t lend itself
> to
> > > any
> > > > > kind of confusion. We will post the new banners to Meta and try to
> > > > address
> > > > > all pending questions before we resume the campaign.
> > > > >
> > > > > This is one of the first times we’re supporting a complex,
> important
> > > > > research initiative like this one, and I apologize for the bumps in
> > the
> > > > > road. We believe that supporting research is part of our mission:
> it
> > > > helps
> > > > > advance our understanding of ourselves. So thanks again for all
> > support
> > > > you
> > > > > can give in making this a success.
> > > > >
> > > > >
> > > > > Dario Taraborelli
> > > > > Senior Research Analyst, Wikimedia Foundation
> > > > >
> > > > > [1]
> http://blog.wikimedia.org/2011/12/08/experiment-decision-making/
> > > > > [2]
> > > > >
> > > >
> > >
> >
> http://en.wikipedia.org/wiki/Wikipedia:Administrators%27_noticeboard/Incidents#Harvard.2FScience_Po_Adverts
> > > > > [3]
> > > > >
> > > >
> > >
> >
> http://en.wikipedia.org/wiki/Wikipedia:Village_pump_%28technical%29#Search_banner_Wikipedia_Research_Committee
> > > > > [4]
> > > > >
> > > >
> > >
> >
> http://lists.wikimedia.org/pipermail/foundation-l/2011-December/070742.html
> > > > > [5]
> > > > >
> > > >
> > >
> >
> https://lists.wikimedia.org/mailman/private/internal-l/2011-December/018842.html
> > > > > [6]
> > > > >
> > > >
> > >
> >
> http://en.wikipedia.org/wiki/Wikipedia:Administrators%27_noticeboard/Archive222#Researchers_requesting_administrators.E2.80.99_advices_to_launch_a_study
> > > > > [7]
> > > > >
> > > >
> > >
> >
> http://meta.wikimedia.org/wiki/Research_talk:Dynamics_of_Online_Interactions_and_Behavior#RCom_review
> > > > > [8]
> > > >
> http://lists.wikimedia.org/pipermail/foundation-l/2011-May/065580.html
> > > > > [9]
> > > >
> http://lists.wikimedia.org/pipermail/foundation-l/2011-May/065558.html
> > > > > [10]
> http://meta.wikimedia.org/wiki/CentralNotice_banner_guidelines
> > > > > [11]
> > > > >
> > > >
> > >
> >
> http://meta.wikimedia.org/w/index.php?title=CentralNotice/Calendar&oldid=3056067
> > > > > [12] http://meta.wikimedia.org/wiki/Research:Subject_recruitment
> > > > > [13]
> > > > >
> > > >
> > >
> >
> meta.wikimedia.org/wiki/Research:Dynamics_of_Online_Interactions_and_Behavior
> > > > >
> > > > >
> > > >
> > > > Dario, nobody in any of the discussions on the English Wikipedia
> (whose
> > > > editors are the target of this research project) suggested that a
> > > *central
> > > > site notice* be used for this or any other research project. The
> > > > discussion in April 2011 showed consensus opposition to bot-delivered
> > > talk
> > > > page notices. One editor involved in the discussion suggested "site
> > > > notices" (which I believe were interpreted by the participants to
> mean
> > a
> > > > local site notice) and two others mentioned watchlist notices. The
> > > > subsequent discussion about central notices discussed the possibility
> > of
> > > > developing a narrowcasting ability for such notices, and discussed
> > > > specifically notices directly related to WMF projects or activities.
> > It
> > > > did not, in any way, address the concept of using a central notice to
> > > > promote a non-WMF activity (such as this research project). Indeed,
> > this
> > > is
> > > > the first use of a central notice for anything not directly related
> to
> > an
> > > > obviously WMF-related activity.
> > > >
> > > > The ability to narrowcast central notices is a positive advancement;
> > > > however, the processes for proposing and determining the
> > appropriateness
> > > of
> > > > a narrowcast are poorly publicized, and some of them don't appear to
> > have
> > > > even existed until after this notice was taken down. There are still
> no
> > > > community-approved guidelines for the use of central notices,
> although
> > a
> > > > draft one is currently up for comment.[1] An RFC initiated in August
> > 2010
> > > > with respect to "global banners"/central notices, well in advance of
> > the
> > > > development of the narrowcasting ability, strongly supported
> consensus
> > > > approval on Meta for non-fundraising global banners.[2] Now that
> there
> > is
> > > > the ability to target central notices to only one project or
> community,
> > > it
> > > > is extremely important that that community be directly notified of
> such
> > > > discussion - a discussion that never took place in any public forum
> > that
> > > I
> > > > can see in advance of this central notice being activated.
> > > >
> > > > The links above include one to a private mailing list that the
> majority
> > > of
> > > > readers of this list have no access to. You may want to consider
> asking
> > > the
> > > > persons whose contributions are contained in that particular message
> to
> > > > grant permission for it to be reproduced here so that the rest of us
> > > aren't
> > > > left in the dark about who said what.
> > > >
> > > > I don't begridge scholars carrying out approved research with
> > Wiki?edians
> > > > who volunteer to do so; in fact, I've responded to several requests
> > > myself.
> > > > I do, however, have concerns about any research that expects to
> contact
> > > > 40,000 editors and involve 1500 of them; that is a very significant
> > > portion
> > > > of our active editorship on the English Wikipedia project. I'm
> curious
> > > to
> > > > know if scholars have shown much interest in studying some of the
> other
> > > > projects as much as they've initiated studies on enwp.
> > > >
> > > > Risker/Anne
> > > >
> > > >
> > > >
> > > > [1] http://meta.wikimedia.org/wiki/CentralNotice_banner_guidelines
> > > > http://meta.wikimedia.org/wiki/Requests_for_comment/Global_banners
> > > > _______________________________________________
> > > > foundation-l mailing list
> > > > foundation-l [at] lists
> > > > Unsubscribe:
> https://lists.wikimedia.org/mailman/listinfo/foundation-l
> > > >
> > > _______________________________________________
> > > foundation-l mailing list
> > > foundation-l [at] lists
> > > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
> > >
> > _______________________________________________
> > foundation-l mailing list
> > foundation-l [at] lists
> > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
> >
> _______________________________________________
> foundation-l mailing list
> foundation-l [at] lists
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
>
_______________________________________________
foundation-l mailing list
foundation-l [at] lists
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l


admin at wikiqueer

Dec 9, 2011, 10:45 PM

Post #7 of 31 (1394 views)
Permalink
Re: Regarding Berkman/Sciences Po study [In reply to]

Having taken the survey - I honestly don't see what all the fuss is about. Sure the banner could have been designed better - but this seems like a disproportionate reaction to that minor mistake in the grand scheme of a one year approval process that obviously was transparent. They're US based organizations, so why are people surprised they'd target en.WP? It's the largest WMF project, so logical to focus on it over say en.WT. Focusing on editors or people with accounts makes a lot of sense. I'm not really sure why people seem to be all caught up in the numbers or target audience. I've worked on research that had much higher target numbers than this. This is one of the more bizarre (although not the most bizarre) reactions I've seen.

I have no stake in the matter nor was I involved in its approval process, but as I was taking it I was thinking of many ways WMF and other nonprofits could utilize the data. Understanding people's altruistic behavior is vital to volunteer recruitment and fundraising efforts. I was impressed with how well it was put together, the explanations were especially well done. I'm not surprised people on enWP objected. You could suggest we had a search box or print feature and I'm confident at least a small vocal group would express displeasure without realizing they're features already present. I'm not saying they should be dismissed outright, but feel they should be taken in better context.

Nothing anyone has said convinces me this is worth such in-depth and cyclical debate. Seems to have gone off the rail a bit... Thank you to the folks involved for providing responses and background information. Already gone beyond what I would have personally expected.

Just my two cents - you may now return to finding problems with it.

-greg aka varnent


-------
Gregory Varnum
Lead, Aequalitas Project
Founding Principal, VarnEnt
@GregVarnum
fb.com/GregVarnum

On Dec 10, 2011, at 1:27 AM, Risker wrote:

> Hi Jerome -
>
> The only documentation from the research team that I have seen so far with
> respect to the target participation is in the initial proposal on enwp back
> in 2010, when it was proposed to leave 40,000 talk page messages; there was
> no indication that 30,000 of them would be newly registered users at that
> time. Not to criticize the genuine attempt at information sharing on
> Dario's part - it is much appreciated - but there is so much change in what
> was put forward from what we had initially been approached about that it's
> preferable to hear it from the researcher's mouth, and to have it well
> documented.
>
> Something that has never been clear is the reason that English Wikipedia
> editors were identified as the preferred target; there does not appear to
> be anything in this study that is particularly oriented toward Wikipedia
> activity.
>
> Risker/Anne
>
> 2011/12/10 Jérôme Hergueux <jerome.hergueux [at] gmail>
>
>> This is actually not the case. Those 30,000 users or so are users who
>> registered their Wikipedia account 30 days prior to the launch of the
>> study. There are no other requirements for those users to be eligible to
>> participate. This is in line with Dario's previous message:
>>
>> the banner has been designed to target a subsample of the English Wikipedia
>> registered editor population. Based on estimates by the research team, the
>> eligibility criteria apply to about 10,000 very active contributors and
>> about 30,000 new editors of the English Wikipedia.
>>
>> Regards,
>>
>> Jérôme.
>>
>> 2011/12/10 Risker <risker.wp [at] gmail>
>>
>>> Hi Jerome - please show me where it says that; I've not been able to
>> verify
>>> that interpretation at all. My understanding is that the 30,000 are
>> users
>>> with fewer than 100 edits per month on average, not that they are new
>>> users.
>>>
>>> Risker/Anne
>>>
>>> 2011/12/10 Jérôme Hergueux <jerome.hergueux [at] gmail>
>>>
>>>> I do, however, have concerns about any research that expects to contact
>>>> 40,000 editors and involve 1500 of them; that is a very significant
>>> portion
>>>> of our active editorship on the English Wikipedia project.
>>>>
>>>> Commenting on this: out of those targeted 40,000 editors, 30,000 or so
>>> are
>>>> *newly registered users*, so that the sample remains somewhat
>>>> representative of the diversity we find on en:wp. The rest of it indeed
>>> are
>>>> active contributors.
>>>>
>>>> Regards,
>>>>
>>>> Jérôme.
>>>>
>>>>
>>>> 2011/12/10 Risker <risker.wp [at] gmail>
>>>>
>>>>> On 9 December 2011 22:51, Dario Taraborelli <
>>> dtaraborelli [at] wikimedia
>>>>>> wrote:
>>>>>
>>>>>> I’d like to give everybody on this list some information on the
>>>>>> Berkman/Sciences Po research project that many of you have been
>>>>> discussing
>>>>>> here.
>>>>>>
>>>>>> On Thursday the Wikimedia Foundation announced the launch of a
>> banner
>>>> to
>>>>>> support a study led by a team at the Berkman Center/Sciences Po and
>>>>>> recruiting participants from the English Wikipedia editor community
>>>> [1].
>>>>>> The banner was taken down within hours of its launch after concerns
>>>>> raised
>>>>>> in various community forums (the Admin Noticeboard [2], the Village
>>>> Pump
>>>>>> Tech [3], various IRC channels and mailing lists such as
>> foundation-l
>>>> [4]
>>>>>> and internal-l [5]) that the design was confusing, that it was
>>>> perceived
>>>>> as
>>>>>> a commercial ad and that the community approval process and privacy
>>>> terms
>>>>>> were unclear and hardly visible.
>>>>>>
>>>>>> Here’s what happened until the launch, what went wrong after the
>>> launch
>>>>>> and what we are planning to do next.
>>>>>>
>>>>>> ==The prequel==
>>>>>> This proposal went through a long review process, involving
>> community
>>>>>> forums, the Research Committee and various WMF departments since
>>> early
>>>>> 2010.
>>>>>>
>>>>>> The Berkman research team first approached WMF to discuss this
>> study
>>> in
>>>>>> January 2010. They suggested a protocol to recruit English
>> Wikipedia
>>>>>> contributors to participate in an early version of this study by
>>> March
>>>>> 2010
>>>>>> and posted a proposal to the Administrators’ noticeboard to get
>>>> community
>>>>>> feedback [6]. The community response at that time opposed the
>>> proposed
>>>>>> recruitment protocol (posting individual invitation messages on
>> user
>>>> talk
>>>>>> pages). It was suggested instead that the recruitment should be
>>> handled
>>>>>> through a CentralNotice banner to be displayed to registered
>> editors,
>>>> but
>>>>>> concerns were raised on how to minimize the disruption.
>>>>>>
>>>>>> To address these concerns, the proposal went through a full review
>>> with
>>>>>> the Wikimedia Research Committee, that was completed in July 2011.
>>> The
>>>>> RCom
>>>>>> evaluated the methods, the recruitment strategy, the language used
>> in
>>>> the
>>>>>> survey and approved the proposal pending a final solution for the
>>>>>> recruitment taking into account the concerns expressed by the
>>> community
>>>>> [7].
>>>>>>
>>>>>> Based on suggestions made by community members (e.g. [8]) the
>>> research
>>>>>> team started to work on a technical solution to selectively
>> display a
>>>>>> banner to a subset of registered editors of the English Wikipedia
>>>> meeting
>>>>>> certain eligibility conditions. WMF agreed to invest engineering
>>> effort
>>>>>> into a system that would allow CentralNotice to serve contents to a
>>>>>> specific set of editors – functionality that would benefit future
>>>>>> campaigns run by the community, chapters or the Foundation [9]
>> [10].
>>>>>>
>>>>>> A new CentralNotice backend was then designed to look up various
>>> editor
>>>>>> metrics (i.e. number of contributions, account registration date
>> and
>>>>> editor
>>>>>> privileges) – all public information available from our database –
>>> and
>>>> to
>>>>>> perform a participant eligibility check against these metrics. A
>>> banner
>>>>>> would then be displayed to eligible participants, posting the above
>>>> data
>>>>>> (user ID + editor metrics) along with a unique token to the server
>>>>> hosting
>>>>>> the survey upon clicking. On the landing page of the survey,
>>>> participants
>>>>>> would have the possibility to read the privacy terms of the survey
>>> and
>>>>>> decide whether to take it or not.
>>>>>>
>>>>>> Throughout the review process of this recruitment protocol, the
>>>> research
>>>>>> team received constant feedback from the Foundation’s legal team,
>> the
>>>>>> community department, the tech department and the communication
>> team
>>>>> before
>>>>>> the campaign went live.
>>>>>>
>>>>>> The campaign was announced in the CentralNotice calendar one month
>>>> before
>>>>>> its launch [11] and the launch was with a post on the Foundation’s
>>>> blog.
>>>>>> The banner was enabled on December 8 at 11:00pm UTC. 800+
>>> participants
>>>>>> completed the study within a few hours since its launch. The banner
>>> was
>>>>>> then taken down by a meta-admin a few hours after the launch due to
>>> the
>>>>>> concerns described above.
>>>>>>
>>>>>> So what went wrong?
>>>>>>
>>>>>> ==A few explanations we owe you==
>>>>>>
>>>>>> • Is the Foundation running ads?
>>>>>> No, this banner is a recruitment campaign for a research project
>> that
>>>> has
>>>>>> been thoroughly reviewed by the Research Committee. We have a long
>>>>>> tradition of supporting recruitment for research about our
>>> communities
>>>>> via
>>>>>> various sitenotices. The methodology of this project is sound and
>> the
>>>>>> recruitment method less invasive than thousands of individual
>>> messages
>>>>>> posted on user talk pages. We believe this research will help
>> advance
>>>> our
>>>>>> understanding of the dynamics of participation in our projects.
>>>> Receiving
>>>>>> support by the Research Committee implies that all published output
>>> and
>>>>>> anonymized data produced by this study will be made available under
>>>> open
>>>>>> licenses. [12] The banner also received full Wikimedia Foundation
>>>>> approval
>>>>>> before its launch.
>>>>>>
>>>>>> • Is this campaign conflicting with the fundraiser?
>>>>>> No, this banner is running only for a subset of logged-in editors
>> for
>>>>> whom
>>>>>> the main fundraiser campaign has already been taken down. We
>>> carefully
>>>>>> timed this campaign to minimize the impact on the fundraiser and we
>>>>>> scheduled it on the CentralNotice calendar with a month notice for
>>> this
>>>>>> reason.
>>>>>>
>>>>>> • Is this campaign running at 100% on the English Wikipedia?
>>>>>> No, the banner has been designed to target a subsample of the
>> English
>>>>>> Wikipedia registered editor population. Based on estimates by the
>>>>> research
>>>>>> team, the eligibility criteria apply to about 10,000 very active
>>>>>> contributors and about 30,000 new editors of the English Wikipedia.
>>> The
>>>>>> target number of completed responses is 1500.
>>>>>>
>>>>>> • Why does the banner include logos of organizations not affiliated
>>>> with
>>>>>> Wikimedia?
>>>>>> The design of the banner was based on the decision to give
>>> participants
>>>>> as
>>>>>> much information as possible about the research team running the
>>>> project
>>>>>> and to set accurate expectations about the study.
>>>>>>
>>>>>>
>>>>>> ==What we are doing now==
>>>>>>
>>>>>> We realize that despite an extensive review, the launch of this
>>> project
>>>>>> was not fully advertised on community forums. We plan to shortly
>>> resume
>>>>> the
>>>>>> campaign (for the time needed by the researchers to complete their
>>>>>> responses) after a full redesign of the recruitment protocol in
>> order
>>>> to
>>>>>> address the concerns raised by many of you over the last 24 hours.
>>>> Here’s
>>>>>> what we are doing:
>>>>>>
>>>>>> • Provide you with better information about the project
>>>>>> We asked the research team to promptly set up a FAQ section on the
>>>>> project
>>>>>> page on Meta [13], and to be available to address any concern about
>>> the
>>>>>> study on the discussion page of this project. The project page on
>>> Meta
>>>>> will
>>>>>> be linked from the recruitment banner itself.
>>>>>>
>>>>>> • Redesign the banner
>>>>>> We understand that the banner design has been interpreted by some
>> as
>>>>>> ad-like (even if the goal was to make clear that this study was not
>>>> being
>>>>>> run by WMF, as it implied a redirection to a third party website
>> for
>>>>>> performing the experiment). In coordination with the research team,
>>> we
>>>>> will
>>>>>> come up with a banner design that will be more in line with the
>>>> concerns
>>>>>> expressed by the community (for instance by removing the logos from
>>> the
>>>>>> banner).
>>>>>>
>>>>>> • Make privacy terms as transparent as possible
>>>>>> Upon clicking on the banner, participants accept to share their
>>>> username,
>>>>>> edit count and user privileges with the research team. The previous
>>>>> version
>>>>>> didn’t make it explicit and we are working to address this problem.
>>> To
>>>>> make
>>>>>> the process totally transparent we will make the acceptance of
>> these
>>>>> terms
>>>>>> explicit in the banner itself.
>>>>>>
>>>>>> Once redirected to the landing page, participants will have to
>> accept
>>>> the
>>>>>> terms of participation in order to enter the study. The project is
>>>> funded
>>>>>> by the European Research Council: the data collected in this study
>> is
>>>>>> subject to strict European privacy protocols. The research team
>> will
>>>> use
>>>>>> this data for research purposes only. The research team is not
>>> exposed
>>>> to
>>>>>> and does not record participants’ IP addresses.
>>>>>>
>>>>>> ==How you can help==
>>>>>>
>>>>>> We would like to hear from you on the redesign of the banner to
>> make
>>>> sure
>>>>>> it meets the expectations of the community and doesn’t lend itself
>> to
>>>> any
>>>>>> kind of confusion. We will post the new banners to Meta and try to
>>>>> address
>>>>>> all pending questions before we resume the campaign.
>>>>>>
>>>>>> This is one of the first times we’re supporting a complex,
>> important
>>>>>> research initiative like this one, and I apologize for the bumps in
>>> the
>>>>>> road. We believe that supporting research is part of our mission:
>> it
>>>>> helps
>>>>>> advance our understanding of ourselves. So thanks again for all
>>> support
>>>>> you
>>>>>> can give in making this a success.
>>>>>>
>>>>>>
>>>>>> Dario Taraborelli
>>>>>> Senior Research Analyst, Wikimedia Foundation
>>>>>>
>>>>>> [1]
>> http://blog.wikimedia.org/2011/12/08/experiment-decision-making/
>>>>>> [2]
>>>>>>
>>>>>
>>>>
>>>
>> http://en.wikipedia.org/wiki/Wikipedia:Administrators%27_noticeboard/Incidents#Harvard.2FScience_Po_Adverts
>>>>>> [3]
>>>>>>
>>>>>
>>>>
>>>
>> http://en.wikipedia.org/wiki/Wikipedia:Village_pump_%28technical%29#Search_banner_Wikipedia_Research_Committee
>>>>>> [4]
>>>>>>
>>>>>
>>>>
>>>
>> http://lists.wikimedia.org/pipermail/foundation-l/2011-December/070742.html
>>>>>> [5]
>>>>>>
>>>>>
>>>>
>>>
>> https://lists.wikimedia.org/mailman/private/internal-l/2011-December/018842.html
>>>>>> [6]
>>>>>>
>>>>>
>>>>
>>>
>> http://en.wikipedia.org/wiki/Wikipedia:Administrators%27_noticeboard/Archive222#Researchers_requesting_administrators.E2.80.99_advices_to_launch_a_study
>>>>>> [7]
>>>>>>
>>>>>
>>>>
>>>
>> http://meta.wikimedia.org/wiki/Research_talk:Dynamics_of_Online_Interactions_and_Behavior#RCom_review
>>>>>> [8]
>>>>>
>> http://lists.wikimedia.org/pipermail/foundation-l/2011-May/065580.html
>>>>>> [9]
>>>>>
>> http://lists.wikimedia.org/pipermail/foundation-l/2011-May/065558.html
>>>>>> [10]
>> http://meta.wikimedia.org/wiki/CentralNotice_banner_guidelines
>>>>>> [11]
>>>>>>
>>>>>
>>>>
>>>
>> http://meta.wikimedia.org/w/index.php?title=CentralNotice/Calendar&oldid=3056067
>>>>>> [12] http://meta.wikimedia.org/wiki/Research:Subject_recruitment
>>>>>> [13]
>>>>>>
>>>>>
>>>>
>>>
>> meta.wikimedia.org/wiki/Research:Dynamics_of_Online_Interactions_and_Behavior
>>>>>>
>>>>>>
>>>>>
>>>>> Dario, nobody in any of the discussions on the English Wikipedia
>> (whose
>>>>> editors are the target of this research project) suggested that a
>>>> *central
>>>>> site notice* be used for this or any other research project. The
>>>>> discussion in April 2011 showed consensus opposition to bot-delivered
>>>> talk
>>>>> page notices. One editor involved in the discussion suggested "site
>>>>> notices" (which I believe were interpreted by the participants to
>> mean
>>> a
>>>>> local site notice) and two others mentioned watchlist notices. The
>>>>> subsequent discussion about central notices discussed the possibility
>>> of
>>>>> developing a narrowcasting ability for such notices, and discussed
>>>>> specifically notices directly related to WMF projects or activities.
>>> It
>>>>> did not, in any way, address the concept of using a central notice to
>>>>> promote a non-WMF activity (such as this research project). Indeed,
>>> this
>>>> is
>>>>> the first use of a central notice for anything not directly related
>> to
>>> an
>>>>> obviously WMF-related activity.
>>>>>
>>>>> The ability to narrowcast central notices is a positive advancement;
>>>>> however, the processes for proposing and determining the
>>> appropriateness
>>>> of
>>>>> a narrowcast are poorly publicized, and some of them don't appear to
>>> have
>>>>> even existed until after this notice was taken down. There are still
>> no
>>>>> community-approved guidelines for the use of central notices,
>> although
>>> a
>>>>> draft one is currently up for comment.[1] An RFC initiated in August
>>> 2010
>>>>> with respect to "global banners"/central notices, well in advance of
>>> the
>>>>> development of the narrowcasting ability, strongly supported
>> consensus
>>>>> approval on Meta for non-fundraising global banners.[2] Now that
>> there
>>> is
>>>>> the ability to target central notices to only one project or
>> community,
>>>> it
>>>>> is extremely important that that community be directly notified of
>> such
>>>>> discussion - a discussion that never took place in any public forum
>>> that
>>>> I
>>>>> can see in advance of this central notice being activated.
>>>>>
>>>>> The links above include one to a private mailing list that the
>> majority
>>>> of
>>>>> readers of this list have no access to. You may want to consider
>> asking
>>>> the
>>>>> persons whose contributions are contained in that particular message
>> to
>>>>> grant permission for it to be reproduced here so that the rest of us
>>>> aren't
>>>>> left in the dark about who said what.
>>>>>
>>>>> I don't begridge scholars carrying out approved research with
>>> Wiki?edians
>>>>> who volunteer to do so; in fact, I've responded to several requests
>>>> myself.
>>>>> I do, however, have concerns about any research that expects to
>> contact
>>>>> 40,000 editors and involve 1500 of them; that is a very significant
>>>> portion
>>>>> of our active editorship on the English Wikipedia project. I'm
>> curious
>>>> to
>>>>> know if scholars have shown much interest in studying some of the
>> other
>>>>> projects as much as they've initiated studies on enwp.
>>>>>
>>>>> Risker/Anne
>>>>>
>>>>>
>>>>>
>>>>> [1] http://meta.wikimedia.org/wiki/CentralNotice_banner_guidelines
>>>>> http://meta.wikimedia.org/wiki/Requests_for_comment/Global_banners
>>>>> _______________________________________________
>>>>> foundation-l mailing list
>>>>> foundation-l [at] lists
>>>>> Unsubscribe:
>> https://lists.wikimedia.org/mailman/listinfo/foundation-l
>>>>>
>>>> _______________________________________________
>>>> foundation-l mailing list
>>>> foundation-l [at] lists
>>>> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
>>>>
>>> _______________________________________________
>>> foundation-l mailing list
>>> foundation-l [at] lists
>>> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
>>>
>> _______________________________________________
>> foundation-l mailing list
>> foundation-l [at] lists
>> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
>>
> _______________________________________________
> foundation-l mailing list
> foundation-l [at] lists
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l

_______________________________________________
foundation-l mailing list
foundation-l [at] lists
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l


nemowiki at gmail

Dec 10, 2011, 1:10 AM

Post #8 of 31 (1396 views)
Permalink
Re: Regarding Berkman/Sciences Po study [In reply to]

Dario Taraborelli, 10/12/2011 04:51:
> • Is the Foundation running ads?
> No, this banner is a recruitment campaign for a research project that has been thoroughly reviewed by the Research Committee. We have a long tradition of supporting recruitment for research about our communities via various sitenotices. The methodology of this project is sound and the recruitment method less invasive than thousands of individual messages posted on user talk pages. We believe this research will help advance our understanding of the dynamics of participation in our projects. Receiving support by the Research Committee implies that all published output and anonymized data produced by this study will be made available under open licenses. [12] The banner also received full Wikimedia Foundation approval before its launch.

I think that this should be made more explicit: we shouldn't advertise
any external initiative; when it comes to research projects, we should
allow them to use our wikis and in particular the site notices only if
they're joint projects (as much ours as theirs). This is the case here,
because it's a research project of common interest where both parties
have some gains and some duties, set with the Research committee and a
signed agreement (IIRC). It's good that they will be explained better now.

Nemo

_______________________________________________
foundation-l mailing list
foundation-l [at] lists
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l


morton.thomas at googlemail

Dec 10, 2011, 2:52 AM

Post #9 of 31 (1392 views)
Permalink
Re: Regarding Berkman/Sciences Po study [In reply to]

(I originally sent this to WikiEn-l - but intended to send it here to
foundation-l... my email reader got confused..)

Hi Dario,

This proposal went through a long review process, involving community
> forums, the Research Committee and various WMF departments since early 2010.
>
> The Berkman research team first approached WMF to discuss this study in
> January 2010. They suggested a protocol to recruit English Wikipedia
> contributors to participate in an early version of this study by March 2010
> and posted a proposal to the Administrators’ noticeboard to get community
> feedback [6]. The community response at that time opposed the proposed
> recruitment protocol (posting individual invitation messages on user talk
> pages). It was suggested instead that the recruitment should be handled
> through a CentralNotice banner to be displayed to registered editors, but
> concerns were raised on how to minimize the disruption.
>

This is not a good summary of the conclusions there at all; and it is
worrying that it has been read that way...

You seem to have taken that discussion as implicit approval to run a
CentralNotice banner - although that was certainly suggested as an option
at the time I think it was reasonably expected for further community input
later down the road. Certainly when I supported the suggestion of some sort
of targeted site notice I envisioned a text link, or something.


> Throughout the review process of this recruitment protocol, the research
> team received constant feedback from the Foundation’s legal team, the
> community department, the tech department and the communication team before
> the campaign went live.
>

But not the community?


> The campaign was announced in the CentralNotice calendar one month before
> its launch [11] and the launch was with a post on the Foundation’s blog.
> The banner was enabled on December 8 at 11:00pm UTC. 800+ participants
> completed the study within a few hours since its launch. The banner was
> then taken down by a meta-admin a few hours after the launch due to the
> concerns described above.
>

Again; not announced to the community. There was a clear an present
communication failure here.


> We realize that despite an extensive review, the launch of this project
> was not fully advertised on community forums. We plan to shortly resume the
> campaign (for the time needed by the researchers to complete their
> responses) after a full redesign of the recruitment protocol in order to
> address the concerns raised by many of you over the last 24 hours. Here’s
> what we are doing:
>
> • Provide you with better information about the project
> We asked the research team to promptly set up a FAQ section on the project
> page on Meta [13], and to be available to address any concern about the
> study on the discussion page of this project. The project page on Meta will
> be linked from the recruitment banner itself.
>
> • Redesign the banner
> We understand that the banner design has been interpreted by some as
> ad-like (even if the goal was to make clear that this study was not being
> run by WMF, as it implied a redirection to a third party website for
> performing the experiment). In coordination with the research team, we will
> come up with a banner design that will be more in line with the concerns
> expressed by the community (for instance by removing the logos from the
> banner).
>
> • Make privacy terms as transparent as possible
> Upon clicking on the banner, participants accept to share their username,
> edit count and user privileges with the research team. The previous version
> didn’t make it explicit and we are working to address this problem. To make
> the process totally transparent we will make the acceptance of these terms
> explicit in the banner itself.
>
> Once redirected to the landing page, participants will have to accept the
> terms of participation in order to enter the study. The project is funded
> by the European Research Council: the data collected in this study is
> subject to strict European privacy protocols. The research team will use
> this data for research purposes only. The research team is not exposed to
> and does not record participants’ IP addresses.
>

You need to tell this *to the community*. Otherwise the discussion will
simply strike up again once you re-enable it. I notice you posted this
exact same message to wikipedia-en-l. The lack of recent discussion on that
list should tell you how effective that is as a communication tool.

The vast majority of English Wikipedia discussion occurs on-wiki, and the
vast majority of editors prefer discussions to occur on-wiki. If you want
to interact with the community, and in this case I think you have to, then
you really have to do so on-wiki :)

We would like to hear from you on the redesign of the banner to make sure
> it meets the expectations of the community and doesn’t lend itself to any
> kind of confusion. We will post the new banners to Meta and try to address
> all pending questions before we resume the campaign.
>

Most en.wiki editors don't hang out on Meta - and I think it is reasonable
not to expect them to. Especially as this is purely a project focused on
English Wikipedia; it needs to be discussed on English Wikipedia. If you
need help with the right places/protocol then I would be happy to oblige.

This once again highlights the huge disconnect between Wikipedians, the
foundations and the various higher level commitees. It's the same every
time - something big appears, the community get cross/upset/confused, the
foundation etc. express incredulity... and no one talks.

My final comment is this; I am fairly active around here. I am on
foundation-l, I read meta for important notices, and I am active on
en.wiki. And the first time I knew about this (since the Admin Noticeboard
discussion some months back) was when the banner appeared. So how anyone
else stood a chance of giving input I do not know :) Whilst much of the
discussion was public, it was "hidden". And the key failing here is in not
making it open and accessible.

Tom
_______________________________________________
foundation-l mailing list
foundation-l [at] lists
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l


rupert.thurner at gmail

Dec 10, 2011, 3:57 AM

Post #10 of 31 (1392 views)
Permalink
Re: Regarding Berkman/Sciences Po study [In reply to]

On Sat, Dec 10, 2011 at 10:10, Federico Leva (Nemo) <nemowiki [at] gmail> wrote:
> Dario Taraborelli, 10/12/2011 04:51:
>> • Is the Foundation running ads?
>> No, this banner is a recruitment campaign for a research project that has been thoroughly reviewed by the Research Committee. We have a long tradition of supporting recruitment for research about our communities via various sitenotices. The methodology of this project is sound and the recruitment method less invasive than thousands of individual messages posted on user talk pages. We believe this research will help advance our understanding of the dynamics of participation in our projects. Receiving support by the Research Committee implies that all published output and anonymized data produced by this study will be made available under open licenses. [12] The banner also received full Wikimedia Foundation approval before its launch.
>
> I think that this should be made more explicit: we shouldn't advertise
> any external initiative; when it comes to research projects, we should
> allow them to use our wikis and in particular the site notices only if
> they're joint projects (as much ours as theirs). This is the case here,
> because it's a research project of common interest where both parties
> have some gains and some duties, set with the Research committee and a
> signed agreement (IIRC). It's good that they will be explained better now.

the promise is: no ad. free knowledge. voluntary work, even with
errors, some waste or non-successes. the whole money inflow is based
on this promise. the whole content is created with this promise in
mind.

joint research projects redirecting to other web sites, using
different logos, than wikipedia related is obviously perceived
borderline. do we have the need to be borderline? imo there is no room
for discussion, the answer must be no. and no exception.

rupert.

_______________________________________________
foundation-l mailing list
foundation-l [at] lists
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l


coldchrist at gmail

Dec 10, 2011, 6:14 AM

Post #11 of 31 (1383 views)
Permalink
Re: Regarding Berkman/Sciences Po study [In reply to]

I'm not a fan of me-too posting, but I am breaking that rule to reinforce
the point that there are those who, like Gregory and me, did not see any
problem with the survey. Those who don't like it are, naturally, posting
to comment; those who found no issues with it are probably not. I would
not like to see anyone deducing what the majority opinion is from these
comments. Having said that, the opposition that has been expressed is
quite rational, and I think the proposed changes to the banner are
sensible, but to me it was unproblematic as originally designed.

Mike

On Sat, Dec 10, 2011 at 1:45 AM, Gregory Varnum <admin [at] wikiqueer> wrote:

> Having taken the survey - I honestly don't see what all the fuss is about.
> Sure the banner could have been designed better - but this seems like a
> disproportionate reaction to that minor mistake in the grand scheme of a
> one year approval process that obviously was transparent. They're US based
> organizations, so why are people surprised they'd target en.WP? It's the
> largest WMF project, so logical to focus on it over say en.WT. Focusing on
> editors or people with accounts makes a lot of sense. I'm not really sure
> why people seem to be all caught up in the numbers or target audience.
> I've worked on research that had much higher target numbers than this.
> This is one of the more bizarre (although not the most bizarre) reactions
> I've seen.
>
> I have no stake in the matter nor was I involved in its approval process,
> but as I was taking it I was thinking of many ways WMF and other nonprofits
> could utilize the data. Understanding people's altruistic behavior is
> vital to volunteer recruitment and fundraising efforts. I was impressed
> with how well it was put together, the explanations were especially well
> done. I'm not surprised people on enWP objected. You could suggest we had
> a search box or print feature and I'm confident at least a small vocal
> group would express displeasure without realizing they're features already
> present. I'm not saying they should be dismissed outright, but feel they
> should be taken in better context.
>
> Nothing anyone has said convinces me this is worth such in-depth and
> cyclical debate. Seems to have gone off the rail a bit... Thank you to
> the folks involved for providing responses and background information.
> Already gone beyond what I would have personally expected.
>
> Just my two cents - you may now return to finding problems with it.
>
> -greg aka varnent
>
>
> -------
> Gregory Varnum
> Lead, Aequalitas Project
> Founding Principal, VarnEnt
> @GregVarnum
> fb.com/GregVarnum
>
> On Dec 10, 2011, at 1:27 AM, Risker wrote:
>
> > Hi Jerome -
> >
> > The only documentation from the research team that I have seen so far
> with
> > respect to the target participation is in the initial proposal on enwp
> back
> > in 2010, when it was proposed to leave 40,000 talk page messages; there
> was
> > no indication that 30,000 of them would be newly registered users at that
> > time. Not to criticize the genuine attempt at information sharing on
> > Dario's part - it is much appreciated - but there is so much change in
> what
> > was put forward from what we had initially been approached about that
> it's
> > preferable to hear it from the researcher's mouth, and to have it well
> > documented.
> >
> > Something that has never been clear is the reason that English Wikipedia
> > editors were identified as the preferred target; there does not appear to
> > be anything in this study that is particularly oriented toward Wikipedia
> > activity.
> >
> > Risker/Anne
> >
> > 2011/12/10 Jérôme Hergueux <jerome.hergueux [at] gmail>
> >
> >> This is actually not the case. Those 30,000 users or so are users who
> >> registered their Wikipedia account 30 days prior to the launch of the
> >> study. There are no other requirements for those users to be eligible to
> >> participate. This is in line with Dario's previous message:
> >>
> >> the banner has been designed to target a subsample of the English
> Wikipedia
> >> registered editor population. Based on estimates by the research team,
> the
> >> eligibility criteria apply to about 10,000 very active contributors and
> >> about 30,000 new editors of the English Wikipedia.
> >>
> >> Regards,
> >>
> >> Jérôme.
> >>
> >> 2011/12/10 Risker <risker.wp [at] gmail>
> >>
> >>> Hi Jerome - please show me where it says that; I've not been able to
> >> verify
> >>> that interpretation at all. My understanding is that the 30,000 are
> >> users
> >>> with fewer than 100 edits per month on average, not that they are new
> >>> users.
> >>>
> >>> Risker/Anne
> >>>
> >>> 2011/12/10 Jérôme Hergueux <jerome.hergueux [at] gmail>
> >>>
> >>>> I do, however, have concerns about any research that expects to
> contact
> >>>> 40,000 editors and involve 1500 of them; that is a very significant
> >>> portion
> >>>> of our active editorship on the English Wikipedia project.
> >>>>
> >>>> Commenting on this: out of those targeted 40,000 editors, 30,000 or so
> >>> are
> >>>> *newly registered users*, so that the sample remains somewhat
> >>>> representative of the diversity we find on en:wp. The rest of it
> indeed
> >>> are
> >>>> active contributors.
> >>>>
> >>>> Regards,
> >>>>
> >>>> Jérôme.
> >>>>
> >>>>
> >>>> 2011/12/10 Risker <risker.wp [at] gmail>
> >>>>
> >>>>> On 9 December 2011 22:51, Dario Taraborelli <
> >>> dtaraborelli [at] wikimedia
> >>>>>> wrote:
> >>>>>
> >>>>>> I’d like to give everybody on this list some information on the
> >>>>>> Berkman/Sciences Po research project that many of you have been
> >>>>> discussing
> >>>>>> here.
> >>>>>>
> >>>>>> On Thursday the Wikimedia Foundation announced the launch of a
> >> banner
> >>>> to
> >>>>>> support a study led by a team at the Berkman Center/Sciences Po and
> >>>>>> recruiting participants from the English Wikipedia editor community
> >>>> [1].
> >>>>>> The banner was taken down within hours of its launch after concerns
> >>>>> raised
> >>>>>> in various community forums (the Admin Noticeboard [2], the Village
> >>>> Pump
> >>>>>> Tech [3], various IRC channels and mailing lists such as
> >> foundation-l
> >>>> [4]
> >>>>>> and internal-l [5]) that the design was confusing, that it was
> >>>> perceived
> >>>>> as
> >>>>>> a commercial ad and that the community approval process and privacy
> >>>> terms
> >>>>>> were unclear and hardly visible.
> >>>>>>
> >>>>>> Here’s what happened until the launch, what went wrong after the
> >>> launch
> >>>>>> and what we are planning to do next.
> >>>>>>
> >>>>>> ==The prequel==
> >>>>>> This proposal went through a long review process, involving
> >> community
> >>>>>> forums, the Research Committee and various WMF departments since
> >>> early
> >>>>> 2010.
> >>>>>>
> >>>>>> The Berkman research team first approached WMF to discuss this
> >> study
> >>> in
> >>>>>> January 2010. They suggested a protocol to recruit English
> >> Wikipedia
> >>>>>> contributors to participate in an early version of this study by
> >>> March
> >>>>> 2010
> >>>>>> and posted a proposal to the Administrators’ noticeboard to get
> >>>> community
> >>>>>> feedback [6]. The community response at that time opposed the
> >>> proposed
> >>>>>> recruitment protocol (posting individual invitation messages on
> >> user
> >>>> talk
> >>>>>> pages). It was suggested instead that the recruitment should be
> >>> handled
> >>>>>> through a CentralNotice banner to be displayed to registered
> >> editors,
> >>>> but
> >>>>>> concerns were raised on how to minimize the disruption.
> >>>>>>
> >>>>>> To address these concerns, the proposal went through a full review
> >>> with
> >>>>>> the Wikimedia Research Committee, that was completed in July 2011.
> >>> The
> >>>>> RCom
> >>>>>> evaluated the methods, the recruitment strategy, the language used
> >> in
> >>>> the
> >>>>>> survey and approved the proposal pending a final solution for the
> >>>>>> recruitment taking into account the concerns expressed by the
> >>> community
> >>>>> [7].
> >>>>>>
> >>>>>> Based on suggestions made by community members (e.g. [8]) the
> >>> research
> >>>>>> team started to work on a technical solution to selectively
> >> display a
> >>>>>> banner to a subset of registered editors of the English Wikipedia
> >>>> meeting
> >>>>>> certain eligibility conditions. WMF agreed to invest engineering
> >>> effort
> >>>>>> into a system that would allow CentralNotice to serve contents to a
> >>>>>> specific set of editors – functionality that would benefit future
> >>>>>> campaigns run by the community, chapters or the Foundation [9]
> >> [10].
> >>>>>>
> >>>>>> A new CentralNotice backend was then designed to look up various
> >>> editor
> >>>>>> metrics (i.e. number of contributions, account registration date
> >> and
> >>>>> editor
> >>>>>> privileges) – all public information available from our database –
> >>> and
> >>>> to
> >>>>>> perform a participant eligibility check against these metrics. A
> >>> banner
> >>>>>> would then be displayed to eligible participants, posting the above
> >>>> data
> >>>>>> (user ID + editor metrics) along with a unique token to the server
> >>>>> hosting
> >>>>>> the survey upon clicking. On the landing page of the survey,
> >>>> participants
> >>>>>> would have the possibility to read the privacy terms of the survey
> >>> and
> >>>>>> decide whether to take it or not.
> >>>>>>
> >>>>>> Throughout the review process of this recruitment protocol, the
> >>>> research
> >>>>>> team received constant feedback from the Foundation’s legal team,
> >> the
> >>>>>> community department, the tech department and the communication
> >> team
> >>>>> before
> >>>>>> the campaign went live.
> >>>>>>
> >>>>>> The campaign was announced in the CentralNotice calendar one month
> >>>> before
> >>>>>> its launch [11] and the launch was with a post on the Foundation’s
> >>>> blog.
> >>>>>> The banner was enabled on December 8 at 11:00pm UTC. 800+
> >>> participants
> >>>>>> completed the study within a few hours since its launch. The banner
> >>> was
> >>>>>> then taken down by a meta-admin a few hours after the launch due to
> >>> the
> >>>>>> concerns described above.
> >>>>>>
> >>>>>> So what went wrong?
> >>>>>>
> >>>>>> ==A few explanations we owe you==
> >>>>>>
> >>>>>> • Is the Foundation running ads?
> >>>>>> No, this banner is a recruitment campaign for a research project
> >> that
> >>>> has
> >>>>>> been thoroughly reviewed by the Research Committee. We have a long
> >>>>>> tradition of supporting recruitment for research about our
> >>> communities
> >>>>> via
> >>>>>> various sitenotices. The methodology of this project is sound and
> >> the
> >>>>>> recruitment method less invasive than thousands of individual
> >>> messages
> >>>>>> posted on user talk pages. We believe this research will help
> >> advance
> >>>> our
> >>>>>> understanding of the dynamics of participation in our projects.
> >>>> Receiving
> >>>>>> support by the Research Committee implies that all published output
> >>> and
> >>>>>> anonymized data produced by this study will be made available under
> >>>> open
> >>>>>> licenses. [12] The banner also received full Wikimedia Foundation
> >>>>> approval
> >>>>>> before its launch.
> >>>>>>
> >>>>>> • Is this campaign conflicting with the fundraiser?
> >>>>>> No, this banner is running only for a subset of logged-in editors
> >> for
> >>>>> whom
> >>>>>> the main fundraiser campaign has already been taken down. We
> >>> carefully
> >>>>>> timed this campaign to minimize the impact on the fundraiser and we
> >>>>>> scheduled it on the CentralNotice calendar with a month notice for
> >>> this
> >>>>>> reason.
> >>>>>>
> >>>>>> • Is this campaign running at 100% on the English Wikipedia?
> >>>>>> No, the banner has been designed to target a subsample of the
> >> English
> >>>>>> Wikipedia registered editor population. Based on estimates by the
> >>>>> research
> >>>>>> team, the eligibility criteria apply to about 10,000 very active
> >>>>>> contributors and about 30,000 new editors of the English Wikipedia.
> >>> The
> >>>>>> target number of completed responses is 1500.
> >>>>>>
> >>>>>> • Why does the banner include logos of organizations not affiliated
> >>>> with
> >>>>>> Wikimedia?
> >>>>>> The design of the banner was based on the decision to give
> >>> participants
> >>>>> as
> >>>>>> much information as possible about the research team running the
> >>>> project
> >>>>>> and to set accurate expectations about the study.
> >>>>>>
> >>>>>>
> >>>>>> ==What we are doing now==
> >>>>>>
> >>>>>> We realize that despite an extensive review, the launch of this
> >>> project
> >>>>>> was not fully advertised on community forums. We plan to shortly
> >>> resume
> >>>>> the
> >>>>>> campaign (for the time needed by the researchers to complete their
> >>>>>> responses) after a full redesign of the recruitment protocol in
> >> order
> >>>> to
> >>>>>> address the concerns raised by many of you over the last 24 hours.
> >>>> Here’s
> >>>>>> what we are doing:
> >>>>>>
> >>>>>> • Provide you with better information about the project
> >>>>>> We asked the research team to promptly set up a FAQ section on the
> >>>>> project
> >>>>>> page on Meta [13], and to be available to address any concern about
> >>> the
> >>>>>> study on the discussion page of this project. The project page on
> >>> Meta
> >>>>> will
> >>>>>> be linked from the recruitment banner itself.
> >>>>>>
> >>>>>> • Redesign the banner
> >>>>>> We understand that the banner design has been interpreted by some
> >> as
> >>>>>> ad-like (even if the goal was to make clear that this study was not
> >>>> being
> >>>>>> run by WMF, as it implied a redirection to a third party website
> >> for
> >>>>>> performing the experiment). In coordination with the research team,
> >>> we
> >>>>> will
> >>>>>> come up with a banner design that will be more in line with the
> >>>> concerns
> >>>>>> expressed by the community (for instance by removing the logos from
> >>> the
> >>>>>> banner).
> >>>>>>
> >>>>>> • Make privacy terms as transparent as possible
> >>>>>> Upon clicking on the banner, participants accept to share their
> >>>> username,
> >>>>>> edit count and user privileges with the research team. The previous
> >>>>> version
> >>>>>> didn’t make it explicit and we are working to address this problem.
> >>> To
> >>>>> make
> >>>>>> the process totally transparent we will make the acceptance of
> >> these
> >>>>> terms
> >>>>>> explicit in the banner itself.
> >>>>>>
> >>>>>> Once redirected to the landing page, participants will have to
> >> accept
> >>>> the
> >>>>>> terms of participation in order to enter the study. The project is
> >>>> funded
> >>>>>> by the European Research Council: the data collected in this study
> >> is
> >>>>>> subject to strict European privacy protocols. The research team
> >> will
> >>>> use
> >>>>>> this data for research purposes only. The research team is not
> >>> exposed
> >>>> to
> >>>>>> and does not record participants’ IP addresses.
> >>>>>>
> >>>>>> ==How you can help==
> >>>>>>
> >>>>>> We would like to hear from you on the redesign of the banner to
> >> make
> >>>> sure
> >>>>>> it meets the expectations of the community and doesn’t lend itself
> >> to
> >>>> any
> >>>>>> kind of confusion. We will post the new banners to Meta and try to
> >>>>> address
> >>>>>> all pending questions before we resume the campaign.
> >>>>>>
> >>>>>> This is one of the first times we’re supporting a complex,
> >> important
> >>>>>> research initiative like this one, and I apologize for the bumps in
> >>> the
> >>>>>> road. We believe that supporting research is part of our mission:
> >> it
> >>>>> helps
> >>>>>> advance our understanding of ourselves. So thanks again for all
> >>> support
> >>>>> you
> >>>>>> can give in making this a success.
> >>>>>>
> >>>>>>
> >>>>>> Dario Taraborelli
> >>>>>> Senior Research Analyst, Wikimedia Foundation
> >>>>>>
> >>>>>> [1]
> >> http://blog.wikimedia.org/2011/12/08/experiment-decision-making/
> >>>>>> [2]
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>
> http://en.wikipedia.org/wiki/Wikipedia:Administrators%27_noticeboard/Incidents#Harvard.2FScience_Po_Adverts
> >>>>>> [3]
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>
> http://en.wikipedia.org/wiki/Wikipedia:Village_pump_%28technical%29#Search_banner_Wikipedia_Research_Committee
> >>>>>> [4]
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>
> http://lists.wikimedia.org/pipermail/foundation-l/2011-December/070742.html
> >>>>>> [5]
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>
> https://lists.wikimedia.org/mailman/private/internal-l/2011-December/018842.html
> >>>>>> [6]
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>
> http://en.wikipedia.org/wiki/Wikipedia:Administrators%27_noticeboard/Archive222#Researchers_requesting_administrators.E2.80.99_advices_to_launch_a_study
> >>>>>> [7]
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>
> http://meta.wikimedia.org/wiki/Research_talk:Dynamics_of_Online_Interactions_and_Behavior#RCom_review
> >>>>>> [8]
> >>>>>
> >> http://lists.wikimedia.org/pipermail/foundation-l/2011-May/065580.html
> >>>>>> [9]
> >>>>>
> >> http://lists.wikimedia.org/pipermail/foundation-l/2011-May/065558.html
> >>>>>> [10]
> >> http://meta.wikimedia.org/wiki/CentralNotice_banner_guidelines
> >>>>>> [11]
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>
> http://meta.wikimedia.org/w/index.php?title=CentralNotice/Calendar&oldid=3056067
> >>>>>> [12] http://meta.wikimedia.org/wiki/Research:Subject_recruitment
> >>>>>> [13]
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>
> meta.wikimedia.org/wiki/Research:Dynamics_of_Online_Interactions_and_Behavior
> >>>>>>
> >>>>>>
> >>>>>
> >>>>> Dario, nobody in any of the discussions on the English Wikipedia
> >> (whose
> >>>>> editors are the target of this research project) suggested that a
> >>>> *central
> >>>>> site notice* be used for this or any other research project. The
> >>>>> discussion in April 2011 showed consensus opposition to bot-delivered
> >>>> talk
> >>>>> page notices. One editor involved in the discussion suggested "site
> >>>>> notices" (which I believe were interpreted by the participants to
> >> mean
> >>> a
> >>>>> local site notice) and two others mentioned watchlist notices. The
> >>>>> subsequent discussion about central notices discussed the possibility
> >>> of
> >>>>> developing a narrowcasting ability for such notices, and discussed
> >>>>> specifically notices directly related to WMF projects or activities.
> >>> It
> >>>>> did not, in any way, address the concept of using a central notice to
> >>>>> promote a non-WMF activity (such as this research project). Indeed,
> >>> this
> >>>> is
> >>>>> the first use of a central notice for anything not directly related
> >> to
> >>> an
> >>>>> obviously WMF-related activity.
> >>>>>
> >>>>> The ability to narrowcast central notices is a positive advancement;
> >>>>> however, the processes for proposing and determining the
> >>> appropriateness
> >>>> of
> >>>>> a narrowcast are poorly publicized, and some of them don't appear to
> >>> have
> >>>>> even existed until after this notice was taken down. There are still
> >> no
> >>>>> community-approved guidelines for the use of central notices,
> >> although
> >>> a
> >>>>> draft one is currently up for comment.[1] An RFC initiated in August
> >>> 2010
> >>>>> with respect to "global banners"/central notices, well in advance of
> >>> the
> >>>>> development of the narrowcasting ability, strongly supported
> >> consensus
> >>>>> approval on Meta for non-fundraising global banners.[2] Now that
> >> there
> >>> is
> >>>>> the ability to target central notices to only one project or
> >> community,
> >>>> it
> >>>>> is extremely important that that community be directly notified of
> >> such
> >>>>> discussion - a discussion that never took place in any public forum
> >>> that
> >>>> I
> >>>>> can see in advance of this central notice being activated.
> >>>>>
> >>>>> The links above include one to a private mailing list that the
> >> majority
> >>>> of
> >>>>> readers of this list have no access to. You may want to consider
> >> asking
> >>>> the
> >>>>> persons whose contributions are contained in that particular message
> >> to
> >>>>> grant permission for it to be reproduced here so that the rest of us
> >>>> aren't
> >>>>> left in the dark about who said what.
> >>>>>
> >>>>> I don't begridge scholars carrying out approved research with
> >>> Wiki?edians
> >>>>> who volunteer to do so; in fact, I've responded to several requests
> >>>> myself.
> >>>>> I do, however, have concerns about any research that expects to
> >> contact
> >>>>> 40,000 editors and involve 1500 of them; that is a very significant
> >>>> portion
> >>>>> of our active editorship on the English Wikipedia project. I'm
> >> curious
> >>>> to
> >>>>> know if scholars have shown much interest in studying some of the
> >> other
> >>>>> projects as much as they've initiated studies on enwp.
> >>>>>
> >>>>> Risker/Anne
> >>>>>
> >>>>>
> >>>>>
> >>>>> [1] http://meta.wikimedia.org/wiki/CentralNotice_banner_guidelines
> >>>>> http://meta.wikimedia.org/wiki/Requests_for_comment/Global_banners
> >>>>> _______________________________________________
> >>>>> foundation-l mailing list
> >>>>> foundation-l [at] lists
> >>>>> Unsubscribe:
> >> https://lists.wikimedia.org/mailman/listinfo/foundation-l
> >>>>>
> >>>> _______________________________________________
> >>>> foundation-l mailing list
> >>>> foundation-l [at] lists
> >>>> Unsubscribe:
> https://lists.wikimedia.org/mailman/listinfo/foundation-l
> >>>>
> >>> _______________________________________________
> >>> foundation-l mailing list
> >>> foundation-l [at] lists
> >>> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
> >>>
> >> _______________________________________________
> >> foundation-l mailing list
> >> foundation-l [at] lists
> >> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
> >>
> > _______________________________________________
> > foundation-l mailing list
> > foundation-l [at] lists
> > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
>
> _______________________________________________
> foundation-l mailing list
> foundation-l [at] lists
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
>
_______________________________________________
foundation-l mailing list
foundation-l [at] lists
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l


web at ajbpearce

Dec 10, 2011, 6:53 AM

Post #12 of 31 (1400 views)
Permalink
Re: Regarding Berkman/Sciences Po study [In reply to]

Speaking personally, now that it has been more explained and developed, I have no problem with the survey in principle, I understand the value that could be obtained though it and the work and effort that was clearly put in behind the scenes and I have no desire to be a Groucho Marx (http://www.youtube.com/watch?v=DtMV44yoXZ0) against such ideas.

The principle thing that still concerns me was the way in which this was communicated and the fact that it just popped up with no warning and totally inadequate discussion. The reason that I posted on ANI about it was it was something that looked like an advert, which I had been given no warning about or real opportunity to discuss and which behaved in a way I didn't understand. Other users have said that their first thought was that they had been attacked by a virus or malware, and I think it is fair to say that when you are making your volunteers mistakenly think they have gotten a virus then there has been a communications breakdown!

Anyone who was a little bit active and involved with the wikipedia community would have been able to see at a glance the potential controversy that this campaign would cause if it was not properly co-ordinated and communicated with the community, especially coming on the back of several foundation initiatives which, regardless of their merits and good intentions have been communicated and developed in ways that have left significant numbers of volunteers feeling like they were working to fix a problem that could have been prevented had the foundation moved more cautiously.

Because of this, the biggest take away from this should be in my view, not that we can never risk any such campaigns again in the future - but that the foundation needs to improve its communication and involvement of the community and develop clearer and more transparent ways of proactively soliciting important discussion in advance and addressing concerns before this kind of problem arises, rather than the current approach, which has been in several cases resulted in unnecessary confusion and bad feeling.


--
Alasdair


On Saturday, 10 December 2011 at 15:14, Mike Christie wrote:

> that rule to reinforce
> the point that there are those who, like Gregory and me, did not see any
> problem with the survey. Those who don't like


_______________________________________________
foundation-l mailing list
foundation-l [at] lists
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l


nawrich at gmail

Dec 10, 2011, 6:53 AM

Post #13 of 31 (1393 views)
Permalink
Re: Regarding Berkman/Sciences Po study [In reply to]

On Sat, Dec 10, 2011 at 9:14 AM, Mike Christie <coldchrist [at] gmail> wrote:
> I'm not a fan of me-too posting, but I am breaking that rule to reinforce
> the point that there are those who, like Gregory and me, did not see any
> problem with the survey.  Those who don't like it are, naturally, posting
> to comment; those who found no issues with it are probably not.  I would
> not like to see anyone deducing what the majority opinion is from these
> comments.  Having said that, the opposition that has been expressed is
> quite rational, and I think the proposed changes to the banner are
> sensible, but to me it was unproblematic as originally designed.
>
> Mike
>


Agree with Mike on almost all points, except that all the opposition
has been quite rational. Some of it hasn't been.

While you wouldn't guess it from the outside, many of the most vocal
Wikipedians are actually quite conservative and risk-averse when it
comes to the project, and there is also (and this won't be surprising)
a very strong anti-authoritarian streak that manifests itself as
pushback against independent decision-making on the part of the
WMF.[1] There is some irony in that what appears to have been the most
key discussion about this particular event took place in secret on
internal-l, even while critics on that list blasted the research
project for not including the whole community. The rejection of
leadership and the insistence on involving everyone in everything has
its benefits, but also fairly significant drawbacks - with the upshot
that we're inconsistent and, to outsiders, confused about what we
want.

~Nathan

[1]Jimmy made some of these points well in a post on his talkpage on
this issue. Hopefully he doesn't mind that I reproduce it here:

"Well, I'm unaware of the Foundation being secretive about anything,
so it seems like an easy enough thing to ask them. But I'd like to
challenge the assumption here that everyone needs or has a right to be
informed about every detail of everything affecting the website at all
times. That's just not generally a good use of the Foundation's time
and resources, and it also reinforces what I think is a very unhealthy
conservativism in the community about change. We need to break out of
the idea that every software feature (for example) needs get
"consensus" support (defined as high as 70% in some people's minds!)
for even some very major software changes. We need to break the idea
that the Foundation needs to get permission to run banners in support
of research projects. We really need to break the idea, which is
preposterous nonsense, that if we don't scream bloody murder and get
out the pitchforks, that the Foundation is going to start running paid
advertising soon. Not every slope is slippery, and most things are
better handled by getting informed before protesting.--Jimbo Wales
(talk) 1:34 pm, Yesterday (UTC−5)"

_______________________________________________
foundation-l mailing list
foundation-l [at] lists
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l


dgerard at gmail

Dec 10, 2011, 6:58 AM

Post #14 of 31 (1388 views)
Permalink
Re: Regarding Berkman/Sciences Po study [In reply to]

On 10 December 2011 14:53, Alasdair <web [at] ajbpearce> wrote:

> Speaking personally, now that it has been more explained and developed, I have no problem with the survey in principle,


Agreed, it's a proper survey by proper researchers and good in
substance, it's just been realised in a clunky manner. Everyone
involved is I think becoming fully aware of this :-)


- d.

_______________________________________________
foundation-l mailing list
foundation-l [at] lists
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l


kim at bruning

Dec 10, 2011, 9:13 AM

Post #15 of 31 (1388 views)
Permalink
Re: Regarding Berkman/Sciences Po study [In reply to]

On Fri, Dec 09, 2011 at 07:51:06PM -0800, Dario Taraborelli wrote:
> So what went wrong?

Local consensus does not override global consensus.

* The research committee failed to adhere to applicable consensus. [1]
* In lieu of consensus, the research committee failed to adhere to or point to any applicable policy permitting or denying their action. [2]
* In lieu of policy, the research committee failed to discuss or explain their actions on-wiki within a reasonable time-frame, nor was any subtantial corrective action undertaken within a reasonable time frame. (IAR, WIARM, BRD) [3][4]

Therefore, the decision to terminate the use of the study banners at
this time was correct, and could be (and was) validly carried out by
any meta admin. (In the end 2 different people drew the same
conclusion almost simultaneously)

It seems reasonable to suggest that the research committee take the
time to obtain an (at least slightly) broader consensus before
restarting their study[5]. It is reasonable to believe that the
research committee might be sanctioned individually or collectively,
should they fail to do so. The making of on-wiki statements is highly
recommended, as off-wiki statements do not contribute to consensus.

In future, before a person or committee starts or continues use of a
wiki-resource, it would be wise to:
* Research, interpret, and adhere to any applicable policy/guideline/essay documents.
* Obtain and/or research and/or interpret relevant consensus, and adhere to it.
* Plan sufficient time and resources for the correction of (inevitable) unforseen issues.

If this is done in a timely manner, this needn't take a lot of time or
difficulty. Fixing errors and misunderstandings post-hoc is more
costly.

sincerely,
Kim Bruning


[1] AN* discussions are not merely "suggestions". Depending on
consensus, statements made on AN* can be actionable. Compliance might
be mandatory, failure to comply may be sanctionable.

The following consensus discussion is applicable:
http://en.wikipedia.org/wiki/Wikipedia:Administrators%27_noticeboard/Archive222#Researchers_requesting_administrators.E2.80.99_advices_to_launch_a_study

This discussion can be interpreted as denying the request to run this survey in the then proposed form, and discouraging the current form. The discussion also provides some minimal requirements to make the survey acceptable. These requirements were only partially met.

[2] So far I've only found the following proposed policy:
http://meta.wikimedia.org/wiki/CentralNotice_banner_guidelines . Note
that this recommends that banners be approved by the community
beforehand. This was not done.

[3] AFAICT, no input at
http://en.wikipedia.org/wiki/Wikipedia:ANI#Harvard.2FScience_Po_Adverts
. It may be open to discussion whether or not the research committee
was properly informed via their page
http://meta.wikimedia.org/wiki/Research_talk:Dynamics_of_Online_Interactions_and_Behavior#Discussions_about_the_banner

[4] Jerome and I did try some last minute IAR-ish defence of the
project (and we convinced a number of people!). Obviously, our last-minute
arguments were insufficient to balance out the previous and continuing
issues at the time. (worth a try though! :-)

[5] WP:POLL suggests that simply discussing and then adhering to a
common position is potentially sufficient. Running an actual poll
might be counter-productive.

_______________________________________________
foundation-l mailing list
foundation-l [at] lists
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l


jerome.hergueux at gmail

Dec 10, 2011, 3:10 PM

Post #16 of 31 (1378 views)
Permalink
Re: Regarding Berkman/Sciences Po study [In reply to]

The only documentation from the research team that I have seen so far with
respect to the target participation is in the initial proposal on enwp back
in 2010, when it was proposed to leave 40,000 talk page messages; there was
no indication that 30,000 of them would be newly registered users at that
time. Not to criticize the genuine attempt at information sharing on
Dario's part - it is much appreciated - but there is so much change in what
was put forward from what we had initially been approached about that it's
preferable to hear it from the researcher's mouth, and to have it well
documented.

Reading through the AN discussion again, I don't feel like the basic gist
of our research proposal has changed much since March 2010. Even if it
does not provide the actual number, one of our posts there does say that "
among the 40 000 users or so that we will invite, many will be newly
registered users".
I do agree that the contact protocol for the study, on the other hand, has
changed much since then. This was the result of a longstanding effort that
we made in collaboration with WMF in order to try to bring this study to
Wikipedians in the most community compliant way possible, following the
discussion that we had on the Admin Board. I'm not saying that we may not
have made some mistakes. Our first banner proposals (there were 13 of them
in total) did not feature our logos. We included them in order to comply
with one of WMF's request, with people (in my view somewhat legitimately)
being concerned about the fact that if this banner was to redirect people
to a third party website for the purpose of performing an experiment, it
should be made as clear as possible from the very beginning that this
project was not run by WMF. But I can tell for myself as well as all the
other researchers who worked on this research project for the past year and
a half that we were acting in good faith and remained 100% committed to
respecting this community throughout the process. I think we have achieved
quite a lot already, developing together a code that could selectively
display banners to specific subset of editors based on their user metrics,
which is a significant improvement in the flexibility of the banner tools
now at the disposal of WMF and the community. I am truly sorry that some
community members considered that our banner had to be taken down only a
few hours after what was indeed a great moment for us as a research team...

Regards,

Jérôme.

2011/12/10 Risker <risker.wp [at] gmail>

> Hi Jerome -
>
> The only documentation from the research team that I have seen so far with
> respect to the target participation is in the initial proposal on enwp back
> in 2010, when it was proposed to leave 40,000 talk page messages; there was
> no indication that 30,000 of them would be newly registered users at that
> time. Not to criticize the genuine attempt at information sharing on
> Dario's part - it is much appreciated - but there is so much change in what
> was put forward from what we had initially been approached about that it's
> preferable to hear it from the researcher's mouth, and to have it well
> documented.
>
> Something that has never been clear is the reason that English Wikipedia
> editors were identified as the preferred target; there does not appear to
> be anything in this study that is particularly oriented toward Wikipedia
> activity.
>
> Risker/Anne
>
> 2011/12/10 Jérôme Hergueux <jerome.hergueux [at] gmail>
>
> > This is actually not the case. Those 30,000 users or so are users who
> > registered their Wikipedia account 30 days prior to the launch of the
> > study. There are no other requirements for those users to be eligible to
> > participate. This is in line with Dario's previous message:
> >
> > the banner has been designed to target a subsample of the English
> Wikipedia
> > registered editor population. Based on estimates by the research team,
> the
> > eligibility criteria apply to about 10,000 very active contributors and
> > about 30,000 new editors of the English Wikipedia.
> >
> > Regards,
> >
> > Jérôme.
> >
> > 2011/12/10 Risker <risker.wp [at] gmail>
> >
> > > Hi Jerome - please show me where it says that; I've not been able to
> > verify
> > > that interpretation at all. My understanding is that the 30,000 are
> > users
> > > with fewer than 100 edits per month on average, not that they are new
> > > users.
> > >
> > > Risker/Anne
> > >
> > > 2011/12/10 Jérôme Hergueux <jerome.hergueux [at] gmail>
> > >
> > > > I do, however, have concerns about any research that expects to
> contact
> > > > 40,000 editors and involve 1500 of them; that is a very significant
> > > portion
> > > > of our active editorship on the English Wikipedia project.
> > > >
> > > > Commenting on this: out of those targeted 40,000 editors, 30,000 or
> so
> > > are
> > > > *newly registered users*, so that the sample remains somewhat
> > > > representative of the diversity we find on en:wp. The rest of it
> indeed
> > > are
> > > > active contributors.
> > > >
> > > > Regards,
> > > >
> > > > Jérôme.
> > > >
> > > >
> > > > 2011/12/10 Risker <risker.wp [at] gmail>
> > > >
> > > > > On 9 December 2011 22:51, Dario Taraborelli <
> > > dtaraborelli [at] wikimedia
> > > > > >wrote:
> > > > >
> > > > > > I’d like to give everybody on this list some information on the
> > > > > > Berkman/Sciences Po research project that many of you have been
> > > > > discussing
> > > > > > here.
> > > > > >
> > > > > > On Thursday the Wikimedia Foundation announced the launch of a
> > banner
> > > > to
> > > > > > support a study led by a team at the Berkman Center/Sciences Po
> and
> > > > > > recruiting participants from the English Wikipedia editor
> community
> > > > [1].
> > > > > > The banner was taken down within hours of its launch after
> concerns
> > > > > raised
> > > > > > in various community forums (the Admin Noticeboard [2], the
> Village
> > > > Pump
> > > > > > Tech [3], various IRC channels and mailing lists such as
> > foundation-l
> > > > [4]
> > > > > > and internal-l [5]) that the design was confusing, that it was
> > > > perceived
> > > > > as
> > > > > > a commercial ad and that the community approval process and
> privacy
> > > > terms
> > > > > > were unclear and hardly visible.
> > > > > >
> > > > > > Here’s what happened until the launch, what went wrong after the
> > > launch
> > > > > > and what we are planning to do next.
> > > > > >
> > > > > > ==The prequel==
> > > > > > This proposal went through a long review process, involving
> > community
> > > > > > forums, the Research Committee and various WMF departments since
> > > early
> > > > > 2010.
> > > > > >
> > > > > > The Berkman research team first approached WMF to discuss this
> > study
> > > in
> > > > > > January 2010. They suggested a protocol to recruit English
> > Wikipedia
> > > > > > contributors to participate in an early version of this study by
> > > March
> > > > > 2010
> > > > > > and posted a proposal to the Administrators’ noticeboard to get
> > > > community
> > > > > > feedback [6]. The community response at that time opposed the
> > > proposed
> > > > > > recruitment protocol (posting individual invitation messages on
> > user
> > > > talk
> > > > > > pages). It was suggested instead that the recruitment should be
> > > handled
> > > > > > through a CentralNotice banner to be displayed to registered
> > editors,
> > > > but
> > > > > > concerns were raised on how to minimize the disruption.
> > > > > >
> > > > > > To address these concerns, the proposal went through a full
> review
> > > with
> > > > > > the Wikimedia Research Committee, that was completed in July
> 2011.
> > > The
> > > > > RCom
> > > > > > evaluated the methods, the recruitment strategy, the language
> used
> > in
> > > > the
> > > > > > survey and approved the proposal pending a final solution for the
> > > > > > recruitment taking into account the concerns expressed by the
> > > community
> > > > > [7].
> > > > > >
> > > > > > Based on suggestions made by community members (e.g. [8]) the
> > > research
> > > > > > team started to work on a technical solution to selectively
> > display a
> > > > > > banner to a subset of registered editors of the English Wikipedia
> > > > meeting
> > > > > > certain eligibility conditions. WMF agreed to invest engineering
> > > effort
> > > > > > into a system that would allow CentralNotice to serve contents
> to a
> > > > > > specific set of editors – functionality that would benefit
> future
> > > > > > campaigns run by the community, chapters or the Foundation [9]
> > [10].
> > > > > >
> > > > > > A new CentralNotice backend was then designed to look up various
> > > editor
> > > > > > metrics (i.e. number of contributions, account registration date
> > and
> > > > > editor
> > > > > > privileges) – all public information available from our database
> –
> > > and
> > > > to
> > > > > > perform a participant eligibility check against these metrics. A
> > > banner
> > > > > > would then be displayed to eligible participants, posting the
> above
> > > > data
> > > > > > (user ID + editor metrics) along with a unique token to the
> server
> > > > > hosting
> > > > > > the survey upon clicking. On the landing page of the survey,
> > > > participants
> > > > > > would have the possibility to read the privacy terms of the
> survey
> > > and
> > > > > > decide whether to take it or not.
> > > > > >
> > > > > > Throughout the review process of this recruitment protocol, the
> > > > research
> > > > > > team received constant feedback from the Foundation’s legal team,
> > the
> > > > > > community department, the tech department and the communication
> > team
> > > > > before
> > > > > > the campaign went live.
> > > > > >
> > > > > > The campaign was announced in the CentralNotice calendar one
> month
> > > > before
> > > > > > its launch [11] and the launch was with a post on the
> Foundation’s
> > > > blog.
> > > > > > The banner was enabled on December 8 at 11:00pm UTC. 800+
> > > participants
> > > > > > completed the study within a few hours since its launch. The
> banner
> > > was
> > > > > > then taken down by a meta-admin a few hours after the launch due
> to
> > > the
> > > > > > concerns described above.
> > > > > >
> > > > > > So what went wrong?
> > > > > >
> > > > > > ==A few explanations we owe you==
> > > > > >
> > > > > > • Is the Foundation running ads?
> > > > > > No, this banner is a recruitment campaign for a research project
> > that
> > > > has
> > > > > > been thoroughly reviewed by the Research Committee. We have a
> long
> > > > > > tradition of supporting recruitment for research about our
> > > communities
> > > > > via
> > > > > > various sitenotices. The methodology of this project is sound and
> > the
> > > > > > recruitment method less invasive than thousands of individual
> > > messages
> > > > > > posted on user talk pages. We believe this research will help
> > advance
> > > > our
> > > > > > understanding of the dynamics of participation in our projects.
> > > > Receiving
> > > > > > support by the Research Committee implies that all published
> output
> > > and
> > > > > > anonymized data produced by this study will be made available
> under
> > > > open
> > > > > > licenses. [12] The banner also received full Wikimedia Foundation
> > > > > approval
> > > > > > before its launch.
> > > > > >
> > > > > > • Is this campaign conflicting with the fundraiser?
> > > > > > No, this banner is running only for a subset of logged-in editors
> > for
> > > > > whom
> > > > > > the main fundraiser campaign has already been taken down. We
> > > carefully
> > > > > > timed this campaign to minimize the impact on the fundraiser and
> we
> > > > > > scheduled it on the CentralNotice calendar with a month notice
> for
> > > this
> > > > > > reason.
> > > > > >
> > > > > > • Is this campaign running at 100% on the English Wikipedia?
> > > > > > No, the banner has been designed to target a subsample of the
> > English
> > > > > > Wikipedia registered editor population. Based on estimates by the
> > > > > research
> > > > > > team, the eligibility criteria apply to about 10,000 very active
> > > > > > contributors and about 30,000 new editors of the English
> Wikipedia.
> > > The
> > > > > > target number of completed responses is 1500.
> > > > > >
> > > > > > • Why does the banner include logos of organizations not
> affiliated
> > > > with
> > > > > > Wikimedia?
> > > > > > The design of the banner was based on the decision to give
> > > participants
> > > > > as
> > > > > > much information as possible about the research team running the
> > > > project
> > > > > > and to set accurate expectations about the study.
> > > > > >
> > > > > >
> > > > > > ==What we are doing now==
> > > > > >
> > > > > > We realize that despite an extensive review, the launch of this
> > > project
> > > > > > was not fully advertised on community forums. We plan to shortly
> > > resume
> > > > > the
> > > > > > campaign (for the time needed by the researchers to complete
> their
> > > > > > responses) after a full redesign of the recruitment protocol in
> > order
> > > > to
> > > > > > address the concerns raised by many of you over the last 24
> hours.
> > > > Here’s
> > > > > > what we are doing:
> > > > > >
> > > > > > • Provide you with better information about the project
> > > > > > We asked the research team to promptly set up a FAQ section on
> the
> > > > > project
> > > > > > page on Meta [13], and to be available to address any concern
> about
> > > the
> > > > > > study on the discussion page of this project. The project page on
> > > Meta
> > > > > will
> > > > > > be linked from the recruitment banner itself.
> > > > > >
> > > > > > • Redesign the banner
> > > > > > We understand that the banner design has been interpreted by some
> > as
> > > > > > ad-like (even if the goal was to make clear that this study was
> not
> > > > being
> > > > > > run by WMF, as it implied a redirection to a third party website
> > for
> > > > > > performing the experiment). In coordination with the research
> team,
> > > we
> > > > > will
> > > > > > come up with a banner design that will be more in line with the
> > > > concerns
> > > > > > expressed by the community (for instance by removing the logos
> from
> > > the
> > > > > > banner).
> > > > > >
> > > > > > • Make privacy terms as transparent as possible
> > > > > > Upon clicking on the banner, participants accept to share their
> > > > username,
> > > > > > edit count and user privileges with the research team. The
> previous
> > > > > version
> > > > > > didn’t make it explicit and we are working to address this
> problem.
> > > To
> > > > > make
> > > > > > the process totally transparent we will make the acceptance of
> > these
> > > > > terms
> > > > > > explicit in the banner itself.
> > > > > >
> > > > > > Once redirected to the landing page, participants will have to
> > accept
> > > > the
> > > > > > terms of participation in order to enter the study. The project
> is
> > > > funded
> > > > > > by the European Research Council: the data collected in this
> study
> > is
> > > > > > subject to strict European privacy protocols. The research team
> > will
> > > > use
> > > > > > this data for research purposes only. The research team is not
> > > exposed
> > > > to
> > > > > > and does not record participants’ IP addresses.
> > > > > >
> > > > > > ==How you can help==
> > > > > >
> > > > > > We would like to hear from you on the redesign of the banner to
> > make
> > > > sure
> > > > > > it meets the expectations of the community and doesn’t lend
> itself
> > to
> > > > any
> > > > > > kind of confusion. We will post the new banners to Meta and try
> to
> > > > > address
> > > > > > all pending questions before we resume the campaign.
> > > > > >
> > > > > > This is one of the first times we’re supporting a complex,
> > important
> > > > > > research initiative like this one, and I apologize for the bumps
> in
> > > the
> > > > > > road. We believe that supporting research is part of our mission:
> > it
> > > > > helps
> > > > > > advance our understanding of ourselves. So thanks again for all
> > > support
> > > > > you
> > > > > > can give in making this a success.
> > > > > >
> > > > > >
> > > > > > Dario Taraborelli
> > > > > > Senior Research Analyst, Wikimedia Foundation
> > > > > >
> > > > > > [1]
> > http://blog.wikimedia.org/2011/12/08/experiment-decision-making/
> > > > > > [2]
> > > > > >
> > > > >
> > > >
> > >
> >
> http://en.wikipedia.org/wiki/Wikipedia:Administrators%27_noticeboard/Incidents#Harvard.2FScience_Po_Adverts
> > > > > > [3]
> > > > > >
> > > > >
> > > >
> > >
> >
> http://en.wikipedia.org/wiki/Wikipedia:Village_pump_%28technical%29#Search_banner_Wikipedia_Research_Committee
> > > > > > [4]
> > > > > >
> > > > >
> > > >
> > >
> >
> http://lists.wikimedia.org/pipermail/foundation-l/2011-December/070742.html
> > > > > > [5]
> > > > > >
> > > > >
> > > >
> > >
> >
> https://lists.wikimedia.org/mailman/private/internal-l/2011-December/018842.html
> > > > > > [6]
> > > > > >
> > > > >
> > > >
> > >
> >
> http://en.wikipedia.org/wiki/Wikipedia:Administrators%27_noticeboard/Archive222#Researchers_requesting_administrators.E2.80.99_advices_to_launch_a_study
> > > > > > [7]
> > > > > >
> > > > >
> > > >
> > >
> >
> http://meta.wikimedia.org/wiki/Research_talk:Dynamics_of_Online_Interactions_and_Behavior#RCom_review
> > > > > > [8]
> > > > >
> > http://lists.wikimedia.org/pipermail/foundation-l/2011-May/065580.html
> > > > > > [9]
> > > > >
> > http://lists.wikimedia.org/pipermail/foundation-l/2011-May/065558.html
> > > > > > [10]
> > http://meta.wikimedia.org/wiki/CentralNotice_banner_guidelines
> > > > > > [11]
> > > > > >
> > > > >
> > > >
> > >
> >
> http://meta.wikimedia.org/w/index.php?title=CentralNotice/Calendar&oldid=3056067
> > > > > > [12] http://meta.wikimedia.org/wiki/Research:Subject_recruitment
> > > > > > [13]
> > > > > >
> > > > >
> > > >
> > >
> >
> meta.wikimedia.org/wiki/Research:Dynamics_of_Online_Interactions_and_Behavior
> > > > > >
> > > > > >
> > > > >
> > > > > Dario, nobody in any of the discussions on the English Wikipedia
> > (whose
> > > > > editors are the target of this research project) suggested that a
> > > > *central
> > > > > site notice* be used for this or any other research project. The
> > > > > discussion in April 2011 showed consensus opposition to
> bot-delivered
> > > > talk
> > > > > page notices. One editor involved in the discussion suggested "site
> > > > > notices" (which I believe were interpreted by the participants to
> > mean
> > > a
> > > > > local site notice) and two others mentioned watchlist notices. The
> > > > > subsequent discussion about central notices discussed the
> possibility
> > > of
> > > > > developing a narrowcasting ability for such notices, and discussed
> > > > > specifically notices directly related to WMF projects or
> activities.
> > > It
> > > > > did not, in any way, address the concept of using a central notice
> to
> > > > > promote a non-WMF activity (such as this research project). Indeed,
> > > this
> > > > is
> > > > > the first use of a central notice for anything not directly related
> > to
> > > an
> > > > > obviously WMF-related activity.
> > > > >
> > > > > The ability to narrowcast central notices is a positive
> advancement;
> > > > > however, the processes for proposing and determining the
> > > appropriateness
> > > > of
> > > > > a narrowcast are poorly publicized, and some of them don't appear
> to
> > > have
> > > > > even existed until after this notice was taken down. There are
> still
> > no
> > > > > community-approved guidelines for the use of central notices,
> > although
> > > a
> > > > > draft one is currently up for comment.[1] An RFC initiated in
> August
> > > 2010
> > > > > with respect to "global banners"/central notices, well in advance
> of
> > > the
> > > > > development of the narrowcasting ability, strongly supported
> > consensus
> > > > > approval on Meta for non-fundraising global banners.[2] Now that
> > there
> > > is
> > > > > the ability to target central notices to only one project or
> > community,
> > > > it
> > > > > is extremely important that that community be directly notified of
> > such
> > > > > discussion - a discussion that never took place in any public forum
> > > that
> > > > I
> > > > > can see in advance of this central notice being activated.
> > > > >
> > > > > The links above include one to a private mailing list that the
> > majority
> > > > of
> > > > > readers of this list have no access to. You may want to consider
> > asking
> > > > the
> > > > > persons whose contributions are contained in that particular
> message
> > to
> > > > > grant permission for it to be reproduced here so that the rest of
> us
> > > > aren't
> > > > > left in the dark about who said what.
> > > > >
> > > > > I don't begridge scholars carrying out approved research with
> > > Wiki?edians
> > > > > who volunteer to do so; in fact, I've responded to several requests
> > > > myself.
> > > > > I do, however, have concerns about any research that expects to
> > contact
> > > > > 40,000 editors and involve 1500 of them; that is a very significant
> > > > portion
> > > > > of our active editorship on the English Wikipedia project. I'm
> > curious
> > > > to
> > > > > know if scholars have shown much interest in studying some of the
> > other
> > > > > projects as much as they've initiated studies on enwp.
> > > > >
> > > > > Risker/Anne
> > > > >
> > > > >
> > > > >
> > > > > [1] http://meta.wikimedia.org/wiki/CentralNotice_banner_guidelines
> > > > > http://meta.wikimedia.org/wiki/Requests_for_comment/Global_banners
> > > > > _______________________________________________
> > > > > foundation-l mailing list
> > > > > foundation-l [at] lists
> > > > > Unsubscribe:
> > https://lists.wikimedia.org/mailman/listinfo/foundation-l
> > > > >
> > > > _______________________________________________
> > > > foundation-l mailing list
> > > > foundation-l [at] lists
> > > > Unsubscribe:
> https://lists.wikimedia.org/mailman/listinfo/foundation-l
> > > >
> > > _______________________________________________
> > > foundation-l mailing list
> > > foundation-l [at] lists
> > > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
> > >
> > _______________________________________________
> > foundation-l mailing list
> > foundation-l [at] lists
> > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
> >
> _______________________________________________
> foundation-l mailing list
> foundation-l [at] lists
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
>
_______________________________________________
foundation-l mailing list
foundation-l [at] lists
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l


renatawiki at gmail

Dec 10, 2011, 7:11 PM

Post #17 of 31 (1379 views)
Permalink
Re: Regarding Berkman/Sciences Po study [In reply to]

Me too. I thought the survey was very nice and interesting. And this
"controversy" is actually upsetting me. Somebody comes in and tries to do a
nice thing ($ to WMF, interesting exercise for volunteers, knowledge for
the world). You think people would be happy. No! Never! Analyze and nitpick
tiniest things to death, shred everything to pieces, and accused them of
being most evil.

When did the community turned into this old grumpy bunch being unhappy
about everything?

Renata



On Sat, Dec 10, 2011 at 9:14 AM, Mike Christie <coldchrist [at] gmail> wrote:

> I'm not a fan of me-too posting, but I am breaking that rule to reinforce
> the point that there are those who, like Gregory and me, did not see any
> problem with the survey. Those who don't like it are, naturally, posting
> to comment; those who found no issues with it are probably not. I would
> not like to see anyone deducing what the majority opinion is from these
> comments. Having said that, the opposition that has been expressed is
> quite rational, and I think the proposed changes to the banner are
> sensible, but to me it was unproblematic as originally designed.
>
> Mike
>
> On Sat, Dec 10, 2011 at 1:45 AM, Gregory Varnum <admin [at] wikiqueer>
> wrote:
>
> > Having taken the survey - I honestly don't see what all the fuss is
> about.
> > Sure the banner could have been designed better - but this seems like a
> > disproportionate reaction to that minor mistake in the grand scheme of a
> > one year approval process that obviously was transparent. They're US
> based
> > organizations, so why are people surprised they'd target en.WP? It's the
> > largest WMF project, so logical to focus on it over say en.WT. Focusing
> on
> > editors or people with accounts makes a lot of sense. I'm not really
> sure
> > why people seem to be all caught up in the numbers or target audience.
> > I've worked on research that had much higher target numbers than this.
> > This is one of the more bizarre (although not the most bizarre)
> reactions
> > I've seen.
> >
> > I have no stake in the matter nor was I involved in its approval process,
> > but as I was taking it I was thinking of many ways WMF and other
> nonprofits
> > could utilize the data. Understanding people's altruistic behavior is
> > vital to volunteer recruitment and fundraising efforts. I was impressed
> > with how well it was put together, the explanations were especially well
> > done. I'm not surprised people on enWP objected. You could suggest we
> had
> > a search box or print feature and I'm confident at least a small vocal
> > group would express displeasure without realizing they're features
> already
> > present. I'm not saying they should be dismissed outright, but feel they
> > should be taken in better context.
> >
> > Nothing anyone has said convinces me this is worth such in-depth and
> > cyclical debate. Seems to have gone off the rail a bit... Thank you to
> > the folks involved for providing responses and background information.
> > Already gone beyond what I would have personally expected.
> >
> > Just my two cents - you may now return to finding problems with it.
> >
> > -greg aka varnent
> >
> >
> > -------
> > Gregory Varnum
> > Lead, Aequalitas Project
> > Founding Principal, VarnEnt
> > @GregVarnum
> > fb.com/GregVarnum
> >
> > On Dec 10, 2011, at 1:27 AM, Risker wrote:
> >
> > > Hi Jerome -
> > >
> > > The only documentation from the research team that I have seen so far
> > with
> > > respect to the target participation is in the initial proposal on enwp
> > back
> > > in 2010, when it was proposed to leave 40,000 talk page messages; there
> > was
> > > no indication that 30,000 of them would be newly registered users at
> that
> > > time. Not to criticize the genuine attempt at information sharing on
> > > Dario's part - it is much appreciated - but there is so much change in
> > what
> > > was put forward from what we had initially been approached about that
> > it's
> > > preferable to hear it from the researcher's mouth, and to have it well
> > > documented.
> > >
> > > Something that has never been clear is the reason that English
> Wikipedia
> > > editors were identified as the preferred target; there does not appear
> to
> > > be anything in this study that is particularly oriented toward
> Wikipedia
> > > activity.
> > >
> > > Risker/Anne
> > >
> > > 2011/12/10 Jérôme Hergueux <jerome.hergueux [at] gmail>
> > >
> > >> This is actually not the case. Those 30,000 users or so are users who
> > >> registered their Wikipedia account 30 days prior to the launch of the
> > >> study. There are no other requirements for those users to be eligible
> to
> > >> participate. This is in line with Dario's previous message:
> > >>
> > >> the banner has been designed to target a subsample of the English
> > Wikipedia
> > >> registered editor population. Based on estimates by the research team,
> > the
> > >> eligibility criteria apply to about 10,000 very active contributors
> and
> > >> about 30,000 new editors of the English Wikipedia.
> > >>
> > >> Regards,
> > >>
> > >> Jérôme.
> > >>
> > >> 2011/12/10 Risker <risker.wp [at] gmail>
> > >>
> > >>> Hi Jerome - please show me where it says that; I've not been able to
> > >> verify
> > >>> that interpretation at all. My understanding is that the 30,000 are
> > >> users
> > >>> with fewer than 100 edits per month on average, not that they are new
> > >>> users.
> > >>>
> > >>> Risker/Anne
> > >>>
> > >>> 2011/12/10 Jérôme Hergueux <jerome.hergueux [at] gmail>
> > >>>
> > >>>> I do, however, have concerns about any research that expects to
> > contact
> > >>>> 40,000 editors and involve 1500 of them; that is a very significant
> > >>> portion
> > >>>> of our active editorship on the English Wikipedia project.
> > >>>>
> > >>>> Commenting on this: out of those targeted 40,000 editors, 30,000 or
> so
> > >>> are
> > >>>> *newly registered users*, so that the sample remains somewhat
> > >>>> representative of the diversity we find on en:wp. The rest of it
> > indeed
> > >>> are
> > >>>> active contributors.
> > >>>>
> > >>>> Regards,
> > >>>>
> > >>>> Jérôme.
> > >>>>
> > >>>>
> > >>>> 2011/12/10 Risker <risker.wp [at] gmail>
> > >>>>
> > >>>>> On 9 December 2011 22:51, Dario Taraborelli <
> > >>> dtaraborelli [at] wikimedia
> > >>>>>> wrote:
> > >>>>>
> > >>>>>> I’d like to give everybody on this list some information on the
> > >>>>>> Berkman/Sciences Po research project that many of you have been
> > >>>>> discussing
> > >>>>>> here.
> > >>>>>>
> > >>>>>> On Thursday the Wikimedia Foundation announced the launch of a
> > >> banner
> > >>>> to
> > >>>>>> support a study led by a team at the Berkman Center/Sciences Po
> and
> > >>>>>> recruiting participants from the English Wikipedia editor
> community
> > >>>> [1].
> > >>>>>> The banner was taken down within hours of its launch after
> concerns
> > >>>>> raised
> > >>>>>> in various community forums (the Admin Noticeboard [2], the
> Village
> > >>>> Pump
> > >>>>>> Tech [3], various IRC channels and mailing lists such as
> > >> foundation-l
> > >>>> [4]
> > >>>>>> and internal-l [5]) that the design was confusing, that it was
> > >>>> perceived
> > >>>>> as
> > >>>>>> a commercial ad and that the community approval process and
> privacy
> > >>>> terms
> > >>>>>> were unclear and hardly visible.
> > >>>>>>
> > >>>>>> Here’s what happened until the launch, what went wrong after the
> > >>> launch
> > >>>>>> and what we are planning to do next.
> > >>>>>>
> > >>>>>> ==The prequel==
> > >>>>>> This proposal went through a long review process, involving
> > >> community
> > >>>>>> forums, the Research Committee and various WMF departments since
> > >>> early
> > >>>>> 2010.
> > >>>>>>
> > >>>>>> The Berkman research team first approached WMF to discuss this
> > >> study
> > >>> in
> > >>>>>> January 2010. They suggested a protocol to recruit English
> > >> Wikipedia
> > >>>>>> contributors to participate in an early version of this study by
> > >>> March
> > >>>>> 2010
> > >>>>>> and posted a proposal to the Administrators’ noticeboard to get
> > >>>> community
> > >>>>>> feedback [6]. The community response at that time opposed the
> > >>> proposed
> > >>>>>> recruitment protocol (posting individual invitation messages on
> > >> user
> > >>>> talk
> > >>>>>> pages). It was suggested instead that the recruitment should be
> > >>> handled
> > >>>>>> through a CentralNotice banner to be displayed to registered
> > >> editors,
> > >>>> but
> > >>>>>> concerns were raised on how to minimize the disruption.
> > >>>>>>
> > >>>>>> To address these concerns, the proposal went through a full review
> > >>> with
> > >>>>>> the Wikimedia Research Committee, that was completed in July 2011.
> > >>> The
> > >>>>> RCom
> > >>>>>> evaluated the methods, the recruitment strategy, the language used
> > >> in
> > >>>> the
> > >>>>>> survey and approved the proposal pending a final solution for the
> > >>>>>> recruitment taking into account the concerns expressed by the
> > >>> community
> > >>>>> [7].
> > >>>>>>
> > >>>>>> Based on suggestions made by community members (e.g. [8]) the
> > >>> research
> > >>>>>> team started to work on a technical solution to selectively
> > >> display a
> > >>>>>> banner to a subset of registered editors of the English Wikipedia
> > >>>> meeting
> > >>>>>> certain eligibility conditions. WMF agreed to invest engineering
> > >>> effort
> > >>>>>> into a system that would allow CentralNotice to serve contents to
> a
> > >>>>>> specific set of editors – functionality that would benefit future
> > >>>>>> campaigns run by the community, chapters or the Foundation [9]
> > >> [10].
> > >>>>>>
> > >>>>>> A new CentralNotice backend was then designed to look up various
> > >>> editor
> > >>>>>> metrics (i.e. number of contributions, account registration date
> > >> and
> > >>>>> editor
> > >>>>>> privileges) – all public information available from our database –
> > >>> and
> > >>>> to
> > >>>>>> perform a participant eligibility check against these metrics. A
> > >>> banner
> > >>>>>> would then be displayed to eligible participants, posting the
> above
> > >>>> data
> > >>>>>> (user ID + editor metrics) along with a unique token to the server
> > >>>>> hosting
> > >>>>>> the survey upon clicking. On the landing page of the survey,
> > >>>> participants
> > >>>>>> would have the possibility to read the privacy terms of the survey
> > >>> and
> > >>>>>> decide whether to take it or not.
> > >>>>>>
> > >>>>>> Throughout the review process of this recruitment protocol, the
> > >>>> research
> > >>>>>> team received constant feedback from the Foundation’s legal team,
> > >> the
> > >>>>>> community department, the tech department and the communication
> > >> team
> > >>>>> before
> > >>>>>> the campaign went live.
> > >>>>>>
> > >>>>>> The campaign was announced in the CentralNotice calendar one month
> > >>>> before
> > >>>>>> its launch [11] and the launch was with a post on the Foundation’s
> > >>>> blog.
> > >>>>>> The banner was enabled on December 8 at 11:00pm UTC. 800+
> > >>> participants
> > >>>>>> completed the study within a few hours since its launch. The
> banner
> > >>> was
> > >>>>>> then taken down by a meta-admin a few hours after the launch due
> to
> > >>> the
> > >>>>>> concerns described above.
> > >>>>>>
> > >>>>>> So what went wrong?
> > >>>>>>
> > >>>>>> ==A few explanations we owe you==
> > >>>>>>
> > >>>>>> • Is the Foundation running ads?
> > >>>>>> No, this banner is a recruitment campaign for a research project
> > >> that
> > >>>> has
> > >>>>>> been thoroughly reviewed by the Research Committee. We have a long
> > >>>>>> tradition of supporting recruitment for research about our
> > >>> communities
> > >>>>> via
> > >>>>>> various sitenotices. The methodology of this project is sound and
> > >> the
> > >>>>>> recruitment method less invasive than thousands of individual
> > >>> messages
> > >>>>>> posted on user talk pages. We believe this research will help
> > >> advance
> > >>>> our
> > >>>>>> understanding of the dynamics of participation in our projects.
> > >>>> Receiving
> > >>>>>> support by the Research Committee implies that all published
> output
> > >>> and
> > >>>>>> anonymized data produced by this study will be made available
> under
> > >>>> open
> > >>>>>> licenses. [12] The banner also received full Wikimedia Foundation
> > >>>>> approval
> > >>>>>> before its launch.
> > >>>>>>
> > >>>>>> • Is this campaign conflicting with the fundraiser?
> > >>>>>> No, this banner is running only for a subset of logged-in editors
> > >> for
> > >>>>> whom
> > >>>>>> the main fundraiser campaign has already been taken down. We
> > >>> carefully
> > >>>>>> timed this campaign to minimize the impact on the fundraiser and
> we
> > >>>>>> scheduled it on the CentralNotice calendar with a month notice for
> > >>> this
> > >>>>>> reason.
> > >>>>>>
> > >>>>>> • Is this campaign running at 100% on the English Wikipedia?
> > >>>>>> No, the banner has been designed to target a subsample of the
> > >> English
> > >>>>>> Wikipedia registered editor population. Based on estimates by the
> > >>>>> research
> > >>>>>> team, the eligibility criteria apply to about 10,000 very active
> > >>>>>> contributors and about 30,000 new editors of the English
> Wikipedia.
> > >>> The
> > >>>>>> target number of completed responses is 1500.
> > >>>>>>
> > >>>>>> • Why does the banner include logos of organizations not
> affiliated
> > >>>> with
> > >>>>>> Wikimedia?
> > >>>>>> The design of the banner was based on the decision to give
> > >>> participants
> > >>>>> as
> > >>>>>> much information as possible about the research team running the
> > >>>> project
> > >>>>>> and to set accurate expectations about the study.
> > >>>>>>
> > >>>>>>
> > >>>>>> ==What we are doing now==
> > >>>>>>
> > >>>>>> We realize that despite an extensive review, the launch of this
> > >>> project
> > >>>>>> was not fully advertised on community forums. We plan to shortly
> > >>> resume
> > >>>>> the
> > >>>>>> campaign (for the time needed by the researchers to complete their
> > >>>>>> responses) after a full redesign of the recruitment protocol in
> > >> order
> > >>>> to
> > >>>>>> address the concerns raised by many of you over the last 24 hours.
> > >>>> Here’s
> > >>>>>> what we are doing:
> > >>>>>>
> > >>>>>> • Provide you with better information about the project
> > >>>>>> We asked the research team to promptly set up a FAQ section on the
> > >>>>> project
> > >>>>>> page on Meta [13], and to be available to address any concern
> about
> > >>> the
> > >>>>>> study on the discussion page of this project. The project page on
> > >>> Meta
> > >>>>> will
> > >>>>>> be linked from the recruitment banner itself.
> > >>>>>>
> > >>>>>> • Redesign the banner
> > >>>>>> We understand that the banner design has been interpreted by some
> > >> as
> > >>>>>> ad-like (even if the goal was to make clear that this study was
> not
> > >>>> being
> > >>>>>> run by WMF, as it implied a redirection to a third party website
> > >> for
> > >>>>>> performing the experiment). In coordination with the research
> team,
> > >>> we
> > >>>>> will
> > >>>>>> come up with a banner design that will be more in line with the
> > >>>> concerns
> > >>>>>> expressed by the community (for instance by removing the logos
> from
> > >>> the
> > >>>>>> banner).
> > >>>>>>
> > >>>>>> • Make privacy terms as transparent as possible
> > >>>>>> Upon clicking on the banner, participants accept to share their
> > >>>> username,
> > >>>>>> edit count and user privileges with the research team. The
> previous
> > >>>>> version
> > >>>>>> didn’t make it explicit and we are working to address this
> problem.
> > >>> To
> > >>>>> make
> > >>>>>> the process totally transparent we will make the acceptance of
> > >> these
> > >>>>> terms
> > >>>>>> explicit in the banner itself.
> > >>>>>>
> > >>>>>> Once redirected to the landing page, participants will have to
> > >> accept
> > >>>> the
> > >>>>>> terms of participation in order to enter the study. The project is
> > >>>> funded
> > >>>>>> by the European Research Council: the data collected in this study
> > >> is
> > >>>>>> subject to strict European privacy protocols. The research team
> > >> will
> > >>>> use
> > >>>>>> this data for research purposes only. The research team is not
> > >>> exposed
> > >>>> to
> > >>>>>> and does not record participants’ IP addresses.
> > >>>>>>
> > >>>>>> ==How you can help==
> > >>>>>>
> > >>>>>> We would like to hear from you on the redesign of the banner to
> > >> make
> > >>>> sure
> > >>>>>> it meets the expectations of the community and doesn’t lend itself
> > >> to
> > >>>> any
> > >>>>>> kind of confusion. We will post the new banners to Meta and try to
> > >>>>> address
> > >>>>>> all pending questions before we resume the campaign.
> > >>>>>>
> > >>>>>> This is one of the first times we’re supporting a complex,
> > >> important
> > >>>>>> research initiative like this one, and I apologize for the bumps
> in
> > >>> the
> > >>>>>> road. We believe that supporting research is part of our mission:
> > >> it
> > >>>>> helps
> > >>>>>> advance our understanding of ourselves. So thanks again for all
> > >>> support
> > >>>>> you
> > >>>>>> can give in making this a success.
> > >>>>>>
> > >>>>>>
> > >>>>>> Dario Taraborelli
> > >>>>>> Senior Research Analyst, Wikimedia Foundation
> > >>>>>>
> > >>>>>> [1]
> > >> http://blog.wikimedia.org/2011/12/08/experiment-decision-making/
> > >>>>>> [2]
> > >>>>>>
> > >>>>>
> > >>>>
> > >>>
> > >>
> >
> http://en.wikipedia.org/wiki/Wikipedia:Administrators%27_noticeboard/Incidents#Harvard.2FScience_Po_Adverts
> > >>>>>> [3]
> > >>>>>>
> > >>>>>
> > >>>>
> > >>>
> > >>
> >
> http://en.wikipedia.org/wiki/Wikipedia:Village_pump_%28technical%29#Search_banner_Wikipedia_Research_Committee
> > >>>>>> [4]
> > >>>>>>
> > >>>>>
> > >>>>
> > >>>
> > >>
> >
> http://lists.wikimedia.org/pipermail/foundation-l/2011-December/070742.html
> > >>>>>> [5]
> > >>>>>>
> > >>>>>
> > >>>>
> > >>>
> > >>
> >
> https://lists.wikimedia.org/mailman/private/internal-l/2011-December/018842.html
> > >>>>>> [6]
> > >>>>>>
> > >>>>>
> > >>>>
> > >>>
> > >>
> >
> http://en.wikipedia.org/wiki/Wikipedia:Administrators%27_noticeboard/Archive222#Researchers_requesting_administrators.E2.80.99_advices_to_launch_a_study
> > >>>>>> [7]
> > >>>>>>
> > >>>>>
> > >>>>
> > >>>
> > >>
> >
> http://meta.wikimedia.org/wiki/Research_talk:Dynamics_of_Online_Interactions_and_Behavior#RCom_review
> > >>>>>> [8]
> > >>>>>
> > >>
> http://lists.wikimedia.org/pipermail/foundation-l/2011-May/065580.html
> > >>>>>> [9]
> > >>>>>
> > >>
> http://lists.wikimedia.org/pipermail/foundation-l/2011-May/065558.html
> > >>>>>> [10]
> > >> http://meta.wikimedia.org/wiki/CentralNotice_banner_guidelines
> > >>>>>> [11]
> > >>>>>>
> > >>>>>
> > >>>>
> > >>>
> > >>
> >
> http://meta.wikimedia.org/w/index.php?title=CentralNotice/Calendar&oldid=3056067
> > >>>>>> [12] http://meta.wikimedia.org/wiki/Research:Subject_recruitment
> > >>>>>> [13]
> > >>>>>>
> > >>>>>
> > >>>>
> > >>>
> > >>
> >
> meta.wikimedia.org/wiki/Research:Dynamics_of_Online_Interactions_and_Behavior
> > >>>>>>
> > >>>>>>
> > >>>>>
> > >>>>> Dario, nobody in any of the discussions on the English Wikipedia
> > >> (whose
> > >>>>> editors are the target of this research project) suggested that a
> > >>>> *central
> > >>>>> site notice* be used for this or any other research project. The
> > >>>>> discussion in April 2011 showed consensus opposition to
> bot-delivered
> > >>>> talk
> > >>>>> page notices. One editor involved in the discussion suggested "site
> > >>>>> notices" (which I believe were interpreted by the participants to
> > >> mean
> > >>> a
> > >>>>> local site notice) and two others mentioned watchlist notices. The
> > >>>>> subsequent discussion about central notices discussed the
> possibility
> > >>> of
> > >>>>> developing a narrowcasting ability for such notices, and discussed
> > >>>>> specifically notices directly related to WMF projects or
> activities.
> > >>> It
> > >>>>> did not, in any way, address the concept of using a central notice
> to
> > >>>>> promote a non-WMF activity (such as this research project). Indeed,
> > >>> this
> > >>>> is
> > >>>>> the first use of a central notice for anything not directly related
> > >> to
> > >>> an
> > >>>>> obviously WMF-related activity.
> > >>>>>
> > >>>>> The ability to narrowcast central notices is a positive
> advancement;
> > >>>>> however, the processes for proposing and determining the
> > >>> appropriateness
> > >>>> of
> > >>>>> a narrowcast are poorly publicized, and some of them don't appear
> to
> > >>> have
> > >>>>> even existed until after this notice was taken down. There are
> still
> > >> no
> > >>>>> community-approved guidelines for the use of central notices,
> > >> although
> > >>> a
> > >>>>> draft one is currently up for comment.[1] An RFC initiated in
> August
> > >>> 2010
> > >>>>> with respect to "global banners"/central notices, well in advance
> of
> > >>> the
> > >>>>> development of the narrowcasting ability, strongly supported
> > >> consensus
> > >>>>> approval on Meta for non-fundraising global banners.[2] Now that
> > >> there
> > >>> is
> > >>>>> the ability to target central notices to only one project or
> > >> community,
> > >>>> it
> > >>>>> is extremely important that that community be directly notified of
> > >> such
> > >>>>> discussion - a discussion that never took place in any public forum
> > >>> that
> > >>>> I
> > >>>>> can see in advance of this central notice being activated.
> > >>>>>
> > >>>>> The links above include one to a private mailing list that the
> > >> majority
> > >>>> of
> > >>>>> readers of this list have no access to. You may want to consider
> > >> asking
> > >>>> the
> > >>>>> persons whose contributions are contained in that particular
> message
> > >> to
> > >>>>> grant permission for it to be reproduced here so that the rest of
> us
> > >>>> aren't
> > >>>>> left in the dark about who said what.
> > >>>>>
> > >>>>> I don't begridge scholars carrying out approved research with
> > >>> Wiki?edians
> > >>>>> who volunteer to do so; in fact, I've responded to several requests
> > >>>> myself.
> > >>>>> I do, however, have concerns about any research that expects to
> > >> contact
> > >>>>> 40,000 editors and involve 1500 of them; that is a very significant
> > >>>> portion
> > >>>>> of our active editorship on the English Wikipedia project. I'm
> > >> curious
> > >>>> to
> > >>>>> know if scholars have shown much interest in studying some of the
> > >> other
> > >>>>> projects as much as they've initiated studies on enwp.
> > >>>>>
> > >>>>> Risker/Anne
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>> [1] http://meta.wikimedia.org/wiki/CentralNotice_banner_guidelines
> > >>>>> http://meta.wikimedia.org/wiki/Requests_for_comment/Global_banners
> > >>>>> _______________________________________________
> > >>>>> foundation-l mailing list
> > >>>>> foundation-l [at] lists
> > >>>>> Unsubscribe:
> > >> https://lists.wikimedia.org/mailman/listinfo/foundation-l
> > >>>>>
> > >>>> _______________________________________________
> > >>>> foundation-l mailing list
> > >>>> foundation-l [at] lists
> > >>>> Unsubscribe:
> > https://lists.wikimedia.org/mailman/listinfo/foundation-l
> > >>>>
> > >>> _______________________________________________
> > >>> foundation-l mailing list
> > >>> foundation-l [at] lists
> > >>> Unsubscribe:
> https://lists.wikimedia.org/mailman/listinfo/foundation-l
> > >>>
> > >> _______________________________________________
> > >> foundation-l mailing list
> > >> foundation-l [at] lists
> > >> Unsubscribe:
> https://lists.wikimedia.org/mailman/listinfo/foundation-l
> > >>
> > > _______________________________________________
> > > foundation-l mailing list
> > > foundation-l [at] lists
> > > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
> >
> > _______________________________________________
> > foundation-l mailing list
> > foundation-l [at] lists
> > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
> >
> _______________________________________________
> foundation-l mailing list
> foundation-l [at] lists
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
>
_______________________________________________
foundation-l mailing list
foundation-l [at] lists
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l


kim at bruning

Dec 10, 2011, 7:31 PM

Post #18 of 31 (1397 views)
Permalink
Re: Regarding Berkman/Sciences Po study [In reply to]

On Sat, Dec 10, 2011 at 10:11:35PM -0500, Renata St wrote:
>
> When did the community turned into this old grumpy bunch being unhappy
> about everything?


Eh? Not in the least. I think Jerome is a nice guy; and so does
practically everyone else who has gotten up to speed on what's going on.

The problem is that the research committee made only a token effort
at finding or following relevant onwiki policy or consensus , nor did
they try to explain or correct their actions onwiki in a timely manner
as per WIARM. Or where they did, they didn't follow up.

Any of those 3 elements (Policy, Consensus, WIARM/BRD) each could and
still can help bring people up to speed and reduce misunderstandings.
That's part of what they're for, after all! I'm sure that people will be
more supportive once things are sorted out in that way.

Hmm, the research committee still hasn't made any onwiki statement at a
relevant location that I can find. If this were a court case, RCom
would pretty much have lost by default and/or forfeit already.

Fortunately, this is not a court case. :-)

Berkman and Science PO are nice people. They've already interacted with
enwiki a bit, and enwiki isn't exactly feeling mean towards them;
mostly just a little surprised and bewildered. I think that in the
worst case, we can cut out the middleman and have enwiki interact
directly with the researchers.


sincerely,
Kim "There's a process for that[*]" Bruning

[*] 'Staunch IAR Supporter States "There's a process for that"; In
other news, Flying Pig Airlines are now offering scenic ski-trips to
hell for the first time this season.'


_______________________________________________
foundation-l mailing list
foundation-l [at] lists
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l


renatawiki at gmail

Dec 11, 2011, 9:48 AM

Post #19 of 31 (1392 views)
Permalink
Re: Regarding Berkman/Sciences Po study [In reply to]

>
> The problem is that the research committee made only a token effort
> at finding or following relevant onwiki policy or consensus , nor did
> they try to explain or correct their actions onwiki in a timely manner
> as per WIARM. Or where they did, they didn't follow up.
>
> Any of those 3 elements (Policy, Consensus, WIARM/BRD) each could and
> still can help bring people up to speed and reduce misunderstandings.
> That's part of what they're for, after all! I'm sure that people will be
> more supportive once things are sorted out in that way.
>
> Hmm, the research committee still hasn't made any onwiki statement at a
> relevant location that I can find. If this were a court case, RCom
> would pretty much have lost by default and/or forfeit already.
>

As I said, analyze and nitpick things to death. Does any of that above *
really* matter?

It distresses me to see the community turned into this insane
policy-enforcing power-hungry gang. Everything must be approved by us
(consensus)! Everything must follow each letter and comma of every goddarn
policy out there! If there is a single comma missing we will shred you to
pieces, treat you like a scum and public enemy number 1, whack you with all
kinds of warnings, AN threads, blocks... Yeah, you go back to where you
came from and stay there![1]

Since when doing something nice and interesting on WP should be treated and
compared to going to a court? Why and when did the community started to
think that compliance with WP:IDHCWTSF[2] is more important than
intentions, than doing the "right thing", than embracing new, different
ideas? Why does everything have to go through nine circles of bureaucracy?

I weep for the memory of Wikipedia that was *free*. Yes, it is still free
[as in $ and *©*], but it is no longer free of the instruction creep that
stifles and regulates your every movement. I weep for the memory of a
feeling that you *can* change, you *can* edit, you *can* do... without that
gripping fear that you are violating some random policy and therefore will
be whacked on your head with some large stick.

Renata

[1] Exaggerated, yes, but isn't this the typical newbie experience these
days?
[2] Wikipedia:I don't have a clue what this stands for
_______________________________________________
foundation-l mailing list
foundation-l [at] lists
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l


kim at bruning

Dec 11, 2011, 9:57 AM

Post #20 of 31 (1383 views)
Permalink
Re: Regarding Berkman/Sciences Po study [In reply to]

On Sun, Dec 11, 2011 at 12:48:05PM -0500, Renata St wrote:
> >
> > Hmm, the research committee still hasn't made any onwiki statement at a
> > relevant location that I can find. If this were a court case, RCom
> > would pretty much have lost by default and/or forfeit already.
> >
>
> As I said, analyze and nitpick things to death. Does any of that above *
> really* matter?
>
> It distresses me to see the community turned into this insane
> policy-enforcing power-hungry gang.

Heh, you're using the right argument in response to the wrong person. What can I say, you're absolutely right. The community really needs to be more flexible.

Oddly, depite all criticism, apparently they're currently still faster on their feet and more flexible than the foundation (or rcom, at least)

There's 3 ways in which I feel you can get away with murder on wikipedia and still get a barnstar. ;-) [1]

These methods still work!

1: If it's in a policy, it's pre-approved. We're done here, let's go home and get lemonade... But what if you don't know policy?
2: Well, you can just go talk with someone, ask, and maybe reach agreement (consensus)... But what if you don't know anyone?
3: Well, just do what you think is right ...but/and if someone comes up and goes "'ello 'ello 'ello, what's this 'ere then?" You should have your
answer ready on-wiki. ;-) ('ignore all rules')


I'm ok with people not knowing policy(1), I'm ok with people not quite grokking consensus(2), and you know what? I'm worlds the biggest fan of
'do what ye will' (modulo 'An it harm no one') (3).

Now when we get to the 'An it harm no one' part; how are people going to figure that out? Well, that's when they start asking questions. They're required to assume good faith on your part, and -conversely- you're not even required to answer their questions!

So far, you still haven't done anything wrong. It's all allowed!

Now, if you don't answer questions, that's perfectly ok, people will simply try to puzzle things out on their own. If they decide that your actions are Mostly Harmless, you're good, carry on.

Of course, if they can't figure it out (or think you are doing something bad) someone might ask you to stop doing what you are doing. If you then stop doing it, once again, you're in the clear.

Only if you continue after being asked to stop, or if you do something after people have told you "don't do that" or "don't do it that way" ... ok, well, then the community will really have to use a little bit of muscle to stop you.

And that's what happened here.

The Rcom can probably STILL solve this issue at almost any moment in time, by simply going on wiki and actually either: Answering the questions asked of them (3), reaching agreement on what to do next(2), or pointing to a policy that says they can run banners(1).

I did take Jerome on-wiki on thursday/friday. This helped a bit :-) But not enough, since the actual rcom weren't around for backup.

Personally, my rule of thumb is to give people 24 hours. Fair's fair: people don't watch the wiki all day, they might live in a different time zone, etc. Rcom have exceeded that limit and they still haven't posted on-wiki (which is the place where the people are who actually have a say).

Well, ok, maybe Rcom decided to start weekend early and already went to the pub? :-)

Nooo, wait, they actually posted on foundation-l more recently. ... Eh? Didn't we all tell them to go talk to the actual wiki-folks A.S.A. effing P?

I'll be frank. Rcom has messed this one up simply by flailing around in circles and failing to do the one. single. thing. they. needed. to. really. really. do.

Rcom: "Do you expect us to die?"
Goldfinger: "No Mr. Rcom, we expect you to talk!"
http://www.youtube.com/watch?v=U1TmeBd9338

sincerely,
Kim "stirred, not shaken" Bruning


--

_______________________________________________
foundation-l mailing list
foundation-l [at] lists
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l


kim at bruning

Dec 11, 2011, 11:24 AM

Post #21 of 31 (1377 views)
Permalink
Re: Regarding Berkman/Sciences Po study [In reply to]

On Sun, Dec 11, 2011 at 11:45:26PM +0400, Yaroslav M. Blanter wrote:
> Do you actually realize that RCom is not a single person
Of course. That's why it's a good idea to assign a liason (which is a role
done by a person). The other solution is to allow rcom members to speak and
think autonomously, like we expect from wikipedians. :-)

> and that there
> is no way we can issue an official statement in 24 hours?
We don't need an official statement. We need people on the ground who
communicate, answer questions and make decisions. To summarise: you need
people to negotiate consensus.

> I am not sure we need to issue anything as a body anyway
I'll help you with this one; I am absolutely certain: You do not.

> but if we need it is absolutely unrealistic to do in 24 hours, since most
> of us have not been involved since June (when we last approved the survey).
Good for you. Would you happen to know when _en.wikipedia_ approved the survey?
;-)


> We do not have any magic means of communication
I have such magic in my pocket. It's my appointment diary.

> that would get every RCom member
We really need only 1 or 2, who are fully briefed on what Rcom is doing, and
only at those times when the Rcom is doing things.

> online immediately
It's very easy to be online "immediately" when you are doing a pre-planned
action, and therefore know the times to be online well in advance.

> and produce a statement
No statement is necessary, per se.

> and answer all the questions.
That's the one good idea here. Also, the members you have online must be authorised to
make decisions. You are required to either follow policy and/or reach
consensus, and this often requires negotiation.

By analogy: imagine you walk into a bank office and start working on their
computers. No one knows who you are, no one knows what you are doing, and you
have no contract, contact info of the local manager, or ID on you.
What's going to happen?

sincerely,
Kim Bruning
--

_______________________________________________
foundation-l mailing list
foundation-l [at] lists
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l


kim at bruning

Dec 11, 2011, 11:33 AM

Post #22 of 31 (1381 views)
Permalink
Re: Regarding Berkman/Sciences Po study [In reply to]

On Mon, Dec 12, 2011 at 12:00:42AM +0400, Yaroslav M. Blanter wrote:
> > With the greatest of respect.. it shouldn't need the whole of RCOM, no
> > statement is needed. Just for someone to go on-wiki and answer queries
> (or
> > if nothing else say - sorry we need to look into this, bear with us).
> >
> > The complaint Kim is making is that no one has done this.
> >
> > Tom
>
> Well, I can do it (not answering the questions, since I am not involved in
> the project, but to say we need more time). What would be the most
> appropriate place to do it?

I've already done some of that for you, together with Jerome. :-)

A new subsection here would work:
http://en.wikipedia.org/wiki/Wikipedia:Administrators%27_noticeboard/Incidents#Harvard.2FScience_Po_Adverts

And what we really need right now is 1 (preferably 2) rcom members who are
involved with the project.

To be sure: this is not just a good idea. This is basic wiki(pedia) policy for
beginners. ;-)

sincerely,
Kim Bruning

--

_______________________________________________
foundation-l mailing list
foundation-l [at] lists
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l


node.ue at gmail

Dec 11, 2011, 11:36 AM

Post #23 of 31 (1380 views)
Permalink
Re: Regarding Berkman/Sciences Po study [In reply to]

How about,

"What we are doing now: In response to community outcry, we have decided
not to repost such banners, rather than talking down to the community and
telling them they have misunderstood people's intentions".

I'm tired of the Foundation making unpopular decisions and then talking
down to the community, implying that the community merely doesn't
understand, and if we did, we'd accept it.


2011/12/9 Dario Taraborelli <dtaraborelli [at] wikimedia>

> I’d like to give everybody on this list some information on the
> Berkman/Sciences Po research project that many of you have been discussing
> here.
>
> On Thursday the Wikimedia Foundation announced the launch of a banner to
> support a study led by a team at the Berkman Center/Sciences Po and
> recruiting participants from the English Wikipedia editor community [1].
> The banner was taken down within hours of its launch after concerns raised
> in various community forums (the Admin Noticeboard [2], the Village Pump
> Tech [3], various IRC channels and mailing lists such as foundation-l [4]
> and internal-l [5]) that the design was confusing, that it was perceived as
> a commercial ad and that the community approval process and privacy terms
> were unclear and hardly visible.
>
> Here’s what happened until the launch, what went wrong after the launch
> and what we are planning to do next.
>
> ==The prequel==
> This proposal went through a long review process, involving community
> forums, the Research Committee and various WMF departments since early 2010.
>
> The Berkman research team first approached WMF to discuss this study in
> January 2010. They suggested a protocol to recruit English Wikipedia
> contributors to participate in an early version of this study by March 2010
> and posted a proposal to the Administrators’ noticeboard to get community
> feedback [6]. The community response at that time opposed the proposed
> recruitment protocol (posting individual invitation messages on user talk
> pages). It was suggested instead that the recruitment should be handled
> through a CentralNotice banner to be displayed to registered editors, but
> concerns were raised on how to minimize the disruption.
>
> To address these concerns, the proposal went through a full review with
> the Wikimedia Research Committee, that was completed in July 2011. The RCom
> evaluated the methods, the recruitment strategy, the language used in the
> survey and approved the proposal pending a final solution for the
> recruitment taking into account the concerns expressed by the community [7].
>
> Based on suggestions made by community members (e.g. [8]) the research
> team started to work on a technical solution to selectively display a
> banner to a subset of registered editors of the English Wikipedia meeting
> certain eligibility conditions. WMF agreed to invest engineering effort
> into a system that would allow CentralNotice to serve contents to a
> specific set of editors – functionality that would benefit future
> campaigns run by the community, chapters or the Foundation [9] [10].
>
> A new CentralNotice backend was then designed to look up various editor
> metrics (i.e. number of contributions, account registration date and editor
> privileges) – all public information available from our database – and to
> perform a participant eligibility check against these metrics. A banner
> would then be displayed to eligible participants, posting the above data
> (user ID + editor metrics) along with a unique token to the server hosting
> the survey upon clicking. On the landing page of the survey, participants
> would have the possibility to read the privacy terms of the survey and
> decide whether to take it or not.
>
> Throughout the review process of this recruitment protocol, the research
> team received constant feedback from the Foundation’s legal team, the
> community department, the tech department and the communication team before
> the campaign went live.
>
> The campaign was announced in the CentralNotice calendar one month before
> its launch [11] and the launch was with a post on the Foundation’s blog.
> The banner was enabled on December 8 at 11:00pm UTC. 800+ participants
> completed the study within a few hours since its launch. The banner was
> then taken down by a meta-admin a few hours after the launch due to the
> concerns described above.
>
> So what went wrong?
>
> ==A few explanations we owe you==
>
> • Is the Foundation running ads?
> No, this banner is a recruitment campaign for a research project that has
> been thoroughly reviewed by the Research Committee. We have a long
> tradition of supporting recruitment for research about our communities via
> various sitenotices. The methodology of this project is sound and the
> recruitment method less invasive than thousands of individual messages
> posted on user talk pages. We believe this research will help advance our
> understanding of the dynamics of participation in our projects. Receiving
> support by the Research Committee implies that all published output and
> anonymized data produced by this study will be made available under open
> licenses. [12] The banner also received full Wikimedia Foundation approval
> before its launch.
>
> • Is this campaign conflicting with the fundraiser?
> No, this banner is running only for a subset of logged-in editors for whom
> the main fundraiser campaign has already been taken down. We carefully
> timed this campaign to minimize the impact on the fundraiser and we
> scheduled it on the CentralNotice calendar with a month notice for this
> reason.
>
> • Is this campaign running at 100% on the English Wikipedia?
> No, the banner has been designed to target a subsample of the English
> Wikipedia registered editor population. Based on estimates by the research
> team, the eligibility criteria apply to about 10,000 very active
> contributors and about 30,000 new editors of the English Wikipedia. The
> target number of completed responses is 1500.
>
> • Why does the banner include logos of organizations not affiliated with
> Wikimedia?
> The design of the banner was based on the decision to give participants as
> much information as possible about the research team running the project
> and to set accurate expectations about the study.
>
>
> ==What we are doing now==
>
> We realize that despite an extensive review, the launch of this project
> was not fully advertised on community forums. We plan to shortly resume the
> campaign (for the time needed by the researchers to complete their
> responses) after a full redesign of the recruitment protocol in order to
> address the concerns raised by many of you over the last 24 hours. Here’s
> what we are doing:
>
> • Provide you with better information about the project
> We asked the research team to promptly set up a FAQ section on the project
> page on Meta [13], and to be available to address any concern about the
> study on the discussion page of this project. The project page on Meta will
> be linked from the recruitment banner itself.
>
> • Redesign the banner
> We understand that the banner design has been interpreted by some as
> ad-like (even if the goal was to make clear that this study was not being
> run by WMF, as it implied a redirection to a third party website for
> performing the experiment). In coordination with the research team, we will
> come up with a banner design that will be more in line with the concerns
> expressed by the community (for instance by removing the logos from the
> banner).
>
> • Make privacy terms as transparent as possible
> Upon clicking on the banner, participants accept to share their username,
> edit count and user privileges with the research team. The previous version
> didn’t make it explicit and we are working to address this problem. To make
> the process totally transparent we will make the acceptance of these terms
> explicit in the banner itself.
>
> Once redirected to the landing page, participants will have to accept the
> terms of participation in order to enter the study. The project is funded
> by the European Research Council: the data collected in this study is
> subject to strict European privacy protocols. The research team will use
> this data for research purposes only. The research team is not exposed to
> and does not record participants’ IP addresses.
>
> ==How you can help==
>
> We would like to hear from you on the redesign of the banner to make sure
> it meets the expectations of the community and doesn’t lend itself to any
> kind of confusion. We will post the new banners to Meta and try to address
> all pending questions before we resume the campaign.
>
> This is one of the first times we’re supporting a complex, important
> research initiative like this one, and I apologize for the bumps in the
> road. We believe that supporting research is part of our mission: it helps
> advance our understanding of ourselves. So thanks again for all support you
> can give in making this a success.
>
>
> Dario Taraborelli
> Senior Research Analyst, Wikimedia Foundation
>
> [1] http://blog.wikimedia.org/2011/12/08/experiment-decision-making/
> [2]
> http://en.wikipedia.org/wiki/Wikipedia:Administrators%27_noticeboard/Incidents#Harvard.2FScience_Po_Adverts
> [3]
> http://en.wikipedia.org/wiki/Wikipedia:Village_pump_%28technical%29#Search_banner_Wikipedia_Research_Committee
> [4]
> http://lists.wikimedia.org/pipermail/foundation-l/2011-December/070742.html
> [5]
> https://lists.wikimedia.org/mailman/private/internal-l/2011-December/018842.html
> [6]
> http://en.wikipedia.org/wiki/Wikipedia:Administrators%27_noticeboard/Archive222#Researchers_requesting_administrators.E2.80.99_advices_to_launch_a_study
> [7]
> http://meta.wikimedia.org/wiki/Research_talk:Dynamics_of_Online_Interactions_and_Behavior#RCom_review
> [8] http://lists.wikimedia.org/pipermail/foundation-l/2011-May/065580.html
> [9] http://lists.wikimedia.org/pipermail/foundation-l/2011-May/065558.html
> [10] http://meta.wikimedia.org/wiki/CentralNotice_banner_guidelines
> [11]
> http://meta.wikimedia.org/w/index.php?title=CentralNotice/Calendar&oldid=3056067
> [12] http://meta.wikimedia.org/wiki/Research:Subject_recruitment
> [13]
> meta.wikimedia.org/wiki/Research:Dynamics_of_Online_Interactions_and_Behavior
>
> _______________________________________________
> foundation-l mailing list
> foundation-l [at] lists
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
>
_______________________________________________
foundation-l mailing list
foundation-l [at] lists
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l


putevod at mccme

Dec 11, 2011, 11:45 AM

Post #24 of 31 (1391 views)
Permalink
Re: Regarding Berkman/Sciences Po study [In reply to]

> Personally, my rule of thumb is to give people 24 hours. Fair's fair:
> people don't watch the wiki all day, they might live in a different time
> zone, etc. Rcom have exceeded that limit and they still haven't posted
> on-wiki (which is the place where the people are who actually have a
say).
>
> Well, ok, maybe Rcom decided to start weekend early and already went to
> the pub? :-)
>
> Nooo, wait, they actually posted on foundation-l more recently. ... Eh?
> Didn't we all tell them to go talk to the actual wiki-folks A.S.A.
effing
> P?
>
> I'll be frank. Rcom has messed this one up simply by flailing around in
> circles and failing to do the one. single. thing. they. needed. to.
> really. really. do.
>
> Rcom: "Do you expect us to die?"
> Goldfinger: "No Mr. Rcom, we expect you to talk!"
> http://www.youtube.com/watch?v=U1TmeBd9338
>
Do you actually realize that RCom is not a single person, and that there
is no way we can issue an official statement in 24 hours? I am not sure we
need to issue anything as a body anyway, but if we need it is absolutely
unrealistic to do in 24 hours, since most of us have not been involved
since June (when we last approved the survey). We do not have any magic
means of communication that would get every RCom member online immediately
and produce a statement and answer all the questions.

Cheers
Yaroslav

_______________________________________________
foundation-l mailing list
foundation-l [at] lists
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l


kim at bruning

Dec 11, 2011, 11:48 AM

Post #25 of 31 (1388 views)
Permalink
Re: Regarding Berkman/Sciences Po study [In reply to]

On Mon, Dec 12, 2011 at 12:27:34AM +0400, Yaroslav M. Blanter wrote:
> I will do it right now, but to make it clear, we have 2 (TWO: twee, zwei,
> deux, dos ...) RCOM members in total who are involved: Dario and Mayo. I do
> not think anybody else would be able to answer any questions.
> And last contribution of Mayo in en.wp, from what I see, is from June.
> So I guess it would be difficult to have two RCom members answering
> questions.

That is most unfortunate.

> Btw trolling on mailing lists is also not just a bad idea, it
> goes against a basic policy for beginners.

With the greatest possible respect; I would suggest that the research
committee does not have the kind of standing required to accuse others of
disruptive behaviour, at this point in time.

sincerely,
Kim Bruning




--

_______________________________________________
foundation-l mailing list
foundation-l [at] lists
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l

First page Previous page 1 2 Next page Last page  View All Wikipedia foundation RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.