Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Wikipedia: Foundation

[Wikimedia-l] Fundraising updates?

 

 

Wikipedia foundation RSS feed   Index | Next | Previous | View Threaded


itzik at infra

Dec 13, 2012, 6:04 AM

Post #1 of 15 (951 views)
Permalink
[Wikimedia-l] Fundraising updates?

Hi,

Could we get some updates (or that I missed them?) about how the
fundraising goes? WMF and Chapters will be great. Since we only focus on
few countries this year, the discussions regarding it is very low, but it
still very interesting to know how much we collect and how the banners
works (and which one of them).

Thanks,

Itzik
_______________________________________________
Wikimedia-l mailing list
Wikimedia-l [at] lists
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l


de10011 at gmail

Dec 13, 2012, 6:12 AM

Post #2 of 15 (926 views)
Permalink
Re: [Wikimedia-l] Fundraising updates? [In reply to]

On Thu, Dec 13, 2012 at 7:34 PM, Itzik Edri <itzik [at] infra> wrote:

> Hi,
>
> Could we get some updates (or that I missed them?) about how the
> fundraising goes? WMF and Chapters will be great. Since we only focus on
> few countries this year, the discussions regarding it is very low, but it
> still very interesting to know how much we collect and how the banners
> works (and which one of them)
>

http://wikimediafoundation.org/wiki/Special:FundraiserStatistics - is a
good start.

-Theo
_______________________________________________
Wikimedia-l mailing list
Wikimedia-l [at] lists
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l


till.mletzko at wikimedia

Dec 13, 2012, 6:14 AM

Post #3 of 15 (924 views)
Permalink
Re: [Wikimedia-l] Fundraising updates? [In reply to]

and: http://meta.wikimedia.org/wiki/Fundraising_2012

Till


Am 13.12.2012 15:12, schrieb Theo10011:
> On Thu, Dec 13, 2012 at 7:34 PM, Itzik Edri <itzik [at] infra> wrote:
>
>> Hi,
>>
>> Could we get some updates (or that I missed them?) about how the
>> fundraising goes? WMF and Chapters will be great. Since we only focus on
>> few countries this year, the discussions regarding it is very low, but it
>> still very interesting to know how much we collect and how the banners
>> works (and which one of them)
>>
> http://wikimediafoundation.org/wiki/Special:FundraiserStatistics - is a
> good start.
>
> -Theo
> _______________________________________________
> Wikimedia-l mailing list
> Wikimedia-l [at] lists
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l


--
Mit freundlichen Grüßen

Till Mletzko
Fundraiser
-------------------------------------
Wikimedia Fördergesellschaft
Obentrautstr. 72
10963 Berlin

Telefon 030 - 219 158 26 -19
www.wikimedia.de

Helfen Sie mit, dass WIKIPEDIA von der UNESCO als erstes digitales Weltkulturerbe anerkannt wird.
Unterzeichnen Sie die Online-Petition unter https://wke.wikimedia.de/wke/Main_Page!

Stellen Sie sich eine Welt vor, in der jeder Mensch freien Zugang zu der Gesamtheit des Wissens der Menschheit hat. Helfen Sie uns dabei! http://spenden.wikimedia.de/

Gemeinnützige Wikimedia Fördergesellschaft mbH.
Eingetragen beim Amtsgericht Berlin-Charlottenburg unter der Nummer 130183 B. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/603/54814.


_______________________________________________
Wikimedia-l mailing list
Wikimedia-l [at] lists
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l


itzik at infra

Dec 13, 2012, 10:01 AM

Post #4 of 15 (916 views)
Permalink
Re: [Wikimedia-l] Fundraising updates? [In reply to]

Indeed, a good start, that I already checked.

The statistics shows only WMF data. Last year we had google docs file that
the chapters shared their numbers every day also.

And there is no indication to which banners running and their performance,
things that I'll like to see.

The fundraising team did some interesting changes this year, which will
be interesting to know their performances.

Itzik.

2012/12/13 Till Mletzko <till.mletzko [at] wikimedia>

> and: http://meta.wikimedia.org/wiki/Fundraising_2012
>
> Till
>
>
> Am 13.12.2012 15:12, schrieb Theo10011:
> > On Thu, Dec 13, 2012 at 7:34 PM, Itzik Edri <itzik [at] infra> wrote:
> >
> >> Hi,
> >>
> >> Could we get some updates (or that I missed them?) about how the
> >> fundraising goes? WMF and Chapters will be great. Since we only focus on
> >> few countries this year, the discussions regarding it is very low, but
> it
> >> still very interesting to know how much we collect and how the banners
> >> works (and which one of them)
> >>
> > http://wikimediafoundation.org/wiki/Special:FundraiserStatistics - is a
> > good start.
> >
> > -Theo
> > _______________________________________________
> > Wikimedia-l mailing list
> > Wikimedia-l [at] lists
> > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l
>
>
> --
> Mit freundlichen Grüßen
>
> Till Mletzko
> Fundraiser
> -------------------------------------
> Wikimedia Fördergesellschaft
> Obentrautstr. 72
> 10963 Berlin
>
> Telefon 030 - 219 158 26 -19
> www.wikimedia.de
>
> Helfen Sie mit, dass WIKIPEDIA von der UNESCO als erstes digitales
> Weltkulturerbe anerkannt wird.
> Unterzeichnen Sie die Online-Petition unter
> https://wke.wikimedia.de/wke/Main_Page!
>
> Stellen Sie sich eine Welt vor, in der jeder Mensch freien Zugang zu der
> Gesamtheit des Wissens der Menschheit hat. Helfen Sie uns dabei!
> http://spenden.wikimedia.de/
>
> Gemeinnützige Wikimedia Fördergesellschaft mbH.
> Eingetragen beim Amtsgericht Berlin-Charlottenburg unter der Nummer 130183
> B. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I
> Berlin, Steuernummer 27/603/54814.
>
>
> _______________________________________________
> Wikimedia-l mailing list
> Wikimedia-l [at] lists
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l
>
_______________________________________________
Wikimedia-l mailing list
Wikimedia-l [at] lists
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l


zexley at wikimedia

Dec 14, 2012, 7:36 AM

Post #5 of 15 (911 views)
Permalink
Re: [Wikimedia-l] Fundraising updates? [In reply to]

Hi Itzik -

I can give a short update -- and there will be more details in the
fundraising report after the campaign.

The banners from last year with the faces of editors, staff or Jimmy and
"Please read a personal appeal from..." stopped working between last year
and this year. We tried very hard to figure out why, but I still can't say
exactly why. It could be a mix of underlying issues that are pulling down
performance of any kind of banner and the fact that everyone on the
internet knows exactly what's in that "personal appeal" now and are no
longer curious enough to click.

What saved us was taking text from the personal appeals and putting it into
the banner itself. These banners did very well. These new message-driven
banners are what made us split the campaign in two -- because we knew we
were going to develop a lot of new messages and not have time to translate
them well. The campaign that started on the 27th of November ran only in
five countries: Australia, Britain, Canada, New Zealand and the United
States.

At first, we had a short version of the new banner:

http://en.wikipedia.org/wiki/Main_Page?banner=B12_1123_Smallinfo_fix

We could have run that for 46 days (the length of last year's campaign) and
probably made our goal. But it performed better the more information we put
into it. Through a series of tests we became confident that, while greater
banner height improved performance, that wasn't as big a factor as the
additional information we put into the banner. We tested many new versions
of messages in the banners and found many improvements. It looked like we
might be able to have a 25 day-long fundraiser.

We launched on the 27th. A few days into the campaign, we were still
paranoid about the goal. We were afraid that maybe the new banners would
burn out faster than the old ones did. Maybe they were good at getting
donations faster, but maybe we were not increasing the overall pool of
donors. We were constantly testing to boost performance. Out of curiosity,
we tested making the banners stick to the top of the screen while the page
scrolled. We knew that would be a dramatic step in a more annoying
direction, but like I said, we were still worried about the goal and just
wanted to know what our options were. The "sticky" banners did about 30%
better for donations. So we decided to keep them and see if we could get
the campaign done with in a very short time.

After 8 days of having the banners up, we were able take banners down and
only display them to people who had not seen them before (or rather,
browsers and computers that had not seen them before). We've never been
able to do this before and only had this feature fully developed several
days into the campaign. Since we took banners down for everyone, we've
mostly been displaying them only 1 or 2 times to people who've never seen
them. Though yesterday we pushed that up to 10 because we're hoping to
reach our US$25 million goal in the next few days.

We also made the banners stop sticking after the first 8 days, and
hopefully we'll never feel we have to use sticky banners again. In total,
we had sticky banners up for 4 or 5 days.

We hope that next week we'll be able to start a sort of "Thank you
campaign." We will feature a thank you message, a video of Wikimedia
editors talking about their experience, interviews and written messages
from editors collected at Wikimania, and an invitation to all our readers
to become editors. The purpose of this campaign is mainly to raise
awareness among readers about how Wikimedia projects work and who is behind
them. The purpose is also to take time to explicitly thank donors for
helping us reach our goal so quickly this year. Thanking is a very
important part of fundraising -- but we've always been so eager to take
down banners that we've never taken enough time to thank donors in the
past. This year we feel its ok since almost no one saw banners for more
than 8 days.

I know there will probably be a lot of detailed questions about income from
different countries, comparisons to last year, to the chapters, etc... We
can't answer those now because we're too busy trying to wrap up the
campaign -- and also because a lot of transactions take time to settle.
Checks flow in slowly. And accurate comparisons take time to prepare. We
don't have truly accurate numbers until later in January.

But the basic result we have is that all 5 countries we ran banners saw a
huge increase in "donations per banner impression" from last year. I can
also say that in our Nov 15th 24-hour "dress rehearsal" in which we ran an
earlier version of the new banners (that didn't perform near as well as the
one we eventually discovered) we also saw a very big increase in almost
every country in the world. The exceptions were countries where we made
some error (such as not turning banners on for half the day). Even in a
country like Greece, where the economy is about as bad as it can get, we
saw something like a 300% increase compared the the equivalent first day of
the campaign last year. Therefore, we think we'll be able to keep our
Spring campaign very low-impact.

I hope this satisfies some of the curiosity about the campaign. Please ask
questions, but please be understanding if we can't answer detailed
questions while we're busy still running the campaign.

Zack


On Thu, Dec 13, 2012 at 12:01 PM, Itzik Edri <itzik [at] infra> wrote:

> Indeed, a good start, that I already checked.
>
> The statistics shows only WMF data. Last year we had google docs file that
> the chapters shared their numbers every day also.
>
> And there is no indication to which banners running and their performance,
> things that I'll like to see.
>
> The fundraising team did some interesting changes this year, which will
> be interesting to know their performances.
>
> Itzik.
>
> 2012/12/13 Till Mletzko <till.mletzko [at] wikimedia>
>
> > and: http://meta.wikimedia.org/wiki/Fundraising_2012
> >
> > Till
> >
> >
> > Am 13.12.2012 15:12, schrieb Theo10011:
> > > On Thu, Dec 13, 2012 at 7:34 PM, Itzik Edri <itzik [at] infra> wrote:
> > >
> > >> Hi,
> > >>
> > >> Could we get some updates (or that I missed them?) about how the
> > >> fundraising goes? WMF and Chapters will be great. Since we only focus
> on
> > >> few countries this year, the discussions regarding it is very low, but
> > it
> > >> still very interesting to know how much we collect and how the banners
> > >> works (and which one of them)
> > >>
> > > http://wikimediafoundation.org/wiki/Special:FundraiserStatistics -
> is a
> > > good start.
> > >
> > > -Theo
> > > _______________________________________________
> > > Wikimedia-l mailing list
> > > Wikimedia-l [at] lists
> > > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l
> >
> >
> > --
> > Mit freundlichen Grüßen
> >
> > Till Mletzko
> > Fundraiser
> > -------------------------------------
> > Wikimedia Fördergesellschaft
> > Obentrautstr. 72
> > 10963 Berlin
> >
> > Telefon 030 - 219 158 26 -19
> > www.wikimedia.de
> >
> > Helfen Sie mit, dass WIKIPEDIA von der UNESCO als erstes digitales
> > Weltkulturerbe anerkannt wird.
> > Unterzeichnen Sie die Online-Petition unter
> > https://wke.wikimedia.de/wke/Main_Page!
> >
> > Stellen Sie sich eine Welt vor, in der jeder Mensch freien Zugang zu der
> > Gesamtheit des Wissens der Menschheit hat. Helfen Sie uns dabei!
> > http://spenden.wikimedia.de/
> >
> > Gemeinnützige Wikimedia Fördergesellschaft mbH.
> > Eingetragen beim Amtsgericht Berlin-Charlottenburg unter der Nummer
> 130183
> > B. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I
> > Berlin, Steuernummer 27/603/54814.
> >
> >
> > _______________________________________________
> > Wikimedia-l mailing list
> > Wikimedia-l [at] lists
> > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l
> >
> _______________________________________________
> Wikimedia-l mailing list
> Wikimedia-l [at] lists
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l
>



--
Zack Exley
Chief Revenue Officer
Wikimedia Foundation
415 506 9225
_______________________________________________
Wikimedia-l mailing list
Wikimedia-l [at] lists
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l


itzik at infra

Dec 15, 2012, 3:03 AM

Post #6 of 15 (905 views)
Permalink
Re: [Wikimedia-l] Fundraising updates? [In reply to]

Zack, Thanks for the detailed email. It was very interesting email and
I appreciate that you took the time to write it. Good work!

I still hope to see information from Chapters about the amount collected
till now, to help us see the full picture of the fundraising this year.


Itzik

On Fri, Dec 14, 2012 at 5:36 PM, Zack Exley <zexley [at] wikimedia> wrote:

> Hi Itzik -
>
> I can give a short update -- and there will be more details in the
> fundraising report after the campaign.
>
> The banners from last year with the faces of editors, staff or Jimmy and
> "Please read a personal appeal from..." stopped working between last year
> and this year. We tried very hard to figure out why, but I still can't say
> exactly why. It could be a mix of underlying issues that are pulling down
> performance of any kind of banner and the fact that everyone on the
> internet knows exactly what's in that "personal appeal" now and are no
> longer curious enough to click.
>
> What saved us was taking text from the personal appeals and putting it into
> the banner itself. These banners did very well. These new message-driven
> banners are what made us split the campaign in two -- because we knew we
> were going to develop a lot of new messages and not have time to translate
> them well. The campaign that started on the 27th of November ran only in
> five countries: Australia, Britain, Canada, New Zealand and the United
> States.
>
> At first, we had a short version of the new banner:
>
> http://en.wikipedia.org/wiki/Main_Page?banner=B12_1123_Smallinfo_fix
>
> We could have run that for 46 days (the length of last year's campaign) and
> probably made our goal. But it performed better the more information we put
> into it. Through a series of tests we became confident that, while greater
> banner height improved performance, that wasn't as big a factor as the
> additional information we put into the banner. We tested many new versions
> of messages in the banners and found many improvements. It looked like we
> might be able to have a 25 day-long fundraiser.
>
> We launched on the 27th. A few days into the campaign, we were still
> paranoid about the goal. We were afraid that maybe the new banners would
> burn out faster than the old ones did. Maybe they were good at getting
> donations faster, but maybe we were not increasing the overall pool of
> donors. We were constantly testing to boost performance. Out of curiosity,
> we tested making the banners stick to the top of the screen while the page
> scrolled. We knew that would be a dramatic step in a more annoying
> direction, but like I said, we were still worried about the goal and just
> wanted to know what our options were. The "sticky" banners did about 30%
> better for donations. So we decided to keep them and see if we could get
> the campaign done with in a very short time.
>
> After 8 days of having the banners up, we were able take banners down and
> only display them to people who had not seen them before (or rather,
> browsers and computers that had not seen them before). We've never been
> able to do this before and only had this feature fully developed several
> days into the campaign. Since we took banners down for everyone, we've
> mostly been displaying them only 1 or 2 times to people who've never seen
> them. Though yesterday we pushed that up to 10 because we're hoping to
> reach our US$25 million goal in the next few days.
>
> We also made the banners stop sticking after the first 8 days, and
> hopefully we'll never feel we have to use sticky banners again. In total,
> we had sticky banners up for 4 or 5 days.
>
> We hope that next week we'll be able to start a sort of "Thank you
> campaign." We will feature a thank you message, a video of Wikimedia
> editors talking about their experience, interviews and written messages
> from editors collected at Wikimania, and an invitation to all our readers
> to become editors. The purpose of this campaign is mainly to raise
> awareness among readers about how Wikimedia projects work and who is behind
> them. The purpose is also to take time to explicitly thank donors for
> helping us reach our goal so quickly this year. Thanking is a very
> important part of fundraising -- but we've always been so eager to take
> down banners that we've never taken enough time to thank donors in the
> past. This year we feel its ok since almost no one saw banners for more
> than 8 days.
>
> I know there will probably be a lot of detailed questions about income from
> different countries, comparisons to last year, to the chapters, etc... We
> can't answer those now because we're too busy trying to wrap up the
> campaign -- and also because a lot of transactions take time to settle.
> Checks flow in slowly. And accurate comparisons take time to prepare. We
> don't have truly accurate numbers until later in January.
>
> But the basic result we have is that all 5 countries we ran banners saw a
> huge increase in "donations per banner impression" from last year. I can
> also say that in our Nov 15th 24-hour "dress rehearsal" in which we ran an
> earlier version of the new banners (that didn't perform near as well as the
> one we eventually discovered) we also saw a very big increase in almost
> every country in the world. The exceptions were countries where we made
> some error (such as not turning banners on for half the day). Even in a
> country like Greece, where the economy is about as bad as it can get, we
> saw something like a 300% increase compared the the equivalent first day of
> the campaign last year. Therefore, we think we'll be able to keep our
> Spring campaign very low-impact.
>
> I hope this satisfies some of the curiosity about the campaign. Please ask
> questions, but please be understanding if we can't answer detailed
> questions while we're busy still running the campaign.
>
> Zack
>
>
> On Thu, Dec 13, 2012 at 12:01 PM, Itzik Edri <itzik [at] infra> wrote:
>
> > Indeed, a good start, that I already checked.
> >
> > The statistics shows only WMF data. Last year we had google docs file
> that
> > the chapters shared their numbers every day also.
> >
> > And there is no indication to which banners running and their
> performance,
> > things that I'll like to see.
> >
> > The fundraising team did some interesting changes this year, which will
> > be interesting to know their performances.
> >
> > Itzik.
> >
> > 2012/12/13 Till Mletzko <till.mletzko [at] wikimedia>
> >
> > > and: http://meta.wikimedia.org/wiki/Fundraising_2012
> > >
> > > Till
> > >
> > >
> > > Am 13.12.2012 15:12, schrieb Theo10011:
> > > > On Thu, Dec 13, 2012 at 7:34 PM, Itzik Edri <itzik [at] infra>
> wrote:
> > > >
> > > >> Hi,
> > > >>
> > > >> Could we get some updates (or that I missed them?) about how the
> > > >> fundraising goes? WMF and Chapters will be great. Since we only
> focus
> > on
> > > >> few countries this year, the discussions regarding it is very low,
> but
> > > it
> > > >> still very interesting to know how much we collect and how the
> banners
> > > >> works (and which one of them)
> > > >>
> > > > http://wikimediafoundation.org/wiki/Special:FundraiserStatistics -
> > is a
> > > > good start.
> > > >
> > > > -Theo
> > > > _______________________________________________
> > > > Wikimedia-l mailing list
> > > > Wikimedia-l [at] lists
> > > > Unsubscribe:
> https://lists.wikimedia.org/mailman/listinfo/wikimedia-l
> > >
> > >
> > > --
> > > Mit freundlichen Grüßen
> > >
> > > Till Mletzko
> > > Fundraiser
> > > -------------------------------------
> > > Wikimedia Fördergesellschaft
> > > Obentrautstr. 72
> > > 10963 Berlin
> > >
> > > Telefon 030 - 219 158 26 -19
> > > www.wikimedia.de
> > >
> > > Helfen Sie mit, dass WIKIPEDIA von der UNESCO als erstes digitales
> > > Weltkulturerbe anerkannt wird.
> > > Unterzeichnen Sie die Online-Petition unter
> > > https://wke.wikimedia.de/wke/Main_Page!
> > >
> > > Stellen Sie sich eine Welt vor, in der jeder Mensch freien Zugang zu
> der
> > > Gesamtheit des Wissens der Menschheit hat. Helfen Sie uns dabei!
> > > http://spenden.wikimedia.de/
> > >
> > > Gemeinnützige Wikimedia Fördergesellschaft mbH.
> > > Eingetragen beim Amtsgericht Berlin-Charlottenburg unter der Nummer
> > 130183
> > > B. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I
> > > Berlin, Steuernummer 27/603/54814.
> > >
> > >
> > > _______________________________________________
> > > Wikimedia-l mailing list
> > > Wikimedia-l [at] lists
> > > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l
> > >
> > _______________________________________________
> > Wikimedia-l mailing list
> > Wikimedia-l [at] lists
> > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l
> >
>
>
>
> --
> Zack Exley
> Chief Revenue Officer
> Wikimedia Foundation
> 415 506 9225
> _______________________________________________
> Wikimedia-l mailing list
> Wikimedia-l [at] lists
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l
>
_______________________________________________
Wikimedia-l mailing list
Wikimedia-l [at] lists
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l


jsalsman at gmail

Dec 15, 2012, 9:01 AM

Post #7 of 15 (902 views)
Permalink
Re: [Wikimedia-l] Fundraising updates? [In reply to]

Hi Zack,

Thanks very much for your updates:

> What saved us was taking text from the personal appeals and putting it into
> the banner itself. These banners did very well. These new message-driven
> banners are what made us split the campaign in two -- because we knew we
> were going to develop a lot of new messages and not have time to translate
> them well....

As you know I've been saying for years that the variance among the
volunteer-supplied messages, originally submitted in 2009 and hundreds
of which have not yet been tested (as far as I know), was large enough
to suggest that some messages would certainly outperform the
traditional banners and appeals. While it's refreshing to be
validated, as you might imagine I feel like Cassandra much of the time
for reasons that have nothing to do with the underlying mathematical
reasoning involved.

The last time I heard from you, you said that you intended to test the
untried messaging from 2009 with multivariate analysis. However,
http://meta.wikimedia.org/wiki/Fundraising_2012/We_Need_A_Breakthrough
shows only three very small-N multivariate tests, the last of which
was in October, and no recent testing.

Do you still intend to test the untried volunteer-submitted messages
with multivariate analysis? If so, when? Thank you.

Sincerely,
James Salsman

_______________________________________________
Wikimedia-l mailing list
Wikimedia-l [at] lists
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l


thomas.dalton at gmail

Dec 15, 2012, 9:16 AM

Post #8 of 15 (903 views)
Permalink
Re: [Wikimedia-l] Fundraising updates? [In reply to]

On Dec 14, 2012 3:37 PM, "Zack Exley" <zexley [at] wikimedia> wrote:
> Since we took banners down for everyone, we've
> mostly been displaying them only 1 or 2 times to people who've never seen
> them. Though yesterday we pushed that up to 10 because we're hoping to
> reach our US$25 million goal in the next few days.

Obviously it's early, but that change doesn't seem to have caused a
noticeable break from the existing trend on the fundraising graph. Have you
been gathering data on how banners perform on the first, second, third,
etc. time of viewing? Presumably the same cookies that let you stop banners
after a certain number of views would let you gather such data. If so, what
kind of an increase were you expecting to see?
_______________________________________________
Wikimedia-l mailing list
Wikimedia-l [at] lists
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l


zexley at wikimedia

Dec 17, 2012, 8:09 AM

Post #9 of 15 (900 views)
Permalink
Re: [Wikimedia-l] Fundraising updates? [In reply to]

On Sat, Dec 15, 2012 at 11:16 AM, Thomas Dalton <thomas.dalton [at] gmail>wrote:

> On Dec 14, 2012 3:37 PM, "Zack Exley" <zexley [at] wikimedia> wrote:
> > Since we took banners down for everyone, we've
> > mostly been displaying them only 1 or 2 times to people who've never seen
> > them. Though yesterday we pushed that up to 10 because we're hoping to
> > reach our US$25 million goal in the next few days.
>
> Obviously it's early, but that change doesn't seem to have caused a
> noticeable break from the existing trend on the fundraising graph.


The graph can be misleading because it counts email revenue too. Over
several days that we were just showing 2 banners to new people, we were
seeing a steady decline in revenue each day. We don't where that will
bottom out -- whether at $50K per day or $1000 per day. The answer to that
question will have a big impact on our fundraiser for next year.


> Have you
> been gathering data on how banners perform on the first, second, third,
> etc. time of viewing?


Yes, we have some data, but it's noisy and confusing. But the overall
picture is that when we turn on banners, a huge majority of donors are
giving upon their first banner view. Everyday, the distribution spreads
out. After a week of solid banners it was more like 50% of donations came
after the first banner view -- and most of the rest coming before the 10th
banner view.



> Presumably the same cookies that let you stop banners
> after a certain number of views would let you gather such data. If so, what
> kind of an increase were you expecting to see?
> _______________________________________________
> Wikimedia-l mailing list
> Wikimedia-l [at] lists
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l
>



--
Zack Exley
Chief Revenue Officer
Wikimedia Foundation
415 506 9225
_______________________________________________
Wikimedia-l mailing list
Wikimedia-l [at] lists
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l


zexley at wikimedia

Dec 17, 2012, 8:28 AM

Post #10 of 15 (901 views)
Permalink
Re: [Wikimedia-l] Fundraising updates? [In reply to]

On Sat, Dec 15, 2012 at 11:01 AM, James Salsman <jsalsman [at] gmail> wrote:

> Hi Zack,
>
> Thanks very much for your updates:
>
> > What saved us was taking text from the personal appeals and putting it
> into
> > the banner itself. These banners did very well. These new message-driven
> > banners are what made us split the campaign in two -- because we knew we
> > were going to develop a lot of new messages and not have time to
> translate
> > them well....
>
> As you know I've been saying for years that the variance among the
> volunteer-supplied messages, originally submitted in 2009 and hundreds
> of which have not yet been tested (as far as I know), was large enough
> to suggest that some messages would certainly outperform the
> traditional banners and appeals. While it's refreshing to be
> validated, as you might imagine I feel like Cassandra much of the time
> for reasons that have nothing to do with the underlying mathematical
> reasoning involved.
>
> The last time I heard from you, you said that you intended to test the
> untried messaging from 2009 with multivariate analysis. However,
> http://meta.wikimedia.org/wiki/Fundraising_2012/We_Need_A_Breakthrough
> shows only three very small-N multivariate tests, the last of which
> was in October, and no recent testing.
>
> Do you still intend to test the untried volunteer-submitted messages
> with multivariate analysis? If so, when? Thank you.
>
>
James -

We can only do big multivariate tests for banner click rates. But banner
click rates have very little to do with donations in our present context.

For example, the new banners have about 30% the click rate of the old ones,
but they make about 3 or 4 times as much money.

To determine how well a banner message does for donations, we usually need
a sample size between 500 and 5,000 donations per banner, depending on the
difference in performance between the banners. That takes from 30 minutes
to several hours to collect -- if we're only testing two banners at a time.

Regarding the banners suggested in past years: I've explained this before,
and will repeat: We tested tons of those banners. I think that we tested
virtually every different (serious) theme that was suggested. They all had
BOTH far lower click rates and even lower donation rates -- usually by
orders of magnitude. This was also true for the new short slogans that we
came up with ourselves on the fundraising team.

Now we're pretty clear on why: A short slogan isn't enough to get people
over all their questions about why they should support Wikipedia. More text
was needed. In our marketing-slogan-obsessed culture, the idea that we'd
have to present people with a long paragraph was very counterintuitive. We
didn't think of it on the fundraising team and none of the volunteers who
submitted suggestions thought of it either. Several marketing professionals
who contacted us with advice even told us to get rid of the appeal on then
landing page altogether because "people don't read!"

As it turns out, Wikipedia users DO like to read -- and want all the facts
before they donate.

Where we're at today, just to emphasize my previous point, is that with the
new banners, changes in messages effect donations totally independently of
click rate. And we typically need an hour or two -- or five -- to detect
even a 10%-%15 percent difference in message performance. That's why we're
not running big multivariate tests with tons of difference banners.

You'll be happy to know, though, that we are running multivariate tests
when we're able. For example, if we have a tweak to the landing pages that
we think is fairly independent of the banner effect, then we sometimes run
a multivariate test. Or if we have a design tweak (like color) that we're
confident will always effect click rate in the same direction as donations,
then we can combine that with message testing.


> Sincerely,
> James Salsman
>



--
Zack Exley
Chief Revenue Officer
Wikimedia Foundation
415 506 9225
_______________________________________________
Wikimedia-l mailing list
Wikimedia-l [at] lists
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l


jsalsman at gmail

Dec 17, 2012, 9:10 AM

Post #11 of 15 (903 views)
Permalink
Re: [Wikimedia-l] Fundraising updates? [In reply to]

Zack,

Thanks very much for helping me understand this.

> We can only do big multivariate tests for banner click rates.

The multivariate tests you ran in May and October list total
donations. What am I missing?

> For example, the new banners have about 30% the click rate of the old ones,
> but they make about 3 or 4 times as much money.

If you have any reason to believe that this is not because of the new
pull-down format, please let me know.

> To determine how well a banner message does for donations, we usually need a
> sample size between 500 and 5,000 donations per banner [requiring] from 30
> minutes to several hours to collect -- if we're only testing two banners at a time.

How about using a much quicker 90% confidence interval on a larger
number of tests, and then confirming the top, say ten performers, at
the 95% confidence level you've been using in subsequent tests?

>.... We tested tons of those banners. I think that we tested
> virtually every different (serious) theme that was suggested.

I'm sorry, but "tons" is not a number. You didn't even get a third of
them and a quick perusal shows that you missed hundreds of sincere
serious submissions.

> They all had BOTH far lower click rates and even lower donation rates

I believe using the word "far" is a factual mistake. I plotted a
sample distribution, and I know that the variance is more than enough
that the population would exceed the top of the sample. Moreover,
there were many discovered which came very close to the "Please read
an appeal from Jimmy" top performer in 2009 and 2010. Isn't that how
you discovered the "If everyone donated $x this fundraiser would be
over in y" which was the top performer for a time?

> usually by orders of magnitude.

That is technically true, but as they fit a lognormal distribution,
it's not surprising.

> Now we're pretty clear on why: A short slogan isn't enough to get people
> over all their questions about why they should support Wikipedia. More text
> was needed. In our marketing-slogan-obsessed culture, the idea that we'd
> have to present people with a long paragraph was very counterintuitive....
> As it turns out, Wikipedia users DO like to read -- and want all the facts
> before they donate.

I completely agree with this, but I note that the longer banners you
started with this month were comprised of three shorter statements,
one of which was an appeal statement.

Let me just cut to the chase and ask it this way: If you were to test
the remaining volunteer submissions in the appeal statement slot in
the existing top performing, fact-based, pull-down appeal,
(1) What would you need that you don't already have?
(2) How long would it take at the 90% confidence level?
(3) How can volunteers help make it easier and faster for you?

_______________________________________________
Wikimedia-l mailing list
Wikimedia-l [at] lists
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l


thomas.dalton at gmail

Dec 17, 2012, 9:23 AM

Post #12 of 15 (901 views)
Permalink
Re: [Wikimedia-l] Fundraising updates? [In reply to]

Have you considered doing some longer tests? Lasting a week, say. It would
enable you to do proper multivariate testing, including dependencies
between variables (which I don't think you have done any real tests of
yet). It would also let you test time dependence. Eg., does a particular
message work better in the morning than in the afternoon? (Different types
of people browse at different times, so it wouldn't surprise me) You could
also model banner fatigue properly, which could be very useful.
On Dec 17, 2012 4:28 PM, "Zack Exley" <zexley [at] wikimedia> wrote:

> On Sat, Dec 15, 2012 at 11:01 AM, James Salsman <jsalsman [at] gmail>
> wrote:
>
> > Hi Zack,
> >
> > Thanks very much for your updates:
> >
> > > What saved us was taking text from the personal appeals and putting it
> > into
> > > the banner itself. These banners did very well. These new
> message-driven
> > > banners are what made us split the campaign in two -- because we knew
> we
> > > were going to develop a lot of new messages and not have time to
> > translate
> > > them well....
> >
> > As you know I've been saying for years that the variance among the
> > volunteer-supplied messages, originally submitted in 2009 and hundreds
> > of which have not yet been tested (as far as I know), was large enough
> > to suggest that some messages would certainly outperform the
> > traditional banners and appeals. While it's refreshing to be
> > validated, as you might imagine I feel like Cassandra much of the time
> > for reasons that have nothing to do with the underlying mathematical
> > reasoning involved.
> >
> > The last time I heard from you, you said that you intended to test the
> > untried messaging from 2009 with multivariate analysis. However,
> > http://meta.wikimedia.org/wiki/Fundraising_2012/We_Need_A_Breakthrough
> > shows only three very small-N multivariate tests, the last of which
> > was in October, and no recent testing.
> >
> > Do you still intend to test the untried volunteer-submitted messages
> > with multivariate analysis? If so, when? Thank you.
> >
> >
> James -
>
> We can only do big multivariate tests for banner click rates. But banner
> click rates have very little to do with donations in our present context.
>
> For example, the new banners have about 30% the click rate of the old ones,
> but they make about 3 or 4 times as much money.
>
> To determine how well a banner message does for donations, we usually need
> a sample size between 500 and 5,000 donations per banner, depending on the
> difference in performance between the banners. That takes from 30 minutes
> to several hours to collect -- if we're only testing two banners at a time.
>
> Regarding the banners suggested in past years: I've explained this before,
> and will repeat: We tested tons of those banners. I think that we tested
> virtually every different (serious) theme that was suggested. They all had
> BOTH far lower click rates and even lower donation rates -- usually by
> orders of magnitude. This was also true for the new short slogans that we
> came up with ourselves on the fundraising team.
>
> Now we're pretty clear on why: A short slogan isn't enough to get people
> over all their questions about why they should support Wikipedia. More text
> was needed. In our marketing-slogan-obsessed culture, the idea that we'd
> have to present people with a long paragraph was very counterintuitive. We
> didn't think of it on the fundraising team and none of the volunteers who
> submitted suggestions thought of it either. Several marketing professionals
> who contacted us with advice even told us to get rid of the appeal on then
> landing page altogether because "people don't read!"
>
> As it turns out, Wikipedia users DO like to read -- and want all the facts
> before they donate.
>
> Where we're at today, just to emphasize my previous point, is that with the
> new banners, changes in messages effect donations totally independently of
> click rate. And we typically need an hour or two -- or five -- to detect
> even a 10%-%15 percent difference in message performance. That's why we're
> not running big multivariate tests with tons of difference banners.
>
> You'll be happy to know, though, that we are running multivariate tests
> when we're able. For example, if we have a tweak to the landing pages that
> we think is fairly independent of the banner effect, then we sometimes run
> a multivariate test. Or if we have a design tweak (like color) that we're
> confident will always effect click rate in the same direction as donations,
> then we can combine that with message testing.
>
>
> > Sincerely,
> > James Salsman
> >
>
>
>
> --
> Zack Exley
> Chief Revenue Officer
> Wikimedia Foundation
> 415 506 9225
> _______________________________________________
> Wikimedia-l mailing list
> Wikimedia-l [at] lists
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l
>
_______________________________________________
Wikimedia-l mailing list
Wikimedia-l [at] lists
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l


meta.sj at gmail

Dec 17, 2012, 9:28 AM

Post #13 of 15 (904 views)
Permalink
Re: [Wikimedia-l] Fundraising updates? [In reply to]

Thomas writes:
> You could also model banner fatigue properly, which could be very useful.

Yes, a detailed model of banner fatigue would be fascinating.

It's certainly something studied by many groups in different contexts;
ideally we'd learn from published analysis, and then see deviations from
the norm in our own context. It's quite likely that the context changes
between donation appeals and other messages; understanding this better
would also help us rotate global sitenotices more effectively.

Zack - thank you for sharing so much detail about the process.
James - thank you for your nuanced statistical comments; something we could
use more of.

SJ


On Mon, Dec 17, 2012 at 12:23 PM, Thomas Dalton <thomas.dalton [at] gmail>wrote:

> Have you considered doing some longer tests? Lasting a week, say. It would
> enable you to do proper multivariate testing, including dependencies
> between variables (which I don't think you have done any real tests of
> yet). It would also let you test time dependence. Eg., does a particular
> message work better in the morning than in the afternoon? (Different types
> of people browse at different times, so it wouldn't surprise me) You could
> also model banner fatigue properly, which could be very useful.
> On Dec 17, 2012 4:28 PM, "Zack Exley" <zexley [at] wikimedia> wrote:
>
> > On Sat, Dec 15, 2012 at 11:01 AM, James Salsman <jsalsman [at] gmail>
> > wrote:
> >
> > > Hi Zack,
> > >
> > > Thanks very much for your updates:
> > >
> > > > What saved us was taking text from the personal appeals and putting
> it
> > > into
> > > > the banner itself. These banners did very well. These new
> > message-driven
> > > > banners are what made us split the campaign in two -- because we knew
> > we
> > > > were going to develop a lot of new messages and not have time to
> > > translate
> > > > them well....
> > >
> > > As you know I've been saying for years that the variance among the
> > > volunteer-supplied messages, originally submitted in 2009 and hundreds
> > > of which have not yet been tested (as far as I know), was large enough
> > > to suggest that some messages would certainly outperform the
> > > traditional banners and appeals. While it's refreshing to be
> > > validated, as you might imagine I feel like Cassandra much of the time
> > > for reasons that have nothing to do with the underlying mathematical
> > > reasoning involved.
> > >
> > > The last time I heard from you, you said that you intended to test the
> > > untried messaging from 2009 with multivariate analysis. However,
> > > http://meta.wikimedia.org/wiki/Fundraising_2012/We_Need_A_Breakthrough
> > > shows only three very small-N multivariate tests, the last of which
> > > was in October, and no recent testing.
> > >
> > > Do you still intend to test the untried volunteer-submitted messages
> > > with multivariate analysis? If so, when? Thank you.
> > >
> > >
> > James -
> >
> > We can only do big multivariate tests for banner click rates. But banner
> > click rates have very little to do with donations in our present context.
> >
> > For example, the new banners have about 30% the click rate of the old
> ones,
> > but they make about 3 or 4 times as much money.
> >
> > To determine how well a banner message does for donations, we usually
> need
> > a sample size between 500 and 5,000 donations per banner, depending on
> the
> > difference in performance between the banners. That takes from 30 minutes
> > to several hours to collect -- if we're only testing two banners at a
> time.
> >
> > Regarding the banners suggested in past years: I've explained this
> before,
> > and will repeat: We tested tons of those banners. I think that we tested
> > virtually every different (serious) theme that was suggested. They all
> had
> > BOTH far lower click rates and even lower donation rates -- usually by
> > orders of magnitude. This was also true for the new short slogans that we
> > came up with ourselves on the fundraising team.
> >
> > Now we're pretty clear on why: A short slogan isn't enough to get people
> > over all their questions about why they should support Wikipedia. More
> text
> > was needed. In our marketing-slogan-obsessed culture, the idea that we'd
> > have to present people with a long paragraph was very counterintuitive.
> We
> > didn't think of it on the fundraising team and none of the volunteers who
> > submitted suggestions thought of it either. Several marketing
> professionals
> > who contacted us with advice even told us to get rid of the appeal on
> then
> > landing page altogether because "people don't read!"
> >
> > As it turns out, Wikipedia users DO like to read -- and want all the
> facts
> > before they donate.
> >
> > Where we're at today, just to emphasize my previous point, is that with
> the
> > new banners, changes in messages effect donations totally independently
> of
> > click rate. And we typically need an hour or two -- or five -- to detect
> > even a 10%-%15 percent difference in message performance. That's why
> we're
> > not running big multivariate tests with tons of difference banners.
> >
> > You'll be happy to know, though, that we are running multivariate tests
> > when we're able. For example, if we have a tweak to the landing pages
> that
> > we think is fairly independent of the banner effect, then we sometimes
> run
> > a multivariate test. Or if we have a design tweak (like color) that we're
> > confident will always effect click rate in the same direction as
> donations,
> > then we can combine that with message testing.
> >
> >
> > > Sincerely,
> > > James Salsman
> > >
> >
> >
> >
> > --
> > Zack Exley
> > Chief Revenue Officer
> > Wikimedia Foundation
> > 415 506 9225
> > _______________________________________________
> > Wikimedia-l mailing list
> > Wikimedia-l [at] lists
> > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l
> >
> _______________________________________________
> Wikimedia-l mailing list
> Wikimedia-l [at] lists
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l
>



--
Samuel Klein @metasj w:user:sj +1 617 529 4266
_______________________________________________
Wikimedia-l mailing list
Wikimedia-l [at] lists
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l


thomas.dalton at gmail

Dec 17, 2012, 10:23 AM

Post #14 of 15 (901 views)
Permalink
Re: [Wikimedia-l] Fundraising updates? [In reply to]

On 17 December 2012 17:28, Samuel Klein <meta.sj [at] gmail> wrote:
> Thomas writes:
>> You could also model banner fatigue properly, which could be very useful.
>
> Yes, a detailed model of banner fatigue would be fascinating.
>
> It's certainly something studied by many groups in different contexts;
> ideally we'd learn from published analysis, and then see deviations from
> the norm in our own context. It's quite likely that the context changes
> between donation appeals and other messages; understanding this better
> would also help us rotate global sitenotices more effectively.

Published analyses would certainly be interesting, but it wouldn't
surprise me if they were completely non-applicable for us. There is
really nothing else like our fundraiser - nobody else uses their own
top 5 website for their fundraising, since no other non-profits have a
top 5 website!

_______________________________________________
Wikimedia-l mailing list
Wikimedia-l [at] lists
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l


mhernandez at wikimedia

Dec 17, 2012, 10:30 AM

Post #15 of 15 (903 views)
Permalink
Re: [Wikimedia-l] Fundraising updates? [In reply to]

Here's the link to the chapter daily reporting numbers. Please keep in
mind that these numbers are still preliminary. The numbers will still
change as donations are still coming in and some donations take a while to
settle (checks, bank transfer, etc).

https://docs.google.com/a/wikimedia.org/spreadsheet/ccc?key=0Alem5893XFmUdDNITmtPTlpmcVlETnphOUEwdFlCUHc#gid=0

Megan

On Mon, Dec 17, 2012 at 9:28 AM, Samuel Klein <meta.sj [at] gmail> wrote:

> Thomas writes:
> > You could also model banner fatigue properly, which could be very useful.
>
> Yes, a detailed model of banner fatigue would be fascinating.
>
> It's certainly something studied by many groups in different contexts;
> ideally we'd learn from published analysis, and then see deviations from
> the norm in our own context. It's quite likely that the context changes
> between donation appeals and other messages; understanding this better
> would also help us rotate global sitenotices more effectively.
>
> Zack - thank you for sharing so much detail about the process.
> James - thank you for your nuanced statistical comments; something we could
> use more of.
>
> SJ
>
>
> On Mon, Dec 17, 2012 at 12:23 PM, Thomas Dalton <thomas.dalton [at] gmail
> >wrote:
>
> > Have you considered doing some longer tests? Lasting a week, say. It
> would
> > enable you to do proper multivariate testing, including dependencies
> > between variables (which I don't think you have done any real tests of
> > yet). It would also let you test time dependence. Eg., does a particular
> > message work better in the morning than in the afternoon? (Different
> types
> > of people browse at different times, so it wouldn't surprise me) You
> could
> > also model banner fatigue properly, which could be very useful.
> > On Dec 17, 2012 4:28 PM, "Zack Exley" <zexley [at] wikimedia> wrote:
> >
> > > On Sat, Dec 15, 2012 at 11:01 AM, James Salsman <jsalsman [at] gmail>
> > > wrote:
> > >
> > > > Hi Zack,
> > > >
> > > > Thanks very much for your updates:
> > > >
> > > > > What saved us was taking text from the personal appeals and putting
> > it
> > > > into
> > > > > the banner itself. These banners did very well. These new
> > > message-driven
> > > > > banners are what made us split the campaign in two -- because we
> knew
> > > we
> > > > > were going to develop a lot of new messages and not have time to
> > > > translate
> > > > > them well....
> > > >
> > > > As you know I've been saying for years that the variance among the
> > > > volunteer-supplied messages, originally submitted in 2009 and
> hundreds
> > > > of which have not yet been tested (as far as I know), was large
> enough
> > > > to suggest that some messages would certainly outperform the
> > > > traditional banners and appeals. While it's refreshing to be
> > > > validated, as you might imagine I feel like Cassandra much of the
> time
> > > > for reasons that have nothing to do with the underlying mathematical
> > > > reasoning involved.
> > > >
> > > > The last time I heard from you, you said that you intended to test
> the
> > > > untried messaging from 2009 with multivariate analysis. However,
> > > >
> http://meta.wikimedia.org/wiki/Fundraising_2012/We_Need_A_Breakthrough
> > > > shows only three very small-N multivariate tests, the last of which
> > > > was in October, and no recent testing.
> > > >
> > > > Do you still intend to test the untried volunteer-submitted messages
> > > > with multivariate analysis? If so, when? Thank you.
> > > >
> > > >
> > > James -
> > >
> > > We can only do big multivariate tests for banner click rates. But
> banner
> > > click rates have very little to do with donations in our present
> context.
> > >
> > > For example, the new banners have about 30% the click rate of the old
> > ones,
> > > but they make about 3 or 4 times as much money.
> > >
> > > To determine how well a banner message does for donations, we usually
> > need
> > > a sample size between 500 and 5,000 donations per banner, depending on
> > the
> > > difference in performance between the banners. That takes from 30
> minutes
> > > to several hours to collect -- if we're only testing two banners at a
> > time.
> > >
> > > Regarding the banners suggested in past years: I've explained this
> > before,
> > > and will repeat: We tested tons of those banners. I think that we
> tested
> > > virtually every different (serious) theme that was suggested. They all
> > had
> > > BOTH far lower click rates and even lower donation rates -- usually by
> > > orders of magnitude. This was also true for the new short slogans that
> we
> > > came up with ourselves on the fundraising team.
> > >
> > > Now we're pretty clear on why: A short slogan isn't enough to get
> people
> > > over all their questions about why they should support Wikipedia. More
> > text
> > > was needed. In our marketing-slogan-obsessed culture, the idea that
> we'd
> > > have to present people with a long paragraph was very counterintuitive.
> > We
> > > didn't think of it on the fundraising team and none of the volunteers
> who
> > > submitted suggestions thought of it either. Several marketing
> > professionals
> > > who contacted us with advice even told us to get rid of the appeal on
> > then
> > > landing page altogether because "people don't read!"
> > >
> > > As it turns out, Wikipedia users DO like to read -- and want all the
> > facts
> > > before they donate.
> > >
> > > Where we're at today, just to emphasize my previous point, is that with
> > the
> > > new banners, changes in messages effect donations totally independently
> > of
> > > click rate. And we typically need an hour or two -- or five -- to
> detect
> > > even a 10%-%15 percent difference in message performance. That's why
> > we're
> > > not running big multivariate tests with tons of difference banners.
> > >
> > > You'll be happy to know, though, that we are running multivariate tests
> > > when we're able. For example, if we have a tweak to the landing pages
> > that
> > > we think is fairly independent of the banner effect, then we sometimes
> > run
> > > a multivariate test. Or if we have a design tweak (like color) that
> we're
> > > confident will always effect click rate in the same direction as
> > donations,
> > > then we can combine that with message testing.
> > >
> > >
> > > > Sincerely,
> > > > James Salsman
> > > >
> > >
> > >
> > >
> > > --
> > > Zack Exley
> > > Chief Revenue Officer
> > > Wikimedia Foundation
> > > 415 506 9225
> > > _______________________________________________
> > > Wikimedia-l mailing list
> > > Wikimedia-l [at] lists
> > > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l
> > >
> > _______________________________________________
> > Wikimedia-l mailing list
> > Wikimedia-l [at] lists
> > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l
> >
>
>
>
> --
> Samuel Klein @metasj w:user:sj +1 617 529 4266
> _______________________________________________
> Wikimedia-l mailing list
> Wikimedia-l [at] lists
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l
>



--

Megan Hernandez

Head of Annual Fundraiser
Wikimedia Foundation
_______________________________________________
Wikimedia-l mailing list
Wikimedia-l [at] lists
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l

Wikipedia foundation RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.