Mailing List Archive: Wikipedia: Wikitech

Production machine config

Index | Next | Previous | View Threaded

william at scissor

Nov 24, 2009, 10:52 AM

Post #1 of 9 (1322 views)
Permalink

Production machine config

Hi, all.

The FlaggedRevs lab site mysteriously got better, which makes me
nervous. To counteract that, I'd like to put together some basic smoke
and load tests. My plan is to do that on Amazon's cloud infrastructure.
A few questions:

1. Does anybody have an AMI (basically, a virtual machine config)
that's already like WMF production servers?
2. If not, is there something like a Puppet config that I can use to
help build the AMI?
3. What's the best way for me to find out details on production
server configs? I see the info in Special:Version, but I'm
wondering a) where to find those packages, and b) if there are
other requirements (e.g., particular kernel versions, supporting
software, etc)?

Once I get the AMI working, I'll post details here so others can easily
spin up pseudo-production boxes.

William
_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

mdale at wikimedia

Nov 24, 2009, 11:46 AM

Post #2 of 9 (1280 views)
Permalink

Re: Production machine config [In reply to]

In terms of configuration this lists most of whats public:
http://noc.wikimedia.org/conf/

There are no virtual machine config that I am aware of.

To check out what is in production use the wmf-deployment branch:
http://svn.wikimedia.org/svnroot/mediawiki/branches/wmf-deployment/

peace,
michael

William Pietri wrote:
> Hi, all.
>
> The FlaggedRevs lab site mysteriously got better, which makes me
> nervous. To counteract that, I'd like to put together some basic smoke
> and load tests. My plan is to do that on Amazon's cloud infrastructure.
> A few questions:
>
> 1. Does anybody have an AMI (basically, a virtual machine config)
> that's already like WMF production servers?
> 2. If not, is there something like a Puppet config that I can use to
> help build the AMI?
> 3. What's the best way for me to find out details on production
> server configs? I see the info in Special:Version, but I'm
> wondering a) where to find those packages, and b) if there are
> other requirements (e.g., particular kernel versions, supporting
> software, etc)?
>
>
> Once I get the AMI working, I'll post details here so others can easily
> spin up pseudo-production boxes.
>
> William
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-l [at] lists
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>

_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

tstarling at wikimedia

Nov 24, 2009, 4:00 PM

Post #3 of 9 (1276 views)
Permalink

Re: Production machine config [In reply to]

William Pietri wrote:
> Hi, all.
>
> The FlaggedRevs lab site mysteriously got better, which makes me
> nervous. To counteract that, I'd like to put together some basic smoke
> and load tests. My plan is to do that on Amazon's cloud infrastructure.
> A few questions:
>
> 1. Does anybody have an AMI (basically, a virtual machine config)
> that's already like WMF production servers?
> 2. If not, is there something like a Puppet config that I can use to
> help build the AMI?
> 3. What's the best way for me to find out details on production
> server configs? I see the info in Special:Version, but I'm
> wondering a) where to find those packages, and b) if there are
> other requirements (e.g., particular kernel versions, supporting
> software, etc)?
>
>
> Once I get the AMI working, I'll post details here so others can easily
> spin up pseudo-production boxes.

I can only assume by this post that you're completely out of the loop
on FlaggedRevs and have received no support from Wikimedia tech staff.

There's no need for performance testing since we've been running it on
our second-biggest wiki for 18 months and the hardware requirements
there can be extrapolated to the English Wikipedia.

Domas tells me that that extrapolation leads to some concern that we
might approach our hardware limits on the s1 cluster. The sums need to
be done and the hardware ordered, if necessary.

I think we need to discuss this off list, preferably in real time with
Aaron and Domas.

-- Tim Starling

_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

william at scissor

Nov 24, 2009, 5:07 PM

Post #4 of 9 (1274 views)
Permalink

Re: Production machine config [In reply to]

Michael Dale wrote:
> In terms of configuration this lists most of whats public:
> http://noc.wikimedia.org/conf/
>
> There are no virtual machine config that I am aware of.
>

Thanks, Michael. That's great to know.

I'll see if I can rummage around that site, as I figure a lot of the
stuff I need must be there.

Thanks,

William

_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

william at scissor

Nov 24, 2009, 5:08 PM

Post #5 of 9 (1275 views)
Permalink

Re: Production machine config [In reply to]

Tim Starling wrote:
> I can only assume by this post that you're completely out of the loop
> on FlaggedRevs and have received no support from Wikimedia tech staff.
>

Naturally, I would appreciate any assistance at getting more in the
loop. I've certainly tried, but you (or anybody) should feel free to
send me suggestions off list. However, Erik and Aaron have been very
helpful, so any fault here would surely be mine.

> There's no need for performance testing since we've been running it on
> our second-biggest wiki for 18 months and the hardware requirements
> there can be extrapolated to the English Wikipedia.
>

That's a reasonable notion. However, given that the labs site was
basically broken for a week for no reason that's apparent yet, an
alternative notion is that there is something about the labs code or the
labs config that blows up, possibly due to use.

If you have an easier way of proving that the new code and config works
just as well as the old code and German config do, I'm all ears.
Automated smoke and tests were just the best solution that I came up
with on my own.

> Domas tells me that that extrapolation leads to some concern that we
> might approach our hardware limits on the s1 cluster. The sums need to
> be done and the hardware ordered, if necessary.
>
> I think we need to discuss this off list, preferably in real time with
> Aaron and Domas.
>

I would love that. I'll contact you off list about arranging that.

William

_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

midom.lists at gmail

Nov 24, 2009, 5:09 PM

Post #6 of 9 (1276 views)
Permalink

Re: Production machine config [In reply to]

> Domas tells me that that extrapolation leads to some concern that we
> might approach our hardware limits on the s1 cluster. The sums need to
> be done and the hardware ordered, if necessary.

flaggedrevs adds 30% database size tax on dewiki.

That would mean that we'd need new-class machines (1TB of fast raid10
storage for upcoming year, may need even bigger storage eventually, if
we continue to double database size yearly, 64-128G of RAM) for enwiki
cluster.
I guess that is much cheaper now, than few years ago. Although, such
data expansion postpones our ability to use enterprise flash at
reasonable price level :)

Alternative is of course analyzing where is all that FR bloat =)

Domas

_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

william at scissor

Dec 9, 2009, 8:53 AM

Post #7 of 9 (1060 views)
Permalink

Re: Production machine config [In reply to]

Somone recently suggested this to me:

> In terms of configuration this lists most of whats public:
> http://noc.wikimedia.org/conf/
>

I just went back to look at these again, and I see that link is now a
404. I take it the configs have moved. Does anybody know the best place
to get that now?

Also, when new boxes are added, how do we build them?

Thanks,

William

_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

roan.kattouw at gmail

Dec 9, 2009, 9:02 AM

Post #8 of 9 (1052 views)
Permalink

Re: Production machine config [In reply to]

2009/12/9 William Pietri <william [at] scissor>:
>
> Somone recently suggested this to me:
>
>> In terms of configuration this lists most of whats public:
>> http://noc.wikimedia.org/conf/
>>
>
> I just went back to look at these again, and I see that link is now a
> 404. I take it the configs have moved. Does anybody know the best place
> to get that now?
>
Right now, noc is broken and the config files are nowhere. I've been
prodding sysadmins to fix this, will continue doing this more
aggressively.

Roan Kattouw (Catrope)

_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

roan.kattouw at gmail

Dec 10, 2009, 9:07 AM

Post #9 of 9 (1057 views)
Permalink

Re: Production machine config [In reply to]

2009/12/9 Roan Kattouw <roan.kattouw [at] gmail>:
> 2009/12/9 William Pietri <william [at] scissor>:
>>
>> Somone recently suggested this to me:
>>
>>> In terms of configuration this lists most of whats public:
>>> http://noc.wikimedia.org/conf/
>>>
>>
>> I just went back to look at these again, and I see that link is now a
>> 404. I take it the configs have moved. Does anybody know the best place
>> to get that now?
>>
> Right now, noc is broken and the config files are nowhere. I've been
> prodding sysadmins to fix this, will continue doing this more
> aggressively.
>
Fred fixed this yesterday.

Roan Kattouw (Catrope)

_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads