Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Wikipedia: Wikitech

mariadb 5.5 in production for english wikipedia

 

 

Wikipedia wikitech RSS feed   Index | Next | Previous | View Threaded


afeldman at wikimedia

Dec 11, 2012, 4:10 PM

Post #1 of 5 (429 views)
Permalink
mariadb 5.5 in production for english wikipedia

Hi,

This afternoon, I migrated one of the main production English Wikipedia
slaves, db59, to MariaDB 5.5.28. We've previously been testing 5.5.27 on
the primary research slave, and I've been testing the current build for the
last few days on a slave in eqiad. All has looked good, and I spent the
last few days adapting our monitoring and metrics collection tools to the
new version, and building binary packages that meet our needs.

A main gotcha in major version upgrades is performance regressions due to
changes in query plans. I've seen no sign of this, and my initial
assessment is that performance for our workload is on par with or slightly
improved over the 5.1 facebook patchset.

Taking the times of 100% of all queries over regular sample windows, the
average query time across all enwiki slave queries is about 8% faster with
MariaDB vs. our production build of 5.1-fb. Some queries types are 10-15%
faster, some are 3% slower, and nothing looks aberrant beyond those
bounds. Overall throughput as measured by qps has generally been improved
by 2-10%. I wouldn't draw any conclusions from this data yet, more is
needed to filter out noise, but it's positive.

MariaDB has some nice performance improvements that our workload doesn't
really hit (better query optimization and index usage during joins, much
better sub query support) but there are also some things, such as full
utilization of the primary key embedded on the right of every secondary
index that we can take advantage of (and improve our schema around) once
prod is fully upgraded, hopefully over the next 1-2 months.

The main goal of migrating to MariaDB is not performance driven. More so,
I think it's in WMF's and the open source communities interest to coalesce
around the MariaDB Foundation as the best route to ensuring a truly open
and well supported future for mysql derived database technology.
Performance gains along the way are icing on the cake.

-Asher
_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


hashar+wmf at free

Dec 12, 2012, 6:45 AM

Post #2 of 5 (406 views)
Permalink
Re: mariadb 5.5 in production for english wikipedia [In reply to]

Le 12/12/12 01:10, Asher Feldman a écrit :
> This afternoon, I migrated one of the main production English Wikipedia
> slaves, db59, to MariaDB 5.5.28.

Congratulations :-)

Out of curiosity, have you looked at Drizzle too?

--
Antoine "hashar" Musso


_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


afeldman at wikimedia

Dec 12, 2012, 9:04 AM

Post #3 of 5 (405 views)
Permalink
Re: mariadb 5.5 in production for english wikipedia [In reply to]

On Wednesday, December 12, 2012, David Gerard wrote:

> On 12 December 2012 15:32, Thomas Fellows <thomas.fellows [at] gmail<javascript:;>>
> wrote:
>
> > This is awesome! Is there any write-up of the migration process floating
> > around?
>
>
> +1
>
> In fact, this would be a nice thing to put on the WMF blog. It'll
> certainly get a lot of linkage and reporting around the geekosphere.


A detailed blog post is definitely my intent, I'm just waiting until
at least one major project is 100% on mariadb and I have more data and
hence confidence in drawn conclusions. I don't think that's far off at all,
potentially later this month. If that occurs and goes well, the eqiad data
center migration in late January may also be a migration to all mariadb.
_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


daniel at nadir-seen-fire

Dec 13, 2012, 2:18 AM

Post #4 of 5 (398 views)
Permalink
Re: mariadb 5.5 in production for english wikipedia [In reply to]

On Wed, 12 Dec 2012 06:45:24 -0800, Antoine Musso <hashar+wmf [at] free>
wrote:

> Le 12/12/12 01:10, Asher Feldman a écrit :
>> This afternoon, I migrated one of the main production English Wikipedia
>> slaves, db59, to MariaDB 5.5.28.
>
> Congratulations :-)
>
> Out of curiosity, have you looked at Drizzle too?

IIRC Drizzle uses a completely different client and we'd need to write a
new database class for it.

...something I've wished I could do for a long time.

--
~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://danielfriesen.name/]


_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


afeldman at wikimedia

Dec 13, 2012, 3:01 PM

Post #5 of 5 (395 views)
Permalink
Re: mariadb 5.5 in production for english wikipedia [In reply to]

On Wed, Dec 12, 2012 at 6:45 AM, Antoine Musso <hashar+wmf [at] free> wrote:

> Le 12/12/12 01:10, Asher Feldman a écrit :
> > This afternoon, I migrated one of the main production English Wikipedia
> > slaves, db59, to MariaDB 5.5.28.
>
> Congratulations :-)
>
> Out of curiosity, have you looked at Drizzle too?
>

I've spoken with Drizzle developers at OSCON in the past. I haven't seen
anyone advocate it as a production quality database though, and it doesn't
currently seem to have a lot of development momentum behind it, with Brian
Aker no longer putting in a lot of time. Lots of interesting ideas and
features, especially around replication, but they make it incompatible with
MySQL in enough ways where a gradual migration wouldn't be practical even
if it was otherwise desirable.

-A
_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Wikipedia wikitech RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.