otto at wikimedia
Apr 27, 2012, 1:53 PM
Post #8 of 10
Here's the migrations library I wrote. :)
On Apr 26, 2012, at 11:30 AM, Andrew Otto wrote:
> I once wrote a pretty decent schema migration tool that fits most if not all of these requirements. It was built for the Kohana PHP framework, but a lot of it is pretty independent of that. If someone ends up working on this I'd love to help and maybe share some code and ideas.
> -Andrew Otto
> On Apr 25, 2012, at 12:58 PM, Asher Feldman wrote:
>> I am generally in favor of all of this and in the meeting that proceeded
>> Rob's email, proposed that we develop a new schema migration tool for
>> mediawiki along similar lines. Such a beast would have to work in all
>> deployment cases without modifications (stock single wiki installs and at
>> wmf with many wikis across multiple masters with tiered replication), be
>> idempotent when run across many databases, track version and state per
>> migration, and include up/down steps in every migration.
>> There are opensource php migration tools modeled along those used by the
>> popular ruby and python frameworks. I deployed
>> https://github.com/davejkiger/mysql-php-migrations at kiva.org a couple
>> years ago where it worked well and is still in use. Nothing will meet our
>> needs off the shelf though. A good project could at best be forked into
>> mediawiki with modifications if the license allows it, or more likely serve
>> as a model for our own development.
>> On Tue, Apr 24, 2012 at 11:27 PM, Faidon Liambotis <faidon [at] wikimedia>wrote:
>>> In other systems I've worked before, such problems have been solved by
>>> each schema-breaking version providing schema *and data* migrations for
>>> both forward *and backward* steps.
>>> This means that the upgrade transition mechanism knew how to add or
>>> remove columns or tables *and* how to fill them with data (say by
>>> concatenating two columns of the old schema). The same program would
>>> also take care to do the exact opposite steps in a the migration's
>>> backward method, in case a rollback was needed.
>> Down migrations aid development; I find them most useful as documentation
>> of prior state, making a migration readable as a diff. They generally
>> aren't useful in production environments at scale though, which developers
>> removed from the workings of production need to be aware of. Even with
>> transparent execution of migrations, the time it takes to apply changes
>> will nearly always be far outside of the acceptable bounds of an emergency
>> response necessitating a code rollback. So except in obvious cases such as
>> adding new tables, care is needed to keep forward migration backwards
>> compatible with code as much as possible.
>> The migrations themselves can be kept in the source tree, perhaps even
>>> versioned and with the schema version kept in the database, so that both
>>> us and external users can at any time forward their database to any
>>> later version, automagically.
>> Yep. That we have to pull in migrations from both core and many extensions
>> (many projects, one migration system) while also running different sets of
>> extensions across different wikis intermingling on the same database
>> servers adds some complexity but we should get there.
>> Wikitech-l mailing list
>> Wikitech-l [at] lists
Wikitech-l mailing list
Wikitech-l [at] lists