Mailing List Archive: Catalyst: Users

Deployment in practice

Index | Next | Previous | View Threaded

orasnita at gmail

Oct 9, 2009, 6:16 AM

Post #1 of 17 (2310 views)
Permalink

Deployment in practice

Hi,

I've recently started to use Mercurial for revision control and the pages
regarding the deployment of a Catalyst app from the Catalyst wiki also help
me very much, but I still don't know what would be the best method for
uploading the files to the server.

How do you do it?
Archive the entire app an dupload to the server then change the permissions
of the files and folders there?
Or upload the modified files one by one?
Or generate a tarball on each revision and unarchive and make, make test it
on the server?
Or something else?

Thank you.

Octavian

_______________________________________________
List: Catalyst [at] lists
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst [at] lists/
Dev site: http://dev.catalyst.perl.org/

tobias.kremer at gmail

Oct 9, 2009, 7:55 AM

Post #2 of 17 (2241 views)
Permalink

Re: Deployment in practice [In reply to]

2009/10/9 Octavian Râşniţă <orasnita [at] gmail>:
> I've recently started to use Mercurial for revision control and the pages
> regarding the deployment of a Catalyst app from the Catalyst wiki also help
> me very much, but I still don't know what would be the best method for
> uploading the files to the server.
> How do you do it?

We do a checkout of the latest version on our staging server, test
everything and then rsync it to the webservers. Works like a charm.

--Tobias

_______________________________________________
List: Catalyst [at] lists
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst [at] lists/
Dev site: http://dev.catalyst.perl.org/

jshirley at gmail

Oct 9, 2009, 9:11 AM

Post #3 of 17 (2239 views)
Permalink

Re: Deployment in practice [In reply to]

On Fri, Oct 9, 2009 at 7:55 AM, Tobias Kremer <tobias.kremer [at] gmail>wrote:

> 2009/10/9 Octavian Râşniţă <orasnita [at] gmail>:
> > I've recently started to use Mercurial for revision control and the pages
> > regarding the deployment of a Catalyst app from the Catalyst wiki also
> help
> > me very much, but I still don't know what would be the best method for
> > uploading the files to the server.
> > How do you do it?
>
> We do a checkout of the latest version on our staging server, test
> everything and then rsync it to the webservers. Works like a charm.
>
>
I have deployment branches, and then push from there. It's git, but I'd
assume you can do the same in hg (though I've never used it)

See: http://blog.woobling.org/2009/05/deployment-branches-with-git.html

Then, you can use this post-hook script on the production machine and simply
push to that repo:
http://github.com/perigrin/soho-web/tree/master/hooks/post-update

This works very well, and you can have that be on a deployment
server/location and update the working tree of the other servers in the
node.

-J

orasnita at gmail

Oct 9, 2009, 9:53 AM

Post #4 of 17 (2237 views)
Permalink

Re: Deployment in practice [In reply to]

From: "J. Shirley" <jshirley [at] gmail>

On Fri, Oct 9, 2009 at 7:55 AM, Tobias Kremer
<tobias.kremer [at] gmail>wrote:

> 2009/10/9 Octavian Râşniţă <orasnita [at] gmail>:
> > I've recently started to use Mercurial for revision control and the
> > pages
> > regarding the deployment of a Catalyst app from the Catalyst wiki also
> help
> > me very much, but I still don't know what would be the best method for
> > uploading the files to the server.
> > How do you do it?
>
> We do a checkout of the latest version on our staging server, test
> everything and then rsync it to the webservers. Works like a charm.
>
>
I have deployment branches, and then push from there. It's git, but I'd
assume you can do the same in hg (though I've never used it)

Thank you Tobias and J.

For the moment I think I will use Tobias' way, because I use Mercurial only
for a few days and I don't know if it supports hooks made in bash.

I know that git is made in C and some perl scripts, that it is faster, much
advanced than Mercurial (which is made in Python), but unfortunately it
doesn't work well under Windows, and I do the development under Windows and
the testing and production use under Linux...

Octavian

_______________________________________________
List: Catalyst [at] lists
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst [at] lists/
Dev site: http://dev.catalyst.perl.org/

jgoulah at gmail

Oct 9, 2009, 12:04 PM

Post #5 of 17 (2239 views)
Permalink

Re: Deployment in practice [In reply to]

2009/10/9 Octavian Râşniţă <orasnita [at] gmail>:
> Hi,
>
> I've recently started to use Mercurial for revision control and the pages
> regarding the deployment of a Catalyst app from the Catalyst wiki also help
> me very much, but I still don't know what would be the best method for
> uploading the files to the server.
>
> How do you do it?
> Archive the entire app an dupload to the server then change the permissions
> of the files and folders there?
> Or upload the modified files one by one?
> Or generate a tarball on each revision and unarchive and make, make test it
> on the server?
> Or something else?
>

I like to have a revision per code push and rsync it out. What I
don't recommend is updating code directly from a repo on the server,
it will get you eventually.

http://blog.johngoulah.com/2009/03/code-deployment-techniques/

_______________________________________________
List: Catalyst [at] lists
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst [at] lists/
Dev site: http://dev.catalyst.perl.org/

amesdaq at websense

Oct 9, 2009, 12:50 PM

Post #6 of 17 (2240 views)
Permalink

RE: Deployment in practice [In reply to]

I will second this recommendation or something similar to this. It’s a lot easier when you have two branches that serve very specific purposes. One branch is production code and that can be sync’d every few minutes with whatever source control system and with a script similar to the one J Shirley posted a link to the webserver can sync to this code with 100% confidence. The other branch for development is where you check in daily changes. This is the branch you manually have your staging machines sync with. Once it passes the tests the code can be merged into the production branch and will auto deploy. The advantage is you will also have a much easier process in trouble shooting production issues because the changes on the files will be much less so it’s easier to find sources of new issues. Some people like to manually update from production code branches which is the safer way to go but at the very least make sure you have the two separate branches. If your checking things into production to fix typos or similar then you’re not using production branch approach properly.

Thanks,
------------------------------------------
Ali Mesdaq (CISSP, GIAC-GREM)
Sr. Security Researcher
Websense Security Labs
http://www.WebsenseSecurityLabs.com<http://www.websensesecuritylabs.com/>
------------------------------------------

From: J. Shirley [mailto:jshirley [at] gmail]
Sent: Friday, October 09, 2009 9:12 AM
To: The elegant MVC web framework
Subject: Re: [Catalyst] Deployment in practice

On Fri, Oct 9, 2009 at 7:55 AM, Tobias Kremer <tobias.kremer [at] gmail<mailto:tobias.kremer [at] gmail>> wrote:
2009/10/9 Octavian Râşniţă <orasnita [at] gmail<mailto:orasnita [at] gmail>>:
> I've recently started to use Mercurial for revision control and the pages
> regarding the deployment of a Catalyst app from the Catalyst wiki also help
> me very much, but I still don't know what would be the best method for
> uploading the files to the server.
> How do you do it?
We do a checkout of the latest version on our staging server, test
everything and then rsync it to the webservers. Works like a charm.

I have deployment branches, and then push from there. It's git, but I'd assume you can do the same in hg (though I've never used it)

See: http://blog.woobling.org/2009/05/deployment-branches-with-git.html

Then, you can use this post-hook script on the production machine and simply push to that repo:
http://github.com/perigrin/soho-web/tree/master/hooks/post-update

This works very well, and you can have that be on a deployment server/location and update the working tree of the other servers in the node.

-J

Click here<https://www.mailcontrol.com/sr/wQw0zmjPoHdJTZGyOCrrhg==> to report this email as spam.

Protected by Websense Hosted Email Security -- www.websense.com

montuori at gmail

Oct 9, 2009, 1:18 PM

Post #7 of 17 (2233 views)
Permalink

Re: Deployment in practice [In reply to]

>>>>> "MA" == Mesdaq, Ali <amesdaq [at] websense> writes:

MA> Once it passes the tests the code can be merged into the
MA> production branch and will auto deploy.

That's a great solution until someone checks a change into the
production branch accidentally. As someone already mentioned, it'll
happen eventually.

I'm a firm believer in not using version control as a deployment tool.
It's more work to create a package, deploy that to test, test, deploy
*the same package* to production, but you at least know that what you
tested is what's in production. Pulling straight from VC you never
really have that assurance.

k.

--
kevin montuori

_______________________________________________
List: Catalyst [at] lists
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst [at] lists/
Dev site: http://dev.catalyst.perl.org/

jshirley at gmail

Oct 9, 2009, 2:08 PM

Post #8 of 17 (2226 views)
Permalink

Re: Re: Deployment in practice [In reply to]

On Fri, Oct 9, 2009 at 1:18 PM, kevin montuori <montuori [at] gmail> wrote:

> >>>>> "MA" == Mesdaq, Ali <amesdaq [at] websense> writes:
>
> MA> Once it passes the tests the code can be merged into the
> MA> production branch and will auto deploy.
>
> That's a great solution until someone checks a change into the
> production branch accidentally. As someone already mentioned, it'll
> happen eventually.
>
> I'm a firm believer in not using version control as a deployment tool.
> It's more work to create a package, deploy that to test, test, deploy
> *the same package* to production, but you at least know that what you
> tested is what's in production. Pulling straight from VC you never
> really have that assurance.
>
>
A lot of these assertions are pretty much FUD.

Even exporting a tarball is sourcing from version control. You're just
bypassing a lot of the built-in aspects of source control and applying
manual steps that can be messed up in a variety of ways.

Unless it is sourced from source control in a way you can easily and
programmatically detect, you really don't know what is in production
outside of "a tarball". You can keep an archive of tarballs, but you lose
context and history... so what's the win?

If something happens and you need to find out the history, what do you do?
You ask version control!

There are plenty of ways to manage how things get into a production branch.
Then, you simply add in another layer using tags. After all, that's pretty
much why tags exist, and how they're used, in every sufficiently modern
system.

You deploy a tag, from a branch, and everybody wins.

Keep in mind that I'm not advocating (nor are the posts I linked to) running
from a -managed- tree. The post-update hook I linked to keeps its own git
repository to itself, it isn't share. You have to explicitly push to it to
update.

These things are very hard to get wrong, and they work very well. You just
have to learn the tools sufficiently.

-J

devin.austin at gmail

Oct 9, 2009, 2:12 PM

Post #9 of 17 (2231 views)
Permalink

Re: Re: Deployment in practice [In reply to]

On Fri, Oct 9, 2009 at 3:08 PM, J. Shirley <jshirley [at] gmail> wrote:

> On Fri, Oct 9, 2009 at 1:18 PM, kevin montuori <montuori [at] gmail> wrote:
>
>> >>>>> "MA" == Mesdaq, Ali <amesdaq [at] websense> writes:
>>
>> MA> Once it passes the tests the code can be merged into the
>> MA> production branch and will auto deploy.
>>
>> That's a great solution until someone checks a change into the
>> production branch accidentally. As someone already mentioned, it'll
>> happen eventually.
>>
>> I'm a firm believer in not using version control as a deployment tool.
>> It's more work to create a package, deploy that to test, test, deploy
>> *the same package* to production, but you at least know that what you
>> tested is what's in production. Pulling straight from VC you never
>> really have that assurance.
>>
>>
> A lot of these assertions are pretty much FUD.
>
> Even exporting a tarball is sourcing from version control. You're just
> bypassing a lot of the built-in aspects of source control and applying
> manual steps that can be messed up in a variety of ways.
>
> Unless it is sourced from source control in a way you can easily and
> programmatically detect, you really don't know what is in production
> outside of "a tarball". You can keep an archive of tarballs, but you lose
> context and history... so what's the win?
>
> If something happens and you need to find out the history, what do you do?
> You ask version control!
>
> There are plenty of ways to manage how things get into a production
> branch. Then, you simply add in another layer using tags. After all,
> that's pretty much why tags exist, and how they're used, in every
> sufficiently modern system.
>
> You deploy a tag, from a branch, and everybody wins.
>
> Keep in mind that I'm not advocating (nor are the posts I linked to)
> running from a -managed- tree. The post-update hook I linked to keeps its
> own git repository to itself, it isn't share. You have to explicitly push
> to it to update.
>
> These things are very hard to get wrong, and they work very well. You just
> have to learn the tools sufficiently.
>
> -J
>
> _______________________________________________
> List: Catalyst [at] lists
> Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
> Searchable archive:
> http://www.mail-archive.com/catalyst [at] lists/
> Dev site: http://dev.catalyst.perl.org/
>
>
I'm coming in late but deploying a tag from trunk/etc and "switching" the
production directory that nginx/lighttpd/apache is looking at is probably
the coolest thing ever. You have a great deal of control when needing to
roll back, and a record of all previous tags/versions.
Anyway, my $0.02.

-Devin

--
Devin Austin
http://www.codedright.net
9702906669 - Cell

amesdaq at websense

Oct 9, 2009, 2:36 PM

Post #10 of 17 (2218 views)
Permalink

RE: Re: Deployment in practice [In reply to]

Solving the issue of someone accidentally checking something into production branch is a lot easier to solve than all the problems associated with not having version control. To compare which do you think will be an easier issue to solve: Someone accidentally checking something into production branch that breaks stuff or someone making a change to production server with no version control that breaks stuff? At least with the first issue you can roll back the change and get back to stable instantly. With the other issue you could be troubleshooting for a long time.

Thanks,
------------------------------------------
Ali Mesdaq (CISSP, GIAC-GREM)
Sr. Security Researcher
Websense Security Labs
http://www.WebsenseSecurityLabs.com
------------------------------------------

-----Original Message-----
From: kevin montuori [mailto:montuori [at] gmail]
Sent: Friday, October 09, 2009 1:19 PM
To: The elegant MVC web framework
Subject: [Catalyst] Re: Deployment in practice

>>>>> "MA" == Mesdaq, Ali <amesdaq [at] websense> writes:

MA> Once it passes the tests the code can be merged into the
MA> production branch and will auto deploy.

That's a great solution until someone checks a change into the
production branch accidentally. As someone already mentioned, it'll
happen eventually.

I'm a firm believer in not using version control as a deployment tool.
It's more work to create a package, deploy that to test, test, deploy
*the same package* to production, but you at least know that what you
tested is what's in production. Pulling straight from VC you never
really have that assurance.

k.

--
kevin montuori

_______________________________________________
List: Catalyst [at] lists
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst [at] lists/
Dev site: http://dev.catalyst.perl.org/

Protected by Websense Hosted Email Security -- www.websense.com

_______________________________________________
List: Catalyst [at] lists
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst [at] lists/
Dev site: http://dev.catalyst.perl.org/

autarch at urth

Oct 9, 2009, 4:16 PM

Post #11 of 17 (2216 views)
Permalink

RE: Re: Deployment in practice [In reply to]

On Fri, 9 Oct 2009, Mesdaq, Ali wrote:

> At least with the first issue you can roll back the change and get back
> to stable instantly. With the other issue you could be troubleshooting
> for a long time.

But I think the OP (other poster) mentioned packages. Presumably you can
roll back to an earlier package as easily as rolling back to an earlier
state of the tree.

-dave

/*============================================================
http://VegGuide.org http://blog.urth.org
Your guide to all that's veg House Absolute(ly Pointless)
============================================================*/

_______________________________________________
List: Catalyst [at] lists
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst [at] lists/
Dev site: http://dev.catalyst.perl.org/

moseley at hank

Oct 10, 2009, 4:28 PM

Post #12 of 17 (2174 views)
Permalink

Re: Deployment in practice [In reply to]

2009/10/9 Octavian Râşniţă <orasnita [at] gmail>

> Hi,
>
> I've recently started to use Mercurial for revision control and the pages
> regarding the deployment of a Catalyst app from the Catalyst wiki also help
> me very much, but I still don't know what would be the best method for
> uploading the files to the server.
>
> How do you do it?
> Archive the entire app an dupload to the server then change the permissions
> of the files and folders there?
> Or upload the modified files one by one?
> Or generate a tarball on each revision and unarchive and make, make test it
> on the server?
>

I have a script on a build server that does an svn export of a specific
branch and version (or latest trunk if no version specified). The script
then runs the test suite, runs the "build" scripts that generate "minified"
css, javascript, and anything else that prepares the package such as create
a meta file with build info, and then builds a tarball named after the
reversion. Also, a symlink is created as "latest" on the build server.

Then on target machines (testing, staging, and production) I have a "push"
script that fetches the tarball (based on a specific version or "latest"),
untars it on the machine. It's untarred into a directory named after the
revision it came from. Tests are run that verify that it can talk to
whatever it needs (database, services). Then do a graceful stop of the
existing server, move symbolic links, start the server back up and then use
wget to validate a number of test URLs. Depending on the update some
servers may get pulled out of the balancer before upgrading to do it in
stages.

Pushing a previous version will just update the symlinks if they revesion
exists.

Not perfect but seems to work ok for now.

--
Bill Moseley
moseley [at] hank

orasnita at gmail

Oct 11, 2009, 12:49 AM

Post #13 of 17 (2176 views)
Permalink

Re: Deployment in practice [In reply to]

From: "Bill Moseley" <moseley [at] hank>
2009/10/9 Octavian Râşniţă <orasnita [at] gmail>

> Hi,
>
> I've recently started to use Mercurial for revision control and the pages
> regarding the deployment of a Catalyst app from the Catalyst wiki also
> help
> me very much, but I still don't know what would be the best method for
> uploading the files to the server.
>
> How do you do it?
> Archive the entire app an dupload to the server then change the
> permissions
> of the files and folders there?
> Or upload the modified files one by one?
> Or generate a tarball on each revision and unarchive and make, make test
> it
> on the server?
>

I have a script on a build server that does an svn export of a specific
branch and version (or latest trunk if no version specified). The script
then runs the test suite, runs the "build" scripts that generate "minified"
css, javascript, and anything else that prepares the package such as create
a meta file with build info, and then builds a tarball named after the
reversion. Also, a symlink is created as "latest" on the build server.

Then on target machines (testing, staging, and production) I have a "push"
script that fetches the tarball (based on a specific version or "latest"),
untars it on the machine. It's untarred into a directory named after the
revision it came from. Tests are run that verify that it can talk to
whatever it needs (database, services). Then do a graceful stop of the
existing server, move symbolic links, start the server back up and then use
wget to validate a number of test URLs. Depending on the update some
servers may get pulled out of the balancer before upgrading to do it in
stages.

Pushing a previous version will just update the symlinks if they revesion
exists.

Not perfect but seems to work ok for now.

Thank you all for your answers. It was very helpful.
What about the database? If starting from a certain revision you need to
make some database changes (adding tables, adding/deleting some table
fields) how can we automate this?

I mean, I think that we could run a script that add some tables and remove
the unneeded fields from other tables, but if something goes wrong and we
need to go back to the previous version, we would need the records from
those fields.

What can we do in these situations? Do the database update and the tests
manually? Or back-up the entire database and then test the latest version of
the app?

Thanks.

Octavian

_______________________________________________
List: Catalyst [at] lists
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst [at] lists/
Dev site: http://dev.catalyst.perl.org/

dab at catapulse

Oct 11, 2009, 2:54 AM

Post #14 of 17 (2172 views)
Permalink

Re: Deployment in practice [In reply to]

> Thank you all for your answers. It was very helpful.
> What about the database? If starting from a certain revision you need
> to make some database changes (adding tables, adding/deleting some
> table fields) how can we automate this?
>
> I mean, I think that we could run a script that add some tables and
> remove the unneeded fields from other tables, but if something goes
> wrong and we need to go back to the previous version, we would need
> the records from those fields.
>
> What can we do in these situations? Do the database update and the
> tests manually? Or back-up the entire database and then test the
> latest version of the app?
>
> Thanks.
>
> Octavian

DBIx::Class::Schema::Versioned perhaps ?

Dab

_______________________________________________
List: Catalyst [at] lists
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst [at] lists/
Dev site: http://dev.catalyst.perl.org/

orasnita at gmail

Oct 11, 2009, 3:59 AM

Post #15 of 17 (2190 views)
Permalink

Re: Deployment in practice [In reply to]

From: "dab" <dab [at] catapulse>
>> Thank you all for your answers. It was very helpful.
>> What about the database? If starting from a certain revision you need
>> to make some database changes (adding tables, adding/deleting some
>> table fields) how can we automate this?
>>
>> I mean, I think that we could run a script that add some tables and
>> remove the unneeded fields from other tables, but if something goes
>> wrong and we need to go back to the previous version, we would need
>> the records from those fields.
>>
>> What can we do in these situations? Do the database update and the
>> tests manually? Or back-up the entire database and then test the
>> latest version of the app?
>>
>> Thanks.
>>
>> Octavian
>
> DBIx::Class::Schema::Versioned perhaps ?
>
> Dab

I don't know... Does anyone use it?

I read in its POD docs that "At the moment, only SQLite and MySQL are
supported.", however in the past I've seen that when creating MySQL queries
from a DBIC schema, some fields were wrongly created, including the size
when not needed, like
date date(10),
so they should be corrected manually. Because of this, I don't know if this
tool can be used for do the change automaticly...
(Or was this issue solved since then?)

Octavian

_______________________________________________
List: Catalyst [at] lists
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst [at] lists/
Dev site: http://dev.catalyst.perl.org/

ash_cpan at firemirror

Oct 11, 2009, 4:10 AM

Post #16 of 17 (2182 views)
Permalink

Re: Deployment in practice [In reply to]

On 11 Oct 2009, at 11:59, Octavian R�snita wrote:

> From: "dab" <dab [at] catapulse>
>>> Thank you all for your answers. It was very helpful.
>>> What about the database? If starting from a certain revision you
>>> need
>>> to make some database changes (adding tables, adding/deleting some
>>> table fields) how can we automate this?
>>>
>>> I mean, I think that we could run a script that add some tables and
>>> remove the unneeded fields from other tables, but if something goes
>>> wrong and we need to go back to the previous version, we would need
>>> the records from those fields.
>>>
>>> What can we do in these situations? Do the database update and the
>>> tests manually? Or back-up the entire database and then test the
>>> latest version of the app?
>>>
>>> Thanks.
>>>
>>> Octavian
>>
>> DBIx::Class::Schema::Versioned perhaps ?
>>
>> Dab
>
> I don't know... Does anyone use it?
>
> I read in its POD docs that "At the moment, only SQLite and MySQL
> are supported.", however in the past I've seen that when creating
> MySQL queries from a DBIC schema, some fields were wrongly created,
> including the size when not needed, like
> date date(10),
> so they should be corrected manually. Because of this, I don't know
> if this tool can be used for do the change automaticly...
> (Or was this issue solved since then?)
>
> Octavian
>
refactoring

Schema::Versioned uses SQL::Translator under the hood to do its job.
Work has been done to make SQL::T v1 better, and as part of this years
Google Summer of Code, Justin Hunter was the internals to make it
easier to fix problems. So the problems should either be fixed
already, or be fixed soon.

-ash
_______________________________________________
List: Catalyst [at] lists
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst [at] lists/
Dev site: http://dev.catalyst.perl.org/

moseley at hank

Oct 11, 2009, 9:03 AM

Post #17 of 17 (2165 views)
Permalink

Re: Deployment in practice [In reply to]

2009/10/11 Octavian Râşniţă <orasnita [at] gmail>

> Thank you all for your answers. It was very helpful.
> What about the database? If starting from a certain revision you need to
> make some database changes (adding tables, adding/deleting some table
> fields) how can we automate this?
>

Every database update is slightly different, so automating hasn't seemed
possible.

The applications includes a script to build a test database. The test
database is saved in svn with each application. The test files can load
this database to test against when running make test. That stays in sync
with development.

I also have a cron job running on a test server that does an svn checkout
when the revision number changes (for branches or trunk) and runs tests
against this test database.

> I mean, I think that we could run a script that add some tables and remove
> the unneeded fields from other tables, but if something goes wrong and we
> need to go back to the previous version, we would need the records from
> those fields.
>

The database is more complex. We create a set of change files that the DBA
applies. I like hand creating these and filling them with comments on why a
column is added or dropped, and the tickets they apply to. They are in svn
(for a history) but also in a web app the DBA uses to track what's been
applied and what is scheduled to apply.

It's very rare that a release includes database changes that have to be done
at the moment of pushing the new version. Schema changes happen first.
That is, the database is ready for the new features before the application
is pushed. Sometimes constraints have to be added in after pushing the
release. Bigger changes require a service window, of course. In those
cases I have created a script that makes the database changes, populates
tables and runs tests.

Once changes are in the database tests can be run against a snapshot of the
production db instead of the test database.

I'd like to have it more automated, but it seems like every database change
requires something special.

--
Bill Moseley
moseley [at] hank

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads