Mailing List Archive: Catalyst: Users

New restarter code in Catalyst::Devel 1.14_01

Index | Next | Previous | View Threaded

autarch at urth

May 10, 2009, 7:25 PM

Post #1 of 9 (1267 views)
Permalink

New restarter code in Catalyst::Devel 1.14_01

I recently did some work to rewrite the Catalyst restarter code and make
it both simpler and less fragile.

The restarter no longer attempts to reload changed modules in an existing
process. This was fragile, and tended to produce a lot of bogus
compilation failures.

The new restarter just _tries_ to restart on every change. If the restart
fails, the server dies, rather than continuing to run with the old code.
However, the restarter keeps watching for changes, and will attempt to
start the server on the next change.

I think this makes for a much saner development process. I always found it
very confusing to make a change, reload a web page, and not see the change
reflected, only to realize that the server didn't actually reload.

Restarting is also much faster. All of the Catalyst core code is preloaded
in a parent process, and the server runs in a child. That means that when
the child is killed and a new child spawned for a new server, it does not
need to load all that Catalyst code again, just your application.

The new restarter has been moved from Catalyst-Runtime, where it lived as
a weird bastardized Engine subclass, to Catalyst-Devel, where it lives as
a standalone Catalyst::Restarter module.

I also switched the actual file monitoring away from the existing
File::Modified implementation. The new implementation uses a new module I
wrote called File::ChangeNotify. This module is much better suited to
Catalyst's needs, as it simply reports on relevant filesystem changes.

File::ChangeNotify is designed so that it is easy to implement per-OS
subclasses to do file monitoring. It currently comes with an Inotify
subclass that uses the Linux inotify interface, as well as a fallback that
is just written in Perl.

I'm hoping that in the future we'll see other OS-specific implementations.

I'd appreciate if people would test out the new code. You'll need to
regenerate your myapp_server.pl script to actually see the changes. Old
scripts will of course continue to work, since the old restarter code is
still in Catalyst-Runtime.

-dave

/*============================================================
http://VegGuide.org http://blog.urth.org
Your guide to all that's veg House Absolute(ly Pointless)
============================================================*/

_______________________________________________
List: Catalyst [at] lists
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst [at] lists/
Dev site: http://dev.catalyst.perl.org/

jon at jrock

May 11, 2009, 3:53 PM

Post #2 of 9 (1204 views)
Permalink

Re: New restarter code in Catalyst::Devel 1.14_01 [In reply to]

* On Sun, May 10 2009, Dave Rolsky wrote:
> I recently did some work to rewrite the Catalyst restarter code and
> make it both simpler and less fragile.
>
> [snip]

Hi Dave,

This is really wonderful. Thanks!

I have begun porting this concept to MooseX::Runnable, so hopefully all
Perl applications will have access to this functionality soon.

(As an aside, someone mentioned factoring out Catalyst stuff into
separate components in another mail earlier today. MooseX::Runnable is
the 0th step in this process. It's actually being used by a few people
in production now, so it's almost time for step 1 -- building a
reusable, componentized (not-necessarily-web) application framework. I
have started this in a project called Eventful, although most of the
Real Work has happened in my current $work_app. The step after that is
to make Eventful speak HTTP (which it does already, via HTTP::Engine).
Finally, on top of those components, we can add the Catalyst stuff as
Yet Another Component. I talked about this on my blog a few weeks ago,
so if you want more details, read that, and bug me on IRC.

The code in the Eventful repo does work, but it doesn't have the
component loading stuff, arguably the most critical part, working
correctly yet... but it will soon, as I finally figured it out this
morning. :)

Regards,
Jonathan Rockway

--
print just => another => perl => hacker => if $,=$"

_______________________________________________
List: Catalyst [at] lists
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst [at] lists/
Dev site: http://dev.catalyst.perl.org/

rodrigolive at gmail

May 18, 2009, 8:57 AM

Post #3 of 9 (1173 views)
Permalink

Re: New restarter code in Catalyst::Devel 1.14_01 [In reply to]

>
>
> Restarting is also much faster. All of the Catalyst core code is preloaded
> in a parent process, and the server runs in a child. That means that when
> the child is killed and a new child spawned for a new server, it does not
> need to load all that Catalyst code again, just your application.
>

I'm getting a misbehaving action dispatcher with the new restarter, even
before the app is restarted:

C:\> catalyst MyApp
C:\MyApp> perl script\myapp_server.pl -r
...
[info] MyApp powered by Catalyst 5.80003
You can connect to your server at http://localhost:3000

The response to the first request to http://localhost:3000 is this:

[info] *** Request 1 (0.333/s) [-2208] [Mon May 18 17:02:38 2009] ***
[debug] "GET" request for "/" from "127.0.0.1"
[debug] Couldn't forward to command "//_DISPATCH": Invalid action or
component.
[error] Couldn't forward to command "//_DISPATCH": Invalid action or
component.
[info] Request took 0.008397s (119.090/s)

The app works just fine without the -r option.

This is a fresh Strawberry Perl 5.8.9 (Apr 2009) install + Catalyst::Devel.

Has anyone tested the new restarter on Win32? It looks like some kindda
issue with the "fork" at Catalyst::Restarter::_fork_and_start(). IMHO, a
parent process loaded with the core code could too advanced for windows'
pseudo-fork. So I was wondering if maybe this was tested on ActivePerl
--I've feeling that their fork works better than Strawberry's.

-rodrigo

autarch at urth

May 18, 2009, 7:19 PM

Post #4 of 9 (1160 views)
Permalink

Re: New restarter code in Catalyst::Devel 1.14_01 [In reply to]

On Mon, 18 May 2009, Rodrigo wrote:

> This is a fresh Strawberry Perl 5.8.9 (Apr 2009) install + Catalyst::Devel.
>
> Has anyone tested the new restarter on Win32? It looks like some kindda
> issue with the "fork" at Catalyst::Restarter::_fork_and_start(). IMHO, a
> parent process loaded with the core code could too advanced for windows'
> pseudo-fork. So I was wondering if maybe this was tested on ActivePerl
> --I've feeling that their fork works better than Strawberry's.

I did test this on WinXP using Strawberry (5.8.8, IIRC), and it worked
fine. Win32 Perl does support a fork emulation that uses thread under the
hood.

I don't think the fork implementation in Strawberry is any different than
ActivePerl. ActiveState wrote the fork implementation (again, IIRC) and
contributed it to the Perl core.

That said, I was thinking it might be good to also have a threads-based
version available, and that could be the default to use on Win32.

I took a stab at doing this at one point, but got stuck on trying to
actually kill the "child" thread.

-dave

/*============================================================
http://VegGuide.org http://blog.urth.org
Your guide to all that's veg House Absolute(ly Pointless)
============================================================*/

_______________________________________________
List: Catalyst [at] lists
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst [at] lists/
Dev site: http://dev.catalyst.perl.org/

rodrigolive at gmail

May 19, 2009, 12:06 AM

Post #5 of 9 (1155 views)
Permalink

Re: New restarter code in Catalyst::Devel 1.14_01 [In reply to]

> I did test this on WinXP using Strawberry (5.8.8, IIRC), and it worked
> fine. Win32 Perl does support a fork emulation that uses thread under the
> hood.
>

Strange. I'll try it with a different/older version of Strawberry Perl and
see what happens.

>
> I don't think the fork implementation in Strawberry is any different than
> ActivePerl. ActiveState wrote the fork implementation (again, IIRC) and
> contributed it to the Perl core.
>

Their forks seem to behave slightly different (it's a feeling more than a
fact), even though I agree that the code is probably the same one Microsoft
helped develop originally. The compilers are different though, VisualStudio
vs MinGW.

>
> That said, I was thinking it might be good to also have a threads-based
> version available, and that could be the default to use on Win32.
>
> I took a stab at doing this at one point, but got stuck on trying to
> actually kill the "child" thread.
>

Since I needed to have the restarter working badly on that machine in
particular, I quickly patched the code with Proc::Background. Something like
this:

sub _fork_and_start() {
my $proc = new Proc::Background("perl $0"); # I'm writing this from
memory; not this exactly, there were more args...
my $pid = $proc->pid;
...}

It did the job since the background process was a non-restarter dev server.
And the restarter was able to kill it just fine when changes were detected
because $pid is a real os pid.

I think I put threads / async { } somewhere in there before using
Proc::Background but I got an error... and don't recall which. Killing the
thread requires it to die from the inside, sending it a signal of some sort.

Couldn't the restarter be subclassed then changed in myapp_script.pl through
an %ENV variable?

-rodrigo

autarch at urth

May 19, 2009, 7:19 AM

Post #6 of 9 (1157 views)
Permalink

Re: New restarter code in Catalyst::Devel 1.14_01 [In reply to]

On Tue, 19 May 2009, Rodrigo wrote:

> Since I needed to have the restarter working badly on that machine in
> particular, I quickly patched the code with Proc::Background. Something like
> this:
>
> sub _fork_and_start() {
> my $proc = new Proc::Background("perl $0"); # I'm writing this from
> memory; not this exactly, there were more args...
> my $pid = $proc->pid;
> ...}

Doing this pretty much defeats the point of the new restarter code, which
was to avoid starting the app from scratch each time.

> I think I put threads / async { } somewhere in there before using
> Proc::Background but I got an error... and don't recall which. Killing the
> thread requires it to die from the inside, sending it a signal of some sort.

Right, that's where I got stuck.

> Couldn't the restarter be subclassed then changed in myapp_script.pl through
> an %ENV variable?

Sure, but the trick is to come up with something that works reliably on
Windows, and ideally, has the benefits of the new restarter code.

-dave

/*============================================================
http://VegGuide.org http://blog.urth.org
Your guide to all that's veg House Absolute(ly Pointless)
============================================================*/

_______________________________________________
List: Catalyst [at] lists
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst [at] lists/
Dev site: http://dev.catalyst.perl.org/

autarch at urth

May 23, 2009, 5:20 PM

Post #7 of 9 (1085 views)
Permalink

Re: New restarter code in Catalyst::Devel 1.14_01 [In reply to]

On Tue, 19 May 2009, Rodrigo wrote:

> Since I needed to have the restarter working badly on that machine in
> particular, I quickly patched the code with Proc::Background. Something like
> this:

I implemented something like this in the latest Catalyst::Devel. I did
more thorough testing on Windows and it seems to really work now.

Please give it a shot on your system.

-dave

/*============================================================
http://VegGuide.org http://blog.urth.org
Your guide to all that's veg House Absolute(ly Pointless)
============================================================*/

_______________________________________________
List: Catalyst [at] lists
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst [at] lists/
Dev site: http://dev.catalyst.perl.org/

rodrigolive at gmail

May 24, 2009, 3:51 PM

Post #8 of 9 (1081 views)
Permalink

Re: New restarter code in Catalyst::Devel 1.14_01 [In reply to]

On Sun, May 24, 2009 at 2:20 AM, Dave Rolsky <autarch [at] urth> wrote:

> On Tue, 19 May 2009, Rodrigo wrote:
>
> Since I needed to have the restarter working badly on that machine in
>> particular, I quickly patched the code with Proc::Background. Something
>> like
>> this:
>>
>
> I implemented something like this in the latest Catalyst::Devel. I did more
> thorough testing on Windows and it seems to really work now.
>
> Please give it a shot on your system.
>
>
>
> -dave
>

That's just hack-tastic! Works great. Thanks.

Shame we can't use the new pre-loading functionality on Windows though. I
tested with different versions of things and errors just got nastier and
nastier.

IMO, to be able to pick restarter engines via %ENV was also a necessary
change anyway. Say, in the future we could have a threaded implementation,
and that would be a whole new ballgame for the restarter.

-rodrigo

autarch at urth

May 24, 2009, 4:29 PM

Post #9 of 9 (1080 views)
Permalink

Re: New restarter code in Catalyst::Devel 1.14_01 [In reply to]

On Mon, 25 May 2009, Rodrigo wrote:

> Shame we can't use the new pre-loading functionality on Windows though. I
> tested with different versions of things and errors just got nastier and
> nastier.
>
> IMO, to be able to pick restarter engines via %ENV was also a necessary
> change anyway. Say, in the future we could have a threaded implementation,
> and that would be a whole new ballgame for the restarter.

For the record, I did try a threaded version, and saw the same weird
errors on Windows. That's not entirely surprising since Perl uses threads
for fork() on Windows.

-dave

/*============================================================
http://VegGuide.org http://blog.urth.org
Your guide to all that's veg House Absolute(ly Pointless)
============================================================*/

_______________________________________________
List: Catalyst [at] lists
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst [at] lists/
Dev site: http://dev.catalyst.perl.org/

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads