Mailing List Archive: Catalyst: Users

How to use local::lib

Index | Next | Previous | View Threaded

tobias.kremer at gmail

Jun 22, 2009, 8:43 AM

Post #1 of 8 (765 views)
Permalink

How to use local::lib

After reading about local::lib and thinking that it's too good to be
true, I just had to try it out as a possible deployment method for one
of our apps. I have the app and all dependencies running on my
development box. The prerequisites are specified in my app's
Makefile.PL. Now, after having bootstrapped local::lib, I ran the
following commands - as suggested by t0m in a recent thread - but
nothing happens because all required modules are already installed in
my system-wide CPAN directory in their correct versions:

$ eval $(perl -I$HOME/perl5/lib/perl5 -Mlocal::lib=--self-contained,local_lib)
$ make installdeps

Am I missing something here or do I really have to checkout my app on
a fresh installation with no prerequisites installed and try the above
there?

Also, am I right that this method won't work for XS-based modules if
I'm building my local_lib, say on a Ubuntu system, while the
production system is Debian-based?

Thanks a lot for any hints!

--Tobias

_______________________________________________
List: Catalyst[at]lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst[at]lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/

robertkrimen at gmail

Jun 22, 2009, 10:02 AM

Post #2 of 8 (722 views)
Permalink

Re: How to use local::lib [In reply to]

On Mon, Jun 22, 2009 at 8:43 AM, Tobias Kremer <tobias.kremer[at]gmail.com>wrote:

> After reading about local::lib and thinking that it's too good to be
> true, I just had to try it out as a possible deployment method for one
> of our apps. I have the app and all dependencies running on my
> development box. The prerequisites are specified in my app's
> Makefile.PL. Now, after having bootstrapped local::lib, I ran the
> following commands - as suggested by t0m in a recent thread - but
> nothing happens because all required modules are already installed in
> my system-wide CPAN directory in their correct versions:
>
> $ eval $(perl -I$HOME/perl5/lib/perl5
> -Mlocal::lib=--self-contained,local_lib)
> $ make installdeps
>
> Am I missing something here or do I really have to checkout my app on
> a fresh installation with no prerequisites installed and try the above
> there?

--self-contained does not work when exporting variables to the environment
because of the way the perl runtime works. Basically perl will always setup
@INC to include
system libraries, regardless of what is in PERL5LIB.

You can try this: PERL5OPT="-Mlocal::lib=--self-contained,local_lib"

(But you may encounter some problems with taint checking)

Rob

bobtfish at bobtfish

Jun 22, 2009, 10:04 AM

Post #3 of 8 (723 views)
Permalink

Re: How to use local::lib [In reply to]

On 22 Jun 2009, at 16:43, Tobias Kremer wrote:

> After reading about local::lib and thinking that it's too good to be
> true, I just had to try it out as a possible deployment method for one
> of our apps. I have the app and all dependencies running on my
> development box. The prerequisites are specified in my app's
> Makefile.PL. Now, after having bootstrapped local::lib, I ran the
> following commands - as suggested by t0m in a recent thread - but
> nothing happens because all required modules are already installed in
> my system-wide CPAN directory in their correct versions:

Yeah, unfortunately, this doesn't work so good unless your
dependencies have been installed in a local::lib themselves, and you
can remove this from PERL5LIB

After rambling about 'just do it like this', I then wandered off and
tried to make the new --self-contained feature of this work. Wasn't
quite as easy to make it work generically than I'd implied I don't
think.

> $ eval $(perl -I$HOME/perl5/lib/perl5 -Mlocal::lib=--self-
> contained,local_lib)
> $ make installdeps
>
> Am I missing something here or do I really have to checkout my app on
> a fresh installation with no prerequisites installed and try the above
> there?

Yeah, pretty much at the moment. If you install all your
prerequisites in a local::lib, then if you blank PERL5LIB, then it
_just works_.

However have a look at:

http://bobtfish.livejournal.com/264777.html
http://github.com/bobtfish/catalyst-app-example-locallibapp/tree/master

This is what I threw together after making that posting, and whilst
it's not quite all there yet - it will at least install a self
contained Catalyst & etc.

> Also, am I right that this method won't work for XS-based modules if
> I'm building my local_lib, say on a Ubuntu system, while the
> production system is Debian-based?

Correct. You'll have to build your local::lib on a system with
similar architecture to your production boxen.

Cheers
t0m

_______________________________________________
List: Catalyst[at]lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst[at]lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/

tobias.kremer at gmail

Jun 23, 2009, 12:59 AM

Post #4 of 8 (710 views)
Permalink

Re: How to use local::lib [In reply to]

On Mon, Jun 22, 2009 at 7:04 PM, Tomas Doran<bobtfish[at]bobtfish.net> wrote:
> On 22 Jun 2009, at 16:43, Tobias Kremer wrote:
>> Am I missing something here or do I really have to checkout my app on
>> a fresh installation with no prerequisites installed and try the above
>> there?
> Yeah, pretty much at the moment. If you install all your prerequisites in a
> local::lib, then if you blank PERL5LIB, then it _just works_.
>> Also, am I right that this method won't work for XS-based modules if
>> I'm building my local_lib, say on a Ubuntu system, while the
>> production system is Debian-based?
> Correct. You'll have to build your local::lib on a system with similar
> architecture to your production boxen.

These things make the local::lib approach a bit more unappealing for
us. For one thing, it appears to be still quite a bit rough around the
edges and I doubt that there's anything I personally could do about it
because my knowledge there is limited. Secondly, we really would like
to have a central repository for our development AND production
systems. What proven alternatives are there if you have developers
working on OS X, Ubuntu etc. and deploy on a different (Debian)
system?

What about CPAN::Mini? At first, it sounds strange to have a whole
copy of the CPAN around (although it's not that big), but then again
it sounds really cool to just "make installdeps" and have everything
installed in the correct version in a cross-platform compatible way.

By the way: Would it theoretically be possible to install specific
versions of a CPAN distribution? Is there possibly already work
underway to achieve this? I think that this would really solve a lot
of deployment trouble ...

So, how are you doing your deployment? :) I believe this is an area
where there's still a lot of homebrew stuff going on and it would be
nice to have one (or 2-3) ways to do it! I'd be willing to write
something up for the wiki if there's enough input.

Thanks a lot!

--Tobias

_______________________________________________
List: Catalyst[at]lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst[at]lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/

bobtfish at bobtfish

Jun 23, 2009, 1:26 AM

Post #5 of 8 (710 views)
Permalink

Re: How to use local::lib [In reply to]

On 23 Jun 2009, at 08:59, Tobias Kremer wrote:
> These things make the local::lib approach a bit more unappealing for
> us. For one thing, it appears to be still quite a bit rough around the
> edges and I doubt that there's anything I personally could do about it
> because my knowledge there is limited. Secondly, we really would like
> to have a central repository for our development AND production
> systems. What proven alternatives are there if you have developers
> working on OS X, Ubuntu etc. and deploy on a different (Debian)
> system?

Erm, well, the new --self-contained feature is a little rough round
the edges, sure.

local::lib itself just works given what it was originally trying to
accomplish.

I think you need a better separation between development and staging.
If you're not running something production like in staging, then IMO
you're really screwed.

Developers can then use what they like (and whatever OS/versions of
modules) they want, if they want to install the app locally, but
that's unsupported. I'd also make a 'production like' (i.e. same OS)
vmware machine available with the 'standard' dependencies - all of
your developers should be given enough horsepower to run vmware,
problem solved.

> What about CPAN::Mini? At first, it sounds strange to have a whole
> copy of the CPAN around (although it's not that big), but then again
> it sounds really cool to just "make installdeps" and have everything
> installed in the correct version in a cross-platform compatible way.

Yes, you can just point an ftp server at a CPAN::Mini, then set your
CPAN shell to use that as your CPAN mirror - it'll just work..

> By the way: Would it theoretically be possible to install specific
> versions of a CPAN distribution? Is there possibly already work
> underway to achieve this? I think that this would really solve a lot
> of deployment trouble ...

Erm, it's already practically possible. You just need to ask the CPAN
shell for a specific revision.

>
> So, how are you doing your deployment? :) I believe this is an area
> where there's still a lot of homebrew stuff going on and it would be
> nice to have one (or 2-3) ways to do it! I'd be willing to write
> something up for the wiki if there's enough input.

At work I debian package everything. But you're still going to make
one package per dependency per platform.

That's a lot of work.. And it totally precludes having different
applications with different versions of the same module on the same
machine.

I guess this comes down to how many environments you're prepared to
'officially' support, and what your constraints are.

Cheers
t0m

_______________________________________________
List: Catalyst[at]lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst[at]lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/

tobias.kremer at gmail

Jun 23, 2009, 1:41 AM

Post #6 of 8 (708 views)
Permalink

Re: How to use local::lib [In reply to]

On Tue, Jun 23, 2009 at 10:26 AM, Tomas Doran<bobtfish[at]bobtfish.net> wrote:
> I think you need a better separation between development and staging. If
> you're not running something production like in staging, then IMO you're
> really screwed.

Actually, we do :)

> Developers can then use what they like (and whatever OS/versions of modules)
> they want, if they want to install the app locally, but that's unsupported.
> I'd also make a 'production like' (i.e. same OS) vmware machine available
> with the 'standard' dependencies - all of your developers should be given
> enough horsepower to run vmware, problem solved.

Yes, that'd be possible. Nevertheless, it would be nice if you could
bring up a new development box with "make installdeps" that "just
works" because it has the right module versions installed and not some
new, probably incompatible version.

>> By the way: Would it theoretically be possible to install specific
>> versions of a CPAN distribution? Is there possibly already work
>> underway to achieve this? I think that this would really solve a lot
>> of deployment trouble ...
> Erm, it's already practically possible. You just need to ask the CPAN shell
> for a specific revision.

Cool, but what I really meant to ask was: Is it possible to require a
specific version in your Makefile.PL and have only this version
installed during "make installdeps"? :) This would possibly solve most
of the CPAN-related deployment problems.

> At work I debian package everything. But you're still going to make one
> package per dependency per platform.
> That's a lot of work.. And it totally precludes having different
> applications with different versions of the same module on the same machine.

I tried that with Catalyst::Runtime a couple of weeks ago and it still hurts :)

Thanks for your help!

--Tobias

_______________________________________________
List: Catalyst[at]lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst[at]lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/

bobtfish at bobtfish

Jun 23, 2009, 5:03 AM

Post #7 of 8 (706 views)
Permalink

Re: How to use local::lib [In reply to]

On 23 Jun 2009, at 09:41, Tobias Kremer wrote:
>> Developers can then use what they like (and whatever OS/versions
>> of modules)
>> they want, if they want to install the app locally, but that's
>> unsupported.
>> I'd also make a 'production like' (i.e. same OS) vmware machine
>> available
>> with the 'standard' dependencies - all of your developers should
>> be given
>> enough horsepower to run vmware, problem solved.
>
> Yes, that'd be possible. Nevertheless, it would be nice if you could
> bring up a new development box with "make installdeps" that "just
> works" because it has the right module versions installed and not some
> new, probably incompatible version.

Aha, well, the obvious correct solution to that is to spin up and
install your environment dependencies with the latest CPAN
$everything on a nightly basis, and run your smoke tests, then yell
at CPAN authors whos newer module broke your code..

If you're not doing something similar (even if more informally), then
you can get into big issues in a couple of years trying to reuse any
modern code (or upgrade something due to a bug) - it has newer
dependencies, and you can't upgrade, and suddenly you're in a nasty
place...

I totally appreciate the need to have a 'known environment' for
production, but at least testing newer CPAN releases automatically is
very worthwhile..

> Cool, but what I really meant to ask was: Is it possible to require a
> specific version in your Makefile.PL and have only this version
> installed during "make installdeps"? :) This would possibly solve most
> of the CPAN-related deployment problems.

No, unfortunately it isn't..

Cheers
t0m

_______________________________________________
List: Catalyst[at]lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst[at]lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/

autarch at urth

Jun 23, 2009, 7:01 AM

Post #8 of 8 (705 views)
Permalink

Re: How to use local::lib [In reply to]

On Tue, 23 Jun 2009, Tobias Kremer wrote:

> Cool, but what I really meant to ask was: Is it possible to require a
> specific version in your Makefile.PL and have only this version
> installed during "make installdeps"? :) This would possibly solve most
> of the CPAN-related deployment problems.

FWIW, Module::Build does support this sort of specificity:

prereqs => { 'Test::More', '== 0.55',
'Moose', '> 0.81, != 0.82',
}

-dave

/*============================================================
http://VegGuide.org http://blog.urth.org
Your guide to all that's veg House Absolute(ly Pointless)
============================================================*/

_______________________________________________
List: Catalyst[at]lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst[at]lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact lists@gossamer-threads.com