Mailing List Archive: Apache: Dev

Linking mod_ssl with a specific OpenSSL version (Re: svn commit: r1358167 - in /httpd/httpd/trunk: acinclude.m4 modules/ssl/ssl_engine_init.c)

1 2

View All

Index | Next | Previous | View Threaded

httpd-dev.2012 at velox

Aug 5, 2012, 1:10 AM

Post #1 of 31 (1226 views)
Permalink

Linking mod_ssl with a specific OpenSSL version (Re: svn commit: r1358167 - in /httpd/httpd/trunk: acinclude.m4 modules/ssl/ssl_engine_init.c)

On 08.07.2012 10:30, Kaspar Brand wrote:
> On 06.07.2012 14:41, ben [at] apache wrote:
>> Author: ben
>> Date: Fri Jul 6 12:41:10 2012
>> New Revision: 1358167
>>
>> URL: http://svn.apache.org/viewvc?rev=1358167&view=rev
>> Log:
>> Work correctly with a development version of OpenSSL. I suspect
>> something similar is needed when there are two OpenSSL installations,
>> one in a default location.

I had another look at this, since it has been proposed for backporting
to 2.4 in the meantime, and still think the following is true:

> If I'm understanding correctly, then this
> patch tries to support building against an OpenSSL source tree (or
> perhaps a build directory where only "make libs" has been executed)?

(should have read "make build_libs" instead)

It's a useful enhancement if mod_ssl can be linked with a specific
OpenSSL version in a non-default location, but the current approach has
at least one problem, AFAICT: it will only work if the directory pointed
to by --with-ssl does not include shared libraries for OpenSSL (by
default, OpenSSL only builds libssl.a and libcrypto.a, so the issue
might not be obvious at first sight).

> I would suggest to use a separate
> configure argument to support this build option, e.g. --with-ssl-srcdir.

I gave it a try, see the attached "work-in-progress" patch. While we're
at it, I think we should also fix a flaw in the handling of the
--with-ssl argument: in
http://svn.apache.org/viewvc?view=revision&revision=730926, acinclude.m4
was modified to always give pkg-config precedence over any argument
specified through --with-ssl. While the rationale for this change
becomes clear from the commit log, I consider it an unfortunate side
effect that pkg-config always trumps any --with-ssl directory argument.

My suggestion would be to handle OpenSSL paths in configure arguments
like this, instead:

1) use --with-ssl-builddir for linking with the static OpenSSL libraries
in that directory (and ignore --with-ssl in this case)

2) use --with-ssl for linking against an installed version of OpenSSL

3) use pkg-config to locate OpenSSL

Does that sound like a reasonable proposal? Comments welcome, and test
feedback would be much appreciated (remember to run "buildconf" after
applying the patch to acinclude.m4, and before calling configure).

Kaspar

Attachments:

mod_ssl-configure-options.patch (7.97 KB)

httpd-dev.2012 at velox

Aug 5, 2012, 2:36 AM

Post #2 of 31 (1206 views)
Permalink

Re: Linking mod_ssl with a specific OpenSSL version (Re: svn commit: r1358167 - in /httpd/httpd/trunk: acinclude.m4 modules/ssl/ssl_engine_init.c) [In reply to]

On 05.08.2012 10:10, Kaspar Brand wrote:
> test feedback would be much appreciated (remember to run "buildconf" after
> applying the patch to acinclude.m4, and before calling configure).

The patch attached to the previous message was missing an important
line, unfortunately (sorry to anybody who already played with it).

Apply the one below on top of mod_ssl-configure-options.patch, and
things should work properly.

Kaspar

Attachments:

mod_ssl-configure-options.incr.patch (0.29 KB)

fuankg at apache

Aug 5, 2012, 5:38 AM

Post #3 of 31 (1203 views)
Permalink

Re: Linking mod_ssl with a specific OpenSSL version (Re: svn commit: r1358167 - in /httpd/httpd/trunk: acinclude.m4 modules/ssl/ssl_engine_init.c) [In reply to]

Hi Kaspar,
Am 05.08.2012 10:10, schrieb Kaspar Brand:
> My suggestion would be to handle OpenSSL paths in configure arguments
> like this, instead:
>
> 1) use --with-ssl-builddir for linking with the static OpenSSL libraries
> in that directory (and ignore --with-ssl in this case)
what about splitting into two arguments:
--with-ssl-include=
--with-ssl-lib=
this would be equal to what many other configure also use ...

just a thought ...

Gün.

httpd-dev.2012 at velox

Aug 5, 2012, 10:10 PM

Post #4 of 31 (1206 views)
Permalink

Re: Linking mod_ssl with a specific OpenSSL version (Re: svn commit: r1358167 - in /httpd/httpd/trunk: acinclude.m4 modules/ssl/ssl_engine_init.c) [In reply to]

On 05.08.2012 14:38, Guenter Knauf wrote:
> Am 05.08.2012 10:10, schrieb Kaspar Brand:
>> 1) use --with-ssl-builddir for linking with the static OpenSSL libraries
>> in that directory (and ignore --with-ssl in this case)
> what about splitting into two arguments:
> --with-ssl-include=
> --with-ssl-lib=
> this would be equal to what many other configure also use ...

That's an option, yes, although the way the proposed option would work
doesn't mirror the typical case of --with-xyz-include/--with-xyz-lib
(i.e., add those args with -I and -L): "--with-ssl-builddir" forces
mod_ssl to always be linked with the static libraries in that directory.
Maybe --with-ssl-static-libdir would be a more appropriate name?

Kaspar

wrowe at rowe-clan

Aug 6, 2012, 1:08 PM

Post #5 of 31 (1201 views)
Permalink

Re: Linking mod_ssl with a specific OpenSSL version (Re: svn commit: r1358167 - in /httpd/httpd/trunk: acinclude.m4 modules/ssl/ssl_engine_init.c) [In reply to]

On 8/5/2012 10:10 PM, Kaspar Brand wrote:
> On 05.08.2012 14:38, Guenter Knauf wrote:
>> Am 05.08.2012 10:10, schrieb Kaspar Brand:
>>> 1) use --with-ssl-builddir for linking with the static OpenSSL libraries
>>> in that directory (and ignore --with-ssl in this case)
>> what about splitting into two arguments:
>> --with-ssl-include=
>> --with-ssl-lib=
>> this would be equal to what many other configure also use ...
>
> That's an option, yes, although the way the proposed option would work
> doesn't mirror the typical case of --with-xyz-include/--with-xyz-lib
> (i.e., add those args with -I and -L): "--with-ssl-builddir" forces
> mod_ssl to always be linked with the static libraries in that directory.
> Maybe --with-ssl-static-libdir would be a more appropriate name?

Why not simply consume the installed openssl.pc?

rainer.jung at kippdata

Aug 6, 2012, 2:36 PM

Post #6 of 31 (1194 views)
Permalink

Re: Linking mod_ssl with a specific OpenSSL version (Re: svn commit: r1358167 - in /httpd/httpd/trunk: acinclude.m4 modules/ssl/ssl_engine_init.c) [In reply to]

On 05.08.2012 10:10, Kaspar Brand wrote:
> On 08.07.2012 10:30, Kaspar Brand wrote:
>> On 06.07.2012 14:41, ben [at] apache wrote:
>>> Author: ben
>>> Date: Fri Jul 6 12:41:10 2012
>>> New Revision: 1358167
>>>
>>> URL: http://svn.apache.org/viewvc?rev=1358167&view=rev
>>> Log:
>>> Work correctly with a development version of OpenSSL. I suspect
>>> something similar is needed when there are two OpenSSL installations,
>>> one in a default location.
>
> I had another look at this, since it has been proposed for backporting
> to 2.4 in the meantime, and still think the following is true:
>
>> If I'm understanding correctly, then this
>> patch tries to support building against an OpenSSL source tree (or
>> perhaps a build directory where only "make libs" has been executed)?

That is my understanding as well.

> (should have read "make build_libs" instead)
>
> It's a useful enhancement if mod_ssl can be linked with a specific
> OpenSSL version in a non-default location, but the current approach has
> at least one problem, AFAICT: it will only work if the directory pointed
> to by --with-ssl does not include shared libraries for OpenSSL (by
> default, OpenSSL only builds libssl.a and libcrypto.a, so the issue
> might not be obvious at first sight).

Why wouldn't it work with shared libs? Because they usually have
dependencies themselves that are ignored?

>> I would suggest to use a separate
>> configure argument to support this build option, e.g. --with-ssl-srcdir.

We could, but I think if it would be easy to just add the logic to
with-ssl to also work with a src directory it would be easier to use. I
have no strong opinion on this though.

> I gave it a try, see the attached "work-in-progress" patch. While we're
> at it, I think we should also fix a flaw in the handling of the
> --with-ssl argument: in
> http://svn.apache.org/viewvc?view=revision&revision=730926, acinclude.m4
> was modified to always give pkg-config precedence over any argument
> specified through --with-ssl. While the rationale for this change
> becomes clear from the commit log, I consider it an unfortunate side
> effect that pkg-config always trumps any --with-ssl directory argument.
>
> My suggestion would be to handle OpenSSL paths in configure arguments
> like this, instead:
>
> 1) use --with-ssl-builddir for linking with the static OpenSSL libraries
> in that directory (and ignore --with-ssl in this case)
>
> 2) use --with-ssl for linking against an installed version of OpenSSL
>
> 3) use pkg-config to locate OpenSSL
>
> Does that sound like a reasonable proposal? Comments welcome, and test
> feedback would be much appreciated (remember to run "buildconf" after
> applying the patch to acinclude.m4, and before calling configure).

I do like the idea to give with-ssl priority over pkgconfig. If a user
chooses the ssl directory explicitly that should not be overwritten by
pkgconfig. Of course one can still use pkgconfig info in the chosen
directory to find dependency libs etc.

Whether 1) and 2) needs to be a separate priority or simply using the
same switch and auto-detect whether it is an installation directory or
only a build directory is undecided to me.

Regards,

Rainer

httpd-dev.2012 at velox

Aug 7, 2012, 10:39 PM

Post #7 of 31 (1195 views)
Permalink

Re: Linking mod_ssl with a specific OpenSSL version (Re: svn commit: r1358167 - in /httpd/httpd/trunk: acinclude.m4 modules/ssl/ssl_engine_init.c) [In reply to]

On 06.08.2012 22:08, William A. Rowe Jr. wrote:
> On 8/5/2012 10:10 PM, Kaspar Brand wrote:
>> On 05.08.2012 14:38, Guenter Knauf wrote:
>>> Am 05.08.2012 10:10, schrieb Kaspar Brand:
>>>> 1) use --with-ssl-builddir for linking with the static OpenSSL libraries
>>>> in that directory (and ignore --with-ssl in this case)
>>> what about splitting into two arguments:
>>> --with-ssl-include=
>>> --with-ssl-lib=
>>> this would be equal to what many other configure also use ...
>>
>> That's an option, yes, although the way the proposed option would work
>> doesn't mirror the typical case of --with-xyz-include/--with-xyz-lib
>> (i.e., add those args with -I and -L): "--with-ssl-builddir" forces
>> mod_ssl to always be linked with the static libraries in that directory.
>> Maybe --with-ssl-static-libdir would be a more appropriate name?
>
> Why not simply consume the installed openssl.pc?

In this particular case we're talking about using an OpenSSL build
directory, i.e. the generated openssl.pc is sitting in a temporary
location in that build tree (and when used, would output incorrect
includedir/libdir locations).

Kaspar

httpd-dev.2012 at velox

Aug 7, 2012, 11:00 PM

Post #8 of 31 (1191 views)
Permalink

Re: Linking mod_ssl with a specific OpenSSL version [In reply to]

On 06.08.2012 23:36, Rainer Jung wrote:
> On 05.08.2012 10:10, Kaspar Brand wrote:
>> It's a useful enhancement if mod_ssl can be linked with a specific
>> OpenSSL version in a non-default location, but the current approach has
>> at least one problem, AFAICT: it will only work if the directory pointed
>> to by --with-ssl does not include shared libraries for OpenSSL (by
>> default, OpenSSL only builds libssl.a and libcrypto.a, so the issue
>> might not be obvious at first sight).
>
> Why wouldn't it work with shared libs? Because they usually have
> dependencies themselves that are ignored?

Sorry, "will only work" is inaccurate... what I was (implicitly)
assuming is that the OpenSSL build directory would not be part of the
standard library search path - i.e., unless LD_LIBRARY_PATH is adapted,
httpd won't find the proper libraries at startup.

Forcing the linker to prefer the static over the shared libraries in a
particular directory (like I tried in the preliminary patch with the
"-Wl,-Bstatic,-lssl,-lcrypto,-Bdynamic" linker options) turns out to be
too linker/platform specific, however - I have abandoned this idea
meanwhile. I'm not sure what to really do about it... perhaps add the
OpenSSL builddir with "-R" and leave the rest to libtool?

>>> I would suggest to use a separate
>>> configure argument to support this build option, e.g. --with-ssl-srcdir.
>
> We could, but I think if it would be easy to just add the logic to
> with-ssl to also work with a src directory it would be easier to use. I
> have no strong opinion on this though.

My thinking was that people should explicitly tell configure that they
want to link with the libs in a build directory (so that they don't
"accidentally" use a directory which might only temporarily exist -
that's also the primary reason for preferring the static over the shared
libs from that dir).

> I do like the idea to give with-ssl priority over pkgconfig. If a user
> chooses the ssl directory explicitly that should not be overwritten by
> pkgconfig. Of course one can still use pkgconfig info in the chosen
> directory to find dependency libs etc.

OpenSSL doesn't have any further dependencies, AFAICT (cf.
http://cvs.openssl.org/fileview?f=openssl/Makefile.org and the
"Requires: " line it adds to openssl.pc). Are you referring to the
--libs-only-other option for pkg-config?

Kaspar

fuankg at apache

Aug 8, 2012, 2:47 AM

Post #9 of 31 (1189 views)
Permalink

Re: Linking mod_ssl with a specific OpenSSL version (Re: svn commit: r1358167 - in /httpd/httpd/trunk: acinclude.m4 modules/ssl/ssl_engine_init.c) [In reply to]

Am 08.08.2012 07:39, schrieb Kaspar Brand:
> On 06.08.2012 22:08, William A. Rowe Jr. wrote:
>> On 8/5/2012 10:10 PM, Kaspar Brand wrote:
>>> On 05.08.2012 14:38, Guenter Knauf wrote:
>>>> Am 05.08.2012 10:10, schrieb Kaspar Brand:
>>>>> 1) use --with-ssl-builddir for linking with the static OpenSSL libraries
>>>>> in that directory (and ignore --with-ssl in this case)
>>>> what about splitting into two arguments:
>>>> --with-ssl-include=
>>>> --with-ssl-lib=
>>>> this would be equal to what many other configure also use ...
>>>
>>> That's an option, yes, although the way the proposed option would work
>>> doesn't mirror the typical case of --with-xyz-include/--with-xyz-lib
>>> (i.e., add those args with -I and -L): "--with-ssl-builddir" forces
>>> mod_ssl to always be linked with the static libraries in that directory.
>>> Maybe --with-ssl-static-libdir would be a more appropriate name?
>>
>> Why not simply consume the installed openssl.pc?
>
> In this particular case we're talking about using an OpenSSL build
> directory, i.e. the generated openssl.pc is sitting in a temporary
> location in that build tree (and when used, would output incorrect
> includedir/libdir locations).
probably a dumb question and I'm missing something, but:
why is this necessary at all? Given someone wants to build/link against
a development version of OpenSSL why should he ommit the last step of
installing OpenSSL to a separate location? Wouldnt everything work fine
using --with-ssl= and building OpenSSL static-only + installing to a
separate development location?

Gün.

ben at links

Aug 8, 2012, 4:41 AM

Post #10 of 31 (1188 views)
Permalink

Re: Linking mod_ssl with a specific OpenSSL version (Re: svn commit: r1358167 - in /httpd/httpd/trunk: acinclude.m4 modules/ssl/ssl_engine_init.c) [In reply to]

On Sun, Aug 5, 2012 at 1:10 AM, Kaspar Brand <httpd-dev.2012 [at] velox> wrote:
> On 08.07.2012 10:30, Kaspar Brand wrote:
>> On 06.07.2012 14:41, ben [at] apache wrote:
>>> Author: ben
>>> Date: Fri Jul 6 12:41:10 2012
>>> New Revision: 1358167
>>>
>>> URL: http://svn.apache.org/viewvc?rev=1358167&view=rev
>>> Log:
>>> Work correctly with a development version of OpenSSL. I suspect
>>> something similar is needed when there are two OpenSSL installations,
>>> one in a default location.
>
> I had another look at this, since it has been proposed for backporting
> to 2.4 in the meantime, and still think the following is true:
>
>> If I'm understanding correctly, then this
>> patch tries to support building against an OpenSSL source tree (or
>> perhaps a build directory where only "make libs" has been executed)?
>
> (should have read "make build_libs" instead)
>
> It's a useful enhancement if mod_ssl can be linked with a specific
> OpenSSL version in a non-default location, but the current approach has
> at least one problem, AFAICT: it will only work if the directory pointed
> to by --with-ssl does not include shared libraries for OpenSSL (by
> default, OpenSSL only builds libssl.a and libcrypto.a, so the issue
> might not be obvious at first sight).

Hmm! This worked for me with shared libraries. Strange.

>> I would suggest to use a separate
>> configure argument to support this build option, e.g. --with-ssl-srcdir.
>
> I gave it a try, see the attached "work-in-progress" patch. While we're
> at it, I think we should also fix a flaw in the handling of the
> --with-ssl argument: in
> http://svn.apache.org/viewvc?view=revision&revision=730926, acinclude.m4
> was modified to always give pkg-config precedence over any argument
> specified through --with-ssl. While the rationale for this change
> becomes clear from the commit log, I consider it an unfortunate side
> effect that pkg-config always trumps any --with-ssl directory argument.
>
> My suggestion would be to handle OpenSSL paths in configure arguments
> like this, instead:
>
> 1) use --with-ssl-builddir for linking with the static OpenSSL libraries
> in that directory (and ignore --with-ssl in this case)
>
> 2) use --with-ssl for linking against an installed version of OpenSSL
>
> 3) use pkg-config to locate OpenSSL
>
> Does that sound like a reasonable proposal? Comments welcome, and test
> feedback would be much appreciated (remember to run "buildconf" after
> applying the patch to acinclude.m4, and before calling configure).

Sounds OK to me.

>
> Kaspar
>

ben at links

Aug 8, 2012, 4:45 AM

Post #11 of 31 (1188 views)
Permalink

Re: Linking mod_ssl with a specific OpenSSL version (Re: svn commit: r1358167 - in /httpd/httpd/trunk: acinclude.m4 modules/ssl/ssl_engine_init.c) [In reply to]

On Wed, Aug 8, 2012 at 2:47 AM, Guenter Knauf <fuankg [at] apache> wrote:
> Am 08.08.2012 07:39, schrieb Kaspar Brand:
>
>> On 06.08.2012 22:08, William A. Rowe Jr. wrote:
>>>
>>> On 8/5/2012 10:10 PM, Kaspar Brand wrote:
>>>>
>>>> On 05.08.2012 14:38, Guenter Knauf wrote:
>>>>>
>>>>> Am 05.08.2012 10:10, schrieb Kaspar Brand:
>>>>>>
>>>>>> 1) use --with-ssl-builddir for linking with the static OpenSSL
>>>>>> libraries
>>>>>> in that directory (and ignore --with-ssl in this case)
>>>>>
>>>>> what about splitting into two arguments:
>>>>> --with-ssl-include=
>>>>> --with-ssl-lib=
>>>>> this would be equal to what many other configure also use ...
>>>>
>>>>
>>>> That's an option, yes, although the way the proposed option would work
>>>> doesn't mirror the typical case of --with-xyz-include/--with-xyz-lib
>>>> (i.e., add those args with -I and -L): "--with-ssl-builddir" forces
>>>> mod_ssl to always be linked with the static libraries in that directory.
>>>> Maybe --with-ssl-static-libdir would be a more appropriate name?
>>>
>>>
>>> Why not simply consume the installed openssl.pc?
>>
>>
>> In this particular case we're talking about using an OpenSSL build
>> directory, i.e. the generated openssl.pc is sitting in a temporary
>> location in that build tree (and when used, would output incorrect
>> includedir/libdir locations).
>
> probably a dumb question and I'm missing something, but:
> why is this necessary at all? Given someone wants to build/link against a
> development version of OpenSSL why should he ommit the last step of
> installing OpenSSL to a separate location?

Time taken to iterate in development is an important determiner of
productivity. Having to do an install for every change introduces
significant delay. It also has a tendency to make debugging hard.

> Wouldnt everything work fine
> using --with-ssl= and building OpenSSL static-only + installing to a
> separate development location?
>
> G�n.
>
>

jorton at redhat

Aug 8, 2012, 9:03 AM

Post #12 of 31 (1194 views)
Permalink

Re: Linking mod_ssl with a specific OpenSSL version [In reply to]

On Wed, Aug 08, 2012 at 08:00:25AM +0200, Kaspar Brand wrote:
> My thinking was that people should explicitly tell configure that they
> want to link with the libs in a build directory (so that they don't
> "accidentally" use a directory which might only temporarily exist -
> that's also the primary reason for preferring the static over the shared
> libs from that dir).

This all seems totally crazy to me. Why are we adding complexity to the
httpd build system so openssl devs can skip typing the " install" part
of running "make install" when testing against modified versions of
OpenSSL? Joe

noel.butler at ausics

Aug 8, 2012, 4:06 PM

Post #13 of 31 (1194 views)
Permalink

Re: Linking mod_ssl with a specific OpenSSL version [In reply to]

On Wed, 2012-08-08 at 17:03 +0100, Joe Orton wrote:

> On Wed, Aug 08, 2012 at 08:00:25AM +0200, Kaspar Brand wrote:
> > My thinking was that people should explicitly tell configure that they
> > want to link with the libs in a build directory (so that they don't
> > "accidentally" use a directory which might only temporarily exist -
> > that's also the primary reason for preferring the static over the shared
> > libs from that dir).
>
> This all seems totally crazy to me. Why are we adding complexity to the
> httpd build system so openssl devs can skip typing the " install" part
> of running "make install" when testing against modified versions of
> OpenSSL? Joe

+1

Attachments:

signature.asc (0.48 KB)

ben at links

Aug 8, 2012, 8:56 PM

Post #14 of 31 (1183 views)
Permalink

Re: Linking mod_ssl with a specific OpenSSL version [In reply to]

On Wed, Aug 8, 2012 at 5:03 PM, Joe Orton <jorton [at] redhat> wrote:
> On Wed, Aug 08, 2012 at 08:00:25AM +0200, Kaspar Brand wrote:
>> My thinking was that people should explicitly tell configure that they
>> want to link with the libs in a build directory (so that they don't
>> "accidentally" use a directory which might only temporarily exist -
>> that's also the primary reason for preferring the static over the shared
>> libs from that dir).
>
> This all seems totally crazy to me. Why are we adding complexity to the
> httpd build system so openssl devs can skip typing the " install" part
> of running "make install" when testing against modified versions of
> OpenSSL? Joe

IIRC, the issue is that it doesn't build correctly even if you do that
(to a non-standard location - and clearly installing to a standard
location is nuts).

The only extra complexity introduced by allowing a dev to skip "make
install" is that the lib is in <base> instead of <base>/lib.

jorton at redhat

Aug 9, 2012, 1:42 AM

Post #15 of 31 (1189 views)
Permalink

Re: Linking mod_ssl with a specific OpenSSL version [In reply to]

On Thu, Aug 09, 2012 at 04:56:03AM +0100, Ben Laurie wrote:
> On Wed, Aug 8, 2012 at 5:03 PM, Joe Orton <jorton [at] redhat> wrote:
> > This all seems totally crazy to me. Why are we adding complexity to the
> > httpd build system so openssl devs can skip typing the " install" part
> > of running "make install" when testing against modified versions of
> > OpenSSL? Joe
>
> IIRC, the issue is that it doesn't build correctly even if you do that
> (to a non-standard location - and clearly installing to a standard
> location is nuts).

Doesn't build correctly... in what way? I've done that loads of times
building against OpenSSL from CVS. All the code is still there so it
better work or else we need to see bug reports.

Regards, Joe

ben at links

Aug 9, 2012, 3:25 AM

Post #16 of 31 (1186 views)
Permalink

Re: Linking mod_ssl with a specific OpenSSL version [In reply to]

On Thu, Aug 9, 2012 at 9:42 AM, Joe Orton <jorton [at] redhat> wrote:
> On Thu, Aug 09, 2012 at 04:56:03AM +0100, Ben Laurie wrote:
>> On Wed, Aug 8, 2012 at 5:03 PM, Joe Orton <jorton [at] redhat> wrote:
>> > This all seems totally crazy to me. Why are we adding complexity to the
>> > httpd build system so openssl devs can skip typing the " install" part
>> > of running "make install" when testing against modified versions of
>> > OpenSSL? Joe
>>
>> IIRC, the issue is that it doesn't build correctly even if you do that
>> (to a non-standard location - and clearly installing to a standard
>> location is nuts).
>
> Doesn't build correctly... in what way? I've done that loads of times
> building against OpenSSL from CVS. All the code is still there so it
> better work or else we need to see bug reports.

In that it uses (parts of) the default install instead of the install
you specify. At least for some versions. I forget the details coz I
fixed it...

>
> Regards, Joe

wrowe at rowe-clan

Aug 9, 2012, 4:29 PM

Post #17 of 31 (1167 views)
Permalink

Re: Linking mod_ssl with a specific OpenSSL version [In reply to]

On 8/8/2012 9:03 AM, Joe Orton wrote:
> On Wed, Aug 08, 2012 at 08:00:25AM +0200, Kaspar Brand wrote:
>> My thinking was that people should explicitly tell configure that they
>> want to link with the libs in a build directory (so that they don't
>> "accidentally" use a directory which might only temporarily exist -
>> that's also the primary reason for preferring the static over the shared
>> libs from that dir).
>
> This all seems totally crazy to me. Why are we adding complexity to the
> httpd build system so openssl devs can skip typing the " install" part
> of running "make install" when testing against modified versions of
> OpenSSL? Joe

I concur.

If lib devs wish to use a build tree, it should not be that hard to
properly craft the right symlinks for an 'install in place' model
used for rapid development.

I agree with Ben that this is desireable, but httpd is not the place
to solve it. openssl.pc can describe such an environment in a useful
manner for developers and solve the entire issue in the lib package.

wrowe at rowe-clan

Aug 9, 2012, 4:31 PM

Post #18 of 31 (1171 views)
Permalink

Re: Linking mod_ssl with a specific OpenSSL version [In reply to]

On 8/8/2012 8:56 PM, Ben Laurie wrote:
> On Wed, Aug 8, 2012 at 5:03 PM, Joe Orton <jorton [at] redhat> wrote:
>> On Wed, Aug 08, 2012 at 08:00:25AM +0200, Kaspar Brand wrote:
>>> My thinking was that people should explicitly tell configure that they
>>> want to link with the libs in a build directory (so that they don't
>>> "accidentally" use a directory which might only temporarily exist -
>>> that's also the primary reason for preferring the static over the shared
>>> libs from that dir).
>>
>> This all seems totally crazy to me. Why are we adding complexity to the
>> httpd build system so openssl devs can skip typing the " install" part
>> of running "make install" when testing against modified versions of
>> OpenSSL? Joe
>
> IIRC, the issue is that it doesn't build correctly even if you do that
> (to a non-standard location - and clearly installing to a standard
> location is nuts).

I have no issues installing to a non-standard location, and with a very
few hacks, can even eliminate the -R compiled-in search paths and create
a build entirely managed by LD_LIBRARY_PATH.

It would help if you explained better what bug you see in handling the
--with-ssl arguement, and I think we could resolve this easily. If the
issue is that openssl.pc is not processed correctly by httpd or not
created correctly by openssl.org, then we can fix either defect.

wrowe at rowe-clan

Aug 9, 2012, 4:55 PM

Post #19 of 31 (1168 views)
Permalink

Re: Linking mod_ssl with a specific OpenSSL version (Re: svn commit: r1358167 - in /httpd/httpd/trunk: acinclude.m4 modules/ssl/ssl_engine_init.c) [In reply to]

On 8/8/2012 4:45 AM, Ben Laurie wrote:
> On Wed, Aug 8, 2012 at 2:47 AM, Guenter Knauf <fuankg [at] apache> wrote:
>> Am 08.08.2012 07:39, schrieb Kaspar Brand:
>>
>>> On 06.08.2012 22:08, William A. Rowe Jr. wrote:
>>>>
>>>> On 8/5/2012 10:10 PM, Kaspar Brand wrote:
>>>>>
>>>>> On 05.08.2012 14:38, Guenter Knauf wrote:
>>>>>>
>>>>>> Am 05.08.2012 10:10, schrieb Kaspar Brand:
>>>>>>>
>>>>>>> 1) use --with-ssl-builddir for linking with the static OpenSSL
>>>>>>> libraries
>>>>>>> in that directory (and ignore --with-ssl in this case)
>>>>>>
>>>>>> what about splitting into two arguments:
>>>>>> --with-ssl-include=
>>>>>> --with-ssl-lib=
>>>>>> this would be equal to what many other configure also use ...
>>>>>
>>>>>
>>>>> That's an option, yes, although the way the proposed option would work
>>>>> doesn't mirror the typical case of --with-xyz-include/--with-xyz-lib
>>>>> (i.e., add those args with -I and -L): "--with-ssl-builddir" forces
>>>>> mod_ssl to always be linked with the static libraries in that directory.
>>>>> Maybe --with-ssl-static-libdir would be a more appropriate name?
>>>>
>>>>
>>>> Why not simply consume the installed openssl.pc?
>>>
>>>
>>> In this particular case we're talking about using an OpenSSL build
>>> directory, i.e. the generated openssl.pc is sitting in a temporary
>>> location in that build tree (and when used, would output incorrect
>>> includedir/libdir locations).
>>
>> probably a dumb question and I'm missing something, but:
>> why is this necessary at all? Given someone wants to build/link against a
>> development version of OpenSSL why should he ommit the last step of
>> installing OpenSSL to a separate location?
>
> Time taken to iterate in development is an important determiner of
> productivity. Having to do an install for every change introduces
> significant delay. It also has a tendency to make debugging hard.

I agree.

An openssl 'make localinstall' could trivially create the lib, include
trees consisting entirely of symlinks to the origin files in the same
build tree, and create an appropriate openssl.pc file describing the
link steps required to

Rather than solving 'for the httpd case', this would allow you to build
most any app or library depending upon openssl in a sensible way, in
order to diagnose openssl's interoperation with any consumer.

This really isn't the repository to solve openssl's contorted build
issues.

httpd-dev.2012 at velox

Aug 12, 2012, 9:23 AM

Post #20 of 31 (1161 views)
Permalink

Re: Linking mod_ssl with a specific OpenSSL version [In reply to]

On 10.08.2012 01:55, William A. Rowe Jr. wrote:
> An openssl 'make localinstall' could trivially create the lib, include
> trees consisting entirely of symlinks to the origin files in the same
> build tree, and create an appropriate openssl.pc file describing the
> link steps required to

Specifically, that "lib" subdirectory would have to be populated like so:

lib
|-- engines
| |-- lib4758cca.so
| |-- ...
| `-- libubsec.so
|-- pkgconfig
| |-- libcrypto.pc
| |-- libssl.pc
| `-- openssl.pc
|-- libcrypto.a
|-- libcrypto.so -> libcrypto.so.1.0.0
|-- libcrypto.so.1.0.0
|-- libssl.a
|-- libssl.so -> libssl.so.1.0.0
`-- libssl.so.1.0.0

(If pkg-config is in $PATH, but lib/pkgconfig/openssl.pc is missing,
then httpd's configure will pick the "default" installation of OpenSSL,
and ignore the directory specified through --with-ssl.)

> Rather than solving 'for the httpd case', this would allow you to build
> most any app or library depending upon openssl in a sensible way, in
> order to diagnose openssl's interoperation with any consumer.
>
> This really isn't the repository to solve openssl's contorted build
> issues.

I tend to agree, and would be in favor of reverting r1358167. As long as
OpenSSL doesn't provide a Makefile target for creating the "lib"
subdirectory in the build tree, a workaround is to call configure with
suitable {CPP,LD}FLAGS, i.e.

CPPFLAGS=-I${openssl_build_dir}/include \
LDFLAGS=-L${openssl_build_dir} \
./configure ...

(when using the shared libssl/libcrypto libraries, adding
"-Wl,-R${openssl_build_dir}" or similar to LDFLAGS might make sense)

One problem with the current version of acinclude.m4 I noticed in the
meantime is that on OS X, the test in line 503
(https://svn.apache.org/viewvc/httpd/httpd/trunk/acinclude.m4?view=markup&pathrev=1358167#l503)
is true whenever --with-ssl is missing from the configure arguments (OS
X doesn't have /lib). configure will then add bogus "-L" strings to
SSL_LIBS and EXTRA_LDFLAGS in build/config_vars.mk... and this breaks
the link commands for the whole tree ("libtool: link: require no space
between `-L' and `-o'").

Kaspar

ben at links

Aug 12, 2012, 11:01 AM

Post #21 of 31 (1156 views)
Permalink

Re: Linking mod_ssl with a specific OpenSSL version [In reply to]

On Sun, Aug 12, 2012 at 5:23 PM, Kaspar Brand <httpd-dev.2012 [at] velox> wrote:
> On 10.08.2012 01:55, William A. Rowe Jr. wrote:
>> An openssl 'make localinstall' could trivially create the lib, include
>> trees consisting entirely of symlinks to the origin files in the same
>> build tree, and create an appropriate openssl.pc file describing the
>> link steps required to
>
> Specifically, that "lib" subdirectory would have to be populated like so:
>
> lib
> |-- engines
> | |-- lib4758cca.so
> | |-- ...
> | `-- libubsec.so
> |-- pkgconfig
> | |-- libcrypto.pc
> | |-- libssl.pc
> | `-- openssl.pc
> |-- libcrypto.a
> |-- libcrypto.so -> libcrypto.so.1.0.0
> |-- libcrypto.so.1.0.0
> |-- libssl.a
> |-- libssl.so -> libssl.so.1.0.0
> `-- libssl.so.1.0.0
>
> (If pkg-config is in $PATH, but lib/pkgconfig/openssl.pc is missing,
> then httpd's configure will pick the "default" installation of OpenSSL,
> and ignore the directory specified through --with-ssl.)
>
>> Rather than solving 'for the httpd case', this would allow you to build
>> most any app or library depending upon openssl in a sensible way, in
>> order to diagnose openssl's interoperation with any consumer.
>>
>> This really isn't the repository to solve openssl's contorted build
>> issues.
>
> I tend to agree, and would be in favor of reverting r1358167. As long as
> OpenSSL doesn't provide a Makefile target for creating the "lib"
> subdirectory in the build tree, a workaround is to call configure with
> suitable {CPP,LD}FLAGS, i.e.
>
> CPPFLAGS=-I${openssl_build_dir}/include \
> LDFLAGS=-L${openssl_build_dir} \
> ./configure ...
>
> (when using the shared libssl/libcrypto libraries, adding
> "-Wl,-R${openssl_build_dir}" or similar to LDFLAGS might make sense)

I haven't had time to retest it, but I think the problem with this
approach is that the default version of OpenSSL gets included first if
anything else uses /usr/local/{include,lib}. IIRC, this was the basis
of all my problems.

>
> One problem with the current version of acinclude.m4 I noticed in the
> meantime is that on OS X, the test in line 503
> (https://svn.apache.org/viewvc/httpd/httpd/trunk/acinclude.m4?view=markup&pathrev=1358167#l503)
> is true whenever --with-ssl is missing from the configure arguments (OS
> X doesn't have /lib). configure will then add bogus "-L" strings to
> SSL_LIBS and EXTRA_LDFLAGS in build/config_vars.mk... and this breaks
> the link commands for the whole tree ("libtool: link: require no space
> between `-L' and `-o'").
>
> Kaspar

httpd-dev.2012 at velox

Aug 16, 2012, 11:36 AM

Post #22 of 31 (1101 views)
Permalink

Re: Linking mod_ssl with a specific OpenSSL version [In reply to]

On 12.8.12 20:01, Ben Laurie wrote:
> On Sun, Aug 12, 2012 at 5:23 PM, Kaspar Brand <httpd-dev.2012 [at] velox> wrote:
>> a workaround is to call configure with
>> suitable {CPP,LD}FLAGS, i.e.
>>
>> CPPFLAGS=-I${openssl_build_dir}/include \
>> LDFLAGS=-L${openssl_build_dir} \
>> ./configure ...
>>
>> (when using the shared libssl/libcrypto libraries, adding
>> "-Wl,-R${openssl_build_dir}" or similar to LDFLAGS might make sense)
>
> I haven't had time to retest it, but I think the problem with this
> approach is that the default version of OpenSSL gets included first if
> anything else uses /usr/local/{include,lib}. IIRC, this was the basis
> of all my problems.

Ok, but then we're talking about a current limitation of the build
system, I think: the compile commands are put together in build/rules.mk
like this:

> ALL_CFLAGS = $(EXTRA_CFLAGS) $(NOTEST_CFLAGS) $(CFLAGS)
> ALL_CPPFLAGS = $(DEFS) $(INTERNAL_CPPFLAGS) $(EXTRA_CPPFLAGS) $(NOTEST_CPPFLAGS) $(CPPFLAGS)
> [...]
> ALL_INCLUDES = $(INCLUDES) $(EXTRA_INCLUDES)
>
> # Compile commands
>
> BASE_CC = $(CC) $(ALL_CFLAGS) $(ALL_CPPFLAGS) $(ALL_INCLUDES)

Both $EXTRA_CPPFLAGS and $EXTRA_INCLUDES are assembled by configure,
where each module can contribute its stuff (appended with APR_ADDTO,
usually). I.e., if you happen to configure mod_deflate with a zlib
in /usr/local, then you end up with $EXTRA_INCLUDES where mod_deflate's
-I/usr/local precedes -I/path/to/openssl/build/dir (since
modules/filter/config.m4 is run before modules/ssl/config.m4).

Similarly, for linking we have:

> ALL_LDFLAGS = $(EXTRA_LDFLAGS) $(NOTEST_LDFLAGS) $(LDFLAGS)
> [...]
> LINK = $(LIBTOOL) --mode=link $(CC) $(ALL_CFLAGS) $(LT_LDFLAGS) $(ALL_LDFLAGS) -o $@

I wonder if we should add support for module-specific CFLAGS etc.,
which would always appear before the EXTRA_XXX stuff in the compile
and link commands, i.e. in rules.mk we would have:

ALL_CFLAGS = $(MOD_CFLAGS) $(EXTRA_CFLAGS) $(NOTEST_CFLAGS) $(CFLAGS)
ALL_CPPFLAGS = $(DEFS) $(INTERNAL_CPPFLAGS) $(MOD_CPPFLAGS) $(EXTRA_CPPFLAGS) $(NOTEST_CPPFLAGS) $(CPPFLAGS)
ALL_INCLUDES = $(INCLUDES) $(MOD_INCLUDES) $(EXTRA_INCLUDES)

ALL_LDFLAGS = $(MOD_LDFLAGS) $(EXTRA_LDFLAGS) $(NOTEST_LDFLAGS) $(LDFLAGS)

A particular module could then set its specific MOD_CFLAGS etc. in
modules.mk, and these would always have priority over those possibly
inserted by other modules.

Kaspar

jorton at redhat

Aug 17, 2012, 1:11 AM

Post #23 of 31 (1092 views)
Permalink

Re: Linking mod_ssl with a specific OpenSSL version [In reply to]

On Thu, Aug 16, 2012 at 08:36:31PM +0200, Kaspar Brand wrote:
> I wonder if we should add support for module-specific CFLAGS etc.,
> which would always appear before the EXTRA_XXX stuff in the compile
> and link commands, i.e. in rules.mk we would have:
>
> ALL_CFLAGS = $(MOD_CFLAGS) $(EXTRA_CFLAGS) $(NOTEST_CFLAGS) $(CFLAGS)
> ALL_CPPFLAGS = $(DEFS) $(INTERNAL_CPPFLAGS) $(MOD_CPPFLAGS) $(EXTRA_CPPFLAGS) $(NOTEST_CPPFLAGS) $(CPPFLAGS)
> ALL_INCLUDES = $(INCLUDES) $(MOD_INCLUDES) $(EXTRA_INCLUDES)
>
> ALL_LDFLAGS = $(MOD_LDFLAGS) $(EXTRA_LDFLAGS) $(NOTEST_LDFLAGS) $(LDFLAGS)
>
> A particular module could then set its specific MOD_CFLAGS etc. in
> modules.mk, and these would always have priority over those possibly
> inserted by other modules.

Doing CFLAGS et al like that doesn't generalise brilliantly, because
they are per-directory (modules/xxx) not strictly per-module, but it
could be done anyway, and that wouldn't matter for mod_ssl. Yeah,
probably a good idea.

It should be possible to override LDFLAGS truly per-module by tweaking
the SH_LINK line generated in modules.mk.

Regards, Joe

ben at links

Aug 20, 2012, 5:19 AM

Post #24 of 31 (1033 views)
Permalink

Re: Linking mod_ssl with a specific OpenSSL version [In reply to]

On Thu, Aug 16, 2012 at 7:36 PM, Kaspar Brand <httpd-dev.2012 [at] velox> wrote:
> On 12.8.12 20:01, Ben Laurie wrote:
>> On Sun, Aug 12, 2012 at 5:23 PM, Kaspar Brand <httpd-dev.2012 [at] velox> wrote:
>>> a workaround is to call configure with
>>> suitable {CPP,LD}FLAGS, i.e.
>>>
>>> CPPFLAGS=-I${openssl_build_dir}/include \
>>> LDFLAGS=-L${openssl_build_dir} \
>>> ./configure ...
>>>
>>> (when using the shared libssl/libcrypto libraries, adding
>>> "-Wl,-R${openssl_build_dir}" or similar to LDFLAGS might make sense)
>>
>> I haven't had time to retest it, but I think the problem with this
>> approach is that the default version of OpenSSL gets included first if
>> anything else uses /usr/local/{include,lib}. IIRC, this was the basis
>> of all my problems.
>
> Ok, but then we're talking about a current limitation of the build
> system, I think: the compile commands are put together in build/rules.mk
> like this:
>
>> ALL_CFLAGS = $(EXTRA_CFLAGS) $(NOTEST_CFLAGS) $(CFLAGS)
>> ALL_CPPFLAGS = $(DEFS) $(INTERNAL_CPPFLAGS) $(EXTRA_CPPFLAGS) $(NOTEST_CPPFLAGS) $(CPPFLAGS)
>> [...]
>> ALL_INCLUDES = $(INCLUDES) $(EXTRA_INCLUDES)
>>
>> # Compile commands
>>
>> BASE_CC = $(CC) $(ALL_CFLAGS) $(ALL_CPPFLAGS) $(ALL_INCLUDES)
>
> Both $EXTRA_CPPFLAGS and $EXTRA_INCLUDES are assembled by configure,
> where each module can contribute its stuff (appended with APR_ADDTO,
> usually). I.e., if you happen to configure mod_deflate with a zlib
> in /usr/local, then you end up with $EXTRA_INCLUDES where mod_deflate's
> -I/usr/local precedes -I/path/to/openssl/build/dir (since
> modules/filter/config.m4 is run before modules/ssl/config.m4).
>
> Similarly, for linking we have:
>
>> ALL_LDFLAGS = $(EXTRA_LDFLAGS) $(NOTEST_LDFLAGS) $(LDFLAGS)
>> [...]
>> LINK = $(LIBTOOL) --mode=link $(CC) $(ALL_CFLAGS) $(LT_LDFLAGS) $(ALL_LDFLAGS) -o $@
>
> I wonder if we should add support for module-specific CFLAGS etc.,
> which would always appear before the EXTRA_XXX stuff in the compile
> and link commands, i.e. in rules.mk we would have:
>
> ALL_CFLAGS = $(MOD_CFLAGS) $(EXTRA_CFLAGS) $(NOTEST_CFLAGS) $(CFLAGS)
> ALL_CPPFLAGS = $(DEFS) $(INTERNAL_CPPFLAGS) $(MOD_CPPFLAGS) $(EXTRA_CPPFLAGS) $(NOTEST_CPPFLAGS) $(CPPFLAGS)
> ALL_INCLUDES = $(INCLUDES) $(MOD_INCLUDES) $(EXTRA_INCLUDES)
>
> ALL_LDFLAGS = $(MOD_LDFLAGS) $(EXTRA_LDFLAGS) $(NOTEST_LDFLAGS) $(LDFLAGS)
>
> A particular module could then set its specific MOD_CFLAGS etc. in
> modules.mk, and these would always have priority over those possibly
> inserted by other modules.

This sounds like it would solve my problem - and seems like about as a
good a solution as we can expect :-)

>
> Kaspar

httpd-dev.2012 at velox

Aug 23, 2012, 12:22 AM

Post #25 of 31 (1016 views)
Permalink

Re: Linking mod_ssl with a specific OpenSSL version [In reply to]

On 17.08.2012 10:11, Joe Orton wrote:
> On Thu, Aug 16, 2012 at 08:36:31PM +0200, Kaspar Brand wrote:
>> I wonder if we should add support for module-specific CFLAGS etc.,
>> which would always appear before the EXTRA_XXX stuff in the compile
>> and link commands, i.e. in rules.mk we would have:
>>
>> ALL_CFLAGS = $(MOD_CFLAGS) $(EXTRA_CFLAGS) $(NOTEST_CFLAGS) $(CFLAGS)
>> ALL_CPPFLAGS = $(DEFS) $(INTERNAL_CPPFLAGS) $(MOD_CPPFLAGS) $(EXTRA_CPPFLAGS) $(NOTEST_CPPFLAGS) $(CPPFLAGS)
>> ALL_INCLUDES = $(INCLUDES) $(MOD_INCLUDES) $(EXTRA_INCLUDES)
>>
>> ALL_LDFLAGS = $(MOD_LDFLAGS) $(EXTRA_LDFLAGS) $(NOTEST_LDFLAGS) $(LDFLAGS)
>>
>> A particular module could then set its specific MOD_CFLAGS etc. in
>> modules.mk, and these would always have priority over those possibly
>> inserted by other modules.
>
> Doing CFLAGS et al like that doesn't generalise brilliantly, because
> they are per-directory (modules/xxx) not strictly per-module, but it
> could be done anyway, and that wouldn't matter for mod_ssl. Yeah,
> probably a good idea.

I gave it a try, and so far it seems to work as expected, see the
attached patch (against r1358166, to reduce clutter). Right now the
following modules are affected:

- modules/cache: mod_socache_dc (--with-distcache)

- modules/filters: mod_deflate (--with-z), mod_xml2enc (--with-libxml2),
mod_proxy_html (--with-libxml2)

- modules/lua: mod_lua (--with-lua, --enable-luajit)

- modules/ssl: mod_ssl (--with-ssl)

I.e. there's currently only a potential clash in modules/filters.

> It should be possible to override LDFLAGS truly per-module by tweaking
> the SH_LINK line generated in modules.mk.

I didn't pursue this option for the time being, as it would currently
only be of potential benefit for mod_deflate and
mod_xml2enc/mod_proxy_html. If you (or other devs) think it's an
important aspect, please let me know and I'll have a closer look.

Feedback and comments about the proposed approach - especially from
build system experts - is very much welcome.

Kaspar

Attachments:

MOD_flags.patch (10.9 KB)

1 2

View All

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads