Mailing List Archive: GnuPG: devel

Re: Adding support for z/OS in gnupg and related libraries

Index | Next | Previous | View Threaded

wk at gnupg

Jul 30, 2009, 6:58 AM

Post #1 of 3 (679 views)
Permalink

Re: Adding support for z/OS in gnupg and related libraries

On Wed, 29 Jul 2009 20:40, amul.shah [at] fnis said:

> Please excuse the cross-post. Some of the hoops that I jumped through

gnupg-devel is fine. We can drop gcrypt-devel.

> gnupg 1.4.9
> libgpg-errors 1.7
> libgcrypt 1.4.4
> libgpgme 1.1.8

Is there a reason why you work on 1.4.x and libgcrypt? Libgcrypt is
only required for 2.x.

> Services. The platform's native character encoding (aka code page or
> code set) is not ASCII, but EBCDIC. The US dialect of EBCDIC is

FWIW, once upon a time gpg had some limited support for EBCDIC but that
was later dropped.

> USS program are compiled as 31bit EBCDIC with no Unicode capabilities.

What does USS mean in this context?

> autotools is weak. Where I wanted to build shared libraries, I compiled
> archives. I unpacked the archives and converted them to shared libs

Support for shared libraries needs to be added to libtool; that is a
wrapper to abstarct the building of shared libs. It is quite possible
that it does not yet support zOS - I have not checked.

> Configuration options:
> ./configure CC=xlc CFLAGS="-qchars=signed -qascii -q64 -qlanglvl=extc99
> -qexportall -qrent -qnocsect -W l,DLL -D_XOPEN_SOURCE=600
> -D_ENHANCED_ASCII_EXT=0xFFFFFFFF -D_IEEEV1_COMPATIBILITY
> -D_OPEN_MSGQ_EXT" LD=xlc LDFLAGS="-qascii -q64 -W l,DLL" CXX=xlc++
> --enable-shared --prefix=/usr/local

In general those required options should go into configure.ac. We can
test there for the zOS and apply specific options.

What is the reason that you used CC=xls? Is there another C compiler
which can't be used? The configure script should be able to detect the
compiler.

> There is an issue with the xlc configuration that defaults to picking up
> include files from '.' (current working directory) and /usr/include
> before picking up command line options. Since the z/OS system has two
> headers with the same name as headers in libgcrypt, we link them into

Doesn't xlc support the -I option? "-I ." should pick up the packages
header files first.

> -- gnupg-1.4.9
> Apply the attached patch gnupg-1.4.9.patch which applies to gnupg
> 1.4.9. I will need help migrating this to the v2 line of gnupg

Is your target GnuPG-1.4 or GnuPG-2? If at all possible I would suggest
to target GnuPG-2.

> Configuration options:
> ./configure CC=xlc CFLAGS="-qchars=signed -qascii -q64 -qlanglvl=extc99
> -qexportall -qrent -qnocsect -W l,DLL -D_XOPEN_SOURCE=600
> -D_ENHANCED_ASCII_EXT=0xFFFFFFFF -D_IEEEV1_COMPATIBILITY
> -D_OPEN_MSGQ_EXT" LD=xlc LDFLAGS="-qascii -q64 -W l,DLL" CXX=xlc++
> --without-pth --without-libassuan --without-ksba --prefix=/usr/local

The --without-* options are not needed but they don't harm for 1.4.
--prefix=/usr/local is anyway the default.

> -- Generating Shared Libs from archives

Weel, we need to look into libtool.

> diff -purN gnupg-1.4.9.orig/checks/conventional-mdc.test gnupg-1.4.9.working-20090430/checks/conventional-mdc.test
> --- gnupg-1.4.9.orig/checks/conventional-mdc.test Tue Oct 23 04:53:20 2007
> +++ gnupg-1.4.9.working-20090430/checks/conventional-mdc.test Wed Apr 29 16:29:29 2009
> @@ -9,8 +9,10 @@ for ciph in `all_cipher_algos`; do
> # *BSD's dd can't cope with a count of 0
> if test "$i" = "0"; then
> : >z
> + chtag -tc ISO8859-1 z

This is a platform specific tool. It needs to be changed to a shell
fucntion which figures out the the latform and calls it only if needed.
Not a big deal.

> diff -purN gnupg-1.4.9.orig/g10/gpg.c gnupg-1.4.9.working-20090430/g10/gpg.c
> --- gnupg-1.4.9.orig/g10/gpg.c Thu Mar 20 05:06:43 2008
> +++ gnupg-1.4.9.working-20090430/g10/gpg.c Thu Apr 30 13:02:17 2009
> @@ -18,6 +18,7 @@
> * along with this program; if not, see <http://www.gnu.org/licenses/>.
> */
>
> +#include "platform_pragma.h"
> #include <config.h>

This platform_pragma.h file should be included by config.h so that
there is no need to chnage all source files. config.h is created by
configure.

> --- gnupg-1.4.9.orig/g10/Makefile.in Wed Mar 26 13:30:47 2008
> +++ gnupg-1.4.9.working-20090430/g10/Makefile.in Thu Apr 30 10:05:30 2009
Don't chnage Makefile.in; the source is Makefile.am.

> clean-generic:
> + rm -f *.dbg *.x

This is done using a CLEAN target in Makefile.am

> +/* platform_pragma.h - platform specific pragmas
> + * Copyright (C) 2009 Fidelity Information Services, Inc

One imortant point to get your code into GnuPG. We require copyright
assignments to the FSF. We should discuss the terms off-list.

> diff -purN gnupg-1.4.9.orig/intl/bindtextdom.c gnupg-1.4.9.working-20090430/intl/bindtextdom.c
> --- gnupg-1.4.9.orig/intl/bindtextdom.c Tue Oct 23 05:25:01 2007
> +++ gnupg-1.4.9.working-20090430/intl/bindtextdom.c Mon Apr 27 14:38:32 2009

These are files installed from the gettext package. It is possible to
change that in GnuPG but the next time we update the gettext stuff it
will be lost. Thus it needs to be integrated into gettext proper.

> +int zos_getpccsid(int fd)
> +{

The GNU project uses a different indendation; The important part is
to align the function name to the first column:

int
zos_getpccsid(int fd)
{

> diff -purN gnupg-1.4.9.orig/util/Makefile.in gnupg-1.4.9.working-20090430/util/Makefile.in

Well, Makefile.am ;-)

> diff -pruN libgcrypt-1.4.4.orig/src/ath.c libgcrypt-1.4.4.new/src/ath.c
> --- libgcrypt-1.4.4.orig/src/ath.c Wed Sep 3 06:04:42 2008
> +++ libgcrypt-1.4.4.new/src/ath.c Thu Apr 30 11:16:28 2009
> @@ -30,6 +30,9 @@
> # include <sys/time.h>
> #endif
> #include <sys/types.h>
> +#if defined(__MVS__)
> +#include <sys/msg.h>
> +#endif

This is better handled using a configure test.

Shalom-Salam,

Werner

--
Die Gedanken sind frei. Auschnahme regelt ein Bundeschgesetz.

_______________________________________________
Gnupg-devel mailing list
Gnupg-devel [at] gnupg
http://lists.gnupg.org/mailman/listinfo/gnupg-devel

amul.shah at fnis

Aug 3, 2009, 10:58 AM

Post #2 of 3 (617 views)
Permalink

Re: Adding support for z/OS in gnupg and related libraries [In reply to]

comments inline:

On 07/30/2009 09:58 AM, Werner Koch wrote:
> On Wed, 29 Jul 2009 20:40, amul.shah [at] fnis said:
>
>
>> Please excuse the cross-post. Some of the hoops that I jumped through
>>
>
> gnupg-devel is fine. We can drop gcrypt-devel.
>

[amul:2] thanks

>> gnupg 1.4.9
>> libgpg-errors 1.7
>> libgcrypt 1.4.4
>> libgpgme 1.1.8
>>
>
> Is there a reason why you work on 1.4.x and libgcrypt? Libgcrypt is
> only required for 2.x.
>

[amul:2] We are using both libgcrypt and gpg. Ideally I would have
compiled gpg 2.x, but we have had better success on various platforms
(AIX, Solaris, HPUX) with 1.4.x. We had issues compiling some of the
libraries needed for the 2.x version.

>> Services. The platform's native character encoding (aka code page or
>> code set) is not ASCII, but EBCDIC. The US dialect of EBCDIC is
>>
>
> FWIW, once upon a time gpg had some limited support for EBCDIC but that
> was later dropped.
>

[amul:2] IMHO, after the port we just went through, it's best to compile
Unix applications as ASCII and let z/OS do its conversion magic.
Working with Unicode adds another layer of complexity.

>> USS program are compiled as 31bit EBCDIC with no Unicode capabilities.
>>
>
> What does USS mean in this context?
>

[amul:2] sorry, USS means "Unix System Services", one of the names the
POSIX environment goes by. See the following link for more information.
http://www-03.ibm.com/servers/eserver/zseries/zos/unix/

>> autotools is weak. Where I wanted to build shared libraries, I compiled
>> archives. I unpacked the archives and converted them to shared libs
>>
>
> Support for shared libraries needs to be added to libtool; that is a
> wrapper to abstarct the building of shared libs. It is quite possible
> that it does not yet support zOS - I have not checked.
>

[amul:2] I spent some time with libtool, but I didn't write down what I
tried. I'll get back to you with some more information.

>> Configuration options:
>> ./configure CC=xlc CFLAGS="-qchars=signed -qascii -q64 -qlanglvl=extc99
>> -qexportall -qrent -qnocsect -W l,DLL -D_XOPEN_SOURCE=600
>> -D_ENHANCED_ASCII_EXT=0xFFFFFFFF -D_IEEEV1_COMPATIBILITY
>> -D_OPEN_MSGQ_EXT" LD=xlc LDFLAGS="-qascii -q64 -W l,DLL" CXX=xlc++
>> --enable-shared --prefix=/usr/local
>>
>
> In general those required options should go into configure.ac. We can
> test there for the zOS and apply specific options.
>

[amul:2] configure.ac is my area of inexperience.

> What is the reason that you used CC=xls? Is there another C compiler
> which can't be used? The configure script should be able to detect the
> compiler.
>

[amul:2] I don't recall what happened when I didn't specify a compiler
CC option. Will let you know after I check it.

>> There is an issue with the xlc configuration that defaults to picking up
>> include files from '.' (current working directory) and /usr/include
>> before picking up command line options. Since the z/OS system has two
>> headers with the same name as headers in libgcrypt, we link them into
>>
>
> Doesn't xlc support the -I option? "-I ." should pick up the packages
> header files first.
>

[amul:2] xlc supports "-I", but it picks up those options _after_ its
default search path. Confused me to no end when I was trying to
compile. There are xlc configuration options that you can customize
(like having a .bashrc, but for xlc) to fix this.

>> -- gnupg-1.4.9
>> Apply the attached patch gnupg-1.4.9.patch which applies to gnupg
>> 1.4.9. I will need help migrating this to the v2 line of gnupg
>>
>
> Is your target GnuPG-1.4 or GnuPG-2? If at all possible I would suggest
> to target GnuPG-2.
>

[amul:2] For now it's GnuPG-1.4. :( As I mentioned before we had
compilation problems on a bunch of platforms.

>> Configuration options:
>> ./configure CC=xlc CFLAGS="-qchars=signed -qascii -q64 -qlanglvl=extc99
>> -qexportall -qrent -qnocsect -W l,DLL -D_XOPEN_SOURCE=600
>> -D_ENHANCED_ASCII_EXT=0xFFFFFFFF -D_IEEEV1_COMPATIBILITY
>> -D_OPEN_MSGQ_EXT" LD=xlc LDFLAGS="-qascii -q64 -W l,DLL" CXX=xlc++
>> --without-pth --without-libassuan --without-ksba --prefix=/usr/local
>>
>
> The --without-* options are not needed but they don't harm for 1.4.
> --prefix=/usr/local is anyway the default.
>

[amul:2] Good to know. I was following configuration options used by
someone else to compile GPG + libs on other platforms.

>> -- Generating Shared Libs from archives
>>
>
> Weel, we need to look into libtool.
>

[amul:2] ok.

>> diff -purN gnupg-1.4.9.orig/checks/conventional-mdc.test gnupg-1.4.9.working-20090430/checks/conventional-mdc.test
>> --- gnupg-1.4.9.orig/checks/conventional-mdc.test Tue Oct 23 04:53:20 2007
>> +++ gnupg-1.4.9.working-20090430/checks/conventional-mdc.test Wed Apr 29 16:29:29 2009
>> @@ -9,8 +9,10 @@ for ciph in `all_cipher_algos`; do
>> # *BSD's dd can't cope with a count of 0
>> if test "$i" = "0"; then
>> : >z
>> + chtag -tc ISO8859-1 z
>>
>
> This is a platform specific tool. It needs to be changed to a shell
> fucntion which figures out the the latform and calls it only if needed.
> Not a big deal.
>

[amul:2] so something like an alias that is a nop on other platforms
would do the trick.

>> diff -purN gnupg-1.4.9.orig/g10/gpg.c gnupg-1.4.9.working-20090430/g10/gpg.c
>> --- gnupg-1.4.9.orig/g10/gpg.c Thu Mar 20 05:06:43 2008
>> +++ gnupg-1.4.9.working-20090430/g10/gpg.c Thu Apr 30 13:02:17 2009
>> @@ -18,6 +18,7 @@
>> * along with this program; if not, see <http://www.gnu.org/licenses/>.
>> */
>>
>> +#include "platform_pragma.h"
>> #include <config.h>
>>
>
> This platform_pragma.h file should be included by config.h so that
> there is no need to chnage all source files. config.h is created by
> configure.
>

[amul:2] I'll move that to confih.h.

>> --- gnupg-1.4.9.orig/g10/Makefile.in Wed Mar 26 13:30:47 2008
>> +++ gnupg-1.4.9.working-20090430/g10/Makefile.in Thu Apr 30 10:05:30 2009
>>
> Don't chnage Makefile.in; the source is Makefile.am.
>
>
>> clean-generic:
>> + rm -f *.dbg *.x
>>
>
> This is done using a CLEAN target in Makefile.am
>

[amul:2] Ok, I'll take a look at the Makefile.am.

>> +/* platform_pragma.h - platform specific pragmas
>> + * Copyright (C) 2009 Fidelity Information Services, Inc
>>
>
> One imortant point to get your code into GnuPG. We require copyright
> assignments to the FSF. We should discuss the terms off-list.
>

[amul:2] sure, we can do that. I thought saying that the code is GPL is
good enough.

>> diff -purN gnupg-1.4.9.orig/intl/bindtextdom.c gnupg-1.4.9.working-20090430/intl/bindtextdom.c
>> --- gnupg-1.4.9.orig/intl/bindtextdom.c Tue Oct 23 05:25:01 2007
>> +++ gnupg-1.4.9.working-20090430/intl/bindtextdom.c Mon Apr 27 14:38:32 2009
>>
>
> These are files installed from the gettext package. It is possible to
> change that in GnuPG but the next time we update the gettext stuff it
> will be lost. Thus it needs to be integrated into gettext proper.
>

[amul:2] The below project right?
http://www.gnu.org/software/gettext/

[amul:2] I'll make sure I send that project a patch.

>> +int zos_getpccsid(int fd)
>> +{
>>
>
> The GNU project uses a different indendation; The important part is
> to align the function name to the first column:
>
> int
> zos_getpccsid(int fd)
> {
>

[amul:2] Thanks. For future reference, here's the link. I should have
thought about that.
http://www.gnu.org/prep/standards/html_node/Formatting.html

>> diff -purN gnupg-1.4.9.orig/util/Makefile.in gnupg-1.4.9.working-20090430/util/Makefile.in
>>
>
> Well, Makefile.am ;-)
>

[amul:2] Yup. I'll fix that. :)

>> diff -pruN libgcrypt-1.4.4.orig/src/ath.c libgcrypt-1.4.4.new/src/ath.c
>> --- libgcrypt-1.4.4.orig/src/ath.c Wed Sep 3 06:04:42 2008
>> +++ libgcrypt-1.4.4.new/src/ath.c Thu Apr 30 11:16:28 2009
>> @@ -30,6 +30,9 @@
>> # include <sys/time.h>
>> #endif
>> #include <sys/types.h>
>> +#if defined(__MVS__)
>> +#include <sys/msg.h>
>> +#endif
>>
>
> This is better handled using a configure test.
>

[amul:2] How would that work?

thanks,
Amul

_____________

The information contained in this message is proprietary and/or confidential. If you are not the
intended recipient, please: (i) delete the message and all copies; (ii) do not disclose,
distribute or use the message in any manner; and (iii) notify the sender immediately. In addition,
please be aware that any message addressed to our domain is subject to archiving and review by
persons other than the intended recipient. Thank you.
_____________

wk at gnupg

Aug 5, 2009, 6:52 AM

Post #3 of 3 (613 views)
Permalink

Re: Adding support for z/OS in gnupg and related libraries [In reply to]

On Mon, 3 Aug 2009 19:58, amul.shah [at] fnis said:

> [amul:2] We are using both libgcrypt and gpg. Ideally I would have
> compiled gpg 2.x, but we have had better success on various platforms
> (AIX, Solaris, HPUX) with 1.4.x. We had issues compiling some of the
> libraries needed for the 2.x version.

GnuPG-2 requires a more modern Unix that GnuPG-1. That is not to say
that Solaris, AIX and HPUX aren't modern Unices. There are just some
little things which need to be adjusted and we usually don't have such a
system for tests.

> [amul:2] IMHO, after the port we just went through, it's best to compile
> Unix applications as ASCII and let z/OS do its conversion magic.
> Working with Unicode adds another layer of complexity.

There must have been a reason that IBM added that magic ASCII support ;-).

> [amul:2] sorry, USS means "Unix System Services", one of the names the
> POSIX environment goes by. See the following link for more information.
> http://www-03.ibm.com/servers/eserver/zseries/zos/unix/

Right. It is 10 years since I last worked a bit on a MVS and the USS
(not sure whether it was already called this back then).

> [amul:2] I spent some time with libtool, but I didn't write down what I
> tried. I'll get back to you with some more information.

Libtool is a nasty beast but it has the advantage that if you get it
right for your platform all software can take advantage of it.

> [amul:2] configure.ac is my area of inexperience.

Just ask me or let me know what you need.

> [amul:2] xlc supports "-I", but it picks up those options _after_ its
> default search path. Confused me to no end when I was trying to
> compile. There are xlc configuration options that you can customize
> (like having a .bashrc, but for xlc) to fix this.

I feared that. What about mangling the vac.cfg and passing a new one
via -F to xlc. It might be possible to extract the -I option from that
file build a new one and pass the extracted -I option on the command
line after our own -I. It might fail if one of the system include files
requires the system memory.h - bad. I think I once had a simialr
problem with OS/2.

So what about changing all

#include "memory.h"

to

#include "../include/memory.h

we might then need to change the name of the include directory as well.
Would that work?

Looking closer at it, this is a problem of gpg 1.4 and not one of
libgcrypt. The strange thing is that libgcrypt includes a memory.h but
does not provide one - thus the system memory.h is used. Must be a
leftover from an earlier version.

I will fix that in libgcrypt 1.5.

> [amul:2] so something like an alias that is a nop on other platforms
> would do the trick.

Something like this (see the FIXME):

===================================================================
--- defs.inc (revision 5106)
+++ defs.inc (working copy)
@@ -112,6 +112,16 @@
# cleanup_files="$cleanup_files $*"
#}

+
+# Special fucntion for zOS.
+my_chtag () {
+ #FIXME: Is there an envvar to test for the OS or do we
+ # need to resort to a configure test
+ #if test "$FOO" = "bar"; then
+ # chtag -tc ISO8859-1 $1
+ #fi
+}
+
have_pubkey_algo () {
if ../g10/gpg --homedir . --version | grep "Pubkey:.*$1" >/dev/null
then
Index: conventional-mdc.test
===================================================================
--- conventional-mdc.test (revision 5106)
+++ conventional-mdc.test (working copy)
@@ -9,6 +9,7 @@
# *BSD's dd can't cope with a count of 0
if test "$i" = "0"; then
+ my_chtag z
else
dd if=data-80000 of=z bs=1 count=$i 2>/dev/null
fi

================================

>>> +#include "platform_pragma.h"
>>> #include <config.h>
>>>
>>
>> This platform_pragma.h file should be included by config.h so that
>> there is no need to chnage all source files. config.h is created by
>> configure.
>>
>
> [amul:2] I'll move that to confih.h.

That's done in configure.ac by adding a line to

AH_TOP([
#ifndef GNUPG_CONFIG_H_INCLUDED
#define GNUPG_CONFIG_H_INCLUDED
])

or using AH_VERBATIM. Easy.

>> This is done using a CLEAN target in Makefile.am
>>
>
> [amul:2] Ok, I'll take a look at the Makefile.am.

You would add something like:

CLEANFILES = prepared.stamp x y yy z out err $(DATA_FILES) \
plain-1 plain-2 plain-3 trustdb.gpg *.lock .\#lk* \
*.test.log gpg_dearmor gpg.conf \
pubring.gpg secring.gpg pubring.pkr secring.skr
DISTCLEANFILES = pubring.gpg~ random_seed

there is also a MOSTLEANFILES target.

>> assignments to the FSF. We should discuss the terms off-list.
>>
>
> [amul:2] sure, we can do that. I thought saying that the code is GPL is
> good enough.

The GNU project requires copyright assignments or dislaimers for all
core software. Without that legal BS I may not include changes into
the package (you may distrubute your own of course, but then you have
the burden of maintaining it). Let's take this off-list.

> [amul:2] The below project right?
> http://www.gnu.org/software/gettext/
>
> [amul:2] I'll make sure I send that project a patch.

Right. It may take a while for a new release. In the meantime we can
apply the patches to the copy in GnuPG.

> [amul:2] Thanks. For future reference, here's the link. I should have
> thought about that.
> http://www.gnu.org/prep/standards/html_node/Formatting.html

It is not a hard rule anyway and all authors have different tastes. I
just make sure that some basic formatting is right.

>> This is better handled using a configure test.
>>
>
> [amul:2] How would that work?

--- configure.ac (revision 1400)
+++ configure.ac (working copy)
@@ -571,7 +571,7 @@
##################################

AC_HEADER_STDC
-AC_CHECK_HEADERS(unistd.h sys/select.h)
+AC_CHECK_HEADERS(unistd.h sys/select.h sys/msg.h)

##########################################
#### Checks for typedefs, structures, ####

--- src/ath.h (revision 1400)
+++ src/ath.h (working copy)
@@ -32,6 +32,9 @@
# include <sys/time.h>
# endif
# include <sys/types.h>
+# ifdef HAVE_SYS_MSG_H
+# include <sys/msg.h> /* (e.g. for zOS) */
+# endif
# include <sys/socket.h>
#endif /* !_WIN32 */
#include <gpg-error.h>

Already commited.

Shalom-Salam,

Werner

--
Die Gedanken sind frei. Auschnahme regelt ein Bundeschgesetz.

_______________________________________________
Gnupg-devel mailing list
Gnupg-devel [at] gnupg
http://lists.gnupg.org/mailman/listinfo/gnupg-devel

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads