Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: GnuPG: gcrypt

Adding support for z/OS in gnupg and related libraries

 

 

GnuPG gcrypt RSS feed   Index | Next | Previous | View Threaded


amul.shah at fnis

Jul 29, 2009, 11:40 AM

Post #1 of 2 (1360 views)
Permalink
Adding support for z/OS in gnupg and related libraries

I updated the makefiles and made some changes to compile and test GnuPG
on z/OS R10. This email exchange covers the steps taken and some
patches with the intent of merging these changes into the main branch
for GnuPG and its libraries.

Please excuse the cross-post. Some of the hoops that I jumped through
are relevant to all libraries and I don't want to split them apart. If
need be, we can stick to one mailing list. Pointers to help me better
explain what I am doing as always appreciated.


- Hardware Information:
Architecture z9
z/OS release V1R10
Crypto Express II card with the ICSF software (enable /dev/random in RACF)


- GnuPG and related library versions:
gnupg 1.4.9
libgpg-errors 1.7
libgcrypt 1.4.4
libgpgme 1.1.8


- Back story:
z/OS provides is POSIX compliant environment called Unix System
Services. The platform's native character encoding (aka code page or
code set) is not ASCII, but EBCDIC. The US dialect of EBCDIC is
IBM-1047, but there are dialects for other languages. I believe that
the USS can only operate in IBM-1047, and I have put in a request to
find out if that is a true statement. When using the IBM-1047 EBCDIC
code page the USS has the ability to auto-magically convert between
ASCII (ISO8859-1) and EBCDIC (IBM-1047).

Auto-magic conversion of data streams and files depends on environment
variables and extended file attributes called tags. Required
environment variables are as follows:
_BPXK_AUTOCVT="ON"
_CEE_RUNOPTS="FILETAG(AUTOCVT,AUTOTAG) POSIX(ON)"
_TAG_REDIR_OUT="txt"
_TAG_REDIR_IN="txt"
_TAG_REDIR_ERR="txt"
You should consider the options for _CEE_RUNOPTS as additions to what
you already have in _CEE_RUNOPTS

Files on disk and file descriptors for a process and process themselves
can be tagged with a character code set ID (CCSID). As of V1R10, all
USS program are compiled as 31bit EBCDIC with no Unicode capabilities.
Breaking this down into a more sensible way:

-- Applications
Applications can have CCSIDs tagged to them defining how they will
internally treat character streams and how they will interact with the
USS. At the moment on the only compilation options are IBM-1047 for
EBCDIC and ISO8859-1 for ASCII. IMHO applications ported to z/OS should
be compiled with the ASCII CCSID to avoid problems such as assuming a-z
is a contiguous block (not true for any form of EBCDIC).

-- Files
Files can be tagged with a CCSID, without a CCSID or as binary. Tagging
a file with a CCSID is meant to tell any opening application the
character encoding inside the file. As of V1R10, this information is
valuable for CCSIDs IBM-1047 (EBCDIC) and ISO8859-1 (ASCII). For the
aforementioned CCSIDs with the

-- File Descriptors
When creating files with file descriptor operations applications must
set file tags correctly to enable conversion. Applications developers
must also decide how to handle untagged files and what to do when file
that they own have been mis-tagged.

-- File Pointers
Using fopen on a file and then fread/fwrite simplifies file tagging and
auto conversion if you have the above environment variables defined.
See the reference link "Steps for setting up Enhanced ASCII" below.

-- Exceptions to the above rules
Files that are tagged with Binary are not subject to auto conversion.
Files tagged with a CCSID but marked as "mixed" text and binary data are
not subject to auto conversion. Files must be tagged as "text" and a
valid CCSID to enable auto conversion.


- Compiling GnuPG et al on z/OS
My team used the XLC compiler from IBM and compiled the libraries and
applications as ASCII (ISO8859-1). We have used a number of MACROs
called Feature Test MACROs to enable specific features and
compatibility. For information on "Feature Test Macros", see
http://publib.boulder.ibm.com/infocenter/zos/v1r10/topic/com.ibm.zos.r10.bpxbd00/ftms.htm


I tried modifying the configure scripts, but my understanding of
autotools is weak. Where I wanted to build shared libraries, I compiled
archives. I unpacked the archives and converted them to shared libs
manually.

-- libgpg-error
Apply the attached patch libgpg-error-1.7.patch which applies to
libgpg-error 1.7.

Configuration options:
./configure CC=xlc CFLAGS="-qchars=signed -qascii -q64 -qlanglvl=extc99
-qexportall -qrent -qnocsect -W l,DLL -D_XOPEN_SOURCE=600
-D_ENHANCED_ASCII_EXT=0xFFFFFFFF -D_IEEEV1_COMPATIBILITY
-D_OPEN_MSGQ_EXT" LD=xlc LDFLAGS="-qascii -q64 -W l,DLL" CXX=xlc++
--enable-shared --prefix=/usr/local

-- libgcrypt
Apply the attached patch libgcrypt-1.4.4.patch which applies to
libgcrypt 1.4.4.

There is an issue with the xlc configuration that defaults to picking up
include files from '.' (current working directory) and /usr/include
before picking up command line options. Since the z/OS system has two
headers with the same name as headers in libgcrypt, we link them into
the mpi subdirectory to force xlc to use the local headers and not the
system headers in /usr/include. From the source directory root, I did
the following:
cd mpi
ln -s ../src/mpi.h .
ln -s ../src/memory.h .
cd -
cd cipher
ln -s ../src/mpi.h .
cd -

It is possible to create an xlc configuration file to get around this
issue, but that is another beast altogether.

Configuration options:
./configure CC=xlc CFLAGS="-qchars=signed -qascii -q64 -qlanglvl=extc99
-qexportall -qrent -qnocsect -W l,DLL -D_XOPEN_SOURCE=600
-D_ENHANCED_ASCII_EXT=0xFFFFFFFF -D_IEEEV1_COMPATIBILITY
-D_OPEN_MSGQ_EXT" LD=xlc LDFLAGS="-qascii -q64 -W l,DLL" CXX=xlc++
--enable-shared --prefix=/usr/local --with-gpg-error-prefix=/usr/local

-- gpgme
Please use the gpgme-1.2.0 sources which have a fix that is needed for
1.1.8 which I built and tested on z/OS.

Configuration options:
./configure CC=xlc CFLAGS="-qchars=signed -qascii -q64 -qlanglvl=extc99
-qexportall -qrent -qnocsect -W l,DLL -D_XOPEN_SOURCE=600
-D_ENHANCED_ASCII_EXT=0xFFFFFFFF -D_IEEEV1_COMPATIBILITY
-D_OPEN_MSGQ_EXT" LD=xlc LDFLAGS="-qascii -q64 -W l,DLL" CXX=xlc++
--enable-shared --prefix=/usr/local--with-gpg-error-prefix=/usr/local

-- gnupg-1.4.9
Apply the attached patch gnupg-1.4.9.patch which applies to gnupg
1.4.9. I will need help migrating this to the v2 line of gnupg

There is an issue with the xlc configuration that defaults to picking up
include files from '.' (current working directory) and /usr/include
before picking up command line options. Since the z/OS system has two
headers with the same name as headers in libgcrypt, we link them into
the mpi subdirectory to force xlc to use the local headers and not the
system headers in /usr/include. From the source directory root, I did
the following:
cd mpi
ln -s ../include/mpi.h .
ln -s ../include/memory.h .
cd -

It is possible to create an xlc configuration file to get around this
issue, but that is another beast altogether.

Configuration options:
./configure CC=xlc CFLAGS="-qchars=signed -qascii -q64 -qlanglvl=extc99
-qexportall -qrent -qnocsect -W l,DLL -D_XOPEN_SOURCE=600
-D_ENHANCED_ASCII_EXT=0xFFFFFFFF -D_IEEEV1_COMPATIBILITY
-D_OPEN_MSGQ_EXT" LD=xlc LDFLAGS="-qascii -q64 -W l,DLL" CXX=xlc++
--without-pth --without-libassuan --without-ksba --prefix=/usr/local

-- Generating Shared Libs from archives
Since we needed shared libraries, I ripped the object files out of each
library and relinked them into a shared library (DLL on z/OS). The
following is an example of what I did to make gpgme a shared library.
mkdir /tmp/tmp-gpgme
cd /tmp/tmp-gpgme
ar -x /usr/local/lib/libgpgme.a
xlc -qascii -q64 -W l,DLL -o libgpgme.dll *.o /usr/local/lib/libgpg-error.x
cp libgpgme.dll /usr/local/lib
cp libgpgme.x /usr/local/lib
cd -
rm -rf /tmp/tmp-gpgme


- References
Porting to z/OS Unix -
http://www-03.ibm.com/servers/eserver/zseries/zos/unix/bpxa1por.html
Steps for setting up Enhanced ASCII -
http://publib.boulder.ibm.com/infocenter/zos/v1r10/topic/com.ibm.zos.r10.bpxb200/stepas.htm#stepas
z/OS Unix -
http://publib.boulder.ibm.com/infocenter/zos/v1r10/topic/com.ibm.zos.r10.bpx/bpx.htm

z/OS Lanuage Env (include programming guide and run time reference) -
http://publib.boulder.ibm.com/infocenter/zos/v1r10/topic/com.ibm.zos.r10.cee/cee.htm
Information about DLLs -
http://publib.boulder.ibm.com/infocenter/zos/v1r10/topic/com.ibm.zos.r10.cbcpx01/dllsim.htm



- Patches
Are attachments.

_____________

The information contained in this message is proprietary and/or confidential. If you are not the
intended recipient, please: (i) delete the message and all copies; (ii) do not disclose,
distribute or use the message in any manner; and (iii) notify the sender immediately. In addition,
please be aware that any message addressed to our domain is subject to archiving and review by
persons other than the intended recipient. Thank you.
_____________
Attachments: gnupg-1.4.9.patch (29.2 KB)
  libgcrypt-1.4.4.patch (1.55 KB)
  libgpg-error-1.7.patch (0.81 KB)


wk at gnupg

Jul 30, 2009, 6:58 AM

Post #2 of 2 (1281 views)
Permalink
Re: Adding support for z/OS in gnupg and related libraries [In reply to]

On Wed, 29 Jul 2009 20:40, amul.shah [at] fnis said:

> Please excuse the cross-post. Some of the hoops that I jumped through

gnupg-devel is fine. We can drop gcrypt-devel.

> gnupg 1.4.9
> libgpg-errors 1.7
> libgcrypt 1.4.4
> libgpgme 1.1.8

Is there a reason why you work on 1.4.x and libgcrypt? Libgcrypt is
only required for 2.x.

> Services. The platform's native character encoding (aka code page or
> code set) is not ASCII, but EBCDIC. The US dialect of EBCDIC is

FWIW, once upon a time gpg had some limited support for EBCDIC but that
was later dropped.

> USS program are compiled as 31bit EBCDIC with no Unicode capabilities.

What does USS mean in this context?

> autotools is weak. Where I wanted to build shared libraries, I compiled
> archives. I unpacked the archives and converted them to shared libs

Support for shared libraries needs to be added to libtool; that is a
wrapper to abstarct the building of shared libs. It is quite possible
that it does not yet support zOS - I have not checked.

> Configuration options:
> ./configure CC=xlc CFLAGS="-qchars=signed -qascii -q64 -qlanglvl=extc99
> -qexportall -qrent -qnocsect -W l,DLL -D_XOPEN_SOURCE=600
> -D_ENHANCED_ASCII_EXT=0xFFFFFFFF -D_IEEEV1_COMPATIBILITY
> -D_OPEN_MSGQ_EXT" LD=xlc LDFLAGS="-qascii -q64 -W l,DLL" CXX=xlc++
> --enable-shared --prefix=/usr/local

In general those required options should go into configure.ac. We can
test there for the zOS and apply specific options.

What is the reason that you used CC=xls? Is there another C compiler
which can't be used? The configure script should be able to detect the
compiler.

> There is an issue with the xlc configuration that defaults to picking up
> include files from '.' (current working directory) and /usr/include
> before picking up command line options. Since the z/OS system has two
> headers with the same name as headers in libgcrypt, we link them into

Doesn't xlc support the -I option? "-I ." should pick up the packages
header files first.


> -- gnupg-1.4.9
> Apply the attached patch gnupg-1.4.9.patch which applies to gnupg
> 1.4.9. I will need help migrating this to the v2 line of gnupg

Is your target GnuPG-1.4 or GnuPG-2? If at all possible I would suggest
to target GnuPG-2.

> Configuration options:
> ./configure CC=xlc CFLAGS="-qchars=signed -qascii -q64 -qlanglvl=extc99
> -qexportall -qrent -qnocsect -W l,DLL -D_XOPEN_SOURCE=600
> -D_ENHANCED_ASCII_EXT=0xFFFFFFFF -D_IEEEV1_COMPATIBILITY
> -D_OPEN_MSGQ_EXT" LD=xlc LDFLAGS="-qascii -q64 -W l,DLL" CXX=xlc++
> --without-pth --without-libassuan --without-ksba --prefix=/usr/local

The --without-* options are not needed but they don't harm for 1.4.
--prefix=/usr/local is anyway the default.

> -- Generating Shared Libs from archives

Weel, we need to look into libtool.

> diff -purN gnupg-1.4.9.orig/checks/conventional-mdc.test gnupg-1.4.9.working-20090430/checks/conventional-mdc.test
> --- gnupg-1.4.9.orig/checks/conventional-mdc.test Tue Oct 23 04:53:20 2007
> +++ gnupg-1.4.9.working-20090430/checks/conventional-mdc.test Wed Apr 29 16:29:29 2009
> @@ -9,8 +9,10 @@ for ciph in `all_cipher_algos`; do
> # *BSD's dd can't cope with a count of 0
> if test "$i" = "0"; then
> : >z
> + chtag -tc ISO8859-1 z

This is a platform specific tool. It needs to be changed to a shell
fucntion which figures out the the latform and calls it only if needed.
Not a big deal.

> diff -purN gnupg-1.4.9.orig/g10/gpg.c gnupg-1.4.9.working-20090430/g10/gpg.c
> --- gnupg-1.4.9.orig/g10/gpg.c Thu Mar 20 05:06:43 2008
> +++ gnupg-1.4.9.working-20090430/g10/gpg.c Thu Apr 30 13:02:17 2009
> @@ -18,6 +18,7 @@
> * along with this program; if not, see <http://www.gnu.org/licenses/>.
> */
>
> +#include "platform_pragma.h"
> #include <config.h>

This platform_pragma.h file should be included by config.h so that
there is no need to chnage all source files. config.h is created by
configure.


> --- gnupg-1.4.9.orig/g10/Makefile.in Wed Mar 26 13:30:47 2008
> +++ gnupg-1.4.9.working-20090430/g10/Makefile.in Thu Apr 30 10:05:30 2009
Don't chnage Makefile.in; the source is Makefile.am.

> clean-generic:
> + rm -f *.dbg *.x

This is done using a CLEAN target in Makefile.am

> +/* platform_pragma.h - platform specific pragmas
> + * Copyright (C) 2009 Fidelity Information Services, Inc

One imortant point to get your code into GnuPG. We require copyright
assignments to the FSF. We should discuss the terms off-list.

> diff -purN gnupg-1.4.9.orig/intl/bindtextdom.c gnupg-1.4.9.working-20090430/intl/bindtextdom.c
> --- gnupg-1.4.9.orig/intl/bindtextdom.c Tue Oct 23 05:25:01 2007
> +++ gnupg-1.4.9.working-20090430/intl/bindtextdom.c Mon Apr 27 14:38:32 2009

These are files installed from the gettext package. It is possible to
change that in GnuPG but the next time we update the gettext stuff it
will be lost. Thus it needs to be integrated into gettext proper.

> +int zos_getpccsid(int fd)
> +{

The GNU project uses a different indendation; The important part is
to align the function name to the first column:

int
zos_getpccsid(int fd)
{


> diff -purN gnupg-1.4.9.orig/util/Makefile.in gnupg-1.4.9.working-20090430/util/Makefile.in

Well, Makefile.am ;-)

> diff -pruN libgcrypt-1.4.4.orig/src/ath.c libgcrypt-1.4.4.new/src/ath.c
> --- libgcrypt-1.4.4.orig/src/ath.c Wed Sep 3 06:04:42 2008
> +++ libgcrypt-1.4.4.new/src/ath.c Thu Apr 30 11:16:28 2009
> @@ -30,6 +30,9 @@
> # include <sys/time.h>
> #endif
> #include <sys/types.h>
> +#if defined(__MVS__)
> +#include <sys/msg.h>
> +#endif

This is better handled using a configure test.


Shalom-Salam,

Werner

--
Die Gedanken sind frei. Auschnahme regelt ein Bundeschgesetz.


_______________________________________________
Gcrypt-devel mailing list
Gcrypt-devel [at] gnupg
http://lists.gnupg.org/mailman/listinfo/gcrypt-devel

GnuPG gcrypt RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.