Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: exim: users

exim 4.70 on NetBSD 5.0.1 failure

 

 

exim users RSS feed   Index | Next | Previous | View Threaded


tux-pinguin at gmx

Nov 16, 2009, 1:55 AM

Post #1 of 6 (944 views)
Permalink
exim 4.70 on NetBSD 5.0.1 failure

Hi there,

I want to use Exim 4.70 on NetBSD 5.0.1.
Every mail outbound is not delivered, because exim quits with sig 6 and
the error message "_res is not supported for multi-threaded programs."

Bug in exim? Bug in NetBSD? What can I do?


root [at] ww:#> exim -d -v dontshow [at] gmx
Exim version 4.70 uid=0 gid=0 pid=11899 D=fbb95cfd
Probably Berkeley DB version 1.8x (native mode)
Support for: crypteq IPv6 TCPwrappers OpenSSL Content_Scanning DKIM
Old_Demime
Lookups: lsearch wildlsearch nwildlsearch iplsearch dbm dbmnz sqlite
Authenticators: cram_md5 dovecot plaintext spa
Routers: accept dnslookup ipliteral manualroute queryprogram redirect
Transports: appendfile/maildir/mailstore/mbx autoreply pipe smtp
Fixed never_users: 0
Size of off_t: 8
OpenSSL compile-time version: OpenSSL 0.9.9-dev 09 May 2008
OpenSSL runtime version: OpenSSL 0.9.9-dev 09 May 2008
changed uid/gid: forcing real = effective
uid=0 gid=0 pid=11899
auxiliary group list: <none>
seeking password data for user "mail": using cached result
getpwnam() succeeded uid=1002 gid=6
seeking password data for user "root": cache not available
getpwnam() succeeded uid=0 gid=0
configuration file is /usr/local/etc/exim/configure
log selectors = 00000ffc 00212001
cwd=/root 4 args: exim -d -v dontshow [at] gmx
trusted user
admin user
changed uid/gid: privilege not needed
uid=1002 gid=6 pid=11899
auxiliary group list: 6
seeking password data for user "mail": cache not available
getpwnam() succeeded uid=1002 gid=6
seeking password data for user "mail": using cached result
getpwnam() succeeded uid=1002 gid=6
seeking password data for user "mail": using cached result
getpwnam() succeeded uid=1002 gid=6
originator: uid=0 gid=0 login=root name=Charlie Root
sender address = root [at] www-online
set_process_info: 11899 accepting a local non-SMTP message from
<root [at] www-online>
Sender: root [at] www-online
Recipients:
dontshow [at] gmx
search_tidyup called
>>Headers received:

rewrite_one_header: type=F:
From: Charlie Root <root [at] www-online>
search_tidyup called
>>Headers after rewriting and local additions:
Date: Mon, 16 Nov 2009 10:16:59 +0100
I Message-Id: <E1N9xhh-00035v-E8 [at] www>
F From: Charlie Root <root [at] www-online>

Data file written for message 1N9xhh-00035v-E8
>>Generated Received: header line
P Received: from root by www.www-online.de with local (Exim 4.70)
(envelope-from <root [at] www-online>)
id 1N9xhh-00035v-E8
for dontshow [at] gmx; Mon, 16 Nov 2009 10:16:59 +0100
calling local_scan(); timeout=300
local_scan() returned 0 NULL
Writing spool header file
Size of headers = 329
LOG: MAIN
<= root [at] www-online U=root P=local S=330
search_tidyup called
search_tidyup called
exec /usr/local/pkg/sbin/exim -d=0xfbb95cfd -Mc 1N9xhh-00035v-E8
>>>>>>>>>>>>>>>> Exim pid=11899 terminating with rc=0 >>>>>>>>>>>>>>>>
Exim version 4.70 uid=1002 gid=6 pid=3335 D=fbb95cfd
root [at] ww:#> Probably Berkeley DB version 1.8x (native mode)
Support for: crypteq IPv6 TCPwrappers OpenSSL Content_Scanning DKIM
Old_Demime/root
Lookups: lsearch wildlsearch nwildlsearch iplsearch dbm dbmnz sqlite
Authenticators: cram_md5 dovecot plaintext spa
Routers: accept dnslookup ipliteral manualroute queryprogram redirect
Transports: appendfile/maildir/mailstore/mbx autoreply pipe smtp
Fixed never_users: 0
Size of off_t: 8
OpenSSL compile-time version: OpenSSL 0.9.9-dev 09 May 2008
OpenSSL runtime version: OpenSSL 0.9.9-dev 09 May 2008
changed uid/gid: forcing real = effective
uid=0 gid=6 pid=3335
auxiliary group list: <none>
seeking password data for user "mail": using cached result
getpwnam() succeeded uid=1002 gid=6
seeking password data for user "root": cache not available
getpwnam() succeeded uid=0 gid=0
configuration file is /usr/local/etc/exim/configure
log selectors = 00000ffc 00212001
cwd=/usr/local/pkg/var/spool/exim 4 args: /usr/local/pkg/sbin/exim
-d=0xfbb95cfd -Mc 1N9xhh-00035v-E8
trusted user
admin user
skipping ACL configuration - not needed
seeking password data for user "mail": cache not available
getpwnam() succeeded uid=1002 gid=6
seeking password data for user "mail": using cached result
getpwnam() succeeded uid=1002 gid=6
seeking password data for user "mail": using cached result
getpwnam() succeeded uid=1002 gid=6
set_process_info: 3335 delivering specified messages
set_process_info: 3335 delivering 1N9xhh-00035v-E8
reading spool file 1N9xhh-00035v-E8-H
user=root uid=0 gid=0 sender=root [at] www-online
sender_local=1 ident=root
Non-recipients:
Empty Tree
---- End of tree ----
recipients_count=1
body_linecount=0 message_linecount=7
Delivery address list:
dontshow [at] gmx
locking /usr/local/pkg/var/spool/exim/db/retry.lockfile
locked /usr/local/pkg/var/spool/exim/db/retry.lockfile
EXIM_DBOPEN(/usr/local/pkg/var/spool/exim/db/retry)
returned from EXIM_DBOPEN
no retry data available
Considering: dontshow [at] gmx
unique = dontshow [at] gmx
no domain retry record
no address retry record
dontshow [at] gmx: queued for routing
routing dontshow [at] gmx
--------> dnslookup router <--------
local_part=dontshow domain=gmx.net
checking domains
search_open: sqlite "/usr/local/etc/mail/mail.sqlite3"
search_find: file="/usr/local/etc/mail/mail.sqlite3"
key="SELECT domain FROM domains WHERE type='local' AND
domain='gmx.net';" partial=-1 affix=NULL starflags=0
LRU list:
internal_search_find: file="/usr/local/etc/mail/mail.sqlite3"
type=sqlite key="SELECT domain FROM domains WHERE type='local' AND
domain='gmx.net';"
file lookup required for SELECT domain FROM domains WHERE type='local' AND
domain='gmx.net';
in /usr/local/etc/mail/mail.sqlite3
lookup forced cache cleanup
lookup failed
gmx.net in "sqlite;/usr/local/etc/mail/mail.sqlite3 SELECT domain FROM
domains WHERE type='local' AND domain='gmx.net';"? no (end of list)
gmx.net in "! +local_domains"? yes (end of list)
calling dnslookup router
dnslookup router called for dontshow [at] gmx
domain = gmx.net
_res is not supported for multi-threaded programs.


root [at] ww:#> uname -a
NetBSD www.bsdworld.private 5.0.1 NetBSD 5.0.1 (GENERIC) #0: Thu Jul 30
01:39:11 UTC 2009
builds [at] b8:/home/builds/ab/netbsd-5-0-1-RELEASE/i386/200907292356Z-obj/home/builds/ab/netbsd-5-0-1-RELEASE/src/sys/arch/i386/compile/GENERIC
i386


Any help would be appreciated
J.R.





--
## List details at http://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/


tom at duncanthrax

Nov 16, 2009, 2:55 AM

Post #2 of 6 (885 views)
Permalink
Re: exim 4.70 on NetBSD 5.0.1 failure [In reply to]

J.R. wrote:

> I want to use Exim 4.70 on NetBSD 5.0.1.
> Every mail outbound is not delivered, because exim quits with sig 6 and
> the error message "_res is not supported for multi-threaded programs."

Looks like a resolver library problem. Exim isn't multi-threaded,
however. Who is generating that message? Dig up that code and check the
prerequisites for it to appear.

/tom



--
## List details at http://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/


tux-pinguin at gmx

Nov 16, 2009, 11:51 PM

Post #3 of 6 (879 views)
Permalink
Re: exim 4.70 on NetBSD 5.0.1 failure [In reply to]

> Looks like a resolver library problem. Exim isn't multi-threaded,
> however. Who is generating that message? Dig up that code and check the
> prerequisites for it to appear.

Got some hints on the netbsd mailing list:
---
_res is a structure used by resolver(3) to resolve domain name queries. As
it is used as a global variable, access to it cannot be thread-safe (the
old API does not define a lock that permits the access of this struct in a
thread-safe manner).

Fix:
There should be some piece of code in exim 4.70 that uses _res directly.
It should be rewritten with the res_n*() (res_ninit, res_nsend, ...)
functions, which are thread-safe.

See also http://netbsd.gw.com/cgi-bin/man-cgi?res_nsend++NetBSD-current
---

It seems exim implemented the use of the resolver library the wrong way.

(I'm the only one who wants to use exim on netbsd in the whole world?)

thx!
JR


--
## List details at http://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/


eximX0902w at linuxwan

Nov 17, 2009, 12:15 AM

Post #4 of 6 (882 views)
Permalink
Re: exim 4.70 on NetBSD 5.0.1 failure [In reply to]

On Tue, 2009-11-17 at 08:51 +0100, J.R. wrote:
> > Looks like a resolver library problem. Exim isn't multi-threaded,
> > however. Who is generating that message? Dig up that code and check the
> > prerequisites for it to appear.
>
> Got some hints on the netbsd mailing list:
> ---
> _res is a structure used by resolver(3) to resolve domain name queries. As
> it is used as a global variable, access to it cannot be thread-safe (the
> old API does not define a lock that permits the access of this struct in a
> thread-safe manner).
>
> Fix:
> There should be some piece of code in exim 4.70 that uses _res directly.
> It should be rewritten with the res_n*() (res_ninit, res_nsend, ...)
> functions, which are thread-safe.
>
> See also http://netbsd.gw.com/cgi-bin/man-cgi?res_nsend++NetBSD-current
> ---
>
> It seems exim implemented the use of the resolver library the wrong way.
>
> (I'm the only one who wants to use exim on netbsd in the whole world?)

Well it is an issue, but again I think you've missed the key point here:
Exim doesn't use threads, thus it does not access _res in an unsafe
manner.

It's entirely fork() based so each new process will have its own copy of
_res which is can with as it pleases.

Even so, if Exim has access to _res implemented incorrectly, we should
probably look at fixing that. Does anyone know of a good reference
program for whatever resolver library we're talking about?


--
The Exim manual - http://docs.exim.org


--
## List details at http://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/


exim-users at spodhuis

Nov 17, 2009, 1:57 PM

Post #5 of 6 (874 views)
Permalink
Re: exim 4.70 on NetBSD 5.0.1 failure [In reply to]

On 2009-11-17 at 19:15 +1100, Ted Cooper wrote:
> Well it is an issue, but again I think you've missed the key point here:
> Exim doesn't use threads, thus it does not access _res in an unsafe
> manner.

I see libthr coming in via libdb (BDB 4.4) and OpenSSL. Doesn't mean
I'm using threads.

> It's entirely fork() based so each new process will have its own copy of
> _res which is can with as it pleases.
>
> Even so, if Exim has access to _res implemented incorrectly, we should
> probably look at fixing that. Does anyone know of a good reference
> program for whatever resolver library we're talking about?

It's the standard Unix resolver library. resolver(3) on FreeBSD
(among others) goes into more detail on _res.

Exim's only usage is to initialise _res in dns_init() and to reference
_res.options to construct a cache key. Oh, and some stuff in the test
harness. Exim sets some items in _res.options (fully documented in the
manpage) and sets _res.retrans and/or _res.retry if the dns_retrans
and/or dns_retry options are set in the main config section.

None of the DNS lookups explicitly reference _res; it's purely an
init-time thing and a read of the options for the cache key.

The proposed replacement functions are peculiar to NetBSD, so this
suggested approach really translates to "implement a custom DNS layer
for NetBSD". That seems to be of little benefit, unless some other OS
is going to adopt these new calls. (But see below)

The current _res usage works across every OS which Exim is built for
except the new NetBSD platform (AIX, HP-UX, IRIX, SCO, OSF1, ULTRIX ...)
so the question really is "What did NetBSD break and is there a simple
way for the Exim binary to persuade the resolver library that it's
behaving safely?"

The use of _res in Exim's dns_init() looks to be compatible with the
description of _res at:
http://netbsd.gw.com/cgi-bin/man-cgi?resolver++NetBSD-current
so I suspect the only issue is the later referencing of _res.options for
the cache key.

If there's a lock which can be grabbed to safely access _res, the right
thing might be to use a global variable internal to Exim to hold
_res.options, init that at the same time that res_init() is called (in
dns_init()), double-check that covers all execution paths and then use
NetBSD-specific #ifdef's to handle lock init. But I don't see such a
lock described in resolver(3) at the URL above. We could set the global
variable in the same way that the resolver options are set, but that
wouldn't pick up default options.

Another approach would be to figure out whether or not we really need
the options as part of the cache key and look at just removing that.

A working debug trace of the part which failed for the OP is:
----------------------------8< cut here >8------------------------------
calling dnslookup router
dnslookup router called for dontshow [at] gmx
domain = gmx.net
DNS lookup of gmx.net (MX) succeeded
DNS lookup of mx1.gmx.net (AAAA) gave NO_DATA
returning DNS_NODATA
DNS lookup of mx1.gmx.net (A) succeeded
----------------------------8< cut here >8------------------------------
so the point which failed was dns.c line 563:
dnsa->answerlen = res_search(CS name, C_IN, type, dnsa->answer, MAXPACKET);

In fact, the DNS resolution is sufficiently isolated that we *could*
implement the res_nsend API. I don't have a NetBSD box to test any
changes on though, so I'm unwilling to write the patch.

-Phil

--
## List details at http://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/


tux-pinguin at gmx

Nov 18, 2009, 1:14 AM

Post #6 of 6 (853 views)
Permalink
Re: exim 4.70 on NetBSD 5.0.1 failure [In reply to]

Hi,

> In fact, the DNS resolution is sufficiently isolated that we *could*
> implement the res_nsend API. I don't have a NetBSD box to test any
> changes on though, so I'm unwilling to write the patch.

there is a person on the netbsd mailing list who can help to develop a
patch. I have his agreement to forward his offer:
http://mail-index.netbsd.org/pkgsrc-users/2009/11/17/msg011148.html
---
Well, I can still make tests on my NetBSD box if required, but please note
that it is in no way dependant of NetBSD.

The res_foo API is deprecated, and the alternative res_nfoo functions have
been around for many, many years. IMHO, using deprecated APIs in new code
is not recommended.

res_nfoo functions use statp instead of _res, which is thread-safe.

Feel free to forward my mail if you need too.

Jean-Yves Migeon
jeanyves.migeon%free.fr [at] localhos
---

Maybe exim *can* run on NetBSD. This would be great!

thx!
J.R.



--
## List details at http://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

exim users RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.