Mailing List Archive: exim: users

Ocasional 'Unrouteable address'-message with LDAP environment

Index | Next | Previous | View Threaded

marks at bit

Apr 22, 2008, 10:12 AM

Post #1 of 4 (256 views)
Permalink

Ocasional 'Unrouteable address'-message with LDAP environment

Hi there,

I'm not on this list, so please make sure that replies go to
marks[at]bit.nl as well, thanks.

I'm not sure if this is an Exim issue, or an nscd issue. But I can't
really find an answer.

We have an Exim-setup with about twenty machines doing email for our
customers. We do about 70 emails/second on a loadbalanced environment
and after scanning for spam we deliver about 10 emails/second in
people's mailboxes. The delivery is loadbalanced over three machines.
(See the config attached to this email)

All works well, except for an 'Unrouteable address', once in a while.
Apr 22 17:47:47 maildrop2.maildrop.dmz.bit.nl exim[29127]:
1JoKih-0007Zi-PN ** stedelyk[at]maildrop.dmz.bit.nl: Unrouteable address
Apr 22 18:04:57 maildrop1.maildrop.dmz.bit.nl exim[16182]:
1JoKzJ-0004Cz-I0 ** hengroep[at]maildrop.dmz.bit.nl: Unrouteable address
Apr 22 18:17:41 maildrop2.maildrop.dmz.bit.nl exim[18194]:
1JoLBd-0004jR-96 ** marina[at]maildrop.dmz.bit.nl: Unrouteable address
Apr 22 18:29:50 maildrop3.maildrop.dmz.bit.nl exim[30770]:
1JoLNO-00080H-Qy ** adm-smit[at]maildrop.dmz.bit.nl: Unrouteable address
Apr 22 18:31:15 maildrop3.maildrop.dmz.bit.nl exim[31710]:
1JoLOl-0008FR-Gz ** nillesen[at]maildrop.dmz.bit.nl: Unrouteable address
Apr 22 18:31:58 maildrop3.maildrop.dmz.bit.nl exim[32264]:
1JoLPS-0008OM-IY ** era214[at]maildrop.dmz.bit.nl: Unrouteable address
Apr 22 18:32:46 maildrop2.maildrop.dmz.bit.nl exim[28466]:
1JoLQE-0007P5-7M ** dlp16[at]maildrop.dmz.bit.nl: Unrouteable address

We do not do any recipient checking as these checks are done on the
frontend-machines, but all of these users exist on the machines where
delivery takes place.
The users exists thanks to libnss-ldap, combined with nscd.

As far as I can tell, the only thing that could cause these messages to
bounce is 'check_local_user'. According to the Exim documentation,
check_local_user does a getpwnam. Should I assume that, if Exim says
'Unrouteable address', the getpwnam-reply was 'User does not exist'? If
so, the problem would lie in nscd, giving false replies. As far as I can
see Exim should tempfail if it sees an LDAP error, and not 'Unrouteable
address'.

If anyone has a cluebat, please hit me with it. :)

Thanks!

--
Mark Schouten, Unix/NOC-engineer
BIT BV | info[at]bit.nl | +31 318 648688
MS8714-RIPE | B1FD 8E60 A184 F89A 450D A128 049B 1B19 9AD6 17FF

Attachments:

exim4.conf.gz (1.47 KB)

exim-users at spodhuis

Apr 24, 2008, 3:09 PM

Post #2 of 4 (215 views)
Permalink

Re: Ocasional 'Unrouteable address'-message with LDAP environment [In reply to]

On 2008-04-22 at 19:12 +0200, Mark Schouten wrote:
> We do not do any recipient checking as these checks are done on the
> frontend-machines, but all of these users exist on the machines where
> delivery takes place.
> The users exists thanks to libnss-ldap, combined with nscd.
>
> As far as I can tell, the only thing that could cause these messages to
> bounce is 'check_local_user'. According to the Exim documentation,
> check_local_user does a getpwnam. Should I assume that, if Exim says
> 'Unrouteable address', the getpwnam-reply was 'User does not exist'? If
> so, the problem would lie in nscd, giving false replies. As far as I can
> see Exim should tempfail if it sees an LDAP error, and not 'Unrouteable
> address'.
>
> If anyone has a cluebat, please hit me with it. :)

nscd flaws are not implausible. Your clue-level seems sufficient. :^)

Take check_local_user out of the picture by using a direct LDAP lookup
in a condition instead and see if that fixes it? Do this on one of the
three boxes, compare error rates.

Make sure that you have sufficient indexing on the LDAP read-only caches
being used for these queries that the servers can handle this without
nscd buffering it, of course.

Check "10.7 Named list caching" which might be useful, depending upon
the queries you end up using, to reduce repeat queries from within one
process. Also, if data from receive-time will be used again at delivery
time and this can be deferred to a queue-runner, consider using an ACL
variable or two to hold the results of the lookup to avoid repeat
queries.

-Phil

--
## List details at http://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

marks at bit

May 7, 2008, 1:28 AM

Post #3 of 4 (150 views)
Permalink

Re: Ocasional 'Unrouteable address'-message with LDAP environment [In reply to]

On Thu, 2008-04-24 at 15:09 -0700, Phil Pennock wrote:
> nscd flaws are not implausible. Your clue-level seems sufficient. :^)
>
> Take check_local_user out of the picture by using a direct LDAP lookup
> in a condition instead and see if that fixes it? Do this on one of the
> three boxes, compare error rates.

I've changed the routers so that they don't use check_local_user
anymore, but:
user = $local_part
router_home_directory = USER_MAILDIR

where USER_MAILDIR is:
USER_MAILDIR = ${lookup ldap{ user=LDAPUSER pass=LDAPPASS
ldap:///ou=mail,ou=services,dc=bit,dc=nl?\
maildirectory?sub?(&(objectClass=bitMailAccount)(uid=
$local_part))}}

This works fine, but I do see an occasional:
Failed to find user "xxxx" from expanded string "$local_part" for the
localuser router

So, again I have the feeling that nscd is letting me down here, since I
expect that Exim uses 'user' to find the uid, and that fails.
Fortunately, Exim now tempfails and delivers the message later on.

Anyways, I think that if I let Exim do the ldap-lookup for the uid
directly, I don't need nscd at all? If I hand Exim a numeric uid for
'user', there's no need for translation of username -> uid.

Please correct me if I'm wrong..

Again, please reply to marks[at]bit.nl too, since I'm not the list itself.
And thanks for the replies.

--
Mark Schouten, Unix/NOC-engineer
BIT BV | info[at]bit.nl | +31 318 648688
MS8714-RIPE | B1FD 8E60 A184 F89A 450D A128 049B 1B19 9AD6 17FF

Attachments:

signature.asc (0.18 KB)

exim-users at spodhuis

May 7, 2008, 1:21 PM

Post #4 of 4 (143 views)
Permalink

Re: Ocasional 'Unrouteable address'-message with LDAP environment [In reply to]

On 2008-05-07 at 10:28 +0200, Mark Schouten wrote:
> So, again I have the feeling that nscd is letting me down here, since I
> expect that Exim uses 'user' to find the uid, and that fails.
> Fortunately, Exim now tempfails and delivers the message later on.

If you use the same LDAP query twice then the result is cached, so if
you search for two attributes and then use ${extract on that to extract
the two fields, you should still do only one LDAP query from Exim. In
fact, since there won't be an nscd->LDAP query, your LDAP load will go
down.

See "9.18 Format of data returned by LDAP"

If you use "exim -be" to test the queries and extracts, then you can use
"exim -d -be" to see the lookups and which results are or are not
cached.

Eg, if I query these:
----------------------------8< cut here >8------------------------------
${extract{uid}{${lookup ldap {ldapi:///ou=People,dc=spodhuis,dc=org?uid,homeDirectory?sub?cn=Phil Pennock}}}}

${extract{homeDirectory}{${lookup ldap {ldapi:///ou=People,dc=spodhuis,dc=org?uid,homeDirectory?sub?cn=Phil Pennock}}}}
----------------------------8< cut here >8------------------------------
then the second one shows the cached data being used.

> Anyways, I think that if I let Exim do the ldap-lookup for the uid
> directly, I don't need nscd at all? If I hand Exim a numeric uid for
> 'user', there's no need for translation of username -> uid.

Correct. spec.txt, "15. GENERIC OPTIONS FOR ROUTERS", "user" option
(search for a line matching ^-user- and you'll find one instance for the
Router option and one for the Transport option).

The other issue which might be increasing the load with nscd is the
groups lookup for the user; that might be unnecessary in your
environment, so explicitly setting "group" to something relevant might
reduce load?

----------------------------8< cut here >8------------------------------
When a router queues an address for a transport, and the transport does not
specify a user, the user given here is used when running the delivery process.
The user may be specified numerically or by name. If expansion fails, the error
is logged and delivery is deferred. This user is also used by the redirect
router when running a filter file. The default is unset, except when
check_local_user is set. In this case, the default is taken from the password
information. If the user is specified as a name, and group is not set, the
group associated with the user is used. See also initgroups and group and the
discussion in chapter 23.
----------------------------8< cut here >8------------------------------

-Phil

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact lists@gossamer-threads.com