Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: exim: dev

[Bug 1153] unknown lookup type

 

 

exim dev RSS feed   Index | Next | Previous | View Threaded


arekm at maven

Sep 15, 2011, 11:51 PM

Post #1 of 11 (1233 views)
Permalink
[Bug 1153] unknown lookup type

------- You are receiving this mail because: -------
You are on the CC list for the bug.

http://bugs.exim.org/show_bug.cgi?id=1153




--- Comment #1 from Arkadiusz Miskiewicz <arekm [at] maven> 2011-09-16 07:51:14 ---
Unfortunately with static I also get:
2011-09-16 08:50:38 [15116] 1R4SFy-0003vo-Pb unknown lookup type ""


--
Configure bugmail: http://bugs.exim.org/userprefs.cgi?tab=email

--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##


arekm at maven

Sep 15, 2011, 11:54 PM

Post #2 of 11 (1228 views)
Permalink
[Bug 1153] unknown lookup type [In reply to]

------- You are receiving this mail because: -------
You are on the CC list for the bug.

http://bugs.exim.org/show_bug.cgi?id=1153




--- Comment #2 from Arkadiusz Miskiewicz <arekm [at] maven> 2011-09-16 07:54:50 ---
I don't see any invalid lookup in here:

# grep '{lookup' /etc/mail/exim.conf | sed -e 's#SELECT.*#SELECT XXX#g'
domainlist local_domains_dns = lsearch;/etc/mail/locals :
lsearch;/etc/mail/locals_allhost : ${lookup mysql{VIRTUAL_DOMAINS}}
domainlist virtual_domains = ${lookup mysql{VIRTUAL_DOMAINS}}
domainlist our_domains = lsearch;/etc/mail/locals :
lsearch;/etc/mail/locals_allhost : ${lookup mysql{VIRTUAL_DOMAINS}}
# server_condition = ${lookup mysql{SELECT XXX
# server_condition = ${lookup mysql{SELECT XXX
# server_secret = ${lookup mysql{SELECT XXX
domains = ${lookup mysql{ROUTE_HOST}}
local_parts = ${lookup mysql{VIRTUAL_ACCOUNTS}}
route_list = * "${lookup mysql{ROUTE_HOST}}"
data = ${lookup{$local_part}wildlsearch{/etc/mail/aliases}}
data = ${lookup mysql{SELECT XXX
#data = ${lookup mysql{SELECT XXX
data = ${lookup mysql{SELECT XXX
data = ${lookup mysql{SELECT XXX
data = ${lookup mysql{SELECT XXX
unseen = ${if eq{${lookup mysql{SELECT XXX
{eq {${lookup mysql{SELECT XXX
local_parts = ${lookup mysql{VIRTUAL_ACCOUNTS}}
data = $local_part@${lookup mysql{ROUTE_HOST}}
command = /var/admin/bin/xx.pl "${lookup mysql{SELECT XXX
once_repeat = ${lookup mysql{SELECT XXX
from = ${escape:${lookup mysql{SELECT XXX
subject = "${rfc2047:${escape:${lookup mysql{SELECT XXX
${lookup mysql{SELECT XXX


--
Configure bugmail: http://bugs.exim.org/userprefs.cgi?tab=email

--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##


tlyons at ivenue

Sep 16, 2011, 6:51 AM

Post #3 of 11 (1222 views)
Permalink
Re: [Bug 1153] unknown lookup type [In reply to]

On Thu, Sep 15, 2011 at 11:54 PM, Arkadiusz Miskiewicz <arekm [at] maven> wrote:

> --- Comment #2 from Arkadiusz Miskiewicz <arekm [at] maven> 2011-09-16 07:54:50 ---
> I don't see any invalid lookup in here:
>

<snip>
> route_list = * "${lookup mysql{ROUTE_HOST}}"
> data = ${lookup{$local_part}wildlsearch{/etc/mail/aliases}}

It's complaining specifically about a lookup type named "". In all of
your lines, you specify a lookup type of "mysql". Well, in the second
line, what if the $local_part is ""? That's likely where your error
is coming from, aka bounce messages with a null envelope.

--
If Americans could eliminate sugary beverages, potatoes, white bread,
pasta, white rice and sugary snacks, we would wipe out almost all the
problems we have with weight and diabetes and other metabolic
diseases. -- Dr. Walter Willett, Harvard School of Public Health

--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##


tlyons at ivenue

Sep 16, 2011, 6:51 AM

Post #4 of 11 (1223 views)
Permalink
[Bug 1153] unknown lookup type [In reply to]

------- You are receiving this mail because: -------
You are on the CC list for the bug.

http://bugs.exim.org/show_bug.cgi?id=1153




--- Comment #3 from Todd Lyons <tlyons [at] ivenue> 2011-09-16 14:51:54 ---
On Thu, Sep 15, 2011 at 11:54 PM, Arkadiusz Miskiewicz <arekm [at] maven> wrote:

> --- Comment #2 from Arkadiusz Miskiewicz <arekm [at] maven>  2011-09-16 07:54:50 ---
> I don't see any invalid lookup in here:
>

<snip>
>  route_list = * "${lookup mysql{ROUTE_HOST}}"
>  data = ${lookup{$local_part}wildlsearch{/etc/mail/aliases}}

It's complaining specifically about a lookup type named "". In all of
your lines, you specify a lookup type of "mysql". Well, in the second
line, what if the $local_part is ""? That's likely where your error
is coming from, aka bounce messages with a null envelope.


--
Configure bugmail: http://bugs.exim.org/userprefs.cgi?tab=email
--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##


pdp at exim

Sep 17, 2011, 4:54 AM

Post #5 of 11 (1211 views)
Permalink
[Bug 1153] unknown lookup type [In reply to]

------- You are receiving this mail because: -------
You are on the CC list for the bug.

http://bugs.exim.org/show_bug.cgi?id=1153




--- Comment #4 from Phil Pennock <pdp [at] exim> 2011-09-17 12:54:01 ---
Empty lookup data does not translate to a missing lookup type. A lookup for a
bounce should be fine. Try it at an {{exim -be}} prompt.


--
Configure bugmail: http://bugs.exim.org/userprefs.cgi?tab=email

--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##


pdp at exim

Sep 17, 2011, 4:57 AM

Post #6 of 11 (1217 views)
Permalink
[Bug 1153] unknown lookup type [In reply to]

------- You are receiving this mail because: -------
You are on the CC list for the bug.

http://bugs.exim.org/show_bug.cgi?id=1153




--- Comment #5 from Phil Pennock <pdp [at] exim> 2011-09-17 12:57:21 ---
Arkadiusz, how common are these log messages?

Any chance of a debug log for a session generating them? Note that you can
turn debugging on in ACLs, going to files; it might be slow on a heavily loaded
system, but I need more data to go on, to find the cause of the problem.


--
Configure bugmail: http://bugs.exim.org/userprefs.cgi?tab=email

--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##


pdp at exim

Sep 17, 2011, 5:00 AM

Post #7 of 11 (1211 views)
Permalink
[Bug 1153] unknown lookup type [In reply to]

------- You are receiving this mail because: -------
You are on the CC list for the bug.

http://bugs.exim.org/show_bug.cgi?id=1153




--- Comment #6 from Phil Pennock <pdp [at] exim> 2011-09-17 13:00:32 ---
This might be ${quote_: somewhere.

Quoting styles for lookups reference code from the lookup; so the
${quote_mysql:foo} syntax takes "quote_mysql", strips off up to the _, and
calls search_findtype() to get the lookup type.

How about repeating that grep, this time for '{quote_' and seeing what crops
up?


--
Configure bugmail: http://bugs.exim.org/userprefs.cgi?tab=email

--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##


arekm at maven

Sep 17, 2011, 6:52 AM

Post #8 of 11 (1211 views)
Permalink
[Bug 1153] unknown lookup type [In reply to]

------- You are receiving this mail because: -------
You are on the CC list for the bug.

http://bugs.exim.org/show_bug.cgi?id=1153




--- Comment #7 from Arkadiusz Miskiewicz <arekm [at] maven> 2011-09-17 14:52:54 ---
Looks like the problem comes from this condition:

condition = ${if and {\
{match_address{${address:$h_From:}}{${address:$h_To:}}}
\
{match_domain{${domain:$h_From:}}{+local_domains_dns}}
\
}\
}


--
Configure bugmail: http://bugs.exim.org/userprefs.cgi?tab=email

--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##


arekm at maven

Sep 24, 2011, 1:14 AM

Post #9 of 11 (1141 views)
Permalink
[Bug 1153] unknown lookup type [In reply to]

------- You are receiving this mail because: -------
You are on the CC list for the bug.

http://bugs.exim.org/show_bug.cgi?id=1153




--- Comment #8 from Arkadiusz Miskiewicz <arekm [at] maven> 2011-09-24 09:14:14 ---
warn message = Test1
log_message = "Test1: ${address:$rh_From:}{${address:$rh_To:}}"

warn message = Test2
log_message = "Test2: ${address:$rh_To:}"
condition = ${if
match_address{${address:$rh_From:}}{${address:$rh_To:}}}

and

# tail -n 1000 /var/log/exim/main.log |exigrep 1R7NBx-0004px-R9
+++ 1R7NBx-0004px-R9 has not completed +++
2011-09-24 10:02:34 [18597] 1R7NBx-0004px-R9 H=(mailin-ng2.xxx)
[193.25.166.32]:60550 I=[aaa.ip]:25 Warning: "Test1:
some [at] email{"Undisclosed-Recipient:;"@mailin-ng2.xxx}"
2011-09-24 10:02:34 [18597] 1R7NBx-0004px-R9 unknown lookup type ""

So two problems here:

1) no idea why it yelds lookup type error on this

2) "unknown lookup type" should also log some things about the message.
Currently the only thing it logs is: 'unknown lookup type ""'. That's the whole
log about message. Nothing else gets logged... so you can't even tell what
message was lost.


${quote} doesn't help:
condition = ${if
match_address{${quote:${address:$rh_From:}}}{${quote:${address:$rh_To:}}}}


--
Configure bugmail: http://bugs.exim.org/userprefs.cgi?tab=email

--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##


pdp at exim

Sep 24, 2011, 3:45 AM

Post #10 of 11 (1152 views)
Permalink
[Bug 1153] unknown lookup type [In reply to]

------- You are receiving this mail because: -------
You are on the CC list for the bug.

http://bugs.exim.org/show_bug.cgi?id=1153




--- Comment #9 from Phil Pennock <pdp [at] exim> 2011-09-24 11:45:09 ---
You keep censoring the samples you provide, which makes it hard to examine
what's happening. You also have something weird happening, because:
To: Undisclosed-Recipient:;
is valid and should not have been treated as a left-hand-side and then
qualified. How did that address come to exist?

Question 1: did you built Exim yourself? If so, why are you using dynamically
loaded lookups? They slow things down and add complexity to the upgrade
process. They're intended for vendors who distribute packages to others, who
might not have all of the dependencies installed. Those packages should take
care of ensuring that a lookup module is never out of sync with the main Exim
binary, to avoid weird failure modes.

The error message comes from search_findtype(), perhaps called via
search_findtype_partial(). It means that the internal table of searches did
not contain the entry and the string being looked up was literally the empty
string.

As previously noted, one way to get that is with quote_<type> without
specifying a type, so:

$ exim -be
> ${quote_:foobar}
Failed: unknown lookup type ""

Normal lookups without a type specified return "missing lookup type" if an
alphabetic character isn't found where the lookup type should be. Which leads
to method 2 for getting this error message:

> ${lookup {foo}partial-{bar}}
Failed: unknown lookup type ""
> ${lookup {foo}partial-*{bar}}
Failed: unknown lookup type ""

The third place it can come from is for string matching, for lookups of the
form:

cdb;/path/to/cdbfile

On a hunch, I added to "domainlist local_domains" the item: ;fred

$ exim -C /etc/exim/test.conf -bt foobar
LOG: MAIN PANIC DIE
unknown lookup type ""
$ exim -C /etc/exim/test.conf -be
> ${if match_domain{example.com}{+local_domains}}
2011-09-24 10:11:08 [81089] unknown lookup type ""

Hrm. You had:

"Undisclosed-Recipient:;"@mailin-ng2.xxx

So let's define:
addresslist foo = Undisclosed-Recipient:;@mailin-ng2.xxx
then:
$ exim -C /etc/exim/test.conf -be
> ${if match_address{snert [at] example}{+foo}}
2011-09-24 10:16:34 [81162] unknown lookup type ""

(Keeping the quotes in that addresslist definition leads to a syntax error)

So let's create "/tmp/datasource" containing one line:
foo [at] example: Undisclosed-Recipient:;@mailin-ng2.xxx
because this lets us have a lookup return data like that, much as your MySQL
might be doing:

$ exim -be
> ${lookup {foo [at] example}lsearch{/tmp/datasource}}
Undisclosed-Recipient:;@mailin-ng2.xxx

I suspect that somewhere, "Undisclosed-Recipient:;" is being included as an
item in an addresslist/... as a result of one of your queries.

addresslist snert = ${lookup {$sender_address}lsearch{/tmp/datasource}}

$ exim -C /etc/exim/test.conf -f foo [at] example -be
> ${if match_address{wibble [at] example}{+snert}}
2011-09-24 10:20:19 [81410] unknown lookup type ""


So at this point I think you're constructing a list somewhere, based on a
${lookup} to insert text into the list as list items, as a string expansion,
rather than using a lookup as member _of_ the list.

String expansion happens first. Results are parsed for list separators.
Result is then iterated over, and a leading semi-colon can be bad. You're
effectively eval'ing your data

You want your lookup to happen as a defined lookup, not as a string expansion
item, so that the results aren't subject to being examined as anything other
than raw data.

Thus for my example above:

addresslist snert = lsearch;/tmp/datasource

and for you, I suggest auditing the data in your MySQL database as returned by
the VIRTUAL_DOMAINS lookup, and look at rewriting the lookup to be of the form:

mysql;SELECT something FROM somewhere WHERE col1 = '${quote_mysql:$domain}'


--
Configure bugmail: http://bugs.exim.org/userprefs.cgi?tab=email

--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##


arekm at maven

Sep 24, 2011, 4:27 AM

Post #11 of 11 (1142 views)
Permalink
[Bug 1153] unknown lookup type [In reply to]

------- You are receiving this mail because: -------
You are on the CC list for the bug.

http://bugs.exim.org/show_bug.cgi?id=1153




--- Comment #10 from Arkadiusz Miskiewicz <arekm [at] maven> 2011-09-24 12:27:51 ---
(In reply to comment #9)
> You keep censoring the samples you provide, which makes it hard to examine
> what's happening.

Sorry about that, can supply raw samples via private/irc/jabber.

> You also have something weird happening, because:
> To: Undisclosed-Recipient:;
> is valid and should not have been treated as a left-hand-side and then
> qualified. How did that address come to exist?

The two warning acl from last my comment were used in acl_smtp_data, so this To
comes from some remote client (it's MX server).

> Question 1: did you built Exim yourself? If so, why are you using dynamically
> loaded lookups? They slow things down and add complexity to the upgrade
> process. They're intended for vendors who distribute packages to others, who
> might not have all of the dependencies installed. Those packages should take
> care of ensuring that a lookup module is never out of sync with the main Exim
> binary, to avoid weird failure modes.

Yes, I use dynamic modules but I always build my own rpm package first, so
nothing is out of sync here. Old ones were uninstalled after upgrading my own
rpm exim package (This exim is used in pld-linux.org distro here).

> On a hunch, I added to "domainlist local_domains" the item: ;fred
>
> $ exim -C /etc/exim/test.conf -bt foobar
> LOG: MAIN PANIC DIE
> unknown lookup type ""
> $ exim -C /etc/exim/test.conf -be
> > ${if match_domain{example.com}{+local_domains}}
> 2011-09-24 10:11:08 [81089] unknown lookup type ""
>
> Hrm. You had:
>
> "Undisclosed-Recipient:;"@mailin-ng2.xxx

Note that here the only censored thing is "xxx" domain.

> So let's define:
> addresslist foo = Undisclosed-Recipient:;@mailin-ng2.xxx
> then:
> $ exim -C /etc/exim/test.conf -be
> > ${if match_address{snert [at] example}{+foo}}
> 2011-09-24 10:16:34 [81162] unknown lookup type ""
>
> (Keeping the quotes in that addresslist definition leads to a syntax error)
>
> So let's create "/tmp/datasource" containing one line:
> foo [at] example: Undisclosed-Recipient:;@mailin-ng2.xxx
> because this lets us have a lookup return data like that, much as your MySQL
> might be doing:
>
> $ exim -be
> > ${lookup {foo [at] example}lsearch{/tmp/datasource}}
> Undisclosed-Recipient:;@mailin-ng2.xxx
>
> I suspect that somewhere, "Undisclosed-Recipient:;" is being included as an
> item in an addresslist/... as a result of one of your queries.
>
> addresslist snert = ${lookup {$sender_address}lsearch{/tmp/datasource}}
>
> $ exim -C /etc/exim/test.conf -f foo [at] example -be
> > ${if match_address{wibble [at] example}{+snert}}
> 2011-09-24 10:20:19 [81410] unknown lookup type ""
>
>
> So at this point I think you're constructing a list somewhere, based on a
> ${lookup} to insert text into the list as list items, as a string expansion,
> rather than using a lookup as member _of_ the list.
>
> String expansion happens first. Results are parsed for list separators.
> Result is then iterated over, and a leading semi-colon can be bad. You're
> effectively eval'ing your data

But why some other things have influence on single acl entry?

warn message = Test2
condition = ${if match_address{${address:$rh_From:}}{${address:$rh_To:}}}

If I remove these two lines from acl_smtp_data then problem is gone. There
aren't any address lists used in it etc.

I looked into last 100 cases here and always $rh_To was in form:
"Undisclosed-Recipient:;"@some.domain.pl

Just tested this on my simple, laptop exim config with above 2 lines added in
content acl:

250-tarm.mavenx.pl Hello test [12.12.12.12]
250-SIZE 52428800
250-DSN
250-PIPELINING
250 HELP
mail from: <arekm [at] mavenx>
250 OK
rcpt to: <arekm [at] tarm>
>>> using ACL "acl_check_rcpt"
>>> processing "accept"
>>> check hosts = :
>>> host in ":"? no (end of list)
>>> accept: condition test failed
>>> processing "deny"
>>> check domains = +local_domains
>>> tarm.maven.pl in "@"? yes (matched "@")
>>> tarm.maven.pl in "+local_domains"? yes (matched "+local_domains")
>>> check local_parts = ^[.] : ^.*[@%!/|]
>>> arekm in "^[.] : ^.*[@%!/|]"? no (end of list)
>>> deny: condition test failed
>>> processing "deny"
>>> check domains = !+local_domains
>>> tarm.maven.pl in "!+local_domains"? no (matched "!+local_domains" - cached)
>>> deny: condition test failed
>>> processing "accept"
>>> check local_parts = postmaster
>>> arekm in "postmaster"? no (end of list)
>>> accept: condition test failed
>>> processing "require"
>>> check verify = sender
>>> maven.pl in "tarm.mavenx.pl"? no (end of list)
>>> arekm [at] mavenx in "arekm [at] tarm"? no (end of list)
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> routing arekm [at] mavenx
>>> maven.pl in "@"? no (end of list)
>>> maven.pl in "! +local_domains"? yes (end of list)
>>> calling relay router
>>> maven.pl in "*"? yes (matched "*")
>>> routed by relay router
>>> ----------- end verify ------------
>>> require: condition test succeeded
>>> processing "accept"
>>> check domains = +local_domains
>>> tarm.mavenx.pl in "+local_domains"? yes (matched "+local_domains" - cached)
>>> check verify = recipient
>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> routing arekm [at] tarm
>>> tarm.mavenx.pl in "! +local_domains"? no (matched "! +local_domains" - cached)
>>> tarm.mavenx.pl in "! +local_domains"? no (matched "! +local_domains" - cached)
>>> calling system_aliases router
>>> system_aliases router declined for arekm [at] tarm
>>> calling localuser router
>>> routed by localuser router
>>> ----------- end verify ------------
>>> accept: condition test succeeded
>>> DSN-AJ(smtp-in): orcpt: NULL flags: 0
250 Accepted
DATA
354 Enter message, ending with "." on a line by itself
From: <arekm [at] mavenx>
To: "Undisclosed-Recipient:;"@some.domain.pl
>>> host in ignore_fromline_hosts? no (option unset)

PDKIM >> Hashed body data, canonicalized >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> mavenx.pl in "tarm.mavenx.pl"? no (end of list)
>>> arekm [at] mavenx in "arekm [at] tarm"? no (end of list)
blabla
.
PDKIM <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
>>> using ACL "acl_check_content"
>>> processing "warn"
>>> mavenx.pl in ""Undisclosed-Recipient"? no (end of list)
LOG: 1R7QBc-0005Sl-Nj unknown lookup type ""


--
Configure bugmail: http://bugs.exim.org/userprefs.cgi?tab=email

--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##

exim dev RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.