Mailing List Archive: exim: dev

testsuite breakage

Index | Next | Previous | View Threaded

jgh at wizmail

May 9, 2012, 11:03 AM

Post #1 of 10 (408 views)
Permalink

testsuite breakage

9ee44ef breaks testsuite case 0022 for me. Not run much further yet.
--
Jeremy

--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##

pdp at exim

May 9, 2012, 12:10 PM

Post #2 of 10 (405 views)
Permalink

Re: testsuite breakage [In reply to]

On 2012-05-09 at 19:03 +0100, Jeremy Harris wrote:
> 9ee44ef breaks testsuite case 0022 for me. Not run much further yet.

A recentish test suite change added regexp syntax to runtest not
supported by my install of Perl. I started the long and painful system
upgrade last night, am still reinstalling Perl modules. When that's all
done, I think I have a way to test the -bw functionality waiting in my
git client, and will take a look at anything else I've broken.

-Phil

--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##

pdp at exim

May 9, 2012, 10:56 PM

Post #3 of 10 (398 views)
Permalink

Re: testsuite breakage [In reply to]

On 2012-05-09 at 19:03 +0100, Jeremy Harris wrote:
> 9ee44ef breaks testsuite case 0022 for me. Not run much further yet.

That's what I get for coding while resetting my body clock. I see the
change, I inserted another code block between a main test and its else.
Clearly I didn't re-test the non-bw case after moving os_getloadavg().

Sorry for the breakage.

Now that I have an upgraded Perl and recovered the resulting breakage, I
have a working runtest and I found the problem fairly quickly. I'm
waiting on the test suite to finish before I push the fix, but it should
be out fairly soon.

-Phil

--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##

pdp at exim

May 9, 2012, 11:32 PM

Post #4 of 10 (397 views)
Permalink

Re: testsuite breakage [In reply to]

On 2012-05-10 at 01:56 -0400, Phil Pennock wrote:
> Now that I have an upgraded Perl and recovered the resulting breakage, I
> have a working runtest and I found the problem fairly quickly. I'm
> waiting on the test suite to finish before I push the fix, but it should
> be out fairly soon.

None of the test complaints I'm seeing now look like the result of any
of my changes.

Test 0152 has an added Sender: header.

Test 0288 has some berkeley DB errors and a missing dir; nothing I've
touched, I don't think.

Test 0358 has changes in retry times.

Test 0369 failed to replace /bin/sh with SHELL. SHELL=/bin/sh in my
environment, I don't know why it's not replacing content.

Test 0373 is missing "sock error". This was changed output accepted in
9d0b48b5; I suspect an OS-specific variance which we should track down.

Test 0548 appears similar to 0358.

Test 1003 test 7 times out on connect. Hope I haven't messed up my
network interfaces recently. I see this test-case was recently fixed.
I'd have to check my mail archives to see if this has worked for me in
the past (and I really need to not be awake right now).

OpenSSL tests need updating for 8BITMIME and for TLS1.2 negotiation,
Jeremy only fixed the GnuTLS ones? I'll do that tomorrow. That's 2100
and 3460 ranges.

confs/3650 appears to have disappeared?

The rest ran okay in a run which:
Omitting tests in 2000-GnuTLS (missing support GnuTLS)
Omitting tests in 3000-Perl (missing support Perl)
Omitting tests in 3100-dlfunc (missing support Expand_dlfunc)
Omitting tests in 3200-testdb (missing lookup testdb)
Omitting tests in 3450-plaintext-GnuTLS (missing support GnuTLS)
Omitting tests in 4000-scanning (missing running SpamAssassin)
Omitting tests in 4950-translate-ip (missing support translate_ip_address)

Fix pushed.

-Phil

--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##

jgh at wizmail

May 10, 2012, 1:25 PM

Post #5 of 10 (394 views)
Permalink

Re: testsuite breakage [In reply to]

On 2012-05-10 07:32, Phil Pennock wrote:
> Test 0152 has an added Sender: header.
>
> Test 0288 has some berkeley DB errors and a missing dir; nothing I've
> touched, I don't think.

I don't get those.

>
> Test 0358 has changes in retry times.

Do you see negative numbers? I get this, intermittently,
and possibly associated with runs close to midnight.....

>
> Test 0369 failed to replace /bin/sh with SHELL. SHELL=/bin/sh in my
> environment, I don't know why it's not replacing content.

Not seen that one.

>
> Test 0373 is missing "sock error". This was changed output accepted in
> 9d0b48b5; I suspect an OS-specific variance which we should track down.

Hmm. One of mine around there is an intermittent. Might be 0393 or that one.

>
> Test 0548 appears similar to 0358.
>
> Test 1003 test 7 times out on connect. Hope I haven't messed up my
> network interfaces recently. I see this test-case was recently fixed.
> I'd have to check my mail archives to see if this has worked for me in
> the past (and I really need to not be awake right now).
>
> OpenSSL tests need updating for 8BITMIME and for TLS1.2 negotiation,
> Jeremy only fixed the GnuTLS ones? I'll do that tomorrow. That's 2100
> and 3460 ranges.
>
> confs/3650 appears to have disappeared?

Looks like it was renumbered 9350 soon after intro,
in 2006:
49c605e7d53699661
ef85a7860b2750dd180
b8b35975734f3136708
7befa435e5664f43d90b

I've got a new fail on 0230. "Client timed out" during the second subtest;
didn't get the 250 after "."

(test run in progress)
Cheers,
Jeremy

I'd say confs/3650 could be wiped?

--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##

jgh at wizmail

May 10, 2012, 1:48 PM

Post #6 of 10 (394 views)
Permalink

Re: testsuite breakage [In reply to]

On 2012-05-10 21:25, Jeremy Harris wrote:
> I've got a new fail on 0230. "Client timed out" during the second subtest;
> didn't get the 250 after "."

Sigh. Not repeatable.
-J

--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##

jgh at wizmail

May 12, 2012, 1:00 PM

Post #7 of 10 (373 views)
Permalink

Re: testsuite breakage [In reply to]

On 2012-05-10 07:32, Phil Pennock wrote:
> Test 1003 test 7 times out on connect. Hope I haven't messed up my
> network interfaces recently. I see this test-case was recently fixed.
> I'd have to check my mail archives to see if this has worked for me in
> the past (and I really need to not be awake right now).

I swear something's changed in dealing with interfaces. I'm getting
intermittent:

Basic-ipv6/1001 recognizing IPv6 address in HELO/EHLO
===============f test-mainlog-munged with log/1001 failed
Line 11 of "test-mainlog-munged" does not match line 11 of "log/1001".
----------
1999-03-02 09:44:33 exim x.yz daemon started: pid=pppp, no queue runs, listening for SMTP on port 1225 (IPv6 with IPv4)
----------
1999-03-02 09:44:33 exim x.yz daemon started: pid=pppp, no queue runs, listening for SMTP on port 1225 (IPv6 and IPv4)
===============
1 difference found.
"test-mainlog-munged" contains 15 lines; "log/1001" contains 15 lines.

Note the one-word difference (with vs. and). It relates to
how interfaces and addresses are laid out, somehow
(cf. daemon.c 1640, 1644).

I'm also seeing intermittent fails on multiple tests, with the
symptom "Client timed out" in stdout. Don't know if that's
relayed to the above but it's also a recent development.

--
Cheers,
Jeremy

--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##

pdp at exim

May 12, 2012, 6:37 PM

Post #8 of 10 (368 views)
Permalink

Re: testsuite breakage [In reply to]

On 2012-05-12 at 21:00 +0100, Jeremy Harris wrote:
> Basic-ipv6/1001 recognizing IPv6 address in HELO/EHLO
> ===============f test-mainlog-munged with log/1001 failed
> Line 11 of "test-mainlog-munged" does not match line 11 of "log/1001".
> ----------
> 1999-03-02 09:44:33 exim x.yz daemon started: pid=pppp, no queue runs, listening for SMTP on port 1225 (IPv6 with IPv4)
> ----------
> 1999-03-02 09:44:33 exim x.yz daemon started: pid=pppp, no queue runs, listening for SMTP on port 1225 (IPv6 and IPv4)

> Note the one-word difference (with vs. and). It relates to
> how interfaces and addresses are laid out, somehow
> (cf. daemon.c 1640, 1644).

On some OSes, an IPv6 socket can accept connections made with both IPv4
and IPv6 protocols, and will use IPv6-mapped IPv4 addresses in that case
(::ffff:ipv4 I think).

On some OSes, an IPv6 socket can't.

On some OSes, it's choosable, both as a system-wide default and as a
per-socket option. OpenBSD defaulted this off, reasoning that unaware
developers not knowing how to handle ::ffff:ipv4 address forms would
mess up their ACL logic.

Eg, FreeBSD sysctl "net.inet6.ip6.v6only" for the system-wide knob. And
later socket standards added IPV6_V6ONLY as a sockopt at socket level
IPPROTO_IPV6.

Are you testing on two different systems with different defaults, or
could some daemon's startup script be doing something stupid like tuning
the system-wide default?

-Phil

--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##

jgh at wizmail

May 13, 2012, 5:45 AM

Post #9 of 10 (357 views)
Permalink

Re: testsuite breakage [In reply to]

On 2012-05-13 02:37, Phil Pennock wrote:
> On 2012-05-12 at 21:00 +0100, Jeremy Harris wrote:
>> Basic-ipv6/1001 recognizing IPv6 address in HELO/EHLO
>> ===============f test-mainlog-munged with log/1001 failed
>> Line 11 of "test-mainlog-munged" does not match line 11 of "log/1001".
>> ----------
>> 1999-03-02 09:44:33 exim x.yz daemon started: pid=pppp, no queue runs, listening for SMTP on port 1225 (IPv6 with IPv4)
>> ----------
>> 1999-03-02 09:44:33 exim x.yz daemon started: pid=pppp, no queue runs, listening for SMTP on port 1225 (IPv6 and IPv4)
>
>> Note the one-word difference (with vs. and). It relates to
>> how interfaces and addresses are laid out, somehow
>> (cf. daemon.c 1640, 1644).
>
> On some OSes, an IPv6 socket can accept connections made with both IPv4
> and IPv6 protocols, and will use IPv6-mapped IPv4 addresses in that case
> (::ffff:ipv4 I think).
>
> On some OSes, an IPv6 socket can't.
>
> On some OSes, it's choosable, both as a system-wide default and as a
> per-socket option. OpenBSD defaulted this off, reasoning that unaware
> developers not knowing how to handle ::ffff:ipv4 address forms would
> mess up their ACL logic.
>
> Eg, FreeBSD sysctl "net.inet6.ip6.v6only" for the system-wide knob. And
> later socket standards added IPV6_V6ONLY as a sockopt at socket level
> IPPROTO_IPV6.
>
> Are you testing on two different systems with different defaults, or
> could some daemon's startup script be doing something stupid like tuning
> the system-wide default?

I'm only testing on the one system; Scientific Linux 6 (a Fedora derivative).

The coding looking for that dual-use socket is (IMHO) highly dubious. It will
false-pos when the IPv4 socket gets a "real" EADDRINUSE. I think I'm
getting such - I'm also getting them for the initial (wildcard) socket
whether IPv4 or v6. They seem to last 5-10 seconds.

The manpage claims IPV6_V6ONLY is supported.

Testcase 0561 is my current repeat-by for IPv4; 1001 for IPv6.

--
Jeremy

--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##

jgh at wizmail

May 13, 2012, 8:03 AM

Post #10 of 10 (354 views)
Permalink

Re: testsuite breakage [In reply to]

On 2012-05-13 13:45, Jeremy Harris wrote:
> The coding looking for that dual-use socket is (IMHO) highly dubious. It will
> false-pos when the IPv4 socket gets a "real" EADDRINUSE. I think I'm
> getting such - I'm also getting them for the initial (wildcard) socket
> whether IPv4 or v6. They seem to last 5-10 seconds.

Right; as far as 1001 goes the issue was a socket being held onto by "runtest"
from the 0564 testcase. I'll be committing a change that takes care to close it.

The dual-use detection still feels wrong to me. I'm tempted to only call the
special-case test when IPV6_V6ONLY has not succeeded - but on further
reflection I wonder why exim should prefer separate v4 / v6 sockets rather
than a dual-use one, when the OS supports dual? Maybe I'll try a testsuite
run in dual-mode to find out.

--
Jeremy

--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads