Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Python: Dev

x86 XP trunk failure

 

 

Python dev RSS feed   Index | Next | Previous | View Threaded


alan.mcintyre at gmail

Sep 5, 2007, 6:09 AM

Post #1 of 8 (334 views)
Permalink
x86 XP trunk failure

Hi all,

My build slave (http://www.python.org/dev/buildbot/trunk/x86%20XP%20trunk)
keeps failing because of a crash that appears to be in the bsddb
module. I assume the master deems the slave to be lost because it's
sitting there waiting on me to make a choice on the "debug/abort"
dialog box.

I can provide details if anybody needs them. I just figured somebody
might want to know that this is actual build/test problem instead of
some kind of issue with the internet connection here.

Thanks,
Alan
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


martin at v

Sep 5, 2007, 6:34 AM

Post #2 of 8 (306 views)
Permalink
Re: x86 XP trunk failure [In reply to]

> My build slave (http://www.python.org/dev/buildbot/trunk/x86%20XP%20trunk)
> keeps failing because of a crash that appears to be in the bsddb
> module. I assume the master deems the slave to be lost because it's
> sitting there waiting on me to make a choice on the "debug/abort"
> dialog box.

What branch, and for how long has this dialog been sitting around?

For crashes in 3.0, there should not be any such dialogs anymore, but
there may have been before I turned them off.

> I can provide details if anybody needs them. I just figured somebody
> might want to know that this is actual build/test problem instead of
> some kind of issue with the internet connection here.

Thanks. You can discard any such dialogs - most likely, they really were
from the 3.0 branch, which is known to crash in bsddb.

Regards,
Martin
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


alan.mcintyre at gmail

Sep 5, 2007, 6:40 AM

Post #3 of 8 (307 views)
Permalink
Re: x86 XP trunk failure [In reply to]

On 9/5/07, "Martin v. Löwis" <martin [at] v> wrote:
> > My build slave (http://www.python.org/dev/buildbot/trunk/x86%20XP%20trunk)
> > keeps failing because of a crash that appears to be in the bsddb
> > module. I assume the master deems the slave to be lost because it's
> > sitting there waiting on me to make a choice on the "debug/abort"
> > dialog box.
>
> What branch, and for how long has this dialog been sitting around?

It's the trunk; at the moment the debugger is sitting there with
python_d.exe at a breakpoint. The current instance of python_d being
debugged is only a day or so old. I don't know when this problem
started happening, but I think it's been a while (it was happening for
all the visible builds on the dashboard when I first noticed it a day
or two ago).

> For crashes in 3.0, there should not be any such dialogs anymore, but
> there may have been before I turned them off.
>
> > I can provide details if anybody needs them. I just figured somebody
> > might want to know that this is actual build/test problem instead of
> > some kind of issue with the internet connection here.
>
> Thanks. You can discard any such dialogs - most likely, they really were
> from the 3.0 branch, which is known to crash in bsddb.

Ok.
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


db3l.net at gmail

Sep 5, 2007, 12:48 PM

Post #4 of 8 (301 views)
Permalink
Re: x86 XP trunk failure [In reply to]

"Martin v. Löwis" <martin [at] v> writes:

>> My build slave (http://www.python.org/dev/buildbot/trunk/x86%20XP%20trunk)
>> keeps failing because of a crash that appears to be in the bsddb
>> module. I assume the master deems the slave to be lost because it's
>> sitting there waiting on me to make a choice on the "debug/abort"
>> dialog box.
>
> What branch, and for how long has this dialog been sitting around?
>
> For crashes in 3.0, there should not be any such dialogs anymore, but
> there may have been before I turned them off.

I think there may actually be an issue here, if only with the tests,
even though 3.0 does suppress the dialog. I think I started noticing
this in the first build after bringing my buildbot online, so I think
on Sep 1. I had manually done on a build on Aug 28 (running the
buildbot batch file interactively) without the problem, but I haven't
been able to find any relevant source tree changes in that interval.
Re-fetching from that date has the problem, and I had blown away my
older tree when starting up the buildbot officially (of course :-().

At least for me, it's happening on 2.5 and trunk (hard to tell about
3.0, but that's dying without a dialog), so I thought it might have
been something backported. But it also appears common to more
platforms than just Windows - it's just Windows that pops up that
dialog.

In my case, the actual dialog doesn't pop up until the end of the
tests, and it seems to be occurring only if test_bsddb3 has run during
the tests. On other platforms, it just shows up as a warning message,
which doesn't serve to mark the tests as failing (e.g., OS X and
FreeBSD) - at the of the test you get a message of:

warning: DBTxn aborted in destructor. No prior commit() or abort().

which I tracked back to an abort() call within the bsddb library as
final destruction is happening at Python exit. (When clearing the
test_bsddb module, and the bsddb wrapper tries to access a log file
related to an open transaction). So perhaps there's an issue with how
one or more of the tests are constructed, or cleanup or something. I
haven't narrowed it down further yet though.

As with Alan, more details are available as needed. While it seems to
show up in the full test run on more platforms, I have a harder time
forcing it by just running test_bsddb3 on FreeBSD, for example, while I
get the dialog consistently on Windows.

-- David

_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


db3l.net at gmail

Sep 5, 2007, 1:25 PM

Post #5 of 8 (305 views)
Permalink
Re: x86 XP trunk failure [In reply to]

I previously wrote:

> (...)
> which I tracked back to an abort() call within the bsddb library as
> final destruction is happening at Python exit. (When clearing the
> test_bsddb module, and the bsddb wrapper tries to access a log file
> related to an open transaction). (...)

For those more familiar with bsddb, it's the test_1413192.py module in
lib/bsddb/test that tickles the problem. It should have been more
obvious, since I saw the 1413192 in the module name during exit
cleanup, but mentally ignored it as an internal identifier of some
sort.

The test module clearly leaves an open transaction, but also purges
its working directory, so maybe that's why the log file is missing.
But since the test was specifically against object destruction, I'm
not sure how best to restructure (maybe make env_name into a class
that only prunes the directory in __del__? Although that would affect
GC and thus destruction order too).

This test has been around a bit, but the pruning of the directory was
backported recently, which is probably the source of the problems.

-- David

_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


martin at v

Sep 5, 2007, 2:18 PM

Post #6 of 8 (303 views)
Permalink
Re: x86 XP trunk failure [In reply to]

> warning: DBTxn aborted in destructor. No prior commit() or abort().

I have seen these as well. bsddb isn't very forgiving when you have
a Python exception inside a bsddb transaction, in the test suite.
IIRC, the exception will abort the transaction, then the unittest
fixture teardown will close the environment, and that will cause
a bsddb crash because something is getting released that does not
exist anymore. When I last looked at it, I did not see an easy way to
fix it; contributions are welcome.

Regards,
Martin
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


db3l.net at gmail

Sep 5, 2007, 3:35 PM

Post #7 of 8 (300 views)
Permalink
Re: x86 XP trunk failure [In reply to]

"Martin v. Löwis" <martin [at] v> writes:

>> warning: DBTxn aborted in destructor. No prior commit() or abort().
>
> I have seen these as well. bsddb isn't very forgiving when you have
> a Python exception inside a bsddb transaction, in the test suite.
> IIRC, the exception will abort the transaction, then the unittest
> fixture teardown will close the environment, and that will cause
> a bsddb crash because something is getting released that does not
> exist anymore. When I last looked at it, I did not see an easy way to
> fix it; contributions are welcome.

One thing I tried that seems to work fairly well for this case is to
encapsulate much of the module-level code in the test into a class
instance. That way the module-level code can instantiate and destroy
the class instance rather than waiting for the interpreter exit for
the latter.

It definitely resolves this current issue, but when I reverted the
changes to _bsddb.c that were originally made in conjunction with this
test, it still seemed to pass the test. So I tried the reverted
module with the original test code and it still passes.

So I'm not entirely sure that the test is enforcing anything at this
point, or at least I'm not sure how to be absolutely positive that the
change will continue to enforce what the existing code used to test.

But I can open a ticket with the proposed changes if that would help.

-- David

_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


db3l.net at gmail

Sep 5, 2007, 4:01 PM

Post #8 of 8 (301 views)
Permalink
Re: x86 XP trunk failure [In reply to]

I wrote:

> But I can open a ticket with the proposed changes if that would help.

Figure it can't hurt - I've created issue 1112 with the proposed patch
to the test_1413192.py module.

-- David

_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

Python dev RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.