Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Trac: Tickets

[The Trac Project] #3504: AssertionError in trac.db.pool.PooledConnection.__del__

  

 
First page Previous page 1 2 Next page Last page  View All Trac tickets RSS feed   Index | Next | Previous | View Threaded


noreply at edgewall

Aug 2, 2006, 4:46 AM

Post #1 of 41 (4180 views)
Permalink
[The Trac Project] #3504: AssertionError in trac.db.pool.PooledConnection.__del__
#3504: AssertionError in trac.db.pool.PooledConnection.__del__
--------------------------------+-------------------------------------------
Reporter: exarkun [at] divmod | Owner: jonas
Type: defect | Status: new
Priority: high | Milestone:
Component: general | Version: 0.9.6
Severity: normal | Keywords:
--------------------------------+-------------------------------------------
On nearly every request, my server logs this error:

{{{
Exception exceptions.AssertionError: <exceptions.AssertionError instance
at 0x89faeacc> in <bound method PooledConnection.__del__ of
<trac.db.pool.PooledConnection object at 0xb47cc88c>> ignored
}}}

--
Ticket URL: <http://trac.edgewall.org/ticket/3504>
The Trac Project <http://trac.edgewall.com/>
_______________________________________________
Trac-Tickets mailing list
Trac-Tickets [at] lists
http://lists.edgewall.com/mailman/listinfo/trac-tickets


noreply at edgewall

Aug 2, 2006, 4:48 AM

Post #2 of 41 (4080 views)
Permalink
Re: [The Trac Project] #3504: AssertionError in trac.db.pool.PooledConnection.__del__ [In reply to]
#3504: AssertionError in trac.db.pool.PooledConnection.__del__
--------------------------------+-------------------------------------------
Reporter: exarkun [at] divmod | Owner: jonas
Type: defect | Status: new
Priority: high | Milestone:
Component: general | Version: 0.9.6
Severity: normal | Resolution:
Keywords: |
--------------------------------+-------------------------------------------
Changes (by anonymous):

* cc: exarkun [at] divmod (added)

--
Ticket URL: <http://trac.edgewall.org/ticket/3504#comment:1>
The Trac Project <http://trac.edgewall.com/>
_______________________________________________
Trac-Tickets mailing list
Trac-Tickets [at] lists
http://lists.edgewall.com/mailman/listinfo/trac-tickets


noreply at edgewall

Aug 23, 2006, 8:57 AM

Post #3 of 41 (4075 views)
Permalink
Re: [The Trac Project] #3504: AssertionError in trac.db.pool.PooledConnection.__del__ [In reply to]
#3504: AssertionError in trac.db.pool.PooledConnection.__del__
--------------------------------+-------------------------------------------
Reporter: exarkun [at] divmod | Owner: jonas
Type: defect | Status: new
Priority: high | Milestone:
Component: general | Version: 0.9.6
Severity: normal | Resolution:
Keywords: |
--------------------------------+-------------------------------------------
Comment (by anonymous):

I sometimes get this error message too on FreeBSD 6.1 AMD64, with Apache
2.2.3 and FastCGI 2.4.2, using Trac 0.10-dev (r3623).

When this error occurs, the message is repeatedly written to the apache
error log until that partition gets filled. Restarting the webserver gets
things going again until the same problem occurs a few days later.

--
Ticket URL: <http://trac.edgewall.org/ticket/3504#comment:2>
The Trac Project <http://trac.edgewall.com/>
_______________________________________________
Trac-Tickets mailing list
Trac-Tickets [at] lists
http://lists.edgewall.com/mailman/listinfo/trac-tickets


noreply at edgewall

Aug 23, 2006, 9:33 AM

Post #4 of 41 (4080 views)
Permalink
Re: [The Trac Project] #3504: AssertionError in trac.db.pool.PooledConnection.__del__ [In reply to]
#3504: AssertionError in trac.db.pool.PooledConnection.__del__
--------------------------------+-------------------------------------------
Reporter: exarkun [at] divmod | Owner: jonas
Type: defect | Status: new
Priority: high | Milestone: 0.10.1
Component: general | Version: 0.9.6
Severity: critical | Resolution:
Keywords: |
--------------------------------+-------------------------------------------
Changes (by cboos):

* severity: normal => critical
* milestone: => 0.10.1

Comment:

Replying to [comment:2 anonymous]:
> ...
> When this error occurs, the message is repeatedly written to the apache
error log until that partition gets filled.

Ouch! that sounds serious. Putting a tentative milestone on this.

--
Ticket URL: <http://trac.edgewall.org/ticket/3504#comment:3>
The Trac Project <http://trac.edgewall.com/>
_______________________________________________
Trac-Tickets mailing list
Trac-Tickets [at] lists
http://lists.edgewall.com/mailman/listinfo/trac-tickets


noreply at edgewall

Aug 23, 2006, 10:52 AM

Post #5 of 41 (4080 views)
Permalink
Re: [The Trac Project] #3504: AssertionError in trac.db.pool.PooledConnection.__del__ [In reply to]
#3504: AssertionError in trac.db.pool.PooledConnection.__del__
--------------------------------+-------------------------------------------
Reporter: exarkun [at] divmod | Owner: jonas
Type: defect | Status: new
Priority: high | Milestone: 0.10.1
Component: general | Version: 0.9.6
Severity: critical | Resolution:
Keywords: |
--------------------------------+-------------------------------------------
Comment (by cboos):

First, what's the database backend you're using, and what are the versions
used (for the bindings and for the db itself)?

--
Ticket URL: <http://trac.edgewall.org/ticket/3504#comment:4>
The Trac Project <http://trac.edgewall.com/>
_______________________________________________
Trac-Tickets mailing list
Trac-Tickets [at] lists
http://lists.edgewall.com/mailman/listinfo/trac-tickets


noreply at edgewall

Aug 23, 2006, 1:46 PM

Post #6 of 41 (4084 views)
Permalink
Re: [The Trac Project] #3504: AssertionError in trac.db.pool.PooledConnection.__del__ [In reply to]
#3504: AssertionError in trac.db.pool.PooledConnection.__del__
--------------------------------+-------------------------------------------
Reporter: exarkun [at] divmod | Owner: jonas
Type: defect | Status: new
Priority: high | Milestone: 0.10.1
Component: general | Version: 0.9.6
Severity: critical | Resolution:
Keywords: |
--------------------------------+-------------------------------------------
Comment (by hpdl [at] oscommerce):

I apologize - inspecting the logs after this happened again shows another
error than originally reported in this ticket is being repeatedly added to
the error log file. I'll open a new ticket to describe the errors there. I
apologize again for the inconvenience caused.

--
Ticket URL: <http://trac.edgewall.org/ticket/3504#comment:5>
The Trac Project <http://trac.edgewall.com/>
_______________________________________________
Trac-Tickets mailing list
Trac-Tickets [at] lists
http://lists.edgewall.com/mailman/listinfo/trac-tickets


noreply at edgewall

Sep 18, 2006, 3:12 AM

Post #7 of 41 (4072 views)
Permalink
Re: [The Trac Project] #3504: AssertionError in trac.db.pool.PooledConnection.__del__ [In reply to]
#3504: AssertionError in trac.db.pool.PooledConnection.__del__
--------------------------------------+-------------------------------------
Reporter: exarkun [at] divmod | Owner: cboos
Type: defect | Status: new
Priority: high | Milestone: 0.10
Component: general | Version: devel
Severity: blocker | Resolution:
Keywords: database connection pool |
--------------------------------------+-------------------------------------
Changes (by cboos):

* keywords: => database connection pool
* owner: jonas => cboos
* version: 0.9.6 => devel
* severity: critical => blocker
* milestone: 0.10.1 => 0.10

Comment:

The `AssertionError` error reported here is indicative of a very serious
issue (see #3742, which I just closed as ''duplicate'' of this one).

Now the good news is that I think I understood the problem... The pool.py
code assumes that garbage collection of a !PooledConnection will happen
''in the same thread'' than the one in which the instance was created. I
believe this is a wrong assumption.
In turn, this messes up the logic of the pool, and after a while, when all
the slots for new connections have been exhausted, every new thread will
become stuck waiting on the `_available` condition variable, giving the
impression that the application freezes.

If that would explain my freezes on #3332, I would be very happy!

--
Ticket URL: <http://trac.edgewall.org/ticket/3504#comment:6>
The Trac Project <http://trac.edgewall.org/>

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Trac Tickets" group.
To post to this group, send email to trac-tickets [at] googlegroups
To unsubscribe from this group, send email to trac-tickets-unsubscribe [at] googlegroups
For more options, visit this group at http://groups.google.se/group/trac-tickets
-~----------~----~----~----~------~----~------~--~---


noreply at edgewall

Sep 19, 2006, 12:23 AM

Post #8 of 41 (4061 views)
Permalink
Re: [The Trac Project] #3504: AssertionError in trac.db.pool.PooledConnection.__del__ [In reply to]
#3504: AssertionError in trac.db.pool.PooledConnection.__del__
--------------------------------------+-------------------------------------
Reporter: exarkun [at] divmod | Owner: cboos
Type: defect | Status: assigned
Priority: high | Milestone: 0.10
Component: general | Version: devel
Severity: blocker | Resolution:
Keywords: database connection pool |
--------------------------------------+-------------------------------------
Changes (by cboos):

* status: new => assigned

Comment:

More detailed explanation:
- with tracd, each request is processed by a dedicated thread. It happens
(1) that some !PooledConnection created by this thread don't get garbage
collected before the thread exits.
- those !PooledConnection were therefore garbage collected in another
thread, and this triggered the assertion failed discussed above.
- however, even if the assertion is avoided by carrying the right thread
identifier around and the proper connection would be freed from the other
thread, a pysqlite !ProgrammingError would then be raised, because
pysqlite doesn't allow a connection object to be manipulated from the
thread in which it has been created.
- therefore, we must ensure that before the thread exists, the pysqlite
Connection object created by it is properly closed.

The attachment:force_close_at_thread_exit.patch implements that.

I'm not entirely satisfied with the patch, though, as we still force the
creation/disposal of a pysqlite connection, even if no db connection is
needed for processing the request (think Chrome).

Perhaps doing a `env.signal_thread_exits(threading._get_ident())`, that
gets propagated by the environment to the db pool would be a way...

----
(1) what I still don't understand is:
- why do we see this bug only ''now''? It's been there since ages.
- why does it seem to be triggered so quickly by the report module (copy
a report, modify the sql, delete it, repeat that a few times and bang, it
hangs)

--
Ticket URL: <http://trac.edgewall.org/ticket/3504#comment:7>
The Trac Project <http://trac.edgewall.org/>

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Trac Tickets" group.
To post to this group, send email to trac-tickets [at] googlegroups
To unsubscribe from this group, send email to trac-tickets-unsubscribe [at] googlegroups
For more options, visit this group at http://groups.google.se/group/trac-tickets
-~----------~----~----~----~------~----~------~--~---


noreply at edgewall

Sep 20, 2006, 2:13 PM

Post #9 of 41 (4042 views)
Permalink
Re: [The Trac Project] #3504: AssertionError in trac.db.pool.PooledConnection.__del__ [In reply to]
#3504: AssertionError in trac.db.pool.PooledConnection.__del__
--------------------------------------+-------------------------------------
Reporter: exarkun [at] divmod | Owner: cboos
Type: defect | Status: closed
Priority: high | Milestone: 0.10
Component: general | Version: devel
Severity: blocker | Resolution: fixed
Keywords: database connection pool |
--------------------------------------+-------------------------------------
Changes (by cboos):

* status: assigned => closed
* resolution: => fixed

Comment:

Fixed in r3756, by applying
attachment:selective_shutdown_at_thread_exit.patch (which also contained
r3755).

--
Ticket URL: <http://trac.edgewall.org/ticket/3504#comment:8>
The Trac Project <http://trac.edgewall.org/>

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Trac Tickets" group.
To post to this group, send email to trac-tickets [at] googlegroups
To unsubscribe from this group, send email to trac-tickets-unsubscribe [at] googlegroups
For more options, visit this group at http://groups.google.se/group/trac-tickets
-~----------~----~----~----~------~----~------~--~---


noreply at edgewall

Sep 20, 2006, 11:03 PM

Post #10 of 41 (4047 views)
Permalink
Re: [The Trac Project] #3504: AssertionError in trac.db.pool.PooledConnection.__del__ [In reply to]
#3504: AssertionError in trac.db.pool.PooledConnection.__del__
--------------------------------------+-------------------------------------
Reporter: exarkun [at] divmod | Owner: cboos
Type: defect | Status: closed
Priority: high | Milestone: 0.10
Component: general | Version: devel
Severity: blocker | Resolution: fixed
Keywords: database connection pool |
--------------------------------------+-------------------------------------
Comment (by cboos):

Ok, after a good night of sleep I just realized that though the fix seems
to work (I couldn't reproduce neither the !AssertionError nor the hangs
with r3756), it's still theoretically possible that a !PooledConnection
created in one thread gets GCed in another one...

I'll follow-up to r3756 shortly.

--
Ticket URL: <http://trac.edgewall.org/ticket/3504#comment:9>
The Trac Project <http://trac.edgewall.org/>

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Trac Tickets" group.
To post to this group, send email to trac-tickets [at] googlegroups
To unsubscribe from this group, send email to trac-tickets-unsubscribe [at] googlegroups
For more options, visit this group at http://groups.google.se/group/trac-tickets
-~----------~----~----~----~------~----~------~--~---


noreply at edgewall

Sep 21, 2006, 3:39 AM

Post #11 of 41 (4044 views)
Permalink
Re: [The Trac Project] #3504: AssertionError in trac.db.pool.PooledConnection.__del__ [In reply to]
#3504: AssertionError in trac.db.pool.PooledConnection.__del__
--------------------------------------+-------------------------------------
Reporter: exarkun [at] divmod | Owner: cboos
Type: defect | Status: closed
Priority: high | Milestone: 0.10
Component: general | Version: devel
Severity: blocker | Resolution: fixed
Keywords: database connection pool |
--------------------------------------+-------------------------------------
Comment (by cboos):

Well, attachment:even_more_threadsafe_pool-r3756.diff implements an even
more robust cleanup, which should really avoid the risk of !AssertError
and closing of a Connection from a different thread.

But this starts to become heavy, especially for backends which don't
support pooling in the first place ;)

--
Ticket URL: <http://trac.edgewall.org/ticket/3504#comment:10>
The Trac Project <http://trac.edgewall.org/>

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Trac Tickets" group.
To post to this group, send email to trac-tickets [at] googlegroups
To unsubscribe from this group, send email to trac-tickets-unsubscribe [at] googlegroups
For more options, visit this group at http://groups.google.se/group/trac-tickets
-~----------~----~----~----~------~----~------~--~---


noreply at edgewall

Sep 21, 2006, 7:14 PM

Post #12 of 41 (4042 views)
Permalink
Re: [The Trac Project] #3504: AssertionError in trac.db.pool.PooledConnection.__del__ [In reply to]
#3504: AssertionError in trac.db.pool.PooledConnection.__del__
--------------------------------------+-------------------------------------
Reporter: exarkun [at] divmod | Owner: cboos
Type: defect | Status: closed
Priority: high | Milestone: 0.10
Component: general | Version: devel
Severity: blocker | Resolution: fixed
Keywords: database connection pool |
--------------------------------------+-------------------------------------
Comment (by anonymous):

FWIW, I updated to r3756 and have seen the ignored AssertionError once
since then.

--
Ticket URL: <http://trac.edgewall.org/ticket/3504#comment:11>
The Trac Project <http://trac.edgewall.org/>

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Trac Tickets" group.
To post to this group, send email to trac-tickets [at] googlegroups
To unsubscribe from this group, send email to trac-tickets-unsubscribe [at] googlegroups
For more options, visit this group at http://groups.google.se/group/trac-tickets
-~----------~----~----~----~------~----~------~--~---


noreply at edgewall

Sep 21, 2006, 11:41 PM

Post #13 of 41 (4047 views)
Permalink
Re: [The Trac Project] #3504: AssertionError in trac.db.pool.PooledConnection.__del__ [In reply to]
#3504: AssertionError in trac.db.pool.PooledConnection.__del__
--------------------------------------+-------------------------------------
Reporter: exarkun [at] divmod | Owner: cboos
Type: defect | Status: reopened
Priority: high | Milestone: 0.10
Component: general | Version: devel
Severity: blocker | Resolution:
Keywords: database connection pool |
--------------------------------------+-------------------------------------
Changes (by cboos):

* status: closed => reopened
* resolution: fixed =>

Comment:

Oh, then I think it's worth applying the latest patch.

--
Ticket URL: <http://trac.edgewall.org/ticket/3504#comment:12>
The Trac Project <http://trac.edgewall.org/>

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Trac Tickets" group.
To post to this group, send email to trac-tickets [at] googlegroups
To unsubscribe from this group, send email to trac-tickets-unsubscribe [at] googlegroups
For more options, visit this group at http://groups.google.se/group/trac-tickets
-~----------~----~----~----~------~----~------~--~---


noreply at edgewall

Sep 22, 2006, 3:24 AM

Post #14 of 41 (4045 views)
Permalink
Re: [The Trac Project] #3504: AssertionError in trac.db.pool.PooledConnection.__del__ [In reply to]
#3504: AssertionError in trac.db.pool.PooledConnection.__del__
--------------------------------------+-------------------------------------
Reporter: exarkun [at] divmod | Owner: cboos
Type: defect | Status: reopened
Priority: high | Milestone: 0.10
Component: general | Version: devel
Severity: blocker | Resolution:
Keywords: database connection pool |
--------------------------------------+-------------------------------------
Comment (by cboos):

Above patch applied in r3763, minus the `print` this time ;)

--
Ticket URL: <http://trac.edgewall.org/ticket/3504#comment:13>
The Trac Project <http://trac.edgewall.org/>

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Trac Tickets" group.
To post to this group, send email to trac-tickets [at] googlegroups
To unsubscribe from this group, send email to trac-tickets-unsubscribe [at] googlegroups
For more options, visit this group at http://groups.google.se/group/trac-tickets
-~----------~----~----~----~------~----~------~--~---


noreply at edgewall

Sep 22, 2006, 6:08 AM

Post #15 of 41 (4059 views)
Permalink
Re: [The Trac Project] #3504: AssertionError in trac.db.pool.PooledConnection.__del__ [In reply to]
#3504: AssertionError in trac.db.pool.PooledConnection.__del__
--------------------------------------+-------------------------------------
Reporter: exarkun [at] divmod | Owner: cboos
Type: defect | Status: reopened
Priority: high | Milestone: 0.10
Component: general | Version: devel
Severity: blocker | Resolution:
Keywords: database connection pool |
--------------------------------------+-------------------------------------
Comment (by anonymous):

I'll update and try it out soon. In the mean time, I woke up this morning
to hundreds or thousands of occurrences of this tracebacks like this in my
logfile:


{{{
2006/09/22 07:06 CDT [-] Exception rendering:
2006/09/22 07:06 CDT [-] Unhandled Error
Traceback (most recent call last):
File "/usr/lib/python2.4/threading.py", line 422, in run
self.__target(*self.__args, **self.__kwargs)
File
"/home/trac/Projects/Twisted/trunk/twisted/python/threadpool.py", line
148, in _worker
context.call(ctx, function, *args, **kwargs)
File
"/home/trac/Projects/Twisted/trunk/twisted/python/context.py", line 59, in
callWithContext
return self.currentContext().callWithContext(ctx, func, *args,
**kw)
File
"/home/trac/Projects/Twisted/trunk/twisted/python/context.py", line 37, in
callWithContext
return func(*args,**kw)
--- <exception caught here> ---
File "/home/trac/Projects/Twisted/trunk/twisted/web2/wsgi.py",
line 138, in run
result = self.application(self.environment,
self.startWSGIResponse)
File "/home/trac/Run/trac/external.py", line 74, in __call__
return self.application(environ, start_response)
File "/home/trac/Run/trac/external.py", line 115, in
tracApplication
return trac.web.main.dispatch_request(environ, start_response)
File "/home/trac/Projects/trac/trunk/trac/web/main.py", line
389, in dispatch_request
req.send_error(sys.exc_info(), status=500)
File "/home/trac/Projects/trac/trunk/trac/web/api.py", line 375,
in send_error
data = self.hdf.render(template)
File "/home/trac/Projects/trac/trunk/trac/web/clearsilver.py",
line 288, in render

neo_util.Error: Traceback (innermost last):
File "csparse.c", line 298, in cs_parse_file()
File "neo_files.c", line 86, in ne_load_file_len()
SystemError: Unable to open file
/home/trac/Projects/trac/trunk/templates/error.cs: [24] Too many open
files
}}}

Before restarting I checked /proc to find out what files were open. About
a thousand of the open file descriptors referred to trac.db.

--
Ticket URL: <http://trac.edgewall.org/ticket/3504#comment:14>
The Trac Project <http://trac.edgewall.org/>

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Trac Tickets" group.
To post to this group, send email to trac-tickets [at] googlegroups
To unsubscribe from this group, send email to trac-tickets-unsubscribe [at] googlegroups
For more options, visit this group at http://groups.google.se/group/trac-tickets
-~----------~----~----~----~------~----~------~--~---


noreply at edgewall

Sep 22, 2006, 6:36 AM

Post #16 of 41 (4039 views)
Permalink
Re: [The Trac Project] #3504: AssertionError in trac.db.pool.PooledConnection.__del__ [In reply to]
#3504: AssertionError in trac.db.pool.PooledConnection.__del__
--------------------------------------+-------------------------------------
Reporter: exarkun [at] divmod | Owner: cboos
Type: defect | Status: reopened
Priority: high | Milestone: 0.10
Component: general | Version: devel
Severity: blocker | Resolution:
Keywords: database connection pool |
--------------------------------------+-------------------------------------
Comment (by cboos):

I see you're running Trac with Twisted... could it be that the
'wsgi.multithread' wsgi environment is not correctly set? Because that
would explain the error. Check by adding a "print" in
source:trunk/trac/web/main.py [at] 376#L361.

''PS: on a somewhat related note, can you point me to some instructions
about how to use Trac within Twisted? Thanks! ''

--
Ticket URL: <http://trac.edgewall.org/ticket/3504#comment:15>
The Trac Project <http://trac.edgewall.org/>

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Trac Tickets" group.
To post to this group, send email to trac-tickets [at] googlegroups
To unsubscribe from this group, send email to trac-tickets-unsubscribe [at] googlegroups
For more options, visit this group at http://groups.google.se/group/trac-tickets
-~----------~----~----~----~------~----~------~--~---


noreply at edgewall

Sep 22, 2006, 8:18 AM

Post #17 of 41 (4040 views)
Permalink
Re: [The Trac Project] #3504: AssertionError in trac.db.pool.PooledConnection.__del__ [In reply to]
#3504: AssertionError in trac.db.pool.PooledConnection.__del__
--------------------------------------+-------------------------------------
Reporter: exarkun [at] divmod | Owner: cboos
Type: defect | Status: reopened
Priority: high | Milestone: 0.10
Component: general | Version: devel
Severity: blocker | Resolution:
Keywords: database connection pool |
--------------------------------------+-------------------------------------
Comment (by anonymous):

The variable is true. Is that right? :) I have no idea. I do know the
server doesn't fork.

The site is running based on the stuff here:

http://cvs.twistedmatrix.com/cvs/sandbox/exarkun/t.web2-wsgi-trac/

It's somewhat specialized for our particular configuration, but it might
still be useful as a starting point if you want to run trac in Twisted.
(I'm not sure this is a great thing to want just yet, Twisted's wsgi
support is very new and not yet entirely stable.)

--
Ticket URL: <http://trac.edgewall.org/ticket/3504#comment:16>
The Trac Project <http://trac.edgewall.org/>

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Trac Tickets" group.
To post to this group, send email to trac-tickets [at] googlegroups
To unsubscribe from this group, send email to trac-tickets-unsubscribe [at] googlegroups
For more options, visit this group at http://groups.google.se/group/trac-tickets
-~----------~----~----~----~------~----~------~--~---


noreply at edgewall

Sep 22, 2006, 8:24 AM

Post #18 of 41 (4056 views)
Permalink
Re: [The Trac Project] #3504: AssertionError in trac.db.pool.PooledConnection.__del__ [In reply to]
#3504: AssertionError in trac.db.pool.PooledConnection.__del__
--------------------------------------+-------------------------------------
Reporter: exarkun [at] divmod | Owner: cboos
Type: defect | Status: reopened
Priority: high | Milestone: 0.10
Component: general | Version: devel
Severity: blocker | Resolution:
Keywords: database connection pool |
--------------------------------------+-------------------------------------
Comment (by cboos):

Replying to [comment:16 anonymous]:
> The variable is true. Is that right? :) I have no idea. I do know the
server doesn't fork.

Yes, it should be true, so that's fine. Well, now please test on r3756
(and also with the upcoming patch).

> The site is running based on the stuff here: ...
Thanks!

--
Ticket URL: <http://trac.edgewall.org/ticket/3504#comment:17>
The Trac Project <http://trac.edgewall.org/>

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Trac Tickets" group.
To post to this group, send email to trac-tickets [at] googlegroups
To unsubscribe from this group, send email to trac-tickets-unsubscribe [at] googlegroups
For more options, visit this group at http://groups.google.se/group/trac-tickets
-~----------~----~----~----~------~----~------~--~---


noreply at edgewall

Sep 22, 2006, 8:26 AM

Post #19 of 41 (4047 views)
Permalink
Re: [The Trac Project] #3504: AssertionError in trac.db.pool.PooledConnection.__del__ [In reply to]
#3504: AssertionError in trac.db.pool.PooledConnection.__del__
--------------------------------------+-------------------------------------
Reporter: exarkun [at] divmod | Owner: cboos
Type: defect | Status: reopened
Priority: high | Milestone: 0.10
Component: general | Version: devel
Severity: blocker | Resolution:
Keywords: database connection pool |
--------------------------------------+-------------------------------------
Comment (by cboos):

I've also stress-tested r3763 with a brand new PosgreSQL 8.1.4
(pyPgSQL-2.5.1.win32-py2.4), and at first it seemed pretty fast and robust
(... and no `database is locked` anymore, joy!).

But upon closer inspection, I discovered that the pool was created over
and over again. Hence the following fix:
{{{
Index: trac/db/api.py
===================================================================
--- trac/db/api.py (revision 3763)
+++ trac/db/api.py (working copy)
@@ -73,7 +73,8 @@
def shutdown(self, tid=None):
if self._cnx_pool:
self._cnx_pool.shutdown(tid)
- self._cnx_pool = None
+ if not tid:
+ self._cnx_pool = None

def _get_connector(self): ### FIXME: Make it public?
scheme, args = _parse_db_str(self.connection_uri)
}}}
After that, things seemed even faster, if possible ;)

Now I'll test back with pysqlite, plus test both backends in single thread
mode.

--
Ticket URL: <http://trac.edgewall.org/ticket/3504#comment:18>
The Trac Project <http://trac.edgewall.org/>

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Trac Tickets" group.
To post to this group, send email to trac-tickets [at] googlegroups
To unsubscribe from this group, send email to trac-tickets-unsubscribe [at] googlegroups
For more options, visit this group at http://groups.google.se/group/trac-tickets
-~----------~----~----~----~------~----~------~--~---


noreply at edgewall

Sep 22, 2006, 8:53 AM

Post #20 of 41 (4043 views)
Permalink
Re: [The Trac Project] #3504: AssertionError in trac.db.pool.PooledConnection.__del__ [In reply to]
#3504: AssertionError in trac.db.pool.PooledConnection.__del__
--------------------------------------+-------------------------------------
Reporter: exarkun [at] divmod | Owner: cboos
Type: defect | Status: reopened
Priority: high | Milestone: 0.10
Component: general | Version: devel
Severity: blocker | Resolution:
Keywords: database connection pool |
--------------------------------------+-------------------------------------
Comment (by cboos):

Ok, with r3764 it seems really solid with sqlite/posgres, with tracd
multi/singlethread on Windows. Now I'll test on Linux...

--
Ticket URL: <http://trac.edgewall.org/ticket/3504#comment:19>
The Trac Project <http://trac.edgewall.org/>

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Trac Tickets" group.
To post to this group, send email to trac-tickets [at] googlegroups
To unsubscribe from this group, send email to trac-tickets-unsubscribe [at] googlegroups
For more options, visit this group at http://groups.google.se/group/trac-tickets
-~----------~----~----~----~------~----~------~--~---


noreply at edgewall

Sep 22, 2006, 9:15 AM

Post #21 of 41 (4039 views)
Permalink
Re: [The Trac Project] #3504: AssertionError in trac.db.pool.PooledConnection.__del__ [In reply to]
#3504: AssertionError in trac.db.pool.PooledConnection.__del__
--------------------------------------+-------------------------------------
Reporter: exarkun [at] divmod | Owner: cboos
Type: defect | Status: closed
Priority: high | Milestone: 0.10
Component: general | Version: devel
Severity: blocker | Resolution: fixed
Keywords: database connection pool |
--------------------------------------+-------------------------------------
Changes (by cboos):

* status: reopened => closed
* resolution: => fixed

Comment:

Tested on Linux with apache/prefork (not that relevant, I know) and tracd,
both with pysqlite. I verified that tracd doesn't leak file handles.

I'm pretty confident we can close this now.

So, if the leak of open files still occurs with Twisted even with r3764,
please open a ''new'' ticket, targeted to [milestone:0.10.1].

--
Ticket URL: <http://trac.edgewall.org/ticket/3504#comment:20>
The Trac Project <http://trac.edgewall.org/>

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Trac Tickets" group.
To post to this group, send email to trac-tickets [at] googlegroups
To unsubscribe from this group, send email to trac-tickets-unsubscribe [at] googlegroups
For more options, visit this group at http://groups.google.se/group/trac-tickets
-~----------~----~----~----~------~----~------~--~---


noreply at edgewall

Sep 25, 2006, 8:31 AM

Post #22 of 41 (3985 views)
Permalink
Re: [The Trac Project] #3504: AssertionError in trac.db.pool.PooledConnection.__del__ [In reply to]
#3504: AssertionError in trac.db.pool.PooledConnection.__del__
--------------------------------------+-------------------------------------
Reporter: exarkun [at] divmod | Owner: cboos
Type: defect | Status: closed
Priority: high | Milestone: 0.10
Component: general | Version: devel
Severity: blocker | Resolution: fixed
Keywords: database connection pool |
--------------------------------------+-------------------------------------
Comment (by anonymous):

The file leak does appear to be fixed. However, the AssertionError still
occurs, although apparently less freqeuntly. Since upgrading on Friday,
it has only happened twice:

2006/09/22 21:10 CDT [HTTPChannel,327,198.49.126.190] Exception
exceptions.AssertionError: <exceptions.AssertionError instance at
0xb7794dac> in <bound method PooledConnection.__del__ of
<trac.db.pool.PooledConnection object at 0xb4b75dec>> ignored
2006/09/25 05:12 CDT [HTTPChannel,25208,198.49.126.190] Exception
exceptions.AssertionError: <exceptions.AssertionError instance at
0xb7770ecc> in <bound method PooledConnection.__del__ of
<trac.db.pool.PooledConnection object at 0x87c248c>> ignored

--
Ticket URL: <http://trac.edgewall.org/ticket/3504#comment:21>
The Trac Project <http://trac.edgewall.org/>

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Trac Tickets" group.
To post to this group, send email to trac-tickets [at] googlegroups
To unsubscribe from this group, send email to trac-tickets-unsubscribe [at] googlegroups
For more options, visit this group at http://groups.google.se/group/trac-tickets
-~----------~----~----~----~------~----~------~--~---


noreply at edgewall

Sep 25, 2006, 9:25 AM

Post #23 of 41 (3983 views)
Permalink
Re: [The Trac Project] #3504: AssertionError in trac.db.pool.PooledConnection.__del__ [In reply to]
#3504: AssertionError in trac.db.pool.PooledConnection.__del__
--------------------------------------+-------------------------------------
Reporter: exarkun [at] divmod | Owner: cboos
Type: defect | Status: closed
Priority: high | Milestone: 0.10
Component: general | Version: devel
Severity: blocker | Resolution: fixed
Keywords: database connection pool |
--------------------------------------+-------------------------------------
Comment (by cboos):

Uh ho, right, there was a bug:
{{{
Index: pool.py
===================================================================
--- pool.py (revision 3788)
+++ pool.py (working copy)
@@ -122,8 +122,8 @@
cnx.rollback() # non-poolable but same thread: close
cnx.close()
self._cursize -= 1
- else: # non-poolable, different thread: do nothing
- self._active[tid][0] = 0
+ else: # non-poolable, different thread: push it back
+ self._active[tid] = (0, cnx)
self._available.notify()

def shutdown(self, tid=None):
}}}

In some rather convoluted circumstances, this could lead to the
!AssertionError...

,,In case anyone's interested in the details: two !PooledConnections
(''a'' and ''b'') in the same thread (say ''A'') are ready to be GCed; if
''a'' happens to be GCed in another thread, it will take the
`self._active[tid][0] = 0` buggy code path and the `self._active[A]` slot
is now empty; after that, if a new !PooledConnection is created in ''A'',
there will be a new cnx created and put into `self._active[A]`; then, when
''b'' gets GCed, the !AssertionError will be raised...,,

--
Ticket URL: <http://trac.edgewall.org/ticket/3504#comment:22>
The Trac Project <http://trac.edgewall.org/>

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Trac Tickets" group.
To post to this group, send email to trac-tickets [at] googlegroups
To unsubscribe from this group, send email to trac-tickets-unsubscribe [at] googlegroups
For more options, visit this group at http://groups.google.se/group/trac-tickets
-~----------~----~----~----~------~----~------~--~---


noreply at edgewall

Sep 25, 2006, 9:39 AM

Post #24 of 41 (3983 views)
Permalink
Re: [The Trac Project] #3504: AssertionError in trac.db.pool.PooledConnection.__del__ [In reply to]
#3504: AssertionError in trac.db.pool.PooledConnection.__del__
--------------------------------------+-------------------------------------
Reporter: exarkun [at] divmod | Owner: cboos
Type: defect | Status: closed
Priority: high | Milestone: 0.10
Component: general | Version: devel
Severity: blocker | Resolution: fixed
Keywords: database connection pool |
--------------------------------------+-------------------------------------
Comment (by cboos):

Follow-up to comment:22:
> In case anyone's interested in the details: ...

Oh well, the above scenario seems not possible, as only one
!PooledConnection can trigger the _cleanup. It must be something else,
similarly hairy...

--
Ticket URL: <http://trac.edgewall.org/ticket/3504#comment:23>
The Trac Project <http://trac.edgewall.org/>

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Trac Tickets" group.
To post to this group, send email to trac-tickets [at] googlegroups
To unsubscribe from this group, send email to trac-tickets-unsubscribe [at] googlegroups
For more options, visit this group at http://groups.google.se/group/trac-tickets
-~----------~----~----~----~------~----~------~--~---


noreply at edgewall

Sep 25, 2006, 12:31 PM

Post #25 of 41 (4001 views)
Permalink
Re: [The Trac Project] #3504: AssertionError in trac.db.pool.PooledConnection.__del__ [In reply to]
#3504: AssertionError in trac.db.pool.PooledConnection.__del__
--------------------------------------+-------------------------------------
Reporter: exarkun [at] divmod | Owner: cboos
Type: defect | Status: closed
Priority: high | Milestone: 0.10
Component: general | Version: devel
Severity: blocker | Resolution: fixed
Keywords: database connection pool |
--------------------------------------+-------------------------------------
Comment (by cboos):

Another follow-up to comment:22:
> Uh ho, right, there was a bug:
> {{{
> ...
> }}}

The correct and tested fix is r3791.

--
Ticket URL: <http://trac.edgewall.org/ticket/3504#comment:24>
The Trac Project <http://trac.edgewall.org/>

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Trac Tickets" group.
To post to this group, send email to trac-tickets [at] googlegroups
To unsubscribe from this group, send email to trac-tickets-unsubscribe [at] googlegroups
For more options, visit this group at http://groups.google.se/group/trac-tickets
-~----------~----~----~----~------~----~------~--~---

First page Previous page 1 2 Next page Last page  View All Trac tickets RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.