Mailing List Archive: Trac: Tickets

[The Trac Project] #3723: Problems with database encoding

Index | Next | Previous | View Threaded

noreply at edgewall

Sep 14, 2006, 2:35 AM

Post #1 of 13 (2624 views)
Permalink

[The Trac Project] #3723: Problems with database encoding

#3723: Problems with database encoding
--------------------+-------------------------------------------------------
Reporter: szymon | Owner: jonas
Type: defect | Status: new
Priority: high | Milestone:
Component: wiki | Version: 0.10b1
Severity: major | Keywords:
--------------------+-------------------------------------------------------
I am using Trac v.0.10-b1 with MySQL and I get the following error:

{{{
Traceback (most recent call last):
File "/usr/lib/python2.4/site-packages/trac/web/main.py", line 335, in
dispatch_request
dispatcher.dispatch(req)
File "/usr/lib/python2.4/site-packages/trac/web/main.py", line 220, in
dispatch
resp = chosen_handler.process_request(req)
File "/usr/lib/python2.4/site-packages/trac/wiki/web_ui.py", line 97, in
process_request
page = WikiPage(self.env, pagename, version, db)
File "/usr/lib/python2.4/site-packages/trac/wiki/model.py", line 32, in
__init__
self._fetch(name, version, db)
File "/usr/lib/python2.4/site-packages/trac/wiki/model.py", line 53, in
_fetch
(name,))
File "/usr/lib/python2.4/site-packages/trac/db/util.py", line 47, in
execute
return self.cursor.execute(sql_escape_percent(sql), args)
File "/usr/lib/python2.4/site-packages/trac/db/util.py", line 47, in
execute
return self.cursor.execute(sql_escape_percent(sql), args)
File "/usr/lib/python2.4/site-packages/MySQLdb/cursors.py", line 163, in
execute
self.errorhandler(self, exc, value)
File "/usr/lib/python2.4/site-packages/MySQLdb/connections.py", line 35,
in defaulterrorhandler
raise errorclass, errorvalue
UnicodeDecodeError: 'utf8' codec can't decode bytes in position 919-922:
invalid data
}}}

Any help?

--
Ticket URL: <http://trac.edgewall.org/ticket/3723>
The Trac Project <http://trac.edgewall.org/>

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Trac Tickets" group.
To post to this group, send email to trac-tickets [at] googlegroups
To unsubscribe from this group, send email to trac-tickets-unsubscribe [at] googlegroups
For more options, visit this group at http://groups.google.se/group/trac-tickets
-~----------~----~----~----~------~----~------~--~---

noreply at edgewall

Sep 20, 2006, 4:06 AM

Post #2 of 13 (2554 views)
Permalink

Re: [The Trac Project] #3723: Problems with database encoding [In reply to]

#3723: Problems with database encoding
----------------------------+-----------------------------------------------
Reporter: szymon | Owner: jonas
Type: defect | Status: new
Priority: highest | Milestone:
Component: wiki | Version: 0.10b1
Severity: blocker | Resolution:
Keywords: trac wiki utf8 |
----------------------------+-----------------------------------------------
Changes (by anonymous):

* keywords: => trac wiki utf8
* priority: high => highest
* severity: major => blocker

Comment:

I'm trying hard to fix this but to no success. Can anyone help as this bug
renders the whole wiki unusable?? Thanks in advance...

--
Ticket URL: <http://trac.edgewall.org/ticket/3723#comment:1>
The Trac Project <http://trac.edgewall.org/>

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Trac Tickets" group.
To post to this group, send email to trac-tickets [at] googlegroups
To unsubscribe from this group, send email to trac-tickets-unsubscribe [at] googlegroups
For more options, visit this group at http://groups.google.se/group/trac-tickets
-~----------~----~----~----~------~----~------~--~---

noreply at edgewall

Sep 24, 2006, 10:59 AM

Post #3 of 13 (2551 views)
Permalink

Re: [The Trac Project] #3723: Problems with database encoding [In reply to]

#3723: Problems with database encoding
----------------------------+-----------------------------------------------
Reporter: szymon | Owner: jonas
Type: defect | Status: new
Priority: highest | Milestone:
Component: wiki | Version: 0.10b1
Severity: blocker | Resolution:
Keywords: trac wiki utf8 |
----------------------------+-----------------------------------------------
Comment (by movex [at] bayern-mail):

Hi,
I'm using also Trac 0.10b1 with Python 2.3.5 and Mysql 4.1.11 and I get
the same error.
Mysql database wokrs in utf-8 constellation. When restarting apache the
error is gone for some hours after that same thing again.
This also affects timeline and tickets.

--
Ticket URL: <http://trac.edgewall.org/ticket/3723#comment:2>
The Trac Project <http://trac.edgewall.org/>

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Trac Tickets" group.
To post to this group, send email to trac-tickets [at] googlegroups
To unsubscribe from this group, send email to trac-tickets-unsubscribe [at] googlegroups
For more options, visit this group at http://groups.google.se/group/trac-tickets
-~----------~----~----~----~------~----~------~--~---

noreply at edgewall

Sep 25, 2006, 6:30 AM

Post #4 of 13 (2535 views)
Permalink

Re: [The Trac Project] #3723: Problems with database encoding [In reply to]

#3723: Problems with database encoding
----------------------------------+-----------------------------------------
Reporter: szymon | Owner: jonas
Type: defect | Status: new
Priority: highest | Milestone: 0.10.1
Component: wiki | Version: 0.10b1
Severity: blocker | Resolution:
Keywords: mysql trac wiki utf8 |
----------------------------------+-----------------------------------------
Changes (by cboos):

* keywords: trac wiki utf8 => mysql trac wiki utf8
* milestone: => 0.10.1

--
Ticket URL: <http://trac.edgewall.org/ticket/3723#comment:3>
The Trac Project <http://trac.edgewall.org/>

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Trac Tickets" group.
To post to this group, send email to trac-tickets [at] googlegroups
To unsubscribe from this group, send email to trac-tickets-unsubscribe [at] googlegroups
For more options, visit this group at http://groups.google.se/group/trac-tickets
-~----------~----~----~----~------~----~------~--~---

noreply at edgewall

Oct 3, 2006, 9:47 AM

Post #5 of 13 (2546 views)
Permalink

Re: [The Trac Project] #3723: Problems with database encoding [In reply to]

#3723: Problems with database encoding
-----------------------------------------+----------------------------------
Reporter: szymon | Owner: jonas
Type: defect | Status: new
Priority: highest | Milestone: 0.10.1
Component: wiki | Version: 0.10b1
Severity: blocker | Resolution:
Keywords: mysql trac wiki utf8 sqlite |
-----------------------------------------+----------------------------------
Changes (by anonymous):

* keywords: mysql trac wiki utf8 => mysql trac wiki utf8 sqlite

Comment:

Seeing the same sort of problem since upgrade to 0.10 with sqlite 3.2.1 &
python 2.4. It only seems to affect searches on tickets, not on the wiki.

{{{
Traceback (most recent call last):
File "/usr/lib/python2.4/site-packages/trac/web/main.py", line 356, in
dispatch_request
dispatcher.dispatch(req)
File "/usr/lib/python2.4/site-packages/trac/web/main.py", line 224, in
dispatch
resp = chosen_handler.process_request(req)
File "/usr/lib/python2.4/site-packages/trac/Search.py", line 181, in
process_request
results += list(source.get_search_results(req, terms, filters))
File "/usr/lib/python2.4/site-packages/trac/ticket/api.py", line 265, in
get_search_results
for summary, desc, author, keywords, tid, date, status in cursor:
File "/usr/lib/python2.4/site-packages/trac/db/util.py", line 40, in
__iter__
row = self.cursor.fetchone()
File "/usr/lib/python2.4/site-packages/trac/db/sqlite_backend.py", line
73, in fetchone
return row and self._convert_row(row) or None
File "/usr/lib/python2.4/site-packages/trac/db/sqlite_backend.py", line
69, in _convert_row
return tuple([(isinstance(v, str) and [v.decode('utf-8')] or [v])[0]
File "/usr/lib/python2.4/encodings/utf_8.py", line 16, in decode
return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode bytes in position 1784-1785:
invalid data
}}}

--
Ticket URL: <http://trac.edgewall.org/ticket/3723#comment:4>
The Trac Project <http://trac.edgewall.org/>

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Trac Tickets" group.
To post to this group, send email to trac-tickets [at] googlegroups
To unsubscribe from this group, send email to trac-tickets-unsubscribe [at] googlegroups
For more options, visit this group at http://groups.google.se/group/trac-tickets
-~----------~----~----~----~------~----~------~--~---

noreply at edgewall

Oct 3, 2006, 10:00 AM

Post #6 of 13 (2528 views)
Permalink

Re: [The Trac Project] #3723: Problems with database encoding [In reply to]

noreply at edgewall

Oct 4, 2006, 1:22 AM

Post #7 of 13 (2512 views)
Permalink

Re: [The Trac Project] #3723: Problems with database encoding [In reply to]

#3723: Problems with database encoding
-----------------------------------------+----------------------------------
Reporter: szymon | Owner: jonas
Type: defect | Status: new
Priority: highest | Milestone: 0.10.1
Component: wiki | Version: 0.10b1
Severity: blocker | Resolution:
Keywords: mysql trac wiki utf8 sqlite |
-----------------------------------------+----------------------------------
Comment (by anonymous):

Hi,

pysqlite 2.3.2 caused apache to segfault, so I tried version 2.0.7, which
worked ok, but I still get an error (although a slightly different one)
{{{
Traceback (most recent call last):
File "/usr/lib/python2.4/site-packages/trac/web/main.py", line 356, in
dispatch_request
dispatcher.dispatch(req)
File "/usr/lib/python2.4/site-packages/trac/web/main.py", line 224, in
dispatch
resp = chosen_handler.process_request(req)
File "/usr/lib/python2.4/site-packages/trac/Search.py", line 181, in
process_request
results += list(source.get_search_results(req, terms, filters))
File "/usr/lib/python2.4/site-packages/trac/ticket/api.py", line 267, in
get_search_results
if status == 'closed':
UnicodeDecodeError: 'utf8' codec can't decode bytes in position 1784-1785:
invalid data
}}}

Again this only happens on ticket searches.

--
Ticket URL: <http://trac.edgewall.org/ticket/3723#comment:6>
The Trac Project <http://trac.edgewall.org/>

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Trac Tickets" group.
To post to this group, send email to trac-tickets [at] googlegroups
To unsubscribe from this group, send email to trac-tickets-unsubscribe [at] googlegroups
For more options, visit this group at http://groups.google.se/group/trac-tickets
-~----------~----~----~----~------~----~------~--~---

noreply at edgewall

Oct 4, 2006, 2:12 AM

Post #8 of 13 (2532 views)
Permalink

Re: [The Trac Project] #3723: Problems with database encoding [In reply to]

noreply at edgewall

Oct 4, 2006, 5:44 AM

Post #9 of 13 (2516 views)
Permalink

Re: [The Trac Project] #3723: Problems with database encoding [In reply to]

noreply at edgewall

Oct 5, 2006, 3:41 AM

Post #10 of 13 (2531 views)
Permalink

Re: [The Trac Project] #3723: Problems with database encoding [In reply to]

#3723: Problems with database encoding
-----------------------------------------+----------------------------------
Reporter: szymon | Owner: jonas
Type: defect | Status: new
Priority: highest | Milestone: 0.10.1
Component: wiki | Version: 0.10b1
Severity: blocker | Resolution:
Keywords: mysql trac wiki utf8 sqlite |
-----------------------------------------+----------------------------------
Comment (by anonymous):

Hopefully of some help...

I tried writing the values out to a file. If I do f.write(tid) I get the
same Unicode error for the write. However if I do f.write(repr(tid)) the
write works fine and just prints a four digit ticket number. I'm learning
python as I go on this one so I'm not sure if that's what you'd expect or
not :o)

jon at the hyphen mill dot com

Replying to [comment:7 cboos]:
> Replying to [comment:6 anonymous]:
> > Hi,
> >
> > pysqlite 2.3.2 caused apache to segfault,
>
> Probably the same issue as pysqlite:ticket:174 (fixed in pysqlite's
trunk).
>
> >so I tried version 2.0.7, which worked ok, but I still get an error
(although a slightly different one)
> > ... [[br]]
> > ` if status == 'closed':` [[br]]
> > `UnicodeDecodeError: 'utf8' codec can't decode bytes in position
1784-1785: invalid data` [[br]]
> > ...
>
> I can't see why the `status` would have such a length ... can you insert
a `print tid, repr(status)` just before line 267?

--
Ticket URL: <http://trac.edgewall.org/ticket/3723#comment:9>
The Trac Project <http://trac.edgewall.org/>

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Trac Tickets" group.
To post to this group, send email to trac-tickets [at] googlegroups
To unsubscribe from this group, send email to trac-tickets-unsubscribe [at] googlegroups
For more options, visit this group at http://groups.google.se/group/trac-tickets
-~----------~----~----~----~------~----~------~--~---

noreply at edgewall

Oct 5, 2006, 11:13 PM

Post #11 of 13 (2528 views)
Permalink

Re: [The Trac Project] #3723: Problems with database encoding [In reply to]

#3723: Problems with database encoding
-------------------------------------------------+--------------------------
Reporter: szymon | Owner: jonas
Type: defect | Status: new
Priority: highest | Milestone: 0.10.1
Component: wiki | Version: 0.10b1
Severity: blocker | Resolution:
Keywords: mysql trac wiki utf8 sqlite unicode |
-------------------------------------------------+--------------------------
Changes (by cboos):

* keywords: mysql trac wiki utf8 sqlite => mysql trac wiki utf8 sqlite
unicode

Comment:

Replying to [comment:9 anonymous]:
> I tried writing the values out to a file. If I do `f.write(tid)` I get
the same Unicode error for the write. However if I do `f.write(repr(tid))`
the write works fine and just prints a four digit ticket number.

That's very surprising...
Please try: `f.write(tid.encode('unicode_internal'))`, and also:
{{{
from locale import getlocale
f.write(repr(getlocale()))
}}}

--
Ticket URL: <http://trac.edgewall.org/ticket/3723#comment:10>
The Trac Project <http://trac.edgewall.org/>

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Trac Tickets" group.
To post to this group, send email to trac-tickets [at] googlegroups
To unsubscribe from this group, send email to trac-tickets-unsubscribe [at] googlegroups
For more options, visit this group at http://groups.google.se/group/trac-tickets
-~----------~----~----~----~------~----~------~--~---

noreply at edgewall

Oct 6, 2006, 1:08 AM

Post #12 of 13 (2514 views)
Permalink

Re: [The Trac Project] #3723: Problems with database encoding [In reply to]

#3723: Problems with database encoding
-------------------------------------------------+--------------------------
Reporter: szymon | Owner: cmlenz
Type: defect | Status: new
Priority: highest | Milestone: 0.10.1
Component: mod_python frontend | Version: 0.10b1
Severity: blocker | Resolution:
Keywords: mysql trac wiki utf8 sqlite unicode |
-------------------------------------------------+--------------------------
Changes (by frederic.duarte):

* owner: jonas => cmlenz
* component: wiki => mod_python frontend

Comment:

I suggest to change the component as the error occurs on every page of any
trac project. Please correct if I'm wrong.

--
Ticket URL: <http://trac.edgewall.org/ticket/3723#comment:11>
The Trac Project <http://trac.edgewall.org/>

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Trac Tickets" group.
To post to this group, send email to trac-tickets [at] googlegroups
To unsubscribe from this group, send email to trac-tickets-unsubscribe [at] googlegroups
For more options, visit this group at http://groups.google.se/group/trac-tickets
-~----------~----~----~----~------~----~------~--~---

noreply at edgewall

Oct 6, 2006, 2:53 AM

Post #13 of 13 (2527 views)
Permalink

Re: [The Trac Project] #3723: Problems with database encoding [In reply to]

#3723: Problems with database encoding
-------------------------------------------------+--------------------------
Reporter: szymon | Owner: cmlenz
Type: defect | Status: new
Priority: highest | Milestone: 0.10.1
Component: mod_python frontend | Version: 0.10b1
Severity: blocker | Resolution:
Keywords: mysql trac wiki utf8 sqlite unicode |
-------------------------------------------------+--------------------------
Comment (by anonymous):

Ok, hopefully some progress! :o)

f.write(tid.encode('unicode_internal')) gives the error: 'int' object has
no attribute 'encode'.

However, if I just put the
{{{
from locale import getlocale
}}}
line in above line 267 and nothing else, the problem goes away and the
search displays correctly. If I put the import line at the top of api.py I
still get the error (not sure if that's expected or not)

If I write out the locale I get: ('en_GB', 'ISO8859-1')

Replying to [comment:10 cboos]:

> Please try: `f.write(tid.encode('unicode_internal'))`, and also:
> {{{
> from locale import getlocale
> f.write(repr(getlocale()))
> }}}

--
Ticket URL: <http://trac.edgewall.org/ticket/3723#comment:12>
The Trac Project <http://trac.edgewall.org/>

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Trac Tickets" group.
To post to this group, send email to trac-tickets [at] googlegroups
To unsubscribe from this group, send email to trac-tickets-unsubscribe [at] googlegroups
For more options, visit this group at http://groups.google.se/group/trac-tickets
-~----------~----~----~----~------~----~------~--~---

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads