Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Python: Dev

Py3K: indirect coupling between raise and exception handler

 

 

Python dev RSS feed   Index | Next | Previous | View Threaded


skip at mojam

Mar 10, 2000, 8:28 AM

Post #1 of 6 (215 views)
Permalink
Py3K: indirect coupling between raise and exception handler

Consider the following snippet of code from MySQLdb.py:

try:
self._query(query % escape_row(args, qc))
except TypeError:
self._query(query % escape_dict(args, qc))

It's not quite right. There are at least four reasons I can think of why
the % operator might raise a TypeError:

1. query has not enough format specifiers
2. query has too many format specifiers
3. argument type mismatch between individual format specifier and
corresponding argument
4. query expects dist-style interpolation

The except clause only handles the last case. That leaves the other three
cases mishandled. The above construct pretends that all TypeErrors possible
are handled by calling escape_dict() instead of escape_row().

I stumbled on case 2 yesterday and got a fairly useless error message when
the code in the except clause also bombed. Took me a few minutes of head
scratching to see that I had an extra %s in my format string. A note to
Andy Dustman, MySQLdb's author, yielded the following modified version:

try:
self._query(query % escape_row(args, qc))
except TypeError, m:
if m.args[0] == "not enough arguments for format string": raise
if m.args[0] == "not all arguments converted": raise
self._query(query % escape_dict(args, qc))

This will do the trick for me for the time being. Note, however, that the
only way for Andy to decide which of the cases occurred (case 3 still isn't
handled above, but should occur very rarely in MySQLdb since it only uses
the more accommodating %s as a format specifier) is to compare the string
value of the message to see which of the four cases was raised.

This strong coupling via the error message text between the exception being
raised (in C code, in this case) and the place where it's caught seems bad
to me and encourages authors to either not recover from errors or to recover
from them in the crudest fashion. If Guido decides to tweak the TypeError
message in any fashion, perhaps to include the count of arguments in the
format string and argument tuple, this code will break. It makes me wonder
if there's not a better mechanism waiting to be discovered. Would it be
possible to publish an interface of some sort via the exceptions module that
would allow symbolic names or dictionary references to be used to decide
which case is being handled? I envision something like the following in
exceptions.py:

UNKNOWN_ERROR_CATEGORY = 0
TYP_SHORT_FORMAT = 1
TYP_LONG_FORMAT = 2
...
IND_BAD_RANGE = 1

message_map = {
# leave
(TypeError, ("not enough arguments for format string",)):
TYP_SHORT_FORMAT,
(TypeError, ("not all arguments converted",)):
TYP_LONG_FORMAT,
...
(IndexError, ("list index out of range",)): IND_BAD_RANGE,
...
}

This would isolate the raw text of exception strings to just a single place
(well, just one place on the exception handling side of things). It would
be used something like

try:
self._query(query % escape_row(args, qc))
except TypeError, m:
from exceptions import *
exc_case = message_map.get((TypeError, m.args), UNKNOWN_ERROR_CATEGORY)
if exc_case in [UNKNOWN_ERROR_CATEGORY,TYP_SHORT_FORMAT,
TYP_LONG_FORMAT]: raise
self._query(query % escape_dict(args, qc))

This could be added to exceptions.py without breaking existing code.

Does this (or something like it) seem like a reasonable enhancement for
Py2K? If we can narrow things down to an implementable solution I'll create
a patch.

Skip Montanaro | http://www.mojam.com/
skip [at] mojam | http://www.musi-cal.com/


guido at python

Mar 10, 2000, 9:17 AM

Post #2 of 6 (212 views)
Permalink
Re: Py3K: indirect coupling between raise and exception handler [In reply to]

> Consider the following snippet of code from MySQLdb.py:

Skip, I'm not familiar with MySQLdb.py, and I have no idea what your
example is about. From the rest of the message I feel it's not about
MySQLdb at all, but about string formatting, butthe point escapes me
because you never quite show what's in the format string and what
error that gives. Could you give some examples based on first
principles? A simple interactive session showing the various errors
would be helpful...

--Guido van Rossum (home page: http://www.python.org/~guido/)


gward at cnri

Mar 10, 2000, 12:05 PM

Post #3 of 6 (214 views)
Permalink
Re: Py3K: indirect coupling between raise and exception handler [In reply to]

On 10 March 2000, Guido van Rossum said:
> Skip, I'm not familiar with MySQLdb.py, and I have no idea what your
> example is about. From the rest of the message I feel it's not about
> MySQLdb at all, but about string formatting, butthe point escapes me
> because you never quite show what's in the format string and what
> error that gives. Could you give some examples based on first
> principles? A simple interactive session showing the various errors
> would be helpful...

I think Skip's point was just this: "TypeError" isn't expressive
enough. If you catch TypeError on a statement with multiple possible
type errors, you don't know which one you caught. Same holds for any
exception type, really: a given statement could blow up with ValueError
for any number of reasons. Etc., etc.

One possible solution, and I think this is what Skip was getting at, is
to add an "error code" to the exception object that identifies the error
more reliably than examining the error message. It's just the
errno/strerror dichotomy: strerror is for users, errno is for code. I
think Skip is just saying that Pythone exception objets need an errno
(although it doesn't have to be a number). It would probably only make
sense to define error codes for exceptions that can be raised by Python
itself, though.

Greg


skip at mojam

Mar 10, 2000, 1:17 PM

Post #4 of 6 (215 views)
Permalink
Re: Py3K: indirect coupling between raise and exception handler [In reply to]

Guido> Skip, I'm not familiar with MySQLdb.py, and I have no idea what
Guido> your example is about. From the rest of the message I feel it's
Guido> not about MySQLdb at all, but about string formatting,

My apologies. You're correct, it's really not about MySQLdb. It's about
handling multiple cases raised by the same exception.

First, a more concrete example that just uses simple string formats:

code exception
"%s" % ("a", "b") TypeError: 'not all arguments converted'
"%s %s" % "a" TypeError: 'not enough arguments for format string'
"%(a)s" % ("a",) TypeError: 'format requires a mapping'
"%d" % {"a": 1} TypeError: 'illegal argument type for built-in operation'

Let's presume hypothetically that it's possible to recover from some subset
of the TypeErrors that are raised, but not all of them. Now, also presume
that the format strings and the tuple, string or dict literals I've given
above can be stored in variables (which they can).

If we wrap the code in a try/except statement, we can catch the TypeError
exception and try to do something sensible. This is precisely the trick
that Andy Dustman uses in MySQLdb: first try expanding the format string
using a tuple as the RH operand, then try with a dict if that fails.

Unfortunately, as you can see from the above examples, there are four cases
that need to be handled. To distinguish them currently, you have to compare
the message you get with the exception to string literals that are generally
defined in C code in the interpreter. Here's what Andy's original code
looked like stripped of the MySQLdb-ese:

try:
x = format % tuple_generating_function(...)
except TypeError:
x = format % dict_generating_function(...)

That doesn't handle the first two cases above. You have to inspect the
message that raise sends out:

try:
x = format % tuple_generating_function(...)
except TypeError, m:
if m.args[0] == "not all arguments converted": raise
if m.args[0] == "not enough arguments for format string": raise
x = format % dict_generating_function(...)

This comparison of except arguments with hard-coded strings (especially ones
the programmer has no direct control over) seems fragile to me. If you
decide to reword the error message strings, you break someone's code.

In my previous message I suggested collecting this fragility in the
exceptions module where it can be better isolated. My solution is a bit
cumbersome, but could probably be cleaned up somewhat, but basically looks
like

try:
x = format % tuple_generating_function(...)
except TypeError, m:
import exceptions
msg_case = exceptions.message_map.get((TypeError, m.args),
exceptions.UNKNOWN_ERROR_CATEGORY)
# punt on the cases we can't recover from
if msg_case == exceptions.TYP_SHORT_FORMAT: raise
if msg_case == exceptions.TYP_LONG_FORMAT: raise
if msg_case == exceptions.UNKNOWN_ERROR_CATEGORY: raise
# handle the one we can
x = format % dict_generating_function(...)

In private email that crossed my original message, Andy suggested defining
more standard exceptions, e.g.:

class FormatError(TypeError): pass
class TooManyElements(FormatError): pass
class TooFewElements(FormatError): pass

then raising the appropriate error based on the circumstance. Code that
catches TypeError exceptions would still work.

So there are two possible changes on the table:

1. define more standard exceptions so you can distinguish classes of
errors on a more fine-grained basis using just the first argument of
the except clause.

2. provide some machinery in exceptions.py to allow programmers a
measure of uncoupling from using hard-coded strings to distinguish
cases.

Skip


skip at mojam

Mar 10, 2000, 1:21 PM

Post #5 of 6 (215 views)
Permalink
Re: Py3K: indirect coupling between raise and exception handler [In reply to]

Greg> One possible solution, and I think this is what Skip was getting
Greg> at, is to add an "error code" to the exception object that
Greg> identifies the error more reliably than examining the error
Greg> message. It's just the errno/strerror dichotomy: strerror is for
Greg> users, errno is for code. I think Skip is just saying that
Greg> Pythone exception objets need an errno (although it doesn't have
Greg> to be a number). It would probably only make sense to define
Greg> error codes for exceptions that can be raised by Python itself,
Greg> though.

I'm actually allowing the string to be used as the error code. If you raise
TypeError with "not all arguments converted" as the argument, then that
string literal will appear in the definition of exceptions.message_map as
part of a key. The programmer would only refer to the args attribute of the
object being raised.

either-or-makes-no-real-difference-to-me-ly y'rs,

Skip


tim_one at email

Mar 11, 2000, 1:51 PM

Post #6 of 6 (216 views)
Permalink
RE: Py3K: indirect coupling between raise and exception handler [In reply to]

[.Skip Montanaro, with an expression that may raise TypeError for any of
several distinct reasons, and wants to figure out which one after the fact]

The existing exception machinery is sufficiently powerful for building a
solution, so nothing new is needed in the language. What you really need
here is an exhaustive list of all exceptions the language can raise, and
when, and why, and a formally supported "detail" field (whether numeric id
or string or whatever) that you can rely on to tell them apart at runtime.

There are at least a thousand cases that need to be so documented and
formalized. That's why not a one of them is now <0.9 wink>.

If P3K is a rewrite from scratch, a rational scheme could be built in from
the start. Else it would seem to require a volunteer with even less of a
life than us <wink>.

Python dev RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.