stefan_ml at behnel
Dec 9, 2011, 12:02 AM
Post #1 of 47
(606 views)
Permalink
|
Hi everyone,
I think Py3.3 would be a good milestone for cleaning up the stdlib support
for XML. Note upfront: you may or may not know me as the maintainer of
lxml, the de-facto non-stdlib standard Python XML tool. This (lengthy) post
was triggered by the following kind of conversation that I keep having with
new XML users in Python (mostly on c.l.py), which hints at some serious
flaw in the stdlib.
User: I'm trying to do XML stuff XYZ in Python and have problem ABC.
Me: What library are you using? Could you show us some code?
User: My code looks like this snippet: ...
Me: You are using minidom which is known to be hard to use, slow and uses
lots of memory. Use the xml.etree.ElementTree package instead, or rather
its C implementation cElementTree, also in the stdlib.
User (coming back after a while): thanks, that was exactly what [I didn't
know] I was looking for.
What does this tell us?
1) MiniDOM is what new users find first. It's highly visible because there
are still lots of ancient "Python and XML" web pages out there that date
back from the time before Python 2.5 (or rather something like 2.2), when
it was the only XML tree library in the stdlib. It's also the first hit
from the top when you search for "XML" on the stdlib docs page and contains
the (to some people) familiar word "DOM", which lets users stop their
search and start writing code, not expecting to find a separate alternative
in the same stdlib, way further down. And the description as "mini",
"simple" and "lightweight" suggests to users that it's going to be easy to
use and efficient.
2) MiniDOM is not what users want. It leads to complicated, unpythonic code
and lots of problems. It is neither easy to use, nor efficient, nor
"lightweight", "simple" or "mini", not in absolute numbers (see
http://bugs.python.org/issue11379#msg148584 and following for a recent
discussion). It's also badly maintained in the sense that its performance
characteristics could likely be improved, but no-one is seriously
interested in doing that, because it would not lead to something that
actually *is* fast or memory friendly compared to any of the 'real'
alternatives that are available right now.
3) ElementTree is what users should use, MiniDOM is not. ElementTree was
added to the stdlib in Py2.5 on popular demand, exactly because it is very
easy to use, very fast, and very memory friendly. And because users did not
want to use MiniDOM any more. Today, ElementTree has a rather straight
upgrade path towards lxml.etree if more XML features like validation or
XSLT are needed. MiniDOM has nothing like that to offer. It's a dead end.
4) In the stdlib, cElementTree is independent of ElementTree, but totally
hidden in the documentation. In conversations like the above, it's
unnecessarily complex to explain to users that there is ElementTree (which
is documented in the stdlib), but that what they want to use is really
cElementTree, which has the same API but does not have a stdlib
documentation page that I can send them to. Note that the other Python
implementations simply provide cElementTree as an alias for ElementTree.
That leaves CPython as the only Python implementation that really has these
two separate modules.
So, there are many problems here. And I think they make it unnecessarily
complicated for users to process XML in Python and that the current
situation helps in turning away new users from Python as a language for XML
processing. Python does have impressively great tools for working with XML.
It's just that the stdlib and its documentation do not reflect or even
appreciate that.
What should change?
a) The stdlib documentation should help users to choose the right tool
right from the start. Instead of using the totally misleading wording that
it uses now, it should be honest about the performance characteristics of
MiniDOM and should actively suggest that those who don't know what to
choose (or even *that* they can choose) should not use MiniDOM in the first
place. I created a ticket (issue11379) for a minor step in this direction,
but given the responses, I'm rather convinced that there's a lot more that
can be done and should be done, and that it should be done now, right for
the next release.
b) cElementTree should finally loose it's "special" status as a separate
library and disappear as an accelerator module behind ElementTree. This has
been suggested a couple of times already, and AFAIR, there was some
opposition because 1) ET was maintained outside of the stdlib and 2) the
APIs of both were not identical. However, getting ET 1.3 into Py2.7 and 3.2
was a U-turn. Today, ET is *only* being maintained in the stdlib by Florent
Xicluna (who is doing a good job with it), and ET 1.3 has basically made
the APIs of both implementations compatible again. So, 3.3 would be the
right milestone for fixing the "two libs for one" quirk.
Given that this is the third time during the last couple of years that I'm
suggesting to finally fix the stdlib and its documentation, I won't provide
any further patches before it has finally been accepted that a) this is a
problem and b) it should be fixed, thus allowing the patches to actually
serve a purpose. If we can agree on that, I'll happily help in making this
change happen.
Stefan
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com
|
1
