Login | Register For Free | Help	Search this list this category for: (Advanced)

Mailing List Archive: Python: Python Code that ought to run fast, but can't due to Python limitations.   Index | Next | Previous | View Flat

nagle at animats Jul 4, 2009, 10:33 AM Views: 709 Permalink Code that ought to run fast, but can't due to Python limitations. As an example of code that really needs to run fast, but is speed-limited by Python's limitations, see "tokenizer.py" in http://code.google.com/p/html5lib/ This is a parser for HTML 5, a piece of code that will be needed in many places and will process large amounts of data. It's written entirely in Python. Take a look at how much work has to be performed per character. This is a good test for Python implementation bottlenecks. Run that tokenizer on HTML, and see where the time goes. ("It should be written in C" is not an acceptable answer.) Python doesn't have a "switch" or "case" statement, and when you need a state machine with many states, that makes for painful, slow code. There's a comment in the code that it would be useful to run a few billion lines of HTML through an instrumented version of the parser to decide in which order the IF statements should be executed. You shouldn't have to do that. Yes, I've read PEP 3103. The big problem is the difficulty of figuring out what's a constant and what might change. If all the cases are constants, case statements are easy. But in Python, the compiler can't tell. Parsers have many named compile-time constants. Python doesn't support named compile-time constants, and this is one of the places where we have to pay the bill for that limitation. Something to think about when you need three more racks of servers because the HTML parser is slow. John Nagle -- http://mail.python.org/mailman/listinfo/python-list

Subject User Time Code that ought to run fast, but can't due to Python limitations. nagle at animats Jul 4, 2009, 10:33 AM     Re: Code that ought to run fast, but can't due to Python limitations. benjamin.kaplan at case Jul 4, 2009, 11:03 AM     Re: Code that ought to run fast, but can't due to Python limitations. mwilson at the-wire Jul 4, 2009, 11:20 AM     Re: Code that ought to run fast, but can't due to Python limitations. http://phr.cx at NOSPAM Jul 4, 2009, 11:40 AM         Re: Code that ought to run fast, but can't due to Python limitations. nagle at animats Jul 4, 2009, 4:35 PM             Re: Code that ought to run fast, but can't due to Python limitations. http://phr.cx at NOSPAM Jul 4, 2009, 4:48 PM                 Re: Code that ought to run fast, but can't due to Python limitations. nagle at animats Jul 4, 2009, 8:15 PM                     Re: Code that ought to run fast, but can't due to Python limitations. aahz at pythoncraft Jul 4, 2009, 8:37 PM                     Re: Code that ought to run fast, but can't due to Python limitations. steve at REMOVE-THIS-cybersource Jul 5, 2009, 1:28 AM                         Re: Code that ought to run fast, but can't due to Python limitations. http://phr.cx at NOSPAM Jul 5, 2009, 1:58 AM                             Re: Code that ought to run fast, but can't due to Python limitations. steve at REMOVE-THIS-cybersource Jul 5, 2009, 2:13 AM                                 Re: Code that ought to run fast, but can't due to Python limitations. http://phr.cx at NOSPAM Jul 5, 2009, 2:38 AM                                     Re: Code that ought to run fast, but can't due to Python limitations. stefan_ml at behnel Jul 5, 2009, 2:48 AM                                     Re: Code that ought to run fast, but can't due to Python limitations. mail at microcorp Jul 5, 2009, 7:03 AM                                         Re: Code that ought to run fast, but can't due to Python limitations. l.mastrodomenico at gmail Jul 5, 2009, 7:25 AM                                 Re: Code that ought to run fast, but can't due to Python limitations. nagle at animats Jul 6, 2009, 11:21 PM                     Re: Code that ought to run fast, but can't due to Python limitations. stefan_ml at behnel Jul 5, 2009, 1:58 AM                     Re: Code that ought to run fast, but can't due to Python limitations. stefan_ml at behnel Jul 5, 2009, 4:52 AM             Re: Code that ought to run fast, but can't due to Python limitations. nobody at nowhere Jul 4, 2009, 6:25 PM             Re: Code that ought to run fast, but can't due to Python limitations. ben+python at benfinney Jul 4, 2009, 7:09 PM             Re: Code that ought to run fast, but can't due to Python limitations. pavlovevidence at gmail Jul 4, 2009, 10:51 PM             Re: Code that ought to run fast, but can't due to Python limitations. stefan_ml at behnel Jul 5, 2009, 2:04 AM     Re: Code that ought to run fast, but can't due to Python limitations. nick at craig-wood Jul 5, 2009, 12:30 AM     Re: Code that ought to run fast, but can't due to Python limitations. mail at microcorp Jul 5, 2009, 1:12 AM         Re: Code that ought to run fast, but can't due to Python limitations. jeanmichel at sequans Jul 6, 2009, 5:54 AM             Re: Code that ought to run fast, but can't due to Python limitations. mail at microcorp Jul 6, 2009, 7:04 AM                 Re: Code that ought to run fast, but can't due to Python limitations. jeanmichel at sequans Jul 6, 2009, 8:24 AM     Re: Code that ought to run fast, but can't due to Python limitations. stefan_ml at behnel Jul 5, 2009, 1:41 AM     Re: Code that ought to run fast, but can't due to Python limitations. steve at REMOVE-THIS-cybersource Jul 5, 2009, 2:07 AM         Re: Code that ought to run fast, but can't due to Python limitations. mail at microcorp Jul 5, 2009, 6:45 AM         Re: Code that ought to run fast, but can't due to Python limitations. nagle at animats Jul 6, 2009, 11:31 PM             Re: Code that ought to run fast, but can't due to Python limitations. stefan_ml at behnel Jul 7, 2009, 2:09 AM     Re: Code that ought to run fast, but can't due to Python limitations. stefan_ml at behnel Jul 5, 2009, 2:41 AM     Re: Code that ought to run fast, but can't due to Python limitations. http://phr.cx at NOSPAM Jul 5, 2009, 2:52 AM         Re: Code that ought to run fast, but can't due to Python limitations. stefan_ml at behnel Jul 5, 2009, 3:06 AM     Re: Code that ought to run fast, but can't due to Python limitations. http://phr.cx at NOSPAM Jul 5, 2009, 2:54 AM         Re: Code that ought to run fast, but can't due to Python limitations. stefan_ml at behnel Jul 5, 2009, 3:04 AM     Re: Code that ought to run fast, but can't due to Python limitations. http://phr.cx at NOSPAM Jul 5, 2009, 4:09 AM         Re: Code that ought to run fast, but can't due to Python limitations. stefan_ml at behnel Jul 5, 2009, 4:23 AM     Re: Code that ought to run fast, but can't due to Python limitations. ptmcg at austin Jul 5, 2009, 4:41 AM     Re: Code that ought to run fast, but can't due to Python limitations. aahz at pythoncraft Jul 5, 2009, 8:08 AM         Re: Code that ought to run fast, but can't due to Python limitations. james at agentultra Jul 6, 2009, 8:06 AM     Re: Code that ought to run fast, but can't due to Python limitations. martin at v Jul 5, 2009, 12:23 PM         Re: Code that ought to run fast, but can't due to Python limitations. david.m.cooke at gmail Jul 6, 2009, 1:16 AM     Re: Code that ought to run fast, but can't due to Python limitations. ldo at geek-central Jul 6, 2009, 1:32 AM     Re: Code that ought to run fast, but can't due to Python limitations. nagle at animats Jul 7, 2009, 9:40 AM

Index | Next | Previous | View Flat   Python     Announce     Python     Dev     Bugs     Checkins

Interested in having your list archived? Contact lists@gossamer-threads.com
 

Web Applications & Managed Hosting Powered by Gossamer Threads Inc.