Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Python: Dev

PEP 389: argparse - new command line parsing module

 

 

First page Previous page 1 2 3 4 5 Next page Last page  View All Python dev RSS feed   Index | Next | Previous | View Threaded


steven.bethard at gmail

Sep 27, 2009, 1:59 PM

Post #1 of 103 (6376 views)
Permalink
PEP 389: argparse - new command line parsing module

Below is a new PEP based on discussions from the stdlib-sig, which
proposes to add the argparse module to the standard library in Python
2.7 and 3.2.

Looking forward to your feedback!

Steve

http://www.python.org/dev/peps/pep-0389/
----------------------------------------------------------------------
PEP: 389
Title: argparse - new command line parsing module
Version: $Revision: 75097 $
Last-Modified: $Date: 2009-09-27 12:42:40 -0700 (Sun, 27 Sep 2009) $
Author: Steven Bethard <steven.bethard [at] gmail>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 25-Sep-2009
Python-Version: 2.7 and 3.2
Post-History:


Abstract
========
This PEP proposes inclusion of the argparse [1]_ module in the Python
standard library in Python 2.7 and 3.2.


Motivation
==========
The argparse module is a command line parsing library which provides
more functionality than the existing command line parsing modules in
the standard library, getopt [2]_ and optparse [3]_. It includes
support for positional arguments (not just options), subcommands,
required options, options syntaxes like "/f" and "+rgb", zero-or-more
and one-or-more style arguments, and many other features the other
two lack.

The argparse module is also already a popular third-party replacement
for these modules. It is used in projects like IPython (the Scipy
Python shell) [4]_, is included in Debian testing and unstable [5]_,
and since 2007 has had various requests for its inclusion in the
standard library [6]_ [7]_ [8]_. This popularity suggests it may be
a valuable addition to the Python libraries.


Why aren't getopt and optparse enough?
======================================
One argument against adding argparse is that thare are "already two
different option parsing modules in the standard library" [9]_. The
following is a list of features provided by argparse but not present
in getopt or optparse:

* While it is true there are two *option* parsing libraries, there
are no full command line parsing libraries -- both getopt and
optparse support only options and have no support for positional
arguments. The argparse module handles both, and as a result, is
able to generate better help messages, avoiding redundancies like
the ``usage=`` string usually required by optparse.

* The argparse module values practicality over purity. Thus, argparse
allows required options and customization of which characters are
used to identify options, while optparse explicitly states "the
phrase 'required option' is self-contradictory" and that the option
syntaxes ``-pf``, ``-file``, ``+f``, ``+rgb``, ``/f`` and ``/file``
"are not supported by optparse, and they never will be".

* The argparse module allows options to accept a variable number of
arguments using ``nargs='?'``, ``nargs='*'`` or ``nargs='+'``. The
optparse module provides an untested recipe for some part of this
functionality [10]_ but admits that "things get hairy when you want
an option to take a variable number of arguments."

* The argparse module supports subcommands, where a main command
line parser dispatches to other command line parsers depending on
the command line arguments. This is a common pattern in command
line interfaces, e.g. ``svn co`` and ``svn up``.


Why isn't the functionality just being added to optparse?
=========================================================
Clearly all the above features offer improvements over what is
available through optparse. A reasonable question then is why these
features are not simply provided as patches to optparse, instead of
introducing an entirely new module. In fact, the original development
of argparse intended to do just that, but because of various fairly
constraining design decisions of optparse, this wasn't really
possible. Some of the problems included:

* The optparse module exposes the internals of its parsing algorithm.
In particular, ``parser.largs`` and ``parser.rargs`` are guaranteed
to be available to callbacks [11]_. This makes it extremely
difficult to improve the parsing algorithm as was necessary in
argparse for proper handling of positional arguments and variable
length arguments. For example, ``nargs='+'`` in argparse is matched
using regular expressions and thus has no notion of things like
``parser.largs``.

* The optparse extension APIs are extremely complex. For example,
just to use a simple custom string-to-object conversion function,
you have to subclass ``Option``, hack class attributes, and then
specify your custom option type to the parser, like this::

class MyOption(Option):
TYPES = Option.TYPES + ("mytype",)
TYPE_CHECKER = copy(Option.TYPE_CHECKER)
TYPE_CHECKER["mytype"] = check_mytype
parser = optparse.OptionParser(option_class=MyOption)
parser.add_option("-m", type="mytype")

For comparison, argparse simply allows conversion functions to be
used as ``type=`` arguments directly, e.g.::

parser = argparse.ArgumentParser()
parser.add_option("-m", type=check_mytype)

But given the baroque customization APIs of optparse, it is unclear
how such a feature should interact with those APIs, and it is
quite possible that introducing the simple argparse API would break
existing custom Option code.

* Both optparse and argparse parse command line arguments and assign
them as attributes to an object returned by ``parse_args``.
However, the optparse module guarantees that the ``take_action``
method of custom actions will always be passed a ``values`` object
which provides an ``ensure_value`` method [12]_, while the argparse
module allows attributes to be assigned to any object, e.g.::

foo_object = ...
parser.parse_args(namespace=foo_object)
foo_object.some_attribute_parsed_from_command_line

Modifying optparse to allow any object to be passed in would be
difficult because simply passing the ``foo_object`` around instead
of a ``Values`` instance will break existing custom actions that
depend on the ``ensure_value`` method.

Because of issues like these, which made it unreasonably difficult
for argparse to stay compatible with the optparse APIs, argparse was
developed as an independent module. Given these issues, merging all
the argparse features into optparse with no backwards
incompatibilities seems unlikely.


Deprecation of getopt and optparse
==================================
There is still some debate over the best way (if at all) to encourage
users to move from getopt and optparse to their replacement,
argparse. The current recommendation of this PEP is the following
conservative deprecation strategy:

* Python 3.2, Python 2.7 and any later Python 2.X releases --
PendingDeprecation warnings, which by default are not displayed,
and documentation notes directing users of getopt and optparse to
argparse.

* Python 3.3 -- Same as above

* Python 3.4 -- Deprecation warnings for getopt and optparse, which
by default *are* displayed.

Though this is slower than the usual deprecation process, it seems
prudent to avoid producing any casually visible Deprecation warnings
until Python 3.X has had some additional time to attract developers.


Open Issues
===========
The argparse module supports Python from 2.3 up through 3.2 and as a
result relies on traditional ``%(foo)s`` style string formatting. It
has been suggested that it might be better to use the new style
``{foo}`` string formatting [13]_. This seems like a good idea, but
would break backwards compatibility for existing argparse-based
scripts unless we can come up with a way to reasonably support both
syntaxes.


References
==========
.. [1] argparse
(http://code.google.com/p/argparse/)

.. [2] getopt
(http://docs.python.org/library/getopt.html)

.. [3] optparse
(http://docs.python.org/library/optparse.html)

.. [4] argparse in IPython
(http://mail.scipy.org/pipermail/ipython-dev/2009-April/005102.html)

.. [5] argparse in Debian
(http://packages.debian.org/search?keywords=python-argparse)

.. [6] 2007-01-03 request for argparse in the standard library
(http://mail.python.org/pipermail/python-list/2007-January/592646.html)

.. [7] 2009-06-09 request for argparse in the standard library
(http://bugs.python.org/issue6247)

.. [8] 2009-09-10 request for argparse in the standard library
(http://mail.python.org/pipermail/stdlib-sig/2009-September/000342.html)

.. [9] Fredrik Lundh response to [6]_
(http://mail.python.org/pipermail/python-list/2007-January/592675.html)

.. [10] optparse variable args
(http://docs.python.org/library/optparse.html#callback-example-6-variable-arguments)

.. [11] parser.largs and parser.rargs
(http://docs.python.org/library/optparse.html#how-callbacks-are-called)

.. [12] take_action values argument
(http://docs.python.org/library/optparse.html#adding-new-actions)

.. [13] use {}-formatting instead of %-formatting
(http://bugs.python.org/msg89279)


Copyright
=========
This document has been placed in the public domain.



..
Local Variables:
mode: indented-text
indent-tabs-mode: nil
sentence-end-double-space: t
fill-column: 70
coding: utf-8
End:
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


brett at python

Sep 27, 2009, 2:57 PM

Post #2 of 103 (6204 views)
Permalink
Re: PEP 389: argparse - new command line parsing module [In reply to]

I am going to state upfront that I am +1 for this and I encouraged
Steven to submit this PEP on the stdlib-SIG. I still remember watching
Steven's lightning talk at PyCon 2009 on argparse and being impressed
by it (along with the rest of the audience which was obviously
impressed as well).

I think argprase is a good improvement over what we have now (getopt
and optparse), it's stable, considered best-of-breed by the community
for a while (as shown as how many times argparse has been suggestion
for inclusion), and Steven is already a core committer so is already
set to maintain the code. That covers the usual checklist we have for
even looking at a PEP to add a module to the standard library.


On Sun, Sep 27, 2009 at 13:59, Steven Bethard <steven.bethard [at] gmail> wrote:
[SNIP]
> Deprecation of getopt and optparse
> ==================================
> There is still some debate over the best way (if at all) to encourage
> users to move from getopt and optparse to their replacement,
> argparse. The current recommendation of this PEP is the following
> conservative deprecation strategy:
>
> * Python 3.2, Python 2.7 and any later Python 2.X releases --
>  PendingDeprecation warnings, which by default are not displayed,
>  and documentation notes directing users of getopt and optparse to
>  argparse.
>
> * Python 3.3 -- Same as above
>
> * Python 3.4 -- Deprecation warnings for getopt and optparse, which
>  by default *are* displayed.
>
> Though this is slower than the usual deprecation process, it seems
> prudent to avoid producing any casually visible Deprecation warnings
> until Python 3.X has had some additional time to attract developers.
>

Just to give people ideas of how long these deprecations would last,
if Python 3.2 is released in June 2010 and we continue on our 18 month
release schedule (and actually release that quickly which we typically
don't), then getopt/optparse would have a pending deprecation for 3
years (until June 2013) and a deprecation warning for 1.5 years (until
January 2015), so 4.5 years total once the deprecations began and six
years since Python 3.0 came out. And obviously the deprecation can be
extended if it turns out Python 3 pick up is now at a rate we expect
(but only if people who plan to port are having issues; this does not
concern those who plan to stay on Python 2 indefinitely and do not
care about Python 3).

And we can also document suggestions on how to transition off of
getopt/optparse like Steven has in the argparse code
(http://argparse.googlecode.com/svn/tags/r101/doc/argparse-vs-optparse.html#upgrading-optparse-code).

>
> Open Issues
> ===========
> The argparse module supports Python from 2.3 up through 3.2 and as a
> result relies on traditional ``%(foo)s`` style string formatting. It
> has been suggested that it might be better to use the new style
> ``{foo}`` string formatting [13]_. This seems like a good idea, but
> would break backwards compatibility for existing argparse-based
> scripts unless we can come up with a way to reasonably support both
> syntaxes.

I see two solutions to this. One is to auto-detect either format and
then do the right thing. Seems potentially messy, although getting the
regex right shouldn't be a problem.

The other option is to rename the module if it is added to the
standard library ala simplejson/json. That would alleviate any issues
with someone including their own copy of argparse with their
application using %()s string interpolation over {}. I don't have a
name to suggest at the moment (nor do I think we should start
discussing that now unless this is a chosen solution), but it would
deal with the problem.

Either way, with {} being the future and talk of someday dropping %, I
think we should make sure the newer syntax is at least supported, if
not the only way to do things.

-Brett
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


fuzzyman at voidspace

Sep 27, 2009, 3:00 PM

Post #3 of 103 (6203 views)
Permalink
Re: PEP 389: argparse - new command line parsing module [In reply to]

Brett Cannon wrote:
> I am going to state upfront that I am +1 for this and I encouraged
> Steven to submit this PEP on the stdlib-SIG. I still remember watching
> Steven's lightning talk at PyCon 2009 on argparse and being impressed
> by it (along with the rest of the audience which was obviously
> impressed as well).
>
> I think argprase is a good improvement over what we have now (getopt
> and optparse), it's stable, considered best-of-breed by the community
> for a while (as shown as how many times argparse has been suggestion
> for inclusion), and Steven is already a core committer so is already
> set to maintain the code. That covers the usual checklist we have for
> even looking at a PEP to add a module to the standard library.
>
>

I've used argparse and really like it. I could also have done with the
subcommand support in recent changes to unittest.

+1 for inclusion.

Michael


> On Sun, Sep 27, 2009 at 13:59, Steven Bethard <steven.bethard [at] gmail> wrote:
> [SNIP]
>
>> Deprecation of getopt and optparse
>> ==================================
>> There is still some debate over the best way (if at all) to encourage
>> users to move from getopt and optparse to their replacement,
>> argparse. The current recommendation of this PEP is the following
>> conservative deprecation strategy:
>>
>> * Python 3.2, Python 2.7 and any later Python 2.X releases --
>> PendingDeprecation warnings, which by default are not displayed,
>> and documentation notes directing users of getopt and optparse to
>> argparse.
>>
>> * Python 3.3 -- Same as above
>>
>> * Python 3.4 -- Deprecation warnings for getopt and optparse, which
>> by default *are* displayed.
>>
>> Though this is slower than the usual deprecation process, it seems
>> prudent to avoid producing any casually visible Deprecation warnings
>> until Python 3.X has had some additional time to attract developers.
>>
>>
>
> Just to give people ideas of how long these deprecations would last,
> if Python 3.2 is released in June 2010 and we continue on our 18 month
> release schedule (and actually release that quickly which we typically
> don't), then getopt/optparse would have a pending deprecation for 3
> years (until June 2013) and a deprecation warning for 1.5 years (until
> January 2015), so 4.5 years total once the deprecations began and six
> years since Python 3.0 came out. And obviously the deprecation can be
> extended if it turns out Python 3 pick up is now at a rate we expect
> (but only if people who plan to port are having issues; this does not
> concern those who plan to stay on Python 2 indefinitely and do not
> care about Python 3).
>
> And we can also document suggestions on how to transition off of
> getopt/optparse like Steven has in the argparse code
> (http://argparse.googlecode.com/svn/tags/r101/doc/argparse-vs-optparse.html#upgrading-optparse-code).
>
>
>> Open Issues
>> ===========
>> The argparse module supports Python from 2.3 up through 3.2 and as a
>> result relies on traditional ``%(foo)s`` style string formatting. It
>> has been suggested that it might be better to use the new style
>> ``{foo}`` string formatting [13]_. This seems like a good idea, but
>> would break backwards compatibility for existing argparse-based
>> scripts unless we can come up with a way to reasonably support both
>> syntaxes.
>>
>
> I see two solutions to this. One is to auto-detect either format and
> then do the right thing. Seems potentially messy, although getting the
> regex right shouldn't be a problem.
>
> The other option is to rename the module if it is added to the
> standard library ala simplejson/json. That would alleviate any issues
> with someone including their own copy of argparse with their
> application using %()s string interpolation over {}. I don't have a
> name to suggest at the moment (nor do I think we should start
> discussing that now unless this is a chosen solution), but it would
> deal with the problem.
>
> Either way, with {} being the future and talk of someday dropping %, I
> think we should make sure the newer syntax is at least supported, if
> not the only way to do things.
>
> -Brett
> _______________________________________________
> Python-Dev mailing list
> Python-Dev [at] python
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>


--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


fperez.net at gmail

Sep 27, 2009, 5:31 PM

Post #4 of 103 (6199 views)
Permalink
Re: PEP 389: argparse - new command line parsing module [In reply to]

On Sun, 27 Sep 2009 14:57:34 -0700, Brett Cannon wrote:

> I am going to state upfront that I am +1 for this and I encouraged
> Steven to submit this PEP on the stdlib-SIG. I still remember watching
> Steven's lightning talk at PyCon 2009 on argparse and being impressed by
> it (along with the rest of the audience which was obviously impressed as
> well).
>
> I think argprase is a good improvement over what we have now (getopt and
> optparse), it's stable, considered best-of-breed by the community for a
> while (as shown as how many times argparse has been suggestion for
> inclusion), and Steven is already a core committer so is already set to
> maintain the code. That covers the usual checklist we have for even
> looking at a PEP to add a module to the standard library.

FWIW, +1 from IPython: we ship a private copy of argparse as well (we use
it in several places but we want the basic ipython to be installable just
with the stdlib). Steven was gracious enough to relicense it as BSD at our
request:

http://mail.scipy.org/pipermail/ipython-dev/2009-September/005537.html

so we could ship this copy without any GPL incompatibilities, but we'd
much rather rely on stdlib components.

Best,

f

_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


jacobs at bioinformed

Sep 27, 2009, 5:43 PM

Post #5 of 103 (6203 views)
Permalink
Re: PEP 389: argparse - new command line parsing module [In reply to]

On Sun, Sep 27, 2009 at 8:31 PM, Fernando Perez <fperez.net [at] gmail>wrote:

> On Sun, 27 Sep 2009 14:57:34 -0700, Brett Cannon wrote:
>
> > I am going to state upfront that I am +1 for this and I encouraged
> > Steven to submit this PEP on the stdlib-SIG. I still remember watching
> > Steven's lightning talk at PyCon 2009 on argparse and being impressed by
> > it (along with the rest of the audience which was obviously impressed as
> > well).
> >
> > I think argprase is a good improvement over what we have now (getopt and
> > optparse), it's stable, considered best-of-breed by the community for a
> > while (as shown as how many times argparse has been suggestion for
> > inclusion), and Steven is already a core committer so is already set to
> > maintain the code. That covers the usual checklist we have for even
> > looking at a PEP to add a module to the standard library.
>
> FWIW, +1 from IPython: we ship a private copy of argparse as well (we use
> it in several places but we want the basic ipython to be installable just
> with the stdlib). Steven was gracious enough to relicense it as BSD at our
> request:
>

+1

Coincidentally I was converting several packages to argparse as I saw the
python-dev mail come in. A great deal of hairy code needed to jury-rig
sub-commands and other "advanced processing" with optparse is simply
disappearing. I like the idea that stdlib can evolve to encompass best of
breed solutions and that effort is being expended to improve existing
functionality even if it means increasing the diversity of APIs.

~Kevin


solipsis at pitrou

Sep 27, 2009, 5:51 PM

Post #6 of 103 (6200 views)
Permalink
Re: PEP 389: argparse - new command line parsing module [In reply to]

Hello,

I am neutral on the idea of adding argparse. However, I'm -1 on deprecating
optparse. It is very widely used (tons of scripts use it), and ok for many uses;
deprecating it is totally unhelpful and gratuitous.

Regards

Antoine.


_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


doug.hellmann at gmail

Sep 27, 2009, 6:22 PM

Post #7 of 103 (6211 views)
Permalink
Re: PEP 389: argparse - new command line parsing module [In reply to]

On Sep 27, 2009, at 6:00 PM, Michael Foord wrote:

> Brett Cannon wrote:
>> I am going to state upfront that I am +1 for this and I encouraged
>> Steven to submit this PEP on the stdlib-SIG. I still remember
>> watching
>> Steven's lightning talk at PyCon 2009 on argparse and being impressed
>> by it (along with the rest of the audience which was obviously
>> impressed as well).
>>
>> I think argprase is a good improvement over what we have now (getopt
>> and optparse), it's stable, considered best-of-breed by the community
>> for a while (as shown as how many times argparse has been suggestion
>> for inclusion), and Steven is already a core committer so is already
>> set to maintain the code. That covers the usual checklist we have for
>> even looking at a PEP to add a module to the standard library.
>>
>>
>
> I've used argparse and really like it. I could also have done with
> the subcommand support in recent changes to unittest.
>
> +1 for inclusion.

+1 as well. I wish I had been able to use this library for a recent
project using sub-commands.

Doug

_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


steven.bethard at gmail

Sep 27, 2009, 7:55 PM

Post #8 of 103 (6209 views)
Permalink
Re: PEP 389: argparse - new command line parsing module [In reply to]

On Sun, Sep 27, 2009 at 5:51 PM, Antoine Pitrou <solipsis [at] pitrou> wrote:
> I am neutral on the idea of adding argparse. However, I'm -1 on deprecating
> optparse. It is very widely used (tons of scripts use it), and ok for many uses;
> deprecating it is totally unhelpful and gratuitous.

Could you elaborate? If you are worried about Python 2.X scripts,
note from the PEP that the only things that will ever show up in
Python 2.X are:

PendingDeprecation warnings, which by default are not displayed,
and documentation notes directing users of getopt and optparse to
argparse.

I think these messages are useful for folks using 2.X who some day
would like to migrate to 3.X, and at the same time should have zero
effect on existing scripts.

If you think getopt and optparse should stick around in 3.X, why is
that? If you think there are things that getopt and optparse do better
than argparse, could you please give some examples?

Steve
--
Where did you get that preposterous hypothesis?
Did Steve tell you that?
--- The Hiphopopotamus
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


benjamin at python

Sep 27, 2009, 8:08 PM

Post #9 of 103 (6198 views)
Permalink
Re: PEP 389: argparse - new command line parsing module [In reply to]

2009/9/27 Steven Bethard <steven.bethard [at] gmail>:
> If you think getopt and optparse should stick around in 3.X, why is
> that? If you think there are things that getopt and optparse do better
> than argparse, could you please give some examples?

Transitioning to Python 3 is already a pain. bytes/str/unicode things
are going to be enough by themselves to drive people nuts. We don't
want to be forcing them to change APIs if optparse is already working
just fine for them. The job is the stdlib is not to force people to
use the "best" or "right" tools. Several years in the future I would
be more supportive of depreacting optparse, but more ways in which 2.x
and 3.x grow farther apart are not going to help jumping the great
version divide.


--
Regards,
Benjamin
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


steven.bethard at gmail

Sep 27, 2009, 8:18 PM

Post #10 of 103 (6197 views)
Permalink
Re: PEP 389: argparse - new command line parsing module [In reply to]

On Sun, Sep 27, 2009 at 8:08 PM, Benjamin Peterson <benjamin [at] python> wrote:
> 2009/9/27 Steven Bethard <steven.bethard [at] gmail>:
>> If you think getopt and optparse should stick around in 3.X, why is
>> that? If you think there are things that getopt and optparse do better
>> than argparse, could you please give some examples?
>
> Transitioning to Python 3 is already a pain. bytes/str/unicode things
> are going to be enough by themselves to drive people nuts. We don't
> want to be forcing them to change APIs if optparse is already working
> just fine for them. The job is the stdlib is not to force people to
> use the "best" or "right" tools. Several years in the future I would
> be more supportive of depreacting optparse, but more ways in which 2.x
> and 3.x grow farther apart are not going to help jumping the great
> version divide.

The first release where any real deprecation message would show up is
Python 3.4, more than 3 years away. If you think 3 years isn't long
enough for people to be over the Python 3 transition, let's stick in
another version in there and make it 4.5 years.

Steve
--
Where did you get that preposterous hypothesis?
Did Steve tell you that?
--- The Hiphopopotamus
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


benjamin at python

Sep 27, 2009, 8:41 PM

Post #11 of 103 (6200 views)
Permalink
Re: PEP 389: argparse - new command line parsing module [In reply to]

2009/9/27 Steven Bethard <steven.bethard [at] gmail>:
> On Sun, Sep 27, 2009 at 8:08 PM, Benjamin Peterson <benjamin [at] python> wrote:
>> 2009/9/27 Steven Bethard <steven.bethard [at] gmail>:
>>> If you think getopt and optparse should stick around in 3.X, why is
>>> that? If you think there are things that getopt and optparse do better
>>> than argparse, could you please give some examples?
>>
>> Transitioning to Python 3 is already a pain. bytes/str/unicode things
>> are going to be enough by themselves to drive people nuts. We don't
>> want to be forcing them to change APIs if optparse is already working
>> just fine for them. The job is the stdlib is not to force people to
>> use the "best" or "right" tools. Several years in the future I would
>> be more supportive of depreacting optparse, but more ways in which 2.x
>> and 3.x grow farther apart are not going to help jumping the great
>> version divide.
>
> The first release where any real deprecation message would show up is
> Python 3.4, more than 3 years away. If you think 3 years isn't long
> enough for people to be over the Python 3 transition, let's stick in
> another version in there and make it 4.5 years.

So, why even bother deprecating it if nobody is going to see the warnings?



--
Regards,
Benjamin
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


steven.bethard at gmail

Sep 27, 2009, 8:46 PM

Post #12 of 103 (6203 views)
Permalink
PEP 389: argparse - new command line parsing module [In reply to]

On Sun, Sep 27, 2009 at 8:41 PM, Benjamin Peterson <benjamin [at] python> wrote:
> 2009/9/27 Steven Bethard <steven.bethard [at] gmail>:
>> The first release where any real deprecation message would show up is
>> Python 3.4, more than 3 years away. If you think 3 years isn't long
>> enough for people to be over the Python 3 transition, let's stick in
>> another version in there and make it 4.5 years.
>
> So, why even bother deprecating it if nobody is going to see the warnings?

I feel like I'm repeating the PEP, but here it is again anyway. There
will be messages in the docs and pending deprecation warnings (which
don't show up by default but can be requested) starting in Python 2.7
and 3.2. Regular deprecation warnings wouldn't show up until Python
3.4, 3 years away. This compromise was intended exactly to address the
issue you brought up about people getting over the Python 3
transition.

Steve
--
Where did you get that preposterous hypothesis?
Did Steve tell you that?
       --- The Hiphopopotamus
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


steven.bethard at gmail

Sep 27, 2009, 8:56 PM

Post #13 of 103 (6209 views)
Permalink
Re: PEP 389: argparse - new command line parsing module [In reply to]

On Sun, Sep 27, 2009 at 8:49 PM, Benjamin Peterson <benjamin [at] python> wrote:
> 2009/9/27 Steven Bethard <steven.bethard [at] gmail>:
>> On Sun, Sep 27, 2009 at 8:41 PM, Benjamin Peterson <benjamin [at] python> wrote:
>>> 2009/9/27 Steven Bethard <steven.bethard [at] gmail>:
>>>> The first release where any real deprecation message would show up is
>>>> Python 3.4, more than 3 years away. If you think 3 years isn't long
>>>> enough for people to be over the Python 3 transition, let's stick in
>>>> another version in there and make it 4.5 years.
>>>
>>> So, why even bother deprecating it if nobody is going to see the warnings?
>>
>> I feel like I'm repeating the PEP, but here it is again anyway. There
>> will be messages in the docs and pending deprecation warnings (which
>> don't show up by default but can be requested) starting in Python 2.7
>> and 3.2. Regular deprecation warnings wouldn't show up until Python
>> 3.4, 3 years away. This compromise was intended exactly to address the
>> issue you brought up about people getting over the Python 3
>> transition.
>
> But that doesn't tell me why we should deprecate optparse, when it may
> work perfectly well for some people.

Because it's basically unmaintained, and anything you can do in
optparse you can do in argparse with almost identical syntax. So
encouraging people to move from getopt and optparse to argparse is a
net gain for us as Python maintainers -- that's two fewer modules in
the standard library that someone has to take care of bug reports for.

Steve
--
Where did you get that preposterous hypothesis?
Did Steve tell you that?
--- The Hiphopopotamus
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


ubershmekel at gmail

Sep 27, 2009, 8:59 PM

Post #14 of 103 (6205 views)
Permalink
Re: PEP 389: argparse - new command line parsing module [In reply to]

-1 for deprecating getopt. getopt is super-simple and especially useful for
c programmers learning python.

+1 for argparse.+1 for eventual deprecation of optparse - optparse and
argparse have a very similar syntax and having both is just
confusing. tsboapooowtdi


On Mon, Sep 28, 2009 at 6:46 AM, Steven Bethard <steven.bethard [at] gmail>wrote:

> On Sun, Sep 27, 2009 at 8:41 PM, Benjamin Peterson <benjamin [at] python>
> wrote:
> > 2009/9/27 Steven Bethard <steven.bethard [at] gmail>:
> >> The first release where any real deprecation message would show up is
> >> Python 3.4, more than 3 years away. If you think 3 years isn't long
> >> enough for people to be over the Python 3 transition, let's stick in
> >> another version in there and make it 4.5 years.
> >
> > So, why even bother deprecating it if nobody is going to see the
> warnings?
>
> I feel like I'm repeating the PEP, but here it is again anyway. There
> will be messages in the docs and pending deprecation warnings (which
> don't show up by default but can be requested) starting in Python 2.7
> and 3.2. Regular deprecation warnings wouldn't show up until Python
> 3.4, 3 years away. This compromise was intended exactly to address the
> issue you brought up about people getting over the Python 3
> transition.
>
> Steve
> --
> Where did you get that preposterous hypothesis?
> Did Steve tell you that?
> --- The Hiphopopotamus
> _______________________________________________
> Python-Dev mailing list
> Python-Dev [at] python
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/ubershmekel%40gmail.com
>


martin at v

Sep 27, 2009, 9:09 PM

Post #15 of 103 (6202 views)
Permalink
Re: PEP 389: argparse - new command line parsing module [In reply to]

> If you think getopt and optparse should stick around in 3.X, why is
> that? If you think there are things that getopt and optparse do better
> than argparse, could you please give some examples?

I personally consider getopt superior to both optparse and argparse.
Those are terribly verbose in specifying arguments, whereas getopt's
sequence-of-letters is really nice and compact.

Regards,
Martin
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


steven.bethard at gmail

Sep 27, 2009, 9:24 PM

Post #16 of 103 (6206 views)
Permalink
Re: PEP 389: argparse - new command line parsing module [In reply to]

On Sun, Sep 27, 2009 at 9:09 PM, "Martin v. Löwis" <martin [at] v> wrote:
>> If you think getopt and optparse should stick around in 3.X, why is
>> that? If you think there are things that getopt and optparse do better
>> than argparse, could you please give some examples?
>
> I personally consider getopt superior to both optparse and argparse.
> Those are terribly verbose in specifying arguments, whereas getopt's
> sequence-of-letters is really nice and compact.

Thanks for the concrete example. Although I'm unconvinced that the
characters you save in the sequence of letters in the getopt.getopt
call aren't afterwards wasted on type conversions, if/else statements
and variable assignments in the subsequent loop, it would be pretty
simple to add to argparse something like::

ArgumentParser.add_getopt_arguments(options[, long_options])

Then you could replace your getopt code::

try:
opts, args = getopt.getopt(sys.argv[1:], "ho:v", ["help", "output="])
except getopt.GetoptError, err:
# print help information and exit:
print str(err) # will print something like "option -a not recognized"
usage()
sys.exit(2)
output = None
verbose = False
for o, a in opts:
if o == "-v":
verbose = True
elif o in ("-h", "--help"):
usage()
sys.exit()
elif o in ("-o", "--output"):
output = a
else:
assert False, "unhandled option"

with argparse code like::

parser = argparse.ArgumentParser()
parser.add_getopt_arguments("ho:v", ["help", "output="])
args = parser.parse_args()
verbose = args.v
output = args.o or args.output

If something like this were available, would you be alright with
deprecating getopt?

Steve
--
Where did you get that preposterous hypothesis?
Did Steve tell you that?
--- The Hiphopopotamus
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


floris.bruynooghe at gmail

Sep 28, 2009, 1:38 AM

Post #17 of 103 (6208 views)
Permalink
Re: PEP 389: argparse - new command line parsing module [In reply to]

On Mon, Sep 28, 2009 at 06:59:45AM +0300, Yuvgoog Greenle wrote:
> -1 for deprecating getopt. getopt is super-simple and especially useful for
> c programmers learning python.
>
> +1 for argparse.+1 for eventual deprecation of optparse - optparse and
> argparse have a very similar syntax and having both is just
> confusing. tsboapooowtdi

+1 on all of this :-)

It would be a shame to see getopt go but optparse -> argparse seems
logical.


Floris
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


ndbecker2 at gmail

Sep 28, 2009, 4:09 AM

Post #18 of 103 (6197 views)
Permalink
Re: PEP 389: argparse - new command line parsing module [In reply to]

If the plan is to migrate from optparse to argparse, this could be made a
bit easier. If it weren't for the fact that some names are different in
argparse than optparse, I believe many optparse usages could port with no
change.

_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


fuzzyman at voidspace

Sep 28, 2009, 4:13 AM

Post #19 of 103 (6194 views)
Permalink
Re: PEP 389: argparse - new command line parsing module [In reply to]

Neal Becker wrote:
> If the plan is to migrate from optparse to argparse, this could be made a
> bit easier. If it weren't for the fact that some names are different in
> argparse than optparse, I believe many optparse usages could port with no
> change.
>

The new names in argparse fit with the fact that argparse has a
different 'rationale' than optparse. What about an optparse
compatibility layer to make porting easier?

Although it can't cover the whole optparse API (as explained in the PEP)
it could perhaps cover most 'normal' usage?

Michael

> _______________________________________________
> Python-Dev mailing list
> Python-Dev [at] python
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>


--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


ncoghlan at gmail

Sep 28, 2009, 5:34 AM

Post #20 of 103 (6194 views)
Permalink
Re: PEP 389: argparse - new command line parsing module [In reply to]

Doug Hellmann wrote:
>
> On Sep 27, 2009, at 6:00 PM, Michael Foord wrote:
>
>> Brett Cannon wrote:
>>> I am going to state upfront that I am +1 for this and I encouraged
>>> Steven to submit this PEP on the stdlib-SIG. I still remember watching
>>> Steven's lightning talk at PyCon 2009 on argparse and being impressed
>>> by it (along with the rest of the audience which was obviously
>>> impressed as well).
>>>
>>> I think argprase is a good improvement over what we have now (getopt
>>> and optparse), it's stable, considered best-of-breed by the community
>>> for a while (as shown as how many times argparse has been suggestion
>>> for inclusion), and Steven is already a core committer so is already
>>> set to maintain the code. That covers the usual checklist we have for
>>> even looking at a PEP to add a module to the standard library.
>>>
>>>
>>
>> I've used argparse and really like it. I could also have done with the
>> subcommand support in recent changes to unittest.
>>
>> +1 for inclusion.
>
> +1 as well. I wish I had been able to use this library for a recent
> project using sub-commands.

+1 here as well (although we should definitely find a way to use
str.format strings instead of %-format ones... come to think of it, does
even the logging module support str.format style formatting in Py3k?)

Reading through the argparse vs optparse comparison in the argparse docs
when the topic of possibly adding it came up a few weeks back I kept
going "yep, I've rolled my own version of that, oh, and that, yeah, that
too...".

argparse probably wouldn't have helped me even if I'd known about it
(due to license review overhead), but it certainly addressed a whole
heap of problems I had encountered in practice, and in ways that were a
lot cleaner than the comparatively hacky approaches I had used for my
own purposes (internal testing and diagnostic scripts are like that...).

Cheers,
Nick.

--
Nick Coghlan | ncoghlan [at] gmail | Brisbane, Australia
---------------------------------------------------------------
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


ncoghlan at gmail

Sep 28, 2009, 5:37 AM

Post #21 of 103 (6187 views)
Permalink
Re: PEP 389: argparse - new command line parsing module [In reply to]

Steven Bethard wrote:
> I feel like I'm repeating the PEP, but here it is again anyway. There
> will be messages in the docs and pending deprecation warnings (which
> don't show up by default but can be requested) starting in Python 2.7
> and 3.2. Regular deprecation warnings wouldn't show up until Python
> 3.4, 3 years away. This compromise was intended exactly to address the
> issue you brought up about people getting over the Python 3
> transition.

For 2.x, I'd go even further and only have the PendingDeprecation
warnings show up under the "-3" command line flag.

Cheers,
Nick.

--
Nick Coghlan | ncoghlan [at] gmail | Brisbane, Australia
---------------------------------------------------------------
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


jon+python-dev at unequivocal

Sep 28, 2009, 6:03 AM

Post #22 of 103 (6199 views)
Permalink
Re: PEP 389: argparse - new command line parsing module [In reply to]

On Mon, Sep 28, 2009 at 09:38:20AM +0100, Floris Bruynooghe wrote:
> On Mon, Sep 28, 2009 at 06:59:45AM +0300, Yuvgoog Greenle wrote:
> > -1 for deprecating getopt. getopt is super-simple and especially useful for
> > c programmers learning python.
> >
> > +1 for argparse.+1 for eventual deprecation of optparse - optparse and
> > argparse have a very similar syntax and having both is just
> > confusing. tsboapooowtdi
>
> +1 on all of this :-)
>
> It would be a shame to see getopt go but optparse -> argparse seems
> logical.

+1 from me too - keep getopt, deprecate optparse.
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


steven.bethard at gmail

Sep 28, 2009, 7:28 AM

Post #23 of 103 (6184 views)
Permalink
Re: PEP 389: argparse - new command line parsing module [In reply to]

On Mon, Sep 28, 2009 at 6:03 AM, Jon Ribbens
<jon+python-dev [at] unequivocal> wrote:
> On Mon, Sep 28, 2009 at 09:38:20AM +0100, Floris Bruynooghe wrote:
>> On Mon, Sep 28, 2009 at 06:59:45AM +0300, Yuvgoog Greenle wrote:
>> > -1 for deprecating getopt. getopt is super-simple and especially useful for
>> > c programmers learning python.
>> >
>> > +1 for argparse.+1 for eventual deprecation of optparse - optparse and
>> > argparse have a very similar syntax and having both is just
>> > confusing. tsboapooowtdi
>>
>> +1 on all of this :-)
>>
>> It would be a shame to see getopt go but optparse -> argparse seems
>> logical.
>
> +1 from me too - keep getopt, deprecate optparse.

Ok, sounds like there are a number of supporters for keeping getopt
around and just deprecating optparse. For those who'd like to keep
getopt around, I have a few questions:

* Would you be opposed to a note in the getopt documentation
suggesting argparse as an alternative?

* Would you like argparse to grow an add_getopt_arguments method (as
in my other post)?

* If argparse grew an add_getopt_arguments, would you still want to
keep getopt around? And if so, why?

Steve
--
Where did you get that preposterous hypothesis?
Did Steve tell you that?
--- The Hiphopopotamus
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


steve at pearwood

Sep 28, 2009, 7:36 AM

Post #24 of 103 (6182 views)
Permalink
Re: PEP 389: argparse - new command line parsing module [In reply to]

On Mon, 28 Sep 2009 06:59:32 am Steven Bethard wrote:

> Why aren't getopt and optparse enough?
[snip]

As a newbie, I found optparse intimidating because it provided so many
features. I expect argparse will be the same, only more so.
(Disclaimer: I haven't used argparse, I'm merely making a prediction
based on the available functionality.)

As it turned out, optparse wasn't anywhere near as difficult to use as I
expected it to be, but I suggest that newbies will have the same
response to argparse as I had to optparse -- "this looks complicated,
I'll just roll my own". At least I had the option to use getopt.

Not every module needs to be newbie-friendly, but something as
fundamental as argument parsing should at least provide a gentle
learning curve.

Currently, getopt more-or-less plays the role of "bare-bones" command
line parsing, which I think is important. Assuming argparse is added to
the std lib, I'm -0.5 on deprecating getopt, 0 on deprecating optparse.



--
Steven D'Aprano
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


steven.bethard at gmail

Sep 28, 2009, 7:43 AM

Post #25 of 103 (6179 views)
Permalink
Re: PEP 389: argparse - new command line parsing module [In reply to]

On Mon, Sep 28, 2009 at 4:09 AM, Neal Becker <ndbecker2 [at] gmail> wrote:
> If the plan is to migrate from optparse to argparse, this could be made a
> bit easier.  If it weren't for the fact that some names are different in
> argparse than optparse, I believe many optparse usages could port with no
> change.

I could conceivably add an OptionParser class which::

* Has an add_option method which delegates to add_argument

* Has a parse_args method which delegates to parse_known_args

* Registers the string names for types that optparse uses

That might allow some users to just switch their import as long as
they're not using any callbacks, catching any exceptions, extending
any APIs, etc. I'm not 100% convinced that it's good to add a subtly
different API that people would have to unlearn, but if there's enough
support for this, I'm open to the idea.

Steve
--
Where did you get that preposterous hypothesis?
Did Steve tell you that?
--- The Hiphopopotamus
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

First page Previous page 1 2 3 4 5 Next page Last page  View All Python dev RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.