Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Python: Python

argparse limitations

 

 

Python python RSS feed   Index | Next | Previous | View Threaded


benoist at ibpc

Jul 27, 2012, 7:26 AM

Post #1 of 15 (868 views)
Permalink
argparse limitations

Hi,

I'm impletting a tool in Python.
I'd like this tool to behave like a standard unix tool, as grep for exemple.
I chose to use the argparse module to parse the command line and I think I'm getting into several limitations of this module.

> First Question.
How can I configure the the ArgumentParser to allow the user to give either an input file or to pipe the output from another program?

$ mytool.py file.txt
$ cat file.txt | mytool.py


> Second Question.
How can I get the nargs options working with subparser?
Cause basically if I've got a positionnal argument with nargs > 1, then the subparsers are recognized as values for the positionnal argument.

$ mytool.py file1.txt file2.txt foo

Here foo is a command I'd like to pass to mytool but argparse considers it's another input file (as are file1.txt and file2.txt).


Any help would be appreciated.
Ben.

--
http://mail.python.org/mailman/listinfo/python-list


oscar.benjamin at bristol

Jul 27, 2012, 7:43 AM

Post #2 of 15 (822 views)
Permalink
Re: argparse limitations [In reply to]

On 27 July 2012 15:26, Benoist Laurent <benoist [at] ibpc> wrote:

> Hi,
>
> I'm impletting a tool in Python.
> I'd like this tool to behave like a standard unix tool, as grep for
> exemple.
> I chose to use the argparse module to parse the command line and I think
> I'm getting into several limitations of this module.
>
> > First Question.
> How can I configure the the ArgumentParser to allow the user to give
> either an input file or to pipe the output from another program?
>
> $ mytool.py file.txt

$ cat file.txt | mytool.py
>

A better way to do that last line is:
$ mytool.py < file.txt

To answer the question, just make the first argument optional defaulting to
None. Then you can do:
if file1 is None:
file1 = sys.stdin


>
>
> > Second Question.
> How can I get the nargs options working with subparser?
> Cause basically if I've got a positionnal argument with nargs > 1, then
> the subparsers are recognized as values for the positionnal argument.
>
> $ mytool.py file1.txt file2.txt foo
>
> Here foo is a command I'd like to pass to mytool but argparse considers
> it's another input file (as are file1.txt and file2.txt).
>

I haven't used subparsers in argparse but I imagine that you would call it
like:
$ mytool.py foo file1.txt file2.txt


Cheers,
Oscar.


>
> Any help would be appreciated.
> Ben.
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>


benoist at ibpc

Jul 27, 2012, 8:19 AM

Post #3 of 15 (819 views)
Permalink
Re: argparse limitations [In reply to]

Le Jul 27, 2012 à 4:43 PM, Oscar Benjamin a écrit :

>
>
> On 27 July 2012 15:26, Benoist Laurent <benoist [at] ibpc> wrote:
> Hi,
>
> I'm impletting a tool in Python.
> I'd like this tool to behave like a standard unix tool, as grep for exemple.
> I chose to use the argparse module to parse the command line and I think I'm getting into several limitations of this module.
>
> > First Question.
> How can I configure the the ArgumentParser to allow the user to give either an input file or to pipe the output from another program?
>
> $ mytool.py file.txt
> $ cat file.txt | mytool.py
>
> A better way to do that last line is:
> $ mytool.py < file.txt
>
> To answer the question, just make the first argument optional defaulting to None. Then you can do:
> if file1 is None:
> file1 = sys.stdin

That's the solution I came to.
But I'm not very happy with this since I can definitively not make my program act as a standard unix tool.
Any other solution?


>
>
>
> > Second Question.
> How can I get the nargs options working with subparser?
> Cause basically if I've got a positionnal argument with nargs > 1, then the subparsers are recognized as values for the positionnal argument.
>
> $ mytool.py file1.txt file2.txt foo
>
> Here foo is a command I'd like to pass to mytool but argparse considers it's another input file (as are file1.txt and file2.txt).
>
> I haven't used subparsers in argparse but I imagine that you would call it like:
> $ mytool.py foo file1.txt file2.txt

As far as I know it is not the case.
Let's get into the code.

parser = argparse.ArgumentParser()
parser.add_argument("-i", help="input files", nargs="+")

subparsers = parser.add_subparsers()
foo_parser = subparser.add_parser("foo")
# ... here come some foo parser options
bar_parser = subparser.add_parser("bar")
# ... here come some bar parser options

What argparse expects is the "-i" arguments coming before the subparsers.

To summarize, if I adopt your solution to my first question, the I should add the "-i" argument to each subparser.
I don't want to since it's ugly and I'd have to write the help myself (which I don't want to).


Cheers


>
>
> Cheers,
> Oscar.
>
>
>
> Any help would be appreciated.
> Ben.
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>

--
Benoist Laurent
Laboratoire de Biochimie Theorique / CNRS UPR 9080
Institut de Biologie Physico-Chimique
13, rue Pierre et Marie Curie
F-75005 Paris
Tel. +33 [0]1 58 41 51 67 or +33 [0]6 21 64 50 56


__peter__ at web

Jul 27, 2012, 8:44 AM

Post #4 of 15 (812 views)
Permalink
Re: argparse limitations [In reply to]

Benoist Laurent wrote:

> I'm impletting a tool in Python.
> I'd like this tool to behave like a standard unix tool, as grep for
> exemple. I chose to use the argparse module to parse the command line and
> I think I'm getting into several limitations of this module.
>
>> First Question.
> How can I configure the the ArgumentParser to allow the user to give
> either an input file or to pipe the output from another program?
>
> $ mytool.py file.txt
> $ cat file.txt | mytool.py

$ echo alpha > in.txt
$ cat in.txt | ./mytool.py
ALPHA
$ cat in.txt | ./mytool.py - out.txt
$ cat out.txt
ALPHA
$ ./mytool.py in.txt
ALPHA
$ ./mytool.py in.txt out2.txt
$ cat out2.txt
ALPHA
$ cat ./mytool.py
#!/usr/bin/env python
assert __name__ == "__main__"

import argparse
import sys

parser = argparse.ArgumentParser()
parser.add_argument("infile", nargs="?", type=argparse.FileType("r"),
default=sys.stdin)
parser.add_argument("outfile", nargs="?", type=argparse.FileType("w"),
default=sys.stdout)
args = parser.parse_args()

args.outfile.writelines(line.upper() for line in args.infile)

Is that good enough?


--
http://mail.python.org/mailman/listinfo/python-list


ian.g.kelly at gmail

Jul 27, 2012, 9:26 AM

Post #5 of 15 (813 views)
Permalink
Re: argparse limitations [In reply to]

On Fri, Jul 27, 2012 at 9:19 AM, Benoist Laurent <benoist [at] ibpc> wrote:
> That's the solution I came to.
> But I'm not very happy with this since I can definitively not make my
> program act as a standard unix tool.
> Any other solution?

I don't understand; that's pretty much the same way that standard unix
tools implement stdin redirection. What specific behavior are you
looking for that you're missing with Oscar or Peter's suggestions?

> As far as I know it is not the case.
> Let's get into the code.
>
> parser = argparse.ArgumentParser()
> parser.add_argument("-i", help="input files", nargs="+")
>
> subparsers = parser.add_subparsers()
> foo_parser = subparser.add_parser("foo")
> # ... here come some foo parser options
> bar_parser = subparser.add_parser("bar")
> # ... here come some bar parser options
>
> What argparse expects is the "-i" arguments coming before the subparsers.
>
> To summarize, if I adopt your solution to my first question, the I should
> add the "-i" argument to each subparser.
> I don't want to since it's ugly and I'd have to write the help myself (which
> I don't want to).

argparse doesn't really handle well options or arguments with
nargs="+" or "*" that come before other non-option arguments or
subparser commands. If it sees the input "mytool.py -i one two foo
...", then is the "foo" part of the -i argument (a file to be read),
or is it the subparser command? At this point in the parsing it can't
really tell the difference, so it tacks it onto the -i list.
Potentially it could figure that out later when it reaches the end of
the command line, but that would require backtracking over an
arbitrarily large part of the command line. Worse, if the string
"foo" or "bar" appears more than once in the command line, then the
command may really be ambiguous. Usually for this reason arguments
with indefinite nargs will be the very last argument so they can
simply consume the rest of the command line.

Here's an alternate approach that does more or less what you want:

parser = argparse.ArgumentParser()
parser.add_argument("-i", help="input files", action="append")

subparsers = parser.add_subparsers()
foo_parser = subparser.add_parser("foo")
# ... here come some foo parser options
bar_parser = subparser.add_parser("bar")
# ... here come some bar parser options

>>> parser.parse_args(['-i', 'one', '-i', 'two', 'foo'])
Namespace(i=['one', 'two'])

It may be a bit awkward having to type "-i" once per file in the
command line, but it does clear up the ambiguity.
--
http://mail.python.org/mailman/listinfo/python-list


ian.g.kelly at gmail

Jul 27, 2012, 9:33 AM

Post #6 of 15 (812 views)
Permalink
Re: argparse limitations [In reply to]

On Fri, Jul 27, 2012 at 10:26 AM, Ian Kelly <ian.g.kelly [at] gmail> wrote:
> It may be a bit awkward having to type "-i" once per file in the
> command line, but it does clear up the ambiguity.

Or you could bite the bullet and make the input files argument part of
the subparser, which you may find ugly, but I believe it's more
typical of how other tools with subcommands normally work. If one of
your subcommands is a "help" command, do you really want that to
accept input files?
--
http://mail.python.org/mailman/listinfo/python-list


benoist at ibpc

Jul 27, 2012, 10:08 AM

Post #7 of 15 (818 views)
Permalink
Re: argparse limitations [In reply to]

Yes basically looks like you get it.
I have to further test it but my first impression is that it's correct.

So actually the point was to use nargs="?".

Thank you very much.
Ben



Le Jul 27, 2012 à 5:44 PM, Peter Otten a écrit :

> Benoist Laurent wrote:
>
>> I'm impletting a tool in Python.
>> I'd like this tool to behave like a standard unix tool, as grep for
>> exemple. I chose to use the argparse module to parse the command line and
>> I think I'm getting into several limitations of this module.
>>
>>> First Question.
>> How can I configure the the ArgumentParser to allow the user to give
>> either an input file or to pipe the output from another program?
>>
>> $ mytool.py file.txt
>> $ cat file.txt | mytool.py
>
> $ echo alpha > in.txt
> $ cat in.txt | ./mytool.py
> ALPHA
> $ cat in.txt | ./mytool.py - out.txt
> $ cat out.txt
> ALPHA
> $ ./mytool.py in.txt
> ALPHA
> $ ./mytool.py in.txt out2.txt
> $ cat out2.txt
> ALPHA
> $ cat ./mytool.py
> #!/usr/bin/env python
> assert __name__ == "__main__"
>
> import argparse
> import sys
>
> parser = argparse.ArgumentParser()
> parser.add_argument("infile", nargs="?", type=argparse.FileType("r"),
> default=sys.stdin)
> parser.add_argument("outfile", nargs="?", type=argparse.FileType("w"),
> default=sys.stdout)
> args = parser.parse_args()
>
> args.outfile.writelines(line.upper() for line in args.infile)
>
> Is that good enough?
>
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>

--
Benoist Laurent
Laboratoire de Biochimie Theorique / CNRS UPR 9080
Institut de Biologie Physico-Chimique
13, rue Pierre et Marie Curie
F-75005 Paris
Tel. +33 [0]1 58 41 51 67 or +33 [0]6 21 64 50 56


benoist at ibpc

Jul 31, 2012, 2:39 AM

Post #8 of 15 (801 views)
Permalink
Re: argparse limitations [In reply to]

Well sorry about that but it seems I was wrong.
It was Friday evening and I guess I've not been careful.

Actually when you specify nargs="?", the doc says "One argument will be consumed from the command line if possible, and produced as a single item".
So you can't pass several arguments to the program.

So, to rephrase the question, how can I get a argument parser that parses the command-line just as Unix grep would do?
i.e.

$ echo 42 > foo.txt
$ echo 172 >> foo.txt
$ cp foo.txt bar.txt
$
$ grep 42 foo.txt
42
$ grep 42 foo.txt bar.txt
foo.txt:42
bar.txt:42
$ cat foo.txt | grep 42
42
$ grep -c 42 foo.txt
1


Cheers,
Ben




Le Jul 27, 2012 à 7:08 PM, Benoist Laurent a écrit :

>
>
> Yes basically looks like you get it.
> I have to further test it but my first impression is that it's correct.
>
> So actually the point was to use nargs="?".
>
> Thank you very much.
> Ben
>
>
>
> Le Jul 27, 2012 à 5:44 PM, Peter Otten a écrit :
>
>> Benoist Laurent wrote:
>>
>>> I'm impletting a tool in Python.
>>> I'd like this tool to behave like a standard unix tool, as grep for
>>> exemple. I chose to use the argparse module to parse the command line and
>>> I think I'm getting into several limitations of this module.
>>>
>>>> First Question.
>>> How can I configure the the ArgumentParser to allow the user to give
>>> either an input file or to pipe the output from another program?
>>>
>>> $ mytool.py file.txt
>>> $ cat file.txt | mytool.py
>>
>> $ echo alpha > in.txt
>> $ cat in.txt | ./mytool.py
>> ALPHA
>> $ cat in.txt | ./mytool.py - out.txt
>> $ cat out.txt
>> ALPHA
>> $ ./mytool.py in.txt
>> ALPHA
>> $ ./mytool.py in.txt out2.txt
>> $ cat out2.txt
>> ALPHA
>> $ cat ./mytool.py
>> #!/usr/bin/env python
>> assert __name__ == "__main__"
>>
>> import argparse
>> import sys
>>
>> parser = argparse.ArgumentParser()
>> parser.add_argument("infile", nargs="?", type=argparse.FileType("r"),
>> default=sys.stdin)
>> parser.add_argument("outfile", nargs="?", type=argparse.FileType("w"),
>> default=sys.stdout)
>> args = parser.parse_args()
>>
>> args.outfile.writelines(line.upper() for line in args.infile)
>>
>> Is that good enough?
>>
>>
>> --
>> http://mail.python.org/mailman/listinfo/python-list
>>
>
> --
> Benoist Laurent
> Laboratoire de Biochimie Theorique / CNRS UPR 9080
> Institut de Biologie Physico-Chimique
> 13, rue Pierre et Marie Curie
> F-75005 Paris
> Tel. +33 [0]1 58 41 51 67 or +33 [0]6 21 64 50 56
>
> --
> http://mail.python.org/mailman/listinfo/python-list

--
Benoist Laurent
Laboratoire de Biochimie Theorique / CNRS UPR 9080
Institut de Biologie Physico-Chimique
13, rue Pierre et Marie Curie
F-75005 Paris
Tel. +33 [0]1 58 41 51 67 or +33 [0]6 21 64 50 56


oscar.j.benjamin at gmail

Jul 31, 2012, 2:55 AM

Post #9 of 15 (798 views)
Permalink
Re: argparse limitations [In reply to]

On Jul 31, 2012 10:32 AM, "Benoist Laurent" <benoist [at] ibpc> wrote:
>
> Well sorry about that but it seems I was wrong.
> It was Friday evening and I guess I've not been careful.
>
> Actually when you specify nargs="?", the doc says "One argument will be
consumed from the command line if possible, and produced as a single item".
> So you can't pass several arguments to the program.

Right below that in the docs it explains about using nargs='*' and
nargs='+'. One of those will do what you want.

Oscar.

>
> So, to rephrase the question, how can I get a argument parser that parses
the command-line just as Unix grep would do?
> i.e.
>
> $ echo 42 > foo.txt
> $ echo 172 >> foo.txt
> $ cp foo.txt bar.txt
> $
> $ grep 42 foo.txt
> 42
> $ grep 42 foo.txt bar.txt
> foo.txt:42
> bar.txt:42
> $ cat foo.txt | grep 42
> 42
> $ grep -c 42 foo.txt
> 1
>
>
> Cheers,
> Ben
>
>
>
>
> Le Jul 27, 2012 à 7:08 PM, Benoist Laurent a écrit :
>
>>
>>
>> Yes basically looks like you get it.
>> I have to further test it but my first impression is that it's correct.
>>
>> So actually the point was to use nargs="?".
>>
>> Thank you very much.
>> Ben
>>
>>
>>
>> Le Jul 27, 2012 à 5:44 PM, Peter Otten a écrit :
>>
>>> Benoist Laurent wrote:
>>>
>>>> I'm impletting a tool in Python.
>>>>
>>>> I'd like this tool to behave like a standard unix tool, as grep for
>>>>
>>>> exemple. I chose to use the argparse module to parse the command line
and
>>>>
>>>> I think I'm getting into several limitations of this module.
>>>>
>>>>
>>>>> First Question.
>>>>
>>>> How can I configure the the ArgumentParser to allow the user to give
>>>>
>>>> either an input file or to pipe the output from another program?
>>>>
>>>>
>>>> $ mytool.py file.txt
>>>>
>>>> $ cat file.txt | mytool.py
>>>
>>>
>>> $ echo alpha > in.txt
>>> $ cat in.txt | ./mytool.py
>>> ALPHA
>>> $ cat in.txt | ./mytool.py - out.txt
>>> $ cat out.txt
>>> ALPHA
>>> $ ./mytool.py in.txt
>>> ALPHA
>>> $ ./mytool.py in.txt out2.txt
>>> $ cat out2.txt
>>> ALPHA
>>> $ cat ./mytool.py
>>> #!/usr/bin/env python
>>> assert __name__ == "__main__"
>>>
>>> import argparse
>>> import sys
>>>
>>> parser = argparse.ArgumentParser()
>>> parser.add_argument("infile", nargs="?", type=argparse.FileType("r"),
>>> default=sys.stdin)
>>> parser.add_argument("outfile", nargs="?", type=argparse.FileType("w"),
>>> default=sys.stdout)
>>> args = parser.parse_args()
>>>
>>> args.outfile.writelines(line.upper() for line in args.infile)
>>>
>>> Is that good enough?
>>>
>>>
>>> --
>>> http://mail.python.org/mailman/listinfo/python-list
>>>
>>
>> --
>> Benoist Laurent
>> Laboratoire de Biochimie Theorique / CNRS UPR 9080
>> Institut de Biologie Physico-Chimique
>> 13, rue Pierre et Marie Curie
>> F-75005 Paris
>> Tel. +33 [0]1 58 41 51 67 or +33 [0]6 21 64 50 56
>>
>> --
>> http://mail.python.org/mailman/listinfo/python-list
>
>
> --
> Benoist Laurent
> Laboratoire de Biochimie Theorique / CNRS UPR 9080
> Institut de Biologie Physico-Chimique
> 13, rue Pierre et Marie Curie
> F-75005 Paris
> Tel. +33 [0]1 58 41 51 67 or +33 [0]6 21 64 50 56
>
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>


benoist at ibpc

Jul 31, 2012, 3:37 AM

Post #10 of 15 (801 views)
Permalink
Re: argparse limitations [In reply to]

Really sorry about that.

So, for the community, below is the full code for a tool that behaves like a Unix standard tool.
It takes in argument the files to process and a command.

"""Just to setup a command-line parser that acts just like a unix
standard tool."""

import argparse
import sys

def define_options():
parser = argparse.ArgumentParser()
parser.add_argument("fname", help="input file", nargs="*")

# create subparsers
subparsers = parser.add_subparsers(dest="cmd", metavar="command")

# create the parser for the "foo" command
get_parser = subparsers.add_parser("foo", help="foo help")
get_parser.add_argument("-n", help="number of foo to print",
type=int, default=10)

# create the parser for the "bar" command
sum_parser = subparsers.add_parser("bar", help="bar help")

return parser


if __name__ == '__main__':
args = define_options().parse_args()

if not args.fname:
content = sys.stdin.read()
# do something
else:
for fname in args.fname:
with(open(fname, "rt")) as f:
content = f.read()
# do somet


Benoist



Le Jul 31, 2012 à 11:55 AM, Oscar Benjamin a écrit :

>
> On Jul 31, 2012 10:32 AM, "Benoist Laurent" <benoist [at] ibpc> wrote:
> >
> > Well sorry about that but it seems I was wrong.
> > It was Friday evening and I guess I've not been careful.
> >
> > Actually when you specify nargs="?", the doc says "One argument will be consumed from the command line if possible, and produced as a single item".
> > So you can't pass several arguments to the program.
>
> Right below that in the docs it explains about using nargs='*' and nargs='+'. One of those will do what you want.
>
> Oscar.
>
> >
> > So, to rephrase the question, how can I get a argument parser that parses the command-line just as Unix grep would do?
> > i.e.
> >
> > $ echo 42 > foo.txt
> > $ echo 172 >> foo.txt
> > $ cp foo.txt bar.txt
> > $
> > $ grep 42 foo.txt
> > 42
> > $ grep 42 foo.txt bar.txt
> > foo.txt:42
> > bar.txt:42
> > $ cat foo.txt | grep 42
> > 42
> > $ grep -c 42 foo.txt
> > 1
> >
> >
> > Cheers,
> > Ben
> >
> >
> >
> >
> > Le Jul 27, 2012 à 7:08 PM, Benoist Laurent a écrit :
> >
> >>
> >>
> >> Yes basically looks like you get it.
> >> I have to further test it but my first impression is that it's correct.
> >>
> >> So actually the point was to use nargs="?".
> >>
> >> Thank you very much.
> >> Ben
> >>
> >>
> >>
> >> Le Jul 27, 2012 à 5:44 PM, Peter Otten a écrit :
> >>
> >>> Benoist Laurent wrote:
> >>>
> >>>> I'm impletting a tool in Python.
> >>>>
> >>>> I'd like this tool to behave like a standard unix tool, as grep for
> >>>>
> >>>> exemple. I chose to use the argparse module to parse the command line and
> >>>>
> >>>> I think I'm getting into several limitations of this module.
> >>>>
> >>>>
> >>>>> First Question.
> >>>>
> >>>> How can I configure the the ArgumentParser to allow the user to give
> >>>>
> >>>> either an input file or to pipe the output from another program?
> >>>>
> >>>>
> >>>> $ mytool.py file.txt
> >>>>
> >>>> $ cat file.txt | mytool.py
> >>>
> >>>
> >>> $ echo alpha > in.txt
> >>> $ cat in.txt | ./mytool.py
> >>> ALPHA
> >>> $ cat in.txt | ./mytool.py - out.txt
> >>> $ cat out.txt
> >>> ALPHA
> >>> $ ./mytool.py in.txt
> >>> ALPHA
> >>> $ ./mytool.py in.txt out2.txt
> >>> $ cat out2.txt
> >>> ALPHA
> >>> $ cat ./mytool.py
> >>> #!/usr/bin/env python
> >>> assert __name__ == "__main__"
> >>>
> >>> import argparse
> >>> import sys
> >>>
> >>> parser = argparse.ArgumentParser()
> >>> parser.add_argument("infile", nargs="?", type=argparse.FileType("r"),
> >>> default=sys.stdin)
> >>> parser.add_argument("outfile", nargs="?", type=argparse.FileType("w"),
> >>> default=sys.stdout)
> >>> args = parser.parse_args()
> >>>
> >>> args.outfile.writelines(line.upper() for line in args.infile)
> >>>
> >>> Is that good enough?
> >>>
> >>>
> >>> --
> >>> http://mail.python.org/mailman/listinfo/python-list
> >>>
> >>
> >> --
> >> Benoist Laurent
> >> Laboratoire de Biochimie Theorique / CNRS UPR 9080
> >> Institut de Biologie Physico-Chimique
> >> 13, rue Pierre et Marie Curie
> >> F-75005 Paris
> >> Tel. +33 [0]1 58 41 51 67 or +33 [0]6 21 64 50 56
> >>
> >> --
> >> http://mail.python.org/mailman/listinfo/python-list
> >
> >
> > --
> > Benoist Laurent
> > Laboratoire de Biochimie Theorique / CNRS UPR 9080
> > Institut de Biologie Physico-Chimique
> > 13, rue Pierre et Marie Curie
> > F-75005 Paris
> > Tel. +33 [0]1 58 41 51 67 or +33 [0]6 21 64 50 56
> >
> >
> > --
> > http://mail.python.org/mailman/listinfo/python-list
> >

--
Benoist Laurent
Laboratoire de Biochimie Theorique / CNRS UPR 9080
Institut de Biologie Physico-Chimique
13, rue Pierre et Marie Curie
F-75005 Paris
Tel. +33 [0]1 58 41 51 67 or +33 [0]6 21 64 50 56


benoist at ibpc

Jul 31, 2012, 4:03 AM

Post #11 of 15 (805 views)
Permalink
Re: argparse limitations [In reply to]

Finally.

The code I proposed doesn't work in this case: if you add any positional argument to one of the subparsers, then the parsing doesn't work anymore.
The reason seems to be that argparse thinks the last argument of the first parser is the last but one argument.
Hence, if a subparser takes some arguments, it fails.

Example: if the "-n" argument of the foo parser is set mandatory (so becomes "n" instead of "-n")

python toto.py foo.txt bar.txt foo 10
usage: toto.py [-h] [fname [fname ...]] command ...
toto.py: error: argument command: invalid choice: '10' (choose from 'foo', 'bar')


Any solution?

Cheers,
Ben



Le Jul 31, 2012 à 12:37 PM, Benoist Laurent a écrit :

> Really sorry about that.
>
> So, for the community, below is the full code for a tool that behaves like a Unix standard tool.
> It takes in argument the files to process and a command.
>
> """Just to setup a command-line parser that acts just like a unix
> standard tool."""
>
> import argparse
> import sys
>
> def define_options():
> parser = argparse.ArgumentParser()
> parser.add_argument("fname", help="input file", nargs="*")
>
> # create subparsers
> subparsers = parser.add_subparsers(dest="cmd", metavar="command")
>
> # create the parser for the "foo" command
> get_parser = subparsers.add_parser("foo", help="foo help")
> get_parser.add_argument("-n", help="number of foo to print",
> type=int, default=10)
>
> # create the parser for the "bar" command
> sum_parser = subparsers.add_parser("bar", help="bar help")
>
> return parser
>
>
> if __name__ == '__main__':
> args = define_options().parse_args()
>
> if not args.fname:
> content = sys.stdin.read()
> # do something
> else:
> for fname in args.fname:
> with(open(fname, "rt")) as f:
> content = f.read()
> # do somet
>
>
> Benoist
>
>
>
> Le Jul 31, 2012 à 11:55 AM, Oscar Benjamin a écrit :
>
>>
>> On Jul 31, 2012 10:32 AM, "Benoist Laurent" <benoist [at] ibpc> wrote:
>> >
>> > Well sorry about that but it seems I was wrong.
>> > It was Friday evening and I guess I've not been careful.
>> >
>> > Actually when you specify nargs="?", the doc says "One argument will be consumed from the command line if possible, and produced as a single item".
>> > So you can't pass several arguments to the program.
>>
>> Right below that in the docs it explains about using nargs='*' and nargs='+'. One of those will do what you want.
>>
>> Oscar.
>>
>> >
>> > So, to rephrase the question, how can I get a argument parser that parses the command-line just as Unix grep would do?
>> > i.e.
>> >
>> > $ echo 42 > foo.txt
>> > $ echo 172 >> foo.txt
>> > $ cp foo.txt bar.txt
>> > $
>> > $ grep 42 foo.txt
>> > 42
>> > $ grep 42 foo.txt bar.txt
>> > foo.txt:42
>> > bar.txt:42
>> > $ cat foo.txt | grep 42
>> > 42
>> > $ grep -c 42 foo.txt
>> > 1
>> >
>> >
>> > Cheers,
>> > Ben
>> >
>> >
>> >
>> >
>> > Le Jul 27, 2012 à 7:08 PM, Benoist Laurent a écrit :
>> >
>> >>
>> >>
>> >> Yes basically looks like you get it.
>> >> I have to further test it but my first impression is that it's correct.
>> >>
>> >> So actually the point was to use nargs="?".
>> >>
>> >> Thank you very much.
>> >> Ben
>> >>
>> >>
>> >>
>> >> Le Jul 27, 2012 à 5:44 PM, Peter Otten a écrit :
>> >>
>> >>> Benoist Laurent wrote:
>> >>>
>> >>>> I'm impletting a tool in Python.
>> >>>>
>> >>>> I'd like this tool to behave like a standard unix tool, as grep for
>> >>>>
>> >>>> exemple. I chose to use the argparse module to parse the command line and
>> >>>>
>> >>>> I think I'm getting into several limitations of this module.
>> >>>>
>> >>>>
>> >>>>> First Question.
>> >>>>
>> >>>> How can I configure the the ArgumentParser to allow the user to give
>> >>>>
>> >>>> either an input file or to pipe the output from another program?
>> >>>>
>> >>>>
>> >>>> $ mytool.py file.txt
>> >>>>
>> >>>> $ cat file.txt | mytool.py
>> >>>
>> >>>
>> >>> $ echo alpha > in.txt
>> >>> $ cat in.txt | ./mytool.py
>> >>> ALPHA
>> >>> $ cat in.txt | ./mytool.py - out.txt
>> >>> $ cat out.txt
>> >>> ALPHA
>> >>> $ ./mytool.py in.txt
>> >>> ALPHA
>> >>> $ ./mytool.py in.txt out2.txt
>> >>> $ cat out2.txt
>> >>> ALPHA
>> >>> $ cat ./mytool.py
>> >>> #!/usr/bin/env python
>> >>> assert __name__ == "__main__"
>> >>>
>> >>> import argparse
>> >>> import sys
>> >>>
>> >>> parser = argparse.ArgumentParser()
>> >>> parser.add_argument("infile", nargs="?", type=argparse.FileType("r"),
>> >>> default=sys.stdin)
>> >>> parser.add_argument("outfile", nargs="?", type=argparse.FileType("w"),
>> >>> default=sys.stdout)
>> >>> args = parser.parse_args()
>> >>>
>> >>> args.outfile.writelines(line.upper() for line in args.infile)
>> >>>
>> >>> Is that good enough?
>> >>>
>> >>>
>> >>> --
>> >>> http://mail.python.org/mailman/listinfo/python-list
>> >>>
>> >>
>> >> --
>> >> Benoist Laurent
>> >> Laboratoire de Biochimie Theorique / CNRS UPR 9080
>> >> Institut de Biologie Physico-Chimique
>> >> 13, rue Pierre et Marie Curie
>> >> F-75005 Paris
>> >> Tel. +33 [0]1 58 41 51 67 or +33 [0]6 21 64 50 56
>> >>
>> >> --
>> >> http://mail.python.org/mailman/listinfo/python-list
>> >
>> >
>> > --
>> > Benoist Laurent
>> > Laboratoire de Biochimie Theorique / CNRS UPR 9080
>> > Institut de Biologie Physico-Chimique
>> > 13, rue Pierre et Marie Curie
>> > F-75005 Paris
>> > Tel. +33 [0]1 58 41 51 67 or +33 [0]6 21 64 50 56
>> >
>> >
>> > --
>> > http://mail.python.org/mailman/listinfo/python-list
>> >
>
> --
> Benoist Laurent
> Laboratoire de Biochimie Theorique / CNRS UPR 9080
> Institut de Biologie Physico-Chimique
> 13, rue Pierre et Marie Curie
> F-75005 Paris
> Tel. +33 [0]1 58 41 51 67 or +33 [0]6 21 64 50 56
>
> --
> http://mail.python.org/mailman/listinfo/python-list

--
Benoist Laurent
Laboratoire de Biochimie Theorique / CNRS UPR 9080
Institut de Biologie Physico-Chimique
13, rue Pierre et Marie Curie
F-75005 Paris
Tel. +33 [0]1 58 41 51 67 or +33 [0]6 21 64 50 56


oscar.j.benjamin at gmail

Jul 31, 2012, 4:45 AM

Post #12 of 15 (808 views)
Permalink
Re: argparse limitations [In reply to]

On 31 July 2012 12:03, Benoist Laurent <benoist [at] ibpc> wrote:

> Finally.
>
> The code I proposed doesn't work in this case: if you add any positional
> argument to one of the subparsers, then the parsing doesn't work anymore.
> The reason seems to be that argparse thinks the last argument of the first
> parser is the last but one argument.
> Hence, if a subparser takes some arguments, it fails.
>
> Example: if the "-n" argument of the foo parser is set mandatory (so
> becomes "n" instead of "-n")
>
> python toto.py foo.txt bar.txt foo 10
> usage: toto.py [-h] [fname [fname ...]] command ...
> toto.py: error: argument command: invalid choice: '10' (choose from 'foo',
> 'bar')
>

What about:

$ python toto.py foo.txt bar.txt foo -n 10

Note that contrary to what you said above, your program does not work like
a "standard unix tool". A standard command line program to do what you want
would normally look like

$ python toto.py foo -n 10 foo.txt bar.txt

or perhaps

$ python toto.py foo foo.txt bar.txt -n 10

so that the algorithm for differentiating the command 'foo' from the
filenames is well defined. How do you propose that your user enters a
filename 'foo'?

Oscar.


>
> Any solution?
>
> Cheers,
> Ben
>
>
>
> Le Jul 31, 2012 à 12:37 PM, Benoist Laurent a écrit :
>
> Really sorry about that.
>
> So, for the community, below is the full code for a tool that behaves like
> a Unix standard tool.
> It takes in argument the files to process and a command.
>
> """Just to setup a command-line parser that acts just like a unix
> standard tool."""
>
> import argparse
> import sys
>
> def define_options():
> parser = argparse.ArgumentParser()
> parser.add_argument("fname", help="input file", nargs="*")
>
> # create subparsers
> subparsers = parser.add_subparsers(dest="cmd", metavar="command")
>
> # create the parser for the "foo" command
> get_parser = subparsers.add_parser("foo", help="foo help")
> get_parser.add_argument("-n", help="number of foo to print",
> type=int, default=10)
>
> # create the parser for the "bar" command
> sum_parser = subparsers.add_parser("bar", help="bar help")
>
> return parser
>
>
> if __name__ == '__main__':
> args = define_options().parse_args()
>
> if not args.fname:
> content = sys.stdin.read()
> # do something
> else:
> for fname in args.fname:
> with(open(fname, "rt")) as f:
> content = f.read()
> # do somet
>
>
> Benoist
>
>
>
> Le Jul 31, 2012 à 11:55 AM, Oscar Benjamin a écrit :
>
>
> On Jul 31, 2012 10:32 AM, "Benoist Laurent" <benoist [at] ibpc> wrote:
> >
> > Well sorry about that but it seems I was wrong.
> > It was Friday evening and I guess I've not been careful.
> >
> > Actually when you specify nargs="?", the doc says "One argument will be
> consumed from the command line if possible, and produced as a single item".
> > So you can't pass several arguments to the program.
>
> Right below that in the docs it explains about using nargs='*' and
> nargs='+'. One of those will do what you want.
>
> Oscar.
>
> >
> > So, to rephrase the question, how can I get a argument parser that
> parses the command-line just as Unix grep would do?
> > i.e.
> >
> > $ echo 42 > foo.txt
> > $ echo 172 >> foo.txt
> > $ cp foo.txt bar.txt
> > $
> > $ grep 42 foo.txt
> > 42
> > $ grep 42 foo.txt bar.txt
> > foo.txt:42
> > bar.txt:42
> > $ cat foo.txt | grep 42
> > 42
> > $ grep -c 42 foo.txt
> > 1
> >
> >
> > Cheers,
> > Ben
> >
> >
> >
> >
> > Le Jul 27, 2012 à 7:08 PM, Benoist Laurent a écrit :
> >
> >>
> >>
> >> Yes basically looks like you get it.
> >> I have to further test it but my first impression is that it's correct.
> >>
> >> So actually the point was to use nargs="?".
> >>
> >> Thank you very much.
> >> Ben
> >>
> >>
> >>
> >> Le Jul 27, 2012 à 5:44 PM, Peter Otten a écrit :
> >>
> >>> Benoist Laurent wrote:
> >>>
> >>>> I'm impletting a tool in Python.
> >>>>
> >>>> I'd like this tool to behave like a standard unix tool, as grep for
> >>>>
> >>>> exemple. I chose to use the argparse module to parse the command line
> and
> >>>>
> >>>> I think I'm getting into several limitations of this module.
> >>>>
> >>>>
> >>>>> First Question.
> >>>>
> >>>> How can I configure the the ArgumentParser to allow the user to give
> >>>>
> >>>> either an input file or to pipe the output from another program?
> >>>>
> >>>>
> >>>> $ mytool.py file.txt
> >>>>
> >>>> $ cat file.txt | mytool.py
> >>>
> >>>
> >>> $ echo alpha > in.txt
> >>> $ cat in.txt | ./mytool.py
> >>> ALPHA
> >>> $ cat in.txt | ./mytool.py - out.txt
> >>> $ cat out.txt
> >>> ALPHA
> >>> $ ./mytool.py in.txt
> >>> ALPHA
> >>> $ ./mytool.py in.txt out2.txt
> >>> $ cat out2.txt
> >>> ALPHA
> >>> $ cat ./mytool.py
> >>> #!/usr/bin/env python
> >>> assert __name__ == "__main__"
> >>>
> >>> import argparse
> >>> import sys
> >>>
> >>> parser = argparse.ArgumentParser()
> >>> parser.add_argument("infile", nargs="?", type=argparse.FileType("r"),
> >>> default=sys.stdin)
> >>> parser.add_argument("outfile", nargs="?", type=argparse.FileType("w"),
> >>> default=sys.stdout)
> >>> args = parser.parse_args()
> >>>
> >>> args.outfile.writelines(line.upper() for line in args.infile)
> >>>
> >>> Is that good enough?
> >>>
> >>>
> >>> --
> >>> http://mail.python.org/mailman/listinfo/python-list
> >>>
> >>
> >> --
> >> Benoist Laurent
> >> Laboratoire de Biochimie Theorique / CNRS UPR 9080
> >> Institut de Biologie Physico-Chimique
> >> 13, rue Pierre et Marie Curie
> >> F-75005 Paris
> >> Tel. +33 [0]1 58 41 51 67 or +33 [0]6 21 64 50 56
> >>
> >> --
> >> http://mail.python.org/mailman/listinfo/python-list
> >
> >
> > --
> > Benoist Laurent
> > Laboratoire de Biochimie Theorique / CNRS UPR 9080
> > Institut de Biologie Physico-Chimique
> > 13, rue Pierre et Marie Curie
> > F-75005 Paris
> > Tel. +33 [0]1 58 41 51 67 or +33 [0]6 21 64 50 56
> >
> >
> > --
> > http://mail.python.org/mailman/listinfo/python-list
> >
>
>
> --
> Benoist Laurent
> Laboratoire de Biochimie Theorique / CNRS UPR 9080
> Institut de Biologie Physico-Chimique
> 13, rue Pierre et Marie Curie
> F-75005 Paris
> Tel. +33 [0]1 58 41 51 67 or +33 [0]6 21 64 50 56
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>
>
> --
> Benoist Laurent
> Laboratoire de Biochimie Theorique / CNRS UPR 9080
> Institut de Biologie Physico-Chimique
> 13, rue Pierre et Marie Curie
> F-75005 Paris
> Tel. +33 [0]1 58 41 51 67 or +33 [0]6 21 64 50 56
>
>


benoist at ibpc

Jul 31, 2012, 5:51 AM

Post #13 of 15 (806 views)
Permalink
Re: argparse limitations [In reply to]

Le Jul 31, 2012 à 1:45 PM, Oscar Benjamin a écrit :

>
>
> On 31 July 2012 12:03, Benoist Laurent <benoist [at] ibpc> wrote:
> Finally.
>
> The code I proposed doesn't work in this case: if you add any positional argument to one of the subparsers, then the parsing doesn't work anymore.
> The reason seems to be that argparse thinks the last argument of the first parser is the last but one argument.
> Hence, if a subparser takes some arguments, it fails.
>
> Example: if the "-n" argument of the foo parser is set mandatory (so becomes "n" instead of "-n")
>
> python toto.py foo.txt bar.txt foo 10
> usage: toto.py [-h] [fname [fname ...]] command ...
> toto.py: error: argument command: invalid choice: '10' (choose from 'foo', 'bar')
>
> What about:
>
> $ python toto.py foo.txt bar.txt foo -n 10
>
> Note that contrary to what you said above, your program does not work like a "standard unix tool". A standard command line program to do what you want would normally look like

You're right.
But then, using argparse, I would have to add the same argument to all my subparsers since argparse does the work sequentially: once it recognized the start of a subparser, everything that follows have to be an argument of this subparser.
Hence, arguments (therefore options) from the main parser are not recognized anymore.

>
> $ python toto.py foo -n 10 foo.txt bar.txt

Same remark as above: if I want the command line to look like this, then I would have setup each subparser with all the main program options.


>
> or perhaps
>
> $ python toto.py foo foo.txt bar.txt -n 10
>
> so that the algorithm for differentiating the command 'foo' from the filenames is well defined. How do you propose that your user enters a filename 'foo'?

Well I guess it is a intrinsec limitation: I think it's quite natural that the user can't enter a filename named as a command.


>
> Oscar.
>
>
>
> Any solution?
>
> Cheers,
> Ben
>
>
>
> Le Jul 31, 2012 à 12:37 PM, Benoist Laurent a écrit :
>
>> Really sorry about that.
>>
>> So, for the community, below is the full code for a tool that behaves like a Unix standard tool.
>> It takes in argument the files to process and a command.
>>
>> """Just to setup a command-line parser that acts just like a unix
>> standard tool."""
>>
>> import argparse
>> import sys
>>
>> def define_options():
>> parser = argparse.ArgumentParser()
>> parser.add_argument("fname", help="input file", nargs="*")
>>
>> # create subparsers
>> subparsers = parser.add_subparsers(dest="cmd", metavar="command")
>>
>> # create the parser for the "foo" command
>> get_parser = subparsers.add_parser("foo", help="foo help")
>> get_parser.add_argument("-n", help="number of foo to print",
>> type=int, default=10)
>>
>> # create the parser for the "bar" command
>> sum_parser = subparsers.add_parser("bar", help="bar help")
>>
>> return parser
>>
>>
>> if __name__ == '__main__':
>> args = define_options().parse_args()
>>
>> if not args.fname:
>> content = sys.stdin.read()
>> # do something
>> else:
>> for fname in args.fname:
>> with(open(fname, "rt")) as f:
>> content = f.read()
>> # do somet
>>
>>
>> Benoist
>>
>>
>>
>> Le Jul 31, 2012 à 11:55 AM, Oscar Benjamin a écrit :
>>
>>>
>>> On Jul 31, 2012 10:32 AM, "Benoist Laurent" <benoist [at] ibpc> wrote:
>>> >
>>> > Well sorry about that but it seems I was wrong.
>>> > It was Friday evening and I guess I've not been careful.
>>> >
>>> > Actually when you specify nargs="?", the doc says "One argument will be consumed from the command line if possible, and produced as a single item".
>>> > So you can't pass several arguments to the program.
>>>
>>> Right below that in the docs it explains about using nargs='*' and nargs='+'. One of those will do what you want.
>>>
>>> Oscar.
>>>
>>> >
>>> > So, to rephrase the question, how can I get a argument parser that parses the command-line just as Unix grep would do?
>>> > i.e.
>>> >
>>> > $ echo 42 > foo.txt
>>> > $ echo 172 >> foo.txt
>>> > $ cp foo.txt bar.txt
>>> > $
>>> > $ grep 42 foo.txt
>>> > 42
>>> > $ grep 42 foo.txt bar.txt
>>> > foo.txt:42
>>> > bar.txt:42
>>> > $ cat foo.txt | grep 42
>>> > 42
>>> > $ grep -c 42 foo.txt
>>> > 1
>>> >
>>> >
>>> > Cheers,
>>> > Ben
>>> >
>>> >
>>> >
>>> >
>>> > Le Jul 27, 2012 à 7:08 PM, Benoist Laurent a écrit :
>>> >
>>> >>
>>> >>
>>> >> Yes basically looks like you get it.
>>> >> I have to further test it but my first impression is that it's correct.
>>> >>
>>> >> So actually the point was to use nargs="?".
>>> >>
>>> >> Thank you very much.
>>> >> Ben
>>> >>
>>> >>
>>> >>
>>> >> Le Jul 27, 2012 à 5:44 PM, Peter Otten a écrit :
>>> >>
>>> >>> Benoist Laurent wrote:
>>> >>>
>>> >>>> I'm impletting a tool in Python.
>>> >>>>
>>> >>>> I'd like this tool to behave like a standard unix tool, as grep for
>>> >>>>
>>> >>>> exemple. I chose to use the argparse module to parse the command line and
>>> >>>>
>>> >>>> I think I'm getting into several limitations of this module.
>>> >>>>
>>> >>>>
>>> >>>>> First Question.
>>> >>>>
>>> >>>> How can I configure the the ArgumentParser to allow the user to give
>>> >>>>
>>> >>>> either an input file or to pipe the output from another program?
>>> >>>>
>>> >>>>
>>> >>>> $ mytool.py file.txt
>>> >>>>
>>> >>>> $ cat file.txt | mytool.py
>>> >>>
>>> >>>
>>> >>> $ echo alpha > in.txt
>>> >>> $ cat in.txt | ./mytool.py
>>> >>> ALPHA
>>> >>> $ cat in.txt | ./mytool.py - out.txt
>>> >>> $ cat out.txt
>>> >>> ALPHA
>>> >>> $ ./mytool.py in.txt
>>> >>> ALPHA
>>> >>> $ ./mytool.py in.txt out2.txt
>>> >>> $ cat out2.txt
>>> >>> ALPHA
>>> >>> $ cat ./mytool.py
>>> >>> #!/usr/bin/env python
>>> >>> assert __name__ == "__main__"
>>> >>>
>>> >>> import argparse
>>> >>> import sys
>>> >>>
>>> >>> parser = argparse.ArgumentParser()
>>> >>> parser.add_argument("infile", nargs="?", type=argparse.FileType("r"),
>>> >>> default=sys.stdin)
>>> >>> parser.add_argument("outfile", nargs="?", type=argparse.FileType("w"),
>>> >>> default=sys.stdout)
>>> >>> args = parser.parse_args()
>>> >>>
>>> >>> args.outfile.writelines(line.upper() for line in args.infile)
>>> >>>
>>> >>> Is that good enough?
>>> >>>
>>> >>>
>>> >>> --
>>> >>> http://mail.python.org/mailman/listinfo/python-list
>>> >>>
>>> >>
>>> >> --
>>> >> Benoist Laurent
>>> >> Laboratoire de Biochimie Theorique / CNRS UPR 9080
>>> >> Institut de Biologie Physico-Chimique
>>> >> 13, rue Pierre et Marie Curie
>>> >> F-75005 Paris
>>> >> Tel. +33 [0]1 58 41 51 67 or +33 [0]6 21 64 50 56
>>> >>
>>> >> --
>>> >> http://mail.python.org/mailman/listinfo/python-list
>>> >
>>> >
>>> > --
>>> > Benoist Laurent
>>> > Laboratoire de Biochimie Theorique / CNRS UPR 9080
>>> > Institut de Biologie Physico-Chimique
>>> > 13, rue Pierre et Marie Curie
>>> > F-75005 Paris
>>> > Tel. +33 [0]1 58 41 51 67 or +33 [0]6 21 64 50 56
>>> >
>>> >
>>> > --
>>> > http://mail.python.org/mailman/listinfo/python-list
>>> >
>>
>> --
>> Benoist Laurent
>> Laboratoire de Biochimie Theorique / CNRS UPR 9080
>> Institut de Biologie Physico-Chimique
>> 13, rue Pierre et Marie Curie
>> F-75005 Paris
>> Tel. +33 [0]1 58 41 51 67 or +33 [0]6 21 64 50 56
>>
>> --
>> http://mail.python.org/mailman/listinfo/python-list
>
>
> --
> Benoist Laurent
> Laboratoire de Biochimie Theorique / CNRS UPR 9080
> Institut de Biologie Physico-Chimique
> 13, rue Pierre et Marie Curie
> F-75005 Paris
> Tel. +33 [0]1 58 41 51 67 or +33 [0]6 21 64 50 56
>
>

--
Benoist Laurent
Laboratoire de Biochimie Theorique / CNRS UPR 9080
Institut de Biologie Physico-Chimique
13, rue Pierre et Marie Curie
F-75005 Paris
Tel. +33 [0]1 58 41 51 67 or +33 [0]6 21 64 50 56


oscar.j.benjamin at gmail

Jul 31, 2012, 6:04 AM

Post #14 of 15 (804 views)
Permalink
Re: argparse limitations [In reply to]

On 31 July 2012 13:51, Benoist Laurent <benoist [at] ibpc> wrote:

>
> Le Jul 31, 2012 à 1:45 PM, Oscar Benjamin a écrit :
>
>
>
> On 31 July 2012 12:03, Benoist Laurent <benoist [at] ibpc> wrote:
>
>> Finally.
>>
>> The code I proposed doesn't work in this case: if you add any positional
>> argument to one of the subparsers, then the parsing doesn't work anymore.
>> The reason seems to be that argparse thinks the last argument of the
>> first parser is the last but one argument.
>> Hence, if a subparser takes some arguments, it fails.
>>
>> Example: if the "-n" argument of the foo parser is set mandatory (so
>> becomes "n" instead of "-n")
>>
>> python toto.py foo.txt bar.txt foo 10
>> usage: toto.py [-h] [fname [fname ...]] command ...
>> toto.py: error: argument command: invalid choice: '10' (choose from
>> 'foo', 'bar')
>>
>
> What about:
>
> $ python toto.py foo.txt bar.txt foo -n 10
>
> Note that contrary to what you said above, your program does not work like
> a "standard unix tool". A standard command line program to do what you want
> would normally look like
>
>
> You're right.
> But then, using argparse, I would have to add the same argument to all my
> subparsers since argparse does the work sequentially: once it recognized
> the start of a subparser, everything that follows have to be an argument of
> this subparser.
> Hence, arguments (therefore options) from the main parser are not
> recognized anymore.
>

If the parsing after the subcommand is to be the same for each subcommand,
then don't use subparsers. You could just make the first argument be the
command name and use one parser for everything.

If the parsing is supposed to be different for different subcommands then
use subparsers and add the files argument to each subparser; you can make a
function to add the common arguments and options if you don't want to
duplicate the code.

Well I guess it is a intrinsec limitation: I think it's quite natural that
> the user can't enter a filename named as a command.
>

There is an intrinsic limitation on any parser that it must be possible to
determine the targets of the arguments uniquely. If you want the parser to
scan through and take the first argument matching 'foo' or 'bar' and parse
the remaining arguments accordingly then you already have your solution. It
just won't work if the user wants to pass in a file called 'foo' or 'bar'
(maybe that's acceptable, though).

The standard way, however, is to have a parser that takes the first
non-option argument as a subcommand name and parses the remaining arguments
according to that subcommand. Your command line users are more likely to be
able to understand how to use the program if it works that way.

Oscar.


benoist at ibpc

Jul 31, 2012, 6:24 AM

Post #15 of 15 (807 views)
Permalink
Re: argparse limitations [In reply to]

> The standard way, however, is to have a parser that takes the first non-option argument as a subcommand name and parses the remaining arguments according to that subcommand. Your command line users are more likely to be able to understand how to use the program if it works that way.

I'll do this way.
Thank a lot.

Ben

Le Jul 31, 2012 à 3:04 PM, Oscar Benjamin a écrit :

>
>
> On 31 July 2012 13:51, Benoist Laurent <benoist [at] ibpc> wrote:
>
> Le Jul 31, 2012 à 1:45 PM, Oscar Benjamin a écrit :
>
>>
>>
>> On 31 July 2012 12:03, Benoist Laurent <benoist [at] ibpc> wrote:
>> Finally.
>>
>> The code I proposed doesn't work in this case: if you add any positional argument to one of the subparsers, then the parsing doesn't work anymore.
>> The reason seems to be that argparse thinks the last argument of the first parser is the last but one argument.
>> Hence, if a subparser takes some arguments, it fails.
>>
>> Example: if the "-n" argument of the foo parser is set mandatory (so becomes "n" instead of "-n")
>>
>> python toto.py foo.txt bar.txt foo 10
>> usage: toto.py [-h] [fname [fname ...]] command ...
>> toto.py: error: argument command: invalid choice: '10' (choose from 'foo', 'bar')
>>
>> What about:
>>
>> $ python toto.py foo.txt bar.txt foo -n 10
>>
>> Note that contrary to what you said above, your program does not work like a "standard unix tool". A standard command line program to do what you want would normally look like
>
> You're right.
> But then, using argparse, I would have to add the same argument to all my subparsers since argparse does the work sequentially: once it recognized the start of a subparser, everything that follows have to be an argument of this subparser.
> Hence, arguments (therefore options) from the main parser are not recognized anymore.
>
> If the parsing after the subcommand is to be the same for each subcommand, then don't use subparsers. You could just make the first argument be the command name and use one parser for everything.
>
> If the parsing is supposed to be different for different subcommands then use subparsers and add the files argument to each subparser; you can make a function to add the common arguments and options if you don't want to duplicate the code.
>
> Well I guess it is a intrinsec limitation: I think it's quite natural that the user can't enter a filename named as a command.
>
> There is an intrinsic limitation on any parser that it must be possible to determine the targets of the arguments uniquely. If you want the parser to scan through and take the first argument matching 'foo' or 'bar' and parse the remaining arguments accordingly then you already have your solution. It just won't work if the user wants to pass in a file called 'foo' or 'bar' (maybe that's acceptable, though).
>
> The standard way, however, is to have a parser that takes the first non-option argument as a subcommand name and parses the remaining arguments according to that subcommand. Your command line users are more likely to be able to understand how to use the program if it works that way.
>
> Oscar.

--
Benoist Laurent
Laboratoire de Biochimie Theorique / CNRS UPR 9080
Institut de Biologie Physico-Chimique
13, rue Pierre et Marie Curie
F-75005 Paris
Tel. +33 [0]1 58 41 51 67 or +33 [0]6 21 64 50 56

Python python RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.