Mailing List Archive: Python: Bugs

[issue14408] Support ./python -m unittest in the stdlib tests

Index | Next | Previous | View Threaded

report at bugs

Mar 28, 2012, 9:44 AM

Post #1 of 20 (113 views)
Permalink

[issue14408] Support ./python -m unittest in the stdlib tests

Changes by Éric Araujo <merwok [at] netwok>:

----------
title: Support the load_tests protocol in the stdlib tests -> Support ./python -m unittest in the stdlib tests

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14408>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Mar 28, 2012, 10:14 AM

Post #2 of 20 (112 views)
Permalink

[issue14408] Support ./python -m unittest in the stdlib tests [In reply to]

R. David Murray <rdmurray [at] bitdance> added the comment:

The test_main functions can be converted to use unittest discovery, though. That's what I did for test_email.

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14408>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Mar 28, 2012, 10:18 AM

Post #3 of 20 (108 views)
Permalink

[issue14408] Support ./python -m unittest in the stdlib tests [In reply to]

Michael Foord <michael [at] voidspace> added the comment:

Test discovery is only needed for finding tests in directories of tests - for a single test module the standard test loader should work fine.

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14408>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Mar 28, 2012, 10:24 AM

Post #4 of 20 (109 views)
Permalink

[issue14408] Support ./python -m unittest in the stdlib tests [In reply to]

R. David Murray <rdmurray [at] bitdance> added the comment:

Right. What I meant to say was "test_main can be converted to use normal unittest test loading". test_email is not an example of that, since it does use test discovery, but is a good example of a test collection that works with both regrtest and unittest without excess boilerplate such as listing individual test suites.

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14408>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Mar 28, 2012, 10:05 PM

Post #5 of 20 (106 views)
Permalink

[issue14408] Support ./python -m unittest in the stdlib tests [In reply to]

Matt Joiner <anacrolix [at] gmail> added the comment:

Michael:

The thread setup and cleanup is not required, AFAICT. You are also correct in that these particular test modules do not run correctly without modification (although test_queue does now that the bug I reported there was fixed).

Sorry by predicated, I mean that we could mask out the base classes that many of the tests are enumerated on top of. Some kind of decorator for the base could prevent the base class being discovered unless it's inherited from. This is the source of the errors one currently gets running the tests via -m unittest.

Eric:

I'm not sure what you mean by the files being long. I've taken the existing test case list and exposed this directly to both load_tests and unittest.main. The load_tests boilerplate is disappointing, but explicit.

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14408>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Mar 29, 2012, 2:52 AM

Post #6 of 20 (103 views)
Permalink

[issue14408] Support ./python -m unittest in the stdlib tests [In reply to]

R. David Murray <rdmurray [at] bitdance> added the comment:

It sounds like we just need to fix the TestCase inheritance, like we did in test_queue.

We should also look more carefully at the threading setup/cleanup. At some point I think we changed the best-practice idiom to be independent of regrtest, but we probably didn't change all the tests. I'm not sure, though.

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14408>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Mar 29, 2012, 3:01 AM

Post #7 of 20 (105 views)
Permalink

[issue14408] Support ./python -m unittest in the stdlib tests [In reply to]

Michael Foord <michael [at] voidspace> added the comment:

One way to exclude base classes from being loaded as tests is to have the base class *not* inherit from TestCase (just from object) - and use it as a mixin class for the actual TestCases.

This isn't particularly elegant (but works fine). A better way of supporting this in unittest would be a reasonable request (a "not_a_test" class decorator or similar perhaps).

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14408>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Mar 29, 2012, 3:22 AM

Post #8 of 20 (104 views)
Permalink

[issue14408] Support ./python -m unittest in the stdlib tests [In reply to]

Matt Joiner <anacrolix [at] gmail> added the comment:

I'm working on a patch using TestCase a la test_queue. Perhaps we should create an issue for a base class test case decorator or something to that effect?

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14408>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Mar 29, 2012, 4:14 AM

Post #9 of 20 (103 views)
Permalink

[issue14408] Support ./python -m unittest in the stdlib tests [In reply to]

Michael Foord <michael [at] voidspace> added the comment:

Yes, feel free to create an issue for that. If you provide a patch for it (with tests) I'll review it.

The decorator itself can be applied to both TestCase and FunctionTestCase in unittest as well. One implementation would be to apply a private attribute to classes and the loader can be taught to ignore any classes that have that attribute in their __dict__ - i.e. not inherited.

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14408>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Mar 29, 2012, 4:54 AM

Post #10 of 20 (104 views)
Permalink

[issue14408] Support ./python -m unittest in the stdlib tests [In reply to]

R. David Murray <rdmurray [at] bitdance> added the comment:

Not particularly elegant? Why not? I find marking tests that should be executed by having them (and only them) inherit from TestCase to fit my sense of what is Pythonic, while having a hidden please-ignore-me attribute doesn't.

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14408>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Mar 29, 2012, 5:20 AM

Post #11 of 20 (103 views)
Permalink

[issue14408] Support ./python -m unittest in the stdlib tests [In reply to]

Michael Foord <michael [at] voidspace> added the comment:

Because you then have classes that inherit from object calling methods that clearly don't exist (until you subclass them *and* TestCase). It looks weird and also means the classes can't be tested in isolation.

With a class decorator the code would *look* straightforward, and the hidden attribute is just an implementation detail.

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14408>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Mar 29, 2012, 6:14 AM

Post #12 of 20 (103 views)
Permalink

[issue14408] Support ./python -m unittest in the stdlib tests [In reply to]

R. David Murray <rdmurray [at] bitdance> added the comment:

Hmm. OK, I guess we can just disagree on what looks straightforward, and since you are the maintainer of unittest you win :) But unless somebody pronounces, I'll probably keep using the mixin pattern for my own modules.

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14408>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Mar 29, 2012, 6:21 AM

Post #13 of 20 (104 views)
Permalink

[issue14408] Support ./python -m unittest in the stdlib tests [In reply to]

R. David Murray <rdmurray [at] bitdance> added the comment:

I guess I'm not really done talking about this, though my bow to you as maintainer still stands.

The mixin tests *can't* be run in isolation, that's the whole point. Otherwise you could just let unittest run them, and wouldn't need to mark them as not-really-a-TestCase. The typical mixin test class uses attributes that don't exist except on the concrete TestCases, so I'm not sure why calling methods that don't exist on the mixin is any worse :)

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14408>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Mar 29, 2012, 6:26 AM

Post #14 of 20 (104 views)
Permalink

[issue14408] Support ./python -m unittest in the stdlib tests [In reply to]

Michael Foord <michael [at] voidspace> added the comment:

It still looks weird to see code calling methods that obviously don't exist, and with no indication *at the call site* where they come from. Making it clearer with naming would help: "TestThingMixin" or similar.

There are classes like this in the unittest test suite, and I was very confused by them initially until I found where and how they were used. It is obviously *not* a pattern that is widely known for test base classes, as we have this problem of it not being done even in the standard library tests.

In contrast I think code similar to the following would be clear and readable without knowing about multiple inheritance and the mixin trick:

@test_base_class
class SomeTestBase(TestCase):
...

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14408>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Mar 29, 2012, 6:30 AM

Post #15 of 20 (107 views)
Permalink

[issue14408] Support ./python -m unittest in the stdlib tests [In reply to]

Michael Foord <michael [at] voidspace> added the comment:

Besides which, the mixin pattern won't *stop* working if we provide this extra functionality - it would just be an alternative for those (like myself) who think it impedes code readability. :-)

At this point we're off topic for the *specific issue*, and I'm fine with our own standard library tests moving to use mixins to support standard unittest invocation. I would suggest the base test cases include Mixin in their name to make it clear how they should be used.

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14408>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Mar 29, 2012, 6:41 AM

Post #16 of 20 (105 views)
Permalink

[issue14408] Support ./python -m unittest in the stdlib tests [In reply to]

R. David Murray <rdmurray [at] bitdance> added the comment:

The convention in the stdlib is to name the mixin classes TestXXXBase. Granted, a lot of those inherit from TestCase. I have no objection to calling them Mixin instead, I'm just pointing out that there is an existing convention.

(As an aside, when I first ran into the Base pattern it was in a file where the Base did subclass TestCase, and it took me forever to figure out that the Base test wasn't actually getting run. So a decorator is definitely superior to listing the test cases that actually run elsewhere in the file!)

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14408>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Mar 29, 2012, 7:55 AM

Post #17 of 20 (103 views)
Permalink

[issue14408] Support ./python -m unittest in the stdlib tests [In reply to]

Matt Joiner <anacrolix [at] gmail> added the comment:

It could in fact be necessary, if the inheritance cannot be juggled to give the right MRO. Fortunately this is not the case, I should have a patch using TestCase inheritance for discovery tomorrow.

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14408>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Apr 1, 2012, 4:05 AM

Post #18 of 20 (105 views)
Permalink

[issue14408] Support ./python -m unittest in the stdlib tests [In reply to]

Matt Joiner <anacrolix [at] gmail> added the comment:

The patch attached, rejigs the TestCase inheritance in test.test_socket so that tests run correctly using unittest discovery. Recent changes have made test_queue, and test_threading run without similar fixes, so I don't think fixes for those are necessary anymore.

----------
Added file: http://bugs.python.org/file25087/test.test_socket-discoverability.patch

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14408>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Apr 1, 2012, 4:29 AM

Post #19 of 20 (103 views)
Permalink

[issue14408] Support ./python -m unittest in the stdlib tests [In reply to]

Matt Joiner <anacrolix [at] gmail> added the comment:

Attached is a patch for test_concurrent_futures, similar to the patch for test_socket.

----------
Added file: http://bugs.python.org/file25088/test_concurrent_futures-unittest-discoverability.patch

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14408>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Apr 13, 2012, 9:26 PM

Post #20 of 20 (86 views)
Permalink

[issue14408] Support ./python -m unittest in the stdlib tests [In reply to]

Changes by Ezio Melotti <ezio.melotti [at] gmail>:

----------
nosy: +ezio.melotti

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14408>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads