Mailing List Archive: Zope: Coders

zLOG warnings and testrunner

Index | Next | Previous | View Threaded

jeremy at zope

Sep 26, 2001, 9:35 PM

Post #1 of 18 (2399 views)
Permalink

zLOG warnings and testrunner

There of the test failures in the Zope suite are caused by warnings
logged by FileStorage. Two of the three are tests that I added
recently; they verify that a storage can be loaded even if the index
file is corrupted.

I don't want these tests to fail, because the code works. I'm not
sure whether I should change the log level or if there is something
wrong with the test framework.

The current FileStorage code logs a warning/problem `if the index file
is corrupted. I chose warning because the description of it in
zLOG/__init__.py seemed to match: "This isn't causing any immediate
problems, but deserves attention." If the index file is corrupted,
the FileStorage can continue, but the operator ought to know that a
file was corrupted and check to make sure that there isn't some more
serious problem.

Some specific questions:

Why does log messages cause tests to fail?

When should I use PROBLEM and higher severity levels with zLOG?

Should I use a different severity for the specific corrupted index
file messages?

Jeremy

jeremy at zope

Sep 26, 2001, 9:43 PM

Post #2 of 18 (2334 views)
Permalink

Re: zLOG warnings and testrunner [In reply to]

It looks like two of the failures in the SearchIndex tests are also
caused by logging errors that turn into test failures. It looks like
the tests are expecting to continue despite the logged error.

Jeremy

andreas at zope

Sep 27, 2001, 5:50 AM

Post #3 of 18 (2334 views)
Permalink

Re: zLOG warnings and testrunner [In reply to]

----- Original Message -----
From: "Jeremy Hylton" <jeremy [at] zope>
To: <evan [at] zope>; <zope-coders [at] zope>
Sent: Thursday, September 27, 2001 00:43
Subject: Re: [Zope-Coders] zLOG warnings and testrunner

> It looks like two of the failures in the SearchIndex tests are also
> caused by logging errors that turn into test failures. It looks like
> the tests are expecting to continue despite the logged error.

All stuff in lib/python/SearchIndex should be regarded as deprecated.
All modules are refactored inside Products/PluginIndexes. We keep
SearchIndex
only for backward compat. for third-party products. Maybe we should use
the warnings module to inform the users that the old SearchIndex stuff will
go
away someday ?

Andreas

chrism at digicool

Sep 27, 2001, 6:29 AM

Post #4 of 18 (2331 views)
Permalink

Re: zLOG warnings and testrunner [In reply to]

Jeremy, you might try setting a STUPID_LOG_FILE envvar (no Im not
kidding) to a file, rerun the tests, and see what happens. I have no
idea why pyunit is choking on Zope log calls, especially because by
default nothing actually makes its way out of Zope without
STUPID_LOG_FILE set.

Jeremy Hylton wrote:
> There of the test failures in the Zope suite are caused by warnings
> logged by FileStorage. Two of the three are tests that I added
> recently; they verify that a storage can be loaded even if the index
> file is corrupted.
>
> I don't want these tests to fail, because the code works. I'm not
> sure whether I should change the log level or if there is something
> wrong with the test framework.
>
> The current FileStorage code logs a warning/problem `if the index file
> is corrupted. I chose warning because the description of it in
> zLOG/__init__.py seemed to match: "This isn't causing any immediate
> problems, but deserves attention." If the index file is corrupted,
> the FileStorage can continue, but the operator ought to know that a
> file was corrupted and check to make sure that there isn't some more
> serious problem.
>
> Some specific questions:
>
> Why does log messages cause tests to fail?
>
> When should I use PROBLEM and higher severity levels with zLOG?
>
> Should I use a different severity for the specific corrupted index
> file messages?
>
> Jeremy
>
>
>
>
>
>
> _______________________________________________
> Zope-Coders mailing list
> Zope-Coders [at] zope
> http://lists.zope.org/mailman/listinfo/zope-coders
>

--
Chris McDonough Zope Corporation
http://www.zope.org http://www.zope.com
"Killing hundreds of birds with thousands of stones"

chrism at digicool

Sep 27, 2001, 6:33 AM

Post #5 of 18 (2332 views)
Permalink

Re: zLOG warnings and testrunner [In reply to]

> All stuff in lib/python/SearchIndex should be regarded as deprecated.
> All modules are refactored inside Products/PluginIndexes. We keep
> SearchIndex
> only for backward compat. for third-party products. Maybe we should use
> the warnings module to inform the users that the old SearchIndex stuff will
> go
> away someday ?

Is it really backwards compat with 3rd-party products or is it just to
allow non-plugin indexes to continue to work after a Zope upgrade from a
version without them?

andreas at zope

Sep 27, 2001, 6:33 AM

Post #6 of 18 (2334 views)
Permalink

Re: zLOG warnings and testrunner [In reply to]

backwards compatible in the sense that old stuff still works
after an upgrade.

Andreas
----- Original Message -----
From: "Chris McDonough" <chrism [at] digicool>
To: "Andreas Jung" <andreas [at] zope>
Cc: <jeremy [at] zope>; <evan [at] zope>; <zope-coders [at] zope>
Sent: Thursday, September 27, 2001 09:33
Subject: Re: [Zope-Coders] zLOG warnings and testrunner

> > All stuff in lib/python/SearchIndex should be regarded as deprecated.
> > All modules are refactored inside Products/PluginIndexes. We keep
> > SearchIndex
> > only for backward compat. for third-party products. Maybe we should use
> > the warnings module to inform the users that the old SearchIndex stuff
will
> > go
> > away someday ?
>
> Is it really backwards compat with 3rd-party products or is it just to
> allow non-plugin indexes to continue to work after a Zope upgrade from a
> version without them?
>
>

jeremy at zope

Sep 27, 2001, 6:58 AM

Post #7 of 18 (2335 views)
Permalink

Re: zLOG warnings and testrunner [In reply to]

>>>>> "AJ" == Andreas Jung <andreas [at] zope> writes:

>> It looks like two of the failures in the SearchIndex tests are
>> also caused by logging errors that turn into test failures. It
>> looks like the tests are expecting to continue despite the logged
>> error.

AJ> All stuff in lib/python/SearchIndex should be regarded as
AJ> deprecated. All modules are refactored inside
AJ> Products/PluginIndexes. We keep SearchIndex only for backward
AJ> compat. for third-party products. Maybe we should use the
AJ> warnings module to inform the users that the old SearchIndex
AJ> stuff will go away someday ?

Then we may also want to prevent testrunner from running the
SearchIndex tests. If we've got failing tests and no plans to fix
them, there's not much point in ever running them.

Jeremy

jeremy at zope

Sep 27, 2001, 7:02 AM

Post #8 of 18 (2330 views)
Permalink

Re: zLOG warnings and testrunner [In reply to]

>>>>> "CMcD" == Chris McDonough <chrism [at] digicool> writes:

CMcD> Jeremy, you might try setting a STUPID_LOG_FILE envvar (no Im
CMcD> not kidding) to a file, rerun the tests, and see what happens.
CMcD> I have no idea why pyunit is choking on Zope log calls,
CMcD> especially because by default nothing actually makes its way
CMcD> out of Zope without STUPID_LOG_FILE set.

The Zope Testing.common module installs a log handler that raises an
AssertionError when a warning or higher is logged. So it's the policy
of the Zope testing framework, but I've no idea why :-).

Jeremy

andreas at zope

Sep 27, 2001, 7:09 AM

Post #9 of 18 (2341 views)
Permalink

Re: zLOG warnings and testrunner [In reply to]

We can remove the tests or just rename the tests directory to
something like "tests_old".

Andreas
----- Original Message -----
From: "Jeremy Hylton" <jeremy [at] zope>
To: "Andreas Jung" <andreas [at] zope>
Cc: <jeremy [at] zope>; <evan [at] zope>; <zope-coders [at] zope>
Sent: Thursday, September 27, 2001 09:58
Subject: Re: [Zope-Coders] zLOG warnings and testrunner

> >>>>> "AJ" == Andreas Jung <andreas [at] zope> writes:
>
> >> It looks like two of the failures in the SearchIndex tests are
> >> also caused by logging errors that turn into test failures. It
> >> looks like the tests are expecting to continue despite the logged
> >> error.
>
> AJ> All stuff in lib/python/SearchIndex should be regarded as
> AJ> deprecated. All modules are refactored inside
> AJ> Products/PluginIndexes. We keep SearchIndex only for backward
> AJ> compat. for third-party products. Maybe we should use the
> AJ> warnings module to inform the users that the old SearchIndex
> AJ> stuff will go away someday ?
>
> Then we may also want to prevent testrunner from running the
> SearchIndex tests. If we've got failing tests and no plans to fix
> them, there's not much point in ever running them.
>
> Jeremy
>
>
>
> _______________________________________________
> Zope-Coders mailing list
> Zope-Coders [at] zope
> http://lists.zope.org/mailman/listinfo/zope-coders
>

evan at zope

Sep 27, 2001, 7:21 AM

Post #10 of 18 (2332 views)
Permalink

Re: zLOG warnings and testrunner [In reply to]

Jeremy Hylton wrote:

> The Zope Testing.common module installs a log handler that raises an
> AssertionError when a warning or higher is logged. So it's the policy
> of the Zope testing framework, but I've no idea why :-).

As with so much in the framework, that's there because several tests
were doing it. There is also a function in there to turn off the log
handler, for tests that expect log messages and don't want to pay
attention to them.

Cheers,

Evan

chrism at zope

Sep 27, 2001, 7:23 AM

Post #11 of 18 (2333 views)
Permalink

Re: zLOG warnings and testrunner [In reply to]

I don't think it's a well thought-through policy. Maybe this behavior
should be settable via a function in Testing.common that should not be
invoked by default. Additionally, maybe instead of failing with a silly
uninformative message, it could say it failed because a log message above a
certain threshhold was seen.

----- Original Message -----
From: "Jeremy Hylton" <jeremy [at] zope>
To: <chrism [at] zope>
Cc: <jeremy [at] zope>; <evan [at] zope>; <zope-coders [at] zope>
Sent: Thursday, September 27, 2001 10:02 AM
Subject: Re: [Zope-Coders] zLOG warnings and testrunner

> >>>>> "CMcD" == Chris McDonough <chrism [at] digicool> writes:
>
> CMcD> Jeremy, you might try setting a STUPID_LOG_FILE envvar (no Im
> CMcD> not kidding) to a file, rerun the tests, and see what happens.
> CMcD> I have no idea why pyunit is choking on Zope log calls,
> CMcD> especially because by default nothing actually makes its way
> CMcD> out of Zope without STUPID_LOG_FILE set.
>
> The Zope Testing.common module installs a log handler that raises an
> AssertionError when a warning or higher is logged. So it's the policy
> of the Zope testing framework, but I've no idea why :-).
>
> Jeremy
>
>

jim at zope

Sep 27, 2001, 7:35 AM

Post #12 of 18 (2332 views)
Permalink

Re: zLOG warnings and testrunner [In reply to]

Andreas Jung wrote:
>
> backwards compatible in the sense that old stuff still works
> after an upgrade.

There are *also* third-party products that depend on them.

Perhaps, in some future version, we should:

- Exclude them from regular distributions

- Provide a distrobution of them so people
can install them if they need to.

Jim

--
Jim Fulton mailto:jim [at] zope Python Powered!
CTO (888) 344-4332 http://www.python.org
Zope Corporation http://www.zope.com http://www.zope.org

jim at zope

Sep 27, 2001, 7:38 AM

Post #13 of 18 (2336 views)
Permalink

Re: zLOG warnings and testrunner [In reply to]

Jeremy Hylton wrote:
>
> >>>>> "CMcD" == Chris McDonough <chrism [at] digicool> writes:
>
> CMcD> Jeremy, you might try setting a STUPID_LOG_FILE envvar (no Im
> CMcD> not kidding) to a file, rerun the tests, and see what happens.
> CMcD> I have no idea why pyunit is choking on Zope log calls,
> CMcD> especially because by default nothing actually makes its way
> CMcD> out of Zope without STUPID_LOG_FILE set.
>
> The Zope Testing.common module installs a log handler that raises an
> AssertionError when a warning or higher is logged. So it's the policy
> of the Zope testing framework, but I've no idea why :-).

It's done beause many errors are logged rather than raised
and indicate failure. This is an important and valuable feature.

I suggest that we make the default log/est threashold "ERROR"
and provide an testing API for lowering it.

Jim

--
Jim Fulton mailto:jim [at] zope Python Powered!
CTO (888) 344-4332 http://www.python.org
Zope Corporation http://www.zope.com http://www.zope.org

c.duncan at nlada

Sep 27, 2001, 8:26 AM

Post #14 of 18 (2333 views)
Permalink

Re: zLOG warnings and testrunner [In reply to]

On Thursday 27 September 2001 08:50 am, Andreas Jung allegedly wrote:
> ----- Original Message -----
> From: "Jeremy Hylton" <jeremy [at] zope>
> To: <evan [at] zope>; <zope-coders [at] zope>
> Sent: Thursday, September 27, 2001 00:43
> Subject: Re: [Zope-Coders] zLOG warnings and testrunner
>
> > It looks like two of the failures in the SearchIndex tests are also
> > caused by logging errors that turn into test failures. It looks like
> > the tests are expecting to continue despite the logged error.
>
> All stuff in lib/python/SearchIndex should be regarded as deprecated.
> All modules are refactored inside Products/PluginIndexes. We keep
> SearchIndex
> only for backward compat. for third-party products. Maybe we should use
> the warnings module to inform the users that the old SearchIndex stuff will
> go
> away someday ?
>
> Andreas

I would say yes and do so as a rule. What is the policy for how long
deprecated stuff hangs around? Until the next maj version?

/---------------------------------------------------\
Casey Duncan, Sr. Web Developer
National Legal Aid and Defender Association
c.duncan [at] nlada
\---------------------------------------------------/

jeremy at zope

Sep 27, 2001, 9:24 AM

Post #15 of 18 (2329 views)
Permalink

Re: zLOG warnings and testrunner [In reply to]

We should have a method available for unit tests that is analogous to
assertRaises(). The method assertLogs() would be used for an
operation where you expect the operation to execute without raising an
exception *and* to log an error message. It would better to be
explicit in the test method about what should log a message and what
shouldn't.

I think we should implement this by producing a zunit module that
extends unittest with whatever Zope specific features we need. The
write our unit tests to import zunit instead of unittest -- and use
zunit.main() instead of unittest.main().

We should be careful not to diverge too much from the pyunit
approach. Eventually, we may be able to push something like zLOG into
the Python standard library and push our zunit changes back into
pyunit.

Jeremy

amos at zope

Sep 27, 2001, 9:48 AM

Post #16 of 18 (2331 views)
Permalink

Re: zLOG warnings and testrunner [In reply to]

> Maybe we should use
> the warnings module to inform the users that the old SearchIndex stuff will
> go
> away someday ?

Yes, please, we need some kind of warnings or something. Folks need some
way to know that code is deprecated. Otherwise they'll never know to
migrate to the new code.

-Amos

--
Amos Latteier amos [at] zope
Zope Corporation http://www.zope.com/

jeremy at zope

Sep 27, 2001, 9:49 AM

Post #17 of 18 (2332 views)
Permalink

Re: zLOG warnings and testrunner [In reply to]

>>>>> "AL" == Amos Latteier <amos [at] zope> writes:

>> Maybe we should use the warnings module to inform the users that
>> the old SearchIndex stuff will go away someday ?

AL> Yes, please, we need some kind of warnings or something. Folks
AL> need some way to know that code is deprecated. Otherwise they'll
AL> never know to migrate to the new code.

Python 2.1 introduced the warnings module. It should do the trick.

Jeremy

chrisw at nipltd

Oct 3, 2001, 2:13 PM

Post #18 of 18 (2331 views)
Permalink

Re: zLOG warnings and testrunner [In reply to]

> invoked by default. Additionally, maybe instead of failing with a silly
> uninformative message, it could say it failed because a log message above
a
> certain threshhold was seen.

How about even printing out what the log message was?

Chris

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads