Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Python: Dev

Drop the new time.wallclock() function?

 

 

Python dev RSS feed   Index | Next | Previous | View Threaded


victor.stinner at gmail

Mar 13, 2012, 4:57 PM

Post #1 of 67 (1986 views)
Permalink
Drop the new time.wallclock() function?

Hi,

I added two functions to the time module in Python 3.3: wallclock()
and monotonic(). I'm unable to explain the difference between these
two functions, even if I wrote them :-) wallclock() is suppose to be
more accurate than time() but has an unspecified starting point.
monotonic() is similar except that it is monotonic: it cannot go
backward. monotonic() may not be available or fail whereas wallclock()
is available/work, but I think that the two functions are redundant.

I prefer to keep only monotonic() because it is not affected by system
clock update and should help to fix issues on NTP update in functions
implementing a timeout.

What do you think?

--

monotonic() has 3 implementations:
* Windows: QueryPerformanceCounter() with QueryPerformanceFrequency()
* Mac OS X: mach_absolute_time() with mach_timebase_info()
* UNIX: clock_gettime(CLOCK_MONOTONIC_RAW) or clock_gettime(CLOCK_MONOTONIC)

wallclock() has 3 implementations:
* Windows: QueryPerformanceCounter() with QueryPerformanceFrequency(),
with a fallback to GetSystemTimeAsFileTime() if
QueryPerformanceFrequency() failed
* UNIX: clock_gettime(CLOCK_MONOTONIC_RAW),
clock_gettime(CLOCK_MONOTONIC) or clock_gettime(CLOCK_REALTIME), with
a fallback to gettimeofday() if clock_gettime(*) failed
* Otherwise: gettimeofday()

(wallclock should also use mach_absolute_time() on Mac OS X)


Victor
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


fuzzyman at voidspace

Mar 13, 2012, 5:03 PM

Post #2 of 67 (1971 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

On 13 Mar 2012, at 16:57, Victor Stinner wrote:

> Hi,
>
> I added two functions to the time module in Python 3.3: wallclock()
> and monotonic(). I'm unable to explain the difference between these
> two functions, even if I wrote them :-) wallclock() is suppose to be
> more accurate than time() but has an unspecified starting point.
> monotonic() is similar except that it is monotonic: it cannot go
> backward. monotonic() may not be available or fail whereas wallclock()
> is available/work, but I think that the two functions are redundant.
>
> I prefer to keep only monotonic() because it is not affected by system
> clock update and should help to fix issues on NTP update in functions
> implementing a timeout.
>
> What do you think?
>


I am in the middle of adding a feature to unittest that involves timing of individual tests. I want the highest resolution cross platform measure of wallclock time - and time.wallclock() looked ideal. If monotonic may not exist or can fail why would that be better?

Michael


> --
>
> monotonic() has 3 implementations:
> * Windows: QueryPerformanceCounter() with QueryPerformanceFrequency()
> * Mac OS X: mach_absolute_time() with mach_timebase_info()
> * UNIX: clock_gettime(CLOCK_MONOTONIC_RAW) or clock_gettime(CLOCK_MONOTONIC)
>
> wallclock() has 3 implementations:
> * Windows: QueryPerformanceCounter() with QueryPerformanceFrequency(),
> with a fallback to GetSystemTimeAsFileTime() if
> QueryPerformanceFrequency() failed
> * UNIX: clock_gettime(CLOCK_MONOTONIC_RAW),
> clock_gettime(CLOCK_MONOTONIC) or clock_gettime(CLOCK_REALTIME), with
> a fallback to gettimeofday() if clock_gettime(*) failed
> * Otherwise: gettimeofday()
>
> (wallclock should also use mach_absolute_time() on Mac OS X)
>
>
> Victor
> _______________________________________________
> Python-Dev mailing list
> Python-Dev [at] python
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>


--
http://www.voidspace.org.uk/


May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing
http://www.sqlite.org/different.html





_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


nadeem.vawda at gmail

Mar 13, 2012, 5:18 PM

Post #3 of 67 (1965 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

So wallclock() falls back to a not-necessarily-monotonic time source
if necessary,
while monotonic() raises an exception in that case? ISTM that these
don't need to
be separate functions - rather, we can have one function that takes a
flag (called
require_monotonic, or something like that) telling it which failure mode to use.
Does that make sense?

Cheers,
Nadeem
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


guido at python

Mar 13, 2012, 5:27 PM

Post #4 of 67 (1950 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

On Tue, Mar 13, 2012 at 4:57 PM, Victor Stinner
<victor.stinner [at] gmail> wrote:
> I added two functions to the time module in Python 3.3: wallclock()
> and monotonic(). I'm unable to explain the difference between these
> two functions, even if I wrote them :-) wallclock() is suppose to be
> more accurate than time() but has an unspecified starting point.
> monotonic() is similar except that it is monotonic: it cannot go
> backward. monotonic() may not be available or fail whereas wallclock()
> is available/work, but I think that the two functions are redundant.
>
> I prefer to keep only monotonic() because it is not affected by system
> clock update and should help to fix issues on NTP update in functions
> implementing a timeout.
>
> What do you think?

I think wallclock() is an awkward name; in other contexts I've seen
"wall clock time" used to mean the time that a clock on the wall would
show, i.e. local time. This matches definition #1 of
http://www.catb.org/jargon/html/W/wall-time.html (while yours matches
#2 :-).

I agree that it's better to have only one of these. I also think if we
offer it we should always have it -- if none of the implementations
are available, I guess you could fall back on returning time.time(),
with some suitable offset so people don't think it is always the same.
Maybe it could be called realtime()?

--
--Guido van Rossum (python.org/~guido)
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


victor.stinner at gmail

Mar 13, 2012, 5:29 PM

Post #5 of 67 (1950 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

On 14/03/2012 01:18, Nadeem Vawda wrote:
> So wallclock() falls back to a not-necessarily-monotonic time source
> if necessary,
> while monotonic() raises an exception in that case? ISTM that these
> don't need to
> be separate functions - rather, we can have one function that takes a
> flag (called
> require_monotonic, or something like that) telling it which failure mode to use.
> Does that make sense?

I don't think that time.monotonic() can fail in practice and it is
available for all modern platforms (Windows, Mac OS X and OS implemented
clock_gettime()).

On Windows, time.monotonic() fails with an OSError if
QueryPerformanceFrequency() failed. QueryPerformanceFrequency() can fail
if "the installed hardware does not support a high-resolution
performance counter" according to Microsoft documentation. Windows uses
the CPU RDTSC instruction, or the ACPI power management timer or event
the old 8245 PIT. I think that you have at least one of this device on
your computer.

I suppose that you can use a manual fallback to time.time() if
time.monotonic() failed. If time.monotonic() fails, it fails directly at
the first call. Example of a fallback working with Python < 3.3:

try:
time.monotonic()
except (OSError, AttributeError):
get_time = time.time
else:
get_time = time.monotonic

Victor
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


kristjan at ccpgames

Mar 13, 2012, 5:45 PM

Post #6 of 67 (1975 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

The reason I originally suggested "wallclock" was because that term is often used to distinguish time measurements (delta) that show real world time from those showing CPU or Kernel time. "number.crunch() took 2 seconds wallclock time but only 1 second CPU!". The original problem was that time.clock() was "wallclock" on some platforms but "cpu" on others, IIRC.
But monotonic is probably even better. I agree removing one or the other, probably wallclock.
K

-----Original Message-----
From: python-dev-bounces+kristjan=ccpgames.com [at] python [mailto:python-dev-bounces+kristjan=ccpgames.com [at] python] On Behalf Of Guido van Rossum
Sent: 13. mars 2012 17:27
To: Victor Stinner
Cc: Python Dev
Subject: Re: [Python-Dev] Drop the new time.wallclock() function?

I think wallclock() is an awkward name; in other contexts I've seen "wall clock time" used to mean the time that a clock on the wall would show, i.e. local time. This matches definition #1 of http://www.catb.org/jargon/html/W/wall-time.html (while yours matches
#2 :-).

I agree that it's better to have only one of these. I also think if we offer it we should always have it -- if none of the implementations are available, I guess you could fall back on returning time.time(), with some suitable offset so people don't think it is always the same.
Maybe it could be called realtime()?



_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


victor.stinner at gmail

Mar 13, 2012, 6:03 PM

Post #7 of 67 (1946 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

> I agree that it's better to have only one of these. I also think if we
> offer it we should always have it -- if none of the implementations
> are available, I guess you could fall back on returning time.time(),
> with some suitable offset so people don't think it is always the same.
> Maybe it could be called realtime()?

For a concrete use case, see for example:
http://bugs.python.org/issue14222

I just wrote two patches, for the queue and threading modules, using
time.monotonic() if available, with a fallback to time.time(). My
patches call time.monotonic() to ensure that it doesn't fail with
OSError.

I suppose that most libraries and programs will have to implement a
similar fallback.

We may merge both functions with a flag to be able to disable the
fallback. Example:

- time.realtime(): best-effort monotonic, with a fallback
- time.realtime(monotonic=True): monotonic, may raise OSError or
NotImplementedError

Victor
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


tismer at stackless

Mar 13, 2012, 6:06 PM

Post #8 of 67 (1977 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

On 3/13/12 5:45 PM, KristjŠn Valur Jůnsson wrote:
> The reason I originally suggested "wallclock" was because that term is often used to distinguish time measurements (delta) that show real world time from those showing CPU or Kernel time. "number.crunch() took 2 seconds wallclock time but only 1 second CPU!". The original problem was that time.clock() was "wallclock" on some platforms but "cpu" on others, IIRC.
> But monotonic is probably even better. I agree removing one or the other, probably wallclock.
> K
>
> -----Original Message-----
> From: python-dev-bounces+kristjan=ccpgames.com [at] python [mailto:python-dev-bounces+kristjan=ccpgames.com [at] python] On Behalf Of Guido van Rossum
> Sent: 13. mars 2012 17:27
> To: Victor Stinner
> Cc: Python Dev
> Subject: Re: [Python-Dev] Drop the new time.wallclock() function?
>
> I think wallclock() is an awkward name; in other contexts I've seen "wall clock time" used to mean the time that a clock on the wall would show, i.e. local time. This matches definition #1 of http://www.catb.org/jargon/html/W/wall-time.html (while yours matches
> #2 :-).
>
> I agree that it's better to have only one of these. I also think if we offer it we should always have it -- if none of the implementations are available, I guess you could fall back on returning time.time(), with some suitable offset so people don't think it is always the same.
> Maybe it could be called realtime()?
>
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev [at] python
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/tismer%40stackless.com

Btw., have you considered virtual machines?
I happen to run windows in Parallels or Virtualbox quite often.
There the "wall clock" stuff notoriously does not work.

It would be good (but difficult?) if the supposed-to-be-accurate
clock could test itself, if it works at all, and replace itself
with a fallback.

In my case, this causes quite a few PyPy tests to fail ;-)

ciao -- Chris

--
Christian Tismer :^)<mailto:tismer [at] stackless>
tismerysoft GmbH : Have a break! Take a ride on Python's
Karl-Liebknecht-Str. 121 : *Starship* http://starship.python.net/
14482 Potsdam : PGP key -> http://pgp.uni-mainz.de
work +49 173 24 18 776 mobile +49 173 24 18 776 fax n.a.
PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04
whom do you want to sponsor today? http://www.stackless.com/

_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


yselivanov.ml at gmail

Mar 13, 2012, 6:09 PM

Post #9 of 67 (1952 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

If we need to decide to which function should be kept - I vote for monotonic.
It's extremely useful (even essential) to track timeouts in various schedulers
implementations, for example. Quick search also shows the demand for it, as
there are questions on stackoverflow.com and few packages on PyPI.

-
Yury


On 2012-03-13, at 9:03 PM, Victor Stinner wrote:

>> I agree that it's better to have only one of these. I also think if we
>> offer it we should always have it -- if none of the implementations
>> are available, I guess you could fall back on returning time.time(),
>> with some suitable offset so people don't think it is always the same.
>> Maybe it could be called realtime()?
>
> For a concrete use case, see for example:
> http://bugs.python.org/issue14222
>
> I just wrote two patches, for the queue and threading modules, using
> time.monotonic() if available, with a fallback to time.time(). My
> patches call time.monotonic() to ensure that it doesn't fail with
> OSError.
>
> I suppose that most libraries and programs will have to implement a
> similar fallback.
>
> We may merge both functions with a flag to be able to disable the
> fallback. Example:
>
> - time.realtime(): best-effort monotonic, with a fallback
> - time.realtime(monotonic=True): monotonic, may raise OSError or
> NotImplementedError
>
> Victor
> _______________________________________________
> Python-Dev mailing list
> Python-Dev [at] python
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/yselivanov.ml%40gmail.com

_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


nadeem.vawda at gmail

Mar 13, 2012, 6:10 PM

Post #10 of 67 (1952 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

On Wed, Mar 14, 2012 at 3:03 AM, Victor Stinner
<victor.stinner [at] gmail> wrote:
> I suppose that most libraries and programs will have to implement a
> similar fallback.
>
> We may merge both functions with a flag to be able to disable the
> fallback. Example:
>
>  - time.realtime(): best-effort monotonic, with a fallback
>  - time.realtime(monotonic=True): monotonic, may raise OSError or
> NotImplementedError

This was my suggestion - I think it's useful to have the fallback
available (since most users will want it), but at the same time we
should also cater to users who need a clock that is *guaranteed* to
be monotonic.

As an aside, I think "monotonic" is a better name than "realtime";
it conveys the functions purpose more clearly. Then we could call
the flag "strict".

Cheers,
Nadeem
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


kristjan at ccpgames

Mar 13, 2012, 6:11 PM

Post #11 of 67 (1943 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

Interesting thought.
Althougn I don't see how that could fail on windows, if the QPC function is really just talking to a clock chip, surely that hasn't been virtualized.
Is there an actual example of windows hardware where this api fails (virtual or not?) Perhaps there is no real need to have a fallback mechanism, and it would even be best to write such a mechanism inside the function itself, and just return getsystemtimeasfiletime() instead.

K

-----Original Message-----
From: Christian Tismer [mailto:tismer [at] stackless]
Sent: 13. mars 2012 18:07
To: KristjŠn Valur Jůnsson
Cc: Guido van Rossum; Victor Stinner; Python Dev
Subject: Re: [Python-Dev] Drop the new time.wallclock() function?

Btw., have you considered virtual machines?
I happen to run windows in Parallels or Virtualbox quite often.
There the "wall clock" stuff notoriously does not work.

It would be good (but difficult?) if the supposed-to-be-accurate clock could test itself, if it works at all, and replace itself with a fallback.

In my case, this causes quite a few PyPy tests to fail ;-)

ciao -- Chris

--
Christian Tismer :^)<mailto:tismer [at] stackless>
tismerysoft GmbH : Have a break! Take a ride on Python's
Karl-Liebknecht-Str. 121 : *Starship* http://starship.python.net/
14482 Potsdam : PGP key -> http://pgp.uni-mainz.de
work +49 173 24 18 776 mobile +49 173 24 18 776 fax n.a.
PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04
whom do you want to sponsor today? http://www.stackless.com/



_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


andrew.svetlov at gmail

Mar 13, 2012, 6:12 PM

Post #12 of 67 (1950 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

On Tue, Mar 13, 2012 at 5:29 PM, Victor Stinner
<victor.stinner [at] gmail> wrote:
> I suppose that you can use a manual fallback to time.time() if
> time.monotonic() failed. If time.monotonic() fails, it fails directly at the
> first call. Example of a fallback working with Python < 3.3:
>
> try:
>   time.monotonic()
> except (OSError, AttributeError):
>   get_time = time.time
> else:
>   get_time = time.monotonic
>

I like 'fallback' solution while `get_time` is not the best name for
high precision timer from my perspective.
Can you call it `monotonic` or `realtime`?
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


guido at python

Mar 13, 2012, 6:34 PM

Post #13 of 67 (1941 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

On Tue, Mar 13, 2012 at 6:03 PM, Victor Stinner
<victor.stinner [at] gmail> wrote:
>> I agree that it's better to have only one of these. I also think if we
>> offer it we should always have it -- if none of the implementations
>> are available, I guess you could fall back on returning time.time(),
>> with some suitable offset so people don't think it is always the same.
>> Maybe it could be called realtime()?
>
> For a concrete use case, see for example:
> http://bugs.python.org/issue14222
>
> I just wrote two patches, for the queue and threading modules, using
> time.monotonic() if available, with a fallback to time.time(). My
> patches call time.monotonic() to ensure that it doesn't fail with
> OSError.
>
> I suppose that most libraries and programs will have to implement a
> similar fallback.

It seems horrible to force everyone to copy the same silly block of
code. The time module itself should implement this once.

> We may merge both functions with a flag to be able to disable the
> fallback. Example:
>
> †- time.realtime(): best-effort monotonic, with a fallback
> †- time.realtime(monotonic=True): monotonic, may raise OSError or
> NotImplementedError

I have no opinions on this or other API details. But please make the
function always exist and return something vaguely resembling a
monotonic real-time clock. (BTW IMO the docs should state explicitly
that it returns a float.)

--
--Guido van Rossum (python.org/~guido)
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


tismer at stackless

Mar 13, 2012, 10:24 PM

Post #14 of 67 (1940 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

The performancecounter is a thing that typically gets intercepted by
the VM infrastructure and does no longer work as a reliable timing
source. In PyPy there are tests which check certain assumptions
how much the performancecounter must advance at least between
a few opcodes, and that does not work in a VM.

cheers - Chris

On 3/13/12 6:11 PM, KristjŠn Valur Jůnsson wrote:
> Interesting thought.
> Althougn I don't see how that could fail on windows, if the QPC function is really just talking to a clock chip, surely that hasn't been virtualized.
> Is there an actual example of windows hardware where this api fails (virtual or not?) Perhaps there is no real need to have a fallback mechanism, and it would even be best to write such a mechanism inside the function itself, and just return getsystemtimeasfiletime() instead.
>
> K
>
> -----Original Message-----
> From: Christian Tismer [mailto:tismer [at] stackless]
> Sent: 13. mars 2012 18:07
> To: KristjŠn Valur Jůnsson
> Cc: Guido van Rossum; Victor Stinner; Python Dev
> Subject: Re: [Python-Dev] Drop the new time.wallclock() function?
>
> Btw., have you considered virtual machines?
> I happen to run windows in Parallels or Virtualbox quite often.
> There the "wall clock" stuff notoriously does not work.
>
> It would be good (but difficult?) if the supposed-to-be-accurate clock could test itself, if it works at all, and replace itself with a fallback.
>
> In my case, this causes quite a few PyPy tests to fail ;-)
>
> ciao -- Chris
>


--
Christian Tismer :^)<mailto:tismer [at] stackless>
tismerysoft GmbH : Have a break! Take a ride on Python's
Karl-Liebknecht-Str. 121 : *Starship* http://starship.python.net/
14482 Potsdam : PGP key -> http://pgp.uni-mainz.de
work +49 173 24 18 776 mobile +49 173 24 18 776 fax n.a.
PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04
whom do you want to sponsor today? http://www.stackless.com/

_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


jyasskin at gmail

Mar 13, 2012, 10:42 PM

Post #15 of 67 (1937 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

On Tue, Mar 13, 2012 at 5:03 PM, Michael Foord
<fuzzyman [at] voidspace> wrote:
>
> On 13 Mar 2012, at 16:57, Victor Stinner wrote:
>
>> Hi,
>>
>> I added two functions to the time module in Python 3.3: wallclock()
>> and monotonic(). I'm unable to explain the difference between these
>> two functions, even if I wrote them :-) wallclock() is suppose to be
>> more accurate than time() but has an unspecified starting point.
>> monotonic() is similar except that it is monotonic: it cannot go
>> backward. monotonic() may not be available or fail whereas wallclock()
>> is available/work, but I think that the two functions are redundant.
>>
>> I prefer to keep only monotonic() because it is not affected by system
>> clock update and should help to fix issues on NTP update in functions
>> implementing a timeout.
>>
>> What do you think?
>>
>
>
> I am in the middle of adding a feature to unittest that involves timing of individual tests. I want the highest resolution cross platform measure of wallclock time - and time.wallclock() looked ideal. If monotonic may not exist or can fail why would that be better?
>

Isn't the highest resolution cross platform measure of "wallclock"
time spelled "time.clock()"? Its docs say "this is the function to use
for benchmarking Python or timing algorithms", and it would be a shame
to add and teach a new function rather than improving clock()'s
definition.

Jeffrey
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


jyasskin at gmail

Mar 13, 2012, 11:16 PM

Post #16 of 67 (1938 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

On Tue, Mar 13, 2012 at 6:10 PM, Nadeem Vawda <nadeem.vawda [at] gmail> wrote:
> On Wed, Mar 14, 2012 at 3:03 AM, Victor Stinner
> <victor.stinner [at] gmail> wrote:
>> I suppose that most libraries and programs will have to implement a
>> similar fallback.
>>
>> We may merge both functions with a flag to be able to disable the
>> fallback. Example:
>>
>>  - time.realtime(): best-effort monotonic, with a fallback
>>  - time.realtime(monotonic=True): monotonic, may raise OSError or
>> NotImplementedError
>
> This was my suggestion - I think it's useful to have the fallback
> available (since most users will want it), but at the same time we
> should also cater to users who need a clock that is *guaranteed* to
> be monotonic.
>
> As an aside, I think "monotonic" is a better name than "realtime";
> it conveys the functions purpose more clearly. Then we could call
> the flag "strict".

While you're bikeshedding:

Some of the drafts of the new C++ standard had a monotonic_clock,
which was guaranteed to only go forwards, but which could be affected
by system clock updates that went forwards. Because of problems in
defining timeouts using an adjustable clock, C++11 instead defines a
"steady_clock", which ticks as steadily as the machine/OS/library can
ensure, and is definitely not affected by any time adjustments:
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2010/n3191.htm. I
realize that Unix's clock_gettime(CLOCK_MONOTONIC) already includes
the steadiness criterion, but the word itself doesn't actually include
the meaning.
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


kristjan at ccpgames

Mar 13, 2012, 11:32 PM

Post #17 of 67 (1955 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

To quote:
"On Unix, return the current processor time as a floating point number expressed in seconds. The precision, and in fact the very definition of the meaning of "processor time", depends on that of the C function of the same name,"

The problem is that it is defined to return "processor time." This is historical baggage that comes from just writing a python wrapper around the unix "clock" function. Of course, "processor time" is quite useless when one is trying to write timeout algorithms or other such things that need to time out in real time, not just cpu cycles.
K

-----Original Message-----
From: python-dev-bounces+kristjan=ccpgames.com [at] python [mailto:python-dev-bounces+kristjan=ccpgames.com [at] python] On Behalf Of Jeffrey Yasskin
Sent: 13. mars 2012 22:42
To: Michael Foord
Cc: Python Dev
Subject: Re: [Python-Dev] Drop the new time.wallclock() function?

Isn't the highest resolution cross platform measure of "wallclock"
time spelled "time.clock()"? Its docs say "this is the function to use for benchmarking Python or timing algorithms", and it would be a shame to add and teach a new function rather than improving clock()'s definition.

Jeffrey
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/kristjan%40ccpgames.com


_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


solipsis at pitrou

Mar 14, 2012, 2:16 AM

Post #18 of 67 (1931 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

On Wed, 14 Mar 2012 02:03:42 +0100
Victor Stinner <victor.stinner [at] gmail> wrote:
>
> We may merge both functions with a flag to be able to disable the
> fallback. Example:
>
> - time.realtime(): best-effort monotonic, with a fallback
> - time.realtime(monotonic=True): monotonic, may raise OSError or
> NotImplementedError

That's a rather awful name. time.time() is *the* real time.

time.monotonic(fallback=False) would be a better API.

Regards

Antoine.


_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


stefan at bytereef

Mar 14, 2012, 2:30 AM

Post #19 of 67 (1928 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

Antoine Pitrou <solipsis [at] pitrou> wrote:
> time.monotonic(fallback=False) would be a better API.

+1


Stefan Krah


_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


victor.stinner at gmail

Mar 14, 2012, 5:27 AM

Post #20 of 67 (1934 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

2012/3/14 Antoine Pitrou <solipsis [at] pitrou>:
> On Wed, 14 Mar 2012 02:03:42 +0100
> Victor Stinner <victor.stinner [at] gmail> wrote:
>>
>> We may merge both functions with a flag to be able to disable the
>> fallback. Example:
>>
>>  - time.realtime(): best-effort monotonic, with a fallback
>>  - time.realtime(monotonic=True): monotonic, may raise OSError or
>> NotImplementedError
>
> That's a rather awful name.  time.time() is *the* real time.
>
> time.monotonic(fallback=False) would be a better API.

I would prefer to enable the fallback by default with a warning in the
doc, just because it is more convinient and it is what user want even
if they don't know that they need a fallback :-)

Enabling the fallback by default allow to write such simple code:

try:
from time import monotonic as get_time
except ImportError:
# Python < 3.3
from time import time as get_time

Use time.monotonic(strict=True) if you need a truly monotonic clock.

monotonic() may not be the best name in this case. Jeffrey Yasskin
proposed time.steady_clock(), so time.steady_clock(monotonic=False)?

Victor
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


solipsis at pitrou

Mar 14, 2012, 5:27 AM

Post #21 of 67 (1917 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

On Wed, 14 Mar 2012 13:27:19 +0100
Victor Stinner <victor.stinner [at] gmail> wrote:
>
> monotonic() may not be the best name in this case. Jeffrey Yasskin
> proposed time.steady_clock(), so time.steady_clock(monotonic=False)?

I don't know what "steady" is supposed to mean here, so perhaps the
best solution is to improve the doc?

Also, "monotonic=False" implies that it won't be monotonic, which is
false.

Regards

Antoine.
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


anacrolix at gmail

Mar 14, 2012, 9:11 AM

Post #22 of 67 (1909 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

I have some observations regarding this:

Victor's existing time.monotonic and time.wallclock make use of
QueryPerformanceCounter, and CLOCK_MONOTONIC_RAW as possible. Both of
these are hardware-based counters, their monotonicity is just a
convenient property of the timer sources. Furthermore, time values can
actually vary depending on the processor the call is served on.
time.hardware()? time.monotonic_raw()?

There are bug reports on Linux that CLOCK_MONOTONIC isn't always
monotonic. This is why CLOCK_MONOTONIC_RAW was created. There's also
the issue of time leaps (forward), which also isn't a problem with the
latter form. time.monotonic(raw_only=False)?

The real value of "monotonic" timers is that they don't leap
backwards, and preferably don't leap forwards. Whether they are
absolute is of no consequence. I would suggest that the API reflect
this, and that more specific time values be obtained using the proper
raw syscall wrapper (like time.clock_gettime) if required.
time.relative(strict=False)?

The ultimate use of the function name being established is for use in
timeouts and relative timings.

Where an option is present, it disallows fallbacks like
CLOCK_MONOTONIC and other weaker forms:
* time.hardware(fallback=True) -> reflects the source of the timing
impeccably. alerts users to possible affinity issues
* time.monotonic_raw() -> a bit linux specific...
* time.relative(strict=False) -> matches the use case. a warning
could be put regarding hardware timings
* time.monotonic(raw_only=False) -> closest to the existing
implementation. the keyword name i think is better.
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


guido at python

Mar 14, 2012, 9:22 AM

Post #23 of 67 (1933 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

I have a totally different observation. Presumably the primary use
case for these timers is to measure real time intervals for the
purpose of profiling various operations. For this purpose we want them
to be as "steady" as possible: tick at a constant rate, don't jump
forward or backward. (And they shouldn't invoke the concept of "CPU"
time -- we already have time.clock() for that, besides it's often
wrong, e.g. you might be measuring some sort of I/O performance.) If
this means that a second as measured by time.time() is sometimes not
the same as a second measured by this timer (due to time.time()
occasionally jumping due to clock adjustments), that's fine with me.
If this means it's unreliable inside a VM, well, it seems that's a
property of the underlying OS timer, and there's not much we can do
about it except letting a knowledgeable user override the timer user.
As for names, I like Jeffrey's idea of having "steady" in the name.

--Guido

On Wed, Mar 14, 2012 at 9:11 AM, Matt Joiner <anacrolix [at] gmail> wrote:
> I have some observations regarding this:
>
> Victor's existing time.monotonic and time.wallclock make use of
> QueryPerformanceCounter, and CLOCK_MONOTONIC_RAW as possible. Both of
> these are hardware-based counters, their monotonicity is just a
> convenient property of the timer sources. Furthermore, time values can
> actually vary depending on the processor the call is served on.
> time.hardware()? time.monotonic_raw()?
>
> There are bug reports on Linux that CLOCK_MONOTONIC isn't always
> monotonic. This is why CLOCK_MONOTONIC_RAW was created. There's also
> the issue of time leaps (forward), which also isn't a problem with the
> latter form. time.monotonic(raw_only=False)?
>
> The real value of "monotonic" timers is that they don't leap
> backwards, and preferably don't leap forwards. Whether they are
> absolute is of no consequence. I would suggest that the API reflect
> this, and that more specific time values be obtained using the proper
> raw syscall wrapper (like time.clock_gettime) if required.
> time.relative(strict=False)?
>
> The ultimate use of the function name being established is for use in
> timeouts and relative timings.
>
> Where an option is present, it disallows fallbacks like
> CLOCK_MONOTONIC and other weaker forms:
> †* time.hardware(fallback=True) -> reflects the source of the timing
> impeccably. alerts users to possible affinity issues
> †* time.monotonic_raw() -> a bit linux specific...
> †* time.relative(strict=False) -> matches the use case. a warning
> could be put regarding hardware timings
> †* time.monotonic(raw_only=False) -> closest to the existing
> implementation. the keyword name i think is better.



--
--Guido van Rossum (python.org/~guido)
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


kristjan at ccpgames

Mar 14, 2012, 9:42 AM

Post #24 of 67 (1916 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

Yes, the intended use is relative timings and timeouts.
I think we are complicating things far too much.
1) Do we really need a fallback on windows? Will QPC ever fail?
2) is it a problem for the intended use if we cannot absolutely guarantee that time won't ever tick backwards?

IMHO, we shouldn't compilicate the api, and make whatever best try we can in C. On windows I would do this (pseudocode)

Static last_time = 0
If (QPC_works) time = QueryPerformanceCounter() else time = GetSystemTimeAsFileTime()
if (time > last_time) last_time=time else time = last_time
return time

in other words:
1) use QPC. If the api indicates that it isn't available (does this ever happen in real life?) use a fallback of system time
2) enforce monotonicity with a static. QPC, if the OS doesn"t is buggy (calls cpu counter rather than timer chip) can cause jitter because it is called on different cores.

No options in the api. No nothing. We simply provide the best api possible and some hardware/software combos might be less accurate.

K

-----Original Message-----
From: python-dev-bounces+kristjan=ccpgames.com [at] python [mailto:python-dev-bounces+kristjan=ccpgames.com [at] python] On Behalf Of Matt Joiner
Sent: 14. mars 2012 09:12
To: Antoine Pitrou; Victor Stinner; Guido van Rossum
Cc: python-dev [at] python
Subject: Re: [Python-Dev] Drop the new time.wallclock() function?

I have some observations regarding this:

Victor's existing time.monotonic and time.wallclock make use of QueryPerformanceCounter, and CLOCK_MONOTONIC_RAW as possible. Both of these are hardware-based counters, their monotonicity is just a convenient property of the timer sources. Furthermore, time values can actually vary depending on the processor the call is served on.
time.hardware()? time.monotonic_raw()?

There are bug reports on Linux that CLOCK_MONOTONIC isn't always monotonic. This is why CLOCK_MONOTONIC_RAW was created. There's also the issue of time leaps (forward), which also isn't a problem with the latter form. time.monotonic(raw_only=False)?

The real value of "monotonic" timers is that they don't leap backwards, and preferably don't leap forwards. Whether they are absolute is of no consequence. I would suggest that the API reflect this, and that more specific time values be obtained using the proper raw syscall wrapper (like time.clock_gettime) if required.
time.relative(strict=False)?

The ultimate use of the function name being established is for use in timeouts and relative timings.

Where an option is present, it disallows fallbacks like CLOCK_MONOTONIC and other weaker forms:
* time.hardware(fallback=True) -> reflects the source of the timing impeccably. alerts users to possible affinity issues
* time.monotonic_raw() -> a bit linux specific...
* time.relative(strict=False) -> matches the use case. a warning could be put regarding hardware timings
* time.monotonic(raw_only=False) -> closest to the existing implementation. the keyword name i think is better.
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/kristjan%40ccpgames.com


_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


nadeem.vawda at gmail

Mar 14, 2012, 9:47 AM

Post #25 of 67 (1913 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

A summary of the discussion so far, as I've understood it:

- We should have *one* monotonic/steady timer function, using the
sources described in Victor's original post.

- By default, it should fall back to time.time if a better source is
not available, but there should be a flag that can disable this
fallback for users who really *need* a monotonic/steady time source.

- Proposed names for the function:
* monotonic
* steady_clock
* wallclock
* realtime

- Proposed names for the flag controlling fallback behavior:
* strict (=False)
* fallback (=True)
* monotonic (=False)


For the function name, I think monotonic() and steady_clock() convey
the purpose of the function much better than the other two; the term
"wallclock" is actively misleading, and "realtime" seems ambiguous.

For the flag name, I'm -1 on "monotonic" -- it sounds like a flag to
decide whether to use a monotonic time source always or never, while
it actually decides between "always" and "sometimes". I think "strict"
is nicer than "fallback", but I'm fine with either one.

Cheers,
Nadeem
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


anacrolix at gmail

Mar 14, 2012, 10:08 AM

Post #26 of 67 (1859 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

On Thu, Mar 15, 2012 at 12:22 AM, Guido van Rossum <guido [at] python> wrote:
> I have a totally different observation. Presumably the primary use
> case for these timers is to measure real time intervals for the
> purpose of profiling various operations. For this purpose we want them
> to be as "steady" as possible: tick at a constant rate, don't jump
> forward or backward. (And they shouldn't invoke the concept of "CPU"
> time -- we already have time.clock() for that, besides it's often
> wrong, e.g. you might be measuring some sort of I/O performance.) If
> this means that a second as measured by time.time() is sometimes not
> the same as a second measured by this timer (due to time.time()
> occasionally jumping due to clock adjustments), that's fine with me.
> If this means it's unreliable inside a VM, well, it seems that's a
> property of the underlying OS timer, and there's not much we can do
> about it except letting a knowledgeable user override the timer user.
> As for names, I like Jeffrey's idea of having "steady" in the name.

In that case I'd suggest either time.hardware(strict=True), or
time.steady(strict=True), since the only timers exposed natively that
are both high resolution and steady are on the hardware. A warning
about CPU affinity is also still wise methinks.
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


kristjan at ccpgames

Mar 14, 2012, 10:09 AM

Post #27 of 67 (1874 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

> - By default, it should fall back to time.time if a better source is
> not available, but there should be a flag that can disable this
> fallback for users who really *need* a monotonic/steady time source.
As pointed out on a different thread, you don"t need this "flag" since the code can easily enforce the monotonic property by maintaining a static value.
This is how we worked around buggy implementations of QueryPerformanceCounter on windows ().
K


-----Original Message-----
From: python-dev-bounces+kristjan=ccpgames.com [at] python [mailto:python-dev-bounces+kristjan=ccpgames.com [at] python] On Behalf Of Nadeem Vawda
Sent: 14. mars 2012 09:47
To: Guido van Rossum
Cc: Antoine Pitrou; python-dev [at] python
Subject: Re: [Python-Dev] Drop the new time.wallclock() function?

A summary of the discussion so far, as I've understood it:

- We should have *one* monotonic/steady timer function, using the
sources described in Victor's original post.

- By default, it should fall back to time.time if a better source is
not available, but there should be a flag that can disable this
fallback for users who really *need* a monotonic/steady time source.

- Proposed names for the function:
* monotonic
* steady_clock
* wallclock
* realtime

- Proposed names for the flag controlling fallback behavior:
* strict (=False)
* fallback (=True)
* monotonic (=False)


For the function name, I think monotonic() and steady_clock() convey the purpose of the function much better than the other two; the term "wallclock" is actively misleading, and "realtime" seems ambiguous.

For the flag name, I'm -1 on "monotonic" -- it sounds like a flag to decide whether to use a monotonic time source always or never, while it actually decides between "always" and "sometimes". I think "strict"
is nicer than "fallback", but I'm fine with either one.

Cheers,
Nadeem
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/kristjan%40ccpgames.com


_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


anacrolix at gmail

Mar 14, 2012, 10:15 AM

Post #28 of 67 (1863 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

FWIW the name is quite important, because these kind of timings are
quite important so I think it's worth the effort.

> - By default, it should fall back to time.time if a better source is
>  not available, but there should be a flag that can disable this
>  fallback for users who really *need* a monotonic/steady time source.

Agreed. As Guido mentioned, some platforms might not be able to access
to hardware times, so falling back should be the default, lest unaware
users trigger unexpected errors.

> - Proposed names for the function:
>  * monotonic

Doesn't indicate that the timing is also prevented from leaping forward.

>  * steady_clock

I think the use of clock might infer CPU time on doc-skimming user.
"clock" is overloaded here.

> For the flag name, I'm -1 on "monotonic" -- it sounds like a flag to
> decide whether to use a monotonic time source always or never, while
> it actually decides between "always" and "sometimes". I think "strict"
> is nicer than "fallback", but I'm fine with either one.

I agree, "strict" fits in with existing APIs.

I think time.hardware(), and time.steady() are still okay here.
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


guido at python

Mar 14, 2012, 10:21 AM

Post #29 of 67 (1876 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

+1 for steady().

On Wed, Mar 14, 2012 at 10:15 AM, Matt Joiner <anacrolix [at] gmail> wrote:
> FWIW the name is quite important, because these kind of timings are
> quite important so I think it's worth the effort.
>
>> - By default, it should fall back to time.time if a better source is
>> †not available, but there should be a flag that can disable this
>> †fallback for users who really *need* a monotonic/steady time source.
>
> Agreed. As Guido mentioned, some platforms might not be able to access
> to hardware times, so falling back should be the default, lest unaware
> users trigger unexpected errors.
>
>> - Proposed names for the function:
>> †* monotonic
>
> Doesn't indicate that the timing is also prevented from leaping forward.
>
>> †* steady_clock
>
> I think the use of clock might infer CPU time on doc-skimming user.
> "clock" is overloaded here.
>
>> For the flag name, I'm -1 on "monotonic" -- it sounds like a flag to
>> decide whether to use a monotonic time source always or never, while
>> it actually decides between "always" and "sometimes". I think "strict"
>> is nicer than "fallback", but I'm fine with either one.
>
> I agree, "strict" fits in with existing APIs.
>
> I think time.hardware(), and time.steady() are still okay here.



--
--Guido van Rossum (python.org/~guido)
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


andrew.svetlov at gmail

Mar 14, 2012, 10:39 AM

Post #30 of 67 (1883 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

On Wed, Mar 14, 2012 at 10:21 AM, Guido van Rossum <guido [at] python> wrote:
> +1 for steady().
>

+1
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


regebro at gmail

Mar 14, 2012, 10:42 AM

Post #31 of 67 (1863 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

On Wed, Mar 14, 2012 at 10:16, Antoine Pitrou <solipsis [at] pitrou> wrote:
> That's a rather awful name. †time.time() is *the* real time.
>
> time.monotonic(fallback=False) would be a better API.

I think calling the function "monotonic" isn't really a good name if
it's not always monotonic.

time.monotonic(fallback=False)

Really just means

time.monotonic(monotonic=False)

And

time.monotonic(strict=True)

Really means

time.monotonic(i_really_mean_it=True)

This is potentially confusing. Therefore

time.clock()
time.time()
time.realtime()
time.wallclock()

Are all better options if there is a flag to switch if it's monotonic or not.

Since time.clock() apparently can mean different things on different
platforms because of it's use of the C-API, we can't use that.
I'm not sure why time.time() would differ from time.realtime().
time.time() is per definition not monotonic, but

time.time(monotonic=True)

is maybe a possibility?


//Lennart
//Lennart
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


regebro at gmail

Mar 14, 2012, 11:07 AM

Post #32 of 67 (1880 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

2012/3/14 KristjŠn Valur Jůnsson <kristjan [at] ccpgames>:
>> - By default, it should fall back to time.time if a better source is
>> †not available, but there should be a flag that can disable this
>> †fallback for users who really *need* a monotonic/steady time source.
> As pointed out on a different thread, you don"t need this "flag" since the code can easily enforce the monotonic property by maintaining a static value.

With this, I think time.steady() would be clear and nice.

//Lennart
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


anacrolix at gmail

Mar 14, 2012, 11:47 AM

Post #33 of 67 (1865 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

I also can live with steady, with strict for the flag.


nadeem.vawda at gmail

Mar 14, 2012, 2:17 PM

Post #34 of 67 (1883 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

+1 for time.steady(strict=False).


On Wed, Mar 14, 2012 at 7:09 PM, Kristján Valur Jónsson
<kristjan [at] ccpgames> wrote:
>> - By default, it should fall back to time.time if a better source is
>>  not available, but there should be a flag that can disable this
>>  fallback for users who really *need* a monotonic/steady time source.
> As pointed out on a different thread, you don"t need this "flag" since the code can easily enforce the monotonic property by maintaining a static value.
> This is how we worked around buggy implementations of QueryPerformanceCounter on windows ().
> K

That's fine if you just need the clock to be monotonic, but it isn't
any help if you also want to prevent it from jumping forward.

Cheers,
Nadeem
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


cs at zip

Mar 14, 2012, 4:38 PM

Post #35 of 67 (1864 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

On 13Mar2012 17:27, Guido van Rossum <guido [at] python> wrote:
| I think wallclock() is an awkward name; in other contexts I've seen
| "wall clock time" used to mean the time that a clock on the wall would
| show, i.e. local time. This matches definition #1 of
| http://www.catb.org/jargon/html/W/wall-time.html (while yours matches
| #2 :-).

I think this also. A "wallclock()" function that did not return real world
elapsed time seconds would be misleading or at least disconcerting.

| Maybe it could be called realtime()?

"elapsedtime()?" It is getting a bit long though.

Cheers,
--
Cameron Simpson <cs [at] zip> DoD#743
http://www.cskk.ezoshosting.com/cs/

"Shot my dog today."
"Was he mad?"
"Well, he weren't too damned pleased."
- Rick Tilson, rtilson [at] Sun
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


kristjan at ccpgames

Mar 14, 2012, 5:28 PM

Post #36 of 67 (1870 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

What does "jumping forward" mean? That's what happens with every clock at every time quantum. The only effect here is that this clock will be slightly noisy, i.e. its precision becomes worse. On average it is still correct. Look at the use cases for this function
1) to enable timeouts for certaing operations, like acquiring locks:
Jumping backwards is bad, because that may cause infinite wait time. But jumping forwards is ok, it may just mean that your lock times out a bit early
2) performance measurements:
If you are running on a platform with a broken runtime clock, you are not likely to be running performance measurements.

Really, I urge you to skip the "strict" keyword. It just adds confusion. Instead, lets just give the best monotonic clock we can do which doesn"t move backwards.
Let's just provide a "practical" real time clock with high resolution that is appropriate for providing timeout functionality and so won't jump backwards for the next 20 years. Let's simply point out to people that it may not be appropriate for high precision timings on old and obsolete hardware and be done with it.

K

-----Original Message-----
From: python-dev-bounces+kristjan=ccpgames.com [at] python [mailto:python-dev-bounces+kristjan=ccpgames.com [at] python] On Behalf Of Nadeem Vawda
Sent: 14. mars 2012 14:18
To: Matt Joiner
Cc: Antoine Pitrou; Guido van Rossum; python-dev [at] python
Subject: Re: [Python-Dev] Drop the new time.wallclock() function?

+1 for time.steady(strict=False).


On Wed, Mar 14, 2012 at 7:09 PM, Kristján Valur Jónsson <kristjan [at] ccpgames> wrote:
>> - By default, it should fall back to time.time if a better source is
>>  not available, but there should be a flag that can disable this
>>  fallback for users who really *need* a monotonic/steady time source.
> As pointed out on a different thread, you don"t need this "flag" since the code can easily enforce the monotonic property by maintaining a static value.
> This is how we worked around buggy implementations of QueryPerformanceCounter on windows ().
> K

That's fine if you just need the clock to be monotonic, but it isn't any help if you also want to prevent it from jumping forward.

Cheers,
Nadeem
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/kristjan%40ccpgames.com
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


victor.stinner at gmail

Mar 14, 2012, 5:29 PM

Post #37 of 67 (1859 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

On 14/03/2012 00:57, Victor Stinner wrote:
> I added two functions to the time module in Python 3.3: wallclock()
> and monotonic(). (...)

I merged the two functions into one function: time.steady(strict=False).

time.steady() should be monotonic most of the time, but may use a fallback.

time.steady(strict=True) fails with OSError or NotImplementedError if
reading the monotonic clock failed or if no monotonic clock is available.

I patched the queue and threading modules to use time.steady() instead
of time.time().

The documentation may need clarification.
http://docs.python.org/dev/library/time.html#time.steady

Victor
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


victor.stinner at gmail

Mar 14, 2012, 5:56 PM

Post #38 of 67 (1858 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

> I merged the two functions into one function: time.steady(strict=False).

I opened the issue #14309 to deprecate time.clock():
http://bugs.python.org/issue14309

time.clock() is a different clock type depending on the OS (Windows vs
UNIX) and so is confusing. You should now decide between time.time()
and time.steady().

time.time():

- known starting point, Epoch (1970.1.1)
- may be update by the system (and so go backward or forward)

=> display time to the user, compare/set file modification time

time.steady():

- unknown starting point
- more accurate than time.time()
- should be monotonic (use strict=True if you want to be sure ;-))

=> benchmark, timeout

Victor
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


anacrolix at gmail

Mar 14, 2012, 6:58 PM

Post #39 of 67 (1841 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

Victor, I think that steady can always be monotonic, there are time sources
enough to ensure this on the platforms I am aware of. Strict in this sense
refers to not being adjusted forward, i.e. CLOCK_MONOTONIC vs
CLOCK_MONOTONIC_RAW.

Non monotonicity of this call should be considered a bug. Strict would be
used for profiling where forward leaps would disqualify the timing.


regebro at gmail

Mar 14, 2012, 9:44 PM

Post #40 of 67 (1857 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

On Thu, Mar 15, 2012 at 02:58, Matt Joiner <anacrolix [at] gmail> wrote:
> Victor, I think that steady can always be monotonic, there are time sources
> enough to ensure this on the platforms I am aware of. Strict in this sense
> refers to not being adjusted forward, i.e. CLOCK_MONOTONIC vs
> CLOCK_MONOTONIC_RAW.
>
> Non monotonicity of this call should be considered a bug. Strict would be
> used for profiling where forward leaps would disqualify the timing.

This makes sense to me.

//Lennart
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


p.f.moore at gmail

Mar 15, 2012, 1:23 AM

Post #41 of 67 (1848 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

On 15 March 2012 01:58, Matt Joiner <anacrolix [at] gmail> wrote:
> Victor, I think that steady can always be monotonic, there are time sources
> enough to ensure this on the platforms I am aware of. Strict in this sense
> refers to not being adjusted forward, i.e. CLOCK_MONOTONIC vs
> CLOCK_MONOTONIC_RAW.

I agree - KristjŠn pointed out that you can ensure that backward jumps
never occur by implementing a cache of the last value.

> Non monotonicity of this call should be considered a bug.

+1

> Strict would be used for profiling where forward leaps would disqualify the timing.

I'm baffled as to how you even identify "forward leaps". In relation
to what? A more accurate time source? I thought that by definition
this was the most accurate time source we have!

+1 on a simple time.steady() with guaranteed monotonicity and no flags
to alter behaviour.

Paul.
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


victor.stinner at gmail

Mar 15, 2012, 2:29 AM

Post #42 of 67 (1854 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

2012/3/15 Matt Joiner <anacrolix [at] gmail>:
> Victor, I think that steady can always be monotonic, there are time sources
> enough to ensure this on the platforms I am aware of. Strict in this sense
> refers to not being adjusted forward, i.e. CLOCK_MONOTONIC vs
> CLOCK_MONOTONIC_RAW.

I don't think that CLOCK_MONOTONIC is available on all platforms.
clock_gettime() and QueryPerformanceFrequency() can fail. In practice,
it should not fail on modern OSses.

But if the monotonic clock fails, Python should use another less
stable clock but provide something. Otherwise, each project would have
to implement its own fallback.

Victor
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


anacrolix at gmail

Mar 15, 2012, 3:06 AM

Post #43 of 67 (1849 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

On Mar 15, 2012 4:23 PM, "Paul Moore" <p.f.moore [at] gmail> wrote:
>
> On 15 March 2012 01:58, Matt Joiner <anacrolix [at] gmail> wrote:
> > Victor, I think that steady can always be monotonic, there are time
sources
> > enough to ensure this on the platforms I am aware of. Strict in this
sense
> > refers to not being adjusted forward, i.e. CLOCK_MONOTONIC vs
> > CLOCK_MONOTONIC_RAW.
>
> I agree - Kristj√°n pointed out that you can ensure that backward jumps
> never occur by implementing a cache of the last value.

Without knowing more, either QPC was buggy on his platform, or he didn't
account for processor affinity (QPC derives from a per processor counter).

>
> > Non monotonicity of this call should be considered a bug.
>
> +1
>
> > Strict would be used for profiling where forward leaps would disqualify
the timing.
>
> I'm baffled as to how you even identify "forward leaps". In relation
> to what? A more accurate time source? I thought that by definition
> this was the most accurate time source we have!

Monotonic clocks are not necessarily hardware based, and may be adjusted
forward by NTP.

>
> +1 on a simple time.steady() with guaranteed monotonicity and no flags
> to alter behaviour.
>
> Paul.

I don't mind since I'll be using it for timeouts, but clearly the strongest
possible guarantee should be made. If forward leaps are okay, then by
definition the timer is monotonic but not steady.


p.f.moore at gmail

Mar 15, 2012, 4:10 AM

Post #44 of 67 (1851 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

On 15 March 2012 10:06, Matt Joiner <anacrolix [at] gmail> wrote:
>> I'm baffled as to how you even identify "forward leaps". In relation
>> to what? A more accurate time source? I thought that by definition
>> this was the most accurate time source we have!
>
> Monotonic clocks are not necessarily hardware based, and may be adjusted
> forward by NTP.

I appreciate that. But I'm still unclear how you would tell that had
happened as part of the implementation. One call to the OS returns
12345. The next returns 13345. Is that because 100 ticks have passed,
or because the clock "leapt forward"? With no point of reference, how
can you tell?

But I agree, the key thing is just to have the strongest guarantee possible.

Paul.
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


nadeem.vawda at gmail

Mar 15, 2012, 5:12 AM

Post #45 of 67 (1850 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

On Thu, Mar 15, 2012 at 1:10 PM, Paul Moore <p.f.moore [at] gmail> wrote:
>> Monotonic clocks are not necessarily hardware based, and may be adjusted
>> forward by NTP.
>
> I appreciate that. But I'm still unclear how you would tell that had
> happened as part of the implementation. One call to the OS returns
> 12345. The next returns 13345. Is that because 100 ticks have passed,
> or because the clock "leapt forward"? With no point of reference, how
> can you tell?

The point (AIUI) is that you *can't* identify such adjustments (in the
absence of some sort of log of NTP updates), so we should provide a
mechanism that is guaranteed not to be affected by them.

Cheers,
Nadeem
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


p.f.moore at gmail

Mar 15, 2012, 5:20 AM

Post #46 of 67 (1849 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

On 15 March 2012 12:12, Nadeem Vawda <nadeem.vawda [at] gmail> wrote:
> On Thu, Mar 15, 2012 at 1:10 PM, Paul Moore <p.f.moore [at] gmail> wrote:
>> I appreciate that. But I'm still unclear how you would tell that had
>> happened as part of the implementation. One call to the OS returns
>> 12345. The next returns 13345. Is that because 100 ticks have passed,
>> or because the clock "leapt forward"? With no point of reference, how
>> can you tell?
>
> The point (AIUI) is that you *can't* identify such adjustments (in the
> absence of some sort of log of NTP updates), so we should provide a
> mechanism that is guaranteed not to be affected by them.

OK, I see (sort of). But if that is the case, what's the use case for
the variation that *is* affected by them? The use cases I've seen
mentioned are timeouts and performance testing, both of which don't
want to see clock adjustments.

Note that when Victor started this thread, he said:

> I prefer to keep only monotonic() because it is not affected by system
> clock update and should help to fix issues on NTP update in functions
> implementing a timeout.

which seems to me to be exactly this point. So I guess I support
Victor's original proposal. (Which is good, because he has thought
about this issue far more than I have :-))

Paul.
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


rowen at uw

Mar 15, 2012, 12:22 PM

Post #47 of 67 (1837 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

In article
<EFE3877620384242A686D52278B7CCD3362609 [at] RKV-IT-EXCH104>,
Kristján Valur Jónsson <kristjan [at] ccpgames> wrote:

> What does "jumping forward" mean? That's what happens with every clock at
> every time quantum. The only effect here is that this clock will be slightly
> noisy, i.e. its precision becomes worse. On average it is still correct.
> Look at the use cases for this function
> 1) to enable timeouts for certaing operations, like acquiring locks:
> Jumping backwards is bad, because that may cause infinite wait time. But
> jumping forwards is ok, it may just mean that your lock times out a bit early
> 2) performance measurements:
> If you are running on a platform with a broken runtime clock, you are not
> likely to be running performance measurements.
>
> Really, I urge you to skip the "strict" keyword. It just adds confusion.
> Instead, lets just give the best monotonic clock we can do which doesn"t move
> backwards.
> Let's just provide a "practical" real time clock with high resolution that is
> appropriate for providing timeout functionality and so won't jump backwards
> for the next 20 years. Let's simply point out to people that it may not be
> appropriate for high precision timings on old and obsolete hardware and be
> done with it.

I agree. I prefer the name time.monotonic with no flags. It will suit
most use cases. I think supplying truly steady time is a low level
hardware function (e.g. buy a GPS timer card) with a driver.

-- Russell


anacrolix at gmail

Mar 15, 2012, 12:55 PM

Post #48 of 67 (1845 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

+1. I now prefer time.monotonic(), no flags. It attempts to be as high
precision as possible and guarantees never to jump backwards. Russell's
comment is right, the only steady sources are from hardware, and these are
too equipment and operating system specific. (For this call anyway).
On Mar 16, 2012 3:23 AM, "Russell E. Owen" <rowen [at] uw> wrote:

> In article
> <EFE3877620384242A686D52278B7CCD3362609 [at] RKV-IT-EXCH104>,
> Kristján Valur Jónsson <kristjan [at] ccpgames> wrote:
>
> > What does "jumping forward" mean? That's what happens with every clock
> at
> > every time quantum. The only effect here is that this clock will be
> slightly
> > noisy, i.e. its precision becomes worse. On average it is still correct.
> > Look at the use cases for this function
> > 1) to enable timeouts for certaing operations, like acquiring locks:
> > Jumping backwards is bad, because that may cause infinite wait
> time. But
> > jumping forwards is ok, it may just mean that your lock times out a bit
> early
> > 2) performance measurements:
> > If you are running on a platform with a broken runtime clock, you
> are not
> > likely to be running performance measurements.
> >
> > Really, I urge you to skip the "strict" keyword. It just adds confusion.
> > Instead, lets just give the best monotonic clock we can do which doesn"t
> move
> > backwards.
> > Let's just provide a "practical" real time clock with high resolution
> that is
> > appropriate for providing timeout functionality and so won't jump
> backwards
> > for the next 20 years. Let's simply point out to people that it may not
> be
> > appropriate for high precision timings on old and obsolete hardware and
> be
> > done with it.
>
> I agree. I prefer the name time.monotonic with no flags. It will suit
> most use cases. I think supplying truly steady time is a low level
> hardware function (e.g. buy a GPS timer card) with a driver.
>
> -- Russell
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev [at] python
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/anacrolix%40gmail.com
>
>


alexander.belopolsky at gmail

Mar 15, 2012, 2:27 PM

Post #49 of 67 (1838 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

On Thu, Mar 15, 2012 at 3:55 PM, Matt Joiner <anacrolix [at] gmail> wrote:
> +1. I now prefer time.monotonic(), no flags.

Am I alone thinking that an adjective is an odd choice for a function
name? I think monotonic_clock or monotonic_time would be a better
option.
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


tjreedy at udel

Mar 15, 2012, 3:45 PM

Post #50 of 67 (1844 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

On 3/15/2012 5:27 PM, Alexander Belopolsky wrote:
> On Thu, Mar 15, 2012 at 3:55 PM, Matt Joiner<anacrolix [at] gmail> wrote:
>> +1. I now prefer time.monotonic(), no flags.
>
> Am I alone thinking that an adjective is an odd choice for a function
> name?

I would normally agree, but in this case, it is a function of a module
whose short name names what the adjective is modifying. I expect that
this will normally be called with the module name.

> I think monotonic_clock or monotonic_time would be a better option.

time.monotonic_time seems redundant.

--
Terry Jan Reedy

_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


steve at pearwood

Mar 15, 2012, 4:18 PM

Post #51 of 67 (817 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

Terry Reedy wrote:
> On 3/15/2012 5:27 PM, Alexander Belopolsky wrote:
>> On Thu, Mar 15, 2012 at 3:55 PM, Matt Joiner<anacrolix [at] gmail> wrote:
>>> +1. I now prefer time.monotonic(), no flags.
>>
>> Am I alone thinking that an adjective is an odd choice for a function
>> name?
>
> I would normally agree, but in this case, it is a function of a module
> whose short name names what the adjective is modifying. I expect that
> this will normally be called with the module name.
>
>> I think monotonic_clock or monotonic_time would be a better option.
>
> time.monotonic_time seems redundant.

Agreed. Same applies to "steady_time", and "steady" on its own is weird.
Steady what?

While we're bike-shedding, I'll toss in another alternative. Early Apple
Macintoshes had a system function that returned the time since last reboot
measured in 1/60th of a second, called "the ticks".

If I have understood correctly, the monotonic timer will have similar
properties: guaranteed monotonic, as accurate as the hardware can provide, but
not directly translatable to real (wall-clock) time. (Wall clocks sometimes go
backwards.)

The two functions are not quite identical: Mac "ticks" were 32-bit integers,
not floating point numbers. But the use-cases seem to be the same.

time.ticks() seems right as a name to me. It suggests a steady heartbeat
ticking along, without making any suggestion that it returns "the time".



--
Steven

_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


anacrolix at gmail

Mar 15, 2012, 5:50 PM

Post #52 of 67 (825 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

Windows also has this albeit course grained and also 32 bit. I don't think
ticks reflects the reason why using the timer is desirable.

monotonic_time seems reasonable, there's no reason to persist short names
when users can import it how they like.
On Mar 16, 2012 7:20 AM, "Steven D&apos;Aprano" <steve [at] pearwood> wrote:

> Terry Reedy wrote:
>
>> On 3/15/2012 5:27 PM, Alexander Belopolsky wrote:
>>
>>> On Thu, Mar 15, 2012 at 3:55 PM, Matt Joiner<anacrolix [at] gmail>
>>> wrote:
>>>
>>>> +1. I now prefer time.monotonic(), no flags.
>>>>
>>>
>>> Am I alone thinking that an adjective is an odd choice for a function
>>> name?
>>>
>>
>> I would normally agree, but in this case, it is a function of a module
>> whose short name names what the adjective is modifying. I expect that this
>> will normally be called with the module name.
>>
>> I think monotonic_clock or monotonic_time would be a better option.
>>>
>>
>> time.monotonic_time seems redundant.
>>
>
> Agreed. Same applies to "steady_time", and "steady" on its own is weird.
> Steady what?
>
> While we're bike-shedding, I'll toss in another alternative. Early Apple
> Macintoshes had a system function that returned the time since last reboot
> measured in 1/60th of a second, called "the ticks".
>
> If I have understood correctly, the monotonic timer will have similar
> properties: guaranteed monotonic, as accurate as the hardware can provide,
> but not directly translatable to real (wall-clock) time. (Wall clocks
> sometimes go backwards.)
>
> The two functions are not quite identical: Mac "ticks" were 32-bit
> integers, not floating point numbers. But the use-cases seem to be the same.
>
> time.ticks() seems right as a name to me. It suggests a steady heartbeat
> ticking along, without making any suggestion that it returns "the time".
>
>
>
> --
> Steven
>
> ______________________________**_________________
> Python-Dev mailing list
> Python-Dev [at] python
> http://mail.python.org/**mailman/listinfo/python-dev<http://mail.python.org/mailman/listinfo/python-dev>
> Unsubscribe: http://mail.python.org/**mailman/options/python-dev/**
> anacrolix%40gmail.com<http://mail.python.org/mailman/options/python-dev/anacrolix%40gmail.com>
>


kristjan at ccpgames

Mar 19, 2012, 2:07 AM

Post #53 of 67 (794 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

Do you really want to add an obscure Boolean flag to the function just so that python can warn you that perhaps your platform is so old and so weird that Python can't guarantee that the performance measurements are to a certain _undefined_ quality?

Please note, that the function makes no claims to the resolution or precision of the timer involved. Only that it moves only forward. It is therefore completely and utterly redundant to add a "strict" value, because we would only be behave "strictly" according to an _undefined specification_.

K

-----Original Message-----
From: python-dev-bounces+kristjan=ccpgames.com [at] python [mailto:python-dev-bounces+kristjan=ccpgames.com [at] python] On Behalf Of Lennart Regebro
Sent: 15. mars 2012 04:44
To: Matt Joiner
Cc: Python Dev
Subject: Re: [Python-Dev] Drop the new time.wallclock() function?

On Thu, Mar 15, 2012 at 02:58, Matt Joiner <anacrolix [at] gmail> wrote:
> Victor, I think that steady can always be monotonic, there are time
> sources enough to ensure this on the platforms I am aware of. Strict
> in this sense refers to not being adjusted forward, i.e.
> CLOCK_MONOTONIC vs CLOCK_MONOTONIC_RAW.
>
> Non monotonicity of this call should be considered a bug. Strict would
> be used for profiling where forward leaps would disqualify the timing.

This makes sense to me.

//Lennart
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/kristjan%40ccpgames.com


_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


zooko at zooko

Mar 23, 2012, 9:55 AM

Post #54 of 67 (812 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

> I merged the two functions into one function: time.steady(strict=False).
>
> time.steady() should be monotonic most of the time, but may use a fallback.
>
> time.steady(strict=True) fails with OSError or NotImplementedError if
> reading the monotonic clock failed or if no monotonic clock is available.

If someone wants time.steady(strict=False), then why don't they just
continue to use time.time()?

I want time.steady(strict=True), and I'm glad you're providing it and
I'm willing to use it this way, although it is slightly annoying
because "time.steady(strict=True)" really means
"time.steady(i_really_mean_it=True)". Else, I would have used
"time.time()".

I am aware of a large number of use cases for a steady clock (event
scheduling, profiling, timeouts), and a large number of uses cases for
a "NTP-respecting wall clock" clock (calendaring, displaying to a
user, timestamping). I'm not aware of any use case for "steady if
implemented, else wall-clock", and it sounds like a mistake to me.

Regards,

Zooko
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


victor.stinner at gmail

Mar 23, 2012, 10:27 AM

Post #55 of 67 (786 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

> I want time.steady(strict=True), and I'm glad you're providing it and
> I'm willing to use it this way, although it is slightly annoying
> because "time.steady(strict=True)" really means
> "time.steady(i_really_mean_it=True)". Else, I would have used
> "time.time()".
>
> I am aware of a large number of use cases for a steady clock (event
> scheduling, profiling, timeouts), and a large number of uses cases for
> a "NTP-respecting wall clock" clock (calendaring, displaying to a
> user, timestamping). I'm not aware of any use case for "steady if
> implemented, else wall-clock", and it sounds like a mistake to me.

time.steady(strict=False) is what you need to implement timeout.

If you use time.steady(strict=True) for timeout, it means that you
cannot use select, threads, etc. if your platform doesn't provide
monotonic clock, whereas it works "well" (except the issue of adjusted
time) with Python < 3.3.

Victor
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


glyph at twistedmatrix

Mar 23, 2012, 1:40 PM

Post #56 of 67 (813 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

On Mar 23, 2012, at 12:55 PM, Zooko Wilcox-O'Hearn wrote:

>> I merged the two functions into one function: time.steady(strict=False).
>>
>> time.steady() should be monotonic most of the time, but may use a fallback.
>>
>> time.steady(strict=True) fails with OSError or NotImplementedError if
>> reading the monotonic clock failed or if no monotonic clock is available.
>
> If someone wants time.steady(strict=False), then why don't they just
> continue to use time.time()?
>
> I want time.steady(strict=True), and I'm glad you're providing it and
> I'm willing to use it this way, although it is slightly annoying
> because "time.steady(strict=True)" really means
> "time.steady(i_really_mean_it=True)". Else, I would have used
> "time.time()".
>
> I am aware of a large number of use cases for a steady clock (event
> scheduling, profiling, timeouts), and a large number of uses cases for
> a "NTP-respecting wall clock" clock (calendaring, displaying to a
> user, timestamping). I'm not aware of any use case for "steady if
> implemented, else wall-clock", and it sounds like a mistake to me.

I think I've lost the thread of this discussion. Is that really what "strict=False" was supposed to mean?

I am aware of use-cases which want to respect slew, but reject steps. The local clock might not be all that reliable, and slew actually keeps it closer to "real" time. My understanding was that strict=True was something like CLOCK_MONOTONIC_RAW and strict=False was just CLOCK_MONOTONIC.

I am increasingly thinking that the first order of business here should be to expose the platform-specific mechanisms directly, then try to come up with a unifying abstraction in the time module later. It's hard enough to understand the substantially dense verbiage around all of these different timers on their respective platforms; understanding which one exactly Python is swaddling up in a portability layer seems bound to generate confusion. Not to mention that you run into awesome little edge cases like this: https://github.com/ThomasHabets/monotonic_clock/blob/master/src/monotonic_win32.c#L62 which means sometimes you really really need to know exactly which clock is getting used, if you want to make it work right (unless Python is going to ship with all of these workarounds on day 1).

(I still object to the "time.steady" naming, because this is what people in the make-believe land of C++ call it. The people who live in the real world of C and POSIX all refer to it as "monotonic". And even the C++ purists slip up sometimes, c.f. <http://en.cppreference.com/w/cpp/chrono/steady_clock>: "Class std::chrono::steady_clock represents a monotonic clock.")

If there really are some applications for which the desired behavior is 'monotonic clock, but if you can't get one, nevermind, wallclock is good enough', it strikes me that this should be as explicit as possible, it should not be the default, and if

try:
value = time.steady()
except OhWowYourComputerReallyHasProblems:
value = time.time()

is generally considered too onerous, it should be spelled time.steady(fallback=time.time).

-glyph


p.f.moore at gmail

Mar 23, 2012, 2:11 PM

Post #57 of 67 (802 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

On 23 March 2012 20:40, Glyph <glyph [at] twistedmatrix> wrote:
> I am increasingly thinking that the first order of business here should be
> to expose the platform-specific mechanisms directly, then try to come up
> with a unifying abstraction in the time module later.

+1.

Paul
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


yselivanov.ml at gmail

Mar 23, 2012, 3:54 PM

Post #58 of 67 (813 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

On 2012-03-23, at 4:40 PM, Glyph wrote:
> (I still object to the "time.steady" naming, because this is what people in the make-believe land of C++ call it. The people who live in the real world of C and POSIX all refer to it as "monotonic". And even the C++ purists slip up sometimes, c.f. <http://en.cppreference.com/w/cpp/chrono/steady_clock>: "Class std::chrono::steady_clock represents a monotonic clock.")

+1.

I also think that the function should be called 'monotonic' and simply fail with OSError on platforms that don't support such clocks. The 'strict' argument is non-intuitive.

-
Yury
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


yselivanov.ml at gmail

Mar 23, 2012, 3:56 PM

Post #59 of 67 (815 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

On 2012-03-23, at 1:27 PM, Victor Stinner wrote:

>> I want time.steady(strict=True), and I'm glad you're providing it and
>> I'm willing to use it this way, although it is slightly annoying
>> because "time.steady(strict=True)" really means
>> "time.steady(i_really_mean_it=True)". Else, I would have used
>> "time.time()".
>>
>> I am aware of a large number of use cases for a steady clock (event
>> scheduling, profiling, timeouts), and a large number of uses cases for
>> a "NTP-respecting wall clock" clock (calendaring, displaying to a
>> user, timestamping). I'm not aware of any use case for "steady if
>> implemented, else wall-clock", and it sounds like a mistake to me.
>
> time.steady(strict=False) is what you need to implement timeout.
>
> If you use time.steady(strict=True) for timeout, it means that you
> cannot use select, threads, etc. if your platform doesn't provide
> monotonic clock, whereas it works "well" (except the issue of adjusted
> time) with Python < 3.3.

Why can't I use select & threads? You mean that if a platform does not
support monotonic clocks it also does not support threads and select sys
call?

-
Yury
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


anacrolix at gmail

Mar 23, 2012, 4:06 PM

Post #60 of 67 (784 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

Yes, call it what it is. monotonic or monotonic_time, because that's why
I'm using it. No flags.

I've followed this thread throughout, and I'm still not sure if "steady"
gives the real guarantees it claims. It's trying to be too much. Existing
bugs complain about backward jumps and demand a clock that doesn't do this.
The function should guarantee monotonicity only, and not get
overcomplicated.


victor.stinner at gmail

Mar 23, 2012, 4:07 PM

Post #61 of 67 (797 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

2012/3/23 Yury Selivanov <yselivanov.ml [at] gmail>:
> Why can't I use select & threads?  You mean that if a platform does not
> support monotonic clocks it also does not support threads and select sys
> call?

Python 3.3 now uses time.steady(strict=False) in the threading and
queue modules. If we replace it by time.steady(strict=True), you may
get an error if your platform doesn't provide a monotonic clock and so
you cannot use these modules.

Victor
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


yselivanov.ml at gmail

Mar 23, 2012, 4:21 PM

Post #62 of 67 (780 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

On 2012-03-23, at 7:07 PM, Victor Stinner wrote:

> 2012/3/23 Yury Selivanov <yselivanov.ml [at] gmail>:
>> Why can't I use select & threads? You mean that if a platform does not
>> support monotonic clocks it also does not support threads and select sys
>> call?
>
> Python 3.3 now uses time.steady(strict=False) in the threading and
> queue modules. If we replace it by time.steady(strict=True), you may
> get an error if your platform doesn't provide a monotonic clock and so
> you cannot use these modules.

Why this won't work?

try:
from time import monotonic as _time
except ImportError:
from time import time as _time

OR (if we decide to fail on first call, instead of ImportError)

import time
try:
time.monotonic()
except OSError:
_time = time
else:
_time = time.monotonic

And then just use '_time' in your code? What's the deal with the
'strict' kwarg?

I really like how it currently works with epoll, for instance. It either
exists in the 'select' module, or not, if the host OS doesn't support it.
I think it should be the same for 'time.monotonic'.

-
Yury

_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


zooko at zooko

Mar 26, 2012, 3:31 PM

Post #63 of 67 (856 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

On Fri, Mar 23, 2012 at 11:27 AM, Victor Stinner
<victor.stinner [at] gmail> wrote:
>
> time.steady(strict=False) is what you need to implement timeout.

No, that doesn't fit my requirements, which are about event
scheduling, profiling, and timeouts. See below for more about my
requirements.

I didn't say this explicitly enough in my previous post:

Some use cases (timeouts, event scheduling, profiling, sensing)
require a steady clock. Others (calendaring, communicating times to
users, generating times for comparison to remote hosts) require a wall
clock.

Now here's the kicker: each use case incur significant risks if it
uses the wrong kind of clock.

If you're implementing event scheduling or sensing and control, and
you accidentally get a wall clock when you thought you had a steady
clock, then your program may go seriously wrong -- events may fire in
the wrong order, measurements of your sensors may be wildly incorrect.
This can lead to serious accidents. On the other hand, if you're
implementing calendaring or display of "real local time of day" to a
user, and you are using a steady clock for some reason, then you risk
displaying incorrect results to the user.

So using one kind of clock and then "falling back" to the other kind
is a choice that should be rare, explicit, and discouraged. The
provision of such a function in the standard library is an attractive
nuisance -- a thing that people naturally think that they want when
they haven't though about it very carefully, but that is actually
dangerous.

If someone has a use case which fits the "steady or else fall back to
wall clock" pattern, I would like to learn about it.

Regards,

Zooko
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


glyph at twistedmatrix

Mar 26, 2012, 3:47 PM

Post #64 of 67 (784 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

On Mar 26, 2012, at 6:31 PM, Zooko Wilcox-O'Hearn wrote:

> On Fri, Mar 23, 2012 at 11:27 AM, Victor Stinner
> <victor.stinner [at] gmail> wrote:
>>
>> time.steady(strict=False) is what you need to implement timeout.
>
> No, that doesn't fit my requirements, which are about event
> scheduling, profiling, and timeouts. See below for more about my
> requirements.
>
> I didn't say this explicitly enough in my previous post:
>
> Some use cases (timeouts, event scheduling, profiling, sensing)
> require a steady clock. Others (calendaring, communicating times to
> users, generating times for comparison to remote hosts) require a wall
> clock.
>
> Now here's the kicker: each use case incur significant risks if it
> uses the wrong kind of clock.
>
> If you're implementing event scheduling or sensing and control, and
> you accidentally get a wall clock when you thought you had a steady
> clock, then your program may go seriously wrong -- events may fire in
> the wrong order, measurements of your sensors may be wildly incorrect.
> This can lead to serious accidents. On the other hand, if you're
> implementing calendaring or display of "real local time of day" to a
> user, and you are using a steady clock for some reason, then you risk
> displaying incorrect results to the user.
>
> So using one kind of clock and then "falling back" to the other kind
> is a choice that should be rare, explicit, and discouraged. The
> provision of such a function in the standard library is an attractive
> nuisance -- a thing that people naturally think that they want when
> they haven't though about it very carefully, but that is actually
> dangerous.
>
> If someone has a use case which fits the "steady or else fall back to
> wall clock" pattern, I would like to learn about it.

I feel that this should be emphasized. Zooko knows what he's talking about here. Listen to him :). (Antoine has the right idea. I think it's well past time for a PEP on this feature.)

-glyph

_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


victor.stinner at gmail

Mar 26, 2012, 4:07 PM

Post #65 of 67 (796 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

> So using one kind of clock and then "falling back" to the other kind
> is a choice that should be rare, explicit, and discouraged. The
> provision of such a function in the standard library is an attractive
> nuisance -- a thing that people naturally think that they want when
> they haven't though about it very carefully, but that is actually
> dangerous.
>
> If someone has a use case which fits the "steady or else fall back to
> wall clock" pattern, I would like to learn about it.

Python 3.2 doesn't provide a monotonic clock, so most program uses
time.time() even if a monotonic clock would be better in some
functions. For these programs, you can replace time.time() by
time.steady() where you need to compute a time delta (e.g. compute a
timeout) to avoid issues with the system clock update. The idea is to
improve the program without refusing to start if no monotonic clock is
available.

Victor
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


zooko at zooko

Mar 26, 2012, 7:41 PM

Post #66 of 67 (796 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

On Mon, Mar 26, 2012 at 5:07 PM, Victor Stinner
<victor.stinner [at] gmail> wrote:
>>
>> If someone has a use case which fits the "steady or else fall back to wall clock" pattern, I would like to learn about it.
>
> Python 3.2 doesn't provide a monotonic clock, so most program uses time.time() even if a monotonic clock would be better in some functions. For these programs, you can replace time.time() by time.steady() where you need to compute a time delta (e.g. compute a timeout) to avoid issues with the system clock update. The idea is to improve the program without refusing to start if no monotonic clock is available.

I agree that this is a reasonable use case. I think of it as basically
being a kind of backward-compatibility, for situations where an
unsteady clock is okay, and a steady clock isn't available. Twisted
faces a similar issue:

http://twistedmatrix.com/trac/ticket/2424

It might good for use cases like this to explicitly implement the
try-and-fallback, since they might have specific needs about how it is
done. For one thing, some such uses may need to emit a warning, or
even to require the caller to explicitly override, such a refusing to
start if a steady clock isn't available unless the user specifies
"--unsteady-clock-ok".

For motivating examples, consider software written using Twisted >
12.0 or Python > 3.2 which is using a clock to drive real world
sensing and control -- measuring the position of a machine and using
time deltas to calculate the machine's velocity, in order to
automatically control the motion of the machine. For some uses, it is
okay if the measurement could, in rare cases, be drastically wrong.
For other uses, that is not an acceptable risk.

One reason I'm sensitive to this issue is that I work in the field of
security, and making the behavior dependent on the system clock
extends the "reliance set", i.e. the set of things that an attacker
could use against you. For example, if your robot depends on the
system clock for its sensing and control, and if your system clock
obeys NTP, then the set of things that an attacker could use against
you includes your NTP servers. If your robot depends instead on a
steady clock, then NTP servers are not in the reliance set.

Now, if your control platform doesn't have a steady clock, you may
choose to go ahead, while making sure that the NTP servers are
authenticated, or you may choose to disable NTP on the control
platform, etc., but that choice might need to be made explicitly by
the operator, rather than automatically by the library.

Regards,

Zooko
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


anacrolix at gmail

Mar 26, 2012, 7:55 PM

Post #67 of 67 (774 views)
Permalink
Re: Drop the new time.wallclock() function? [In reply to]

Inside time.steady, there are two different clocks trying to get out.

I think this steady business should be removed sooner rather than later.

Python dev RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.