Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Python: Dev

this is why we shouldn't call it a "monotonic clock" (was: PEP 418 is too divisive and confusing and should be postponed)

 

 

First page Previous page 1 2 Next page Last page  View All Python dev RSS feed   Index | Next | Previous | View Threaded


cs at zip

Apr 8, 2012, 7:54 PM

Post #26 of 32 (347 views)
Permalink
Re: this is why we shouldn't call it a "monotonic clock" (was: PEP 418 is too divisive and confusing and should be postponed) [In reply to]

On 09Apr2012 02:00, Victor Stinner <victor.stinner [at] gmail> wrote:
| > I personally have a need for one potentially different clock -- to
| > measure short intervals for benchmarks and profiling. This might be
| > called time.performancetimer()?
|
| I deferred this topic because it is unclear to me if such timer has to
| count elapsed time during a sleep or not. For example, time.clock()
| does on UNIX, whereas it doesn't on Windows. You may need two clocks
| for this:
| * time.perf_counter(): high-resolution timer for benchmarking, count
| time elasped during a sleep

For POSIX, sounds like CLOCK_MONOTONIC_RAW to me.

| * time.process_time(): High-resolution (?) per-process timer from the
| CPU. (other possible names: time.process_cpu_time() or
| time.cpu_time())

POSIX offers CLOCK_PROCESS_CPUTIME_ID and CLOCK_THREAD_CPUTIME_ID that
seem to suit this need, depending on your threading situation (and what
you're measuring).

| On Windows, GetProcessTimes() has not a "high-resolution": it has a
| accuracy of 1 ms in the best case.

This page:
http://msdn.microsoft.com/en-us/library/windows/desktop/ms683223%28v=vs.85%29.aspx
says "100-nanosecond time units".

Am I going to the wrong place to learn about these functions?
--
Cameron Simpson <cs [at] zip> DoD#743
http://www.cskk.ezoshosting.com/cs/

I distrust a research person who is always obviously busy on a task.
- Robert Frosch, VP, GM Research
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


guido at python

Apr 8, 2012, 9:14 PM

Post #27 of 32 (350 views)
Permalink
Re: this is why we shouldn't call it a "monotonic clock" (was: PEP 418 is too divisive and confusing and should be postponed) [In reply to]

On Sun, Apr 8, 2012 at 5:00 PM, Victor Stinner <victor.stinner [at] gmail> wrote:
>> IOW "What's good enough for sleep() is good enough for
>> user-implemented timeouts and scheduling." as a way to reach at least
>> one decision for a platform with agreed-upon cross-platform
>> characteristics that are useful.
>
> sleep() is implemented in the kernel. The kernel is notified when a
> clock is set, and so can choose how to handle time adjustement. Most
> "sleeping" functions use the system clock but don't care of clock
> adjustement.

We're going around in circles. I'm not asking what sleep does, I want
on principle a timer that does the same thing as sleep(), regardless
of how sleep() works. So if on some OS sleep() uses the same algorithm
as CLOCK_MONOTONIC_RAW, I want my timer to use that too. But if on
some other OS sleep() uses CLOCK_MONOTONIC, I want my timer there to
use that. And if on some OS sleep() is buggy and uses the time-of-day
clock, well, I wouldn't mind if my timer used the same thing.

>> I personally have a need for one potentially different clock -- to
>> measure short intervals for benchmarks and profiling. This might be
>> called time.performancetimer()?
>
> I deferred this topic because it is unclear to me if such timer has to
> count elapsed time during a sleep or not. For example, time.clock()
> does on UNIX, whereas it doesn't on Windows.

I will declare that that was a mistake in clock(), but one that's too
late to fix, because fixing it would break too many programs (those on
*nix that use it to measure CPU time, and those on Windows that use it
to measure real time).

>You may need two clocks
> for this:
> * time.perf_counter(): high-resolution timer for benchmarking, count
> time elasped during a sleep
> * time.process_time(): High-resolution (?) per-process timer from the
> CPU. (other possible names: time.process_cpu_time() or
> time.cpu_time())

TBH I don't need another timer that measures CPU time (not even on
Windows). In a sense, measuring CPU time is a relic from the age of
mainframes and timesharing, where CPU time was the most precious
resource (and in some cases the unit in which other resources were
expressed for accounting purposes). In modern days, it's much more
likely that the time you're measuring is somehow related to how long a
use has to wait for some result (e.g. web response times) and here
"wait time" is just as real as CPU time.

> On Windows, GetProcessTimes() has not a "high-resolution": it has a
> accuracy of 1 ms in the best case. QueryPerformanceCounter() counts
> time elapsed during a sleep, I don't know for GetProcessTimes.

--
--Guido van Rossum (python.org/~guido)
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


victor.stinner at gmail

Apr 9, 2012, 4:24 AM

Post #28 of 32 (349 views)
Permalink
Re: this is why we shouldn't call it a "monotonic clock" (was: PEP 418 is too divisive and confusing and should be postponed) [In reply to]

2012/4/9 Guido van Rossum <guido [at] python>:
>>You may need two clocks
>> for this:
>>  * time.perf_counter(): high-resolution timer for benchmarking, count
>> time elasped during a sleep
>>  * time.process_time(): High-resolution (?) per-process timer from the
>> CPU. (other possible names: time.process_cpu_time() or
>> time.cpu_time())
>
> TBH I don't need another timer that measures CPU time (not even on
> Windows). In a sense, measuring CPU time is a relic from the age of
> mainframes and timesharing, where CPU time was the most precious
> resource (and in some cases the unit in which other resources were
> expressed for accounting purposes). In modern days, it's much more
> likely that the time you're measuring is somehow related to how long a
> use has to wait for some result (e.g. web response times) and here
> "wait time" is just as real as CPU time.

Ah. In this case, my initial proposition is correct. I re-added the pseudo-code:
http://www.python.org/dev/peps/pep-0418/#deferred-api-time-perf-counter

Use QueryPerformanceCounter(), or time.monotonic() or time.time().

Victor
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


victor.stinner at gmail

Apr 9, 2012, 4:26 AM

Post #29 of 32 (349 views)
Permalink
Re: this is why we shouldn't call it a "monotonic clock" (was: PEP 418 is too divisive and confusing and should be postponed) [In reply to]

> |  * time.process_time(): High-resolution (?) per-process timer from the
> | CPU. (other possible names: time.process_cpu_time() or
> | time.cpu_time())
>
> POSIX offers CLOCK_PROCESS_CPUTIME_ID and CLOCK_THREAD_CPUTIME_ID that
> seem to suit this need, depending on your threading situation (and what
> you're measuring).

Yep.

> | On Windows, GetProcessTimes() has not a "high-resolution": it has a
> | accuracy of 1 ms in the best case.
>
> This page:
>  http://msdn.microsoft.com/en-us/library/windows/desktop/ms683223%28v=vs.85%29.aspx
> says "100-nanosecond time units".
>
> Am I going to the wrong place to learn about these functions?

Yes, the resolution is 100 ns, but the accuracy is only 1 ms in the
best case (but it usually 15 ms or 10 ms).

Resolution != accuracy, and only accuracy matters :-)
http://www.python.org/dev/peps/pep-0418/#resolution

Victor
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


cs at zip

Apr 9, 2012, 3:26 PM

Post #30 of 32 (348 views)
Permalink
Re: this is why we shouldn't call it a "monotonic clock" (was: PEP 418 is too divisive and confusing and should be postponed) [In reply to]

On 09Apr2012 13:26, Victor Stinner <victor.stinner [at] gmail> wrote:
| > | On Windows, GetProcessTimes() has not a "high-resolution": it has a
| > | accuracy of 1 ms in the best case.
| >
| > This page:
| >  http://msdn.microsoft.com/en-us/library/windows/desktop/ms683223%28v=vs.85%29.aspx
| > says "100-nanosecond time units".
| >
| > Am I going to the wrong place to learn about these functions?
|
| Yes, the resolution is 100 ns, but the accuracy is only 1 ms in the
| best case (but it usually 15 ms or 10 ms).

I understand the difference, but I can't see mention of the accuracy on
the cited page, hence my question as to whether I'm looking in the right
place. I need to mark up clocks with their accuracy (I've got their
resolution:-)

| Resolution != accuracy, and only accuracy matters :-)
| http://www.python.org/dev/peps/pep-0418/#resolution

I agree. But finding the accuracy seems harder than one would like.
--
Cameron Simpson <cs [at] zip> DoD#743
http://www.cskk.ezoshosting.com/cs/

Thomas R. Collins<brimisha [at] ix> wrote
> This is NOT alt.peeves, as I previously suspected, but
>alt.talk-about-what-you-want-but-sooner-or-later-you'll-get-flamed.

alt.peeves "as you suspected" doesn't exist and never has. The _real_
alt.peeves is, and for at least the past six years has been, the
literate and flamminiferous counterpart of alt.flame and the refined
and brutal alternative to alt.tasteless.
- Charlie Stross <charlie [at] antipope>, educating a newbie
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


victor.stinner at gmail

Apr 9, 2012, 4:33 PM

Post #31 of 32 (352 views)
Permalink
Re: this is why we shouldn't call it a "monotonic clock" (was: PEP 418 is too divisive and confusing and should be postponed) [In reply to]

>> sleep() is implemented in the kernel. The kernel is notified when a
>> clock is set, and so can choose how to handle time adjustement. Most
>> "sleeping" functions use the system clock but don't care of clock
>> adjustement.
>
> We're going around in circles. I'm not asking what sleep does, I want
> on principle a timer that does the same thing as sleep(), regardless
> of how sleep() works. So if on some OS sleep() uses the same algorithm
> as CLOCK_MONOTONIC_RAW, I want my timer to use that too. But if on
> some other OS sleep() uses CLOCK_MONOTONIC, I want my timer there to
> use that. And if on some OS sleep() is buggy and uses the time-of-day
> clock, well, I wouldn't mind if my timer used the same thing.

sleep() takes a number of seconds as argument, so CLOCK_MONOTONIC
should be used, not CLOCK_MONOTONIC_RAW. If I understood correctly,
the unit of CLOCK_MONOTONIC is a second, whereas CLOCK_MONOTONIC_RAW
may be faster or slower than a second.

It looks like CLOCK_MONOTONIC_RAW was added to write a NTP server in
user-space. Extract of the mail including the patch adding
CLOCK_MONOTONIC_RAW to the Linux kernel:
"In talking with Josip Loncaric, and his work on clock synchronization
(see btime.sf.net), he mentioned that for really close synchronization,
it is useful to have access to "hardware time", that is a notion of time
that is not in any way adjusted by the clock slewing done to keep close
time sync.

Part of the issue is if we are using the kernel's ntp adjusted
representation of time in order to measure how we should correct time,
we can run into what Paul McKenney aptly described as "Painting a road
using the lines we're painting as the guide".

I had been thinking of a similar problem, and was trying to come up with
a way to give users access to a purely hardware based time
representation that avoided users having to know the underlying
frequency and mask values needed to deal with the wide variety of
possible underlying hardware counters."
https://lkml.org/lkml/2008/3/19/260

Victor
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


victor.stinner at gmail

Apr 10, 2012, 4:06 PM

Post #32 of 32 (342 views)
Permalink
Re: this is why we shouldn't call it a "monotonic clock" (was: PEP 418 is too divisive and confusing and should be postponed) [In reply to]

>> We're going around in circles. I'm not asking what sleep does, I want
>> on principle a timer that does the same thing as sleep(), regardless
>> of how sleep() works. So if on some OS sleep() uses the same algorithm
>> as CLOCK_MONOTONIC_RAW, I want my timer to use that too. But if on
>> some other OS sleep() uses CLOCK_MONOTONIC, I want my timer there to
>> use that. And if on some OS sleep() is buggy and uses the time-of-day
>> clock, well, I wouldn't mind if my timer used the same thing.
>
> sleep() takes a number of seconds as argument, so CLOCK_MONOTONIC
> should be used, not CLOCK_MONOTONIC_RAW. If I understood correctly,
> the unit of CLOCK_MONOTONIC is a second, whereas CLOCK_MONOTONIC_RAW
> may be faster or slower than a second.

sleep() is not affected by system clock update on any OS: I tested
Linux, FreeBSD, Mac OS X and OpenIndiana.

By the way, CLOCK_BOOTTIME was added to Linux 2.6.39: it includes time
elapsed during system suspend, whereas CLOCK_MONOTONIC doesn't include
time elapsed during system suspend. I updated the "Monotonic clocks"
table to indicate if the clock includes the elapsed time or not.
http://www.python.org/dev/peps/pep-0418/#monotonic-clocks

Victor
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

First page Previous page 1 2 Next page Last page  View All Python dev RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.