[Bug libstdc++/63400] [C++11]precision of std::chrono::high_resolution_clock

[Bug libstdc++/63400] [C++11]precision of std::chrono::high_resolution_clock

[redi at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63400

Jonathan Wakely <redi at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Target|                            |mingw-w64

--- Comment #1 from Jonathan Wakely <redi at gcc dot gnu.org> ---
Which of macros are defined on mingw-w64?

Preprocessing this should tell you

#include <chrono>
#ifdef _GLIBCXX_USE_CLOCK_REALTIME
#ifdef _GLIBCXX_USE_CLOCK_GETTIME_SYSCALL
#warning system_clock using syscall(SYS_clock_gettime, CLOCK_REALTIME, &tp);
#else
#warning system_clock using clock_gettime(CLOCK_REALTIME, &tp);
#endif
#elif defined(_GLIBCXX_USE_GETTIMEOFDAY)
#warning system_clock using gettimeofday(&tv, 0);
#else
#warning system_clock using std::time(0);
#endif

#ifdef _GLIBCXX_USE_CLOCK_MONOTONIC
#ifdef _GLIBCXX_USE_CLOCK_GETTIME_SYSCALL
#warning steady_clock using syscall(SYS_clock_gettime, CLOCK_MONOTONIC, &tp);
#else
#warning steady_clock using clock_gettime(CLOCK_MONOTONIC, &tp);
#endif
#else
#warning steady_clock using time_point(system_clock::now().time_since_epoch());
#endif

[Bug libstdc++/63400] [C++11]precision of std::chrono::high_resolution_clock

[redi at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63400

--- Comment #3 from Jonathan Wakely <redi at gcc dot gnu.org> ---
What libstdc++ is doing is sensible, why is the realtime clock so much coarser
than the monotonic clock on mingw-w64?

[Bug libstdc++/63400] [C++11]precision of std::chrono::high_resolution_clock

[frankhb1989 at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63400

--- Comment #4 from frankhb1989 at gmail dot com ---
(In reply to Jonathan Wakely from comment #3)
> What libstdc++ is doing is sensible, why is the realtime clock so much
> coarser than the monotonic clock on mingw-w64?

It is not always true that the real time clock would have a higher resolution
than monotonic clock. At least I found nothing about resolution guaranteed by
POSIX, in fact, "clock resolutions are implementation-defined". 

With mingw-w64, the following program shows that the monotonic clock is far
more precise:

#include <pthread_time.h>
#include <iostream>

int main()
{
    using namespace std;
    timespec ts;

    if(clock_getres(CLOCK_REALTIME, &ts) == 0)
        cout << "CLOCK_REALTIME: " << ts.tv_sec << ',' << ts.tv_nsec << endl;
    if(clock_getres(CLOCK_MONOTONIC, &ts) == 0)
        cout << "CLOCK_MONOTONIC: " << ts.tv_sec << ',' << ts.tv_nsec << endl;
}

The result on my machine:

CLOCK_REALTIME: 0,15625000
CLOCK_MONOTONIC: 0,489

I then found the actual implementation in winpthreads (clock.c):

int clock_gettime(clockid_t clock_id, struct timespec *tp)
{
    unsigned __int64 t;
    LARGE_INTEGER pf, pc;
    union {
        unsigned __int64 u64;
        FILETIME ft;
    }  ct, et, kt, ut;

    switch(clock_id) {
    case CLOCK_REALTIME:
        {
            GetSystemTimeAsFileTime(&ct.ft);
            t = ct.u64 - DELTA_EPOCH_IN_100NS;
            tp->tv_sec = t / POW10_7;
            tp->tv_nsec = ((int) (t % POW10_7)) * 100;

            return 0;
        }

    case CLOCK_MONOTONIC:
        {
            if (QueryPerformanceFrequency(&pf) == 0)
                return lc_set_errno(EINVAL);

            if (QueryPerformanceCounter(&pc) == 0)
                return lc_set_errno(EINVAL);

            tp->tv_sec = pc.QuadPart / pf.QuadPart;
            tp->tv_nsec = (int) (((pc.QuadPart % pf.QuadPart) * POW10_9 +
(pf.QuadPart >> 1)) / pf.QuadPart);
            if (tp->tv_nsec >= POW10_9) {
                tp->tv_sec ++;
                tp->tv_nsec -= POW10_9;
            }

            return 0;
        }

    case CLOCK_PROCESS_CPUTIME_ID:
        {
        if(0 == GetProcessTimes(GetCurrentProcess(), &ct.ft, &et.ft, &kt.ft,
&ut.ft))
            return lc_set_errno(EINVAL);
        t = kt.u64 + ut.u64;
        tp->tv_sec = t / POW10_7;
        tp->tv_nsec = ((int) (t % POW10_7)) * 100;

        return 0;
        }

    case CLOCK_THREAD_CPUTIME_ID: 
        {
            if(0 == GetThreadTimes(GetCurrentThread(), &ct.ft, &et.ft, &kt.ft,
&ut.ft))
                return lc_set_errno(EINVAL);
            t = kt.u64 + ut.u64;
            tp->tv_sec = t / POW10_7;
            tp->tv_nsec = ((int) (t % POW10_7)) * 100;

            return 0;
        }

    default:
        break;
    }

    return lc_set_errno(EINVAL);
}

For CLOCK_REALTIME, the Windows API GetSystemTimeAsFileTime is used.
GetSystemTimePreciseAsFileTime is an improved version which provide "the
highest possible level of precision (<1us)". Unfortunately, the latter is only
available since Windows 8/Windows 2012, which is not suited for winpthreads for
compatibility reason IMO. See this MSDN page for details:
http://msdn.microsoft.com/en-us/library/windows/desktop/hh706895(v=vs.85).aspx
For CLOCK_MONOTONIC, QPC(Query Performance Counter) APIs are used. This method
is reasonably portable (among different versions of Windows, since Windows
2000), and Microsoft strongly suggested it when high-resolution time stamps
needed, see
http://msdn.microsoft.com/en-us/library/windows/desktop/dn553408(v=vs.85).aspx.
However, QPC is not suited for a system-wide clock.

[Bug libstdc++/63400] [C++11]precision of std::chrono::high_resolution_clock

[frankhb1989 at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63400

--- Comment #5 from frankhb1989 at gmail dot com ---
BTW, what if the clock_gettime call failed? The current implementation does
nothing about error handling... (Though for QPC on Windows it should rarely
fail for machines within a decade.)

[Bug libstdc++/63400] [C++11]precision of std::chrono::high_resolution_clock

[redi at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63400

--- Comment #6 from Jonathan Wakely <redi at gcc dot gnu.org> ---
Maybe we should make high_resolution_clock a typedef for steady_clock on
mingw-w64 .... but maybe only when using winpthreads.

[Bug libstdc++/63400] [C++11]precision of std::chrono::high_resolution_clock

[redi at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63400

--- Comment #7 from Jonathan Wakely <redi at gcc dot gnu.org> ---
(In reply to frankhb1989 from comment #5)
> BTW, what if the clock_gettime call failed? The current implementation does
> nothing about error handling... (Though for QPC on Windows it should rarely
> fail for machines within a decade.)

None of the errors specified by POSIX or Linux are possible, because we don't
pass invalid clock IDs or invalid pointers to the function.

I don't know if the winpthreads wrapper has additional failure cases. It should
not be possible for it to fail when given valid arguments.

[Bug libstdc++/63400] [C++11]precision of std::chrono::high_resolution_clock

[redi at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63400

--- Comment #8 from Jonathan Wakely <redi at gcc dot gnu.org> ---
(In reply to frankhb1989 from comment #4)
> With mingw-w64, the following program shows that the monotonic clock is far
> more precise:
> 
> #include <pthread_time.h>
> #include <iostream>
> 
> int main()
> {
> 	using namespace std;
> 	timespec ts;
> 
> 	if(clock_getres(CLOCK_REALTIME, &ts) == 0)
> 		cout << "CLOCK_REALTIME: " << ts.tv_sec << ',' << ts.tv_nsec << endl;
> 	if(clock_getres(CLOCK_MONOTONIC, &ts) == 0)
> 		cout << "CLOCK_MONOTONIC: " << ts.tv_sec << ',' << ts.tv_nsec << endl;
> }
> 
> The result on my machine:
> 
> CLOCK_REALTIME: 0,15625000
> CLOCK_MONOTONIC: 0,489

Is this still an issue in 2022?

Using a mingw-w64 cross-compiler and running under Wine I get:

CLOCK_REALTIME: 0,100
CLOCK_MONOTONIC: 0,100

Is that just because I'm using Wine not real Windows?

[Bug libstdc++/63400] [C++11]precision of std::chrono::high_resolution_clock

[redi at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63400

--- Comment #9 from Jonathan Wakely <redi at gcc dot gnu.org> ---
(In reply to frankhb1989 from comment #4)
> For CLOCK_REALTIME, the Windows API GetSystemTimeAsFileTime is used.
> GetSystemTimePreciseAsFileTime is an improved version which provide "the
> highest possible level of precision (<1us)". Unfortunately, the latter is
> only available since Windows 8/Windows 2012, which is not suited for
> winpthreads for compatibility reason IMO. See this MSDN page for details:
> http://msdn.microsoft.com/en-us/library/windows/desktop/hh706895(v=vs.85).
> aspx

But libstdc++ could call that directly, instead of going through the
clock_gettime API.

[Bug libstdc++/63400] [C++11]precision of std::chrono::high_resolution_clock

[redi at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63400

--- Comment #10 from Jonathan Wakely <redi at gcc dot gnu.org> ---
GetSystemTimePreciseAsFileTime gives UTC, so would need adjustment for leap
seconds to turn it into a sys_time. That's doable though.

Alternatively, we could use it to implement a high performance
chrono::utc_clock, and then define system_clock::now() in terms of that.

The standard specifies utc_clock::now() as:

"Returns: from_sys(system_clock::now()), or a more accurate value of utc_time."


Also, if Windows FILETIME is measured in UTC then chrono::file_clock should use
that too. So we might want to use a custom implementation of chrono::file_clock
for Windows.

[Bug libstdc++/63400] [C++11]precision of std::chrono::high_resolution_clock

[redi at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63400

--- Comment #11 from Jonathan Wakely <redi at gcc dot gnu.org> ---
(In reply to Jonathan Wakely from comment #8)
> Is this still an issue in 2022?
> 
> Using a mingw-w64 cross-compiler and running under Wine I get:
> 
> CLOCK_REALTIME: 0,100
> CLOCK_MONOTONIC: 0,100
> 
> Is that just because I'm using Wine not real Windows?

I still want an answer to this question though.

[Bug libstdc++/63400] [C++11]precision of std::chrono::high_resolution_clock

[redi at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63400

--- Comment #12 from Jonathan Wakely <redi at gcc dot gnu.org> ---
(In reply to Jonathan Wakely from comment #10)
> GetSystemTimePreciseAsFileTime gives UTC, so would need adjustment for leap
> seconds to turn it into a sys_time. That's doable though.

Doable, but it would make calls to system_clock::now() slower, because
converting UTC to system requires a lookup in the leap second table, and then
adjusting the time.

Another option would be to leave system_clock alone, but make utc_clock fast
and precise on Windows, and make high_resolution_clock a separate type (not a
typedef for system_clock) which happens to use the same implementation as
utc_clock.

> Also, if Windows FILETIME is measured in UTC then chrono::file_clock should
> use that too. So we might want to use a custom implementation of
> chrono::file_clock for Windows.

And replace its from_sys/to_sys members with from_utc/to_utc.

[Bug libstdc++/63400] [C++11]precision of std::chrono::high_resolution_clock

[vz-gcc at zeitlins dot org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63400

--- Comment #13 from Vadim Zeitlin <vz-gcc at zeitlins dot org> ---
(In reply to Jonathan Wakely from comment #11)
> (In reply to Jonathan Wakely from comment #8)
> > Is this still an issue in 2022?
> > 
> > Using a mingw-w64 cross-compiler and running under Wine I get:
> > 
> > CLOCK_REALTIME: 0,100
> > CLOCK_MONOTONIC: 0,100
> > 
> > Is that just because I'm using Wine not real Windows?
> 
> I still want an answer to this question though.

FWIW I can confirm that running the program above compiled with gcc-mingw-w64
12.2.0-14+25.2 from Debian 12 under Windows 10.0.19044.3803 produces exactly
the same output.

[Bug libstdc++/63400] [C++11]precision of std::chrono::high_resolution_clock

[redi at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63400

--- Comment #14 from Jonathan Wakely <redi at gcc dot gnu.org> ---
Thanks!

So does that mean mingw-w64 fixed the issue by improving the resolution of
CLOCK_REALTIME?
In that case, this bug could be closed WORKSFORME.

Or maybe the testcase makes invalid assumptions and isn't really measuring what
it thinks it's measuring?

In any case, we might want to keep it open to track the suggestion of using
GetSystemTimePreciseAsFileTime for utc_clock, as an enhancement.

[Bug libstdc++/63400] [C++11]precision of std::chrono::high_resolution_clock

[redi at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63400

--- Comment #15 from Jonathan Wakely <redi at gcc dot gnu.org> ---
(In reply to Jonathan Wakely from comment #14)
> Or maybe the testcase makes invalid assumptions and isn't really measuring
> what it thinks it's measuring?

e.g. maybe clock_getres says 100ns even though the clocks aren't really that
precise.

[Bug libstdc++/63400] [C++11]precision of std::chrono::high_resolution_clock

[vz-gcc at zeitlins dot org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63400

--- Comment #16 from Vadim Zeitlin <vz-gcc at zeitlins dot org> ---
(In reply to Jonathan Wakely from comment #15)
> (In reply to Jonathan Wakely from comment #14)
> > Or maybe the testcase makes invalid assumptions and isn't really measuring
> > what it thinks it's measuring?
> 
> e.g. maybe clock_getres says 100ns even though the clocks aren't really that
> precise.

No, the clock is not precise. The original test case

int main() {
  for (unsigned long long size = 1; size < 10000000; size *= 10) {
    auto start = std::chrono::high_resolution_clock::now();
    std::vector<int> v(size, 42);
    auto end = std::chrono::high_resolution_clock::now();
    auto elapsed = end - start;
    std::cout << size << ": " << elapsed.count() << '\n';
  }
}

outputs the following under Linux

% ./a.out               
1: 436
10: 114
100: 80
1000: 1019
10000: 9499
100000: 104850
1000000: 686436

and this under Windows:

% ./a.exe
1: 1000
10: 0
100: 0
1000: 1000
10000: 9000
100000: 69000
1000000: 523000