[Bug libstdc++/57440] Memory usage with future and std containers

[Bug libstdc++/57440] Memory usage with future and std containers

[redi at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=57440

--- Comment #1 from Jonathan Wakely <redi at gcc dot gnu.org> ---
(In reply to DrD from comment #0)
>         // ... launch the threads
>         vector<std::future<double> > values;
>         for (uint w=0; w<nThreads; ++w) {
>             values.push_back(std::async(TestFutures));
>         }

One quick comment, without analysing the issue:

No threads are launched in your program.  Currently GCC's std::async always
behaves as std::async(std::launch::deferred, ...) so to run in a new thread you
need to explicitly call std::async(std::launch::async, TestFutures)

[Bug libstdc++/57440] Memory usage with future and std containers

[demonskull1 at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=57440

--- Comment #2 from DrD <demonskull1 at gmail dot com> ---
(In reply to Jonathan Wakely from comment #1)
> (In reply to DrD from comment #0)
> >         // ... launch the threads
> >         vector<std::future<double> > values;
> >         for (uint w=0; w<nThreads; ++w) {
> >             values.push_back(std::async(TestFutures));
> >         }
> 
> One quick comment, without analysing the issue:
> 
> No threads are launched in your program.  Currently GCC's std::async always
> behaves as std::async(std::launch::deferred, ...) so to run in a new thread
> you need to explicitly call std::async(std::launch::async, TestFutures)

Oh! thank you very much for the tip!

[Bug libstdc++/57440] Memory usage with future and std containers

[redi at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=57440

Jonathan Wakely <redi at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |WAITING
   Last reconfirmed|                            |2013-05-28
     Ever confirmed|0                           |1

[Bug libstdc++/57440] Memory usage with future and std containers

[redi at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=57440

--- Comment #4 from Jonathan Wakely <redi at gcc dot gnu.org> ---
If my guess is right you should be able to reproduce the unbounded memory usage
with this:

#include <future>

int f() { return 0; }

int main()
{
    for (int i=0; i < 100000; ++i)
        std::async(f).get();
}


And you should not see it happen with this:

#define _GTHREAD_USE_MUTEX_INIT_FUNC
#define _GTHREAD_USE_COND_INIT_FUNC

#include <future>

int f() { return 0; }

int main()
{
    for (int i=0; i < 100000; ++i)
        std::async(f).get();
}

Please check and update the bug report, thanks.

[Bug libstdc++/57440] Memory usage with future and std containers

[demonskull1 at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=57440

--- Comment #5 from DrD <demonskull1 at gmail dot com> ---
(In reply to Jonathan Wakely from comment #3)
> Also I can't reproduce this on GNU/Linux, the memory usage is bounded as
> expected.
> 
> I'm surprised this even compiles with Mingw, are you using Mingw-w64 with
> libpthread-win32?  Since it seems to be platform-specific it could be a
> resource leak of mutexes or condition variables, so my wild guess is that
> libpthread-win32 requires pthread_mutex_destroy or pthread_cond_destroy to
> be used.
> 
> Does it make any difference if you add these to the very top of the source
> file, before including any headers?
> 
> #define _GTHREAD_USE_MUTEX_INIT_FUNC
> #define _GTHREAD_USE_COND_INIT_FUNC

It does compile in MinGw. Unfortunately the two #define lines don't help
either.

I forgot to mention that I had already been warned that the test code works OK
in Linux, see:
http://stackoverflow.com/questions/16489389/memory-usage-with-future-and-containers-in-qt-mingw

[Bug libstdc++/57440] Memory usage with future and std containers

[demonskull1 at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=57440

--- Comment #6 from DrD <demonskull1 at gmail dot com> ---
(In reply to Jonathan Wakely from comment #4)
> If my guess is right you should be able to reproduce the unbounded memory
> usage with this:
> 
> #include <future>
> 
> int f() { return 0; }
> 
> int main()
> {
>     for (int i=0; i < 100000; ++i)
>         std::async(f).get();
> }
> 
> 
> And you should not see it happen with this:
> 
> #define _GTHREAD_USE_MUTEX_INIT_FUNC
> #define _GTHREAD_USE_COND_INIT_FUNC
> 
> #include <future>
> 
> int f() { return 0; }
> 
> int main()
> {
>     for (int i=0; i < 100000; ++i)
>         std::async(f).get();
> }
> 
> Please check and update the bug report, thanks.

Hi Jonathan,

Thanks for your help. I just tried this code and the memory problem is there
with and without the #define lines.

I haven't managed to get this bug tested on Windows by anyone else yet. I have
tried in two different computers, though (not enough, I think).

Any suggestions?

[Bug libstdc++/57440] Memory usage with future and std containers

[redi at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=57440

--- Comment #7 from Jonathan Wakely <redi at gcc dot gnu.org> ---
You didn't answer the question about which mingw compiler you are using, the
standard mingw gcc does not support std::async.

[Bug libstdc++/57440] Memory usage with future and std containers

[demonskull1 at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=57440

--- Comment #8 from DrD <demonskull1 at gmail dot com> ---
(In reply to Jonathan Wakely from comment #7)
> You didn't answer the question about which mingw compiler you are using, the
> standard mingw gcc does not support std::async.

>From my first post:
"I compile it with gcc 4.7.2 (MinGW 4.7.2) in Qt 5.0.1, Qt Creator 2.6.2."

Is that what you need or are you after some other detail I have omitted?

[Bug libstdc++/57440] Memory usage with future and std containers

[redi at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=57440

--- Comment #9 from Jonathan Wakely <redi at gcc dot gnu.org> ---
Some other detail.  I don't care about Qt Creator, it's not a compiler, and the
version of Qt is compeltely irrelevant because you're not using Qt.

I don't believe you can be using Mingw 4.7.2, because that doesn't support
std::async().  What does 'gcc -v' show?

[Bug libstdc++/57440] Memory usage with future and std containers

[demonskull1 at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=57440

--- Comment #10 from DrD <demonskull1 at gmail dot com> ---
(In reply to Jonathan Wakely from comment #7)
> You didn't answer the question about which mingw compiler you are using, the
> standard mingw gcc does not support std::async.

>From my first post:
"I compile it with gcc 4.7.2 (MinGW 4.7.2) in Qt 5.0.1, Qt Creator 2.6.2."

Is that what you need or are you after some other detail I have omitted?(In
reply to Jonathan Wakely from comment #9)
> Some other detail.  I don't care about Qt Creator, it's not a compiler, and
> the version of Qt is compeltely irrelevant because you're not using Qt.
> 
> I don't believe you can be using Mingw 4.7.2, because that doesn't support
> std::async().  What does 'gcc -v' show?

I am using the MinGW copy that comes with Qt. If I go to the bin folder I
obtain:


C:\Qt\Qt5.0.1\Tools\MinGW\bin>gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=c:/qt/qt5.0.1/tools/mingw/bin/../libexec/gcc/i686-w64-mingw32/4.7.2/lto-wrapper.exe
Target: i686-w64-mingw32
Configured with: ../../../src/gcc-4.7.2/configure --host=i686-w64-mingw32
--build=i686-w64-mingw32 --target=i686-w64-mingw32
--prefix=/temp/x32-4.7.2-posix-sjlj-r8/prefix
--with-sysroot=/temp/x32-4.7.2-posix-sj
lj-r8/prefix --enable-shared --enable-static --enable-targets=all
--enable-multilib --enable-languages=c,c++,fortran,lto
--enable-libstdcxx-time=yes --enable-threads=posix --enable-libgomp
--enable-lto --enable
-graphite --enable-cloog-backend=isl --enable-checking=release
--enable-fully-dynamic-string --enable-version-specific-runtime-libs
--enable-sjlj-exceptions --disable-ppl-version-check --disable-cloog-version-c
heck --disable-libstdcxx-pch --disable-libstdcxx-debug --disable-bootstrap
--disable-rpath --disable-win32-registry --disable-nls --disable-werror
--disable-symvers --with-gnu-as --with-gnu-ld --with-arch-32=i6
86 --with-arch-64=nocona --with-tune-32=core2 --with-tune-64=core2
--with-host-libstdcxx='-static -lstdc++' --with-libiconv --with-system-zlib
--with-gmp=/temp/mingw-prereq/i686-w64-mingw32-static --with-mpfr=/
temp/mingw-prereq/i686-w64-mingw32-static
--with-mpc=/temp/mingw-prereq/i686-w64-mingw32-static
--with-ppl=/temp/mingw-prereq/i686-w64-mingw32-static
--with-cloog=/temp/mingw-prereq/i686-w64-mingw32-static --wi
th-pkgversion='Built by MinGW-builds project'
--with-bugurl=http://sourceforge.net/projects/mingwbuilds/ CFLAGS='-O2 -pipe
-fomit-frame-pointer -I/temp/x32-4.7.2-posix-sjlj-r8/libs/include
-I/temp/mingw-prereq/
x32-zlib/include -I/temp/mingw-prereq/i686-w64-mingw32-static/include'
CXXFLAGS='-O2 -pipe -fomit-frame-pointer
-I/temp/x32-4.7.2-posix-sjlj-r8/libs/include
-I/temp/mingw-prereq/x32-zlib/include -I/temp/mingw-p
rereq/i686-w64-mingw32-static/include' CPPFLAGS= LDFLAGS='-pipe
-L/temp/x32-4.7.2-posix-sjlj-r8/libs/lib -L/temp/mingw-prereq/x32-zlib/lib
-L/temp/mingw-prereq/i686-w64-mingw32-static/lib -L/temp/x32-4.7.2-posi
x-sjlj-r8/prefix/opt/lib'
Thread model: posix
gcc version 4.7.2 (Built by MinGW-builds project)

I am relatively new to MinGw (it shows), so I am not too familiar with the
particularities of Qt's copy.

[Bug libstdc++/57440] Memory usage with future and std containers

[redi at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=57440

--- Comment #12 from Jonathan Wakely <redi at gcc dot gnu.org> ---
Using Qt Creator I have confirmed the leak, and can reproduce it with this 

#include <pthread.h>
#include <unistd.h>

int main()
{
    for (int i=0; i < 100000; ++i) {
        pthread_mutex_t mx = PTHREAD_MUTEX_INITIALIZER;
        pthread_mutex_lock(&mx);
        pthread_mutex_unlock(&mx);
        usleep(10000);
    }
}

This is a simplified version of the code in libstdc++, here's another leaky
program using libstdc++:

//#define _GTHREAD_USE_MUTEX_INIT_FUNC

#include <mutex>
#include <unistd.h>

int main()
{
    for (int i=0; i < 100000; ++i) {
        std::mutex mx;
        mx.lock();
        mx.unlock();
        usleep(10000);
    }
}

Uncommenting the first line fixes it, so we should define that macro in
libstdc++-v3/config/os/mingw32-w64/os_defines.h

[Bug libstdc++/57440] Memory usage with future and std containers

[redi at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=57440

--- Comment #13 from Jonathan Wakely <redi at gcc dot gnu.org> ---
dup of PR 59807

[Bug libstdc++/57440] Memory usage with future and std containers

[ktietz at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=57440

Kai Tietz <ktietz at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |ahanins at gmail dot com

--- Comment #14 from Kai Tietz <ktietz at gcc dot gnu.org> ---
*** Bug 59807 has been marked as a duplicate of this bug. ***

[Bug libstdc++/57440] Memory usage with future and std containers

[ktietz at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=57440

--- Comment #15 from Kai Tietz <ktietz at gcc dot gnu.org> ---
Posted patch to ML for defining _GTHREAD_USE_MUTEX_INIT_FUNC in mingw-w64, and
mingw32 case.  It is true that posix-threading support for Windows is right now
only provided by mingw-w64 based toolchains, nevertheless is the issue a
general one on native Windows platforms using posix-threading.

[Bug libstdc++/57440] Memory usage with future and std containers

[ktietz at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=57440

--- Comment #16 from Kai Tietz <ktietz at gcc dot gnu.org> ---
Author: ktietz
Date: Tue Oct 14 16:58:37 2014
New Revision: 216210

URL: https://gcc.gnu.org/viewcvs?rev=216210&root=gcc&view=rev
Log:
2014-10-14  Kai Tietz  <ktietz@redhat.com>

        PR libstdc++/57440
        * config/os/mingw32/os_defines.h (_GTHREAD_USE_MUTEX_INIT_FUNC):
        Define to avoid leak.
        * config/os/mingw32-w64/os_defines.h: Likewise.


Modified:
    trunk/libstdc++-v3/ChangeLog
    trunk/libstdc++-v3/config/os/mingw32-w64/os_defines.h
    trunk/libstdc++-v3/config/os/mingw32/os_defines.h

[Bug libstdc++/57440] Memory usage with future and std containers

[ktietz at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=57440

--- Comment #18 from Kai Tietz <ktietz at gcc dot gnu.org> ---
Author: ktietz
Date: Tue Oct 14 17:06:27 2014
New Revision: 216213

URL: https://gcc.gnu.org/viewcvs?rev=216213&root=gcc&view=rev
Log:
    PR libstdc++/57440
    * config/os/mingw32/os_defines.h (_GTHREAD_USE_MUTEX_INIT_FUNC):
    Define to avoid leak.
    * config/os/mingw32-w64/os_defines.h: Likewise.

Modified:
    branches/gcc-4_8-branch/libstdc++-v3/ChangeLog
    branches/gcc-4_8-branch/libstdc++-v3/config/os/mingw32-w64/os_defines.h
    branches/gcc-4_8-branch/libstdc++-v3/config/os/mingw32/os_defines.h

[Bug libstdc++/57440] Memory usage with future and std containers

[ktietz at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=57440

Kai Tietz <ktietz at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|---                         |FIXED

--- Comment #19 from Kai Tietz <ktietz at gcc dot gnu.org> ---
Applied patch to trunk, and all open branches (4.9, 4.8).  Closed as fixed.