RE: Memory leak problem in MSVS6.0 C RTL

RE: Memory leak problem in MSVS6.0 C RTL

[Bossom, John]

Like I said, if you do NOT the win32-pthreads DLL, you will not get the
automatic
cleanup of thread specific data (i.e. the destroy routine getting called on
the
data you registered with the key)...

-----Original Message-----
From: Mattias Ernelli [ mailto:Mattias.Ernelli@melody.se ]
Sent: December 22, 2000 10:28 AM
To: 'Bossom, John'
Cc: Pthreads Win32 (E-mail)
Subject: RE: Memory leak problem in MSVS6.0 C RTL


Thanks for your quick reply.
I noticed what caused the problem when I link pthreads with my application.
I specify statically linked C RTL (libcmt/libcmtd) instead of dynamically
linked (msvcrt/msvcrtd).

What happends is that the C RTL routines that uses the C RTL local storage
allocates data on the fly, for example when accessing errno or rand(). The C
RTL designers probably ensured that code calling CreateThread instead of
_beginthreadex should not break.

If I build my application using dynamically linked C RTL, the memory leak
goes away, probably because the RTL code that pthread calls to initialise
the RTL thread data is different from the statically linked RTL functions
that my application calls, and additionally data is allocated  by the
statically linked C RTL code.

My question is then: 

Is it possible to build the pthread-win32 library for statically linking?
Booth regarding the linkage specification for the C RTL as well as
pthreadXX.lib itself?

The reason for using statically linked libraries is in general that our
application consists mainly of separate modules, each as a static library,
each compiled using a static linkage specification for the C RTL, we could
switch to dynamically linked C RTL, but I would prefer statically linked
code. Its easier to keep things together and to avoid "dll hell"...

//Mattias

-----Original Message-----
From: Bossom, John [ mailto:John.Bossom@Cognos.COM ]
Sent: den 22 december 2000 13:56
To: 'Mattias Ernelli'; Pthreads Win32 (E-mail)
Subject: RE: Memory leak problem in MSVS6.0 C library when using
pthread_c reat e


The code should be using beginthreadex and endthreadex... These
interfaces ensure that the C RTL is initialized and cleaned up properly.
If you do not use these thread routines and you attempt a C RTL call,
you will probably blow up since C RTL initialization will not have
taken place.

Now, the implementation of TSD for pthreads requires that you use the
DLL version of pthreads... If you check the DllMain, you will see
that it should, if DLL_THREAD_DETACH is passed, cause the TSD destroy
routines to be called for this terminating thread, on all thread keys
that have a destroy routine.
If you are using the static link library for pthreads, you will have to
perform this cleanup yourself.

-

RE: Memory leak problem in MSVS6.0 C RTL

[Laurence F. Wood]

I have implemented pthreads-win32 as a statically linked library.  As
earlier described, the only thing that is necessary is to have two routines
that are called prior to application init and close which replicate what the
DLL would normally do.  We have tested this on some hairy massively threaded
qauntum chemistry codes and it works fine.

-----Original Message-----
From: pthreads-win32-owner@sources.redhat.com
[ mailto:pthreads-win32-owner@sources.redhat.com]On Behalf Of Bossom,
John
Sent: Friday, December 22, 2000 10:37 AM
To: 'Mattias Ernelli'; Bossom, John
Cc: Pthreads Win32 (E-mail)
Subject: RE: Memory leak problem in MSVS6.0 C RTL


Like I said, if you do NOT the win32-pthreads DLL, you will not get the
automatic
cleanup of thread specific data (i.e. the destroy routine getting called on
the
data you registered with the key)...

-----Original Message-----
From: Mattias Ernelli [ mailto:Mattias.Ernelli@melody.se ]
Sent: December 22, 2000 10:28 AM
To: 'Bossom, John'
Cc: Pthreads Win32 (E-mail)
Subject: RE: Memory leak problem in MSVS6.0 C RTL


Thanks for your quick reply.
I noticed what caused the problem when I link pthreads with my application.
I specify statically linked C RTL (libcmt/libcmtd) instead of dynamically
linked (msvcrt/msvcrtd).

What happends is that the C RTL routines that uses the C RTL local storage
allocates data on the fly, for example when accessing errno or rand(). The C
RTL designers probably ensured that code calling CreateThread instead of
_beginthreadex should not break.

If I build my application using dynamically linked C RTL, the memory leak
goes away, probably because the RTL code that pthread calls to initialise
the RTL thread data is different from the statically linked RTL functions
that my application calls, and additionally data is allocated  by the
statically linked C RTL code.

My question is then:

Is it possible to build the pthread-win32 library for statically linking?
Booth regarding the linkage specification for the C RTL as well as
pthreadXX.lib itself?

The reason for using statically linked libraries is in general that our
application consists mainly of separate modules, each as a static library,
each compiled using a static linkage specification for the C RTL, we could
switch to dynamically linked C RTL, but I would prefer statically linked
code. Its easier to keep things together and to avoid "dll hell"...

//Mattias

-----Original Message-----
From: Bossom, John [ mailto:John.Bossom@Cognos.COM ]
Sent: den 22 december 2000 13:56
To: 'Mattias Ernelli'; Pthreads Win32 (E-mail)
Subject: RE: Memory leak problem in MSVS6.0 C library when using
pthread_c reat e


The code should be using beginthreadex and endthreadex... These
interfaces ensure that the C RTL is initialized and cleaned up properly.
If you do not use these thread routines and you attempt a C RTL call,
you will probably blow up since C RTL initialization will not have
taken place.

Now, the implementation of TSD for pthreads requires that you use the
DLL version of pthreads... If you check the DllMain, you will see
that it should, if DLL_THREAD_DETACH is passed, cause the TSD destroy
routines to be called for this terminating thread, on all thread keys
that have a destroy routine.
If you are using the static link library for pthreads, you will have to
perform this cleanup yourself.

-

RE: Memory leak problem in MSVS6.0 C RTL

[Laurence F. Wood]

Here is how I have modified pthreads-win32 with _LIB conditional
compilations for building
as a statically linked library. There is a new conditionally compiled
function in pthread.h that is implemented in dll.c.  I use this
implementation successfully to run massively threaded quantum chemistry
codes.  It should be obvious how to make use of the new call from the code
in dll.c.  If this is of use would someone please make certain that this
gets to the correct archive or pthreads development people.

Happy Holiday,

Larry Wood

-----Original Message-----
From: pthreads-win32-owner@sources.redhat.com
[ mailto:pthreads-win32-owner@sources.redhat.com]On Behalf Of Laurence F.
Wood
Sent: Friday, December 22, 2000 4:00 PM
To: Bossom, John; 'Mattias Ernelli'
Cc: Pthreads Win32 (E-mail)
Subject: RE: Memory leak problem in MSVS6.0 C RTL


I have implemented pthreads-win32 as a statically linked library.  As
earlier described, the only thing that is necessary is to have two routines
that are called prior to application init and close which replicate what the
DLL would normally do.  We have tested this on some hairy massively threaded
qauntum chemistry codes and it works fine.

-----Original Message-----
From: pthreads-win32-owner@sources.redhat.com
[ mailto:pthreads-win32-owner@sources.redhat.com]On Behalf Of Bossom,
John
Sent: Friday, December 22, 2000 10:37 AM
To: 'Mattias Ernelli'; Bossom, John
Cc: Pthreads Win32 (E-mail)
Subject: RE: Memory leak problem in MSVS6.0 C RTL


Like I said, if you do NOT the win32-pthreads DLL, you will not get the
automatic
cleanup of thread specific data (i.e. the destroy routine getting called on
the
data you registered with the key)...

-----Original Message-----
From: Mattias Ernelli [ mailto:Mattias.Ernelli@melody.se ]
Sent: December 22, 2000 10:28 AM
To: 'Bossom, John'
Cc: Pthreads Win32 (E-mail)
Subject: RE: Memory leak problem in MSVS6.0 C RTL


Thanks for your quick reply.
I noticed what caused the problem when I link pthreads with my application.
I specify statically linked C RTL (libcmt/libcmtd) instead of dynamically
linked (msvcrt/msvcrtd).

What happends is that the C RTL routines that uses the C RTL local storage
allocates data on the fly, for example when accessing errno or rand(). The C
RTL designers probably ensured that code calling CreateThread instead of
_beginthreadex should not break.

If I build my application using dynamically linked C RTL, the memory leak
goes away, probably because the RTL code that pthread calls to initialise
the RTL thread data is different from the statically linked RTL functions
that my application calls, and additionally data is allocated  by the
statically linked C RTL code.

My question is then:

Is it possible to build the pthread-win32 library for statically linking?
Booth regarding the linkage specification for the C RTL as well as
pthreadXX.lib itself?

The reason for using statically linked libraries is in general that our
application consists mainly of separate modules, each as a static library,
each compiled using a static linkage specification for the C RTL, we could
switch to dynamically linked C RTL, but I would prefer statically linked
code. Its easier to keep things together and to avoid "dll hell"...

//Mattias

-----Original Message-----
From: Bossom, John [ mailto:John.Bossom@Cognos.COM ]
Sent: den 22 december 2000 13:56
To: 'Mattias Ernelli'; Pthreads Win32 (E-mail)
Subject: RE: Memory leak problem in MSVS6.0 C library when using
pthread_c reat e


The code should be using beginthreadex and endthreadex... These
interfaces ensure that the C RTL is initialized and cleaned up properly.
If you do not use these thread routines and you attempt a C RTL call,
you will probably blow up since C RTL initialization will not have
taken place.

Now, the implementation of TSD for pthreads requires that you use the
DLL version of pthreads... If you check the DllMain, you will see
that it should, if DLL_THREAD_DETACH is passed, cause the TSD destroy
routines to be called for this terminating thread, on all thread keys
that have a destroy routine.
If you are using the static link library for pthreads, you will have to
perform this cleanup yourself.

-

RE: Memory leak problem in MSVS6.0 C RTL

[Laurence F. Wood]

It's to bad you had to switch to the dll version.  I wager that approach
will start rippling through your application. Good idea about the _CRTIMP
extern.


--Larry

-----Original Message-----
From: pthreads-win32-owner@sources.redhat.com
[ mailto:pthreads-win32-owner@sources.redhat.com]On Behalf Of Mattias
Ernelli
Sent: Wednesday, December 27, 2000 12:41 PM
To: 'Laurence F. Wood'
Cc: Pthreads Win32 (E-mail)
Subject: RE: Memory leak problem in MSVS6.0 C RTL


Intresting, I noticed also that you have renamed the _errno declaration in
pthread.h to _errnoPthread to avoid nameclash with the _errno() declared in
need_errno/errno.h. I get compiletime warnings about inconsistant linkage
specification since in need_errno/errno.h it is declared "_CRTIMP extern int
* _errno( void ); " which when compiled as dll yields dllexport extern int *
_errno( void ).

I was looking for solutions, one is to declare NEED_ERRNO as well as
declaring _MD but it felt awkward since NEED_ERRNO includes need_errno.h
which is just a copy of errno.h.

So I solved it by adding _CRTIMP extern to the declaration in pthread.h.

I wonder if this has been addressed by the pthread-32 comunity before? Or is
everybody ignoring the compiler warning? I do think that the declaration
should go away since its declared in either need_errno.h or errno.h.

Anyway, what I meant about static linking is that our application is is
using the static version of the C RTL (libcmt.lib) not msvcrt.lib, ant that
caused a conflict with the pthread linking of msvcrt.lib/dll. Now I have
changed so that our application links to msvcrt.lib/dll as well, although I
have to recompile pthread depending on whether I need to link to the
debug/release version of msvcrt. Maybe that should be part of the build
system to build a version that links to either release/debug version of the
msvcrt.dll.

//Mattias

-----Original Message-----
From: Laurence F. Wood [ mailto:LaurenceWood@SunyataSystems.Com ]
Sent: den 22 december 2000 23:35
To: Laurence F. Wood; Bossom, John; 'Mattias Ernelli'
Cc: Pthreads Win32 (E-mail)
Subject: RE: Memory leak problem in MSVS6.0 C RTL


Here is how I have modified pthreads-win32 with _LIB conditional
compilations for building
as a statically linked library. There is a new conditionally compiled
function in pthread.h that is implemented in dll.c.  I use this
implementation successfully to run massively threaded quantum chemistry
codes.  It should be obvious how to make use of the new call from the code
in dll.c.  If this is of use would someone please make certain that this
gets to the correct archive or pthreads development people.

Happy Holiday,

Larry Wood

RE: Memory leak problem in MSVS6.0 C RTL

[Mattias Ernelli]

Intresting, I noticed also that you have renamed the _errno declaration in
pthread.h to _errnoPthread to avoid nameclash with the _errno() declared in
need_errno/errno.h. I get compiletime warnings about inconsistant linkage
specification since in need_errno/errno.h it is declared "_CRTIMP extern int
* _errno( void ); " which when compiled as dll yields dllexport extern int *
_errno( void ).

I was looking for solutions, one is to declare NEED_ERRNO as well as
declaring _MD but it felt awkward since NEED_ERRNO includes need_errno.h
which is just a copy of errno.h.

So I solved it by adding _CRTIMP extern to the declaration in pthread.h.

I wonder if this has been addressed by the pthread-32 comunity before? Or is
everybody ignoring the compiler warning? I do think that the declaration
should go away since its declared in either need_errno.h or errno.h.

Anyway, what I meant about static linking is that our application is is
using the static version of the C RTL (libcmt.lib) not msvcrt.lib, ant that
caused a conflict with the pthread linking of msvcrt.lib/dll. Now I have
changed so that our application links to msvcrt.lib/dll as well, although I
have to recompile pthread depending on whether I need to link to the
debug/release version of msvcrt. Maybe that should be part of the build
system to build a version that links to either release/debug version of the
msvcrt.dll.

//Mattias

-----Original Message-----
From: Laurence F. Wood [ mailto:LaurenceWood@SunyataSystems.Com ]
Sent: den 22 december 2000 23:35
To: Laurence F. Wood; Bossom, John; 'Mattias Ernelli'
Cc: Pthreads Win32 (E-mail)
Subject: RE: Memory leak problem in MSVS6.0 C RTL


Here is how I have modified pthreads-win32 with _LIB conditional
compilations for building
as a statically linked library. There is a new conditionally compiled
function in pthread.h that is implemented in dll.c.  I use this
implementation successfully to run massively threaded quantum chemistry
codes.  It should be obvious how to make use of the new call from the code
in dll.c.  If this is of use would someone please make certain that this
gets to the correct archive or pthreads development people.

Happy Holiday,

Larry Wood

RE: Memory leak problem in MSVS6.0 C RTL

[Mattias Ernelli]

Thanks for your quick reply.
I noticed what caused the problem when I link pthreads with my application.
I specify statically linked C RTL (libcmt/libcmtd) instead of dynamically
linked (msvcrt/msvcrtd).

What happends is that the C RTL routines that uses the C RTL local storage
allocates data on the fly, for example when accessing errno or rand(). The C
RTL designers probably ensured that code calling CreateThread instead of
_beginthreadex should not break.

If I build my application using dynamically linked C RTL, the memory leak
goes away, probably because the RTL code that pthread calls to initialise
the RTL thread data is different from the statically linked RTL functions
that my application calls, and additionally data is allocated  by the
statically linked C RTL code.

My question is then: 

Is it possible to build the pthread-win32 library for statically linking?
Booth regarding the linkage specification for the C RTL as well as
pthreadXX.lib itself?

The reason for using statically linked libraries is in general that our
application consists mainly of separate modules, each as a static library,
each compiled using a static linkage specification for the C RTL, we could
switch to dynamically linked C RTL, but I would prefer statically linked
code. Its easier to keep things together and to avoid "dll hell"...

//Mattias

-----Original Message-----
From: Bossom, John [ mailto:John.Bossom@Cognos.COM ]
Sent: den 22 december 2000 13:56
To: 'Mattias Ernelli'; Pthreads Win32 (E-mail)
Subject: RE: Memory leak problem in MSVS6.0 C library when using
pthread_c reat e


The code should be using beginthreadex and endthreadex... These
interfaces ensure that the C RTL is initialized and cleaned up properly.
If you do not use these thread routines and you attempt a C RTL call,
you will probably blow up since C RTL initialization will not have
taken place.

Now, the implementation of TSD for pthreads requires that you use the
DLL version of pthreads... If you check the DllMain, you will see
that it should, if DLL_THREAD_DETACH is passed, cause the TSD destroy
routines to be called for this terminating thread, on all thread keys
that have a destroy routine.
If you are using the static link library for pthreads, you will have to
perform this cleanup yourself.

-