RE: Mutex implementation questions

RE: Mutex implementation questions

[Scott McCaskill]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 1031 bytes --]

Title: RE: Mutex implementation questions







> -----Original Message-----
> From: Tristan Savatier [ mailto:tristan@mpegtv.com ]
> Sent: Monday, October 02, 2000 5:10 PM
> To: 'pthreads-win32@sources.redhat.com'
> Subject: Mutex implementation questions
> 
> 
> I noticed that if _pthread_try_enter_critical_section has been set
> to a non-NULL value by DllMain, Mutexes are implemented using
> critical sections (InitializeCriticalSection) instead of 
> CreateMutex, regardless of the value of the implemetation-specific
> forcecs
> mutex attribute.
> 
> According to "Win32 programming", critical sections are light weight
> compared to mutexes, they are not managed by the kernel, and they
> are much faster than mutexes.  So why no use critical sections
> all the time to implement pthread mutexes ?
> 
> Are there cases where critical sections are not available or
> wouldn't work well ?
> 


Yes, I'm pretty sure critical sections can't be shared between processes (but mutexes can).  So this may be the reason.

Re: Mutex implementation questions

[Tristan Savatier]

> Scott McCaskill wrote:
> 
> 
> Yes, I'm pretty sure critical sections can't be shared between
> processes (but mutexes can).  So this may be the reason.

Yes, but does the pthread API  permit the creation of pthread mutexes
that
are shared between processes ?

If not, then what would be the advantage of using  CreateMutex
rather than InitializeCriticalSection ?

-t

Re: Mutex implementation questions

[Ross Johnson]

Tristan Savatier wrote:
> 
> > Scott McCaskill wrote:
> >
> >
> > Yes, I'm pretty sure critical sections can't be shared between
> > processes (but mutexes can).  So this may be the reason.
> 
> Yes, but does the pthread API  permit the creation of pthread mutexes
> that
> are shared between processes ?

Pthreads-win32 doesn't have them yet, but it does have the
functions (pthread_*_*pshared) defined. Those and some others emit
an error if you try to use process shared objects.

_POSIX_THREAD_PROCESS_SHARED is not defined in pthreads-win32.

Ross

-- 
+----------------------+---+
| Ross Johnson         |   | E-Mail: rpj@ise.canberra.edu.au
| Info Sciences and Eng|___|
| University of Canberra   | FAX:    +61 6 2015227
| PO Box 1                 |
| Belconnen  ACT    2616   | WWW:    http://willow.canberra.edu.au/~rpj/
| AUSTRALIA                |
+--------------------------+

RE: Mutex implementation questions

[Bossom, John]

Hi Ross!

The ptw32_threadStart() is only used (correct me if I am wrong)
if someone explicitly calls pthread_create. Hence, anyone using
other simulated features of the win32-pthread library within
a "foreign" thread body will encounter cleanup/leak problems.
For example, using TSD with a destructor... the destructor will
not be called if a value is stored from within a foreign thread
body (such as the main process). You will then be forced to only
use any of the pthread types from within a thread explicitly
created with pthread_create. Perhaps this is a valid restriction
for those wanting to use a static link library as opposed to
a DLL.

The reason for the introduction of a DllMain was to ensure that
you can use any of the pthread types exactly like on UNIX. I
understand the concern over lack of DLL versioning from Microsoft.
However, it could be implemented ourselves through the use of
a thin library that dynamically loads the Dll by name (including
version codes).

Cheers,

John.

-----Original Message-----
From: Ross Johnson [ mailto:rpj@ise.canberra.edu.au ]
Sent: Monday, October 09, 2000 12:58 AM
To: Tristan Savatier
Cc: Bossom, John; 'pthreads-win32@sources.redhat.com'
Subject: Re: Mutex implementation questions


With regard to possibly eliminating dllMain for applications
linking pthread*.lib statically - rather than require all threads
to explicitly call pthread_exit(), is there any reason why the
DLL_THREAD_DETACH code in dllMain can't be moved to
private.c:ptw32_threadStart(), which is the wrapper for the
applications threads? It looks doable to me. That leaves
just the DLL_PROCESS_DETACH code which could be called
explicitly as suggested.

Ross

Tristan Savatier wrote:
> 
> Bossom, John wrote:
> > 
> > 4) By eliminating the DllMain, whose purpose was to perform
> >    behind the scenes cleanup of the "self" structure as well as calling
> >    all the destructors for thread-specific data keys, what alternate
> >    approach did you take for doing this thread-based cleanup?
> >    (i.e. I leveraged the fact that DllMain is called each time a thread
> >          is started/finished to perform thread based
initialization/cleanup)
> 
> You are right. One thing that can be done is to make sure that all
> thread
> terminate by calling pthread_exit(void *), and do not "fall through the
> bottom"
> of the threan entry-point routine (which is allowed by the standard).
> 
> That's what we do, and to make it work, we modified pthread_exit
> by replacing:
> 
>   _pthread_callUserDestroyRoutines((pthread_t)
> pthread_getspecific(_pthread_selfThreadKey));
> 
> by:
>   _pthread_threadDestroy((pthread_t)
> pthread_getspecific(_pthread_selfThreadKey));
> 
> which calls _pthread_callUserDestroyRoutines and then does the necessary
> cleanup
> of the thread TSD.
> 
> That takes are of the thread cleanup.  Regarding the process cleanup, I
> suppose
> that _pthread_processTerminate() should be called explicitely if the dll
> is not used.
> 
> Now, maybe you wonder why we prefer to link statically rather that
> to use the DLL (pthread.dll).  The reason is that Windows does not
> support
> dll versionning, i.e. if several applications rely on pthread.dll, but
> on various versions of it, things break.  There would be a solution
> which is
> to place the dll in the same directory as the application, rather than
> in
> the \Windows directory, but this also breaks due to the stupid rule used
> by Windows to look for dll's: first it looks in \Windows, then in the
> other places... which means that if another application places an
> incompatible pthread.dll in \windows, it will break you application
> (we had the experience with applications using xaudio.dll).
> 
> And pthread is so small that the gain of having it shared (compared
> to the trouble) is small.
> 
> That's why we think that using pthread.dll is a BAD idea, and that
> static linking with pthread.lib is a safer solution...
> 
> -t
> 
> > I hope this clears up some of your questions.
> >
> > Cheers,
> >
> > John.
> >
> > -----Original Message-----
> > From: Tristan Savatier [ mailto:tristan@mpegtv.com ]
> > Sent: Monday, October 02, 2000 6:10 PM
> > To: 'pthreads-win32@sources.redhat.com'
> > Subject: Mutex implementation questions
> >
> > I noticed that if _pthread_try_enter_critical_section has been set
> > to a non-NULL value by DllMain, Mutexes are implemented using
> > critical sections (InitializeCriticalSection) instead of
> > CreateMutex, regardless of the value of the implemetation-specific
> > forcecs
> > mutex attribute.
> >
> > According to "Win32 programming", critical sections are light weight
> > compared to mutexes, they are not managed by the kernel, and they
> > are much faster than mutexes.  So why no use critical sections
> > all the time to implement pthread mutexes ?
> >
> > Are there cases where critical sections are not available or
> > wouldn't work well ?
> >
> > The DllMain seems to do some tests to check if InitializeCriticalSection
> > exists and works.  What are the cases where critical sections
> > would not be available ?
> >
> > Also, I have a suggestion:
> >
> > It might be a good idea to add a compile flag to
> > allow the use of pthread-win32 with static linking
> > (i.e. to make just a pthread.lib, no dll).
> >
> > In this case, a compilation flag should be added to exclude the DllMain
> > routine. Also, the code that sets _pthread_try_enter_critical_section
> > should be moved from DllMain to _pthread_processInitialize.  The
> > _pthread_processInitialize should be made global and it should
> > be called by the application to initialize pthread.
> >
> > We had to change the pthread code to do all that in our
> > WinCE application.
> >
> > -t
> 
> --
> Regards, -- Tristan Savatier (President, MpegTV LLC)
> 
> MpegTV:   http://www.mpegtv.com   - Tel: (415) 864 6466
> PocketTV: http://www.pockettv.com - Fax: (415) 704 3224

-- 
+----------------------+---+
| Ross Johnson         |   | E-Mail: rpj@ise.canberra.edu.au
| Info Sciences and Eng|___|
| University of Canberra   | FAX:    +61 6 2015227
| PO Box 1                 |
| Belconnen  ACT    2616   | WWW:    http://willow.canberra.edu.au/~rpj/
| AUSTRALIA                |
+--------------------------+

Re: Mutex implementation questions

[Ross Johnson]

With regard to possibly eliminating dllMain for applications
linking pthread*.lib statically - rather than require all threads
to explicitly call pthread_exit(), is there any reason why the
DLL_THREAD_DETACH code in dllMain can't be moved to
private.c:ptw32_threadStart(), which is the wrapper for the
applications threads? It looks doable to me. That leaves
just the DLL_PROCESS_DETACH code which could be called
explicitly as suggested.

Ross

Tristan Savatier wrote:
> 
> Bossom, John wrote:
> > 
> > 4) By eliminating the DllMain, whose purpose was to perform
> >    behind the scenes cleanup of the "self" structure as well as calling
> >    all the destructors for thread-specific data keys, what alternate
> >    approach did you take for doing this thread-based cleanup?
> >    (i.e. I leveraged the fact that DllMain is called each time a thread
> >          is started/finished to perform thread based initialization/cleanup)
> 
> You are right. One thing that can be done is to make sure that all
> thread
> terminate by calling pthread_exit(void *), and do not "fall through the
> bottom"
> of the threan entry-point routine (which is allowed by the standard).
> 
> That's what we do, and to make it work, we modified pthread_exit
> by replacing:
> 
>   _pthread_callUserDestroyRoutines((pthread_t)
> pthread_getspecific(_pthread_selfThreadKey));
> 
> by:
>   _pthread_threadDestroy((pthread_t)
> pthread_getspecific(_pthread_selfThreadKey));
> 
> which calls _pthread_callUserDestroyRoutines and then does the necessary
> cleanup
> of the thread TSD.
> 
> That takes are of the thread cleanup.  Regarding the process cleanup, I
> suppose
> that _pthread_processTerminate() should be called explicitely if the dll
> is not used.
> 
> Now, maybe you wonder why we prefer to link statically rather that
> to use the DLL (pthread.dll).  The reason is that Windows does not
> support
> dll versionning, i.e. if several applications rely on pthread.dll, but
> on various versions of it, things break.  There would be a solution
> which is
> to place the dll in the same directory as the application, rather than
> in
> the \Windows directory, but this also breaks due to the stupid rule used
> by Windows to look for dll's: first it looks in \Windows, then in the
> other places... which means that if another application places an
> incompatible pthread.dll in \windows, it will break you application
> (we had the experience with applications using xaudio.dll).
> 
> And pthread is so small that the gain of having it shared (compared
> to the trouble) is small.
> 
> That's why we think that using pthread.dll is a BAD idea, and that
> static linking with pthread.lib is a safer solution...
> 
> -t
> 
> > I hope this clears up some of your questions.
> >
> > Cheers,
> >
> > John.
> >
> > -----Original Message-----
> > From: Tristan Savatier [ mailto:tristan@mpegtv.com ]
> > Sent: Monday, October 02, 2000 6:10 PM
> > To: 'pthreads-win32@sources.redhat.com'
> > Subject: Mutex implementation questions
> >
> > I noticed that if _pthread_try_enter_critical_section has been set
> > to a non-NULL value by DllMain, Mutexes are implemented using
> > critical sections (InitializeCriticalSection) instead of
> > CreateMutex, regardless of the value of the implemetation-specific
> > forcecs
> > mutex attribute.
> >
> > According to "Win32 programming", critical sections are light weight
> > compared to mutexes, they are not managed by the kernel, and they
> > are much faster than mutexes.  So why no use critical sections
> > all the time to implement pthread mutexes ?
> >
> > Are there cases where critical sections are not available or
> > wouldn't work well ?
> >
> > The DllMain seems to do some tests to check if InitializeCriticalSection
> > exists and works.  What are the cases where critical sections
> > would not be available ?
> >
> > Also, I have a suggestion:
> >
> > It might be a good idea to add a compile flag to
> > allow the use of pthread-win32 with static linking
> > (i.e. to make just a pthread.lib, no dll).
> >
> > In this case, a compilation flag should be added to exclude the DllMain
> > routine. Also, the code that sets _pthread_try_enter_critical_section
> > should be moved from DllMain to _pthread_processInitialize.  The
> > _pthread_processInitialize should be made global and it should
> > be called by the application to initialize pthread.
> >
> > We had to change the pthread code to do all that in our
> > WinCE application.
> >
> > -t
> 
> --
> Regards, -- Tristan Savatier (President, MpegTV LLC)
> 
> MpegTV:   http://www.mpegtv.com   - Tel: (415) 864 6466
> PocketTV: http://www.pockettv.com - Fax: (415) 704 3224

-- 
+----------------------+---+
| Ross Johnson         |   | E-Mail: rpj@ise.canberra.edu.au
| Info Sciences and Eng|___|
| University of Canberra   | FAX:    +61 6 2015227
| PO Box 1                 |
| Belconnen  ACT    2616   | WWW:    http://willow.canberra.edu.au/~rpj/
| AUSTRALIA                |
+--------------------------+

Re: Mutex implementation questions

[Tristan Savatier]

Bossom, John wrote:
> 
> You are correct... critical sections are almost 100 times faster than
> actual kernel based mutexes... however...

That's right. Regarding pthread-win32 running on WinCE (PocketPC),
I was able to get a very significant gain of performances on
the overall application (5% to 15%) now that our mutexes use
critical sections (TryEnterCriticalSection works on WinCE 3.0 / Pocket
PC).

Note that we had to make a minor change to fix mutexes on WinCE:

#ifdef UNDER_CE
	  _pthread_h_kernel32 = LoadLibrary(TEXT("COREDLL.DLL"));
#else
	  _pthread_h_kernel32 = LoadLibrary(TEXT("KERNEL32.DLL"));
#endif

Because on WinCE, the symbol TryEnterCriticalSection is part of
COREDLL.DLL (there is no KERNEL32.DLL).

> 1) Mutexes (i.e. kernel based) are useful in that you can support a
>    lock-with-timeout... a feature that is being pondered as an upgrade
>    to pthreads.

true.

> 2) Mutexes can use WaitForMultipleObjects (sp?) which allows you to simulate
>    a cancel by waiting on a cancel event as well as the mutex.

true, but in some cases cancellation is not a problem, and performances
is
critical.

> 3) TryEnterCriticalSection is not supported on Win95/98 (originally it was
>    completely left out... then they added a stub that simply returns
>    "function not supported", thus there is no way to implement
>    pthread_mutex_trylock; hence if you need "trylock" you need to use a
>    real mutex.

true. but it works on NT/2000 (and WinCE 3.0)

> 4) By eliminating the DllMain, whose purpose was to perform
>    behind the scenes cleanup of the "self" structure as well as calling
>    all the destructors for thread-specific data keys, what alternate
>    approach did you take for doing this thread-based cleanup?
>    (i.e. I leveraged the fact that DllMain is called each time a thread
>          is started/finished to perform thread based initialization/cleanup)

You are right. One thing that can be done is to make sure that all
thread
terminate by calling pthread_exit(void *), and do not "fall through the
bottom"
of the threan entry-point routine (which is allowed by the standard).

That's what we do, and to make it work, we modified pthread_exit
by replacing:

  _pthread_callUserDestroyRoutines((pthread_t)
pthread_getspecific(_pthread_selfThreadKey));

by: 
  _pthread_threadDestroy((pthread_t)
pthread_getspecific(_pthread_selfThreadKey));

which calls _pthread_callUserDestroyRoutines and then does the necessary
cleanup
of the thread TSD.

That takes are of the thread cleanup.  Regarding the process cleanup, I
suppose
that _pthread_processTerminate() should be called explicitely if the dll
is not used.

Now, maybe you wonder why we prefer to link statically rather that
to use the DLL (pthread.dll).  The reason is that Windows does not
support
dll versionning, i.e. if several applications rely on pthread.dll, but
on various versions of it, things break.  There would be a solution
which is
to place the dll in the same directory as the application, rather than
in
the \Windows directory, but this also breaks due to the stupid rule used
by Windows to look for dll's: first it looks in \Windows, then in the
other places... which means that if another application places an
incompatible pthread.dll in \windows, it will break you application
(we had the experience with applications using xaudio.dll).

And pthread is so small that the gain of having it shared (compared
to the trouble) is small.

That's why we think that using pthread.dll is a BAD idea, and that
static linking with pthread.lib is a safer solution...

-t

> I hope this clears up some of your questions.
> 
> Cheers,
> 
> John.
> 
> -----Original Message-----
> From: Tristan Savatier [ mailto:tristan@mpegtv.com ]
> Sent: Monday, October 02, 2000 6:10 PM
> To: 'pthreads-win32@sources.redhat.com'
> Subject: Mutex implementation questions
> 
> I noticed that if _pthread_try_enter_critical_section has been set
> to a non-NULL value by DllMain, Mutexes are implemented using
> critical sections (InitializeCriticalSection) instead of
> CreateMutex, regardless of the value of the implemetation-specific
> forcecs
> mutex attribute.
> 
> According to "Win32 programming", critical sections are light weight
> compared to mutexes, they are not managed by the kernel, and they
> are much faster than mutexes.  So why no use critical sections
> all the time to implement pthread mutexes ?
> 
> Are there cases where critical sections are not available or
> wouldn't work well ?
> 
> The DllMain seems to do some tests to check if InitializeCriticalSection
> exists and works.  What are the cases where critical sections
> would not be available ?
> 
> Also, I have a suggestion:
> 
> It might be a good idea to add a compile flag to
> allow the use of pthread-win32 with static linking
> (i.e. to make just a pthread.lib, no dll).
> 
> In this case, a compilation flag should be added to exclude the DllMain
> routine. Also, the code that sets _pthread_try_enter_critical_section
> should be moved from DllMain to _pthread_processInitialize.  The
> _pthread_processInitialize should be made global and it should
> be called by the application to initialize pthread.
> 
> We had to change the pthread code to do all that in our
> WinCE application.
> 
> -t

-- 
Regards, -- Tristan Savatier (President, MpegTV LLC)

MpegTV:   http://www.mpegtv.com   - Tel: (415) 864 6466
PocketTV: http://www.pockettv.com - Fax: (415) 704 3224

RE: Mutex implementation questions

[Bossom, John]

You are correct... critical sections are almost 100 times faster than
actual kernel based mutexes... however...

1) Mutexes (i.e. kernel based) are useful in that you can support a
   lock-with-timeout... a feature that is being pondered as an upgrade
   to pthreads.
2) Mutexes can use WaitForMultipleObjects (sp?) which allows you to simulate
   a cancel by waiting on a cancel event as well as the mutex.
3) TryEnterCriticalSection is not supported on Win95/98 (originally it was
   completely left out... then they added a stub that simply returns
   "function not supported", thus there is no way to implement
   pthread_mutex_trylock; hence if you need "trylock" you need to use a
   real mutex.
3) By eliminating the DllMain, whose purpose was to perform
   behind the scenes cleanup of the "self" structure as well as calling
   all the destructors for thread-specific data keys, what alternate
   approach did you take for doing this thread-based cleanup?
   (i.e. I leveraged the fact that DllMain is called each time a thread
         is started/finished to perform thread based initialization/cleanup)

I hope this clears up some of your questions.

Cheers,

John.

-----Original Message-----
From: Tristan Savatier [ mailto:tristan@mpegtv.com ]
Sent: Monday, October 02, 2000 6:10 PM
To: 'pthreads-win32@sources.redhat.com'
Subject: Mutex implementation questions


I noticed that if _pthread_try_enter_critical_section has been set
to a non-NULL value by DllMain, Mutexes are implemented using
critical sections (InitializeCriticalSection) instead of 
CreateMutex, regardless of the value of the implemetation-specific
forcecs
mutex attribute.

According to "Win32 programming", critical sections are light weight
compared to mutexes, they are not managed by the kernel, and they
are much faster than mutexes.  So why no use critical sections
all the time to implement pthread mutexes ?

Are there cases where critical sections are not available or
wouldn't work well ?

The DllMain seems to do some tests to check if InitializeCriticalSection
exists and works.  What are the cases where critical sections
would not be available ?

Also, I have a suggestion:

It might be a good idea to add a compile flag to
allow the use of pthread-win32 with static linking
(i.e. to make just a pthread.lib, no dll).

In this case, a compilation flag should be added to exclude the DllMain
routine. Also, the code that sets _pthread_try_enter_critical_section
should be moved from DllMain to _pthread_processInitialize.  The
_pthread_processInitialize should be made global and it should
be called by the application to initialize pthread.

We had to change the pthread code to do all that in our
WinCE application.

-t

Re: Mutex implementation questions

[Ross Johnson]

Tristan Savatier wrote:
> 
> I noticed that if _pthread_try_enter_critical_section has been set
> to a non-NULL value by DllMain, Mutexes are implemented using
> critical sections (InitializeCriticalSection) instead of
> CreateMutex, regardless of the value of the implemetation-specific
> forcecs
> mutex attribute.
> 
> According to "Win32 programming", critical sections are light weight
> compared to mutexes, they are not managed by the kernel, and they
> are much faster than mutexes.  So why no use critical sections
> all the time to implement pthread mutexes ?
> 

Win9x doesn't support TryEnterCriticalSection, hence the fallback
to Win32 mutexes on those platforms. On Win98 that function
appears to be a stub that does nothing hence the need to test it
by actually initialising a CS and attempting to acquire it.

The forcecs hook is there if your application never needs to use
pthread_mutex_trylock() and you want the performance advantage.
It could perhaps be better implemented as a non-portable per
mutex attribute (or either option could be provided).

> It might be a good idea to add a compile flag to
> allow the use of pthread-win32 with static linking
> (i.e. to make just a pthread.lib, no dll).
> 
> In this case, a compilation flag should be added to exclude the DllMain
> routine. Also, the code that sets _pthread_try_enter_critical_section
> should be moved from DllMain to _pthread_processInitialize.  The
> _pthread_processInitialize should be made global and it should
> be called by the application to initialize pthread.

This sounds like a good idea for static linked libraries.

Cheers.
Ross

-- 
+----------------------+---+
| Ross Johnson         |   | E-Mail: rpj@ise.canberra.edu.au
| Info Sciences and Eng|___|
| University of Canberra   | FAX:    +61 6 2015227
| PO Box 1                 |
| Belconnen  ACT    2616   | WWW:    http://willow.canberra.edu.au/~rpj/
| AUSTRALIA                |
+--------------------------+

Mutex implementation questions

[Tristan Savatier]

I noticed that if _pthread_try_enter_critical_section has been set
to a non-NULL value by DllMain, Mutexes are implemented using
critical sections (InitializeCriticalSection) instead of 
CreateMutex, regardless of the value of the implemetation-specific
forcecs
mutex attribute.

According to "Win32 programming", critical sections are light weight
compared to mutexes, they are not managed by the kernel, and they
are much faster than mutexes.  So why no use critical sections
all the time to implement pthread mutexes ?

Are there cases where critical sections are not available or
wouldn't work well ?

The DllMain seems to do some tests to check if InitializeCriticalSection
exists and works.  What are the cases where critical sections
would not be available ?

Also, I have a suggestion:

It might be a good idea to add a compile flag to
allow the use of pthread-win32 with static linking
(i.e. to make just a pthread.lib, no dll).

In this case, a compilation flag should be added to exclude the DllMain
routine. Also, the code that sets _pthread_try_enter_critical_section
should be moved from DllMain to _pthread_processInitialize.  The
_pthread_processInitialize should be made global and it should
be called by the application to initialize pthread.

We had to change the pthread code to do all that in our
WinCE application.

-t