Serious problem with win32 pthreads crashing and c++ class

Serious problem with win32 pthreads crashing and c++ class

[Arash Partow]

Hi All,

I'm a long-time reader, first-time poster. I've been doing some work lately
with posix threads, mainly trying to build a very simple c++ wrapper for
Posix threads, and well have come up to a stumbling point on the win32
platform with a simple prototype i had built utilizing this c++ wrapper.

On the Linux and BSD(free and open) platforms the simple prototype works
perfectly, it does not crash and does not seem to produce any zombie
processes. In one instance I've had it running for 3 days none stop(on RH-
Linux 9.0) and during this time it had created and execute over 16 million
threads.

The prototype initially creates 700 threads all of which are contained in a
vector (threadList), each thread does some "simple" string processing
(basically tokenize a string) and then exists. On completion of the thread,
the thread sets its own state in the thread-class to dead.

Another thread called GarbageCollector, which is created before the other
threads are created, is continually traversing the threadList, looking for
dead threads, once a dead thread is found, it deletes the pointer pointing
to the thread class in the list, creates a new thread and adds it to the
end of the list. Hence continually maintaining the number of threads being
run at any one time, (hopefully)

The problem is on Windows2000 and WindowsXP (with SP4) the prototype only
ever gets to about 50000 threads or so, never getting past that amount, it
always crashes. several times I've tried using GDB to figure out where its
crashing however its always seems to be crashing in different place. I've
made sure i don't use any methods from the standard libs that use globals,
and tried implementing a reentrant way of development.

I've tried reducing the number of overall threads (MAX_THREADS), down to
100, 50, 10, it still crashes its just takes longer to crash.

I've tried turning optimization on and off for GCC, nothing changes. except
for a slightly longer compile time which is due to something else. What
really baffles me is that it works well under unix platforms but falls over
on win32.


My questions are:

1.) could you please look at the code from this URL 
http://www.partow.net/downloads/thread-question.zip
    and tell me if any of you can replicate what i am seeing.

2.) if there is a problem, is the problem from my code or is it from the
    implementation of win32 pthreads, or is it a windows issue?

3.) is the general way the Thread class been written correct, or is there a
    better way to do it?

4.) Is there a better way to implement a Thread class?


My setup is as follows:

1.) GCC version 3.3.1 (cygming special)
2.) P4 2.4, 1024GB DDR
3.) Windows 2000


My intention is to hopefully extent the class base and combine it with some
sort of sockets class the implement a per-thread-per-connection based
client/server system for doing something interesting. I'm not looking for
windows based implementations cause I'm trying to keep the code as portable
as possible.

Any help regarding this matter would be very much appreciated.


Regards


Arash


PS: To make and run the source code, you gotta have GCC installed 2.95 or
    later plus win32-pthreads installed. Other than that type:
    make ThreadTest
    ThreadTest


__________________________________________________
Be one who knows what they don't know,
Instead of being one who knows not what they don't know,
Thinking they know everything about all things.
http://www.partow.net

_________________________________________________________________
Protect your inbox from harmful viruses with new ninemsn Premium. Click here 
  http://ninemsn.com.au/premium/landing.asp

Re: Serious problem with win32 pthreads crashing and c++ class

[Ross Johnson]

Looking at your code, I see that you have [when starting a new thread]:

pthread_create(&threadID, ...);
pthread_detach(&threadID);

In pthreads-win32, pthread_detach() just sets a flag in the thread, and 
if the
thread is created, runs and completes before pthread_detach() can set 
the flag, the thread
will never be properly cleaned up, and possibly not even close the Win32 
handle.

If this is the case then you could try creating detached threads using a 
suitably
initialised and set pthread_attr_t as arg 2 in the call to 
pthread_create(). I.e.

pthread_attr_t attr;
pthread_attr_init(&attr);
pthread_attr_setdetachedstate(&attr, PTHREAD_CREATE_DETACHED);
...
pthread_create(&threadID, &attr, ...);

If that stops your test code from crashing then it remains to fix 
pthreads-win32's current
behaviour. It might simply need changing the detachstate flag into a 
semaphore in pthread_t
('sem_join' say). The thread exit logic might then just call 
sem_wait(&thr->sem_join)
after running the cleanup handlers and tsd destructors. Pthread_detach() 
would just call
sem_post(&thr->sem_join), as would pthread_join(), but after recording 
the thread's return
value.

Ross

Arash Partow wrote:

> Hi All,
>
> I'm a long-time reader, first-time poster. I've been doing some work 
> lately
> with posix threads, mainly trying to build a very simple c++ wrapper for
> Posix threads, and well have come up to a stumbling point on the win32
> platform with a simple prototype i had built utilizing this c++ wrapper.
>
> On the Linux and BSD(free and open) platforms the simple prototype works
> perfectly, it does not crash and does not seem to produce any zombie
> processes. In one instance I've had it running for 3 days none stop(on 
> RH-
> Linux 9.0) and during this time it had created and execute over 16 
> million
> threads.
>
> The prototype initially creates 700 threads all of which are contained 
> in a
> vector (threadList), each thread does some "simple" string processing
> (basically tokenize a string) and then exists. On completion of the 
> thread,
> the thread sets its own state in the thread-class to dead.
>
> Another thread called GarbageCollector, which is created before the other
> threads are created, is continually traversing the threadList, looking 
> for
> dead threads, once a dead thread is found, it deletes the pointer 
> pointing
> to the thread class in the list, creates a new thread and adds it to the
> end of the list. Hence continually maintaining the number of threads 
> being
> run at any one time, (hopefully)
>
> The problem is on Windows2000 and WindowsXP (with SP4) the prototype only
> ever gets to about 50000 threads or so, never getting past that 
> amount, it
> always crashes. several times I've tried using GDB to figure out where 
> its
> crashing however its always seems to be crashing in different place. I've
> made sure i don't use any methods from the standard libs that use 
> globals,
> and tried implementing a reentrant way of development.
>
> I've tried reducing the number of overall threads (MAX_THREADS), down to
> 100, 50, 10, it still crashes its just takes longer to crash.
>
> I've tried turning optimization on and off for GCC, nothing changes. 
> except
> for a slightly longer compile time which is due to something else. What
> really baffles me is that it works well under unix platforms but falls 
> over
> on win32.
>
>
> My questions are:
>
> 1.) could you please look at the code from this URL 
> http://www.partow.net/downloads/thread-question.zip
>    and tell me if any of you can replicate what i am seeing.
>
> 2.) if there is a problem, is the problem from my code or is it from the
>    implementation of win32 pthreads, or is it a windows issue?
>
> 3.) is the general way the Thread class been written correct, or is 
> there a
>    better way to do it?
>
> 4.) Is there a better way to implement a Thread class?
>
>
> My setup is as follows:
>
> 1.) GCC version 3.3.1 (cygming special)
> 2.) P4 2.4, 1024GB DDR
> 3.) Windows 2000
>
>
> My intention is to hopefully extent the class base and combine it with 
> some
> sort of sockets class the implement a per-thread-per-connection based
> client/server system for doing something interesting. I'm not looking for
> windows based implementations cause I'm trying to keep the code as 
> portable
> as possible.
>
> Any help regarding this matter would be very much appreciated.
>
>
> Regards
>
>
> Arash
>
>
> PS: To make and run the source code, you gotta have GCC installed 2.95 or
>    later plus win32-pthreads installed. Other than that type:
>    make ThreadTest
>    ThreadTest
>
>
> __________________________________________________
> Be one who knows what they don't know,
> Instead of being one who knows not what they don't know,
> Thinking they know everything about all things.
> http://www.partow.net
>
> _________________________________________________________________
> Protect your inbox from harmful viruses with new ninemsn Premium. 
> Click here  http://ninemsn.com.au/premium/landing.asp