RE: Return values

RE: Return values

[Hujsak, Jonathan T]

Ross, 

That was a amazingly diplomatic response. The flames from
that post almost melted my inbox. It's hard for anyone to
imagine that such an oversight would have been intentional 
given the implicit difficulty of developing pthreads on 
win32 in the first place (most threading experts I work 
with insisted it could not be done). There is hope for NT 
yet.

We continue to use your work with great success.

--Jon Hujsak/Marconi Integrated Systems

-----Original Message-----
From: Ross Johnson [ mailto:rpj@ise.canberra.edu.au ]
Sent: Wednesday, August 18, 1999 9:31 PM
To: Milan Gardian
Cc: John.Bossom@Cognos.COM; pthreads-win32@sourceware.cygnus.com
Subject: Re: Return values


On Wed, 18 Aug 1999, Milan Gardian wrote:

> I came across this "feature" of your pthread implementation some time ago
> (and it is another reason why I don't trust your pthreads at all) - the
> value returned by a thread function IS BEING IGNORED!!! I think every
decent
> implementation of threads should care about the value (void *) that si
> returned by thread functions (why else would the pointer to thread
function
> be declared as "returning pointer to void", ha?). It is not the case of
your
> pthreads. Please take a look at:
> ---
> file: pthreads/private.c
> line: 192
> 
> Snip from the file, lines 187 to 194; Visual C++:
> __try
> {
>   /*
>    * Run the caller's routine;
>    */
>   (*start) (arg);
>   status = (void *) 0;
> }
> ---
> CAN you please EXPLAIN why the hell have you done it this way? Is there a
> reason? WHY do you set status value apriori to ZERO??? Why not
> status = (*start) (arg);

Dear Milan,

Yes, this would appear to be an obvious bug. I'll make the change
and credit you with finding it. Perhaps it was a case of not seeing
the trees because the forest was in the way, as distinct from not
seeing the forest because the trees are in the way.

Thanks also for providing your test code, which I will add to the
test suite.

This is an active project and it relies heavily on people like
yourself using it in the real world and reporting problems. I hope
that as you look through the code you'll find that it is really a
very well conceived implementation of POSIX threads on top of Win32
threads.

Any other problems or suggestions for improvement will be more than
happily received.

Cheers.
Ross

+----------------------+---+
| Ross Johnson         |   | E-Mail: rpj@ise.canberra.edu.au
| Info Sciences and Eng|___|
| University of Canberra   | FAX:    +61 6 2015227
| PO Box 1                 |
| Belconnen  ACT    2616   | WWW:    http://willow.canberra.edu.au/~rpj/
| AUSTRALIA                |
+--------------------------+

pthread-win32: WinCE port

[Tristan Savatier]

Some interesting news:

I have been able to port pthread-win32 to Windows-CE,
which uses a subset of the WIN32 API.

Since we intend to keep using pthread-win32 for our
commercial WinCE developments, I would be very interested
if WinCE support could be added to the main source tree
of pthread-win32.  Also, I would like to be credited
for this port :-)

Now, here is the story...

The port was performed and tested on a Casio "Cassiopeia"
PalmSize PC, which runs a MIP processor.  The OS in the
Casio is WinCE version 2.11, but I used VC++ 6.0 with
the WinCE SDK for version 2.01.

I used pthread-win32 to port a heavily multithreaded
commercial application (real-time MPEG video player)
from Linux to WinCE.  I consider the changes that
I have done to be quite well tested.

Overall the modifications that we had to do are minor.

The WinCE port were based on pthread-win32-snap-1999-05-30,
but I am certain that they can be integrated very easiely
to more recent versions of the source.

I have attached the modified source code:
pthread-win32-snap-1999-05-30-WinCE.

All the changes do not affect the code compiled on non-WinCE
environment, provided that the macros used for WinCE compilation
are not used, of course!

Overall description of the WinCE port:
-------------------------------------

Most of the changes had to be made in areas where
pthread-win32 was relying on some standard-C librairies
(e.g. _ftime, calloc, errno), which are not available
on WinCE. We have changed the code to use native Win32
API instead (or in some cases we made wrappers).

The Win32 Semaphores are not available,
so we had to re-implement Semaphores using mutexes
and events.

Limitations / known problems of the WinCE port:
----------------------------------------------

Not all the semaphore routines have been ported
(semaphores are defined by Posix but are not part
pf pthread).  I have just done enough to make
pthread routines (that rely internally on semaphores)
work, like signal conditions.

I noticed that the Win32 threads work slightly
differently on WinCE.  This may have some impact
on some tricky parts of pthread-win32, but I have
not really investigated.  For example, on WinCE,
the process is killed if the main thread falls off
the bottom (or calls pthread_exit), regardless
of the existence of any other detached thread.
Microsoft manual indicates that this behavior is
deffirent from that of Windows Threads for other
Win32 platforms.


Detailed descriptions of the changes and rationals:

------------------------------------
- use a new macro NO_ERRNO.

If defined, the code in errno.c that defines a reentrant errno
is compiled, regardless of _MT and _REENTRANT.

Rational: On WinCE, there is no support for <stdio.h>, <errno.h> or
any other standard C library, i.e. even if _MT or _REENTRANT
is defined, errno is not provided by any library.  NO_ERRNO
must be set to compile for WinCE.

------------------------------------
- In implement.h, change #include <semaphore.h> to #include
"semaphore.h".

Rational: semaphore.h is provided in pthread-win32 and should not
be searched in the systems standard include.  would not compile.
This change does not seem to create problems on "classic" win32
(e.g. win95).

------------------------------------
- use a new macro NO_CALLOC.

If defined, some code in misc.c will provide a replacement
for calloc, which is not available on Win32.


------------------------------------
- use a new macro _PTHREAD_CREATETHREAD.

If defined, implement.h defined the macro _beginthreadex
and _endthreadex, regardless of the value of __CYGWIN32__
and __CYGWIN__.

Rational: On WinCE, the wrappers _beginthreadex and _endthreadex
do not exist. The native Win32 routines must be used.

------------------------------------
- in misc.c:

#ifdef UNDER_CE
	  /* DuplicateHandle does not exist on WinCE */
	  self->threadH = GetCurrentThread();
#else
	  if( !DuplicateHandle(
			       GetCurrentProcess(),
			       GetCurrentThread(),
			       GetCurrentProcess(),
			       &self->threadH,
			       0,
			       FALSE,
			       DUPLICATE_SAME_ACCESS ) )
	    {
	      free( self );
	      return (NULL);
	    }
#endif

Rational: On WinCE, DuplicateHandle does not exist.  I could not
understand
why DuplicateHandle must be used.  It seems to me that getting the
current
thread handle with GetCurrentThread() is sufficient, and it seems to
work
perfectly fine, so maybe DuplicateHandle was just plain useless to begin
with ?

------------------------------------
- In private.c, added some code at the beginning of
_pthread_processInitialize
to detect the case of multiple calls to _pthread_processInitialize.

Rational: In order to debug pthread-win32, it is easier to compile
it as a regular library (it is not possible to debug DLL's on winCE).
In that case, the application must call _pthread_processInitialize()
explicitely, to initialize pthread-win32.  It is safer in this
circumstance
to handle the case where _pthread_processInitialize() is called on
an already initialized library:

int
_pthread_processInitialize (void)
{
	if (_pthread_processInitialized) {
		/* 
		 * ignore if already initialized. this is useful for 
		 * programs that uses a non-dll pthread
		 * library. such programs must call _pthread_processInitialize()
explicitely,
		 * since this initialization routine is automatically called only when
		 * the dll is loaded.
		 */
		return TRUE;
	}
    _pthread_processInitialized = TRUE;
  	[...]
}

------------------------------------
- in private.c, if macro NO_FTIME is defined, add routines to
convert timespec_to_filetime and filetime_to_timespec, and modified
code that was using _ftime() to use Win32 API instead.

Rational: _ftime is not available on WinCE.  It is necessary to use
the native Win32 time API instead.

Note: the routine timespec_to_filetime is provided as a convenience and
a mean
to test that filetime_to_timespec works, but it is not used by the
library.

------------------------------------
- in semaphore.c, if macro NO_SEM is defined, add code for the routines
_increase_semaphore and _decrease_semaphore, and modify significantly
the implementation of the semaphores so that it does not use
CreateSemaphore.

Rational: CreateSemaphore is not available on WinCE.  I had to
re-implement
semaphores using mutexes and Events.

Note: Only the semaphore routines that are used by pthread are
implemented
(i.e. signal conditions rely on a subset of the semaphores routines, and
this subset works). Some other semaphore routines (e.g. sem_trywait) are
not yet supported on my WinCE port (and since I don't need them, I am
not
planning to do anything about them).

------------------------------------
- in tsd.c, changed the code that defines TLS_OUT_OF_INDEXES

/* TLS_OUT_OF_INDEXES not defined on WinCE */
#ifndef TLS_OUT_OF_INDEXES
#define TLS_OUT_OF_INDEXES 0xffffffff
#endif

Rational: TLS_OUT_OF_INDEXES is not defined in any standard include file
on WinCE.

------------------------------------
- added file pthread_errno.h

Rational: On WinCE, there is no errno.h file. pthread_errno.h is just a
copy of windows version of errno.h, with minor modifications due to the
fact
that some of the error codes are defined by the WinCE socket library.
In pthread.h, if NO_ERRNO is defined, the file pthread_errno.h is
included (instead of <errno.h>).


-- 
Regards, -- Tristan Savatier (President, MpegTV LLC)

MpegTV: http://www.mpegtv.com - Xaudio: http://www.xaudio.com
MPEG.ORG: http://www.mpeg.org - Tel: (415) 864 6466
pthreads-win32-snap-1999-05-30-WinCE.tgz

Re: Return values

[Ross Johnson]

On Wed, 18 Aug 1999, Milan Gardian wrote:

> I came across this "feature" of your pthread implementation some time ago
> (and it is another reason why I don't trust your pthreads at all) - the
> value returned by a thread function IS BEING IGNORED!!! I think every decent
> implementation of threads should care about the value (void *) that si
> returned by thread functions (why else would the pointer to thread function
> be declared as "returning pointer to void", ha?). It is not the case of your
> pthreads. Please take a look at:
> ---
> file: pthreads/private.c
> line: 192
> 
> Snip from the file, lines 187 to 194; Visual C++:
> __try
> {
>   /*
>    * Run the caller's routine;
>    */
>   (*start) (arg);
>   status = (void *) 0;
> }
> ---
> CAN you please EXPLAIN why the hell have you done it this way? Is there a
> reason? WHY do you set status value apriori to ZERO??? Why not
> status = (*start) (arg);

Dear Milan,

Yes, this would appear to be an obvious bug. I'll make the change
and credit you with finding it. Perhaps it was a case of not seeing
the trees because the forest was in the way, as distinct from not
seeing the forest because the trees are in the way.

Thanks also for providing your test code, which I will add to the
test suite.

This is an active project and it relies heavily on people like
yourself using it in the real world and reporting problems. I hope
that as you look through the code you'll find that it is really a
very well conceived implementation of POSIX threads on top of Win32
threads.

Any other problems or suggestions for improvement will be more than
happily received.

Cheers.
Ross

+----------------------+---+
| Ross Johnson         |   | E-Mail: rpj@ise.canberra.edu.au
| Info Sciences and Eng|___|
| University of Canberra   | FAX:    +61 6 2015227
| PO Box 1                 |
| Belconnen  ACT    2616   | WWW:    http://willow.canberra.edu.au/~rpj/
| AUSTRALIA                |
+--------------------------+

RE: Return values

[Bossom, John]

Dear Milan,

First of all, thank you for finding a flaw in the implementation.
I am sure that Ross will duely note your contribution
and make the appropriate change.

However, I am quite offended by your method and
tone of reporting a problem. Your level of maturity 
and professionalism in communicating with other people
leaves much to be desired.

John.


-----Original Message-----
From: Milan Gardian [ mailto:mg@tatramed.sk ]
Sent: Wednesday, August 18, 1999 7:41 AM
To: John.Bossom@cognos.com; rpj@ise.canberra.edu.au
Cc: pthreads-win32@sourceware.cygnus.com
Subject: Return values


> With regards to the return code not working...
> In my original contributed work, the pthread_t structure contained
> an unused attribute, "exitStatus" that can be used precisely for
> the purpose of implementing the return code.
>
> I didn't bother to use it since _endthreadex and GetExitCodeThread
> did the trick just fine for WIN32.
>
> However, if you need to use an alternative method, you simply can
> do the following:
>
> 1) pthread_exit - stuff the result in t->exitStatus.
> 2) In _pthread_threadStart (?) if the user's thread routine actually
>    returned, then stuff the return value in t->exitStatus.
> 3) In pthread_join, if the thread had terminated, simply extract
>    the exit status from the instance of the thread structure.
[SNIP]

I came across this "feature" of your pthread implementation some time ago
(and it is another reason why I don't trust your pthreads at all) - the
value returned by a thread function IS BEING IGNORED!!! I think every decent
implementation of threads should care about the value (void *) that si
returned by thread functions (why else would the pointer to thread function
be declared as "returning pointer to void", ha?). It is not the case of your
pthreads. Please take a look at:
---
file: pthreads/private.c
line: 192

Snip from the file, lines 187 to 194; Visual C++:
__try
{
  /*
   * Run the caller's routine;
   */
  (*start) (arg);
  status = (void *) 0;
}
---
CAN you please EXPLAIN why the hell have you done it this way? Is there a
reason? WHY do you set status value apriori to ZERO??? Why not
status = (*start) (arg);

Attached please find a simple program that tests return values (MSVC6
project) acquired by 'pthread_join' call. One thread (1) returns a value
directly using return statement, the other one (2) returns another value
using 'pthread_exit'. I have run it on both WinNT machine and UNIX machine.
Here are the outputs:

---
Platform: M$ WinNT 4, SP5
Compiler: M$ Visual C++ 6, no SP
pthreads: Pthreads-win32 snapshot 1999-08-12
- (output) -
Creating thread 1
Creating thread 2
Worker thread 1 running (returning 10)
Worker thread 2 running (returning 20)
Thread 1 returned 0
Thread 2 returned 20

Using Win32 'CreateThread' to create thread 1
Worker thread 1 running (returning 10)
Using Win32, thread 1 returned 10


---
Platform: DIGITAL UNIX V4.0 (Rev. 564)
Compiler: DIGITAL C++ V6.0-010
pthreads: Default DIGITAL pthreads
- (output) -
Creating thread 1
Creating thread 2
Worker thread 1 running (returning 10)
Worker thread 2 running (returning 20)
Thread 1 returned 10
Thread 2 returned 20
---

I have used the same "ReturnValue.cpp" file for both Win32 and UNIX version
of the compiled program (I used "cxx -g
ReturnValue.cpp -pthread -D__USE_STD_IOSTREAM" line to compile it in UNIX).
Please note that in your pthreads, join to thread 1 returns zero (and no
suprise...), while on UNIX it returns correct value -> you have a SERIOUS
FLAW in the BASIC functionality of pthreads (if somebody relies on return
values from threads (and not solving it with this NASTY pthread_exit)...
they are out of luck with your implementation).

Please take a look at this issue,
Best regards,
	Milan Gardian

Return values

[Milan Gardian]

> With regards to the return code not working...
> In my original contributed work, the pthread_t structure contained
> an unused attribute, "exitStatus" that can be used precisely for
> the purpose of implementing the return code.
>
> I didn't bother to use it since _endthreadex and GetExitCodeThread
> did the trick just fine for WIN32.
>
> However, if you need to use an alternative method, you simply can
> do the following:
>
> 1) pthread_exit - stuff the result in t->exitStatus.
> 2) In _pthread_threadStart (?) if the user's thread routine actually
>    returned, then stuff the return value in t->exitStatus.
> 3) In pthread_join, if the thread had terminated, simply extract
>    the exit status from the instance of the thread structure.
[SNIP]

I came across this "feature" of your pthread implementation some time ago
(and it is another reason why I don't trust your pthreads at all) - the
value returned by a thread function IS BEING IGNORED!!! I think every decent
implementation of threads should care about the value (void *) that si
returned by thread functions (why else would the pointer to thread function
be declared as "returning pointer to void", ha?). It is not the case of your
pthreads. Please take a look at:
---
file: pthreads/private.c
line: 192

Snip from the file, lines 187 to 194; Visual C++:
__try
{
  /*
   * Run the caller's routine;
   */
  (*start) (arg);
  status = (void *) 0;
}
---
CAN you please EXPLAIN why the hell have you done it this way? Is there a
reason? WHY do you set status value apriori to ZERO??? Why not
status = (*start) (arg);

Attached please find a simple program that tests return values (MSVC6
project) acquired by 'pthread_join' call. One thread (1) returns a value
directly using return statement, the other one (2) returns another value
using 'pthread_exit'. I have run it on both WinNT machine and UNIX machine.
Here are the outputs:

---
Platform: M$ WinNT 4, SP5
Compiler: M$ Visual C++ 6, no SP
pthreads: Pthreads-win32 snapshot 1999-08-12
- (output) -
Creating thread 1
Creating thread 2
Worker thread 1 running (returning 10)
Worker thread 2 running (returning 20)
Thread 1 returned 0
Thread 2 returned 20

Using Win32 'CreateThread' to create thread 1
Worker thread 1 running (returning 10)
Using Win32, thread 1 returned 10


---
Platform: DIGITAL UNIX V4.0 (Rev. 564)
Compiler: DIGITAL C++ V6.0-010
pthreads: Default DIGITAL pthreads
- (output) -
Creating thread 1
Creating thread 2
Worker thread 1 running (returning 10)
Worker thread 2 running (returning 20)
Thread 1 returned 10
Thread 2 returned 20
---

I have used the same "ReturnValue.cpp" file for both Win32 and UNIX version
of the compiled program (I used "cxx -g
ReturnValue.cpp -pthread -D__USE_STD_IOSTREAM" line to compile it in UNIX).
Please note that in your pthreads, join to thread 1 returns zero (and no
suprise...), while on UNIX it returns correct value -> you have a SERIOUS
FLAW in the BASIC functionality of pthreads (if somebody relies on return
values from threads (and not solving it with this NASTY pthread_exit)...
they are out of luck with your implementation).

Please take a look at this issue,
Best regards,
	Milan Gardian