RE: snap-2004-11-03 breakage

RE: snap-2004-11-03 breakage

[Bossom, John]

I will awaken from my stealth lurking
(Yes Ross, I'm still here!)

The whole goal of the pthreads-win32 initiative is to provide
an implemenation of the POSIX Threads library that is true to
the standard in order to permit code portability from UNIX (& (UNIX-like i.e linux) to Win32.

Thus, any deviation from the standard will render code leveraging
these features non-portable, especially if they are making any assumptions as to the implementation
of this library.

There are implementations of PThreads on unix that use scalar values for ID's,
there are also implementations that use non-scalar ids...


Cheers, (and back to lurking...)

John E. Bossom

P.S. The original implementation of pthreads-win32 made all ID's completely opaque pointers in
order to ensure that a product that leverages the DLL doesn't have to be re-built in order to
pick up a bug fix (i.e. just drop in the fixed DLL).


-----Original Message-----
From: pthreads-win32-owner@sources.redhat.com
[mailto:pthreads-win32-owner@sources.redhat.com]On Behalf Of Ross
Johnson
Sent: Wednesday, November 03, 2004 9:33 AM
To: pthreads-win32
Subject: Re: snap-2004-11-03 breakage


Hi,

I definitely don't want to break applications unnecessarily and 
particularly applications that have been ported to several different 
POSIX systems, but on the other hand POSIX thread IDs are not required 
to be scalar. See the rationale from the definition of pthread_equal() 
in the Single Unix Specification version 3, which says:


        RATIONALE

    Implementations may choose to define a thread ID as a structure.
    This allows additional flexibility and robustness over using an
    *int*. For example, a thread ID could include a sequence number that
    allows detection of "dangling IDs" (copies of a thread ID that has
    been detached). Since the C language does not support comparison on
    structure types, the /pthread_equal/() function is provided to
    compare thread IDs.

I don't think this is new to SUS or SUS version 3 either.

Nor (given the above) does the value 0 (zero) for a thread ID have any 
special meaning in the standard, although it obviously does as the value 
returned by ec_thread_getpid() in ettercap. If you want to test if a 
thread exists given it's thread ID, use:

    if (pthread_kill(pid, 0) == 0)  /* or (... != ESRCH) */

That should be portable to all POSIX implementations.

I'm not dismissing your problem, but in this case I think it would be 
better to fix the application.

Regards.
Ross

Gisle Vanem wrote:

> snap-2004-11-03 breaks a lot of applications by the way 'pthread_t' is 
> defined:
>
> typedef struct {
>    void * p;                   /* Pointer to actual object */
>    unsigned int x;             /* Extra information - reuse count etc */
> } ptw32_handle_t;
>
> typedef ptw32_handle_t pthread_t;
>
> Code like (from Ettercap)
>  pthread_t pid = ec_thread_getpid("golem");
>  if (pid != 0)
>    ec_thread_destroy(pid);
>
> no longer works; you cannot compare a struct against 0.
>
> I'm not sure you really meant to do that or if the typedef should be
> typedef ptw32_handle_t *pthread_t;
>
> --gv
> 
  
       This message may contain privileged and/or confidential information.  If you have received this e-mail in error or are not the intended recipient, you may not use, copy, disseminate or distribute it; do not open any attachments, delete it immediately from your system and notify the sender promptly by e-mail that you have done so.  Thank you. 
 

snap-2004-11-03 breakage

[Gisle Vanem]

snap-2004-11-03 breaks a lot of applications by the way 'pthread_t' is 
defined:

typedef struct {
    void * p;                   /* Pointer to actual object */
    unsigned int x;             /* Extra information - reuse count etc */
} ptw32_handle_t;

typedef ptw32_handle_t pthread_t;

Code like (from Ettercap)
  pthread_t pid = ec_thread_getpid("golem");
  if (pid != 0)
    ec_thread_destroy(pid);

no longer works; you cannot compare a struct against 0.

I'm not sure you really meant to do that or if the typedef should be
typedef ptw32_handle_t *pthread_t;

--gv

RE: snap-2004-11-03 breakage

[Robert Kindred]

On the other hand, having pthread_t to be a pointer forces me to put
compiler switches in my code that runs on both Windows and QNX.  I would
appreciate knowing the general direction things are taking.

Robert Kindred

-----Original Message-----
From: pthreads-win32-owner@sources.redhat.com
[mailto:pthreads-win32-owner@sources.redhat.com]On Behalf Of Gisle Vanem
Sent: Wednesday, November 03, 2004 7:18 AM
To: pthreads-win32
Subject: snap-2004-11-03 breakage


snap-2004-11-03 breaks a lot of applications by the way 'pthread_t' is
defined:

typedef struct {
    void * p;                   /* Pointer to actual object */
    unsigned int x;             /* Extra information - reuse count etc */
} ptw32_handle_t;

typedef ptw32_handle_t pthread_t;

Code like (from Ettercap)
  pthread_t pid = ec_thread_getpid("golem");
  if (pid != 0)
    ec_thread_destroy(pid);

no longer works; you cannot compare a struct against 0.

I'm not sure you really meant to do that or if the typedef should be
typedef ptw32_handle_t *pthread_t;

--gv

Re: snap-2004-11-03 breakage

[Ross Johnson]

Robert Kindred wrote:

>On the other hand, having pthread_t to be a pointer forces me to put
>compiler switches in my code that runs on both Windows and QNX.  I would
>appreciate knowing the general direction things are taking.
>
>  
>
What specifics of pthread_t are causing you to have to treat 
it differently on different systems? Applications shouldn't 
have to know how pthread_t is actually defined.

Re the general direction for the library:
The library has reached a point where it's stable, except 
for some relatively quiet issues, such as thread ID reuse, 
left to fix. Defining pthread_t as a pointer meant that a 
new thread could acquire the same ID as a recently detached 
or joined thread, which could be disasterous. There were no 
serious bug fixes to the last snapshot and no significant
fundamentally new features added.

The changes to mutexes in this snapshot are also helping 
prepare the way for [possibly] making headway on 
POSIX_PROCESS_SHARED mutexes, semaphores etc. This may still 
prove difficult, but some more definite impedements
have now been removed.

This is the first time in over 6 years, if I recall 
correctly, that the library's ABI has changed, and there are 
no further changes planned.

Regards.
Ross

>Robert Kindred
>
>-----Original Message-----
>From: pthreads-win32-owner@sources.redhat.com
>[mailto:pthreads-win32-owner@sources.redhat.com]On Behalf Of Gisle Vanem
>Sent: Wednesday, November 03, 2004 7:18 AM
>To: pthreads-win32
>Subject: snap-2004-11-03 breakage
>
>
>snap-2004-11-03 breaks a lot of applications by the way 'pthread_t' is
>defined:
>
>typedef struct {
>    void * p;                   /* Pointer to actual object */
>    unsigned int x;             /* Extra information - reuse count etc */
>} ptw32_handle_t;
>
>typedef ptw32_handle_t pthread_t;
>
>Code like (from Ettercap)
>  pthread_t pid = ec_thread_getpid("golem");
>  if (pid != 0)
>    ec_thread_destroy(pid);
>
>no longer works; you cannot compare a struct against 0.
>
>I'm not sure you really meant to do that or if the typedef should be
>typedef ptw32_handle_t *pthread_t;
>
>--gv
>
>
>  
>

Re: snap-2004-11-03 breakage

[Peter Slacik]

After looong sleep...

By the way, what means the "ABI" shortcut?

With regards
Peter Slacik

Ross Johnson wrote:

> This is the first time in over 6 years, if I recall correctly, that 
> the library's ABI has changed, and there are no further changes planned.
>
> Regards.
> Ross
>

RE: snap-2004-11-03 breakage

[Robert Kindred]

> -----Original Message-----
> From: pthreads-win32-owner@sources.redhat.com
> [mailto:pthreads-win32-owner@sources.redhat.com]On Behalf Of Ross
> Johnson
> Sent: Wednesday, November 03, 2004 10:18 AM
> To: pthreads-win32
> Subject: Re: snap-2004-11-03 breakage
>
>
> Robert Kindred wrote:
>
> >On the other hand, having pthread_t to be a pointer forces me to put
> >compiler switches in my code that runs on both Windows and QNX.  I would
> >appreciate knowing the general direction things are taking.
> >
> >
> >
> What specifics of pthread_t are causing you to have to treat
> it differently on different systems? Applications shouldn't
> have to know how pthread_t is actually defined.

Let me take a quick look at my code...    Oops, I typed too soon.  I am not
actually switching on a pthread_t.  I am switching code on a
pthread_mutex_t.  I must create and initialize these differently on the two
platforms.  Perhaps there is a portable way that I don't know about?

>
> Re the general direction for the library:
> The library has reached a point where it's stable, except
> for some relatively quiet issues, such as thread ID reuse,
> left to fix. Defining pthread_t as a pointer meant that a
> new thread could acquire the same ID as a recently detached
> or joined thread, which could be disasterous. There were no
> serious bug fixes to the last snapshot and no significant
> fundamentally new features added.
>
> The changes to mutexes in this snapshot are also helping
> prepare the way for [possibly] making headway on
> POSIX_PROCESS_SHARED mutexes, semaphores etc. This may still
> prove difficult, but some more definite impedements
> have now been removed.
>
> This is the first time in over 6 years, if I recall
> correctly, that the library's ABI has changed, and there are
> no further changes planned.
>
> Regards.
> Ross
>
> >Robert Kindred
> >
> >-----Original Message-----
> >From: pthreads-win32-owner@sources.redhat.com
> >[mailto:pthreads-win32-owner@sources.redhat.com]On Behalf Of Gisle Vanem
> >Sent: Wednesday, November 03, 2004 7:18 AM
> >To: pthreads-win32
> >Subject: snap-2004-11-03 breakage
> >
> >
> >snap-2004-11-03 breaks a lot of applications by the way 'pthread_t' is
> >defined:
> >
> >typedef struct {
> >    void * p;                   /* Pointer to actual object */
> >    unsigned int x;             /* Extra information - reuse count etc */
> >} ptw32_handle_t;
> >
> >typedef ptw32_handle_t pthread_t;
> >
> >Code like (from Ettercap)
> >  pthread_t pid = ec_thread_getpid("golem");
> >  if (pid != 0)
> >    ec_thread_destroy(pid);
> >
> >no longer works; you cannot compare a struct against 0.
> >
> >I'm not sure you really meant to do that or if the typedef should be
> >typedef ptw32_handle_t *pthread_t;
> >
> >--gv
> >
> >
> >
> >
>
>

Re: snap-2004-11-03 breakage

[Ross Johnson]

Hi,

I definitely don't want to break applications unnecessarily and 
particularly applications that have been ported to several different 
POSIX systems, but on the other hand POSIX thread IDs are not required 
to be scalar. See the rationale from the definition of pthread_equal() 
in the Single Unix Specification version 3, which says:


        RATIONALE

    Implementations may choose to define a thread ID as a structure.
    This allows additional flexibility and robustness over using an
    *int*. For example, a thread ID could include a sequence number that
    allows detection of "dangling IDs" (copies of a thread ID that has
    been detached). Since the C language does not support comparison on
    structure types, the /pthread_equal/() function is provided to
    compare thread IDs.

I don't think this is new to SUS or SUS version 3 either.

Nor (given the above) does the value 0 (zero) for a thread ID have any 
special meaning in the standard, although it obviously does as the value 
returned by ec_thread_getpid() in ettercap. If you want to test if a 
thread exists given it's thread ID, use:

    if (pthread_kill(pid, 0) == 0)  /* or (... != ESRCH) */

That should be portable to all POSIX implementations.

I'm not dismissing your problem, but in this case I think it would be 
better to fix the application.

Regards.
Ross

Gisle Vanem wrote:

> snap-2004-11-03 breaks a lot of applications by the way 'pthread_t' is 
> defined:
>
> typedef struct {
>    void * p;                   /* Pointer to actual object */
>    unsigned int x;             /* Extra information - reuse count etc */
> } ptw32_handle_t;
>
> typedef ptw32_handle_t pthread_t;
>
> Code like (from Ettercap)
>  pthread_t pid = ec_thread_getpid("golem");
>  if (pid != 0)
>    ec_thread_destroy(pid);
>
> no longer works; you cannot compare a struct against 0.
>
> I'm not sure you really meant to do that or if the typedef should be
> typedef ptw32_handle_t *pthread_t;
>
> --gv
>

Re: snap-2004-11-03 breakage

[Gisle Vanem]

"Ross Johnson" wrote:

> I definitely don't want to break applications unnecessarily and 
> particularly applications that have been ported to several different 
> POSIX systems, but on the other hand POSIX thread IDs are not required 
> to be scalar. See the rationale from the definition of pthread_equal() 
> in the Single Unix Specification version 3, which says:

Great, but when was pthread_equal() introduced? I'd like to support
old versions of pthread-win32 too.

BTW. I cannot find a compile-time VERSION define to check against.
If there where, I could #ifdef around this problem.

BTW2. I'm, getting 2 copies of every reply from you. I would prefer replies
to go to the list only. Could you please make the mailing-list program add 
a "Reply-to" header?

--gv

Re: snap-2004-11-03 breakage

[Ross Johnson]

Gisle Vanem wrote:

> "Ross Johnson" wrote:
>
>> I definitely don't want to break applications unnecessarily and 
>> particularly applications that have been ported to several different 
>> POSIX systems, but on the other hand POSIX thread IDs are not 
>> required to be scalar. See the rationale from the definition of 
>> pthread_equal() in the Single Unix Specification version 3, which says:
>
>
> Great, but when was pthread_equal() introduced? I'd like to support
> old versions of pthread-win32 too.
>
pthread_equal() has been in pthreads-win32 since 1998. pthread_kill() 
with the sigzero functionality is a lot newer - introduced in 
snapshot-2003-08-15.

> BTW. I cannot find a compile-time VERSION define to check against.
> If there where, I could #ifdef around this problem.

I should have done that for this snapshot. Don't be surprised to see a 
new snapshot in a day or two just to include that.

>
> BTW2. I'm, getting 2 copies of every reply from you. I would prefer 
> replies
> to go to the list only. Could you please make the mailing-list program 
> add a "Reply-to" header?
>
I have no control over the mailing list, but I can save you from getting 
two copies from me.

> --gv
>

Re: snap-2004-11-03 breakage

[Alexander Terekhov]

> Code like (from Ettercap)
>   pthread_t pid = ec_thread_getpid("golem");
>   if (pid != 0)
>     ec_thread_destroy(pid);

Code it like

  pthread_t tid;
  if (ec_thread_getpid("golem",&tid)
    ec_thread_destroy(tid);

Don't rely on idiotic pthread_kill() "testing". The standard 
doesn't preclude the use of pointers for thread IDs (e.g. 
inside pthread_t structure). Pointers become indeterminate and
trigger undefined behavior for any nonassignment access once 
the reference storage gets freed. Conforming implementations 
are allowed to reclaim/free that referenced storage when 
detached thread is terminated or joinable thread is terminated 
and joined. Use of "retired" pthread_t values (e.g. as 
pthread_kill() argument) after that moment will trigger 
undefined behavior. 

regards,
alexander.

Re: snap-2004-11-03 breakage

[Ross Johnson]

Alexander Terekhov wrote:

>>Code like (from Ettercap)
>>  pthread_t pid = ec_thread_getpid("golem");
>>  if (pid != 0)
>>    ec_thread_destroy(pid);
>>    
>>
>
>Code it like
>
>  pthread_t tid;
>  if (ec_thread_getpid("golem",&tid)
>    ec_thread_destroy(tid);
>
>  
>
But how does ec_thread_getpid() portably determine if the thread
exists or not?

>Don't rely on idiotic pthread_kill() "testing". The standard 
>doesn't preclude the use of pointers for thread IDs (e.g. 
>inside pthread_t structure).
>
Oops. I should have recognised that because ...

>Pointers become indeterminate and
>trigger undefined behavior for any nonassignment access once 
>the reference storage gets freed. Conforming implementations 
>are allowed to reclaim/free that referenced storage when 
>detached thread is terminated or joinable thread is terminated 
>and joined. Use of "retired" pthread_t values (e.g. as 
>pthread_kill() argument) after that moment will trigger 
>undefined behavior. 
>
>  
>
... that's exactly why pthread_t was changed in pthreads-win32.

Every function that takes a thread ID argument has the same problem
as pthread_kill() if pointers are used.

Ross

>regards,
>alexander.
>
>  
>

Re: snap-2004-11-03 breakage

[Alexander Terekhov]

> But how does ec_thread_getpid() portably determine if the thread
> exists or not?

Define "exists" first. If you mean "tryjoin", see 
<http://www.terekhov.de/mythread.c> illustration.

> Every function that takes a thread ID argument has the same problem
> as pthread_kill() if pointers are used.

Yes, but it's an application bug.

regards,
alexander.



Sent by:        pthreads-win32-owner@sources.redhat.com
To:     pthreads-win32 <pthreads-win32@sources.redhat.com>
cc:      
Subject:        Re: snap-2004-11-03 breakage


Alexander Terekhov wrote:

>>Code like (from Ettercap)
>>  pthread_t pid = ec_thread_getpid("golem");
>>  if (pid != 0)
>>    ec_thread_destroy(pid);
>> 
>>
>
>Code it like
>
>  pthread_t tid;
>  if (ec_thread_getpid("golem",&tid)
>    ec_thread_destroy(tid);
>
> 
>
But how does ec_thread_getpid() portably determine if the thread
exists or not?

>Don't rely on idiotic pthread_kill() "testing". The standard 
>doesn't preclude the use of pointers for thread IDs (e.g. 
>inside pthread_t structure).
>
Oops. I should have recognised that because ...

>Pointers become indeterminate and
>trigger undefined behavior for any nonassignment access once 
>the reference storage gets freed. Conforming implementations 
>are allowed to reclaim/free that referenced storage when 
>detached thread is terminated or joinable thread is terminated 
>and joined. Use of "retired" pthread_t values (e.g. as 
>pthread_kill() argument) after that moment will trigger 
>undefined behavior. 
>
> 
>
... that's exactly why pthread_t was changed in pthreads-win32.

Every function that takes a thread ID argument has the same problem
as pthread_kill() if pointers are used.

Ross

>regards,
>alexander.
>
> 
>