RE: asynchronous cancellation

RE: asynchronous cancellation

[Bossom, John]

Hi Jason,

No flames intended... (and there are none contained within this email
either)
I'm not trying to say "ours is better than yours" either. 

(Better grab a coffee... I got kind of long winded here!)

The situation I am trying to describe has nothing to do with the
pthreads-win32 implementation or your implementation, or any other
implementation for that matter.

It is a general comment I have that the C++ features

	- automatic C++ object destructors
	- C++ exception handling

are in direct conflict with the "philosophy" of the PThreads Cancellation
programming paradigm (specifically cleanup) regardless of
platform/implementation.

Summary:
=======
As a C++ programmer, I wouldn't use the pthreads cancellation mechanism
simply because it does prevent cleaning up resources held by automatic
(stack) based C++ objects (or, conversely, it prevents me from using these
C++ features). The cleanup code one would put in their C++ 'catch' would 
have to be duplicated in their pthread_cleanup_push routines... so a C++
programmer might as well choose one methodology or the other:

	a) pthread cancellation mechanism, with push/pop for cleanup:
		- never using automatic objects that retain resources,
		- never using/relying on the C++ catch code being executed
for cancellation

	b) C++ features:
		- automatic C++ object destructors
		- C++ Exception Handling
		- your own method of cancelling a thread
(flag/event/condition/etc.
		  and unwind the stack by throwing an exception)

Attempting to mix the two such that you have no resource leaks wouldn't be
pretty.

Details
-------
There is no standard that specifies the behaviour between C++ and PThreads.
As a result, the behaviour of a PThreads library and these C++ features is
implementation specific and should not be relied upon.

My point is, you cannot rely on the two mechanisms working together from
machine to machine as there is no standard enforcing this. 

As you have stated, your implementation bypasses stack-unwinding... there
is nothing wrong with this... as far as I know, the native PThreads
implementations
on all UNIX platforms also bypass C++ stack unwinding - this is their
defined 
behaviour - PThreads knows nothing about C++.

I am not touting the pthreads-win32 mechanism either... I personally
wouldn't use
the pthreads cancellation mechanism on any platform (including
pthreads-win32)
if I was writing code in C++ because it limits/prevents the use of these C++
features.

Most PThreads books I have read state that you should avoid using
the PThreads cancellation mechanism since it can result in complex code
and bypasses all C++ stack unwinding resulting in lost resources (see my
little C++ example below. These books typically recommend that C++
programmers
implement their own cancellation mechanism (i.e. setting a variable and
checking it periodically within the thread, or by some other means - then
throw a C++
exception out to the thread's main routine, and exit the thread there by
simply
returning from the main routine.) This is the philosophy taken by Threads++
library
by RogueWave(?). The programmer defines cancellation points within their
code. The cancelation check simply returns whether you should carry on or
cancel. It is up to the programmer to then shutdown the thread however they
choose. This model works for
both C and C++ without having to sacrifice language features.

More Detail:

i.e.) C++ guarantees that objects declared on the stack have their
      destructors invoked no matter how you leave the scope in which they
      are defined. There are two valid methods of leaving the scope, as
	far as C++ is concerned:
		1) simply executing to the end of the scope in which the
object is
		   declared, or
		2) jumping out of the scope through the use of the C++
exception
		   handler.
	In both cases, the C++ compiler places sufficient implicit hooks
into
	the generated code to ensure that your automatic destructors for
these
	stack based objects are called.

The programming paradigm for the PThreads cancellation mechanism is that
only those destructors explicitly registered with pthread_cleanup_push are
called; then the thread exists in place (i.e. doesn't return through the
call stack)... this step bypasses the cleanup of all stack based C++ objects
... any resources being held by these objects are forever lost).


Here is an example snippet of code (sorry if it isn't accurate
C++, but you'll get the idea...


class MyClass
{
	public:

	MyClass()
	{
		acquireResource();
	}
	~MyClass()
	{
		releaseResource();
	}
};

someRoutine()
{
	{
		MyClass	classInstance;
		
		doSomeStuff();

		// compiler generated implicit code to call
		// classInstance.~MyClass() here...
	}

By stack unwinding I mean that, whether or not you use C++ exception
handling, when you leave the scope in which an automatic (i.e. stack
based variable) is created, the destructors are implicitly called
(i.e. the C++ compiler generates an implicit call to
classInstance::~MyClass()
 in the above example).

Now, if someRoutine was called within a thread, and we are currently running
in doSomeStuff() when the cancellation is recognised, the object
classInstance
would not have it's destructor called and any resources held by it will be
leaked/lost...
A C++ programmer wishing to use the pthread cancellation mechanism must take
special care in order to get these features to work together... They must
either
only use deferred cancellation and only enable it in safe situations, or
re-write the above to look something like

someRoutine()
{
	{
		MyClass	* classInstance = New MyClass();
		pthread_cleanup_push( classInstance, MyClass::~MyClass );

		doSomeStuff();

		pthread_cleanup_pop( classInstance, 1 );
	}
}

I hope I have clarified myself and I do thank you for showing me how to
implement an asynchronous cancel by basically telling the CPU to start
executing code at a specific address...

-----Original Message-----
From: Jason Nye [ mailto:jnye@nbnet.nb.ca ]
Sent: Tuesday, November 16, 1999 3:57 PM
To: Ross Johnson
Cc: 'Pthreads-win32'
Subject: Re: asynchronous cancellation


John Bossom wrote:

> > Interesting solution...
> >
> > To work with the pthreads-win32 library, your AsyncCancelPoint
> > needs to get pSelf from dynamically declared TLS... (i.e. pthread_self,
> > or it's internal implementation).
> >
> > Just a note to C++ users: one shouldn't use the built-in pthreads
> > cancellation if they expect C++ stack unwinding to work....
> >

I'm a C++ developer, not necessarily a guru in the ways of pthreads -- I
do know some its specifications, however -- please, no flames!

Why does the stack have to be unwound when a thread is cancelled? Is
this a requirement of pthreads (I'm pretty sure this is not the case) or
is it just the way that pthreads-win32 has always implemented it and you
are sticking to this design? My ObjectThread library does not unwind the
stack when a thread is cancelled and the net result is that it has no
weird implementation-specific side effects from using exception handling
(whether SEH or C++) and there is no unnecessary performance overhead
when a thread gets cancelled -- no unwinding the stack, just executing
cleanup handlers/destructors and calling _exitthreadex().

IMHO using SEH and C++ exceptions ties pthreads-win32 to specific
compilers/languages and introduces unnecessary implementation-specific
side-effects that will catch some developers by surprise (it caught me
by surprise -- and note John's last point). Cancellation can be easily
implemented without using such constructs and the result is a
compiler-independent library in which cancellation acts as one would
expect.

Note that I'm not trying to say "Mine is better than yours", I'm just
saying that there is an another way to achieve cancellation that may be
more reliable and more compliant to the standard.

Cheerio,
Jason

Re: asynchronous cancellation

[Jason Nye]

Just thought I'd point out a mistake I made in my code I showed you. The 

   if (::WaitForSingleObject(hThread, 0) != WAIT_TIMEDOUT)

line should be

   if (::WaitForSingleObject(hThread, 0) == WAIT_TIMEDOUT)

You want to time out on the wait -- it means the thread is still
running.

Cheers.
Jason

Re: asynchronous cancellation

[Ross Johnson]

John, Jason, all,

The following applies to the C++ version of things, but SEH versions
should hopefully be similar.

Following on from John's explanation of C++ stack unwinding etc, and
the fact that pthreads-win32 implements cancelation by throwing an
exception, and the fact that the C++ version of pthread_cleanup_pop
is implemented as a destructor (after John's original
implementation):

For deferred cancelation:
1) stack unwinding should occur properly IMO

2) cleanup handlers should be called.

For async cancelation:
1) Jason's method could be used with the following simple change to
   AsyncCancelPoint to work with the current pthreads-win32
   mechanism:

void cancelThread(HANDLE hThread)
{
    ::SuspendThread(hThread);
    if (::WaitForSingleObject(hThread, 0) != WAIT_TIMEDOUT) {
            // Ok, thread did not exit before we got to it.
           CONTEXT context;
           context.ContextFlags = CONTEXT_CONTROL;
           ::GetThreadContext(hThread, &context);
           // _x86 only!!!
           context.Eip = (DWORD)AsyncCancelPoint;
           ::SetThreadContext(hThread, &context);
           ::ResumeThread(hThread);
    }
}

// declare AsyncCancelPoint:
void AsyncCancelPoint()
{
    // This is all it needs to do to call cleanup handlers,
    // call object destructors, and exit the thread.
    throw Pthread_exception_cancel();
}

For cancelation in general:
1) the following macro should be included in pthread.h:

#define catch(E) \
	catch(Pthread_exception_cancel) { \
		throw(); \
	} \
	catch(E)


Is this getting closer to the ideal, or am I way off the track?

Ross

+----------------------+---+
| Ross Johnson         |   | E-Mail: rpj@ise.canberra.edu.au
| Info Sciences and Eng|___|
| University of Canberra   | FAX:    +61 6 2015227
| PO Box 1                 |
| Belconnen  ACT    2616   | WWW:    http://willow.canberra.edu.au/~rpj/
| AUSTRALIA                |
+--------------------------+

Re: asynchronous cancellation

[Jason Nye]

John Bossom wrote:

> > Interesting solution...
> >
> > To work with the pthreads-win32 library, your AsyncCancelPoint
> > needs to get pSelf from dynamically declared TLS... (i.e. pthread_self,
> > or it's internal implementation).
> >
> > Just a note to C++ users: one shouldn't use the built-in pthreads
> > cancellation if they expect C++ stack unwinding to work....
> >

I'm a C++ developer, not necessarily a guru in the ways of pthreads -- I
do know some its specifications, however -- please, no flames!

Why does the stack have to be unwound when a thread is cancelled? Is
this a requirement of pthreads (I'm pretty sure this is not the case) or
is it just the way that pthreads-win32 has always implemented it and you
are sticking to this design? My ObjectThread library does not unwind the
stack when a thread is cancelled and the net result is that it has no
weird implementation-specific side effects from using exception handling
(whether SEH or C++) and there is no unnecessary performance overhead
when a thread gets cancelled -- no unwinding the stack, just executing
cleanup handlers/destructors and calling _exitthreadex().

IMHO using SEH and C++ exceptions ties pthreads-win32 to specific
compilers/languages and introduces unnecessary implementation-specific
side-effects that will catch some developers by surprise (it caught me
by surprise -- and note John's last point). Cancellation can be easily
implemented without using such constructs and the result is a
compiler-independent library in which cancellation acts as one would
expect.

Note that I'm not trying to say "Mine is better than yours", I'm just
saying that there is an another way to achieve cancellation that may be
more reliable and more compliant to the standard.

Cheerio,
Jason

RE: asynchronous cancellation

[Ross Johnson]

On Mon, 15 Nov 1999, Bossom, John wrote:

> Interesting solution...
> 
> To work with the pthreads-win32 library, your AsyncCancelPoint
> needs to get pSelf from dynamically declared TLS... (i.e. pthread_self,
> or it's internal implementation).
> 
> Just a note to C++ users: one shouldn't use the built-in pthreads
> cancellation if they expect C++ stack unwinding to work....
> 

[Most of C++ is out of scope for me so I've been hanging back
waiting for more informed comment, which John has provided, but I'm
still not clear about this issue.]

Is the last paragraph above meant as general advice or is it
specific to either the pthreads-win32 or Jason's implementation, or
both? I believe that the current pthreads-win32 method of using an
exception properly unwinds the stack. There are other problems with
the method though (apart from the lack of async cancelation), which
Jason has observed as explained later.

Looking at Jason's code: I'm assuming that scope is retained at the
point the thread is resumed (because stack pointers etc are
untouched). It isn't clear to me what happens when the thread exits.
Jason includes the statement "callDestructors(pSelf)". Is this
logical pseudo code for what needs to happen? Are destructors all
run properly in Jason's implementation, or can they be made to run
if not?


Back to the existing method of cancelation:
Jason points out in his README file (in his package) that the
current cancelation method using exceptions is prone to the
following problems (C++ exceptions of course):

void f() throw()
{
	/* Cancelation won't work in here */
	...
}

and:

try {
	...
}
catch (...) {
	/* This will catch any cancelation exception prematurely */
	...
}


My response to these is perhaps naive, but nevertheless could form a
reasonable solution, at least in part, if we persist with the
current method.

While both problems can be handled easily if the application
programmer knows the internal implementation of pthread-win32 (and
could therefore insert an appropriate catch block etc), we want
really to keep it opaque.

The following macro:

#define catch(E) \
	catch(Pthread_exception_cancel) { \
		throw(); \
	} \
	catch(E)

appears to work around the second case. Even though this can result
in redundant repetition of catch blocks for
Pthread_exception_cancel, this doesn't appear to be a problem for
the compiler (tested with g++).

The first case appears to be a little more difficult to provide a
workaround for. The following works for the empty list case (tested
only with gnu cpp/g++):

#define throw() throw(Pthread_exception_cancel)

but obviously not for the case where a list of exceptions is given.
It's a pity cpp can't differentiate between macros with empty and
non-empty arg lists.

Is there a more sophisticated way to do any of this? Is it worth
putting at least the "catch" macro into pthread.h? Is Jason's method
a better one (will require some remodelling of the existing cleanup
handler push/pop code)?

Ross

> 
> -----Original Message-----
> From: Jason Nye [ mailto:jnye@nbnet.nb.ca ]
> Sent: Friday, November 12, 1999 9:20 PM
> To: 'Pthreads-win32'
> Subject: asynchronous cancellation
> 
> 
> Hi, all
> 
> I've noticed a lot of you discussing asynchronous cancellation and
> mentioning how difficult it is to do under win32. I did some research
> and found a bullet-proof way of doing it (from a J. Richter example).
> Here is a sample of the code I used in my library:
> 
> If thread x wants to cancel thread y asynchronously, thread x should
> call cancelThread(y):
> 
> ----------------------------------------------------------------------------
> -----------------------
> 
> void cancelThread(HANDLE hThread)
> {
>     ::SuspendThread(hThread);
>     if (::WaitForSingleObject(hThread, 0) != WAIT_TIMEDOUT) {
>             // Ok, thread did not exit before we got to it.
>            CONTEXT context;
>            context.ContextFlags = CONTEXT_CONTROL;
>            ::GetThreadContext(hThread, &context);
>            // _x86 only!!!
>            context.Eip = (DWORD)AsyncCancelPoint;
>            ::SetThreadContext(hThread, &context);
>            ::ResumeThread(hThread);
>     }
> }
> 
> // declare AsyncCancelPoint:
> void AsyncCancelPoint()
> {
>     // pSelf is a pointer to a ThreadInfo (each thread has one --
> declared as __declspec(thread)).
>     popCancelCleanupHandlers(pSelf);
>     callDestructors(pSelf);
>     _endthreadex(PTHREAD_CANCELLED);
> }
> ----------------------------------------------------------------------------
> -----------------------
> 
> That is it. If a thread's cancel state is asynchronous and another
> thread requests that it be cancelled, the thread will suddenly find
> itself executing AsyncCancelPoint which is exactly what you want.
> 
> If you want to see it in action in my C++ library, go to
> http://www3.nbnet.nb.ca/jnye , follow the "current software projects"
> link and download the latest version of ObjectThread. You'll see how it
> fits into a complete library.
> 
> Hopefully this is useful,
> Jason
> 

+----------------------+---+
| Ross Johnson         |   | E-Mail: rpj@ise.canberra.edu.au
| Info Sciences and Eng|___|
| University of Canberra   | FAX:    +61 6 2015227
| PO Box 1                 |
| Belconnen  ACT    2616   | WWW:    http://willow.canberra.edu.au/~rpj/
| AUSTRALIA                |
+--------------------------+

RE: asynchronous cancellation

[Bossom, John]

Interesting solution...

One little problem though... this solution makes use of _declspec( thread ).
Use of this functionality effectively prevents the use of any
dynamically loaded DLLs that want to use the threading library.
(memory for variables declared as declspec( thread ) is only accounted
 for at initial process load time; any declarations within dynamically
 loaded DLLs are missed - you end up with unpredictable behaviour).

To work with the pthreads-win32 library, your AsyncCancelPoint
needs to get pSelf from dynamically declared TLS... (i.e. pthread_self,
or it's internal implementation).

Just a note to C++ users: one shouldn't use the built-in pthreads
cancellation if they expect C++ stack unwinding to work....

Thanks for your contribution!

John.


-----Original Message-----
From: Jason Nye [ mailto:jnye@nbnet.nb.ca ]
Sent: Friday, November 12, 1999 9:20 PM
To: 'Pthreads-win32'
Subject: asynchronous cancellation


Hi, all

I've noticed a lot of you discussing asynchronous cancellation and
mentioning how difficult it is to do under win32. I did some research
and found a bullet-proof way of doing it (from a J. Richter example).
Here is a sample of the code I used in my library:

If thread x wants to cancel thread y asynchronously, thread x should
call cancelThread(y):

----------------------------------------------------------------------------
-----------------------

void cancelThread(HANDLE hThread)
{
    ::SuspendThread(hThread);
    if (::WaitForSingleObject(hThread, 0) != WAIT_TIMEDOUT) {
            // Ok, thread did not exit before we got to it.
           CONTEXT context;
           context.ContextFlags = CONTEXT_CONTROL;
           ::GetThreadContext(hThread, &context);
           // _x86 only!!!
           context.Eip = (DWORD)AsyncCancelPoint;
           ::SetThreadContext(hThread, &context);
           ::ResumeThread(hThread);
    }
}

// declare AsyncCancelPoint:
void AsyncCancelPoint()
{
    // pSelf is a pointer to a ThreadInfo (each thread has one --
declared as __declspec(thread)).
    popCancelCleanupHandlers(pSelf);
    callDestructors(pSelf);
    _endthreadex(PTHREAD_CANCELLED);
}
----------------------------------------------------------------------------
-----------------------

That is it. If a thread's cancel state is asynchronous and another
thread requests that it be cancelled, the thread will suddenly find
itself executing AsyncCancelPoint which is exactly what you want.

If you want to see it in action in my C++ library, go to
http://www3.nbnet.nb.ca/jnye , follow the "current software projects"
link and download the latest version of ObjectThread. You'll see how it
fits into a complete library.

Hopefully this is useful,
Jason

asynchronous cancellation

[Jason Nye]

Hi, all

I've noticed a lot of you discussing asynchronous cancellation and
mentioning how difficult it is to do under win32. I did some research
and found a bullet-proof way of doing it (from a J. Richter example).
Here is a sample of the code I used in my library:

If thread x wants to cancel thread y asynchronously, thread x should
call cancelThread(y):

---------------------------------------------------------------------------------------------------

void cancelThread(HANDLE hThread)
{
    ::SuspendThread(hThread);
    if (::WaitForSingleObject(hThread, 0) != WAIT_TIMEDOUT) {
            // Ok, thread did not exit before we got to it.
           CONTEXT context;
           context.ContextFlags = CONTEXT_CONTROL;
           ::GetThreadContext(hThread, &context);
           // _x86 only!!!
           context.Eip = (DWORD)AsyncCancelPoint;
           ::SetThreadContext(hThread, &context);
           ::ResumeThread(hThread);
    }
}

// declare AsyncCancelPoint:
void AsyncCancelPoint()
{
    // pSelf is a pointer to a ThreadInfo (each thread has one --
declared as __declspec(thread)).
    popCancelCleanupHandlers(pSelf);
    callDestructors(pSelf);
    _endthreadex(PTHREAD_CANCELLED);
}
---------------------------------------------------------------------------------------------------

That is it. If a thread's cancel state is asynchronous and another
thread requests that it be cancelled, the thread will suddenly find
itself executing AsyncCancelPoint which is exactly what you want.

If you want to see it in action in my C++ library, go to
http://www3.nbnet.nb.ca/jnye , follow the "current software projects"
link and download the latest version of ObjectThread. You'll see how it
fits into a complete library.

Hopefully this is useful,
Jason