Re: more cancellation work

Re: more cancellation work

[Steven Munroe]

Found that glibc cvs will not build -enable-add-ons=linuxthreads any more
because of cancelation changes made to misc/syslog.c 

syslog.c:238: warning: implicit declaration of function `__libc_cleanup_push'
syslog.c:238: warning: implicit declaration of function `__libc_cleanup_push'
syslog.c:238: warning: implicit declaration of function `__libc_cleanup_push'
/home/sjmunroe/work/build-23/libc.so.6: undefined reference to
`__libc_cleanup_push'
/home/sjmunroe/work/build-23/libc.so.6: undefined reference to
`__libc_cleanup_push'

the __libc_cleanup_push and __libc_cleanup_pop macros are defined in:

nptl/sysdeps/pthread/bits/libc-lock.h

but not in:

linuxthreads/sysdeps/pthread/bits/libc-lock.h

What is the intent for this? Should linuxthreads just noop these macros or are
there equalent cleanup function fo linuxthreads?


-- 
Steven Munroe
sjmunroe@us.ibm.com
Linux on PowerPC-64 Development
GLIBC for PowerPC-64 Development

Re: more cancellation work

[David Mosberger]

>>>>> On Thu, 17 Jul 2003 11:42:07 -0700, Richard Henderson <rth@twiddle.net> said:

  Richard> On Thu, Jul 17, 2003 at 10:43:51AM -0700, David Mosberger wrote:
  >> What I meant is: when will libc unwind?  As a result of
  >> pthread_cancel()?  If so, how does it work precisely?

  >> But since you brought it up: what API will this "external library"
  >> follow?  The C++ exception handling API? (_Unwind_FOO)?

  Richard> Yes, and by calling _Unwind_ForceUnwind.  What else did you
  Richard> want to know?

That explains it.

Thanks,

	--david

Re: more cancellation work

[Richard Henderson]

On Thu, Jul 17, 2003 at 10:43:51AM -0700, David Mosberger wrote:
> What I meant is: when will libc unwind?  As a result of
> pthread_cancel()?  If so, how does it work precisely?
> 
> But since you brought it up: what API will this "external library"
> follow?  The C++ exception handling API? (_Unwind_FOO)?

Yes, and by calling _Unwind_ForceUnwind.  What else did you 
want to know?


r~

Re: more cancellation work

[David Mosberger]

>>>>> On Wed, 16 Jul 2003 21:11:49 -0700, Richard Henderson <rth@twiddle.net> said:

  Richard> On Tue, Jul 15, 2003 at 04:37:12PM -0700, David Mosberger
  Richard> wrote:
  >> BTW: is there a summary somewhere that libc will be doing?

  Richard> Relying on an external library for the actual unwinding.

What I meant is: when will libc unwind?  As a result of
pthread_cancel()?  If so, how does it work precisely?

But since you brought it up: what API will this "external library"
follow?  The C++ exception handling API? (_Unwind_FOO)?

	--david

Re: more cancellation work

[Richard Henderson]

On Tue, Jul 15, 2003 at 04:37:12PM -0700, David Mosberger wrote:
> BTW: is there a summary somewhere that libc will be doing?

Relying on an external library for the actual unwinding.


r~

Re: more cancellation work

[David Mosberger]

>>>>> On Tue, 15 Jul 2003 18:23:23 -0700, Richard Henderson <rth@twiddle.net> said:

  Richard> On Tue, Jul 15, 2003 at 04:27:00PM -0700, David Mosberger
  Richard> wrote:

  >> You do know that the ia64 unwinder in gcc is broken, however.

  Richard> No, I don't.

It is.  The biggest issue is unwinding across alternate signal stacks.
The changes needed to make this work are fairly extensive and I
decided against trying to backport them to the gcc unwinder (since
that would have likely introduced merging bugs).

	--david

Re: more cancellation work

[Richard Henderson]

On Tue, Jul 15, 2003 at 04:27:00PM -0700, David Mosberger wrote:
> You do know that the ia64 unwinder in gcc is broken, however.

No, I don't.


r~

Re: more cancellation work

[David Mosberger]

>>>>> On Tue, 15 Jul 2003 15:36:14 -0700, Ulrich Drepper <drepper@redhat.com> said:

  Uli> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1

  Uli> David Mosberger wrote:

  >> In any case, are you worried at all about dynamically generated
  >> code?  If so, how are you planning to handle that?

  Uli> Why should I do anything about this?  This has nothing to do
  Uli> with the exception handling we do in libc.

That's why I was asking.  I don't know whether the libc exception
handling would ever have to unwind through dynamically generated code.
If not, there is no issue, of course.

BTW: is there a summary somewhere that libc will be doing?

	--david

Re: more cancellation work

[David Mosberger]

>>>>> On Tue, 15 Jul 2003 15:13:30 -0700, Richard Henderson <rth@twiddle.net> said:

  Richard> On Tue, Jul 15, 2003 at 10:56:52AM -0700, David Mosberger
  Richard> wrote:
  >> Well, gcc on ia64 _does_ use libunwind.

  Richard> Not by default.

You do know that the ia64 unwinder in gcc is broken, however.

	--david

Re: more cancellation work

[Ulrich Drepper]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

David Mosberger wrote:

> In any case, are you worried at all about dynamically generated code?
> If so, how are you planning to handle that?

Why should I do anything about this?  This has nothing to do with the
exception handling we do in libc.

- -- 
- --------------.                        ,-.            444 Castro Street
Ulrich Drepper \    ,-----------------'   \ Mountain View, CA 94041 USA
Red Hat         `--' drepper at redhat.com `---------------------------
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.1 (GNU/Linux)

iD8DBQE/FIHe2ijCOnn/RHQRAhciAJ9nD2AsJO9lEQFDijKXfNNE+2E/CwCfeHM6
DVIhYmxan1ixYpIdlcn392Q=
=+cqE
-----END PGP SIGNATURE-----

Re: more cancellation work

[David Mosberger]

>>>>> On Tue, 15 Jul 2003 14:27:36 -0700, Ulrich Drepper <drepper@redhat.com> said:

  Uli> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1

  Uli> David Mosberger wrote:

  >> - gdb never had any problems with libunwind itself (it was
  >> initially released under GPL, now it's even under the MIT license
  >> so that the *BSD folks can use it for their kernel)

  Uli> That's not enough.  The code has to be assigned to the FSF.

libunwind is not part of gdb, so I don't see why it would need an
assignment.

  Uli> But all this is pointless.  If anything should change it's
  Uli> ia64.  There was no reason to invent something new but still it
  Uli> was done.  For no good reason.

That sounds like a lot of sour grapes.  I don't know the history
behind the ABI work, but changing it now sure seems like a bit
late... ;-)

In any case, are you worried at all about dynamically generated code?
If so, how are you planning to handle that?

	--david

Re: more cancellation work

[Richard Henderson]

On Tue, Jul 15, 2003 at 10:56:52AM -0700, David Mosberger wrote:
> Well, gcc on ia64 _does_ use libunwind.

Not by default.


r~

Re: more cancellation work

[Ulrich Drepper]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

David Mosberger wrote:

>  - gdb never had any problems with libunwind itself (it was initially
>    released under GPL, now it's even under the MIT license so that the
>    *BSD folks can use it for their kernel)

That's not enough.  The code has to be assigned to the FSF.


But all this is pointless.  If anything should change it's ia64.  There
was no reason to invent something new but still it was done.  For no
good reason.

- -- 
- --------------.                        ,-.            444 Castro Street
Ulrich Drepper \    ,-----------------'   \ Mountain View, CA 94041 USA
Red Hat         `--' drepper at redhat.com `---------------------------
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.1 (GNU/Linux)

iD8DBQE/FHHI2ijCOnn/RHQRAnrlAKCu1H4kUmEnSkeGjNTIrhx4ubi29gCfWLYR
1LXUNBWdMXMJbeuDfBXa8II=
=Mdly
-----END PGP SIGNATURE-----

Re: more cancellation work

[David Mosberger]

>>>>> On Tue, 15 Jul 2003 11:48:50 -0700, Ulrich Drepper <drepper@redhat.com> said:

  Uli> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1

  Uli> David Mosberger wrote:

  >> Huh?  What "legal problems" does libunwind have?  Care to
  >> elaborate?

  Uli> gdb apparently still cannot use libunwind.

You're confusing different things here:

 - gdb never had any problems with libunwind itself (it was initially
   released under GPL, now it's even under the MIT license so that the
   *BSD folks can use it for their kernel)

 - there is a gdb-specific patch which I wrote which needed a
   copyright assignment; this took a long time to settle but it's in
   place now

Summary: there are no legal issues in using libunwind.

	--david

Re: more cancellation work

[Ulrich Drepper]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

David Mosberger wrote:

> Huh?  What "legal problems" does libunwind have?  Care to elaborate?

gdb apparently still cannot use libunwind.

- -- 
- --------------.                        ,-.            444 Castro Street
Ulrich Drepper \    ,-----------------'   \ Mountain View, CA 94041 USA
Red Hat         `--' drepper at redhat.com `---------------------------
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.1 (GNU/Linux)

iD8DBQE/FEyS2ijCOnn/RHQRAleTAJ9z5j2T67KPmyg4eVinAcVMGydWZQCgmmGf
ql+nONi3VJvDpZ4vgXra+9k=
=pT2A
-----END PGP SIGNATURE-----

Re: more cancellation work

[David Mosberger]

>>>>> On Tue, 15 Jul 2003 11:06:38 -0700, Ulrich Drepper <drepper@redhat.com> said:

  Uli> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1

  Uli> David Mosberger wrote:

  >> Well, gcc on ia64 _does_ use libunwind.  There is no reason the
  >> same couldn't be true for x86 and all the other platforms (apart
  >> from a small matter of some implementation work...).

  Uli> The current implementatin is just fine and does not have any
  Uli> legal problems attached to it.

Huh?  What "legal problems" does libunwind have?  Care to elaborate?

	--david

Re: more cancellation work

[Ulrich Drepper]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

David Mosberger wrote:

> Well, gcc on ia64 _does_ use libunwind.  There is no reason the same
> couldn't be true for x86 and all the other platforms (apart from a
> small matter of some implementation work...).

The current implementatin is just fine and does not have any legal
problems attached to it.  So I think we are much better off with the
status quo.

- -- 
- --------------.                        ,-.            444 Castro Street
Ulrich Drepper \    ,-----------------'   \ Mountain View, CA 94041 USA
Red Hat         `--' drepper at redhat.com `---------------------------
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.1 (GNU/Linux)

iD8DBQE/FEKu2ijCOnn/RHQRAoWhAJ93rzMRmV4a4j2JTG0sYiipmcp6PwCfQFD2
dU3Oh3nZvPARd/kM3sybZb8=
=OMMY
-----END PGP SIGNATURE-----

Re: more cancellation work

[David Mosberger]

>>>>> On Tue, 15 Jul 2003 10:30:29 -0700, Ulrich Drepper <drepper@redhat.com> said:

  Uli> We have to use whatever gcc uses so libunwind is no option.

Well, gcc on ia64 _does_ use libunwind.  There is no reason the same
couldn't be true for x86 and all the other platforms (apart from a
small matter of some implementation work...).

	--david

Re: more cancellation work

[Ulrich Drepper]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

David Mosberger wrote:

> Have you considered using libunwind?  The API is portable, full
> implementation exists for ia64, and a partial one for x86.  See:
> 
> 	http://www.hpl.hp.com/research/linux/libunwind/
> 
> for some more info.  I'm not sure exactly what functionality you need,
> but if something is missing, let me know.

We have to use whatever gcc uses so libunwind is no option.

- -- 
- --------------.                        ,-.            444 Castro Street
Ulrich Drepper \    ,-----------------'   \ Mountain View, CA 94041 USA
Red Hat         `--' drepper at redhat.com `---------------------------
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.1 (GNU/Linux)

iD8DBQE/FDo12ijCOnn/RHQRAuWvAKCdZrEmzZ60CqKNzMle3SKJkMsRZgCghyOV
/FIm6SdTyr2Sgpl8bnsT+bg=
=KQQD
-----END PGP SIGNATURE-----

Re: more cancellation work

[David Mosberger]

>>>>> On Tue, 15 Jul 2003 00:22:01 -0700, Ulrich Drepper <drepper@redhat.com> said:

  Uli> rth said he doesn't want to split the file to avoid exposing
  Uli> more interfaces.  This means we have to pull in all of the
  Uli> unwinder code which might mean we get maintainance problems in
  Uli> future where old glibcs cannot be compiled with new gccs.  I
  Uli> see no other solution for this.

  Uli> Another problem is ia64.  The unwinder code for ia64 is
  Uli> different from the rest.  We probably need to pull in that code
  Uli> from gcc as well.  I haven't done it yet which means ia64 might
  Uli> not link/run correctly.

Have you considered using libunwind?  The API is portable, full
implementation exists for ia64, and a partial one for x86.  See:

	http://www.hpl.hp.com/research/linux/libunwind/

for some more info.  I'm not sure exactly what functionality you need,
but if something is missing, let me know.

	--david

more cancellation work

[Ulrich Drepper]

I've checked in some more cancellation work today.  It took a while to
get this done mainly because I went back and forth on some of the
decisions.  This is the time when policies have to be made and I think I
got it down to some reasonable set now.

The problem is the set of functions which POSIX lists as possible
cancellation points plus all the functions not in POSIX.  If the
implementation of the functions uses a function which is a cancellation
point it can also become cancelable.  And in fact it will unless steps
are taken.  For each function a decision has to be made.  In case the
decision is to not make a function cancelable a not-cancelable version
of the syscall wrapper has to be used.

The decisions I made so far are as follows:

~ functions needed for error messages etc are not cancelable.  This
includes catgets, gettext, strerror.  This will allow generating some
output before a thread dies.

~ if possible, all functions related to closing a descriptor are not
cancelable.  Unfortunately close() itself must be cancelable since POSIX
says so.  But think a second what this means.  If close() is canceled it
is not known whether the descriptor has been closed or not.  I.e., the
program leaks.  Better avoid this if possible.

~ the dlfcn code is not cancelable.  Mostly because most of the code is
in ld.so and there it is not cancelable.

~ the headers reflect exactly the cancelability of the function.  I.e.,
even if a function is listed in POSIX as a possible cancellation point
but it is not cancelable in glibc, it still gets the __THROW marker.


I have also introduced some code which inside glibc itself uses the C
cleanup handling.  The use of this feature will increase significantly
in future.  But using it brought up a problem: the unwinder code is now
used internally.  This causes problem since we already compile in parts
of unwind-dw2.c.  If now the other unwind code is pulled in from
libgcc_eh.a we get a collision of symbol due to the few symbols we
already have from our own copy of unwind-dw2.c.  rth said he doesn't
want to split the file to avoid exposing more interfaces.  This means we
have to pull in all of the unwinder code which might mean we get
maintainance problems in future where old glibcs cannot be compiled with
new gccs.  I see no other solution for this.

Another problem is ia64.  The unwinder code for ia64 is different from
the rest.  We probably need to pull in that code from gcc as well.  I
haven't done it yet which means ia64 might not link/run correctly.


Anyway, there is a lot more work to be done.  All possible cancellation
points have to be investigated.  Then all functions implementations need
to be investigated whether they are using (transitively) and cancelable
function.  In case this is the case and the function is in POSIX the
code must e rewritten to avoid the cancellation points.  Otherwise we
have to make a decision.

The whole matter is worse since I know this feature will hardly ever be
used (and rightly so).  It's such a waste.

-- 
--------------.                        ,-.            444 Castro Street
Ulrich Drepper \    ,-----------------'   \ Mountain View, CA 94041 USA
Red Hat         `--' drepper at redhat.com `---------------------------