[Bug nptl/27880] New: Please provide a pthread pid accessor

[Bug nptl/27880] New: Please provide a pthread pid accessor

[hpa at zytor dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=27880

            Bug ID: 27880
           Summary: Please provide a pthread pid accessor
           Product: glibc
           Version: unspecified
            Status: UNCONFIRMED
          Severity: enhancement
          Priority: P2
         Component: nptl
          Assignee: unassigned at sourceware dot org
          Reporter: hpa at zytor dot com
                CC: drepper.fsp at gmail dot com
  Target Milestone: ---

There is currently no way to access the kernel tid for another pthread, except
via the very cumbersome thread_db interface. The intent is good, to provide
interface isolation, but sometimes having such cross-layer interfaces (e.g.
fileno()) are at the very least serve a "harm reduction" function.

As it is, there are enough reasons access to a tid is necessary that
programmers are employing multiple unsafe techniques:

1. Calling gettid() early in the thread main function and caching that value
indefinitely;
2. Mixing pthreads and raw calls to clone();
3. Extracting or even hard-coding the offset for tid in struct pthread (almost
never using any kind of hard version tying.)

Having an interface like:

pid_t pthread_gettid_np(pthread_t thread)

... would at least allow the interface to be kept in sync with the current
libpthread, and provides a path to returning an error if no meaningful tid is
available (e.g. in the case of M:N threads, or the underlying kernel thread no
longer existing.)

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug nptl/27880] Please provide a pthread pid accessor

[hpa at zytor dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=27880

--- Comment #1 from hpa at zytor dot com <hpa at zytor dot com> ---
>From what I can read, MacOS X seems to have a pthread_threadid_np(thread, &tid)
interface for this purpose; AIX has can get it via the pthread_getthrds_np()
interface.

FreeBSD has pthread_getthreadid_np(), but that only applies to the self thread,
i.e. is equivalent to Linux gettid().

Reusing the MacOS X interface seems most sensible; the pthread_getthrds_np()
interface seems very complex.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug nptl/27880] Please provide a pthread pid accessor

[joseph at codesourcery dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=27880

--- Comment #2 from joseph at codesourcery dot com <joseph at codesourcery dot com> ---
This was previously requested in bug 14300 (but then that was marked as a 
duplicate of bug 6399, then 6399 was marked FIXED with gettid but not 
pthread_gettid_np having been added).

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug nptl/27880] Please provide a pthread pid accessor

[hpa at zytor dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=27880

--- Comment #3 from hpa at zytor dot com <hpa at zytor dot com> ---
Right. I'm asking for the ability to get the tid of a *different* thread,
whereas gettid() gets the tid of the *current* thread.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug nptl/27880] Please provide a pthread pid accessor

[adhemerval.zanella at linaro dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=27880

Adhemerval Zanella <adhemerval.zanella at linaro dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |adhemerval.zanella at linaro dot o
                   |                            |rg
           Assignee|unassigned at sourceware dot org   |adhemerval.zanella at linaro dot o
                   |                            |rg

--- Comment #4 from Adhemerval Zanella <adhemerval.zanella at linaro dot org> ---
I think it is a fair addition, I will work on add this.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug nptl/27880] Please provide a pthread pid accessor

[fweimer at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=27880

Florian Weimer <fweimer at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |fweimer at redhat dot com
           See Also|                            |https://sourceware.org/bugz
                   |                            |illa/show_bug.cgi?id=6399,
                   |                            |https://sourceware.org/bugz
                   |                            |illa/show_bug.cgi?id=14300
              Flags|                            |security-

--- Comment #5 from Florian Weimer <fweimer at redhat dot com> ---
I already have a patch somewhere. I'll see if I can post it.

Some operating systems have a thread ID functionality that provides TIDs that
aren't reused. I think that's probably not what you have in mind. You want the
kernel TID, right?

What about threads that have exited and have not yet been joined? Should we
report the TID that was used when the thread was running?

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug nptl/27880] Please provide a pthread pid accessor

[hpa at zytor dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=27880

--- Comment #6 from hpa at zytor dot com <hpa at zytor dot com> ---
Yes, the kernel tid (which is valid as long as there is a zombie process.)

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug nptl/27880] Please provide a pthread pid accessor

[adhemerval.zanella at linaro dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=27880

--- Comment #7 from Adhemerval Zanella <adhemerval.zanella at linaro dot org> ---
I think the TID value is what he is has in mind, assuming that prototype in
comment #1 returns 'pid_t'.

For thread exit/cancelled I think the sensible way would to return an error to
indicate it.  Another issue, which I think also there is no guaranteee, is
calling the newer interface with pthread_self() on a vfork process (since the
tid won't be update and I am assuming the straightforward solution would be to
just atomic read the TID field in struct pthread).

Also I am not sure about the name, pthread_gettid_np seems sensible but I am
not sure if it make sense for non-Linux systems.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug nptl/27880] Please provide a pthread pid accessor

[hpa at zytor dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=27880

--- Comment #8 from hpa at zytor dot com <hpa at zytor dot com> ---
Since MacOS X already has pthread_threadid_np(), I think we should mimic that.

And yes ->tid seems like the sane thing.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug nptl/27880] Please provide a pthread pid accessor

[fweimer at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=27880

--- Comment #9 from Florian Weimer <fweimer at redhat dot com> ---
(In reply to hpa@zytor.com from comment #8)
> Since MacOS X already has pthread_threadid_np(), I think we should mimic
> that.
> 
> And yes ->tid seems like the sane thing.

pthread_threadid_np seems to return a 64-bit unique ID. Our interface won't do
that, so I think we should use a different function name.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug nptl/27880] Please provide a pthread pid accessor

[mtk.manpages at gmail dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=27880

Michael Kerrisk <mtk.manpages at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |mtk.manpages at gmail dot com

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug nptl/27880] Please provide a pthread pid accessor

[adhemerval.zanella at linaro dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=27880

--- Comment #10 from Adhemerval Zanella <adhemerval.zanella at linaro dot org> ---
It seems that this is trickier than it seems, some issues we might considere
first:

  1. What should we do with detached threads? As for pthread_kill, issuing a
pthread_gettid_np might use an invalid handler (since the pthread_t identifier
might be reused).  This only solution I have is to define it as undefined
behavior, this is not great but to proper support it would incur to keep
tracking or all possible pthread_t identifiers (we already keep the glibc
provided stacks, dl_stack_cache, so it would be a matter to include the user
provided one in the list as special entries).

  2. I think that once we provide this API, developers will start to use o
query if a thread is alive and I am not sure if this is really the proper API
for this. This is the same issue as 1.

  3. How do we handle the concurrent access between pthread_join and
pthread_gettid_np? Once a pthread_join is issued, the pthread_t identifier
might be reused and accessing it should be invalid. pthread_join first
synchronizes using 'joinid' to avoid concurrent pthread_join and then wait the
kernel signal on 'tid' that the thread has finished.  The 'pthread_gettid_np'
naive check would just do a atomic load on tid, however it might read a
transient value between pthread_join 'joinid' setup and the futex wait.  I am
not sure how to handle it correctly.

Also, MacOSX signature is:

  int pthread_gettid_np (pthread_t thread, uint64_t *thread_id)

And it returns the current thread identification if THREAD is NULL, returns
ESRCH for invalid handle (the 1. and 2. issue below), and also consults the
kernel if the identifier can no be obtained.

I think for possible glibc symbols we should use a pid_t instead, the NULL
special arguments is also tricky because if by POSIX pthread_t might be NULL
(this is an implementation detail), for ESRCH we can our
INVALID_NOT_TERMINATED_TD_P (which not the best solution, but it the current
pratice).

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug nptl/27880] Please provide a pthread pid accessor

[hpa at zytor dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=27880

--- Comment #11 from hpa at zytor dot com <hpa at zytor dot com> ---
As far as a pthread_id being reused - it doesn't seem to be fundamentally
different from what can happen with any other use of pthread_id? It is actually
a pointer, so it isn't any different from any other stale pointer?

It really seems to be absolutely no difference to me: if an old pthread_id can
be reused and this causing failures, then *any* of the pthread functions are
affected.

In the case of (2), this actually makes it a bit safer than what the pthreads
interface currently provides: at least there is a second thing to test (the tid
is at least likely to change.)

When a pthread is detached from its underlying kernel thread, this should
return zero or -1 with ESRCH; and as you correctly point out this is pretty
much exactly what your INVALID_NOT_TERMINATED_TD_P() test does...

If this kind of robustness is an issue, you really can't just use a plain
pointer, but this is really unrelated to this specific use case; you would
probably need to do something like a {table index, generation number} token.

The MacOS interface does seem a bit needlessly complicated;
pid_t pthread_gettid_np(pthread_t thread) is definitely simpler.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug nptl/27880] Please provide a pthread pid accessor

[sam at gentoo dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=27880

Sam James <sam at gentoo dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |sam at gentoo dot org

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug nptl/27880] Please provide a pthread pid accessor

[jkriegshauser at gmail dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=27880

Josh Kriegshauser <jkriegshauser at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jkriegshauser at gmail dot com

--- Comment #12 from Josh Kriegshauser <jkriegshauser at gmail dot com> ---
I would really like to see this addition make it in, even if it is only exposed
for the calling thread. Sadly, it seems as though work on it has stalled.

In my use case it's a performance issue. The gettid() syscall takes ~700 ns on
my machine, but reading pthread_self() is ~9 ns.

It would be great to have a performant way of reading the TID out of the struct
pthread.

-- 
You are receiving this mail because:
You are on the CC list for the bug.