Pthreads-win32 and static linking

Pthreads-win32 and static linking

[Martin Lambers]

Hello!

The Mingw-cross-env project provides a MinGW cross-compiling environment
for POSIX systems. For various reasons, all included libraries are
static, including the pthreads-win32 library.

This means that all pthreads users need to call
pthread_win32_thread_attach_np() and pthread_win32_thread_detach_np() in
each thread function.

We would like to avoid patching all libraries that use pthread, and
instead change pthreads-win32: instead of using a thread main function
given to pthread_create() directly, pthreads-win32 could use an internal
wrapper that calls the thread main function and also calls the
attach/detach functions if necessary.

To avoid the pthread_win32_process_attach() function, pthreads-win32
could check on each function call whether the library was initialized,
and if not, call pthread_win32_process_attach() itself.

This would induce some overhead, but would allow us to use a large
number of existing libraries unchanged.

Before we start to work on a patch, we would like to know if this
approach has a chance to work. Do you know any technical reasons why
this might fail?

Best regards,
Martin

Re: Pthreads-win32 and static linking

[Ramiro Polla]

[-- Attachment #1: Type: text/plain, Size: 1599 bytes --]

Hi,

On Mon, May 10, 2010 at 4:50 PM, Martin Lambers <marlam@marlam.de> wrote:
> The Mingw-cross-env project provides a MinGW cross-compiling environment
> for POSIX systems. For various reasons, all included libraries are
> static, including the pthreads-win32 library.
>
> This means that all pthreads users need to call
> pthread_win32_thread_attach_np() and pthread_win32_thread_detach_np() in
> each thread function.
>
> We would like to avoid patching all libraries that use pthread, and
> instead change pthreads-win32: instead of using a thread main function
> given to pthread_create() directly, pthreads-win32 could use an internal
> wrapper that calls the thread main function and also calls the
> attach/detach functions if necessary.
>
> To avoid the pthread_win32_process_attach() function, pthreads-win32
> could check on each function call whether the library was initialized,
> and if not, call pthread_win32_process_attach() itself.
>
> This would induce some overhead, but would allow us to use a large
> number of existing libraries unchanged.
>
> Before we start to work on a patch, we would like to know if this
> approach has a chance to work. Do you know any technical reasons why
> this might fail?

This has been discussed before. Search this list's archive. The
solution I'm currently using is attached (to patch latest CVS). It
removes the false wsock32 dependency, remove the need to define
something before using the static lib (in mingw32) and hints the
linker to automatically initialize the library. The library can be
built/used by both MSVC and GCC.

Ramiro Polla

[-- Attachment #2: autostatic.diff --]
[-- Type: application/octet-stream, Size: 6176 bytes --]

Only in pthreads-win32-20091019: autostatic.c
diff -ur pthreads-win32-20091019.orig/GNUmakefile pthreads-win32-20091019/GNUmakefile
--- pthreads-win32-20091019.orig/GNUmakefile	2009-10-19 08:07:32.000000000 -0200
+++ pthreads-win32-20091019/GNUmakefile	2010-02-01 14:45:28.471432337 -0200
@@ -63,7 +63,6 @@
 XOPT	=
 
 RCFLAGS		= --include-dir=.
-LFLAGS		= -lwsock32
 
 # ----------------------------------------------------------------------
 # The library can be built with some alternative behaviour to
diff -ur pthreads-win32-20091019.orig/need_errno.h pthreads-win32-20091019/need_errno.h
--- pthreads-win32-20091019.orig/need_errno.h	2009-10-19 08:07:32.000000000 -0200
+++ pthreads-win32-20091019/need_errno.h	2010-02-02 16:51:10.829325013 -0200
@@ -60,7 +60,7 @@
 #endif
 
 /* declare reference to errno */
-#ifndef PTW32_STATIC_LIB
+#if !defined(PTW32_STATIC_LIB) && !defined(__MINGW32__)
 #  ifdef PTW32_BUILD
 #    define PTW32_DLLPORT __declspec (dllexport)
 #  else
Only in pthreads-win32-20091019: need_errno.h~
diff -ur pthreads-win32-20091019.orig/pthread.c pthreads-win32-20091019/pthread.c
--- pthreads-win32-20091019.orig/pthread.c	2009-10-19 08:07:32.000000000 -0200
+++ pthreads-win32-20091019/pthread.c	2010-02-01 14:26:48.601487556 -0200
@@ -49,6 +49,7 @@
 #include "condvar.c"
 #include "create.c"
 #include "dll.c"
+#include "autostatic.c"
 #include "errno.c"
 #include "exit.c"
 #include "fork.c"
diff -ur pthreads-win32-20091019.orig/pthread_getspecific.c pthreads-win32-20091019/pthread_getspecific.c
--- pthreads-win32-20091019.orig/pthread_getspecific.c	2009-10-19 08:07:32.000000000 -0200
+++ pthreads-win32-20091019/pthread_getspecific.c	2010-02-01 14:28:36.603981857 -0200
@@ -72,12 +72,10 @@
   else
     {
       int lasterror = GetLastError ();
-      int lastWSAerror = WSAGetLastError ();
 
       ptr = TlsGetValue (key->key);
 
       SetLastError (lasterror);
-      WSASetLastError (lastWSAerror);
     }
 
   return ptr;
diff -ur pthreads-win32-20091019.orig/pthread.h pthreads-win32-20091019/pthread.h
--- pthreads-win32-20091019.orig/pthread.h	2009-10-19 08:07:32.000000000 -0200
+++ pthreads-win32-20091019/pthread.h	2010-02-02 16:51:05.047931915 -0200
@@ -533,7 +533,7 @@
  * do NOT define PTW32_BUILD, and then the variables/functions will
  * be imported correctly.
  */
-#ifndef PTW32_STATIC_LIB
+#if !defined(PTW32_STATIC_LIB) && !defined(__MINGW32__)
 #  ifdef PTW32_BUILD
 #    define PTW32_DLLPORT __declspec (dllexport)
 #  else
Only in pthreads-win32-20091019: pthread.h~
diff -ur pthreads-win32-20091019.orig/sched.h pthreads-win32-20091019/sched.h
--- pthreads-win32-20091019.orig/sched.h	2009-10-19 08:07:32.000000000 -0200
+++ pthreads-win32-20091019/sched.h	2010-02-02 16:51:00.665887720 -0200
@@ -76,7 +76,7 @@
  * do NOT define PTW32_BUILD, and then the variables/functions will
  * be imported correctly.
  */
-#ifndef PTW32_STATIC_LIB
+#if !defined(PTW32_STATIC_LIB) && !defined(__MINGW32__)
 #  ifdef PTW32_BUILD
 #    define PTW32_DLLPORT __declspec (dllexport)
 #  else
Only in pthreads-win32-20091019: sched.h~
diff -ur pthreads-win32-20091019.orig/semaphore.h pthreads-win32-20091019/semaphore.h
--- pthreads-win32-20091019.orig/semaphore.h	2009-10-19 08:07:32.000000000 -0200
+++ pthreads-win32-20091019/semaphore.h	2010-02-02 16:50:45.916576916 -0200
@@ -75,7 +75,7 @@
  * do NOT define PTW32_BUILD, and then the variables/functions will
  * be imported correctly.
  */
-#ifndef PTW32_STATIC_LIB
+#if !defined(PTW32_STATIC_LIB) && !defined(__MINGW32__)
 #  ifdef PTW32_BUILD
 #    define PTW32_DLLPORT __declspec (dllexport)
 #  else
Only in pthreads-win32-20091019: semaphore.h~
--- /dev/null	2010-01-29 12:57:37.677072272 -0200
+++ pthreads-win32-20091019/autostatic.c	2010-02-01 14:26:48.601487556 -0200
@@ -0,0 +1,67 @@
+/*
+ * autostatic.c
+ *
+ * Description:
+ * This translation unit implements static library initialisation.
+ *
+ * --------------------------------------------------------------------------
+ *
+ *      Pthreads-win32 - POSIX Threads Library for Win32
+ *      Copyright(C) 1998 John E. Bossom
+ *      Copyright(C) 1999,2005 Pthreads-win32 contributors
+ * 
+ *      Contact Email: rpj@callisto.canberra.edu.au
+ * 
+ *      The current list of contributors is contained
+ *      in the file CONTRIBUTORS included with the source
+ *      code distribution. The list can also be seen at the
+ *      following World Wide Web location:
+ *      http://sources.redhat.com/pthreads-win32/contributors.html
+ * 
+ *      This library is free software; you can redistribute it and/or
+ *      modify it under the terms of the GNU Lesser General Public
+ *      License as published by the Free Software Foundation; either
+ *      version 2 of the License, or (at your option) any later version.
+ * 
+ *      This library is distributed in the hope that it will be useful,
+ *      but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *      MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ *      Lesser General Public License for more details.
+ * 
+ *      You should have received a copy of the GNU Lesser General Public
+ *      License along with this library in the file COPYING.LIB;
+ *      if not, write to the Free Software Foundation, Inc.,
+ *      59 Temple Place - Suite 330, Boston, MA 02111-1307, USA
+ */
+
+#ifdef PTW32_STATIC_LIB
+
+#include "pthread.h"
+#include "implement.h"
+
+static void on_process_init(void)
+{
+    pthread_win32_process_attach_np ();
+}
+
+static void on_process_exit(void)
+{
+    pthread_win32_thread_detach_np  ();
+    pthread_win32_process_detach_np ();
+}
+
+#ifdef __MINGW32__
+#    define attribute_section(a) __attribute__((section(a)))
+#elif defined(_MSC_VER)
+#    define attribute_section(a) __pragma(section(a,long,read)); __declspec(allocate(a))
+#else
+#error compiler not supported!
+#endif
+
+attribute_section(".CRT$XCU") void *msc_ctor = on_process_init;
+attribute_section(".CRT$XPU") void *msc_dtor = on_process_exit;
+
+attribute_section(".ctors"  ) void *gcc_ctor = on_process_init;
+attribute_section(".dtors"  ) void *gcc_dtor = on_process_exit;
+
+#endif /* PTW32_STATIC_LIB */

Re: Pthreads-win32 and static linking

[Ross Johnson]

Martin Lambers wrote:
> Hello!
>
> The Mingw-cross-env project provides a MinGW cross-compiling environment
> for POSIX systems. For various reasons, all included libraries are
> static, including the pthreads-win32 library.
>
> This means that all pthreads users need to call
> pthread_win32_thread_attach_np() and pthread_win32_thread_detach_np() in
> each thread function.
>   
Only Windows native threads that call POSIX threads routines. See note 
below.
> We would like to avoid patching all libraries that use pthread, and
> instead change pthreads-win32: instead of using a thread main function
> given to pthread_create() directly, pthreads-win32 could use an internal
> wrapper that calls the thread main function and also calls the
> attach/detach functions if necessary.
>   
Hi Martin,

Apologies for the slow response.

Please try the current pthreads-win32 CVS repository head version. 
Anonymous CVS access instructions are on our web page at 
http://sourceware.org/pthreads-win32
I've applied most of Ramiro Polla's patches and tested the MinGW32 GCC 
static build across the full test suite. The patches that I've omitted 
are just conditional compiler directives that broke the DLL build and 
did not appear to affect the static build. I've emailed Ramiro privately 
for comment.

Everyone may like to note:
It is not necessary to call pthread_win32_thread_*_np() for any threads 
created through pthread_create(). That is only necessary for Windows 
native threads that call POSIX API routines in order to interact with 
POSIX threads and only when statically linked (often only the primary 
Windows thread). That is, pthread_create() already effectively does what 
Martin has suggested. I've updated the README.NONPORTABLE file to 
hopefully make that clearer.

So with Ramiro's patches most applications should now run unmodified 
when statically linked.

This otherwise transparent ability for Windows and POSIX threads to 
freely interact (via either API) is an aspect of John Bossom's original 
code that has been deliberately retained throughout the evolution of the 
library.

Regards.
Ross

Re: Pthreads-win32 and static linking

[John E. Bossom]

Thanks, Ross...

The original intent when I first wrote the original library was to
have the pthreads-win32 library be usable for those that also wrote
IIS plugin/dlls ... i.e. Windows creates the thread of execution prior
to calling your own plugin... I wanted to seamlessly plug-into the
already created thread.
(Yes... I'm still lurking around ;^)

John E. Bossom


Quoting Ross Johnson <Ross.Johnson@homemail.com.au>:

> Martin Lambers wrote:
>> Hello!
>>
>> The Mingw-cross-env project provides a MinGW cross-compiling environment
>> for POSIX systems. For various reasons, all included libraries are
>> static, including the pthreads-win32 library.
>>
>> This means that all pthreads users need to call
>> pthread_win32_thread_attach_np() and pthread_win32_thread_detach_np() in
>> each thread function.
>>
> Only Windows native threads that call POSIX threads routines. See note below.
>> We would like to avoid patching all libraries that use pthread, and
>> instead change pthreads-win32: instead of using a thread main function
>> given to pthread_create() directly, pthreads-win32 could use an internal
>> wrapper that calls the thread main function and also calls the
>> attach/detach functions if necessary.
>>
> Hi Martin,
>
> Apologies for the slow response.
>
> Please try the current pthreads-win32 CVS repository head version.
> Anonymous CVS access instructions are on our web page at
> http://sourceware.org/pthreads-win32
> I've applied most of Ramiro Polla's patches and tested the MinGW32 GCC
> static build across the full test suite. The patches that I've omitted
> are just conditional compiler directives that broke the DLL build and
> did not appear to affect the static build. I've emailed Ramiro
> privately for comment.
>
> Everyone may like to note:
> It is not necessary to call pthread_win32_thread_*_np() for any threads
> created through pthread_create(). That is only necessary for Windows
> native threads that call POSIX API routines in order to interact with
> POSIX threads and only when statically linked (often only the primary
> Windows thread). That is, pthread_create() already effectively does
> what Martin has suggested. I've updated the README.NONPORTABLE file to
> hopefully make that clearer.
>
> So with Ramiro's patches most applications should now run unmodified
> when statically linked.
>
> This otherwise transparent ability for Windows and POSIX threads to
> freely interact (via either API) is an aspect of John Bossom's original
> code that has been deliberately retained throughout the evolution of
> the library.
>
> Regards.
> Ross