* dll_list::load_after_fork() blues (was Re: [ python-Bugs-489709 ] Building Fails ...) [not found] <Pine.LNX.4.33.0112061015100.898-100000@starship.python.net> @ 2001-12-06 9:44 ` Jason Tishler 2001-12-06 10:12 ` Charles Wilson 2001-12-07 6:04 ` Jason Tishler 0 siblings, 2 replies; 32+ messages in thread From: Jason Tishler @ 2001-12-06 9:44 UTC (permalink / raw) To: Michael Hudson; +Cc: david_abrahams, Cygwin Michael, On Thu, Dec 06, 2001 at 10:15:42AM -0500, Michael Hudson wrote: > know anything about this? > > "no" is a perfectly acceptable answer :) Unfortunately, yes. See below... > building under Cygwin ends with the following error: > > building 'gdbm' extension > gcc -g -O3 -Wall -Wstrict-prototypes -DUSE_DL_IMPORT -I. -I/cygdrive/e/Python-2.2b2/./Include -I/usr/local/include -IInclude/ -c /cygdrive/e/Python-2.2b2/Modules/gdbmmodule.c -o build/temp.cygwin-1.3.5-i686-2.2/gdbmmodule.o > e:\buildpy\python.exe: *** unable to remap > C:\cygnus\bin\cygssl.dll to same address as parent -- 0x1A7C0000 > 0 [main] python 1292 sync_with_child: child 1108 (0xF0) died before initialization with status code 0x1 > 25901 [main] python 1292 sync_with_child: *** child state child loading dlls > error: Resource temporarily unavailable > make: *** [sharedmods] Error 1 The above occurs during Cygwin's fork() when the Cygwin DLL cannot load a DLL to the same address in the child that it had in the parent. I have seen this during Python 2.1.1 regression tests with threads enabled. My WAG is that during a regression test with threads enabled more DLLs (i.e., extension modules) have been loaded (i.e., imported) into the parent's address space. Hence, the probability of a DLL address clash is higher than without threads enabled. Otherwise, just to be clear, I feel that this problem has nothing to do with threads. I have just reproduced this build problem with Python 2.2b2 too. However, I don't believe that it is related to the particular Python version one attempts to build. The reason that we are observing this problem now is that very recently some packages (e.g., OpenSSL) were just updated with their static libraries replaced by their shared (i.e., DLL) counterparts. Hence, the probability of an address clash just increased some more. Previously, I tried suggestion (a) from: http://sources.redhat.com/ml/cygwin/2001-09/msg01165.html Unfortunately, even the non-threads Python 2.1.1 failed its regression test with this error. Also, I searched the MSDN and found the rebase utility: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/tools/perfutil_2z39.asp which could possibly solve this problem. I haven't tried it yet, but even if it does solve the problem, then people without the MS SDK will not be able to build Cygwin Python. :,( Solving this problem has been on my list, but I thought that I could procrastinate until after the threads issue was resolved. Apparently, I was wrong. And as typical with my luck, major Cygwin Python problems are synchronized with Python releases. Sigh... A few months ago, I did some investigation on this problem. What I learned from adding some printfs to dll_list::load_after_fork() was that the DLL in question was *already* remapped from the its preferred image-base in the parent while this address was still available in the child. This empirical evidence seems to contradict the following source code comment: /* It loaded in the wrong place. Dunno why this happens but it always seems to happen when there are multiple DLLs attempting to load into the same address space. In the "forked" process, the second DLL always loads into a different location. */ In this case, it seems like the DLL was loaded in a "different" location in the parent not the child. Since this code *and* my understanding of the issues seemed tenuous at best, I placed this one very gingerly back onto my "round tuit" list. Any help with this problem would be greatly appreciated. > Comment By: Michael Hudson (mwh) > Date: 2001-12-06 05:53 > > PS: can we add jason as a developer? then we can assign > bugs to him <wink>. Hmm... :,) Thanks, Jason -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/ ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: dll_list::load_after_fork() blues (was Re: [ python-Bugs-489709 ] Building Fails ...) 2001-12-06 9:44 ` dll_list::load_after_fork() blues (was Re: [ python-Bugs-489709 ] Building Fails ...) Jason Tishler @ 2001-12-06 10:12 ` Charles Wilson 2001-12-07 6:54 ` Michael Hudson 2001-12-07 15:11 ` Robert Collins 2001-12-07 6:04 ` Jason Tishler 1 sibling, 2 replies; 32+ messages in thread From: Charles Wilson @ 2001-12-06 10:12 UTC (permalink / raw) To: Jason Tishler; +Cc: Michael Hudson, david_abrahams, Cygwin Jason Tishler wrote: >> >>building 'gdbm' extension >>gcc -g -O3 -Wall -Wstrict-prototypes -DUSE_DL_IMPORT -I. -I/cygdrive/e/Python-2.2b2/./Include -I/usr/local/include -IInclude/ -c /cygdrive/e/Python-2.2b2/Modules/gdbmmodule.c -o build/temp.cygwin-1.3.5-i686-2.2/gdbmmodule.o >>e:\buildpy\python.exe: *** unable to remap >>C:\cygnus\bin\cygssl.dll to same address as parent -- 0x1A7C0000 >> 0 [main] python 1292 sync_with_child: child 1108 (0xF0) died before initialization with status code 0x1 >> 25901 [main] python 1292 sync_with_child: *** child state child loading dlls >>error: Resource temporarily unavailable >>make: *** [sharedmods] Error 1 >> > > The above occurs during Cygwin's fork() when the Cygwin DLL cannot > load a DLL to the same address in the child that it had in the parent. > I have seen this during Python 2.1.1 regression tests with threads > enabled. Part of the problem may be that cyggdbm.dll was built with --auto-image-base. It was later demonstrated that this can cause problems with fork; you're better off just letting ld assign the default dllbase, which means that EVERY process will remap the dll at runtime. Thus, no hardcoded conflicts. Downside: *very* slightly delay in loading DLLs -- probably unnoticeable. (Did I get that right, robert?) Anyway, I plan to redo cyggdbm "eventually" without the --auto-image-base. Doing so *may* fix this problem, but I'm not sure... --Chuck -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/ ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: dll_list::load_after_fork() blues (was Re: [ python-Bugs-489709 ] Building Fails ...) 2001-12-06 10:12 ` Charles Wilson @ 2001-12-07 6:54 ` Michael Hudson 2001-12-07 7:03 ` David Abrahams 2001-12-07 15:11 ` Robert Collins 1 sibling, 1 reply; 32+ messages in thread From: Michael Hudson @ 2001-12-07 6:54 UTC (permalink / raw) To: Charles Wilson; +Cc: Jason Tishler, david_abrahams, Cygwin On Thu, 6 Dec 2001, Charles Wilson wrote: > Jason Tishler wrote: > > > >> > >>building 'gdbm' extension > >>gcc -g -O3 -Wall -Wstrict-prototypes -DUSE_DL_IMPORT -I. -I/cygdrive/e/Python-2.2b2/./Include -I/usr/local/include -IInclude/ -c /cygdrive/e/Python-2.2b2/Modules/gdbmmodule.c -o build/temp.cygwin-1.3.5-i686-2.2/gdbmmodule.o > >>e:\buildpy\python.exe: *** unable to remap > >>C:\cygnus\bin\cygssl.dll to same address as parent -- 0x1A7C0000 > >> 0 [main] python 1292 sync_with_child: child 1108 (0xF0) died before initialization with status code 0x1 > >> 25901 [main] python 1292 sync_with_child: *** child state child loading dlls > >>error: Resource temporarily unavailable > >>make: *** [sharedmods] Error 1 > >> > > > > The above occurs during Cygwin's fork() when the Cygwin DLL cannot > > load a DLL to the same address in the child that it had in the parent. > > I have seen this during Python 2.1.1 regression tests with threads > > enabled. > > > Part of the problem may be that cyggdbm.dll was built with > --auto-image-base. It was later demonstrated that this can cause > problems with fork; you're better off just letting ld assign the default > dllbase, which means that EVERY process will remap the dll at runtime. > Thus, no hardcoded conflicts. Downside: *very* slightly delay in > loading DLLs -- probably unnoticeable. > > (Did I get that right, robert?) > > Anyway, I plan to redo cyggdbm "eventually" without the > --auto-image-base. Doing so *may* fix this problem, but I'm not sure... I didn't get Jason's email. Odd that. Anyway, it's more than just gdbm; if I stop that building, then termios fails in a similar manner; disable that and it's resource. And so on. This is really ******* annoying (tm); I'm trying to fix generic Python bugs and have to scp my changes onto another machine to test them... Cheers, M. -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/ ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: dll_list::load_after_fork() blues (was Re: [ python-Bugs-489709 ] Building Fails ...) 2001-12-07 6:54 ` Michael Hudson @ 2001-12-07 7:03 ` David Abrahams 2001-12-07 7:32 ` Jason Tishler 0 siblings, 1 reply; 32+ messages in thread From: David Abrahams @ 2001-12-07 7:03 UTC (permalink / raw) To: Michael Hudson, Charles Wilson; +Cc: Jason Tishler, david_abrahams, Cygwin FWIW, Jason's patch worked for me, except for curses of course which seems to be broken in cygwin. -Dave ----- Original Message ----- From: "Michael Hudson" <mwh@python.net> To: "Charles Wilson" <cwilson@ece.gatech.edu> Cc: "Jason Tishler" <jason@tishler.net>; <david_abrahams@users.sourceforge.net>; "Cygwin" <cygwin@sources.redhat.com> Sent: Friday, December 07, 2001 9:50 AM Subject: Re: dll_list::load_after_fork() blues (was Re: [ python-Bugs-489709 ] Building Fails ...) > On Thu, 6 Dec 2001, Charles Wilson wrote: > > > Jason Tishler wrote: > > > > > > >> > > >>building 'gdbm' extension > > >>gcc -g -O3 -Wall -Wstrict-prototypes -DUSE_DL_IMPORT -I. -I/cygdrive/e/Pyt hon-2.2b2/./Include -I/usr/local/include -IInclude/ -c /cygdrive/e/Python-2.2b2/Modules/gdbmmodule.c -o build/temp.cygwin-1.3.5-i686-2.2/gdbmmodule.o > > >>e:\buildpy\python.exe: *** unable to remap > > >>C:\cygnus\bin\cygssl.dll to same address as parent -- 0x1A7C0000 > > >> 0 [main] python 1292 sync_with_child: child 1108 (0xF0) died before initialization with status code 0x1 > > >> 25901 [main] python 1292 sync_with_child: *** child state child loading dlls > > >>error: Resource temporarily unavailable > > >>make: *** [sharedmods] Error 1 > > >> > > > > > > The above occurs during Cygwin's fork() when the Cygwin DLL cannot > > > load a DLL to the same address in the child that it had in the parent. > > > I have seen this during Python 2.1.1 regression tests with threads > > > enabled. > > > > > > Part of the problem may be that cyggdbm.dll was built with > > --auto-image-base. It was later demonstrated that this can cause > > problems with fork; you're better off just letting ld assign the default > > dllbase, which means that EVERY process will remap the dll at runtime. > > Thus, no hardcoded conflicts. Downside: *very* slightly delay in > > loading DLLs -- probably unnoticeable. > > > > (Did I get that right, robert?) > > > > Anyway, I plan to redo cyggdbm "eventually" without the > > --auto-image-base. Doing so *may* fix this problem, but I'm not sure... > > I didn't get Jason's email. Odd that. > > Anyway, it's more than just gdbm; if I stop that building, then termios > fails in a similar manner; disable that and it's resource. And so on. > > This is really ******* annoying (tm); I'm trying to fix generic Python > bugs and have to scp my changes onto another machine to test them... > > Cheers, > M. > -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/ ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: dll_list::load_after_fork() blues (was Re: [ python-Bugs-489709 ] Building Fails ...) 2001-12-07 7:03 ` David Abrahams @ 2001-12-07 7:32 ` Jason Tishler 0 siblings, 0 replies; 32+ messages in thread From: Jason Tishler @ 2001-12-07 7:32 UTC (permalink / raw) To: David Abrahams; +Cc: Michael Hudson, Charles Wilson, david_abrahams, Cygwin Dave, On Fri, Dec 07, 2001 at 10:03:24AM -0500, David Abrahams wrote: > FWIW, Jason's patch worked for me, Thanks for the feedback. > except for curses of course which seems to be broken in cygwin. The above is a known problem, see the following for details: http://sources.redhat.com/ml/cygwin-apps/2001-11/msg00260.html Jason -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/ ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: dll_list::load_after_fork() blues (was Re: [ python-Bugs-489709 ] Building Fails ...) 2001-12-06 10:12 ` Charles Wilson 2001-12-07 6:54 ` Michael Hudson @ 2001-12-07 15:11 ` Robert Collins 2001-12-07 17:02 ` Charles Wilson ` (2 more replies) 1 sibling, 3 replies; 32+ messages in thread From: Robert Collins @ 2001-12-07 15:11 UTC (permalink / raw) To: Charles Wilson, Jason Tishler; +Cc: Michael Hudson, david_abrahams, Cygwin ----- Original Message ----- From: "Charles Wilson" <cwilson@ece.gatech.edu> > > The above occurs during Cygwin's fork() when the Cygwin DLL cannot > > load a DLL to the same address in the child that it had in the parent. > > I have seen this during Python 2.1.1 regression tests with threads > > enabled. > > > Part of the problem may be that cyggdbm.dll was built with > --auto-image-base. It was later demonstrated that this can cause > problems with fork; you're better off just letting ld assign the default > dllbase, which means that EVERY process will remap the dll at runtime. > Thus, no hardcoded conflicts. Downside: *very* slightly delay in > loading DLLs -- probably unnoticeable. > > (Did I get that right, robert?) Yes. There is actually a longer term solution... which is to 'rebase' every cygwin linked .dll on a particular system to not conflict with each other - which has to be done by setup.exe. Rob > Anyway, I plan to redo cyggdbm "eventually" without the > --auto-image-base. Doing so *may* fix this problem, but I'm not sure... > > --Chuck > > > > > > > -- > Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple > Bug reporting: http://cygwin.com/bugs.html > Documentation: http://cygwin.com/docs.html > FAQ: http://cygwin.com/faq/ > > -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/ ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: dll_list::load_after_fork() blues (was Re: [ python-Bugs-489709 ] Building Fails ...) 2001-12-07 15:11 ` Robert Collins @ 2001-12-07 17:02 ` Charles Wilson 2001-12-07 17:29 ` Christopher Faylor 2001-12-07 18:41 ` dll_list::load_after_fork() blues (was Re: [ python-Bugs-489709 ] Building Fails ...) Robert Collins 2001-12-07 18:19 ` Norman Vine 2001-12-10 4:46 ` Jason Tishler 2 siblings, 2 replies; 32+ messages in thread From: Charles Wilson @ 2001-12-07 17:02 UTC (permalink / raw) To: Robert Collins; +Cc: Jason Tishler, Michael Hudson, david_abrahams, Cygwin Robert Collins wrote: > > Part of the problem may be that cyggdbm.dll was built with > > --auto-image-base. It was later demonstrated that this can cause > > problems with fork; you're better off just letting ld assign the > default > > dllbase, which means that EVERY process will remap the dll at runtime. > > Thus, no hardcoded conflicts. Downside: *very* slightly delay in > > loading DLLs -- probably unnoticeable. > > > > (Did I get that right, robert?) > > Yes. There is actually a longer term solution... which is to 'rebase' > every cygwin linked .dll on a particular system to not conflict with > each other - which has to be done by setup.exe. Yes, but only with the apps and dll's that setup knows about. Recall the discussion on this list some months ago concerning sybase DLL's (I think). Somebody had a custom cygwin app that linked to vendor-supplied database DLLs as well as cygwin stuff (which is fine as long as the resulting app is not distributed...GPL conflicts notwithstanding). Anyway, they had a problem after upgrading to a new cygwinish dll (cygncurses?? I think) w.r.t. load-on-fork. There's no way setup/rebase can be used to avoid that problem a_priori...is there? (As I recall, the person did a 'hand rebase' on his own system to work around it...aware that he would have to repeat the process every time he updated that problem DLL from the cygwin dist) --Chuck -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/ ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: dll_list::load_after_fork() blues (was Re: [ python-Bugs-489709 ] Building Fails ...) 2001-12-07 17:02 ` Charles Wilson @ 2001-12-07 17:29 ` Christopher Faylor 2001-12-12 5:14 ` rebase utility (was Re: dll_list::load_after_fork() blues ...) Jason Tishler 2001-12-07 18:41 ` dll_list::load_after_fork() blues (was Re: [ python-Bugs-489709 ] Building Fails ...) Robert Collins 1 sibling, 1 reply; 32+ messages in thread From: Christopher Faylor @ 2001-12-07 17:29 UTC (permalink / raw) To: cygwin On Fri, Dec 07, 2001 at 07:54:32PM -0500, Charles Wilson wrote: >Anyway, they had a problem after upgrading to a new cygwinish dll >(cygncurses?? I think) w.r.t. load-on-fork. There's no way setup/rebase >can be used to avoid that problem a_priori...is there? (As I recall, >the person did a 'hand rebase' on his own system to work around >it...aware that he would have to repeat the process every time he >updated that problem DLL from the cygwin dist) A rebase utility for cygwin would be pretty nice, I think. We could even include a .bat file which could iterate over all of the dlls in /bin and make sure there were no conflicts. cgf -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/ ^ permalink raw reply [flat|nested] 32+ messages in thread
* rebase utility (was Re: dll_list::load_after_fork() blues ...) 2001-12-07 17:29 ` Christopher Faylor @ 2001-12-12 5:14 ` Jason Tishler 2001-12-12 5:45 ` Robert Collins 2001-12-12 6:04 ` Jason Tishler 0 siblings, 2 replies; 32+ messages in thread From: Jason Tishler @ 2001-12-12 5:14 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 578 bytes --] Chris, On Fri, Dec 07, 2001 at 08:02:22PM -0500, Christopher Faylor wrote: > A rebase utility for cygwin would be pretty nice, I think. Attached is the beginning of a rebase utility. It is modeled after the MS one: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/tools/perfutil_2z39.asp So far I have only implemented the -b and -d options. If a stand-alone utility is still deemed useful (as opposed to integrating the functionality directly into setup.exe), then I will "GNU-ize", clean up, and submit it to cygwin-patches for consideration. Jason [-- Attachment #2: rebase.cc --] [-- Type: text/plain, Size: 3236 bytes --] /* * Copyright (c) 2001 Jason Tishler * * This program is free software; you can redistribute it and/or * modify it under the terms of the GNU General Public License * as published by the Free Software Foundation; either version 2 * of the License, or (at your option) any later version. * * This program is distributed in the hope that it will be useful, * but WITHOUT ANY WARRANTY; without even the implied warranty of * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the * GNU General Public License for more details. * * You should have received a copy of the GNU General Public License * along with this program; if not, write to the Free Software * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. * * $Id: rebase.cc,v 1.4 2001/12/12 12:36:57 jtishler Exp $ */ #include <iostream> #include <sstream> #include <string> #include <time.h> #include <stdlib.h> #include <getopt.h> #include <sys/cygwin.h> #include <windows.h> #include <imagehlp.h> using namespace std; string PosixToWin32(const string& aPosixPath); void ParseArgs(int argc, char* argv[]); unsigned long StringToUlong(const string& aString); void Usage(); ULONG theImageBase = 0; BOOL theDownFlag = FALSE; int theArgsIndex = 0; void main(int argc, char* argv[]) { ParseArgs(argc, argv); ULONG aNewImageBase = theImageBase; for (int i = theArgsIndex; i < argc; i++) { string aFile = PosixToWin32(argv[i]); ULONG anOldImageSize, anOldImageBase, aNewImageSize; ULONG aPrevNewImageBase = aNewImageBase; BOOL aStatus = ReBaseImage( const_cast<char*>(aFile.c_str()), // CurrentImageName 0, // SymbolPath TRUE, // fReBase FALSE, // fRebaseSysfileOk theDownFlag, // fGoingDown 0, // CheckImageSize &anOldImageSize, // OldImageSize &anOldImageBase, // OldImageBase &aNewImageSize, // NewImageSize &aNewImageBase, // NewImageBase time(0)); // TimeStamp // ReBaseImage seems to never returns false! DWORD aStatus2 = GetLastError(); if (aStatus2 != 0) { cerr << "ReBaseImage() failed with last error = " << GetLastError() << endl; exit(2); } cout << aFile << hex << ": new base = " << ((theDownFlag) ? aNewImageBase : aPrevNewImageBase) << ", new size = " << aNewImageSize << endl; } exit(0); } string PosixToWin32(const string& aPosixPath) { char aWin32Path[MAX_PATH]; cygwin_conv_to_win32_path(aPosixPath.c_str(), aWin32Path); return aWin32Path; } void ParseArgs(int argc, char* argv[]) { const char* anOptions = "b:d"; for (int anOption; (anOption = getopt(argc, argv, anOptions)) != -1;) { switch (anOption) { case 'b': theImageBase = StringToUlong(optarg); break; case 'd': theDownFlag = TRUE; break; default: Usage(); exit(1); break; } } if (theImageBase == 0) { Usage(); exit(1); } theArgsIndex = optind; } unsigned long StringToUlong(const string& aString) { stringstream ss; unsigned long aUlong; string::size_type anIndex = aString.find("0x"); if (anIndex == 0) ss << hex << string(aString, 2, aString.size() - 2); else ss << aString; ss >> aUlong; return aUlong; } void Usage() { cerr << "usage: rebase -b BaseAddress [-d] ImageFileName ..." << endl; } [-- Attachment #3: Type: text/plain, Size: 214 bytes --] -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/ ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: rebase utility (was Re: dll_list::load_after_fork() blues ...) 2001-12-12 5:14 ` rebase utility (was Re: dll_list::load_after_fork() blues ...) Jason Tishler @ 2001-12-12 5:45 ` Robert Collins 2001-12-12 6:04 ` Jason Tishler 1 sibling, 0 replies; 32+ messages in thread From: Robert Collins @ 2001-12-12 5:45 UTC (permalink / raw) To: Jason Tishler, cygwin ----- Original Message ----- From: "Jason Tishler" <jason@tishler.net> To: <cygwin@cygwin.com> Sent: Thursday, December 13, 2001 12:07 AM Subject: rebase utility (was Re: dll_list::load_after_fork() blues ...) > Chris, > > On Fri, Dec 07, 2001 at 08:02:22PM -0500, Christopher Faylor wrote: > > A rebase utility for cygwin would be pretty nice, I think. > > Attached is the beginning of a rebase utility. It is modeled after the > MS one: > > http://msdn.microsoft.com/library/default.asp?url=/library/en-us/tools/p erfutil_2z39.asp > > So far I have only implemented the -b and -d options. > > If a stand-alone utility is still deemed useful (as opposed to integrating > the functionality directly into setup.exe), then I will "GNU-ize", clean > up, and submit it to cygwin-patches for consideration. If I may add two cents here. Binutils could use this. Setup could use this. A standalone tool may be useful. libtool could use a binutils or standalone util. install.sh could " So I'd suggest you make your utilitie a verrry thin wrapper around a library, and put everything functional in the library. Rob -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/ ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: rebase utility (was Re: dll_list::load_after_fork() blues ...) 2001-12-12 5:14 ` rebase utility (was Re: dll_list::load_after_fork() blues ...) Jason Tishler 2001-12-12 5:45 ` Robert Collins @ 2001-12-12 6:04 ` Jason Tishler 1 sibling, 0 replies; 32+ messages in thread From: Jason Tishler @ 2001-12-12 6:04 UTC (permalink / raw) To: cygwin On Wed, Dec 12, 2001 at 08:07:59AM -0500, Jason Tishler wrote: > On Fri, Dec 07, 2001 at 08:02:22PM -0500, Christopher Faylor wrote: > > A rebase utility for cygwin would be pretty nice, I think. > > Attached is the beginning of a rebase utility. It is modeled after the > MS one: > > [snip] Oops! I forgot to mention how to build rebase: $ g++ -O2 -o rebase rebase.cc -limagehlp Jason -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/ ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: dll_list::load_after_fork() blues (was Re: [ python-Bugs-489709 ] Building Fails ...) 2001-12-07 17:02 ` Charles Wilson 2001-12-07 17:29 ` Christopher Faylor @ 2001-12-07 18:41 ` Robert Collins 1 sibling, 0 replies; 32+ messages in thread From: Robert Collins @ 2001-12-07 18:41 UTC (permalink / raw) To: Charles Wilson; +Cc: Jason Tishler, Michael Hudson, david_abrahams, Cygwin ----- Original Message ----- From: "Charles Wilson" <cwilson@ece.gatech.edu> > > Yes. There is actually a longer term solution... which is to 'rebase' > > every cygwin linked .dll on a particular system to not conflict with > > each other - which has to be done by setup.exe. > > Yes, but only with the apps and dll's that setup knows about. Recall > the discussion on this list some months ago concerning sybase DLL's (I > think). Somebody had a custom cygwin app that linked to > vendor-supplied database DLLs as well as cygwin stuff (which is fine as > long as the resulting app is not distributed...GPL conflicts > notwithstanding). Actually what I have in mind is * changes to install (pseudo code): if (installing a .dll or .exe) rebase and store info in /etc/setup * changes to setup if (installing a .dll or .exe) rebase and store info in /etc/setup rebase: find object size - sz lookup a gap of sz in the address table in /etc/setup find object dependencies foreach if (a cygwin .dll) rebase this if (a non-cygwin .dll) store the base and size info in /etc/setup > Anyway, they had a problem after upgrading to a new cygwinish dll > (cygncurses?? I think) w.r.t. load-on-fork. There's no way setup/rebase > can be used to avoid that problem a_priori...is there? (As I recall, Yes - following the pseudo code above should be ok (because its install system dependent, not build system dependent). MS actually have a tool for developers to do this with - as part of their programs setup.exe. We may even be able to use that tool - which would use the MS local machine database, not one of our own. Rob -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/ ^ permalink raw reply [flat|nested] 32+ messages in thread
* RE: dll_list::load_after_fork() blues (was Re: [ python-Bugs-489709 ] Building Fails ...) 2001-12-07 15:11 ` Robert Collins 2001-12-07 17:02 ` Charles Wilson @ 2001-12-07 18:19 ` Norman Vine 2001-12-07 18:44 ` Robert Collins 2001-12-10 4:46 ` Jason Tishler 2 siblings, 1 reply; 32+ messages in thread From: Norman Vine @ 2001-12-07 18:19 UTC (permalink / raw) To: 'Robert Collins', 'Charles Wilson', 'Jason Tishler' Cc: 'Michael Hudson', david_abrahams, 'Cygwin' Robert Collins writes: > >----- Original Message ----- >From: "Charles Wilson" <cwilson@ece.gatech.edu> >> > The above occurs during Cygwin's fork() when the Cygwin DLL cannot >> > load a DLL to the same address in the child that it had in the parent. >> > I have seen this during Python 2.1.1 regression tests with threads >> > enabled. >> >> >> Part of the problem may be that cyggdbm.dll was built with >> --auto-image-base. It was later demonstrated that this can cause >> problems with fork; you're better off just letting ld assign the default >> dllbase, which means that EVERY process will remap the dll at runtime. >> Thus, no hardcoded conflicts. Downside: *very* slightly delay in >> loading DLLs -- probably unnoticeable. >> >> (Did I get that right, robert?) > >Yes. There is actually a longer term solution... which is to 'rebase' >every cygwin linked .dll on a particular system to not conflict with >each other - which has to be done by setup.exe. FWIW I just tried patching Cygwin's dlopen() function to always rebase dll's on loading with a 'slightly hacked to force rebasing' version of niftyload.cpp < available from http://thecodeproject/dll/niftyload.asp > and got the same errors as Jason reported in his original message. C:\home\jtishler\src\PythonCvs\nothreads\python.exe: *** unable to remap C:\cygwin\bin\cygssl.dll to same address as parent -- 0x1A2F0000 0 [main] python 852 sync_with_child: child 2772(0x158) died before initialization with status code 0x1 996 [main] python 852 sync_with_child: *** child state child loading dlls Traceback (most recent call last): File "test2.py", line 4, in ? pid = os.fork() OSError: [Errno 11] Resource temporarily unavailable The author of the code states that the base address is chosen the wrong way. "Rebased dll's should start from the top of address space and work down" BTW If it isn't obvious I know next to nothing about dll rebaseing issues Cheers Norman -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/ ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: dll_list::load_after_fork() blues (was Re: [ python-Bugs-489709 ] Building Fails ...) 2001-12-07 18:19 ` Norman Vine @ 2001-12-07 18:44 ` Robert Collins 0 siblings, 0 replies; 32+ messages in thread From: Robert Collins @ 2001-12-07 18:44 UTC (permalink / raw) To: nhv, 'Charles Wilson', 'Jason Tishler' Cc: 'Michael Hudson', david_abrahams, 'Cygwin' ----- Original Message ----- From: "Norman Vine" <nhv@cape.com> > FWIW > I just tried patching Cygwin's dlopen() function to always rebase dll's > on loading with a 'slightly hacked to force rebasing' version of Interesting idea. Unfortunately it only covers the dlopened dll's - and we need to ensure the imported .dll's relocate (or not) consistently. It's the consistency thats the problem, not the actual relocation. Rob -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/ ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: dll_list::load_after_fork() blues (was Re: [ python-Bugs-489709 ] Building Fails ...) 2001-12-07 15:11 ` Robert Collins 2001-12-07 17:02 ` Charles Wilson 2001-12-07 18:19 ` Norman Vine @ 2001-12-10 4:46 ` Jason Tishler 2001-12-10 5:13 ` Robert Collins ` (2 more replies) 2 siblings, 3 replies; 32+ messages in thread From: Jason Tishler @ 2001-12-10 4:46 UTC (permalink / raw) To: Cygwin; +Cc: Michael Hudson, david_abrahams [-- Attachment #1: Type: text/plain, Size: 2264 bytes --] On Sat, Dec 08, 2001 at 10:08:14AM +1100, Robert Collins wrote: > Yes. There is actually a longer term solution... which is to 'rebase' > every cygwin linked .dll on a particular system to not conflict with > each other - which has to be done by setup.exe. I just tried a hand rebase of my system using the MS supplied rebase tool to see if this will fix the problem at least for the Python case. Specifically, I rebased the following DLLs: o Python DLL (e.g., libpython2.2.dll) o all Python standard shared extension modules (e.g., _socket.dll) o all Cygwin DLLs in /usr/bin that match cyg*.dll except for the following: - cygwin1.dll: since I believe that it relies on being based at 0x61000000 - cygcurl-2.dll: because it gets "whacked" by rebase and AFAICT is not used by Python anyway - cygtclpip80.dll: because it appears not to be relocatable Additionally, following the suggestions from the MSDN, I rebased from 0x68000000 down. So, all of the above DLLs were rebased into the range of 0x672e0000 - 0x68000000. After rebasing, the minimal test case that previously exhibited the problem: http://cygwin.com/ml/cygwin/2001-12/msg00419.html now works fine. Unfortunately, when I run the complete Python regression test, I still get the same three test failures as reported by Michael without rebasing: test_popen2 test_pty test_socket When I run these tests individually (i.e., not part of the complete test suite), then they pass. Hence, the rebasing appears not to completely solve this problem. See the attached snippet of output from a regression test run (and search for 0x1A). It shows that although there should not be DLL base address conflicts, some DLLs are being rebased in the parent anyway. A few examples are: _socket.dll: rebased to 0x67f50000 loaded at 0x1A260000 cygz.dll: rebased to 0x678b0000 loaded at 0x1A310000 Would other Python users (with access to MS's rebase tool) be willing to try to reproduce my findings to eliminate the possibility of cockpit error on my part? Does anyone understand why the DLLs are being rebased even though there theoretically is no chance of a base address conflict anymore? Thanks, Jason [-- Attachment #2: rebase.txt --] [-- Type: text/plain, Size: 7627 bytes --] test_popen2 0 [main] python 648 dll_list::load_after_fork: remap failed for C:\home\jtishler\src\PythonCvs\nothreads\build\lib.cygwin-1.3.5-i686-2.2\strop.dll with old handle = 0x1A230000, new handle = 0x420000 97411 [main] python 648 dll_list::load_after_fork: remap failed for C:\home\jtishler\src\PythonCvs\nothreads\build\lib.cygwin-1.3.5-i686-2.2\readline.dll with old handle = 0x1A340000, new handle = 0x1A790000 115881 [main] python 648 dll_list::load_after_fork: remap failed twice for C:\home\jtishler\src\PythonCvs\nothreads\build\lib.cygwin-1.3.5-i686-2.2\readline.dll with old handle = 0x1A340000, new handle = 0x1A790000 123685 [main] python 648 dll_list::load_after_fork: name = C:\home\jtishler\src\PythonCvs\nothreads\libpython2.2.dll handle = 0x675E0000 123997 [main] python 648 dll_list::load_after_fork: name = C:\home\jtishler\src\PythonCvs\nothreads\build\lib.cygwin-1.3.5-i686-2.2\math.dll handle = 0x675C0000 124122 [main] python 648 dll_list::load_after_fork: name = C:\home\jtishler\src\PythonCvs\nothreads\build\lib.cygwin-1.3.5-i686-2.2\time.dll handle = 0x67370000 124742 [main] python 648 dll_list::load_after_fork: name = C:\home\jtishler\src\PythonCvs\nothreads\build\lib.cygwin-1.3.5-i686-2.2\errno.dll handle = 0x67890000 124875 [main] python 648 dll_list::load_after_fork: name = C:\home\jtishler\src\PythonCvs\nothreads\build\lib.cygwin-1.3.5-i686-2.2\struct.dll handle = 0x673D0000 125032 [main] python 648 dll_list::load_after_fork: name = C:\home\jtishler\src\PythonCvs\nothreads\build\lib.cygwin-1.3.5-i686-2.2\_codecs.dll handle = 0x67FE0000 125155 [main] python 648 dll_list::load_after_fork: name = C:\home\jtishler\src\PythonCvs\nothreads\build\lib.cygwin-1.3.5-i686-2.2\strop.dll handle = 0x1A230000 125473 [main] python 648 dll_list::load_after_fork: name = C:\home\jtishler\src\PythonCvs\nothreads\build\lib.cygwin-1.3.5-i686-2.2\_testcapi.dll handle = 0x67F30000 125600 [main] python 648 dll_list::load_after_fork: name = C:\home\jtishler\src\PythonCvs\nothreads\build\lib.cygwin-1.3.5-i686-2.2\cStringIO.dll handle = 0x67E70000 126484 [main] python 648 dll_list::load_after_fork: name = C:\cygwin\bin\cygcrypto.dll handle = 0x67D40000 126616 [main] python 648 dll_list::load_after_fork: name = C:\cygwin\bin\cygssl.dll handle = 0x67A60000 126924 [main] python 648 dll_list::load_after_fork: name = C:\home\jtishler\src\PythonCvs\nothreads\build\lib.cygwin-1.3.5-i686-2.2\_socket.dll handle = 0x1A260000 127044 [main] python 648 dll_list::load_after_fork: name = C:\home\jtishler\src\PythonCvs\nothreads\build\lib.cygwin-1.3.5-i686-2.2\cPickle.dll handle = 0x1A2A0000 127650 [main] python 648 dll_list::load_after_fork: name = C:\home\jtishler\src\PythonCvs\nothreads\build\lib.cygwin-1.3.5-i686-2.2\binascii.dll handle = 0x67EC0000 127777 [main] python 648 dll_list::load_after_fork: name = C:\home\jtishler\src\PythonCvs\nothreads\build\lib.cygwin-1.3.5-i686-2.2\termios.dll handle = 0x1A2E0000 127924 [main] python 648 dll_list::load_after_fork: name = C:\cygwin\bin\cygz.dll handle = 0x1A310000 128040 [main] python 648 dll_list::load_after_fork: name = C:\home\jtishler\src\PythonCvs\nothreads\build\lib.cygwin-1.3.5-i686-2.2\zlib.dll handle = 0x672E0000 128337 [main] python 648 dll_list::load_after_fork: name = C:\home\jtishler\src\PythonCvs\nothreads\build\lib.cygwin-1.3.5-i686-2.2\_locale.dll handle = 0x67F80000 128459 [main] python 648 dll_list::load_after_fork: name = C:\home\jtishler\src\PythonCvs\nothreads\build\lib.cygwin-1.3.5-i686-2.2\pcre.dll handle = 0x67500000 129262 [main] python 648 dll_list::load_after_fork: name = C:\home\jtishler\src\PythonCvs\nothreads\build\lib.cygwin-1.3.5-i686-2.2\select.dll handle = 0x67430000 129388 [main] python 648 dll_list::load_after_fork: name = C:\home\jtishler\src\PythonCvs\nothreads\build\lib.cygwin-1.3.5-i686-2.2\regex.dll handle = 0x674B0000 129694 [main] python 648 dll_list::load_after_fork: name = C:\cygwin\bin\cygncurses5.dll handle = 0x67BB0000 129814 [main] python 648 dll_list::load_after_fork: name = C:\cygwin\bin\cygreadline5.dll handle = 0x67AB0000 133807 [main] python 648 dll_list::load_after_fork: name = C:\home\jtishler\src\PythonCvs\nothreads\build\lib.cygwin-1.3.5-i686-2.2\readline.dll handle = 0x1A340000 133958 [main] python 648 dll_list::load_after_fork: name = C:\home\jtishler\src\PythonCvs\nothreads\build\lib.cygwin-1.3.5-i686-2.2\fcntl.dll handle = 0x67870000 143197 [main] python 648 dll_list::load_after_fork: name = C:\home\jtishler\src\PythonCvs\nothreads\build\lib.cygwin-1.3.5-i686-2.2\_weakref.dll handle = 0x67F20000 143353 [main] python 648 dll_list::load_after_fork: name = C:\home\jtishler\src\PythonCvs\nothreads\build\lib.cygwin-1.3.5-i686-2.2\array.dll handle = 0x67F00000 143780 [main] python 648 dll_list::load_after_fork: name = C:\home\jtishler\src\PythonCvs\nothreads\build\lib.cygwin-1.3.5-i686-2.2\audioop.dll handle = 0x410000 143907 [main] python 648 dll_list::load_after_fork: name = C:\home\jtishler\src\PythonCvs\nothreads\build\lib.cygwin-1.3.5-i686-2.2\cmath.dll handle = 0x67E50000 144735 [main] python 648 dll_list::load_after_fork: name = C:\home\jtishler\src\PythonCvs\nothreads\build\lib.cygwin-1.3.5-i686-2.2\crypt.dll handle = 0x67E30000 144864 [main] python 648 dll_list::load_after_fork: name = C:\cygwin\bin\cyggdbm.dll handle = 0x67D10000 145173 [main] python 648 dll_list::load_after_fork: name = C:\home\jtishler\src\PythonCvs\nothreads\build\lib.cygwin-1.3.5-i686-2.2\gdbm.dll handle = 0x67840000 145295 [main] python 648 dll_list::load_after_fork: name = C:\home\jtishler\src\PythonCvs\nothreads\build\lib.cygwin-1.3.5-i686-2.2\grp.dll handle = 0x67830000 145425 [main] python 648 dll_list::load_after_fork: name = C:\home\jtishler\src\PythonCvs\nothreads\build\lib.cygwin-1.3.5-i686-2.2\md5.dll handle = 0x675A0000 145541 [main] python 648 dll_list::load_after_fork: name = C:\home\jtishler\src\PythonCvs\nothreads\build\lib.cygwin-1.3.5-i686-2.2\sha.dll handle = 0x67410000 146303 [main] python 648 dll_list::load_after_fork: name = C:\home\jtishler\src\PythonCvs\nothreads\build\lib.cygwin-1.3.5-i686-2.2\_hotshot.dll handle = 0x640000 146453 [main] python 648 dll_list::load_after_fork: name = C:\home\jtishler\src\PythonCvs\nothreads\build\lib.cygwin-1.3.5-i686-2.2\parser.dll handle = 0x670000 146761 [main] python 648 dll_list::load_after_fork: name = C:\home\jtishler\src\PythonCvs\nothreads\build\lib.cygwin-1.3.5-i686-2.2\imageop.dll handle = 0x6B0000 146880 [main] python 648 dll_list::load_after_fork: name = C:\home\jtishler\src\PythonCvs\nothreads\build\lib.cygwin-1.3.5-i686-2.2\rgbimg.dll handle = 0x67470000 147536 [main] python 648 dll_list::load_after_fork: name = C:\home\jtishler\src\PythonCvs\nothreads\build\lib.cygwin-1.3.5-i686-2.2\xreadlines.dll handle = 0x6E0000 147664 [main] python 648 dll_list::load_after_fork: name = C:\home\jtishler\src\PythonCvs\nothreads\build\lib.cygwin-1.3.5-i686-2.2\operator.dll handle = 0x67560000 147969 [main] python 648 dll_list::load_after_fork: name = C:\home\jtishler\src\PythonCvs\nothreads\build\lib.cygwin-1.3.5-i686-2.2\mmap.dll handle = 0x700000 C:\home\jtishler\src\PythonCvs\nothreads\python.exe: *** unable to remap C:\home\jtishler\src\PythonCvs\nothreads\build\lib.cygwin-1.3.5-i686-2.2\mmap.dll to same address as parent -- 0x1A790000 0 [main] python 1628 sync_with_child: child 648(0x1D8) died before initialization with status code 0x1 225 [main] python 1628 sync_with_child: *** child state child loading dlls test test_popen2 crashed -- exceptions.OSError: [Errno 11] Resource temporarily unavailable [-- Attachment #3: Type: text/plain, Size: 214 bytes --] -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/ ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: dll_list::load_after_fork() blues (was Re: [ python-Bugs-489709 ] Building Fails ...) 2001-12-10 4:46 ` Jason Tishler @ 2001-12-10 5:13 ` Robert Collins 2001-12-10 7:44 ` Jason Tishler 2001-12-10 7:23 ` Michael Hudson 2001-12-16 0:31 ` dll_list::load_after_fork() blues (was Re: [ python-Bugs-489709 ] Building Fails ...) Robert Collins 2 siblings, 1 reply; 32+ messages in thread From: Robert Collins @ 2001-12-10 5:13 UTC (permalink / raw) To: Jason Tishler, Cygwin; +Cc: Michael Hudson, david_abrahams Possibly because the cygwin heap is getting allocated across where those .dll's would go. Rob === ----- Original Message ----- From: "Jason Tishler" <jason@tishler.net> To: "Cygwin" <cygwin@sources.redhat.com> Cc: "Michael Hudson" <mwh@python.net>; <david_abrahams@users.sourceforge.net> Sent: Monday, December 10, 2001 11:46 PM Subject: Re: dll_list::load_after_fork() blues (was Re: [ python-Bugs-489709 ] Building Fails ...) > On Sat, Dec 08, 2001 at 10:08:14AM +1100, Robert Collins wrote: > > Yes. There is actually a longer term solution... which is to 'rebase' > > every cygwin linked .dll on a particular system to not conflict with > > each other - which has to be done by setup.exe. > > I just tried a hand rebase of my system using the MS supplied rebase > tool to see if this will fix the problem at least for the Python case. > Specifically, I rebased the following DLLs: > > o Python DLL (e.g., libpython2.2.dll) > o all Python standard shared extension modules (e.g., _socket.dll) > o all Cygwin DLLs in /usr/bin that match cyg*.dll except for the > following: > > - cygwin1.dll: since I believe that it relies on being based > at 0x61000000 > - cygcurl-2.dll: because it gets "whacked" by rebase and > AFAICT is not used by Python anyway > - cygtclpip80.dll: because it appears not to be relocatable > > Additionally, following the suggestions from the MSDN, I rebased from > 0x68000000 down. So, all of the above DLLs were rebased into the range > of 0x672e0000 - 0x68000000. > > After rebasing, the minimal test case that previously exhibited the > problem: > > http://cygwin.com/ml/cygwin/2001-12/msg00419.html > > now works fine. > > Unfortunately, when I run the complete Python regression test, I still > get the same three test failures as reported by Michael without rebasing: > > test_popen2 > test_pty > test_socket > > When I run these tests individually (i.e., not part of the complete test > suite), then they pass. Hence, the rebasing appears not to completely > solve this problem. > > See the attached snippet of output from a regression test run (and > search for 0x1A). It shows that although there should not be DLL base > address conflicts, some DLLs are being rebased in the parent anyway. > A few examples are: > > _socket.dll: rebased to 0x67f50000 loaded at 0x1A260000 > cygz.dll: rebased to 0x678b0000 loaded at 0x1A310000 > > Would other Python users (with access to MS's rebase tool) be willing > to try to reproduce my findings to eliminate the possibility of cockpit > error on my part? > > Does anyone understand why the DLLs are being rebased even though there > theoretically is no chance of a base address conflict anymore? > > Thanks, > Jason > ------------------------------------------------------------------------ -------- > -- > Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple > Bug reporting: http://cygwin.com/bugs.html > Documentation: http://cygwin.com/docs.html > FAQ: http://cygwin.com/faq/ -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/ ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: dll_list::load_after_fork() blues (was Re: [ python-Bugs-489709 ] Building Fails ...) 2001-12-10 5:13 ` Robert Collins @ 2001-12-10 7:44 ` Jason Tishler 2001-12-16 0:12 ` Robert Collins 0 siblings, 1 reply; 32+ messages in thread From: Jason Tishler @ 2001-12-10 7:44 UTC (permalink / raw) To: Cygwin; +Cc: Michael Hudson, david_abrahams Rob, On Mon, Dec 10, 2001 at 11:42:01PM +1100, Robert Collins wrote: > Possibly because the cygwin heap is getting allocated across where those > .dll's would go. That is what I figured. Any suggestions on a good address to use during the rebasing? Thanks, Jason -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/ ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: dll_list::load_after_fork() blues (was Re: [ python-Bugs-489709 ] Building Fails ...) 2001-12-10 7:44 ` Jason Tishler @ 2001-12-16 0:12 ` Robert Collins 0 siblings, 0 replies; 32+ messages in thread From: Robert Collins @ 2001-12-16 0:12 UTC (permalink / raw) To: Jason Tishler, Cygwin; +Cc: Michael Hudson, david_abrahams ----- Original Message ----- From: "Jason Tishler" <jason@tishler.net> > Rob, > > On Mon, Dec 10, 2001 at 11:42:01PM +1100, Robert Collins wrote: > > Possibly because the cygwin heap is getting allocated across where those > > .dll's would go. > > That is what I figured. Any suggestions on a good address to use during > the rebasing? Leave ~ the size of the cygwin heap between each .dll needed to load bash + (say) vim. Then use process explorer to see where the heap is occuring, and rebase everything second time avoiding that base + the size of the maximum heap. Rob -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/ ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: dll_list::load_after_fork() blues (was Re: [ python-Bugs-489709 ] Building Fails ...) 2001-12-10 4:46 ` Jason Tishler 2001-12-10 5:13 ` Robert Collins @ 2001-12-10 7:23 ` Michael Hudson 2001-12-10 10:46 ` Norman Vine 2001-12-16 0:31 ` dll_list::load_after_fork() blues (was Re: [ python-Bugs-489709 ] Building Fails ...) Robert Collins 2 siblings, 1 reply; 32+ messages in thread From: Michael Hudson @ 2001-12-10 7:23 UTC (permalink / raw) To: Jason Tishler; +Cc: Cygwin, david_abrahams On Mon, 10 Dec 2001, Jason Tishler wrote: > Unfortunately, when I run the complete Python regression test, I still > get the same three test failures as reported by Michael without rebasing: > > test_popen2 > test_pty > test_socket > > When I run these tests individually (i.e., not part of the complete test > suite), then they pass. Hence, the rebasing appears not to completely > solve this problem. FWIW, and I don't know how much that is, all tests pass if I link _socket statically. Oh, and this is building without threads, it seems. I'll do a new build with threads and see if anything changes, but I doubt it. So what is it that dynamically loading _socket does that screws fork()? Cheers, M. -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/ ^ permalink raw reply [flat|nested] 32+ messages in thread
* RE: dll_list::load_after_fork() blues (was Re: [ python-Bugs-489709 ] Building Fails ...) 2001-12-10 7:23 ` Michael Hudson @ 2001-12-10 10:46 ` Norman Vine 2001-12-12 5:56 ` Jason Tishler 0 siblings, 1 reply; 32+ messages in thread From: Norman Vine @ 2001-12-10 10:46 UTC (permalink / raw) To: 'Michael Hudson', 'Jason Tishler' Cc: 'Cygwin', david_abrahams Michael Hudson writes: > >On Mon, 10 Dec 2001, Jason Tishler wrote: > >> Unfortunately, when I run the complete Python regression test, I still >> get the same three test failures as reported by Michael without rebasing: >> >> test_popen2 >> test_pty >> test_socket >> >> When I run these tests individually (i.e., not part of the complete test >> suite), then they pass. Hence, the rebasing appears not to completely >> solve this problem. > >FWIW, and I don't know how much that is, all tests pass if I link _socket >statically. Oh, and this is building without threads, it seems. I'll do >a new build with threads and see if anything changes, but I doubt it. GREAT IDEA ! I just rebuilt python 2.1.1 with threads and linking _socket statically and all seems to work :-) === / usr / src / python-2.1.1 / Modules / Setup.local # Edit this file for local setup changes # socket line above, and possibly edit the SSL variable: SSL=/usr _socket socketmodule.c \ -DUSE_SSL -I$(SSL)/include -I$(SSL)/include/openssl \ -L$(SSL)/lib -lssl -lcrypto Thanks Michael Norman -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/ ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: dll_list::load_after_fork() blues (was Re: [ python-Bugs-489709 ] Building Fails ...) 2001-12-10 10:46 ` Norman Vine @ 2001-12-12 5:56 ` Jason Tishler 2001-12-18 13:12 ` Cygwin fork() rebase solution (was Re: dll_list::load_after_fork() blues ...) Jason Tishler 0 siblings, 1 reply; 32+ messages in thread From: Jason Tishler @ 2001-12-12 5:56 UTC (permalink / raw) To: Norman Vine; +Cc: 'Michael Hudson', 'Cygwin', david_abrahams On Mon, Dec 10, 2001 at 11:52:08AM -0500, Norman Vine wrote: > Michael Hudson writes: > >FWIW, and I don't know how much that is, all tests pass if I link _socket > >statically. Oh, and this is building without threads, it seems. I'll do > >a new build with threads and see if anything changes, but I doubt it. > > GREAT IDEA ! > > I just rebuilt python 2.1.1 with threads and linking _socket statically > and all seems to work :-) > > === / usr / src / python-2.1.1 / Modules / Setup.local > # Edit this file for local setup changes > # socket line above, and possibly edit the SSL variable: > SSL=/usr > _socket socketmodule.c \ > -DUSE_SSL -I$(SSL)/include -I$(SSL)/include/openssl \ > -L$(SSL)/lib -lssl -lcrypto I just tried the above and it works. However, it only works if one has not fiddled around with rebasing their DLLs. Although this is a good short-term workaround (and the one that I will probably use when I release Python 2.2), I think that we should focus our efforts on trying to solve this problem at its root cause -- Cygwin fork() and DLL base address conflicts. Otherwise, when something else changes we will be back in the same situation. I encourage the interested parties to try the rebase utility that I just posted: http://cygwin.com/ml/cygwin/2001-12/msg00635.html I would be interested if others can reproduce my findings and/or determine how to rebase to fix this Python problem without having to build _socket statically. Thanks, Jason -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/ ^ permalink raw reply [flat|nested] 32+ messages in thread
* Cygwin fork() rebase solution (was Re: dll_list::load_after_fork() blues ...) 2001-12-12 5:56 ` Jason Tishler @ 2001-12-18 13:12 ` Jason Tishler 2001-12-18 14:07 ` Robert Collins 2001-12-19 22:46 ` Cygwin " Norman Vine 0 siblings, 2 replies; 32+ messages in thread From: Jason Tishler @ 2001-12-18 13:12 UTC (permalink / raw) To: Norman Vine, 'Michael Hudson', david_abrahams Cc: 'Cygwin', Python-List [-- Attachment #1: Type: text/plain, Size: 3496 bytes --] On Wed, Dec 12, 2001 at 08:57:44AM -0500, Jason Tishler wrote: > On Mon, Dec 10, 2001 at 11:52:08AM -0500, Norman Vine wrote: > > Michael Hudson writes: > > >FWIW, and I don't know how much that is, all tests pass if I link _socket > > >statically. Oh, and this is building without threads, it seems. I'll do > > >a new build with threads and see if anything changes, but I doubt it. > > > > GREAT IDEA ! > > > > I just rebuilt python 2.1.1 with threads and linking _socket statically > > and all seems to work :-) > > > > [snip] > > I just tried the above and it works. However, it only works if one has > not fiddled around with rebasing their DLLs. > > Although this is a good short-term workaround (and the one that I will > probably use when I release Python 2.2), I think that we should focus > our efforts on trying to solve this problem at its root cause -- Cygwin > fork() and DLL base address conflicts. Otherwise, when something else > changes we will be back in the same situation. I believe that I have found a rebase solution to the Cygwin fork() problem that has been causing Cygwin Python some grief lately. I added an offset option to the attached rebase tool. If I spread the DLLs out by an extra 0x10000, then the fork() problem seems to be mitigate. The following is the command line necessary to build rebase: $ g++ -O2 -o rebase rebase.cc -limagehlp After rebasing the necessary Cygwin DLLs: $ cd /usr/bin $ rebase -d -b 0x68000000 -o 0x10000 cygXpm-X4.dll cygXpm-noX4.dll \ cygbz21.0.dll cygcrypto.dll cygform5.dll cygform6.dll \ cyggdbm.dll cyghistory4.dll cyghistory5.dll cygintl.dll \ cygitcl30.dll cygitk30.dll cygjbig1.dll cygjpeg6b.dll cygmenu5.dll \ cygmenu6.dll cygncurses++5.dll cygncurses++6.dll cygncurses5.dll \ cygncurses6.dll cygpanel5.dll cygpanel6.dll cygpcre.dll \ cygpcreposix.dll cygpng2.dll cygreadline4.dll cygreadline5.dll \ cygregex.dll cygssl.dll cygtcl80.dll cygtclreg80.dll cygtiff3.dll \ cygtk80.dll cygz.dll I am able to build Python *without* the following patch: http://sourceforge.net/tracker/index.php?func=detail&aid=491107&group_id=5470&atid=305470 After rebasing the necessary Cygwin and Python DLLs: $ cd /usr/bin $ rebase -d -b 0x68000000 -o 0x10000 cygXpm-X4.dll cygXpm-noX4.dll \ cygbz21.0.dll cygcrypto.dll cygform5.dll cygform6.dll \ cyggdbm.dll cyghistory4.dll cyghistory5.dll cygintl.dll \ cygitcl30.dll cygitk30.dll cygjbig1.dll cygjpeg6b.dll cygmenu5.dll \ cygmenu6.dll cygncurses++5.dll cygncurses++6.dll cygncurses5.dll \ cygncurses6.dll cygpanel5.dll cygpanel6.dll cygpcre.dll \ cygpcreposix.dll cygpng2.dll cygreadline4.dll cygreadline5.dll \ cygregex.dll cygssl.dll cygtcl80.dll cygtclreg80.dll cygtiff3.dll \ cygtk80.dll cygz.dll ~/src/PythonCvs/nothreads/libpython2.2.dll \ ~/src/PythonCvs/nothreads/build/lib.cygwin-1.3.6-i686-2.2/*.dll I can run the full regression test without any failures *even though* _socket is a shared module. I encourage those interested in Cygwin Python to determine whether or not the above rebase solution works for them too. I'm particular interested in 9x/Me reports since I do not have access to those platforms. Please post your results to the Cygwin and Python mailing lists. If my findings are corroborated, then I will work with the Cygwin team to integrate rebase into setup.exe so that rebasing automatically occurs every time that setup.exe is run. Thanks, Jason [-- Attachment #2: rebase.cc --] [-- Type: text/plain, Size: 3447 bytes --] /* * Copyright (c) 2001 Jason Tishler * * This program is free software; you can redistribute it and/or * modify it under the terms of the GNU General Public License * as published by the Free Software Foundation; either version 2 * of the License, or (at your option) any later version. * * This program is distributed in the hope that it will be useful, * but WITHOUT ANY WARRANTY; without even the implied warranty of * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the * GNU General Public License for more details. * * You should have received a copy of the GNU General Public License * along with this program; if not, write to the Free Software * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. * * $Id: rebase.cc,v 1.5 2001/12/18 20:21:19 jtishler Exp $ */ #include <iostream> #include <sstream> #include <string> #include <time.h> #include <stdlib.h> #include <getopt.h> #include <sys/cygwin.h> #include <windows.h> #include <imagehlp.h> using namespace std; string PosixToWin32(const string& aPosixPath); void ParseArgs(int argc, char* argv[]); unsigned long StringToUlong(const string& aString); void Usage(); ULONG theImageBase = 0; BOOL theDownFlag = FALSE; ULONG theOffset = 0; int theArgsIndex = 0; void main(int argc, char* argv[]) { ParseArgs(argc, argv); ULONG aNewImageBase = theImageBase; for (int i = theArgsIndex; i < argc; i++) { if (theDownFlag) aNewImageBase -= theOffset; string aFile = PosixToWin32(argv[i]); ULONG anOldImageSize, anOldImageBase, aNewImageSize; ULONG aPrevNewImageBase = aNewImageBase; BOOL aStatus = ReBaseImage( const_cast<char*>(aFile.c_str()), // CurrentImageName 0, // SymbolPath TRUE, // fReBase FALSE, // fRebaseSysfileOk theDownFlag, // fGoingDown 0, // CheckImageSize &anOldImageSize, // OldImageSize &anOldImageBase, // OldImageBase &aNewImageSize, // NewImageSize &aNewImageBase, // NewImageBase time(0)); // TimeStamp // ReBaseImage seems to never returns false! DWORD aStatus2 = GetLastError(); if (aStatus2 != 0) { cerr << "ReBaseImage() failed with last error = " << GetLastError() << endl; exit(2); } cout << aFile << hex << ": new base = " << ((theDownFlag) ? aNewImageBase : aPrevNewImageBase) << ", new size = " << aNewImageSize + theOffset << endl; if (!theDownFlag) aNewImageBase += theOffset; } exit(0); } string PosixToWin32(const string& aPosixPath) { char aWin32Path[MAX_PATH]; cygwin_conv_to_win32_path(aPosixPath.c_str(), aWin32Path); return aWin32Path; } void ParseArgs(int argc, char* argv[]) { const char* anOptions = "b:do:"; for (int anOption; (anOption = getopt(argc, argv, anOptions)) != -1;) { switch (anOption) { case 'b': theImageBase = StringToUlong(optarg); break; case 'd': theDownFlag = TRUE; break; case 'o': theOffset = StringToUlong(optarg); break; default: Usage(); exit(1); break; } } if (theImageBase == 0) { Usage(); exit(1); } theArgsIndex = optind; } unsigned long StringToUlong(const string& aString) { stringstream ss; unsigned long aUlong; string::size_type anIndex = aString.find("0x"); if (anIndex == 0) ss << hex << string(aString, 2, aString.size() - 2); else ss << aString; ss >> aUlong; return aUlong; } void Usage() { cerr << "usage: rebase -b BaseAddress [-d] -o Offset ImageFileName ..." << endl; } [-- Attachment #3: Type: text/plain, Size: 214 bytes --] -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/ ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: Cygwin fork() rebase solution (was Re: dll_list::load_after_fork() blues ...) 2001-12-18 13:12 ` Cygwin fork() rebase solution (was Re: dll_list::load_after_fork() blues ...) Jason Tishler @ 2001-12-18 14:07 ` Robert Collins 2001-12-18 19:56 ` cygwin " Christopher Faylor 2001-12-19 22:46 ` Cygwin " Norman Vine 1 sibling, 1 reply; 32+ messages in thread From: Robert Collins @ 2001-12-18 14:07 UTC (permalink / raw) To: Jason Tishler, Norman Vine, 'Michael Hudson', david_abrahams Cc: 'Cygwin', Python-List Your results are enough for me Jason - I'm happy to include this in setup in some form. Rob -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/ ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: cygwin fork() rebase solution (was Re: dll_list::load_after_fork() blues ...) 2001-12-18 14:07 ` Robert Collins @ 2001-12-18 19:56 ` Christopher Faylor 2001-12-21 5:02 ` Jason Tishler 0 siblings, 1 reply; 32+ messages in thread From: Christopher Faylor @ 2001-12-18 19:56 UTC (permalink / raw) To: cygwin On Wed, Dec 19, 2001 at 08:33:50AM +1100, Robert Collins wrote: >Your results are enough for me Jason - I'm happy to include this in >setup in some form. Nice job, Jason. Thanks for following this through. cgf -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/ ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: cygwin fork() rebase solution (was Re: dll_list::load_after_fork() blues ...) 2001-12-18 19:56 ` cygwin " Christopher Faylor @ 2001-12-21 5:02 ` Jason Tishler 2001-12-22 17:08 ` Chris McDonough 0 siblings, 1 reply; 32+ messages in thread From: Jason Tishler @ 2001-12-21 5:02 UTC (permalink / raw) To: cygwin Chris, On Tue, Dec 18, 2001 at 10:28:13PM -0500, Christopher Faylor wrote: > On Wed, Dec 19, 2001 at 08:33:50AM +1100, Robert Collins wrote: > >Your results are enough for me Jason - I'm happy to include this in > >setup in some form. > > Nice job, Jason. Thanks for following this through. You are very welcome. I still have more following through to do: 1. formal contribution of the tool (most likely to binutils) 2. integration with setup.exe Jason -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/ ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: cygwin fork() rebase solution (was Re: dll_list::load_after_fork() blues ...) 2001-12-21 5:02 ` Jason Tishler @ 2001-12-22 17:08 ` Chris McDonough 2002-01-02 10:45 ` Jason Tishler 0 siblings, 1 reply; 32+ messages in thread From: Chris McDonough @ 2001-12-22 17:08 UTC (permalink / raw) To: Jason Tishler, cygwin Hi Jason et. al, FYI, Zope was broken under out-of-the-box Cygwin 1.3.6 because of the fork issue. The rebase solution allows Zope to work again. Many thanks, - C ----- Original Message ----- From: "Jason Tishler" <jason@tishler.net> To: <cygwin@cygwin.com> Sent: Friday, December 21, 2001 7:51 AM Subject: Re: cygwin fork() rebase solution (was Re: dll_list::load_after_fork() blues ...) > Chris, > > On Tue, Dec 18, 2001 at 10:28:13PM -0500, Christopher Faylor wrote: > > On Wed, Dec 19, 2001 at 08:33:50AM +1100, Robert Collins wrote: > > >Your results are enough for me Jason - I'm happy to include this in > > >setup in some form. > > > > Nice job, Jason. Thanks for following this through. > > You are very welcome. I still have more following through to do: > > 1. formal contribution of the tool (most likely to binutils) > 2. integration with setup.exe > > Jason > > -- > Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple > Bug reporting: http://cygwin.com/bugs.html > Documentation: http://cygwin.com/docs.html > FAQ: http://cygwin.com/faq/ > > -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/ ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: cygwin fork() rebase solution (was Re: dll_list::load_after_fork() blues ...) 2001-12-22 17:08 ` Chris McDonough @ 2002-01-02 10:45 ` Jason Tishler 0 siblings, 0 replies; 32+ messages in thread From: Jason Tishler @ 2002-01-02 10:45 UTC (permalink / raw) To: Chris McDonough; +Cc: cygwin Chris, On Sat, Dec 22, 2001 at 10:22:37AM -0500, Chris McDonough wrote: > FYI, Zope was broken under out-of-the-box Cygwin 1.3.6 because of the fork > issue. The rebase solution allows Zope to work again. > > Many thanks, You are very welcome! Thanks for taking the time to see if this solution solved your problem too. Jason -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/ ^ permalink raw reply [flat|nested] 32+ messages in thread
* RE: Cygwin fork() rebase solution (was Re: dll_list::load_after_fork() blues ...) 2001-12-18 13:12 ` Cygwin fork() rebase solution (was Re: dll_list::load_after_fork() blues ...) Jason Tishler 2001-12-18 14:07 ` Robert Collins @ 2001-12-19 22:46 ` Norman Vine 1 sibling, 0 replies; 32+ messages in thread From: Norman Vine @ 2001-12-19 22:46 UTC (permalink / raw) To: 'Jason Tishler', 'Michael Hudson', david_abrahams Cc: 'Cygwin', 'Python-List' Jason Tishler writes: > >I believe that I have found a rebase solution to the Cygwin fork() >problem that has been causing Cygwin Python some grief lately. I added >an offset option to the attached rebase tool. If I spread the DLLs out >by an extra 0x10000, then the fork() problem seems to be mitigate. > >After rebasing the necessary Cygwin and Python DLLs: >I can run the full regression test without any failures *even though* >_socket is a shared module. > If my findings are corroborated, then I will work with the Cygwin team > to integrate rebase into setup.exe so that rebasing automatically occurs > every time that setup.exe is run. HI Jason 'Rebase'ing appears to work :-) Following report on a Win98 box with a 'fresh' Cygwin install Let me know if you need more testing done Cheers Norman ========== bash-2.05a$ uname -a CYGWIN_98-4.10 NHV 1.3.6(0.47/3/2) 2001-12-08 17:02 i686 unknown ========== bash-2.05a$ ./rebase.sh cygcrypto.dll: new base = 67f40000, new size = c0000 cygform5.dll: new base = 67f20000, new size = 20000 cygform6.dll: new base = 67f00000, new size = 20000 cyggdbm.dll: new base = 67ee0000, new size = 20000 cyghistory4.dll: new base = 67ec0000, new size = 20000 cyghistory5.dll: new base = 67ea0000, new size = 20000 cygmenu5.dll: new base = 67e80000, new size = 20000 cygmenu6.dll: new base = 67e60000, new size = 20000 cygncurses++5.dll: new base = 67e20000, new size = 40000 cygncurses++6.dll: new base = 67de0000, new size = 40000 cygncurses5.dll: new base = 67d90000, new size = 50000 cygncurses6.dll: new base = 67d40000, new size = 50000 cygpanel5.dll: new base = 67d20000, new size = 20000 cygpanel6.dll: new base = 67d00000, new size = 20000 cygreadline4.dll: new base = 67cd0000, new size = 30000 cygreadline5.dll: new base = 67c90000, new size = 40000 cygregex.dll: new base = 67c60000, new size = 30000 cygz.dll: new base = 67c30000, new size = 30000 cygssl.dll: new base = 67bf0000, new size = 40000 ======== bash-2.05a$ python regrtest.py Python 2.1.1 (#34, Dec 19 2001, 20:24:16) [GCC 2.95.3-5 (cygwin special)] on cygwin ....... 15 tests skipped: test_al test_bsddb test_cd test_cl test_dbm test_dl test_gl test_imgfile test_largefile test_linuxaudiodev test_locale test_nis test_socketserver test_sunaudiodev test_sundry ========= Cygwin DLL version info: DLL version: 1.3.6 DLL epoch: 19 DLL bad signal mask: 19005 DLL old termios: 5 DLL malloc env: 28 API major: 0 API minor: 47 Shared data: 3 DLL identifier: cygwin1 Mount registry: 2 Cygnus registry name: Cygnus Solutions Cygwin registry name: Cygwin Program options name: Program Options Cygwin mount registry name: mounts v2 Cygdrive flags: cygdrive flags Cygdrive prefix: cygdrive prefix Cygdrive default prefix: Build date: Sat Dec 8 17:02:30 EST 2001 CVS tag: cygwin-1-3-6-6 Shared id: cygwin1S3 -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/ ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: dll_list::load_after_fork() blues (was Re: [ python-Bugs-489709 ] Building Fails ...) 2001-12-10 4:46 ` Jason Tishler 2001-12-10 5:13 ` Robert Collins 2001-12-10 7:23 ` Michael Hudson @ 2001-12-16 0:31 ` Robert Collins 2001-12-18 5:31 ` Jason Tishler 2 siblings, 1 reply; 32+ messages in thread From: Robert Collins @ 2001-12-16 0:31 UTC (permalink / raw) To: Jason Tishler, Cygwin; +Cc: Michael Hudson, david_abrahams ----- Original Message ----- From: "Jason Tishler" <jason@tishler.net> > - cygwin1.dll: since I believe that it relies on being based > at 0x61000000 It does not rely on any base address per se. It *does* reply on the base address being consistent between forked/spawned and exec'd parent and children. That means that the base address must be one of the following: a) Whatever is in the .dll, in all cygwin processs (for a given process hierarchy). (ie it can be different for different invocations of cygwin.bat). b) Relocated consistently on the fly in all cygwin processs (for a given process hierarchy). (ie it can be different for different invocations of cygwin.bat). > - cygcurl-2.dll: because it gets "whacked" by rebase and > AFAICT is not used by Python anyway Can you detail 'whack' a little more? > - cygtclpip80.dll: because it appears not to be relocatable What error do you get? > Unfortunately, when I run the complete Python regression test, I still > get the same three test failures as reported by Michael without rebasing: > > test_popen2 > test_pty > test_socket > > When I run these tests individually (i.e., not part of the complete test > suite), then they pass. Hence, the rebasing appears not to completely > solve this problem. Can you get process explorer to show their address space - get them to fail, and stop rather than terminating (usually there is a dialog you could leave open or you can use the CYGWIN debug flags). Rob -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/ ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: dll_list::load_after_fork() blues (was Re: [ python-Bugs-489709 ] Building Fails ...) 2001-12-16 0:31 ` dll_list::load_after_fork() blues (was Re: [ python-Bugs-489709 ] Building Fails ...) Robert Collins @ 2001-12-18 5:31 ` Jason Tishler 2001-12-18 6:35 ` Robert Collins 0 siblings, 1 reply; 32+ messages in thread From: Jason Tishler @ 2001-12-18 5:31 UTC (permalink / raw) To: Robert Collins; +Cc: Cygwin, Michael Hudson, david_abrahams Rob, On Sun, Dec 16, 2001 at 07:26:37PM +1100, Robert Collins wrote: > From: "Jason Tishler" <jason@tishler.net> > > - cygwin1.dll: since I believe that it relies on being based > > at 0x61000000 > > It does not rely on any base address per se. It *does* reply on the base > address being consistent between forked/spawned and exec'd parent and > children. > [snip] I knew that I was going to put my foot in my mouth with the above feeble explanation. Thanks for setting the record straight. > > - cygcurl-2.dll: because it gets "whacked" by rebase and > > AFAICT is not used by Python anyway > > Can you detail 'whack' a little more? $ cp /usr/bin/cygcurl-2.dll . $ file cygcurl-2.dll cygcurl-2.dll: MS Windows PE Intel 80386 console DLL $ rebase -b 0x68000000 -d cygcurl-2.dll cygcurl-2.dll: new base = 68000000, new size = 20000 $ file cygcurl-2.dll cygcurl-2.dll: MS-DOS executable (EXE), OS/2 or MS Windows $ objdump -p cygcurl-2.dll objdump: cygcurl-2.dll: File format not recognized The above seems to indicated that cygcurl-2.dll did not "survive" a rebase which is why I used the term "whacked." > > - cygtclpip80.dll: because it appears not to be relocatable > > What error do you get? None. The rebase is just ineffective which should not be surprising since: $ file cygtclpip80.dll cygtclpip80.dll: MS Windows PE Intel 80386 ... not relocatable ^^^^^^^^^^^ > > Hence, the rebasing appears not to completely solve this problem. > > Can you get process explorer to show their address space - get them to > fail, and stop rather than terminating (usually there is a dialog you > could leave open or you can use the CYGWIN debug flags). By "process explorer," do you mean Process Explorer (aka HandleEx) from Sysinternals? Or, Process Viewer that is part of Visual Studio? Or, something else? Thanks, Jason -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/ ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: dll_list::load_after_fork() blues (was Re: [ python-Bugs-489709 ] Building Fails ...) 2001-12-18 5:31 ` Jason Tishler @ 2001-12-18 6:35 ` Robert Collins 0 siblings, 0 replies; 32+ messages in thread From: Robert Collins @ 2001-12-18 6:35 UTC (permalink / raw) To: Jason Tishler; +Cc: Cygwin, Michael Hudson, david_abrahams > By "process explorer," do you mean Process Explorer (aka HandleEx) from > Sysinternals? Yes. -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/ ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: dll_list::load_after_fork() blues (was Re: [ python-Bugs-489709 ] Building Fails ...) 2001-12-06 9:44 ` dll_list::load_after_fork() blues (was Re: [ python-Bugs-489709 ] Building Fails ...) Jason Tishler 2001-12-06 10:12 ` Charles Wilson @ 2001-12-07 6:04 ` Jason Tishler 1 sibling, 0 replies; 32+ messages in thread From: Jason Tishler @ 2001-12-07 6:04 UTC (permalink / raw) To: Michael Hudson, david_abrahams; +Cc: Cygwin, Python-List [-- Attachment #1: Type: text/plain, Size: 917 bytes --] The ASSume principle applies once again... On Thu, Dec 06, 2001 at 12:44:26PM -0500, Jason Tishler wrote: > I have just reproduced this build problem with Python 2.2b2 too. However, > I don't believe that it is related to the particular Python version one > attempts to build. Actually, setup.py has been enhanced to check the validity of each module by importing it during the build problem. Hence, the probability of a DLL address clash during fork has increased significantly. The attached patch workarounds this problem. I intend to submit this patch (redone against CVS) to the Python patch collector ASAP so that Python 2.2 does not go out the door on 12/19 busted for Cygwin. If you have any objections or better solutions, please speak up sooner rather than later. Note that I still intend to fix the Cygwin Python fork problem, if possible. Hopefully, this workaround will be only temporary. Jason [-- Attachment #2: setup.py.diff --] [-- Type: text/plain, Size: 601 bytes --] --- setup.py.orig Fri Dec 7 08:21:56 2001 +++ setup.py Fri Dec 7 08:31:21 2001 @@ -159,6 +159,11 @@ if 'Carbon' in ext.extra_link_args: self.announce('WARNING: skipping import check for Carbon-based "%s"' % ext.name) return + # Workaround for Cygwin: Cygwin currently has fork issues when many + # modules have been imported + if self.get_platform() == 'cygwin': + self.announce('WARNING: skipping import check for Cygwin-based "%s"' % ext.name) + return try: __import__(ext.name) except ImportError: [-- Attachment #3: Type: text/plain, Size: 214 bytes --] -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/ ^ permalink raw reply [flat|nested] 32+ messages in thread
end of thread, other threads:[~2002-01-02 18:45 UTC | newest] Thread overview: 32+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- [not found] <Pine.LNX.4.33.0112061015100.898-100000@starship.python.net> 2001-12-06 9:44 ` dll_list::load_after_fork() blues (was Re: [ python-Bugs-489709 ] Building Fails ...) Jason Tishler 2001-12-06 10:12 ` Charles Wilson 2001-12-07 6:54 ` Michael Hudson 2001-12-07 7:03 ` David Abrahams 2001-12-07 7:32 ` Jason Tishler 2001-12-07 15:11 ` Robert Collins 2001-12-07 17:02 ` Charles Wilson 2001-12-07 17:29 ` Christopher Faylor 2001-12-12 5:14 ` rebase utility (was Re: dll_list::load_after_fork() blues ...) Jason Tishler 2001-12-12 5:45 ` Robert Collins 2001-12-12 6:04 ` Jason Tishler 2001-12-07 18:41 ` dll_list::load_after_fork() blues (was Re: [ python-Bugs-489709 ] Building Fails ...) Robert Collins 2001-12-07 18:19 ` Norman Vine 2001-12-07 18:44 ` Robert Collins 2001-12-10 4:46 ` Jason Tishler 2001-12-10 5:13 ` Robert Collins 2001-12-10 7:44 ` Jason Tishler 2001-12-16 0:12 ` Robert Collins 2001-12-10 7:23 ` Michael Hudson 2001-12-10 10:46 ` Norman Vine 2001-12-12 5:56 ` Jason Tishler 2001-12-18 13:12 ` Cygwin fork() rebase solution (was Re: dll_list::load_after_fork() blues ...) Jason Tishler 2001-12-18 14:07 ` Robert Collins 2001-12-18 19:56 ` cygwin " Christopher Faylor 2001-12-21 5:02 ` Jason Tishler 2001-12-22 17:08 ` Chris McDonough 2002-01-02 10:45 ` Jason Tishler 2001-12-19 22:46 ` Cygwin " Norman Vine 2001-12-16 0:31 ` dll_list::load_after_fork() blues (was Re: [ python-Bugs-489709 ] Building Fails ...) Robert Collins 2001-12-18 5:31 ` Jason Tishler 2001-12-18 6:35 ` Robert Collins 2001-12-07 6:04 ` Jason Tishler
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).