when to rebase?

when to rebase?

[Thomas L Roche]

Christopher Faylor Fri, 20 Feb 2004 10:41:45 -0500 -----
> I run rebaseall on all of my systems.

When? After every install? Periodically? Only on weirdness?

Is this just CYA, or is it UG- or FAQ-worthy? (Note that searching 
either's big-HTML versions for 'rebas' fails.)


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: when to rebase?

[Larry Hall]

At 01:23 PM 2/20/2004, Thomas L Roche you wrote:
>Christopher Faylor Fri, 20 Feb 2004 10:41:45 -0500 -----
>> I run rebaseall on all of my systems.
>
>When? After every install? Periodically? Only on weirdness?
>
>Is this just CYA, or is it UG- or FAQ-worthy? (Note that searching 
>either's big-HTML versions for 'rebas' fails.)


It should never hurt to 'rebase' so you could do it after every install.
Rebasing will eventually become automatic, done by setup.  Some 
discussion of 'rebase' could be added to the User's Guide though IMO.



--
Larry Hall                              http://www.rfk.com
RFK Partners, Inc.                      (508) 893-9779 - RFK Office
838 Washington Street                   (508) 893-9889 - FAX
Holliston, MA 01746                     


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: when to rebase?

[Brian Dessent]

Thomas L Roche wrote:
> 
> Christopher Faylor Fri, 20 Feb 2004 10:41:45 -0500 -----
> > I run rebaseall on all of my systems.
> 
> When? After every install? Periodically? Only on weirdness?
> 
> Is this just CYA, or is it UG- or FAQ-worthy? (Note that searching
> either's big-HTML versions for 'rebas' fails.)

It should only be necessary if you get an "Unable to map..." error, as
far as I can tell.

Brian

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: when to rebase?

[Christopher Faylor]

On Fri, Feb 20, 2004 at 11:25:21AM -0800, Brian Dessent wrote:
>Thomas L Roche wrote:
>> 
>> Christopher Faylor Fri, 20 Feb 2004 10:41:45 -0500 -----
>> > I run rebaseall on all of my systems.
>> 
>> When? After every install? Periodically? Only on weirdness?
>> 
>> Is this just CYA, or is it UG- or FAQ-worthy? (Note that searching
>> either's big-HTML versions for 'rebas' fails.)
>
>It should only be necessary if you get an "Unable to map..." error, as
>far as I can tell.

Actually, it should provide a performance boost regardless of whether
you get that error or not.

cgf

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: when to rebase?

[Jason Tishler]

On Fri, Feb 20, 2004 at 01:30:29PM -0500, Larry Hall wrote:
> At 01:23 PM 2/20/2004, Thomas L Roche you wrote:
> >Christopher Faylor Fri, 20 Feb 2004 10:41:45 -0500 -----
> >> I run rebaseall on all of my systems.
> >
> >When? After every install? Periodically? Only on weirdness?
> 
> It should never hurt to 'rebase' so you could do it after every
> install.

Unfortunately, rebasing apparently breaks zsh:

    http://www.cygwin.com/ml/cygwin/2003-12/msg00415.html

Jason

-- 
PGP/GPG Key: http://www.tishler.net/jason/pubkey.asc or key servers
Fingerprint: 7A73 1405 7F2B E669 C19D  8784 1AFD E4CC ECF4 8EF6

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: when to rebase?

[Larry Hall]

At 03:32 PM 2/20/2004, Jason Tishler you wrote:
>On Fri, Feb 20, 2004 at 01:30:29PM -0500, Larry Hall wrote:
>> At 01:23 PM 2/20/2004, Thomas L Roche you wrote:
>> >Christopher Faylor Fri, 20 Feb 2004 10:41:45 -0500 -----
>> >> I run rebaseall on all of my systems.
>> >
>> >When? After every install? Periodically? Only on weirdness?
>> 
>> It should never hurt to 'rebase' so you could do it after every
>> install.
>
>Unfortunately, rebasing apparently breaks zsh:
>
>    http://www.cygwin.com/ml/cygwin/2003-12/msg00415.html


Right.  This is the exception that proves the rule, as they say.  

Hopefully, this zsh problem is a temporary glitch.


--
Larry Hall                              http://www.rfk.com
RFK Partners, Inc.                      (508) 893-9779 - RFK Office
838 Washington Street                   (508) 893-9889 - FAX
Holliston, MA 01746                     


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: when to rebase?

[Peter A. Castro]

On Fri, 20 Feb 2004, Jason Tishler wrote:

> On Fri, Feb 20, 2004 at 01:30:29PM -0500, Larry Hall wrote:
> > At 01:23 PM 2/20/2004, Thomas L Roche you wrote:
> > >Christopher Faylor Fri, 20 Feb 2004 10:41:45 -0500 -----
> > >> I run rebaseall on all of my systems.
> > >
> > >When? After every install? Periodically? Only on weirdness?
> >
> > It should never hurt to 'rebase' so you could do it after every
> > install.
>
> Unfortunately, rebasing apparently breaks zsh:
>
>     http://www.cygwin.com/ml/cygwin/2003-12/msg00415.html

As the zsh maintainer, I'm still waiting for the rebase and gcc
maintainers to make a ruling on just where the problem is.  See:

http://www.cygwin.com/ml/cygwin/2004-02/msg00573.html

> Jason

-- 
Peter A. Castro <doctor@fruitbat.org> or <Peter.Castro@oracle.com>
	"Cats are just autistic Dogs" -- Dr. Tony Attwood

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: when to rebase?

[Jason Tishler]

Peter,

On Fri, Feb 20, 2004 at 12:52:28PM -0800, Peter A. Castro wrote:
> On Fri, 20 Feb 2004, Jason Tishler wrote:
> > Unfortunately, rebasing apparently breaks zsh:
> >
> >     http://www.cygwin.com/ml/cygwin/2003-12/msg00415.html
> 
> As the zsh maintainer, I'm still waiting for the rebase and gcc
> maintainers to make a ruling on just where the problem is.  See:
> 
> http://www.cygwin.com/ml/cygwin/2004-02/msg00573.html

FWIW, I just tried the above with Microsoft's rebase and observed the
same behavior.  Note this is not a "ruling" just another data point.

Jason

-- 
PGP/GPG Key: http://www.tishler.net/jason/pubkey.asc or key servers
Fingerprint: 7A73 1405 7F2B E669 C19D  8784 1AFD E4CC ECF4 8EF6

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: when to rebase?

[Peter A. Castro]

On Fri, 20 Feb 2004, Jason Tishler wrote:

> Peter,
>
> On Fri, Feb 20, 2004 at 12:52:28PM -0800, Peter A. Castro wrote:
> > On Fri, 20 Feb 2004, Jason Tishler wrote:
> > > Unfortunately, rebasing apparently breaks zsh:
> > >
> > >     http://www.cygwin.com/ml/cygwin/2003-12/msg00415.html
> >
> > As the zsh maintainer, I'm still waiting for the rebase and gcc
> > maintainers to make a ruling on just where the problem is.  See:
> >
> > http://www.cygwin.com/ml/cygwin/2004-02/msg00573.html
>
> FWIW, I just tried the above with Microsoft's rebase and observed the
> same behavior.  Note this is not a "ruling" just another data point.

Well, considering what the root problem is, I wouldn't expect MS's rebase
to work any better than the cygwin version.  However, I would like to see
some cooperation on how to resolve it.  This problem is not specific to
zsh.  It has more to do with how exported symbols are treated/managed.
I'd almost propose that rebase change such symbols' value, being rebase
is the one changing the image base of the dll in the first place.
However, I would really like to understand why this symbol exists and if
there is some way to eliminate it and use the dll's real image base
instead.

> Jason

-- 
Peter A. Castro <doctor@fruitbat.org> or <Peter.Castro@oracle.com>
	"Cats are just autistic Dogs" -- Dr. Tony Attwood

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: when to rebase?

[Jason Tishler]

[-- Attachment #1: Type: text/plain, Size: 1670 bytes --]

Peter,

On Fri, Feb 20, 2004 at 10:19:09PM -0800, Peter A. Castro wrote:
> On Fri, 20 Feb 2004, Jason Tishler wrote:
> > FWIW, I just tried the above with Microsoft's rebase and observed
> > the same behavior.  Note this is not a "ruling" just another data
> > point.
> 
> Well, considering what the root problem is, I wouldn't expect MS's
> rebase to work any better than the cygwin version.

Understood.  However, one of my design goals for rebase was to behave as
similar to Microsoft's as possible.  This became especially true when I
replaced Microsoft's ReBaseImage() with the one from Ralf Habacker's
libimagehelper.

> However, I would like to see some cooperation on how to resolve it.

See below for one possible solution...

> This problem is not specific to zsh.

Understood.

> It has more to do with how exported symbols are treated/managed.

Or built...

> I'd almost propose that rebase change such symbols' value, being
> rebase is the one changing the image base of the dll in the first
> place.

Do others agree?  If so, does anyone know how hard it would be to enhance
libimagehelper to handle this case?

> However, I would really like to understand why this symbol exists and
> if there is some way to eliminate it and use the dll's real image base
> instead.

It appears that dllwrap *may* be the culprit.  If I replace dllwrap with
gcc -shared in your testscript, then the problem is avoided.  See
attached patch for the details.

Can you use gcc -shared instead of dllwrap when you build zsh?

Thanks,
Jason

-- 
PGP/GPG Key: http://www.tishler.net/jason/pubkey.asc or key servers
Fingerprint: 7A73 1405 7F2B E669 C19D  8784 1AFD E4CC ECF4 8EF6

[-- Attachment #2: testscript.sh.diff --]
[-- Type: text/plain, Size: 408 bytes --]

--- testscript.sh.orig	2004-02-24 15:52:50.398697600 -0500
+++ testscript.sh	2004-02-24 15:53:52.097416000 -0500
@@ -34,7 +34,7 @@
 echodo "gcc -g -c testdll.c"

 

 echo "## Creating testdll.dll"

-echodo "dllwrap -g --export-all-symbols -o testdll.dll testdll.o -lc"

+echodo "gcc -shared -g -o testdll.dll testdll.o -lc"

 

 echo "## Compiling testmain.c -> testmain.o"

 echodo "gcc -g -c testmain.c"



[-- Attachment #3: Type: text/plain, Size: 218 bytes --]

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: when to rebase?

[Peter A. Castro]

On Tue, 24 Feb 2004, Jason Tishler wrote:

> Peter,
>
> On Fri, Feb 20, 2004 at 10:19:09PM -0800, Peter A. Castro wrote:
> > On Fri, 20 Feb 2004, Jason Tishler wrote:
> > > FWIW, I just tried the above with Microsoft's rebase and observed
> > > the same behavior.  Note this is not a "ruling" just another data
> > > point.
> >
> > Well, considering what the root problem is, I wouldn't expect MS's
> > rebase to work any better than the cygwin version.
>
> Understood.  However, one of my design goals for rebase was to behave as
> similar to Microsoft's as possible.  This became especially true when I
> replaced Microsoft's ReBaseImage() with the one from Ralf Habacker's
> libimagehelper.

A worthy goal.

> > However, I would like to see some cooperation on how to resolve it.
>
> See below for one possible solution...
>
> > This problem is not specific to zsh.
>
> Understood.
>
> > It has more to do with how exported symbols are treated/managed.
>
> Or built...

Yes, yes...

> > I'd almost propose that rebase change such symbols' value, being
> > rebase is the one changing the image base of the dll in the first
> > place.
>
> Do others agree?  If so, does anyone know how hard it would be to enhance
> libimagehelper to handle this case?
>
> > However, I would really like to understand why this symbol exists and
> > if there is some way to eliminate it and use the dll's real image base
> > instead.
>
> It appears that dllwrap *may* be the culprit.  If I replace dllwrap with
> gcc -shared in your testscript, then the problem is avoided.  See
> attached patch for the details.

I was a bit curious about this.  dllwrap is just a convenient wrapper
around dlltool, which is supposed to be the tool of choice for creating
dlls, yes?  Or has 'gcc -shared' supplanted dlltool?  And, ultimately,
dlltool simply calls gcc with some different options.  You can see this
yourself by adding a '-v' option to the dllwrap command in testscript.sh.

Here's a sample gcc line output by dllwrap/dlltool to create the dll:
-------------------------------------------------------------------------
gcc testdll.exp -Wl,--dll -nostartfiles -o testdll.dll
 -Wl,-e,__cygwin_dll_entry@12 -Wl,--image-base,0x63780000 -v -g testdll.o -lc
-------------------------------------------------------------------------

And what gcc does (because of the -v):
-------------------------------------------------------------------------
Reading specs from /usr/lib/gcc-lib/i686-pc-cygwin/3.3.1/specs
Configured with: /netrel/src/gcc-3.3.1-2/configure
 --enable-languages=c,c++,f77,java --enable-libgcj --enable-threads=posix
 --with-system-zlib --enable-nls --without-included-gettext
 --enable-interpreter --enable-sjlj-exceptions
 --disable-version-specific-runtime-libs --enable-shared
 --build=i686-pc-linux --host=i686-pc-cygwin --target=i686-pc-cygwin
 --prefix=/usr --exec-prefix=/usr --sysconfdir=/etc --libdir=/usr/lib
 --includedir=/nonexistent/include --libexecdir=/usr/sbin

Thread model: posix
gcc version 3.3.1 (cygming special)
/usr/lib/gcc-lib/i686-pc-cygwin/3.3.1/collect2.exe -Bdynamic
 --dll-search-prefix=cyg -o testdll.dll
 -L/usr/lib/gcc-lib/i686-pc-cygwin/3.3.1
 -L/usr/lib/gcc-lib/i686-pc-cygwin/3.3.1/../../.. testdll.exp
 --dll -e __cygwin_dll_entry@12 --image-base 0x63780000 testdll.o -lc -lgcc
 -lcygwin -luser32 -lkernel32 -ladvapi32 -lshell32 -lgcc
-------------------------------------------------------------------------

So, what's the difference?  Apparently, only the options supplied to the
linker.  I don't really understand, completely, the differences between
'-shared' and '-Wl,--dll', but I suspect it has more to do with Windows
compatability than anything else.  '-shared' creates shared objects,
while '--dll' creates dlls (like shared objects) which supposedly can
interoperate with other compiler generated dlls (Visual C++, perhaps?).
I'm not sure that this is an advantage, give that within the cygwin
environment you can't really interoperate with Visual C++ generated dlls
anyways, so perhaps it's better to simple create cygwin only compatable
dlls instead.

> Can you use gcc -shared instead of dllwrap when you build zsh?

After a bit of experimenting and a few rebuilds using different options,
I've determined that using:
  gcc -shared -Wl,--export-all-symbols
works best.  There's apparently a subtle difference between this and
using '-Wl,-dll', which causes a conflict with rebasing.

Zsh uses global variables as well as exported functions, so using just
'gcc -shared' alone was insufficient (because the global vars weren't
being exported).  While the build process for zsh takes care to extract
out exported functions and variables, there doesn't seem to be a
convenient way to actually use that info in building the shared libraries
themselves.  One option is to explicitly mark for dllexport those globals
which are needed by other parts of zsh, but that's tedious.  I also toyed
with hacking the makefiles to use the export list in the shared lib gen,
but I didn't really want to customize the makefiles that much (it
would've been invasive surgery to some extent), so I opted to make shared
library creation simply export everything, hence the
'-Wl,--export-all-symbols' option.

I've got some cleanup to do, then I'll post a new release of zsh which can
be safely rebased. :)

As a side note, this change appears to have fixed some other hidden
problem which was causing some of the Test cases for zsh to fail or hang
randomly.  Perhaps this is an indication of just how unstable the --dll
linker option is, but that's just an observation (and probably wrong at
that :).

In anycase, zsh, for the moment, has side-stepped the issue.  That's not
to say someone else won't hit this, but at least we have a known work
around. :)

> Thanks,
> Jason

-- 
Peter A. Castro <doctor@fruitbat.org> or <Peter.Castro@oracle.com>
	"Cats are just autistic Dogs" -- Dr. Tony Attwood

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/