Re: cygwin Digest 25 Sep 2005 20:51:19 -0000 Issue 4542

Re: cygwin Digest 25 Sep 2005 20:51:19 -0000 Issue 4542

[Fred Smith]

> ---------- Forwarded message ----------
> From: Jason Tishler <jason@tishler.net>
> To: cygwin@cygwin.com
> Date: Sat, 24 Sep 2005 16:51:11 -0400
> Subject: Re: rebase: rebaseall growing pains
> Yaakov,
>
> On Fri, Sep 23, 2005 at 01:34:20PM -0500, Yaakov S wrote:
> > Rebaseall is subject to "growing pains".  In addition to the 500-some
> > packages in the distro, I have my Cygwin Ports repository installed as
> > well, which makes for a total of almost 1300 binary packages.
>
> Actually, the above brings up an interesting question.  Did you run out
> of rebase address space?  Please run rebaseall as follows:
>
>     $ rebaseall -v | tail -1
>
> and report back to the list.
>
> Thanks,
> Jason
>

Which leads me to the dumb question (patience would be appreciated :)
What is rebase? what does it do? under what circumstances would I
want/need to do it?

Gracias!

Fred

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: cygwin Digest 25 Sep 2005 20:51:19 -0000 Issue 4542

[Brian Dessent]

Fred Smith wrote:

> Which leads me to the dumb question (patience would be appreciated :)
> What is rebase? what does it do? under what circumstances would I
> want/need to do it?

When created, every DLL has a base address which is the location in the
virtual memory space that it was intended to be loaded into.  The DLL
can be loaded elsewhere, but doing so requires modifying any stored
addresses that are not relative.  This relocation takes a bit of time at
startup, and it also decreases the overall efficiency of the shared
library system.  Normally all processes that use a particular DLL can
share one copy of it in physical memory, but if it was relocated a copy
must be made.

To help alleviate this, the MS tools have a system whereby the image
base for a DLL is assigned based on some kind of hash signature of the
DLL. (I believe they also have a kind of internal central registry of
image bases for the core system DLLs, but I'm not sure about that since
the load addresses of even core DLLs varies by OS version and even
serive pack version.)

Gcc/binutils also has a similar mechanism, but it is not turned on by
default at present.  This means that DLLs created by Cygwin gcc all have
an image base of 0x10000000.  There is nothing wrong with this, as the
DLLs can be relocated as needed.  However, the system decides where to
relocate these DLLs, and this apparently is not always a deterministic
operation -- especially if DLLs are dynamically loaded by the program
with dlopen / LoadLibrary.  The Cygwin fork emulation code relies on
being able to recreate the same[*] memory layout in the child as the
parent, and if DLLs load into different locations in the child you will
get a fatal error from Cygwin, normally something along the lines of
"Unable to remap...."

The rebase tool can change the base address of a DLL on disk, so that
the relocation is permanent.  When you run rebaseall, it generates a
list of all the Cygwin DLLs, and then assigns each of them a unique
location that does not overlap with any of the other DLLs in the list
(based on their sizes.)  This is analogous to defragmenting a hard
drive.  It causes the memory layout to be deterministic as there is no
need for relocation, and fixes the problems with fork errors.

At the end of the day it would be cleaner to enable the automatic image
base assignment in ld, but I believe there were some problems associated
with this -- they may be fixed now.  But until then, rebaseall is fairly
simple and straightforward.

As to when to use it, just run it any time you get an "unable to map" or
"MapViewOfFileEx failed" or similar fatal error.  If you don't see any
fatal errors you don't need to worry.

[*] Cygwin can do fixups on its own data structures so that they can be
relocated, so the layouts aren't always exactly the same.

Brian

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/