RFC: Add zlib source to src CVS resposity

RFC: Add zlib source to src CVS resposity

[H.J. Lu]

Hi,

Binutils can compress/decompress debug sections if zlib is available.
I imported zlib from gcc source tree to binutils source tree. I changed
binutils to use zlib unconditionally. By default, the in-tree zlib is used.
If you configure binutis using --with-system-zlib, system zlib will be used.
My zlib changes are on hjl/zlib branch at

http://git.kernel.org/?p=devel/binutils/hjl/x86.git;a=summary

Any comments?

Thanks.

-- 
H.J.

Re: RFC: Add zlib source to src CVS resposity

[John Reiser]

On 10/30/2010, H.J. Lu wrote:
> Binutils can compress/decompress debug sections if zlib is available.
> I imported zlib from gcc source tree to binutils source tree. ...

Please consider using -DNO_GZIP in zlib version 1.2.5 (April 19, 2010)
from  http://zlib.net/ .  Compiling zlib with -DNO_GZIP forces adler32
as the checksum algorithm instead of crc32, and zlib streams for the
data format instead of gzip streams.  adler32 is significantly faster,
less energy intensive, and several kilobytes smaller in static size
because it uses no CRC table. The cost is a loss of at most 0.023%
in detecting corruption.

-- 
John Reiser, jreiser@BitWagon.com

Re: RFC: Add zlib source to src CVS resposity

[Nick Clifton]

Hi H.J.

 > On 30/10/10 23:19, John Reiser wrote:
> Please consider using -DNO_GZIP in zlib version 1.2.5 (April 19, 2010)
> from  http://zlib.net/.

I think that this sounds like a good idea.  What do you think ?

Cheers
   Nick

Re: RFC: Add zlib source to src CVS resposity

[Nathan Froyd]

On Mon, Nov 01, 2010 at 04:49:15PM +0000, Nick Clifton wrote:
> > On 30/10/10 23:19, John Reiser wrote:
> >Please consider using -DNO_GZIP in zlib version 1.2.5 (April 19, 2010)
> >from  http://zlib.net/.
> 
> I think that this sounds like a good idea.  What do you think ?

Will that cause an incompatibility with existing objects?  (i.e. if the
zlib binutils was using was compiled without -DNO_GZIP)

-Nathan

Re: RFC: Add zlib source to src CVS resposity

[Mike Frysinger]

[-- Attachment #1: Type: Text/Plain, Size: 630 bytes --]

On Tuesday, November 02, 2010 09:22:24 Ian Lance Taylor wrote:
> Nick Clifton <nickc@redhat.com> writes:
> >   Right - this decision has been made.  We are not going to include
> > 
> > zlib the in the binutils sources.
> > 
> >   Thanks for suggesting the idea and working on the patch, but in the
> > 
> > end it was just not a path we wanted to go down.
> 
> I think the next decision is whether to require the host system to have
> zlib, much as gcc requires the host system to have gmp, mpfr, and mpc.

since the functionality in binutils is minor and optional, i dont think 
requiring zlib makes sense
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: RFC: Add zlib source to src CVS resposity

[Ian Lance Taylor]

Nick Clifton <nickc@redhat.com> writes:

>   Right - this decision has been made.  We are not going to include
> zlib the in the binutils sources.
>
>   Thanks for suggesting the idea and working on the patch, but in the
> end it was just not a path we wanted to go down.

I think the next decision is whether to require the host system to have
zlib, much as gcc requires the host system to have gmp, mpfr, and mpc.

Ian

Re: RFC: Add zlib source to src CVS resposity

[Paolo Bonzini]

On 10/31/2010 08:12 PM, Ian Lance Taylor wrote:
> I assume that the reason we do that for intl is because it has complex
> interactions with the rest of the C library, so using the wrong intl
> library will cause confusing behaviour when the LC_ environment
> variables are set.  That case does not arise for zlib.  I think that if
> we do ship zlib with the binutils, we might as well always build it
> rather than using complex configure tests.

I think it's also for historical reasons.  Nowadays most GNU utilities 
are not shipping with a copy intl anymore.

Paolo

Re: RFC: Add zlib source to src CVS resposity

[Nick Clifton]

Hi H.J.

   Right - this decision has been made.  We are not going to include 
zlib the in the binutils sources.

   Thanks for suggesting the idea and working on the patch, but in the 
end it was just not a path we wanted to go down.

Cheers
   Nick Clifton

Re: RFC: Add zlib source to src CVS resposity

[Christopher Faylor]

On Tue, Nov 02, 2010 at 10:25:50AM +1030, Alan Modra wrote:
>On Mon, Nov 01, 2010 at 05:13:44PM +0000, Nick Clifton wrote:
>>   * We have to make sure that zlib will build on all of the
>>     hosts that we care about.  Should the situation arise
>>     where the zlib does not build on a particular host, and
>>     the zlib maintainers are not interested in making it
>>     build there, then it will be down to us to fix it.  Or
>>     else abandon compression support on that host.
>
>This would mean we need to keep machinery to conditionally compile
>in compressed debug support, removal of said support being HJ's stated
>reason for importing zlib.
>
>I'm against importing zlib into binutils, and I think we should keep
>support of compressed debug sections conditional, to avoid potential
>bootstrap problems or circular dependencies.

FWIW, I agree.  I think that having to keep zlib up-to-date wrt
potential security issues or other serious bugs is a burden that we
shouldn't take on.  Shouldn't we be trying to use any system shared
libraries for these types of things specifically so that the vendor
can handle problems for us?

cgf

Re: RFC: Add zlib source to src CVS resposity

[Alan Modra]

On Mon, Nov 01, 2010 at 05:13:44PM +0000, Nick Clifton wrote:
>   * We have to make sure that zlib will build on all of the
>     hosts that we care about.  Should the situation arise
>     where the zlib does not build on a particular host, and
>     the zlib maintainers are not interested in making it
>     build there, then it will be down to us to fix it.  Or
>     else abandon compression support on that host.

This would mean we need to keep machinery to conditionally compile
in compressed debug support, removal of said support being HJ's stated
reason for importing zlib.

I'm against importing zlib into binutils, and I think we should keep
support of compressed debug sections conditional, to avoid potential
bootstrap problems or circular dependencies.

-- 
Alan Modra
Australia Development Lab, IBM

Re: RFC: Add zlib source to src CVS resposity

[Daniel Jacobowitz]

On Mon, Nov 01, 2010 at 05:13:44PM +0000, Nick Clifton wrote:
> At the moment I feel that the pros outweigh the cons.  What do other
> people think ?

I was asked not to include expat in GDB, which was a similar
situation.  I don't remember if this was an FSF issue; I know that the
FSF, in general, dislikes duplicated source code in packages, but I
don't know if they care when the duplicated bits are non-GNU.  There
were definitely GDB developers that disliked bundling expat, so
perhaps you can find the reasons in the archives.

-- 
Daniel Jacobowitz
CodeSourcery

Re: RFC: Add zlib source to src CVS resposity

[Nick Clifton]

Hi Guys,

 >>> So this becomes a question for the binutils maintainers: do
>>> the binutils want to be self-contained, or do they want to follow the
>>> path of gcc and require additional libraries to be installed before a
>>> build can succeed?

As I see it the pros of having a copy of the zlib sources in the 
binutils tree are that:

   * The tools can be built on a host that does not have
     zlib installed.

   * We can be sure exactly which version of zlib is being
     used.

   * It simplifies our configure scripts and sources.  We
     always know that zlib is present and the API to use.

Whereas the cons of having our own copy of zlib are that:

   * We have to manually import any bug-fixes or enhancements
     to the official zlib sources.  Which means that we have
     to watch the zlib sources and be ready to evaluate any
     changes.

   * We have to make sure that zlib will build on all of the
     hosts that we care about.  Should the situation arise
     where the zlib does not build on a particular host, and
     the zlib maintainers are not interested in making it
     build there, then it will be down to us to fix it.  Or
     else abandon compression support on that host.

   * It is essentially a waste of space on hosts that already
     have zlib installed.

At the moment I feel that the pros outweigh the cons.  What do other 
people think ?

Cheers
   Nick

RE: RFC: Add zlib source to src CVS resposity

[Weddington, Eric]

 

> -----Original Message-----
> From: Ian Lance Taylor [mailto:iant@google.com] 
> Sent: Sunday, October 31, 2010 12:42 PM
> To: Frank Ch. Eigler
> Cc: H.J. Lu; Binutils; GCC Development; GDB
> Subject: Re: RFC: Add zlib source to src CVS resposity
> 
> fche@redhat.com (Frank Ch. Eigler) writes:
> 
> > "H.J. Lu" <hjl.tools@gmail.com> writes:
> >
> >> [...]  By default, the in-tree zlib is used.  If you configure
> >> binutis using --with-system-zlib, system zlib will be used.  [...]
> >
> > Can you summarize what modern platforms lack a system zlib, and what
> > justifies using the proposed in-tree copy by default?
> 
> This is a good point.  We need zlib in the gcc repository because we
> build it for the target, but this issue does not arise in the src
> repository.  So this becomes a question for the binutils 
> maintainers: do
> the binutils want to be self-contained, or do they want to follow the
> path of gcc and require additional libraries to be installed before a
> build can succeed?

Typically, building for host=mingw runs into all sorts of non-standard stuff. I don't know for sure, but I highly doubt that there is a system zlib on that platform. For gcc dependencies, it just so happens that gmp, mpfr, and mpc build just fine for mingw.

Eric Weddington

Re: RFC: Add zlib source to src CVS resposity

[Ian Lance Taylor]

"H.J. Lu" <hjl.tools@gmail.com> writes:

> On Sun, Oct 31, 2010 at 11:42 AM, Ian Lance Taylor <iant@google.com> wrote:
>> fche@redhat.com (Frank Ch. Eigler) writes:
>>
>>> "H.J. Lu" <hjl.tools@gmail.com> writes:
>>>
>>>> [...]  By default, the in-tree zlib is used.  If you configure
>>>> binutis using --with-system-zlib, system zlib will be used.  [...]
>>>
>>> Can you summarize what modern platforms lack a system zlib, and what
>>> justifies using the proposed in-tree copy by default?
>>
>> This is a good point.  We need zlib in the gcc repository because we
>> build it for the target, but this issue does not arise in the src
>> repository.  So this becomes a question for the binutils maintainers: do
>> the binutils want to be self-contained, or do they want to follow the
>> path of gcc and require additional libraries to be installed before a
>> build can succeed?
>
> zlib is in similar situation as intl. We include intl in binutils src and
> it can be disabled at configure time. For host zlib, should we check if
> it is available and fail back to in-tree zlib if there is no suitable host
> zlib?

I assume that the reason we do that for intl is because it has complex
interactions with the rest of the C library, so using the wrong intl
library will cause confusing behaviour when the LC_ environment
variables are set.  That case does not arise for zlib.  I think that if
we do ship zlib with the binutils, we might as well always build it
rather than using complex configure tests.

This is just my opinion, and really I think the more active binutils and
gdb maintainers should decide what to do here.

Ian

Re: RFC: Add zlib source to src CVS resposity

[H.J. Lu]

On Sun, Oct 31, 2010 at 11:42 AM, Ian Lance Taylor <iant@google.com> wrote:
> fche@redhat.com (Frank Ch. Eigler) writes:
>
>> "H.J. Lu" <hjl.tools@gmail.com> writes:
>>
>>> [...]  By default, the in-tree zlib is used.  If you configure
>>> binutis using --with-system-zlib, system zlib will be used.  [...]
>>
>> Can you summarize what modern platforms lack a system zlib, and what
>> justifies using the proposed in-tree copy by default?
>
> This is a good point.  We need zlib in the gcc repository because we
> build it for the target, but this issue does not arise in the src
> repository.  So this becomes a question for the binutils maintainers: do
> the binutils want to be self-contained, or do they want to follow the
> path of gcc and require additional libraries to be installed before a
> build can succeed?

zlib is in similar situation as intl. We include intl in binutils src and
it can be disabled at configure time. For host zlib, should we check if
it is available and fail back to in-tree zlib if there is no suitable host
zlib?


-- 
H.J.

Re: RFC: Add zlib source to src CVS resposity

[Ian Lance Taylor]

fche@redhat.com (Frank Ch. Eigler) writes:

> "H.J. Lu" <hjl.tools@gmail.com> writes:
>
>> [...]  By default, the in-tree zlib is used.  If you configure
>> binutis using --with-system-zlib, system zlib will be used.  [...]
>
> Can you summarize what modern platforms lack a system zlib, and what
> justifies using the proposed in-tree copy by default?

This is a good point.  We need zlib in the gcc repository because we
build it for the target, but this issue does not arise in the src
repository.  So this becomes a question for the binutils maintainers: do
the binutils want to be self-contained, or do they want to follow the
path of gcc and require additional libraries to be installed before a
build can succeed?

Ian

Re: RFC: Add zlib source to src CVS resposity

[H.J. Lu]

On Sat, Oct 30, 2010 at 2:37 PM, Frank Ch. Eigler <fche@redhat.com> wrote:
> "H.J. Lu" <hjl.tools@gmail.com> writes:
>
>> [...]  By default, the in-tree zlib is used.  If you configure
>> binutis using --with-system-zlib, system zlib will be used.  [...]
>
> Can you summarize what modern platforms lack a system zlib, and what
> justifies using the proposed in-tree copy by default?
>

I don't know. I just want to remove those zlib checks.


-- 
H.J.

Re: RFC: Add zlib source to src CVS resposity

[Frank Ch. Eigler]

"H.J. Lu" <hjl.tools@gmail.com> writes:

> [...]  By default, the in-tree zlib is used.  If you configure
> binutis using --with-system-zlib, system zlib will be used.  [...]

Can you summarize what modern platforms lack a system zlib, and what
justifies using the proposed in-tree copy by default?

- FChE