Re: make the shared library optional

Re: make the shared library optional

[Mark Wielaard]

[-- Attachment #1: Type: text/plain, Size: 3041 bytes --]

On Fri, Nov 07, 2014 at 03:47:32PM +0000, Vicente Olivert Riera wrote:
> > It looks like the issue is with the crtbeginT.o code, not with any of
> > the elfutils objects? I don't immediately know why that particular
> > object is linked into the shared library. Does the toolchain pick up the
> > correct version?
> 
> I'm sorry, the correct version of what? Also, that's an external
> toolchain and I don't have much information about it :S

I guess the issue is just that the setup tries to do a static build,
but doesn't patch the elfutils build enough, so some parts still don't
get build staticly and when linking against the static crtbeginT.o
things fail. As Petr explained building elfutils fully static is not
really supported and might need some ugly hacks.

> > It looks like that is a fairly old 0.155 build with several patches
> > applied. Have you tried latest elfutils 0.160 or current git?
> 
> I have tried elfutils-0.160, but I'm having this error:
> 
> /br/output/host/usr/bin/mipsel-linux-gcc -D_GNU_SOURCE -DHAVE_CONFIG_H
> -DLOCALEDIR='"/usr/share/locale"' -I. -I..  -I. -I. -I../lib -I..
> -I./../libelf -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -std=gnu99 -Wall
> -Wshadow -Werror -Wunused -Wextra -fgnu89-inline -Wformat=2   -fpic
> -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -pipe -Os -g2 -static -c -o
> color.o color.c
> color.c: In function ‘parse_opt’:
> color.c:135:5: error: passing argument 4 of ‘argp_help’ discards ‘const’
> qualifier from pointer target type [-Werror]
>      program_invocation_short_name);
>      ^
> In file included from color.c:34:0:
> /home/ldap/vriera/work/mips-buildroots/mips32/output/host/usr/mipsel-buildroot-linux-uclibc/sysroot/usr/include/argp.h:469:13:
> note: expected ‘char *’ but argument is of type ‘const char *’
>  extern void argp_help (__const struct argp *__restrict __argp,
>              ^
> cc1: all warnings being treated as errors
> make[3]: *** [color.o] Error 1

That is strange. argp_help does indeed take a non-const char *.
But program_invocation_short_name is defined as non-const char * too
in errno.h. You might be using a non-GNU glibc? Some libcs are slightly
broken.

> > If you think any of those patches are useful could you submit them so we
> > can review them for inclusion upstream?
> 
> Please have a look to those patches here. I think some of them can be
> useful:
> 
> http://git.buildroot.net/buildroot/tree/package/elfutils

Those seem hacks to work around bugs/missing features in uClibc.
Normally elfutils relies on glibc features whenever possible.
I am not against supporting other libc implementations, but it shouldn't
create ugly hacks. The memcpy one is a workaround for something odd in our
code.  I'll propose a simpler cleanup for that. But for the others we
really need someone that understands the patches to propose and explain
them on the list so we can discuss and see whether or not they make sense
to apply to mainline.

Thanks,

Mark

Re: make the shared library optional

[Mark Wielaard]

[-- Attachment #1: Type: text/plain, Size: 2004 bytes --]

On Mon, 2014-11-10 at 16:49 +0000, Vicente Olivert Riera wrote:
> On 11/08/2014 03:14 PM, Mark Wielaard wrote:
> >>> It looks like that is a fairly old 0.155 build with several patches
> >>> applied. Have you tried latest elfutils 0.160 or current git?
> >>
> >> I have tried elfutils-0.160, but I'm having this error:
> >>
> >> /br/output/host/usr/bin/mipsel-linux-gcc -D_GNU_SOURCE -DHAVE_CONFIG_H
> >> -DLOCALEDIR='"/usr/share/locale"' -I. -I..  -I. -I. -I../lib -I..
> >> -I./../libelf -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -std=gnu99 -Wall
> >> -Wshadow -Werror -Wunused -Wextra -fgnu89-inline -Wformat=2   -fpic
> >> -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -pipe -Os -g2 -static -c -o
> >> color.o color.c
> >> color.c: In function ‘parse_opt’:
> >> color.c:135:5: error: passing argument 4 of ‘argp_help’ discards ‘const’
> >> qualifier from pointer target type [-Werror]
> >>      program_invocation_short_name);
> >>      ^
> >> In file included from color.c:34:0:
> >> /home/ldap/vriera/work/mips-buildroots/mips32/output/host/usr/mipsel-buildroot-linux-uclibc/sysroot/usr/include/argp.h:469:13:
> >> note: expected ‘char *’ but argument is of type ‘const char *’
> >>  extern void argp_help (__const struct argp *__restrict __argp,
> >>              ^
> >> cc1: all warnings being treated as errors
> >> make[3]: *** [color.o] Error 1
> > 
> > That is strange. argp_help does indeed take a non-const char *.
> > But program_invocation_short_name is defined as non-const char * too
> > in errno.h. You might be using a non-GNU glibc? Some libcs are slightly
> > broken.
> 
> It's uClibc.

Yeah, that seems to be the problem.
glibc errno.h defines:
extern char *program_invocation_name, *program_invocation_short_name;
While uClibc has:
extern const char *program_invocation_name,
*program_invocation_short_name;

If you fix uClibc errno.h to drop the const in these definitions the
code will compile as is.

Thanks,

Mark

Re: make the shared library optional

[Vicente Olivert Riera]

[-- Attachment #1: Type: text/plain, Size: 3607 bytes --]

On 11/08/2014 03:14 PM, Mark Wielaard wrote:
> On Fri, Nov 07, 2014 at 03:47:32PM +0000, Vicente Olivert Riera wrote:
>>> It looks like the issue is with the crtbeginT.o code, not with any of
>>> the elfutils objects? I don't immediately know why that particular
>>> object is linked into the shared library. Does the toolchain pick up the
>>> correct version?
>>
>> I'm sorry, the correct version of what? Also, that's an external
>> toolchain and I don't have much information about it :S
> 
> I guess the issue is just that the setup tries to do a static build,
> but doesn't patch the elfutils build enough, so some parts still don't
> get build staticly and when linking against the static crtbeginT.o
> things fail. As Petr explained building elfutils fully static is not
> really supported and might need some ugly hacks.

That's fine, we already disabled elfutils for static builds in Buildroot:

http://git.buildroot.net/buildroot/commit/?id=9406d4dc02b5fa452ff487b1c5ce9a891388b919

>>> It looks like that is a fairly old 0.155 build with several patches
>>> applied. Have you tried latest elfutils 0.160 or current git?
>>
>> I have tried elfutils-0.160, but I'm having this error:
>>
>> /br/output/host/usr/bin/mipsel-linux-gcc -D_GNU_SOURCE -DHAVE_CONFIG_H
>> -DLOCALEDIR='"/usr/share/locale"' -I. -I..  -I. -I. -I../lib -I..
>> -I./../libelf -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -std=gnu99 -Wall
>> -Wshadow -Werror -Wunused -Wextra -fgnu89-inline -Wformat=2   -fpic
>> -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -pipe -Os -g2 -static -c -o
>> color.o color.c
>> color.c: In function ‘parse_opt’:
>> color.c:135:5: error: passing argument 4 of ‘argp_help’ discards ‘const’
>> qualifier from pointer target type [-Werror]
>>      program_invocation_short_name);
>>      ^
>> In file included from color.c:34:0:
>> /home/ldap/vriera/work/mips-buildroots/mips32/output/host/usr/mipsel-buildroot-linux-uclibc/sysroot/usr/include/argp.h:469:13:
>> note: expected ‘char *’ but argument is of type ‘const char *’
>>  extern void argp_help (__const struct argp *__restrict __argp,
>>              ^
>> cc1: all warnings being treated as errors
>> make[3]: *** [color.o] Error 1
> 
> That is strange. argp_help does indeed take a non-const char *.
> But program_invocation_short_name is defined as non-const char * too
> in errno.h. You might be using a non-GNU glibc? Some libcs are slightly
> broken.

It's uClibc.

>>> If you think any of those patches are useful could you submit them so we
>>> can review them for inclusion upstream?
>>
>> Please have a look to those patches here. I think some of them can be
>> useful:
>>
>> http://git.buildroot.net/buildroot/tree/package/elfutils
> 
> Those seem hacks to work around bugs/missing features in uClibc.
> Normally elfutils relies on glibc features whenever possible.
> I am not against supporting other libc implementations, but it shouldn't
> create ugly hacks. The memcpy one is a workaround for something odd in our
> code.  I'll propose a simpler cleanup for that. But for the others we
> really need someone that understands the patches to propose and explain
> them on the list so we can discuss and see whether or not they make sense
> to apply to mainline.

I'm sure that if you contact with the guy who signed-of-by those patches
(Thomas Petazzoni) he will be happy to help :-)

Best regards,
-- 
Vicente Olivert Riera
Graduate Software Engineer, MIPS Processor IP
Imagination Technologies Limited
t: +44 (0)113 2429814
www.imgtec.com

Re: make the shared library optional

[Vicente Olivert Riera]

[-- Attachment #1: Type: text/plain, Size: 4346 bytes --]

Dear Mark Wielaard,

On 11/07/2014 03:47 PM, Vicente Olivert Riera wrote:
> Dear Mark Wielaard,
> 
> On 11/07/2014 02:16 PM, Mark Wielaard wrote:
>> On Mon, 2014-11-03 at 14:45 +0000, Vicente Olivert Riera wrote:
>>> I'm having a failure when doing a static build of elfutils because it
>>> tries to build a shared library (.so). Could it be possible to add a
>>> configure option to disable the shared library so only the static one
>>> (.a) would be built?
>>>
>>> This is the failure I'm talking about:
>>>
>>> /home/chroot/media/code/buildroot/autobuilder/instance-0/output/host/opt/ext-toolchain/bin/../lib/gcc/mipsel-buildroot-linux-uclibc/4.8.3/../../../../mipsel-buildroot-linux-uclibc/bin/ld:
>>> /home/chroot/media/code/buildroot/autobuilder/instance-0/output/host/opt/ext-toolchain/bin/../lib/gcc/mipsel-buildroot-linux-uclibc/4.8.3/crtbeginT.o:
>>> relocation R_MIPS_HI16 against `a local symbol' can not be used when
>>> making a shared object; recompile with -fPIC
>>> /home/chroot/media/code/buildroot/autobuilder/instance-0/output/host/opt/ext-toolchain/bin/../lib/gcc/mipsel-buildroot-linux-uclibc/4.8.3/crtbeginT.o:
>>> could not read symbols: Bad value
>>> collect2: error: ld returned 1 exit status
>>> Makefile:939: recipe for target 'libelf.so' failed
>>> make[3]: *** [libelf.so] Error 1
> 
> 
> thanks a lot for your reply.
> 
> 
>> Does that failure also happen with a normal (not patched) build?
> 
> Those patches are necessary. Otherwise it will fail at many points. You
> can see the comments on the patches:
> 
> http://git.buildroot.net/buildroot/tree/package/elfutils
> 
>> It looks like the issue is with the crtbeginT.o code, not with any of
>> the elfutils objects? I don't immediately know why that particular
>> object is linked into the shared library. Does the toolchain pick up the
>> correct version?
> 
> I'm sorry, the correct version of what? Also, that's an external
> toolchain and I don't have much information about it :S
> 
>>> The full build log is here:
>>>
>>> http://autobuild.buildroot.net/results/16f/16f6956d5a215fe678a45cc4b24ba196309ee05c/build-end.log
>>
>> It looks like that is a fairly old 0.155 build with several patches
>> applied. Have you tried latest elfutils 0.160 or current git?
> 
> I have tried elfutils-0.160, but I'm having this error:
> 
> /br/output/host/usr/bin/mipsel-linux-gcc -D_GNU_SOURCE -DHAVE_CONFIG_H
> -DLOCALEDIR='"/usr/share/locale"' -I. -I..  -I. -I. -I../lib -I..
> -I./../libelf -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -std=gnu99 -Wall
> -Wshadow -Werror -Wunused -Wextra -fgnu89-inline -Wformat=2   -fpic
> -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -pipe -Os -g2 -static -c -o
> color.o color.c
> color.c: In function ‘parse_opt’:
> color.c:135:5: error: passing argument 4 of ‘argp_help’ discards ‘const’
> qualifier from pointer target type [-Werror]
>      program_invocation_short_name);
>      ^
> In file included from color.c:34:0:
> /home/ldap/vriera/work/mips-buildroots/mips32/output/host/usr/mipsel-buildroot-linux-uclibc/sysroot/usr/include/argp.h:469:13:
> note: expected ‘char *’ but argument is of type ‘const char *’
>  extern void argp_help (__const struct argp *__restrict __argp,
>              ^
> cc1: all warnings being treated as errors
> make[3]: *** [color.o] Error 1

After removing all -Werror options (passing --disable-werror to the
configure script doesn't work or is not enough) I got rid of that error
and now I see the static problem again (version 0.160):

mipsel-buildroot-linux-uclibc/4.8.3/crtbeginT.o: relocation R_MIPS_HI16
against `a local symbol' can not be used when making a shared object;
recompile with -fPIC
/br/output/host/opt/ext-toolchain/bin/../lib/gcc/mipsel-buildroot-linux-uclibc/4.8.3/crtbeginT.o:
could not read symbols: Bad value

>> If you think any of those patches are useful could you submit them so we
>> can review them for inclusion upstream?
> 
> Please have a look to those patches here. I think some of them can be
> useful:
> 
> http://git.buildroot.net/buildroot/tree/package/elfutils
> 
> Best regards,
> 

Best regards,
-- 
Vicente Olivert Riera
Graduate Software Engineer, MIPS Processor IP
Imagination Technologies Limited
t: +44 (0)113 2429814
www.imgtec.com

Re: make the shared library optional

[Vicente Olivert Riera]

[-- Attachment #1: Type: text/plain, Size: 3624 bytes --]

Dear Mark Wielaard,

On 11/07/2014 02:16 PM, Mark Wielaard wrote:
> On Mon, 2014-11-03 at 14:45 +0000, Vicente Olivert Riera wrote:
>> I'm having a failure when doing a static build of elfutils because it
>> tries to build a shared library (.so). Could it be possible to add a
>> configure option to disable the shared library so only the static one
>> (.a) would be built?
>>
>> This is the failure I'm talking about:
>>
>> /home/chroot/media/code/buildroot/autobuilder/instance-0/output/host/opt/ext-toolchain/bin/../lib/gcc/mipsel-buildroot-linux-uclibc/4.8.3/../../../../mipsel-buildroot-linux-uclibc/bin/ld:
>> /home/chroot/media/code/buildroot/autobuilder/instance-0/output/host/opt/ext-toolchain/bin/../lib/gcc/mipsel-buildroot-linux-uclibc/4.8.3/crtbeginT.o:
>> relocation R_MIPS_HI16 against `a local symbol' can not be used when
>> making a shared object; recompile with -fPIC
>> /home/chroot/media/code/buildroot/autobuilder/instance-0/output/host/opt/ext-toolchain/bin/../lib/gcc/mipsel-buildroot-linux-uclibc/4.8.3/crtbeginT.o:
>> could not read symbols: Bad value
>> collect2: error: ld returned 1 exit status
>> Makefile:939: recipe for target 'libelf.so' failed
>> make[3]: *** [libelf.so] Error 1


thanks a lot for your reply.


> Does that failure also happen with a normal (not patched) build?

Those patches are necessary. Otherwise it will fail at many points. You
can see the comments on the patches:

http://git.buildroot.net/buildroot/tree/package/elfutils

> It looks like the issue is with the crtbeginT.o code, not with any of
> the elfutils objects? I don't immediately know why that particular
> object is linked into the shared library. Does the toolchain pick up the
> correct version?

I'm sorry, the correct version of what? Also, that's an external
toolchain and I don't have much information about it :S

>> The full build log is here:
>>
>> http://autobuild.buildroot.net/results/16f/16f6956d5a215fe678a45cc4b24ba196309ee05c/build-end.log
> 
> It looks like that is a fairly old 0.155 build with several patches
> applied. Have you tried latest elfutils 0.160 or current git?

I have tried elfutils-0.160, but I'm having this error:

/br/output/host/usr/bin/mipsel-linux-gcc -D_GNU_SOURCE -DHAVE_CONFIG_H
-DLOCALEDIR='"/usr/share/locale"' -I. -I..  -I. -I. -I../lib -I..
-I./../libelf -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -std=gnu99 -Wall
-Wshadow -Werror -Wunused -Wextra -fgnu89-inline -Wformat=2   -fpic
-D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -pipe -Os -g2 -static -c -o
color.o color.c
color.c: In function ‘parse_opt’:
color.c:135:5: error: passing argument 4 of ‘argp_help’ discards ‘const’
qualifier from pointer target type [-Werror]
     program_invocation_short_name);
     ^
In file included from color.c:34:0:
/home/ldap/vriera/work/mips-buildroots/mips32/output/host/usr/mipsel-buildroot-linux-uclibc/sysroot/usr/include/argp.h:469:13:
note: expected ‘char *’ but argument is of type ‘const char *’
 extern void argp_help (__const struct argp *__restrict __argp,
             ^
cc1: all warnings being treated as errors
make[3]: *** [color.o] Error 1

> If you think any of those patches are useful could you submit them so we
> can review them for inclusion upstream?

Please have a look to those patches here. I think some of them can be
useful:

http://git.buildroot.net/buildroot/tree/package/elfutils

Best regards,
-- 
Vicente Olivert Riera
Graduate Software Engineer, MIPS Processor IP
Imagination Technologies Limited
t: +44 (0)113 2429814
www.imgtec.com

Re: make the shared library optional

[Mark Wielaard]

[-- Attachment #1: Type: text/plain, Size: 1856 bytes --]

On Mon, 2014-11-03 at 14:45 +0000, Vicente Olivert Riera wrote:
> I'm having a failure when doing a static build of elfutils because it
> tries to build a shared library (.so). Could it be possible to add a
> configure option to disable the shared library so only the static one
> (.a) would be built?
> 
> This is the failure I'm talking about:
> 
> /home/chroot/media/code/buildroot/autobuilder/instance-0/output/host/opt/ext-toolchain/bin/../lib/gcc/mipsel-buildroot-linux-uclibc/4.8.3/../../../../mipsel-buildroot-linux-uclibc/bin/ld:
> /home/chroot/media/code/buildroot/autobuilder/instance-0/output/host/opt/ext-toolchain/bin/../lib/gcc/mipsel-buildroot-linux-uclibc/4.8.3/crtbeginT.o:
> relocation R_MIPS_HI16 against `a local symbol' can not be used when
> making a shared object; recompile with -fPIC
> /home/chroot/media/code/buildroot/autobuilder/instance-0/output/host/opt/ext-toolchain/bin/../lib/gcc/mipsel-buildroot-linux-uclibc/4.8.3/crtbeginT.o:
> could not read symbols: Bad value
> collect2: error: ld returned 1 exit status
> Makefile:939: recipe for target 'libelf.so' failed
> make[3]: *** [libelf.so] Error 1

Does that failure also happen with a normal (not patched) build?

It looks like the issue is with the crtbeginT.o code, not with any of
the elfutils objects? I don't immediately know why that particular
object is linked into the shared library. Does the toolchain pick up the
correct version?

> The full build log is here:
> 
> http://autobuild.buildroot.net/results/16f/16f6956d5a215fe678a45cc4b24ba196309ee05c/build-end.log

It looks like that is a fairly old 0.155 build with several patches
applied. Have you tried latest elfutils 0.160 or current git?
If you think any of those patches are useful could you submit them so we
can review them for inclusion upstream?

Thanks,

Mark

Re: make the shared library optional

[Vicente Olivert Riera]

[-- Attachment #1: Type: text/plain, Size: 1736 bytes --]

Dear Petr Machata,

On 11/04/2014 05:01 PM, Petr Machata wrote:
> Vicente Olivert Riera <Vincent.Riera@imgtec.com> writes:
> 
>> I'm having a failure when doing a static build of elfutils because it
>> tries to build a shared library (.so). Could it be possible to add a
>> configure option to disable the shared library so only the static one
>> (.a) would be built?
> 
> Note that fully static builds are problematic.  elfutils uses dlopen to
> open the EBL backends (the CPU-specific support snippets), so even if
> you link statically, the final binaries are still considerably dynamic.
> 
> One option (that we use in Red Hat Developer Toolset) is to statically
> link in just the backends that you are interested in.  There were some
> problems with mutual dependencies between elfutils libraries, and I
> ended up having to split the libraries into smaller parts and adding
> linker scripts that brought the pieces together.
> 
> Mark and I have branches with this code--his is more recent, as he's
> been doing that work lately.  They are named {pmachata,mjw}/RH-DTS.
> 
> (Note that we still build the dynamic parts though, as eu-readelf
> et.al. can still use the dynamism, it's just customer code that needs
> the static bits.  I'm not really sure what your bug is, just dropping my
> two cents WRT lack of real static support in elfutils.)

thank you very much for your reply and your detailed explanation. I have
decided to send a patch to not allow elfutils to be built statically:

http://patchwork.ozlabs.org/patch/406941/

Best regards,
-- 
Vicente Olivert Riera
Graduate Software Engineer, MIPS Processor IP
Imagination Technologies Limited
t: +44 (0)113 2429814
www.imgtec.com

Re: make the shared library optional

[Petr Machata]

[-- Attachment #1: Type: text/plain, Size: 1285 bytes --]

Vicente Olivert Riera <Vincent.Riera@imgtec.com> writes:

> I'm having a failure when doing a static build of elfutils because it
> tries to build a shared library (.so). Could it be possible to add a
> configure option to disable the shared library so only the static one
> (.a) would be built?

Note that fully static builds are problematic.  elfutils uses dlopen to
open the EBL backends (the CPU-specific support snippets), so even if
you link statically, the final binaries are still considerably dynamic.

One option (that we use in Red Hat Developer Toolset) is to statically
link in just the backends that you are interested in.  There were some
problems with mutual dependencies between elfutils libraries, and I
ended up having to split the libraries into smaller parts and adding
linker scripts that brought the pieces together.

Mark and I have branches with this code--his is more recent, as he's
been doing that work lately.  They are named {pmachata,mjw}/RH-DTS.

(Note that we still build the dynamic parts though, as eu-readelf
et.al. can still use the dynamism, it's just customer code that needs
the static bits.  I'm not really sure what your bug is, just dropping my
two cents WRT lack of real static support in elfutils.)

Thanks,
Petr

make the shared library optional

[Vicente Olivert Riera]

[-- Attachment #1: Type: text/plain, Size: 1361 bytes --]

Hello,

I'm having a failure when doing a static build of elfutils because it
tries to build a shared library (.so). Could it be possible to add a
configure option to disable the shared library so only the static one
(.a) would be built?

This is the failure I'm talking about:

/home/chroot/media/code/buildroot/autobuilder/instance-0/output/host/opt/ext-toolchain/bin/../lib/gcc/mipsel-buildroot-linux-uclibc/4.8.3/../../../../mipsel-buildroot-linux-uclibc/bin/ld:
/home/chroot/media/code/buildroot/autobuilder/instance-0/output/host/opt/ext-toolchain/bin/../lib/gcc/mipsel-buildroot-linux-uclibc/4.8.3/crtbeginT.o:
relocation R_MIPS_HI16 against `a local symbol' can not be used when
making a shared object; recompile with -fPIC
/home/chroot/media/code/buildroot/autobuilder/instance-0/output/host/opt/ext-toolchain/bin/../lib/gcc/mipsel-buildroot-linux-uclibc/4.8.3/crtbeginT.o:
could not read symbols: Bad value
collect2: error: ld returned 1 exit status
Makefile:939: recipe for target 'libelf.so' failed
make[3]: *** [libelf.so] Error 1

The full build log is here:


http://autobuild.buildroot.net/results/16f/16f6956d5a215fe678a45cc4b24ba196309ee05c/build-end.log

Best regards,
-- 
Vicente Olivert Riera
Graduate Software Engineer, MIPS Processor IP
Imagination Technologies Limited
t: +44 (0)113 2429814
www.imgtec.com