[Bug target/56533] New: Linker problem on avr with lto and main function inside archive

[Bug target/56533] New: Linker problem on avr with lto and main function inside archive

[matthijs at stdin dot nl]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56533

             Bug #: 56533
           Summary: Linker problem on avr with lto and main function
                    inside archive
    Classification: Unclassified
           Product: gcc
           Version: 4.7.2
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: target
        AssignedTo: unassigned@gcc.gnu.org
        ReportedBy: matthijs@stdin.nl


When trying to add lto to my Arduino program, it stopped compiling complaining
about missing symbols. I've managed to reduce the problem to below minimal
example. Note that removing anything from below example makes the problem
disappear. In particular, the problem disappears when:
 * any of the linker options is removed: -mmcu=atmega328p -Os -flto
-fwhole-program
 * the -flto compiler option is removed
 * using normal gcc (amd64) instead of avr-gcc
 * linking main.o instead of main.a
 * declaring realmain as externally_visible in realmain.c

Note that in this example, the actual main() function is inside an archive,
which is probably the reason for this bug / problem.



$ avr-gcc --version
    avr-gcc (GCC) 4.7.2
    Copyright (C) 2012 Free Software Foundation, Inc.
    This is free software; see the source for copying conditions.  There is NO
    warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

$ avr-ld --version
    GNU ld (GNU Binutils) 2.20.1.20100303
    Copyright 2009 Free Software Foundation, Inc.
    This program is free software; you may redistribute it under the terms of
    the GNU General Public License version 3 or (at your option) a later
version.
    This program has absolutely no warranty.

$ cat main.c
    int realmain(void);

    int main(void)
    {
        return realmain();
    }

$ cat realmain.c
    int realmain(void) {
    }

$ cat do
    #!/bin/sh
    set -x

    rm -f main.a

    /usr/bin/avr-gcc -c main.c -o main.o
    /usr/bin/avr-ar rcs main.a  main.o
    /usr/bin/avr-gcc -c -flto realmain.c -o realmain.o
    /usr/bin/avr-gcc -mmcu=atmega328p -Os -flto -fwhole-program realmain.o
main.a

$ ./do
    + rm -f main.a
    + /usr/bin/avr-gcc -c main.c -o main.o
    + /usr/bin/avr-ar rcs main.a main.o
    + /usr/bin/avr-gcc -c -flto realmain.c -o realmain.o
    + /usr/bin/avr-gcc -mmcu=atmega328p -Os -flto -fwhole-program realmain.o
main.a
    main.a(main.o): In function `main':
    main.c:(.text+0x8): undefined reference to `realmain'
    collect2: error: ld returned 1 exit status

[Bug target/56533] Linker problem on avr with lto and main function inside archive

[rguenth at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56533

--- Comment #1 from Richard Biener <rguenth at gcc dot gnu.org> 2013-03-05 12:38:51 UTC ---
    + /usr/bin/avr-gcc -mmcu=atmega328p -Os -flto -fwhole-program realmain.o
main.a

please provide the console output produced when adding -v to the above
commandline.  I suspect that you fall foul of using -fwhole-program without
making the whole program visible to gcc via -flto by means of not using
a compiler that can handle LTO of static archives.

In general, don't use -fwhole-program with -flto, -flto is smart enough
to discover things more intelligently.

[Bug target/56533] Linker problem on avr with lto and main function inside archive

[matthijs at stdin dot nl]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56533

--- Comment #2 from Matthijs Kooijman <matthijs at stdin dot nl> 2013-03-05 12:55:47 UTC ---
+ /usr/bin/avr-gcc -v -mmcu=atmega328p -Os -flto -fwhole-program realmain.o
main.a
Using built-in specs.
COLLECT_GCC=/usr/bin/avr-gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/avr/4.7.2/lto-wrapper
Target: avr
Configured with: ../src/configure -v --enable-languages=c,c++ --prefix=/usr/lib
--infodir=/usr/share/info --mandir=/usr/share/man --bindir=/usr/bin
--libexecdir=/usr/lib --libdir=/usr/lib --enable-shared --with-system-zlib
--enable-long-long --enable-nls --without-included-gettext --disable-libssp
--build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=avr
Thread model: single
gcc version 4.7.2 (GCC) 
COMPILER_PATH=/usr/lib/gcc/avr/4.7.2/:/usr/lib/gcc/avr/4.7.2/:/usr/lib/gcc/avr/:/usr/lib/gcc/avr/4.7.2/:/usr/lib/gcc/avr/:/usr/lib/gcc/avr/4.7.2/../../../avr/bin/
LIBRARY_PATH=/usr/lib/gcc/avr/4.7.2/avr5/:/usr/lib/gcc/avr/4.7.2/../../../avr/lib/avr5/:/usr/lib/gcc/avr/4.7.2/:/usr/lib/gcc/avr/4.7.2/../../../avr/lib/
COLLECT_GCC_OPTIONS='-v' '-mmcu=atmega328p' '-Os' '-flto' '-fwhole-program'
 /usr/lib/gcc/avr/4.7.2/collect2 -flto -m avr5 -Tdata 0x800100
/usr/lib/gcc/avr/4.7.2/../../../avr/lib/avr5/crtm328p.o
-L/usr/lib/gcc/avr/4.7.2/avr5 -L/usr/lib/gcc/avr/4.7.2/../../../avr/lib/avr5
-L/usr/lib/gcc/avr/4.7.2 -L/usr/lib/gcc/avr/4.7.2/../../../avr/lib realmain.o
main.a -lgcc -lc -lgcc
 /usr/bin/avr-gcc @/tmp/ccYrSTvi.args
Using built-in specs.
COLLECT_GCC=/usr/bin/avr-gcc
Target: avr
Configured with: ../src/configure -v --enable-languages=c,c++ --prefix=/usr/lib
--infodir=/usr/share/info --mandir=/usr/share/man --bindir=/usr/bin
--libexecdir=/usr/lib --libdir=/usr/lib --enable-shared --with-system-zlib
--enable-long-long --enable-nls --without-included-gettext --disable-libssp
--build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=avr
Thread model: single
gcc version 4.7.2 (GCC) 
COLLECT_GCC_OPTIONS='-c' '-v' '-mmcu=atmega328p' '-Os' '-fwhole-program'
'-fltrans-output-list=/tmp/ccZEu3t3.ltrans.out' '-fwpa'
 /usr/lib/gcc/avr/4.7.2/lto1 -quiet -dumpbase realmain.o -mmcu=atmega328p
-auxbase realmain -Os -version -fwhole-program
-fltrans-output-list=/tmp/ccZEu3t3.ltrans.out -fwpa @/tmp/ccOsOe32
GNU GIMPLE (GCC) version 4.7.2 (avr)
        compiled by GNU C version 4.7.2, GMP version 5.0.5, MPFR version
3.1.0-p10, MPC version 0.9
GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
GNU GIMPLE (GCC) version 4.7.2 (avr)
        compiled by GNU C version 4.7.2, GMP version 5.0.5, MPFR version
3.1.0-p10, MPC version 0.9
GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
COMPILER_PATH=/usr/lib/gcc/avr/4.7.2/:/usr/lib/gcc/avr/4.7.2/:/usr/lib/gcc/avr/:/usr/lib/gcc/avr/4.7.2/:/usr/lib/gcc/avr/:/usr/lib/gcc/avr/4.7.2/../../../avr/bin/:/usr/lib/gcc/avr/4.7.2/:/usr/lib/gcc/avr/4.7.2/:/usr/lib/gcc/avr/:/usr/lib/gcc/avr/4.7.2/:/usr/lib/gcc/avr/:/usr/lib/gcc/avr/4.7.2/../../../avr/bin/
LIBRARY_PATH=/usr/lib/gcc/avr/4.7.2/avr5/:/usr/lib/gcc/avr/4.7.2/../../../avr/lib/avr5/:/usr/lib/gcc/avr/4.7.2/:/usr/lib/gcc/avr/4.7.2/../../../avr/lib/
COLLECT_GCC_OPTIONS='-c' '-v' '-mmcu=atmega328p' '-Os' '-fwhole-program'
'-fltrans-output-list=/tmp/ccZEu3t3.ltrans.out' '-fwpa'
 /usr/bin/avr-gcc @/tmp/ccoysJBM.args
Using built-in specs.
COLLECT_GCC=/usr/bin/avr-gcc
Target: avr
Configured with: ../src/configure -v --enable-languages=c,c++ --prefix=/usr/lib
--infodir=/usr/share/info --mandir=/usr/share/man --bindir=/usr/bin
--libexecdir=/usr/lib --libdir=/usr/lib --enable-shared --with-system-zlib
--enable-long-long --enable-nls --without-included-gettext --disable-libssp
--build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=avr
Thread model: single
gcc version 4.7.2 (GCC) 
COLLECT_GCC_OPTIONS='-c' '-v' '-mmcu=atmega328p' '-Os' '-fwhole-program'
'-fltrans-output-list=/tmp/ccZEu3t3.ltrans.out' '-fltrans' '-o'
'/tmp/ccZEu3t3.ltrans0.ltrans.o'
 /usr/lib/gcc/avr/4.7.2/lto1 -quiet -dumpbase ccZEu3t3.ltrans0.o
-mmcu=atmega328p -auxbase-strip /tmp/ccZEu3t3.ltrans0.ltrans.o -Os -version
-fwhole-program -fltrans-output-list=/tmp/ccZEu3t3.ltrans.out -fltrans
@/tmp/ccAudUT3 -o /tmp/ccyDScYi.s
GNU GIMPLE (GCC) version 4.7.2 (avr)
        compiled by GNU C version 4.7.2, GMP version 5.0.5, MPFR version
3.1.0-p10, MPC version 0.9
GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
GNU GIMPLE (GCC) version 4.7.2 (avr)
        compiled by GNU C version 4.7.2, GMP version 5.0.5, MPFR version
3.1.0-p10, MPC version 0.9
GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
COLLECT_GCC_OPTIONS='-c' '-v' '-mmcu=atmega328p' '-Os' '-fwhole-program'
'-fltrans-output-list=/tmp/ccZEu3t3.ltrans.out' '-fltrans' '-o'
'/tmp/ccZEu3t3.ltrans0.ltrans.o'
 /usr/lib/gcc/avr/4.7.2/../../../avr/bin/as -mmcu=atmega328p -mno-skip-bug -o
/tmp/ccZEu3t3.ltrans0.ltrans.o /tmp/ccyDScYi.s
COMPILER_PATH=/usr/lib/gcc/avr/4.7.2/:/usr/lib/gcc/avr/4.7.2/:/usr/lib/gcc/avr/:/usr/lib/gcc/avr/4.7.2/:/usr/lib/gcc/avr/:/usr/lib/gcc/avr/4.7.2/../../../avr/bin/:/usr/lib/gcc/avr/4.7.2/:/usr/lib/gcc/avr/4.7.2/:/usr/lib/gcc/avr/:/usr/lib/gcc/avr/4.7.2/:/usr/lib/gcc/avr/:/usr/lib/gcc/avr/4.7.2/../../../avr/bin/
LIBRARY_PATH=/usr/lib/gcc/avr/4.7.2/avr5/:/usr/lib/gcc/avr/4.7.2/../../../avr/lib/avr5/:/usr/lib/gcc/avr/4.7.2/:/usr/lib/gcc/avr/4.7.2/../../../avr/lib/
COLLECT_GCC_OPTIONS='-c' '-v' '-mmcu=atmega328p' '-Os' '-fwhole-program'
'-fltrans-output-list=/tmp/ccZEu3t3.ltrans.out' '-fltrans' '-o'
'/tmp/ccZEu3t3.ltrans0.ltrans.o'
main.a(main.o): In function `main':
main.c:(.text+0x8): undefined reference to `realmain'
collect2: error: ld returned 1 exit status

[Bug target/56533] Linker problem on avr with lto and main function inside archive

[matthijs at stdin dot nl]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56533

--- Comment #3 from Matthijs Kooijman <matthijs at stdin dot nl> 2013-03-05 13:06:18 UTC ---
Seems I made a wrong observation in my original report: When I link main.o
instead of main.a, the problem does _not_ go away. In fact, I can remove a few
more flags then, while still keeping the problem around:

$ ./do
    + rm -f main.a main.o realmain.o
    + /usr/bin/avr-gcc -c main.c -o main.o
    + /usr/bin/avr-gcc -c -flto realmain.c -o realmain.o
    + /usr/bin/avr-gcc -flto -fwhole-program realmain.o main.o
    main.o: In function `main':
    main.c:(.text+0x8): undefined reference to `realmain'
    collect2: error: ld returned 1 exit status

main.c and realmain.c are the same as before.

However, adding -flto to the main.c compilation makes the problem disappear
again. I suspect that this means that without -flto, main.o is passed straight
to the linker and with -flto it is included in link-time optimization, which
would mean your previous analysis still holds.

$ ./do
    + rm -f *.a main.o realmain.o
    + /usr/bin/avr-gcc -c -flto main.c -o main.o
    + /usr/bin/avr-gcc -c -flto realmain.c -o realmain.o
    + /usr/bin/avr-gcc -flto -fwhole-program realmain.o main.o

[Bug target/56533] Linker problem on avr with lto and main function inside archive

[rguenth at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56533

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |RESOLVED
         Resolution|                            |INVALID

--- Comment #4 from Richard Biener <rguenth at gcc dot gnu.org> 2013-03-05 13:41:01 UTC ---
Yes.  You are not using a linker-plugin which would end up basically
ignoring -fwhole-program.  With -fwhole-program you are basically lieing to
GCC.

[Bug target/56533] Linker problem on avr with lto and main function inside archive

[matthijs at stdin dot nl]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56533

Matthijs Kooijman <matthijs at stdin dot nl> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |UNCONFIRMED
         Resolution|INVALID                     |

--- Comment #5 from Matthijs Kooijman <matthijs at stdin dot nl> 2013-03-05 14:38:36 UTC ---
Just for future reference, the problem here seems to be that I'm using
-fwhole-program, but the GCC LTO cannot actually look at the whole program. In
particular, .a archives and .o object files that were compiled without -flto,
are passed directly to the linker and not included in LTO. Since
-fwhole-program makes the compiler assume that all files that are included in
LTO compose the whole program, the compiler removes symbols that look unused,
but then turn up missing in the final link.

So, I shouldn't have been using -fwhole-program, or I should be aware of the
above and set externally_visible attributes as needed if I insist on using
-fwhole-program.

Ideally, the compiler would ask the linker about which symbols are used in
these "non-LTO" objects, which is done by -fuse-linker-plugin (which is implied
by -flto). However, on the AVR target, it seems there is no linker plugin (at
least not in this particular case), which means that without -fwhole-program,
the compiler cannot optimize as much (since it has to assume that all symbols
are externally visible).

[Bug target/56533] Linker problem on avr with lto and main function inside archive

[matthijs at stdin dot nl]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56533

Matthijs Kooijman <matthijs at stdin dot nl> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |RESOLVED
         Resolution|                            |INVALID

--- Comment #6 from Matthijs Kooijman <matthijs at stdin dot nl> 2013-03-05 14:40:15 UTC ---
w00ps, didn't mean to change the resolution.