[Bug lto/41858] New: -flto -O2 crashes when many source files.

[Bug lto/41858] New: -flto -O2 crashes when many source files.

[bstarynk at gcc dot gnu dot org]

When compiling a lot (e.g. 4000) source files with
   gcc-trunk -flto -O2 [A-Z]*.c -o big
the lto component crashes with sigsegv.

Version:
gcc version 4.5.0 20091028 (experimental) (GCC) 

that it the trunk rev 153680


-- 
           Summary: -flto -O2 crashes when many source files.
           Product: gcc
           Version: 4.5.0
            Status: UNCONFIRMED
          Keywords: lto
          Severity: major
          Priority: P3
         Component: lto
        AssignedTo: unassigned at gcc dot gnu dot org
        ReportedBy: bstarynk at gcc dot gnu dot org
 GCC build triplet: x86_64-unknown-linux-gnu
  GCC host triplet: x86_64-unknown-linux-gnu
GCC target triplet: x86_64-unknown-linux-gnu


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=41858

[Bug lto/41858] -flto -O2 crashes when many source files.

[bstarynk at gcc dot gnu dot org]

------- Comment #1 from bstarynk at gcc dot gnu dot org  2009-10-28 21:40 -------
Created an attachment (id=18927)
 --> (http://gcc.gnu.org/bugzilla/attachment.cgi?id=18927&action=view)
a small program which generates many files to exhibit the bug

The attached main.c file generates many files.

Put it in a new directory (empty).

Compile it with 
   gcc -O main.c -o main

Run it to generate many files, for example
  ./main 4000

it should display messages ending with

generated function #3904 in E_150
generated function #3968 in Q_152
generated 4000 files last is W_153.c
try to compile all 4000 files with:
         gcc-trunk -flto -O2 -DGENERATEDALL main.c [A-Z]*.c -o big


Now, run the suggest command (replacing gcc-trunk with what is appropriate for
you).

Regards.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=41858

[Bug lto/41858] -flto -O2 crashes when many source files.

[bstarynk at gcc dot gnu dot org]

------- Comment #2 from bstarynk at gcc dot gnu dot org  2009-10-28 21:47 -------
The attachment main.c is a source file generator and test case.

Put it in an empty directory.

First, compile it just with
  gcc main.c -o main
[you could add other flags to gcc, you can use any flavor of gcc]

Then run it once to generate many files.
  ./main 4000

Then it shows a lot of messages, ending with
 generated function #3968 in Q_152
generated 4000 files last is W_153.c
try to compile all 4000 files with:
         gcc-trunk -flto -O2 -DGENERATEDALL main.c [A-Z]*.c -o big

Do what it tells, so run the gcc trunk as said.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=41858

[Bug lto/41858] -flto -O2 crashes when many source files.

[rguenth at gcc dot gnu dot org]

------- Comment #3 from rguenth at gcc dot gnu dot org  2009-10-28 22:10 -------
Well, it looks like you generated a pretty degenerate callgraph.  Trying
to reproduce.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=41858

[Bug lto/41858] -flto -O2 crashes when many source files.

[bstarynk at gcc dot gnu dot org]

------- Comment #4 from bstarynk at gcc dot gnu dot org  2009-10-28 22:22 -------
The linker is gold on Debian/Sid/Amd64 
% /usr/bin/ld -v
GNU gold (GNU Binutils for Debian 2.20) 1.9

It could be that it is that gold which crashes. Apparently, it open a lot of
files, and when getting the MFILE errno, something has a sigegv. Perhaps that
something is lto1.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=41858

[Bug lto/41858] -flto -O2 crashes when many source files.

[rguenth at gcc dot gnu dot org]

------- Comment #5 from rguenth at gcc dot gnu dot org  2009-10-28 23:10 -------
Confirmed.  There's a file-descriptor leak (well, we kepp to many of them
open).


-- 

rguenth at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |NEW
     Ever Confirmed|0                           |1
   Last reconfirmed|0000-00-00 00:00:00         |2009-10-28 23:10:07
               date|                            |


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=41858

[Bug lto/41858] -flto -O2 crashes when many source files.

[rguenth at gcc dot gnu dot org]

------- Comment #6 from rguenth at gcc dot gnu dot org  2009-10-28 23:12 -------
I have a patch.


-- 

rguenth at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         AssignedTo|unassigned at gcc dot gnu   |rguenth at gcc dot gnu dot
                   |dot org                     |org
             Status|NEW                         |ASSIGNED
   Last reconfirmed|2009-10-28 23:10:07         |2009-10-28 23:12:19
               date|                            |


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=41858

[Bug lto/41858] -flto -O2 crashes when many source files.

[bstarynk at gcc dot gnu dot org]

------- Comment #7 from bstarynk at gcc dot gnu dot org  2009-10-29 16:06 -------
(In reply to comment #6)
> I have a patch.
> 

Richard's patch on http://gcc.gnu.org/ml/gcc/2009-10/msg00602.html
seems to solve the original problem I've got 
http://gcc.gnu.org/ml/gcc-patches/2009-10/msg01741.html

I hope that Richard's patch will be accepted in 4.5.
I am marking this PR as a resolved bug (or should I wait for the patch to be
accepted?)


-- 

bstarynk at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |RESOLVED
         Resolution|                            |FIXED


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=41858

[Bug lto/41858] -flto -O2 crashes when many source files.

[rguenth at gcc dot gnu dot org]

------- Comment #8 from rguenth at gcc dot gnu dot org  2009-10-30 15:59 -------
Subject: Bug 41858

Author: rguenth
Date: Fri Oct 30 15:58:57 2009
New Revision: 153758

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=153758
Log:
2009-10-30  Richard Guenther  <rguenther@suse.de>

        PR lto/41858
        * lto-streamer.h (struct lto_file_decl_data): Remove fd member.

        lto/
        * lto.c (lto_file_read): Do not set file_data->fd.
        (lto_read_section_data): Use a single-entry file-descriptor cache.
        Do not check the result of xmalloc.
        (free_section_data): Do not use file_data->fd.
        (lto_read_all_file_options): Likewise.

Modified:
    trunk/gcc/ChangeLog
    trunk/gcc/lto-streamer.h
    trunk/gcc/lto/ChangeLog
    trunk/gcc/lto/lto.c


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=41858