[Bug lto/64837] New: lto plugun doesn't call ld_plugin_release_input_file

[Bug lto/64837] New: lto plugun doesn't call ld_plugin_release_input_file

[hjl.tools at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64837

            Bug ID: 64837
           Summary: lto plugun doesn't call ld_plugin_release_input_file
           Product: gcc
           Version: 5.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: lto
          Assignee: unassigned at gcc dot gnu.org
          Reporter: hjl.tools at gmail dot com
                CC: ccoutant at gcc dot gnu.org

Since lto plugin doesn't call ld_plugin_release_input_file,
plugin_maybe_claim in ld/plugin.c does

  /* fd belongs to us, not the plugin; but we don't need it.  */
  close (file->fd);

It may not work for other plugins, like LLVM plugin.  It
may also leak file descriptor when gold is used.

[Bug lto/64837] lto plugun doesn't call ld_plugin_release_input_file

[trippels at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64837

Markus Trippelsdorf <trippels at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |NEW
   Last reconfirmed|                            |2015-01-28
                 CC|                            |trippels at gcc dot gnu.org
     Ever confirmed|0                           |1

--- Comment #1 from Markus Trippelsdorf <trippels at gcc dot gnu.org> ---
> It may also leak file descriptor when gold is used.

It certainly does. I ran into this issue when building Chromium with LTO.
(I just increased ulimit -n as a workaround)

[Bug lto/64837] lto plugun doesn't call ld_plugin_release_input_file

[hjl.tools at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64837

H.J. Lu <hjl.tools at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|---                         |5.0

--- Comment #2 from H.J. Lu <hjl.tools at gmail dot com> ---
Let's fix it for 5.0.

[Bug lto/64837] lto plugin doesn't call ld_plugin_release_input_file

[hjl.tools at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64837

--- Comment #4 from H.J. Lu <hjl.tools at gmail dot com> ---
A patch is posted at

https://gcc.gnu.org/ml/gcc-patches/2015-01/msg02508.html

[Bug lto/64837] lto plugin doesn't call ld_plugin_release_input_file

[hjl at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64837

--- Comment #5 from hjl at gcc dot gnu.org <hjl at gcc dot gnu.org> ---
Author: hjl
Date: Wed Jan 28 19:39:42 2015
New Revision: 220212

URL: https://gcc.gnu.org/viewcvs?rev=220212&root=gcc&view=rev
Log:
Call release_input_file in claim_file_handler

    PR lto/64837
    * lto-plugin.c (release_input_file): New.
    (claim_file_handler): Call release_input_file.
    (onload): Set release_input_file.

Modified:
    trunk/lto-plugin/ChangeLog
    trunk/lto-plugin/lto-plugin.c

[Bug lto/64837] lto plugin doesn't call ld_plugin_release_input_file

[hjl.tools at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64837

H.J. Lu <hjl.tools at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|---                         |FIXED

--- Comment #6 from H.J. Lu <hjl.tools at gmail dot com> ---
Fixed for GCC 5.

[Bug lto/64837] lto plugin doesn't call ld_plugin_release_input_file

[burnus at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64837

Tobias Burnus <burnus at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |burnus at gcc dot gnu.org

--- Comment #7 from Tobias Burnus <burnus at gcc dot gnu.org> ---
(In reply to H.J. Lu from comment #3)
> There is also a gold bug:
> 
> https://sourceware.org/bugzilla/show_bug.cgi?id=17896

Which has been closed as invalid. Quote from that PR: "The LTO plugin should
*not* use release_input_file during the claim_file handler."


Additionally, the causes ICEs with (all or all but the newest version of) gold:
https://gcc.gnu.org/ml/gcc-patches/2015-02/msg00398.html

[Bug lto/64837] lto plugin doesn't call ld_plugin_release_input_file

[hjl at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64837

--- Comment #8 from hjl at gcc dot gnu.org <hjl at gcc dot gnu.org> ---
Author: hjl
Date: Fri Feb  6 13:55:42 2015
New Revision: 220477

URL: https://gcc.gnu.org/viewcvs?rev=220477&root=gcc&view=rev
Log:
Revert the PR lto/64837 fix

It breaks gold:

% echo "int main () {}" | gcc -fuse-ld=gold -flto -x c++ -
ld.gold: internal error in remove_writer, at token.h:132
collect2: error: ld returned 1 exit status

    * lto-plugin.c (release_input_file): Removed.
    (claim_file_handler): Don't call release_input_file.
    (onload): Don't set release_input_file.

Modified:
    trunk/lto-plugin/ChangeLog
    trunk/lto-plugin/lto-plugin.c

[Bug lto/64837] lto plugin doesn't call ld_plugin_release_input_file

[hjl.tools at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64837

H.J. Lu <hjl.tools at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |REOPENED
         Resolution|FIXED                       |---

--- Comment #9 from H.J. Lu <hjl.tools at gmail dot com> ---
Fix was reverted.

[Bug lto/64837] lto plugin doesn't call ld_plugin_release_input_file

[hjl.tools at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64837

--- Comment #10 from H.J. Lu <hjl.tools at gmail dot com> ---
Created attachment 34725
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=34725&action=edit
A patch

[Bug lto/64837] lto plugin doesn't call ld_plugin_release_input_file

[hjl.tools at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64837

H.J. Lu <hjl.tools at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|REOPENED                    |NEW

--- Comment #11 from H.J. Lu <hjl.tools at gmail dot com> ---
(In reply to Markus Trippelsdorf from comment #1)
> > It may also leak file descriptor when gold is used.
> 
> It certainly does. I ran into this issue when building Chromium with LTO.
> (I just increased ulimit -n as a workaround)

Can you try the patch

https://gcc.gnu.org/bugzilla/attachment.cgi?id=34725

[Bug lto/64837] lto plugin doesn't call ld_plugin_release_input_file

[hjl.tools at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64837

H.J. Lu <hjl.tools at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #34725|0                           |1
        is obsolete|                            |

--- Comment #12 from H.J. Lu <hjl.tools at gmail dot com> ---
Created attachment 34727
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=34727&action=edit
A patch

[Bug lto/64837] lto plugin doesn't call ld_plugin_release_input_file

[trippels at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64837

Markus Trippelsdorf <trippels at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |marxin at gcc dot gnu.org

--- Comment #13 from Markus Trippelsdorf <trippels at gcc dot gnu.org> ---
(In reply to H.J. Lu from comment #11)
> (In reply to Markus Trippelsdorf from comment #1)
> > > It may also leak file descriptor when gold is used.
> > 
> > It certainly does. I ran into this issue when building Chromium with LTO.
> > (I just increased ulimit -n as a workaround)
> 
> Can you try the patch
> 
> https://gcc.gnu.org/bugzilla/attachment.cgi?id=34725

Unfortunately no. It was possible to build Chromium with LTO
on my 8GB machine two years ago, but not anymore.

Maybe Martin can give it a try? CCing him.

[Bug lto/64837] lto plugin doesn't call ld_plugin_release_input_file

[trippels at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64837

--- Comment #14 from Markus Trippelsdorf <trippels at gcc dot gnu.org> ---
Well, at least your patch survives a Firefox LTO build using gold
on a ppc64 test machine.

[Bug lto/64837] lto plugin doesn't call ld_plugin_release_input_file

[hjl.tools at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64837

--- Comment #15 from H.J. Lu <hjl.tools at gmail dot com> ---
(In reply to Markus Trippelsdorf from comment #14)
> Well, at least your patch survives a Firefox LTO build using gold
> on a ppc64 test machine.

Does it build without my patch?

[Bug lto/64837] lto plugin doesn't call ld_plugin_release_input_file

[trippels at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64837

--- Comment #16 from Markus Trippelsdorf <trippels at gcc dot gnu.org> ---
(In reply to H.J. Lu from comment #15)
> (In reply to Markus Trippelsdorf from comment #14)
> > Well, at least your patch survives a Firefox LTO build using gold
> > on a ppc64 test machine.
> 
> Does it build without my patch?

Yes. So all I've tested is that it doesn't crash.

[Bug lto/64837] lto plugin doesn't call ld_plugin_release_input_file

[hjl.tools at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64837

--- Comment #17 from H.J. Lu <hjl.tools at gmail dot com> ---
(In reply to Markus Trippelsdorf from comment #16)
> (In reply to H.J. Lu from comment #15)
> > (In reply to Markus Trippelsdorf from comment #14)
> > > Well, at least your patch survives a Firefox LTO build using gold
> > > on a ppc64 test machine.
> > 
> > Does it build without my patch?
> 
> Yes. So all I've tested is that it doesn't crash.

Can you lower file descriptor limit such that GCC
crashes without my patch and works with my patch?

[Bug lto/64837] lto plugin doesn't call ld_plugin_release_input_file

[trippels at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64837

--- Comment #18 from Markus Trippelsdorf <trippels at gcc dot gnu.org> ---
(In reply to H.J. Lu from comment #17)
> (In reply to Markus Trippelsdorf from comment #16)
> > (In reply to H.J. Lu from comment #15)
> > > (In reply to Markus Trippelsdorf from comment #14)
> > > > Well, at least your patch survives a Firefox LTO build using gold
> > > > on a ppc64 test machine.
> > > 
> > > Does it build without my patch?
> > 
> > Yes. So all I've tested is that it doesn't crash.
> 
> Can you lower file descriptor limit such that GCC
> crashes without my patch and works with my patch?

Martin is currently running these tests with Chromium...

(Also Carry apparently thinks that the
https://sourceware.org/bugzilla/show_bug.cgi?id=15660
issue is caused by archives and not by normal object files.)

[Bug lto/64837] lto plugin doesn't call ld_plugin_release_input_file

[marxin at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64837

--- Comment #19 from Martin Liška <marxin at gcc dot gnu.org> ---
(In reply to Markus Trippelsdorf from comment #18)
> (In reply to H.J. Lu from comment #17)
> > (In reply to Markus Trippelsdorf from comment #16)
> > > (In reply to H.J. Lu from comment #15)
> > > > (In reply to Markus Trippelsdorf from comment #14)
> > > > > Well, at least your patch survives a Firefox LTO build using gold
> > > > > on a ppc64 test machine.
> > > > 
> > > > Does it build without my patch?
> > > 
> > > Yes. So all I've tested is that it doesn't crash.
> > 
> > Can you lower file descriptor limit such that GCC
> > crashes without my patch and works with my patch?
> 
> Martin is currently running these tests with Chromium...
> 
> (Also Carry apparently thinks that the
> https://sourceware.org/bugzilla/show_bug.cgi?id=15660
> issue is caused by archives and not by normal object files.)

Hello.

I can confirm that following patch:
https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;a=blobdiff;f=gold/archive.cc;h=6d259803185368ccbe3b71cda4040d90105b66d3;hp=69107f5fc072ecce25df57cbe22464f74d2cc6da;hb=2cfbf2fece582c29df348104b28677c38a8301f4;hpb=8265ef950202a4bf0d3444802ad5d7087b4b185d

works for me. Before the patch was applied, ulimit -n 10000 wasn't enough to
survive chrome link. With the patch, even 100 is enough.

Thanks,
Martin
>From gcc-bugs-return-476977-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org Thu Feb 12 09:13:04 2015
Return-Path: <gcc-bugs-return-476977-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org>
Delivered-To: listarch-gcc-bugs@gcc.gnu.org
Received: (qmail 27516 invoked by alias); 12 Feb 2015 09:13:04 -0000
Mailing-List: contact gcc-bugs-help@gcc.gnu.org; run by ezmlm
Precedence: bulk
List-Id: <gcc-bugs.gcc.gnu.org>
List-Archive: <http://gcc.gnu.org/ml/gcc-bugs/>
List-Post: <mailto:gcc-bugs@gcc.gnu.org>
List-Help: <mailto:gcc-bugs-help@gcc.gnu.org>
Sender: gcc-bugs-owner@gcc.gnu.org
Delivered-To: mailing list gcc-bugs@gcc.gnu.org
Received: (qmail 27412 invoked by uid 48); 12 Feb 2015 09:13:00 -0000
From: "ktietz at gcc dot gnu.org" <gcc-bugzilla@gcc.gnu.org>
To: gcc-bugs@gcc.gnu.org
Subject: [Bug gcov-profile/61889] [5 Regression] gcov-tool.c uses nftw, ftw.h
Date: Thu, 12 Feb 2015 09:13:00 -0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: changed
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Product: gcc
X-Bugzilla-Component: gcov-profile
X-Bugzilla-Version: 5.0
X-Bugzilla-Keywords: build
X-Bugzilla-Severity: normal
X-Bugzilla-Who: ktietz at gcc dot gnu.org
X-Bugzilla-Status: RESOLVED
X-Bugzilla-Priority: P1
X-Bugzilla-Assigned-To: xur at google dot com
X-Bugzilla-Target-Milestone: 5.0
X-Bugzilla-Flags:
X-Bugzilla-Changed-Fields:
Message-ID: <bug-61889-4-AOQvoCdf9U@http.gcc.gnu.org/bugzilla/>
In-Reply-To: <bug-61889-4@http.gcc.gnu.org/bugzilla/>
References: <bug-61889-4@http.gcc.gnu.org/bugzilla/>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-Bugzilla-URL: http://gcc.gnu.org/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0
X-SW-Source: 2015-02/txt/msg01310.txt.bz2
Content-length: 298

https://gcc.gnu.org/bugzilla/show_bug.cgi?ida889

--- Comment #36 from Kai Tietz <ktietz at gcc dot gnu.org> ---
Well, I guess that you missed to reconfigure gcc.  By checking current source
is the include of ftw.h guarded by HAVE_FTW_H check, which get defined by
configure if header is found.

[Bug lto/64837] lto plugin doesn't call ld_plugin_release_input_file

[hjl.tools at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64837

H.J. Lu <hjl.tools at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|---                         |FIXED

--- Comment #20 from H.J. Lu <hjl.tools at gmail dot com> ---
Invalid.