[Bug c++/51878] New: ICE or OOM with decltype + variadic templates + "indirect" function call

[Bug c++/51878] New: ICE or OOM with decltype + variadic templates + "indirect" function call

[b.r.longbons at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=51878

             Bug #: 51878
           Summary: ICE or OOM with decltype + variadic templates +
                    "indirect" function call
    Classification: Unclassified
           Product: gcc
           Version: 4.6.2
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: c++
        AssignedTo: unassigned@gcc.gnu.org
        ReportedBy: b.r.longbons@gmail.com


Created attachment 26347
  --> http://gcc.gnu.org/bugzilla/attachment.cgi?id=26347
testcase that is probably minimal

(Bugzilla is suggesting lots of duplicates for me, but they all seem to be old
...)

While trying to work around what may or may not be bug 44175, I tried not
calling the function directly. I hit compile and my computer became
unresponsive. With a little investigation, I discovered GCC was trying to
allocate an infinite amount of memory. Running gcc with niceness 19, my system
was responsive enough for me to watch 'top' until it consumed all my swap and
got OOM'ed.

Interestingly, after reducing my testcase, it allocates a lot of memory, but
eventually ICEs instead (SEGV in cc1plus). I swear I didn't change anything in
the testcase since the last time I got an OOM. Nonetheless, I've left an #error
for your safety.

Also, at least in some variants (including the original), the memory explosion
can happen after an Error. Sadly, reproducing this bug with after an error has
occurred is not possible with this minimal testcase. (The particular error was
caused by forgetting to include the pack in one of my if-branches, so it was
calling the functor with too few arguments, leading to "no match found for
apply, candidates are" and then listed the one candidate.)

Expected result:
The following code works (but is admittedly unnecessary if other GCC bugs are
fixed)

template<class F, class... T>
auto indirect_call(F f, T... t) -> decltype(f(t...))
{
    return f(t...);
}

Actual result:
If you're lucky, ICE. Else, an unresponsive system and eventually OOM.

Versions:
Here's where it gets interesting ...
* gcc <= 4.6.0 fail for various reasons unrelated to this bug.
* 4.6.1 and 4.6.2 from Gentoo, and svn branch 4.6 r183147, and whatever svn
snapshot Debian is using, have the effect described in this bug report, but
work if the call is made directly.
* svn trunk (r182728 and r183173) has some sort of regression such that even
the direct call ICEs. My gut says that should probably file a separate bug
about whatever causes that (if there isn't one already), but I don't feel I
know enough.
(My workaround for bug 44175 that works in 4.6 is also broken on trunk. I don't
know if that is the same regression that makes this one fail)

[Bug c++/51878] ICE or OOM with decltype + variadic templates + "indirect" function call

[b.r.longbons at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=51878

--- Comment #1 from Ben Longbons <b.r.longbons at gmail dot com> 2012-01-17 08:48:12 UTC ---
Created attachment 26348
  --> http://gcc.gnu.org/bugzilla/attachment.cgi?id=26348
variant of the testcase that compiles with gcc 4.6.{1,2}

[Bug c++/51878] ICE or OOM with decltype + variadic templates + "indirect" function call

[paolo.carlini at oracle dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=51878

Paolo Carlini <paolo.carlini at oracle dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jason at gcc dot gnu.org

--- Comment #2 from Paolo Carlini <paolo.carlini at oracle dot com> 2012-01-17 11:28:43 UTC ---
If I apply the below patchlet (parts of it are probably sane changes anyway) I
see a clean ICE with template instantiation depth exceeded. Then I wonder if
modulo the small tweaks we have just a duplicate of, eg, PR51304.

Maybe Jason can help triaging... (thanks in advance)

//////////////////

Index: error.c
===================================================================
--- error.c    (revision 183238)
+++ error.c    (working copy)
@@ -2471,6 +2471,10 @@ dump_expr (tree t, int flags)
       dump_expr (resolve_virtual_fun_from_obj_type_ref (t), flags);
       break;

+    case NONTYPE_ARGUMENT_PACK:
+      dump_template_argument (t, flags);
+      break;
+
       /*  This list is incomplete, but should suffice for now.
       It is very important that `sorry' does not call
       `report_error_function'.  That could cause an infinite loop.  */
Index: mangle.c
===================================================================
--- mangle.c    (revision 183238)
+++ mangle.c    (working copy)
@@ -2319,6 +2319,9 @@ write_bare_function_type (const tree type, const i
 {
   int java_method_p;

+  if (type == error_mark_node)
+    return;
+
   MANGLE_TRACE_TREE ("bare-function-type", type);

   /* Detect Java methods and emit special encoding.  */

[Bug c++/51878] ICE or OOM with decltype + variadic templates + "indirect" function call

[paolo.carlini at oracle dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=51878

Paolo Carlini <paolo.carlini at oracle dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
      Known to work|                            |4.8.0

--- Comment #3 from Paolo Carlini <paolo.carlini at oracle dot com> 2012-10-11 23:20:26 UTC ---
Works fine in mainline, I'm adding the testcase and closing the PR as fixed.

[Bug c++/51878] ICE or OOM with decltype + variadic templates + "indirect" function call

[paolo at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=51878

--- Comment #4 from paolo at gcc dot gnu.org <paolo at gcc dot gnu.org> 2012-10-11 23:37:55 UTC ---
Author: paolo
Date: Thu Oct 11 23:37:48 2012
New Revision: 192381

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=192381
Log:
2012-10-11  Paolo Carlini  <paolo.carlini@oracle.com>

    PR c++/51878
    * g++.dg/cpp0x/decltype45.C: New.

Added:
    trunk/gcc/testsuite/g++.dg/cpp0x/decltype45.C
Modified:
    trunk/gcc/testsuite/ChangeLog

[Bug c++/51878] ICE or OOM with decltype + variadic templates + "indirect" function call

[paolo.carlini at oracle dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=51878

Paolo Carlini <paolo.carlini at oracle dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |RESOLVED
         Resolution|                            |FIXED
   Target Milestone|---                         |4.8.0

--- Comment #5 from Paolo Carlini <paolo.carlini at oracle dot com> 2012-10-11 23:39:14 UTC ---
Done.