[Bug rtl-optimization/43520] New: [4.5 Regression] gcc.dg/pr43058.c causes kernel panic on Linux/ia64

[Bug rtl-optimization/43520] New: [4.5 Regression] gcc.dg/pr43058.c causes kernel panic on Linux/ia64

[hjl dot tools at gmail dot com]

From

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=43058#c16

The testcase checked into mainline is causing kernel panics on my debian
testing ia64-linux machine when I run the gcc testsuite.  The kernel panic is
coming from the out-of-memory killer, when it runs out of processes to kill.  I
have 2GB main memory and 2GB swap.  I tried a x86-linux hosted cross compiler
to ia64-linux, and I see the cc1 process uses 3GB before the kernel kills it. 
I suspect a 32-bit x86-linux process can't use more than 3GB.  I don't know how
much memory is needed for this testcase, but it is clearly too much.


-- 
           Summary: [4.5 Regression] gcc.dg/pr43058.c causes kernel panic on
                    Linux/ia64
           Product: gcc
           Version: 4.5.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: rtl-optimization
        AssignedTo: unassigned at gcc dot gnu dot org
        ReportedBy: hjl dot tools at gmail dot com
GCC target triplet: ia64-unknown-linux-gnu


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=43520

[Bug rtl-optimization/43520] [4.5 Regression] gcc.dg/pr43058.c causes kernel panic on Linux/ia64

[hjl dot tools at gmail dot com]

------- Comment #1 from hjl dot tools at gmail dot com  2010-03-25 16:15 -------
From

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=43058#c17

But clearly it is not var-tracking that eats all the memory, instead it is the
scheduler, and it happens also with -g0, and doesn't happen with
-fno-schedule-insns -fno-schedule-insns2.  So, please open a separate bugreport
about it, reopening this one for unrelated reasons will just lead to confusion.
 I guess I could add { target { ! "ia64-*-*" } } as a quick workaround.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=43520

[Bug rtl-optimization/43520] gcc.dg/pr43058.c uses way too memory on ia64

[pinskia at gcc dot gnu dot org]

------- Comment #2 from pinskia at gcc dot gnu dot org  2010-03-25 19:05 -------
Did you test earlier versions of GCC with the testcase?

Also it should not hard to figure out where in the scheduler the memory is
being used.  The kernel panic is not our fault so make sure the kernel guys get
a bug report also; the memory usage is though.


-- 

pinskia at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
 GCC target triplet|ia64-unknown-linux-gnu      |ia64-*-*
            Summary|[4.5 Regression]            |gcc.dg/pr43058.c uses way
                   |gcc.dg/pr43058.c causes     |too memory on ia64
                   |kernel panic on Linux/ia64  |


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=43520

[Bug rtl-optimization/43520] gcc.dg/pr43058.c uses way too memory on ia64

[wilson at codesourcery dot com]

------- Comment #3 from wilson at codesourcery dot com  2010-03-25 23:21 -------
Subject: Re:  gcc.dg/pr43058.c uses way too
 memory on ia64

On Thu, 2010-03-25 at 19:05 +0000, pinskia at gcc dot gnu dot org wrote:
> Did you test earlier versions of GCC with the testcase?

I hadn't gotten around to it yet.  I am at Cisco Tue-Thu, and I can't do
much IA-64 work these days.

It compiles OK with gcc-4.4.

> Also it should not hard to figure out where in the scheduler the memory is
> being used.  The kernel panic is not our fault so make sure the kernel guys get
> a bug report also; the memory usage is though.

Another thing I hadn't gotten around to yet.  The problem appears to be
Vlad's -fsched-pressure stuff.  reg_last->implicit_sets has about 5000
instructions on the list.  If we have 5000 insns each of which is
getting a dependency on each other, that is 25M dependencies, at 40
bytes each, which is 1GB, not counting other overhead.  There is also
memory being allocated somewhere else, as this testcase needs more than
3GB to be compiled.  It appears that these lists need some kind of
throttling to prevent them from getting too big.  Or maybe they aren't
being constructed correctly for IA-64. -fno-sched-pressure doesn't help,
as that does not disable creation of the implicit register sets.  I can
spend more time looking at this later.

Everybody knows the linux kernel oom killer has problems, and that there
are no easy solutions.  I don't need to report that.

Jim


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=43520

[Bug rtl-optimization/43520] gcc.dg/pr43058.c uses way too memory on ia64

[wilson at gcc dot gnu dot org]

------- Comment #4 from wilson at gcc dot gnu dot org  2010-03-30 23:58 -------
I've confirmed that Vlad's -fsched-pressure patch is the problem, and done a
bit more analysis to see why.

The testcase has approximately 20,000 movdi_internal insns, 6000 movsi_internal
insns, and 10,000 call_value_gp insns.

The ia64 movdi_internal patterns has constraints 'd' for memory pipeline
application registers and 'e' for integer pipeline application registers.  The
'd' class has 2 regs, both fixed.  The 'e' class has 3 regs, two fixed.  Since
the two classes have only 0 and 1 allocatable register each, the
-fsched-pressure patch puts them in implicit_reg_pending_clobbers.  The
movsi_internal pattern also uses 'd', so it is handled similarly.  So we end up
with all 26,000 move insns on the reg_last->implicit_sets list.

Meanwhile, because we have 4 fixed registers, every call insn is assumes to use
these registers.  Thus we end up with all 10,000 call insns on reg_last->uses.

Since we create a dependency between implicit_sets and uses, that means we end
up with approx 260M dependencies, each 40 bytes each, which is 10GB.  Plus
memory for insns lists and other stuff.

The convention is that there is only supposed to be one mov* pattern, since
reload does not re-recognize, so it doesn't appear to me that the ia64.md file
is doing anything wrong.

We could reduce the problem if the fixed registers were split out into separate
move insns though.  We still have the one not-fixed register (ar.lc), but it
isn't call clobbered either, so I think that one might be OK.  I haven't tested
that theory yet.  We would need to split the 'e' class to separate the fixed
and non-fixed registers.

Another option here would be to have a special letter like '*' and '#' except
that it can be used to disable the register-pressure code.  We still need to
split the 'e' class for this, since we still want the register-pressure code
for the non-fixed register (ar.lc).  This would require fewer changes to the
ia64 port than the above option.

Another option here is to throttle the reg_last->use and/or
reg_last->implicit_sets lists somehow.  We could just keep a count of them and
arbitrarily flush them when they get too big.  There is already code to do this
that uses uses_length and clobbers_length.  Adding a implicit_sets_length might
help.  There is code to free a list when we see a set, but the testcase does
not have sets for 4 of the 5 registers, so the lists do not get freed this way.
 The current code that tests uses_length and clobber_length only triggers if
there is a clobber, and again there are no clobbers of any of the 5 registers
here.  The lists will only get freed if we flush them some how in the
implicit_sets handling code.

Or maybe we could try to keep trace of reads and writes better.  If we have a
series of implicit sets followed by a series of uses followed by some more
implicit sets, then we can actually flush the implicit_set list when we see the
second set of implicit sets.  This is because every use will depend on every
implicit set in the first group, and every implicit set in the second group
will depend on every use, so there is no need to retain the insns in the first
group in the implicit set list.  Similarly, we can free insns from the use
group when we see a second set of uses.

Another option here is to make -fno-sched-pressure disable the code that sets
implicit_reg_pending_clobbers, though this isn't a fix, just a workaround.


-- 

wilson at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |vmakarov at redhat dot com
             Status|UNCONFIRMED                 |NEW
     Ever Confirmed|0                           |1
   Last reconfirmed|0000-00-00 00:00:00         |2010-03-30 23:58:23
               date|                            |


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=43520

[Bug rtl-optimization/43520] gcc.dg/pr43058.c uses way too memory on ia64

[wilson at gcc dot gnu dot org]

------- Comment #5 from wilson at gcc dot gnu dot org  2010-04-14 21:57 -------
I think the first part of the solution here is to make ira stop handling
zero-size classes as ones that have a potential register pressure problem.

The second part, is that I think we can make ar.lc a fixed register.  I think
the only reason why it is call-saved is because the ABI says it is call-saved,
but the ia64 prologue code has to handle the register specially anyways, so it
doesn't matter whether we mark it as call-saved or call-clobbered.  Also, all
references to it are generated as hard-registers, so it is OK for it to be
fixed.  There is the off change that someone is incorrectly using a 'e'
constraint to put values in the register.  We should check for that in a few
obvious places like the linux kernel and glibc.  If this is true, then we will
have to split the 'e' class in two, and ar.lc will have to remain in 'e' and
the other two regs will have to move.

I did a little checking to see if making ar.lc fixed would work, and I noticed
that the handling of doloops is wrong.  The default handling is to reject loops
with calls, but since ia64 ar.lc is call-saved, we should allow them.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=43520

[Bug rtl-optimization/43520] gcc.dg/pr43058.c uses way too memory on ia64

[wilson at gcc dot gnu dot org]

------- Comment #6 from wilson at gcc dot gnu dot org  2010-04-14 21:59 -------
Created an attachment (id=20381)
 --> (http://gcc.gnu.org/bugzilla/attachment.cgi?id=20381&action=view)
untested patch that works for testcase

This is logically 3 different patches that can be handled separately.  The
doloop hook change needs performance testing to verify that it is useful.  The
patch to make ar.lc fixed needs to be double checked with the kernel and glibc
to make sure we don't break them.  The ira patch needs approval from an ira
maintainer.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=43520

[Bug rtl-optimization/43520] gcc.dg/pr43058.c uses way too memory on ia64

[wilson at gcc dot gnu dot org]

------- Comment #7 from wilson at gcc dot gnu dot org  2010-04-20 01:17 -------
Subject: Bug 43520

Author: wilson
Date: Tue Apr 20 01:16:59 2010
New Revision: 158539

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=158539
Log:
partial fix, make all 'e' class regs fixed
PR rtl-optimization/43520
* config/ia64/ia64.h (FIXED_REGISTERS, CALL_USED_REGISTERS): Make
ar.lc fixed and call-used.

Modified:
    trunk/gcc/ChangeLog
    trunk/gcc/config/ia64/ia64.h


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=43520

[Bug rtl-optimization/43520] gcc.dg/pr43058.c uses way too memory on ia64

[wilson at gcc dot gnu dot org]

------- Comment #8 from wilson at gcc dot gnu dot org  2010-04-21 05:29 -------
Subject: Bug 43520

Author: wilson
Date: Wed Apr 21 05:29:11 2010
New Revision: 158584

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=158584
Log:
PR rtl-optimization/43520
* ira-lives.c (ira_implicitly_set_insn_hard_regs): Exclude classes with
zero available registers.

Modified:
    trunk/gcc/ChangeLog
    trunk/gcc/ira-lives.c


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=43520

[Bug rtl-optimization/43520] gcc.dg/pr43058.c uses way too memory on ia64

[wilson at gcc dot gnu dot org]

-- 

wilson at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         AssignedTo|unassigned at gcc dot gnu   |wilson at gcc dot gnu dot
                   |dot org                     |org
             Status|NEW                         |ASSIGNED
   Last reconfirmed|2010-03-30 23:58:23         |2010-04-21 05:30:12
               date|                            |


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=43520

[Bug rtl-optimization/43520] gcc.dg/pr43058.c uses way too memory on ia64

[wilson at gcc dot gnu dot org]

------- Comment #9 from wilson at gcc dot gnu dot org  2010-04-21 05:31 -------
Fixed on mainline.


-- 

wilson at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |RESOLVED
         Resolution|                            |FIXED


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=43520