[Bug rtl-optimization/52629] New: global buffer overflow in gcc/reload1.c

[Bug rtl-optimization/52629] New: global buffer overflow in gcc/reload1.c

[konstantin.s.serebryany at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52629

             Bug #: 52629
           Summary: global buffer overflow in gcc/reload1.c
    Classification: Unclassified
           Product: gcc
           Version: 4.8.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: rtl-optimization
        AssignedTo: unassigned@gcc.gnu.org
        ReportedBy: konstantin.s.serebryany@gmail.com


Building gcc trunk (r185531, linux, x86_64) with AddressSanitizer (a memory
error detector)
shows a global buffer overflow in gcc/reload1.c while compiling
libgcc/libgcc2.c with ./gcc/xgcc

==21687== ERROR: AddressSanitizer global-buffer-overflow on address
0x000002ddc551 at pc 0xf98a5f bp 0x7fffdcbe1050 sp 0x7fffdcbe1048               
READ of size 1 at 0x000002ddc551 thread T0                                      
    #0 0xf98a5f in count_spilled_pseudo gcc/reload1.c:1830                      
    #1 0xf715a6 in bmp_iter_set_init gcc/bitmap.h:385                           
    #2 0xd55dc2 in do_reload gcc/ira.c:3733                                     
    #3 0xea9e0d in execute_one_pass gcc/passes.c:2084                           
    #4 0xeaaf4d in execute_pass_list gcc/passes.c:2139                          
    #5 0xeaaf71 in execute_pass_list gcc/passes.c:2141                          
    #6 0x1200802 in invoke_plugin_callbacks gcc/plugin.h:57                     
    #7 0x81917d in cgraph_expand_function gcc/cgraphunit.c:1840                 
    #8 0x820425 in cgraph_expand_all_functions gcc/cgraphunit.c:1904            
    #9 0x81c655 in timevar_pop gcc/timevar.h:110                                
    #10 0x4ebc2e in c_write_global_declarations gcc/c-decl.c:10053              
    #11 0x10b82c5 in compile_file gcc/toplev.c:575                              
    #12 0x2b1cbf568c4d in __libc_start_main
/build/buildd/eglibc-2.11.1/csu/libc-start.c:258                                
0x000002ddc551 is located 17 bytes to the right of global variable
'default_target_hard_regs (../../gcc_trunk/gcc/reginfo.c)' (0x2dd9d80) of size
10176  

The buggy lines: 

static void                                                                     
count_spilled_pseudo (int spilled, int spilled_nregs, int reg)                  
{                                                                               
  int freq = REG_FREQ (reg);                                                    
  int r = reg_renumber[reg];                                                    
  int nregs = hard_regno_nregs[r][PSEUDO_REGNO_MODE (reg)];             


the value of 'r' can be -1, thus the buffer overflow. 

One can confirm this bug by changing the code like this and performing full gcc
build:

count_spilled_pseudo (int spilled, int spilled_nregs, int reg)
{
  int freq = REG_FREQ (reg);
  int r = reg_renumber[reg];
  int nregs;
  gcc_assert(r >= 0);  /*<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<*/
  nregs = hard_regno_nregs[r][PSEUDO_REGNO_MODE (reg)];

[Bug rtl-optimization/52629] global buffer overflow in gcc/reload1.c

[ebotcazou at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52629

Eric Botcazou <ebotcazou at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |ASSIGNED
   Last reconfirmed|                            |2012-03-21
                 CC|                            |ebotcazou at gcc dot
                   |                            |gnu.org
         AssignedTo|unassigned at gcc dot       |ebotcazou at gcc dot
                   |gnu.org                     |gnu.org
     Ever Confirmed|0                           |1

--- Comment #1 from Eric Botcazou <ebotcazou at gcc dot gnu.org> 2012-03-21 16:21:21 UTC ---
Ugh.  count_pseudo has the correct implementation.

[Bug rtl-optimization/52629] out-of-bounds access in reload1.c

[ebotcazou at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52629

--- Comment #2 from Eric Botcazou <ebotcazou at gcc dot gnu.org> 2012-03-26 08:41:14 UTC ---
Author: ebotcazou
Date: Mon Mar 26 08:41:02 2012
New Revision: 185787

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=185787
Log:
    PR rtl-optimization/52629
    * reload1.c (count_pseudo): Short-circuit common case.
    (count_spilled_pseudo): Return early for pseudos without hard regs.
    Assert that the pseudo has got a hard reg before manipulating it.

Modified:
    trunk/gcc/ChangeLog
    trunk/gcc/reload1.c

[Bug rtl-optimization/52629] out-of-bounds access in reload1.c

[ebotcazou at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52629

Eric Botcazou <ebotcazou at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |RESOLVED
         Resolution|                            |FIXED
   Target Milestone|---                         |4.8.0

--- Comment #3 from Eric Botcazou <ebotcazou at gcc dot gnu.org> 2012-03-28 21:51:51 UTC ---
On the trunk only.