[Bug plugins/101810] New: libiberty/simple-object-xcoff.c segmentation fault

[Bug plugins/101810] New: libiberty/simple-object-xcoff.c segmentation fault

[amodra at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101810

            Bug ID: 101810
           Summary: libiberty/simple-object-xcoff.c segmentation fault
           Product: gcc
           Version: 12.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: plugins
          Assignee: unassigned at gcc dot gnu.org
          Reporter: amodra at gmail dot com
  Target Milestone: ---

>From https://sourceware.org/bugzilla/show_bug.cgi?id=28179

binutils/nm-new --plugin ~/build/gcc-virgin/lto-plugin/.libs/liblto_plugin.so
-a pr28179
AddressSanitizer:DEADLYSIGNAL
=================================================================
==3630013==ERROR: AddressSanitizer: SEGV on unknown address 0x60200001000a (pc
0x7fc28ca928ea bp 0x000000000000 sp 0x7ffd425c36d0 T0)
==3630013==The signal is caused by a READ memory access.
    #0 0x7fc28ca928ea in simple_object_xcoff_find_sections
/home/alan/src/gcc-virgin/libiberty/simple-object-xcoff.c:529:26
    #1 0x7fc28ca874f7 in claim_file_handler
/home/alan/src/gcc-virgin/lto-plugin/lto-plugin.c:1189:16
    #2 0x9ad923 in try_claim /home/alan/src/binutils-gdb/bfd/plugin.c:323:7
[snip]

A little analysis of the binutils testcase reveals the xcoff file header has
nsyms of 0x80000000.  The file contains a number of places where ocr->nsyms *
SYMESZ is calculated.  Since ocr->nsyms is an unsigned int and SYMESZ a plain
number (18), the expression overflows to zero.  That results in a zero length
buffer being allocated and read from file, but 0x80000000 syms processed from
the buffer.

[Bug plugins/101810] libiberty/simple-object-xcoff.c segmentation fault

[amodra at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101810

--- Comment #1 from Alan Modra <amodra at gmail dot com> ---
Created attachment 51272
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=51272&action=edit
Proposed fix

[Bug plugins/101810] libiberty/simple-object-xcoff.c segmentation fault

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101810

--- Comment #2 from Richard Biener <rguenth at gcc dot gnu.org> ---
How about fixing the places instad?  Making SYMESZ a size_t looks like a
complete fix?

[Bug plugins/101810] libiberty/simple-object-xcoff.c segmentation fault

[amodra at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101810

--- Comment #3 from Alan Modra <amodra at gmail dot com> ---
Making SYMESZ a size_t as the patch does, is a complete fix if the code is only
compiled for 64-bit hosts where unsigned int is smaller than size_t.  If
compiled for 32-bit then the expression calculating buffer size can overflow
leading to similar segfaults on fuzzed object files.  As explained by the
comment.

[Bug other/101810] libiberty/simple-object-xcoff.c segmentation fault

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101810

Andrew Pinski <pinskia at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Last reconfirmed|                            |2022-01-09
          Component|lto                         |other
             Status|UNCONFIRMED                 |NEW
     Ever confirmed|0                           |1
           Keywords|                            |lto

--- Comment #4 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
Confirmed.