LTO: buffer overflow in lto_output_init_mode_table

LTO: buffer overflow in lto_output_init_mode_table

[Jivan Hakobyan]

[-- Attachment #1.1: Type: text/plain, Size: 398 bytes --]

In the case when enabled -flto=N GCC aborted compilation.
The reason is the overflowing streamer_mode_table buffer.
It has 1 << 8 bytes but lto_output_init_mode_table() tries to fill
with MAX_MACHINE_MODE bytes.

gcc/ChangeLog:
       * tree-streamer.h (streamer_mode_table): Changed buffer size
       * tree-streamer.cc (streamer_mode_table): Likewise.


-- 
With the best regards
Jivan Hakobyan

[-- Attachment #2: lto.patch --]
[-- Type: text/x-patch, Size: 1232 bytes --]

diff --git a/gcc/tree-streamer.cc b/gcc/tree-streamer.cc
index ed65a7692e3..a28ef9c7920 100644
--- a/gcc/tree-streamer.cc
+++ b/gcc/tree-streamer.cc
@@ -35,7 +35,7 @@ along with GCC; see the file COPYING3.  If not see
    During streaming in, we translate the on the disk mode using this
    table.  For normal LTO it is set to identity, for ACCEL_COMPILER
    depending on the mode_table content.  */
-unsigned char streamer_mode_table[1 << 8];
+unsigned char streamer_mode_table[MAX_MACHINE_MODE];
 
 /* Check that all the TS_* structures handled by the streamer_write_* and
    streamer_read_* routines are exactly ALL the structures defined in
diff --git a/gcc/tree-streamer.h b/gcc/tree-streamer.h
index 170d61cf20b..51a292c8d80 100644
--- a/gcc/tree-streamer.h
+++ b/gcc/tree-streamer.h
@@ -75,7 +75,7 @@ void streamer_write_tree_body (struct output_block *, tree);
 void streamer_write_integer_cst (struct output_block *, tree);
 
 /* In tree-streamer.cc.  */
-extern unsigned char streamer_mode_table[1 << 8];
+extern unsigned char streamer_mode_table[MAX_MACHINE_MODE];
 void streamer_check_handled_ts_structures (void);
 bool streamer_tree_cache_insert (struct streamer_tree_cache_d *, tree,
 				 hashval_t, unsigned *);

Re: LTO: buffer overflow in lto_output_init_mode_table

[Robin Dapp]

Hi Jivan,

I think Pan is already on this problem.  Please see this thread:
https://gcc.gnu.org/pipermail/gcc-patches/2023-June/622129.html

Regards
 Robin

Re: LTO: buffer overflow in lto_output_init_mode_table

[Jivan Hakobyan]

[-- Attachment #1: Type: text/plain, Size: 373 bytes --]

Hi Robbin.

Thank you for responding.
I will defer my thread.


On Thu, Jun 22, 2023 at 3:42 PM Robin Dapp <rdapp.gcc@gmail.com> wrote:

> Hi Jivan,
>
> I think Pan is already on this problem.  Please see this thread:
> https://gcc.gnu.org/pipermail/gcc-patches/2023-June/622129.html
>
> Regards
>  Robin
>


-- 
With the best regards
Jivan Hakobyan