[Bug c/68065] Size calculations for VLAs can overflow

["ebotcazou at gcc dot gnu.org" <gcc-bugzilla@gcc.gnu.org>]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=68065

Eric Botcazou <ebotcazou at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |ebotcazou at gcc dot gnu.org

--- Comment #12 from Eric Botcazou <ebotcazou at gcc dot gnu.org> ---
> Stack overflows are detected with -fstack-check, or at least they would be
> if the option worked properly:
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66479
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65958

Yes, it works, i.e. it detects stack overflows in real life.  The first PR is
certainly annoying but largely artificial and the second PR is actually a
generic bug in the gimplifier with VLAs and alloca that the old implementation
happens to run into; the modern one doesn't.

> I've always found it quite bad that well-defined code with GCC can actually
> be exploited (arbitrary write vulnerabilities) due to the fact that
> -fstack-check is not enabled by default. MSVC++ and Clang on Windows
> guarantee that stack overflows from well-defined code (large stack frames,
> VLAs) will be caught.

Same for GCC on Windows (but it does out-of-line stack checking).

> However, the switch seems to cause a significant performance hit for functions 
> where it triggers (which are rare but sometimes performance critical, a good 
> example is jemalloc's rbtree implementation which uses arrays rather than 
> recursion) and compatibility issues due to the way it's currently implemented:
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67265/.

This one is more of a register allocation issue actually.