[Bug c/52635] New: gcc fails to diagnose invalid type in unused sizeof() when optimizing

[Bug c/52635] New: gcc fails to diagnose invalid type in unused sizeof() when optimizing

[mikpe at it dot uu.se]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52635

             Bug #: 52635
           Summary: gcc fails to diagnose invalid type in unused sizeof()
                    when optimizing
    Classification: Unclassified
           Product: gcc
           Version: 4.8.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: c
        AssignedTo: unassigned@gcc.gnu.org
        ReportedBy: mikpe@it.uu.se


Created attachment 26930
  --> http://gcc.gnu.org/bugzilla/attachment.cgi?id=26930
test case

> cat bug.c
extern int bar;
void foo(void)
{
    (void)sizeof(char[1 - 2 * !__builtin_constant_p(bar)]);
}
> objdir/gcc/xgcc -Bobjdir/gcc -O1 -S bug.c
> objdir/gcc/xgcc -Bobjdir/gcc -O0 -S bug.c
bug.c: In function 'foo':
bug.c:4:5: error: size of unnamed array is negative

The test case deliberately contains an invalid type expression, but gcc only
diagnoses it at -O0, at -O1 and above there is no error or warning.

Reproduced on x86_64-linux with gcc 4.8-20120318, 4.7.0-RC-20120314, every
4.x.latest release, 3.3.6, and 3.2.3.  gcc-3.4.6 fails to diagnose also at -O0.

The test case is based on the Linux kernel's BUILD_BUG_ON() macro.

[Bug c/52635] gcc fails to diagnose invalid type in unused sizeof() when optimizing

[jakub at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52635

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jakub at gcc dot gnu.org

--- Comment #1 from Jakub Jelinek <jakub at gcc dot gnu.org> 2012-03-20 14:31:16 UTC ---
I don't think this is a bug.  For -std=c89 -pedantic-errors you get a proper
error - C89 doesn't have VLAs, but otherwise when optimizing the type is
considered just as a VLA type.  You don't get any diagnostic even when you
actually use the sizeof (e.g. return it from the function), negative VLA size
doesn't have any diagnostic (most often it isn't even possible to diagnose it
at compile time).

[Bug c/52635] gcc fails to diagnose invalid type in unused sizeof() when optimizing

[joseph at codesourcery dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52635

--- Comment #2 from joseph at codesourcery dot com <joseph at codesourcery dot com> 2012-03-20 15:13:13 UTC ---
On Tue, 20 Mar 2012, mikpe at it dot uu.se wrote:

> > cat bug.c
> extern int bar;
> void foo(void)
> {
>     (void)sizeof(char[1 - 2 * !__builtin_constant_p(bar)]);
> }
> > objdir/gcc/xgcc -Bobjdir/gcc -O1 -S bug.c
> > objdir/gcc/xgcc -Bobjdir/gcc -O0 -S bug.c
> bug.c: In function 'foo':
> bug.c:4:5: error: size of unnamed array is negative
> 
> The test case deliberately contains an invalid type expression, but gcc only
> diagnoses it at -O0, at -O1 and above there is no error or warning.

Do you have an example not involving __builtin_constant_p?  When 
optimizing, this array will be considered a VLA (i.e. the size will be 
considered nonconstant) as __builtin_constant_p is only evaluated later 
(after inlining, for example), long after decisions about whether arrays 
are VLAs have to be made.

[Bug c/52635] gcc fails to diagnose invalid type in unused sizeof() when optimizing

[mikpe at it dot uu.se]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52635

--- Comment #3 from Mikael Pettersson <mikpe at it dot uu.se> 2012-03-20 17:26:23 UTC ---
(In reply to comment #2)
> Do you have an example not involving __builtin_constant_p?

Unfortunately no.  The example is just a cleaned up and reduced version of the
one from PR52632.  I've tried a few variations with __builtin_offsetof() and
sizeof(), but they all work, i.e., generate diagnostics at all optimization
levels.

>  When 
> optimizing, this array will be considered a VLA (i.e. the size will be 
> considered nonconstant) as __builtin_constant_p is only evaluated later 
> (after inlining, for example), long after decisions about whether arrays 
> are VLAs have to be made.

I see.  The Linux kernel also uses the following variation:

#define BUILD_BUG_ON_ZERO(e) (sizeof(struct { int:-!!(e); })

which seems to work with all of __builtin_constant_p, __builtin_offsetof,
sizeof, at both -O0 and -O2.

Perhaps I ought to submit a kernel patch to make BUILD_BUG_ON() use the struct
trick rather than the array size trick?

[Bug c/52635] gcc fails to diagnose invalid type in unused sizeof() when optimizing

[jakub at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52635

--- Comment #4 from Jakub Jelinek <jakub at gcc dot gnu.org> 2012-03-20 18:06:33 UTC ---
Not sure what kernel you are looking at, but in current kernel at least since
2009-09 BUILD_BUG_ON is BUILD_BUG_ON_ZERO and only MAYBE_BUILD_BUG_ON macro is
using this sizeof and from the description it sounds like it wants the warnings
only if the condition is known to be false already during parsing.

[Bug c/52635] gcc fails to diagnose invalid type in unused sizeof() when optimizing

[mikpe at it dot uu.se]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52635

--- Comment #5 from Mikael Pettersson <mikpe at it dot uu.se> 2012-03-20 19:09:01 UTC ---
(In reply to comment #4)
> Not sure what kernel you are looking at, but in current kernel at least since
> 2009-09 BUILD_BUG_ON is BUILD_BUG_ON_ZERO and only MAYBE_BUILD_BUG_ON macro is
> using this sizeof and from the description it sounds like it wants the warnings
> only if the condition is known to be false already during parsing.

I'm looking at the 3.3 kernel released only a few days ago.  It's kernel.h
doesn't define BUILD_BUG_ON in terms of BUILD_BUG_ON_ZERO (which perhaps it
should be) and there is no MAYBE_BUILD_BUG_ON anywhere.