[Bug c/109956] New: GCC reserves 9 bytes for struct s { int a; char b; char t[]; } x = {1, 2, 3};

[Bug c/109956] New: GCC reserves 9 bytes for struct s { int a; char b; char t[]; } x = {1, 2, 3};

[pascal_cuoq at hotmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109956

            Bug ID: 109956
           Summary: GCC reserves 9 bytes for struct s { int a; char b;
                    char t[]; } x = {1, 2, 3};
           Product: gcc
           Version: 14.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: c
          Assignee: unassigned at gcc dot gnu.org
          Reporter: pascal_cuoq at hotmail dot com
  Target Milestone: ---

Static-lifetime variables of type “struct with FAM” (flexible array member)
with an initializer for the FAM are a GCC extension.

As of GCC 13.1 and Compiler Explorer “trunk”, targeting x86, the definition
“struct s { int a; char b; char t[]; } x = {1, 2, 3};” reserves 9 bytes for x,
and in fact, with various initializers, the trailing padding for variables of
type “struct s” is always 3, as if the size to reserve for the variable was
computed as “sizeof (struct s) + n * sizeof(element)”.

Input file:
struct s { int a; char b; char t[]; } x = {1, 2, 3};

Command:
gcc -S fam_init.c

Result (with Ubuntu 9.4.0-1ubuntu1~20.04.1 which exhibits the same behavior as
the recent versions on Compiler Explorer):
        .align 8
        .type   x, @object
        .size   x, 9
x:
        .long   1
        .byte   2
        .byte   3
        .zero   3


Clang up to version 14 used to round up the size of the variable to a multiple
of the alignment of the struct, but even this is not necessary. It is only
necessary that the size reserved for a variable of type t is at least
“sizeof(t)” bytes, and also to reserve enough space for the initializer. Clang
15 and later uses the optimal formula:

max(sizeof (struct s), offsetof(struct s, t[n]))

Compiler Explorer link: https://gcc.godbolt.org/z/5W7h4KWT1

This ticket is to suggest that GCC uses the same optimal formula as Clang 15
and later.

[Bug c/109956] GCC reserves 9 bytes for struct s { int a; char b; char t[]; } x = {1, 2, 3};

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109956

Andrew Pinski <pinskia at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Severity|normal                      |trivial

--- Comment #1 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
Considering this is an extension, I think GCC is still correct.

[Bug c/109956] GCC reserves 9 bytes for struct s { int a; char b; char t[]; } x = {1, 2, 3};

[muecker at gwdg dot de]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109956

Martin Uecker <muecker at gwdg dot de> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |muecker at gwdg dot de

--- Comment #2 from Martin Uecker <muecker at gwdg dot de> ---
To me it seems that the C standard requires that the object has 
size sizeof(struct s) + n * sizeof(struct t) if you want to store n elements
even when the array then starts at a smaller offset.

[Bug c/109956] GCC reserves 9 bytes for struct s { int a; char b; char t[]; } x = {1, 2, 3};

[pascal_cuoq at hotmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109956

--- Comment #3 from Pascal Cuoq <pascal_cuoq at hotmail dot com> ---
@Andrew Pinski

You don't even need to invoke the fact that this is an extension. GCC could
reserve 17 bytes for each variable i of type “int”, and as long as “sizeof i”
continued to evaluate to 4 (4 being the value of “sizeof(int)” for x86), no-one
would be able to claim that GCC is not generating “correct” assembly code.

This ticket is pointing out that the current behavior for initialized FAMs is
suboptimal for programs that rely on the GCC extension, just like it would be
suboptimal to reserve 17 bytes for each “int” variable for standard C programs
(and I would open a ticket for it if I noticed such a behavior).

It's not breaking anything and it may be inconvenient to change, and as a
ticket that does not affect correctness, it can be ignored indefinitely. It's
just a suggestion for smaller binaries that might also end up marginally faster
as a result.

@Martin Uecker

Considering how casually phrased the description of FAMs was in C99 and
remained in later standards (see https://stackoverflow.com/q/73497572/139746
for me trying to make sense of some of the relevant words), I doubt that the
standard has anything to say about the compiler extension being discussed. But
if you have convincing arguments, you could spend a few minutes filing a bug
against Clang to tell them that they are making the binaries they generate too
small and efficient.

[Bug c/109956] GCC reserves 9 bytes for struct s { int a; char b; char t[]; } x = {1, 2, 3};

[muecker at gwdg dot de]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109956

--- Comment #4 from Martin Uecker <muecker at gwdg dot de> ---

The concern would be that a program relying on the size of an object being
larger may then have out of bounds accesses.  But rereading the standard, I am
also not not seeing that this is required. (for the extension nothing is
required anyway, but it should be consistent with it).

[Bug c/109956] GCC reserves 9 bytes for struct s { int a; char b; char t[]; } x = {1, 2, 3};

[muecker at gwdg dot de]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109956

--- Comment #5 from Martin Uecker <muecker at gwdg dot de> ---
Clang bug:
https://github.com/llvm/llvm-project/issues/62929

[Bug c/109956] GCC reserves 9 bytes for struct s { int a; char b; char t[]; } x = {1, 2, 3};

[joseph at codesourcery dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109956

--- Comment #6 from joseph at codesourcery dot com <joseph at codesourcery dot com> ---
For the standard, dynamically allocated case, you should only need to 
allocate enough memory to contain the initial part of the struct and the 
array members being accessed - not any padding after that array.  (There 
were wording problems before C99 TC2; see DR#282.)

[Bug c/109956] GCC reserves 9 bytes for struct s { int a; char b; char t[]; } x = {1, 2, 3};

[joseph at codesourcery dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109956

--- Comment #7 from joseph at codesourcery dot com <joseph at codesourcery dot com> ---
I suppose the question is how to interpret "the longest array (with the 
same element type) that would not make the structure larger than the 
object being accessed".  The difficulty of interpreting "make the 
structure larger" in terms of including post-array padding in the 
replacement structure is that there might not be a definition of what that 
post-array padding should be given the offset of the array need not be the 
same as the offset with literal replacement in the struct definition.

[Bug c/109956] GCC reserves 9 bytes for struct s { int a; char b; char t[]; } x = {1, 2, 3};

[amonakov at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109956

Alexander Monakov <amonakov at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |amonakov at gcc dot gnu.org

--- Comment #8 from Alexander Monakov <amonakov at gcc dot gnu.org> ---
(In reply to joseph@codesourcery.com from comment #6)
> For the standard, dynamically allocated case, you should only need to 
> allocate enough memory to contain the initial part of the struct and the 
> array members being accessed - not any padding after that array.  (There 
> were wording problems before C99 TC2; see DR#282.)

I think the following testcase indicates that GCC assumes that tail padding is
accessible:

struct S {
        int i;
        char c;
        char fam[];
};

void f(struct S *p, struct S *q)
{
        *p = *q;
}

f:
        movq    (%rsi), %rax
        movq    %rax, (%rdi)
        ret

Sorry for the tangential remark, but there seems to be a contradiction.

[Bug c/109956] GCC reserves 9 bytes for struct s { int a; char b; char t[]; } x = {1, 2, 3};

[muecker at gwdg dot de]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109956

--- Comment #9 from Martin Uecker <muecker at gwdg dot de> ---
Clang as well, but that would be only padding inside the first part without
taking into account extra element in the FAM. 

I am more concert about programmers using the formula sizeof(.) + n * sizeof
for memcpy etc.  (and we have an example in the standard using this formula).
Creating objects smaller than this seems a bit dangerous.

[Bug c/109956] GCC reserves 9 bytes for struct s { int a; char b; char t[]; } x = {1, 2, 3};

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109956

--- Comment #10 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
(In reply to Alexander Monakov from comment #8)
> I think the following testcase indicates that GCC assumes that tail padding
> is accessible: 

Well it aligned accesses are always accessable ....
the alignment of `struct S` in this case is 4 byte aligned after all.

[Bug c/109956] GCC reserves 9 bytes for struct s { int a; char b; char t[]; } x = {1, 2, 3};

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109956

--- Comment #11 from Richard Biener <rguenth at gcc dot gnu.org> ---
(In reply to Alexander Monakov from comment #8)
> (In reply to joseph@codesourcery.com from comment #6)
> > For the standard, dynamically allocated case, you should only need to 
> > allocate enough memory to contain the initial part of the struct and the 
> > array members being accessed - not any padding after that array.  (There 
> > were wording problems before C99 TC2; see DR#282.)
> 
> I think the following testcase indicates that GCC assumes that tail padding
> is accessible:
> 
> struct S {
> 	int i;
> 	char c;
> 	char fam[];
> };
> 
> void f(struct S *p, struct S *q)
> {
> 	*p = *q;
> }
> 
> f:
>         movq    (%rsi), %rax
>         movq    %rax, (%rdi)
>         ret
> 
> Sorry for the tangential remark, but there seems to be a contradiction.

Not only accessible but also not used by sth else.  That is, a following
'char' variable may not be placed into the padding.

[Bug c/109956] GCC reserves 9 bytes for struct s { int a; char b; char t[]; } x = {1, 2, 3};

[muecker at gwdg dot de]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109956

--- Comment #12 from Martin Uecker <muecker at gwdg dot de> ---

The C standard says "However, when a . (or -> ) operator has a left operand
that is (a pointer to) a structure with a flexible array member and the right
operand names that member, it behaves as if that member were replaced with the
longest array (with the same element type) that would not make the structure
larger than the object being accessed;" 

This would imply that also for GCC not all elements can be accessed in the
following structure without invoking UB. For x86_64 'x' has 11 bytes (for clang
9) but a struct with replacement array actually needs 12.

struct foo { int a; short b; char t[]; } x = { .t = { 1, 2, 3 } }; 
// x has 11 bytes

struct bar { int a; short b; char t[3]; }; // 12 bytes


One can argue that this does not matter for an extension and maybe the C
standard should not be read in this way as itself also contains an example
using the sizeof() + n * sizeof() rule which implies that this amount of
storage is enough for n elements.

But the question is what programmers should use when using memcpy of statically
initialized structs which have a FAM?   sizeof() + n * sizeof() works for GCC
but not for clang.  sizeof(struct bar) works for neither.  One can use
MAX(sizeof(struct foo, offsetof(struct foo, t) + n * sizeof(char)) but I have
not seen anybody use it.

[Bug c/109956] GCC reserves 9 bytes for struct s { int a; char b; char t[]; } x = {1, 2, 3};

[pascal_cuoq at hotmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109956

--- Comment #13 from Pascal Cuoq <pascal_cuoq at hotmail dot com> ---
@Martin

I completely agree with comment 12, however about the last paragraph, I would
like to point out that for purposes of memcpy'ing to or from such a struct with
initialized FAM, it is enough to recommend that programmers use the simple
formula “offsetof(struct foo, t) + n * sizeof(char)” (or “offsetof(struct foo,
t[n])”. The part that is not copied is the part that they did not intend to use
when they chose the initializer of the FAM, and that they cannot portably use
because of the padding that may or may not exist for a different target
architecture.

So since:

First, GCC currently does not always reserve enough room to allow “memcpy(…, …,
sizeof(struct foo) + n * sizeof(char))”, and 

second, using the time-proven formula as argument of malloc technically does
not always allocate enough space to make it valid to access p->t[n-1] according
to the strict interpretation of the words “it behaves as if that member were
replaced with the longest array (with the same element type) that would not
make the structure larger than the object being accessed”,

we might as well start recommending that C programmers use “offsetof(struct
foo, t) + n * sizeof(char)” as argument of memcpy, and either clarify the
meaning of the words “it behaves as if…” in the C standard or prepare for a
very unpleasant discussion when we have to tell them the formula they have to
use as argument of malloc.

[Bug c/109956] GCC reserves 9 bytes for struct s { int a; char b; char t[]; } x = {1, 2, 3};

[muecker at gwdg dot de]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109956

--- Comment #14 from Martin Uecker <muecker at gwdg dot de> ---


Maybe. 

On the other hand, I wonder whether a struct with FAM should not rather always
have the same size, and alignment, and representation as the corresponding
struct with a conventional array. This would conceptually be cleaner, easier to
understand, and less error prone.

[Bug c/109956] GCC reserves 9 bytes for struct s { int a; char b; char t[]; } x = {1, 2, 3};

[muecker at gwdg dot de]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109956

--- Comment #15 from Martin Uecker <muecker at gwdg dot de> ---
GCC seems to allocate enough for sizeof(struct foo) + n * sizeof(char) but not
for sizeof(struct { int a; char b; char t[n]; }).

[Bug c/109956] GCC reserves 9 bytes for struct s { int a; char b; char t[]; } x = {1, 2, 3};

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109956

Andrew Pinski <pinskia at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |DUPLICATE
             Status|UNCONFIRMED                 |RESOLVED

--- Comment #16 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
Dup.

*** This bug has been marked as a duplicate of bug 91672 ***