[wish] Flexible array members in unions

[wish] Flexible array members in unions

[Alejandro Colomar]

[-- Attachment #1.1: Type: text/plain, Size: 2869 bytes --]

Hi!

Currently, one can have pseudo-flexible array members in unions with
[0] syntax, but it's not allowed with [] syntax.

Here's an example of how it is possible today:

struct s {
	...

	size_t  n;
	union {
		ptrdiff_t  off[0];  // [n]; offsets from s->data.
		char       data[0];
	};
};

which is useful to have a structure with two (or really several)
consecutive flexible arrays: one of offsets, which mark the positions
of data, and another with the actual data.  Below goes an example
program, which works fine with GCC, and I believe rewriting it to
not use the union would make it less clear, since I'd need to add
casts to it.

It works thanks to [0] pseudo-flexible arrays, but it doesn't
compile with C99 flexible arrays.  And of course, [0] arrays have
issues with -fstrict-flex-arrays=3.


$ cat flexi4.c 
#include <stddef.h>
#include <stdlib.h>
#include <stdio.h>
#include <string.h>

struct s {
	size_t     n;
	union {
		ptrdiff_t  off[0];
		char       data[0];
	};
};

int
main(void)
{
	char      *p;
	struct s  *s;

	s = malloc(offsetof(struct s, off) +
	           sizeof(ptrdiff_t) * 2 +
	           sizeof("foobar") + sizeof("baz"));

	s->n = 2;
	p = s->data + sizeof(ptrdiff_t) * s->n;

	s->off[0] = p - s->data;
	p = stpcpy(p, "foobar") + 1;
	s->off[1] = p - s->data;
	p = stpcpy(p, "baz") + 1;

	puts(s->data + s->off[0]);
	puts(s->data + s->off[1]);

	free(s);
}
$ gcc-13 -Wall -Wextra -Werror -fanalyzer \
         -fsanitize=undefined -fsanitize=address \
         -D_FORTIFY_SOURCE=3 -fstrict-flex-arrays=2 \
         flexi4.c 
$ ./a.out 
foobar
baz
$ gcc-13 -Wall -Wextra -Werror -fanalyzer \
         -fsanitize=undefined -fsanitize=address \
         -D_FORTIFY_SOURCE=3 -fstrict-flex-arrays=3 \
         flexi4.c 
$ ./a.out 
flexi4.c:27:8: runtime error: index 0 out of bounds for type 'ptrdiff_t [*]'
flexi4.c:29:8: runtime error: index 1 out of bounds for type 'ptrdiff_t [*]'
flexi4.c:32:23: runtime error: index 0 out of bounds for type 'ptrdiff_t [*]'
foobar
flexi4.c:33:23: runtime error: index 1 out of bounds for type 'ptrdiff_t [*]'
baz


Would you allow flexible array members in unions?  Is there any
strong reason to disallow them?

Currently, I get:

$ gcc-13 -Wall -Wextra -fanalyzer \
         -fsanitize=undefined -fsanitize=address \
         -D_FORTIFY_SOURCE=3 -fstrict-flex-arrays=3 \
         flexi4-true.c 
flexi4-true.c:9:28: error: flexible array member in union
    9 |                 ptrdiff_t  off[];
      |                            ^~~
flexi4-true.c:10:28: error: flexible array member in union
   10 |                 char       data[];
      |                            ^~~~


Cheers,
Alex

-- 
<http://www.alejandro-colomar.es/>
GPG key fingerprint: A9348594CE31283A826FBDD8D57633D441E25BB5

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [wish] Flexible array members in unions

[Alejandro Colomar]

[-- Attachment #1.1: Type: text/plain, Size: 4271 bytes --]

[CC += Kees, Andrew]

[start of thread: <https://inbox.sourceware.org/gcc/ac2550e0-2f5c-3680-08e6-0c224d043036@gmail.com/T/#u>]

On 5/11/23 18:07, Alejandro Colomar wrote:
> Hi!
> 
> Currently, one can have pseudo-flexible array members in unions with
> [0] syntax, but it's not allowed with [] syntax.
> 
> Here's an example of how it is possible today:
> 
> struct s {
> 	...
> 
> 	size_t  n;
> 	union {
> 		ptrdiff_t  off[0];  // [n]; offsets from s->data.
> 		char       data[0];
> 	};
> };
> 
> which is useful to have a structure with two (or really several)
> consecutive flexible arrays: one of offsets, which mark the positions
> of data, and another with the actual data.  Below goes an example
> program, which works fine with GCC, and I believe rewriting it to
> not use the union would make it less clear, since I'd need to add
> casts to it.
> 
> It works thanks to [0] pseudo-flexible arrays, but it doesn't
> compile with C99 flexible arrays.  And of course, [0] arrays have
> issues with -fstrict-flex-arrays=3.
> 
> 
> $ cat flexi4.c 
> #include <stddef.h>
> #include <stdlib.h>
> #include <stdio.h>
> #include <string.h>
> 
> struct s {
> 	size_t     n;
> 	union {
> 		ptrdiff_t  off[0];
> 		char       data[0];
> 	};
> };
> 
> int
> main(void)
> {
> 	char      *p;
> 	struct s  *s;
> 
> 	s = malloc(offsetof(struct s, off) +
> 	           sizeof(ptrdiff_t) * 2 +
> 	           sizeof("foobar") + sizeof("baz"));
> 
> 	s->n = 2;
> 	p = s->data + sizeof(ptrdiff_t) * s->n;
> 
> 	s->off[0] = p - s->data;
> 	p = stpcpy(p, "foobar") + 1;
> 	s->off[1] = p - s->data;
> 	p = stpcpy(p, "baz") + 1;
> 
> 	puts(s->data + s->off[0]);
> 	puts(s->data + s->off[1]);
> 
> 	free(s);
> }
> $ gcc-13 -Wall -Wextra -Werror -fanalyzer \
>          -fsanitize=undefined -fsanitize=address \
>          -D_FORTIFY_SOURCE=3 -fstrict-flex-arrays=2 \
>          flexi4.c 
> $ ./a.out 
> foobar
> baz
> $ gcc-13 -Wall -Wextra -Werror -fanalyzer \
>          -fsanitize=undefined -fsanitize=address \
>          -D_FORTIFY_SOURCE=3 -fstrict-flex-arrays=3 \
>          flexi4.c 
> $ ./a.out 
> flexi4.c:27:8: runtime error: index 0 out of bounds for type 'ptrdiff_t [*]'
> flexi4.c:29:8: runtime error: index 1 out of bounds for type 'ptrdiff_t [*]'
> flexi4.c:32:23: runtime error: index 0 out of bounds for type 'ptrdiff_t [*]'
> foobar
> flexi4.c:33:23: runtime error: index 1 out of bounds for type 'ptrdiff_t [*]'
> baz
> 
> 
> Would you allow flexible array members in unions?  Is there any
> strong reason to disallow them?
> 
> Currently, I get:
> 
> $ gcc-13 -Wall -Wextra -fanalyzer \
>          -fsanitize=undefined -fsanitize=address \
>          -D_FORTIFY_SOURCE=3 -fstrict-flex-arrays=3 \
>          flexi4-true.c 
> flexi4-true.c:9:28: error: flexible array member in union
>     9 |                 ptrdiff_t  off[];
>       |                            ^~~
> flexi4-true.c:10:28: error: flexible array member in union
>    10 |                 char       data[];
>       |                            ^~~~
> 
> 

Currently, the Linux kernel has to go through some hoops due to this
restriction:


$ grepc -tm __DECLARE_FLEX_ARRAY *
include/uapi/linux/stddef.h:42:
#define __DECLARE_FLEX_ARRAY(TYPE, NAME)	\
	struct { \
		struct { } __empty_ ## NAME; \
		TYPE NAME[]; \
	}


tools/include/uapi/linux/stddef.h:42:
#define __DECLARE_FLEX_ARRAY(TYPE, NAME)	\
	struct { \
		struct { } __empty_ ## NAME; \
		TYPE NAME[]; \
	}
$ grep -rnB2 __DECLARE_FLEX_ARRAY * | head
include/uapi/rdma/rdma_user_rxe.h-153-	__u32			reserved;
include/uapi/rdma/rdma_user_rxe.h-154-	union {
include/uapi/rdma/rdma_user_rxe.h:155:		__DECLARE_FLEX_ARRAY(__u8, inline_data);
include/uapi/rdma/rdma_user_rxe.h:156:		__DECLARE_FLEX_ARRAY(__u8, atomic_wr);
include/uapi/rdma/rdma_user_rxe.h:157:		__DECLARE_FLEX_ARRAY(struct rxe_sge, sge);
--
include/uapi/scsi/scsi_netlink_fc.h-53-	union {
include/uapi/scsi/scsi_netlink_fc.h-54-		__u32 event_data;
include/uapi/scsi/scsi_netlink_fc.h:55:		__DECLARE_FLEX_ARRAY(__u8, event_data_flex);
--




-- 
<http://www.alejandro-colomar.es/>
GPG key fingerprint: A9348594CE31283A826FBDD8D57633D441E25BB5

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [wish] Flexible array members in unions

[Kees Cook]

On Thu, May 11, 2023 at 06:29:10PM +0200, Alejandro Colomar wrote:
> On 5/11/23 18:07, Alejandro Colomar wrote:
> [...]
> > Would you allow flexible array members in unions?  Is there any
> > strong reason to disallow them?

Yes please!! And alone in a struct, too.

AFAICT, there is no mechanical/architectural reason to disallow them
(especially since they _can_ be constructed with some fancy tricks,
and they behave as expected.) My understanding is that it's disallowed
due to an overly strict reading of the very terse language that created
flexible arrays in C99.

> [...]
> Currently, the Linux kernel has to go through some hoops due to this
> restriction:
> 
> 
> $ grepc -tm __DECLARE_FLEX_ARRAY *
> include/uapi/linux/stddef.h:42:
> #define __DECLARE_FLEX_ARRAY(TYPE, NAME)	\
> 	struct { \
> 		struct { } __empty_ ## NAME; \
> 		TYPE NAME[]; \
> 	}

Yes, we've had to do this as we eradicate all the fake flexible arrays
in the kernel which cause endless bugs[1]. Additionally, we'll be
using -fstrict-flex-arrays=3 soon to let existing array bounds checking
mitigations gain coverage over trailing arrays. All of this means that
we're converting a lot of code that is happily using dynamically sized
arrays in unions, etc.

[1] https://people.kernel.org/kees/bounded-flexible-arrays-in-c

-- 
Kees Cook

Re: [wish] Flexible array members in unions

[Joseph Myers]

On Thu, 11 May 2023, Kees Cook via Gcc wrote:

> On Thu, May 11, 2023 at 06:29:10PM +0200, Alejandro Colomar wrote:
> > On 5/11/23 18:07, Alejandro Colomar wrote:
> > [...]
> > > Would you allow flexible array members in unions?  Is there any
> > > strong reason to disallow them?
> 
> Yes please!! And alone in a struct, too.
> 
> AFAICT, there is no mechanical/architectural reason to disallow them
> (especially since they _can_ be constructed with some fancy tricks,
> and they behave as expected.) My understanding is that it's disallowed
> due to an overly strict reading of the very terse language that created
> flexible arrays in C99.

Standard C has no such thing as a zero-size object or type, which would 
lead to problems with a struct or union that only contains a flexible 
array member there.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: [wish] Flexible array members in unions

[Kees Cook]

On Thu, May 11, 2023 at 08:53:52PM +0000, Joseph Myers wrote:
> On Thu, 11 May 2023, Kees Cook via Gcc wrote:
> 
> > On Thu, May 11, 2023 at 06:29:10PM +0200, Alejandro Colomar wrote:
> > > On 5/11/23 18:07, Alejandro Colomar wrote:
> > > [...]
> > > > Would you allow flexible array members in unions?  Is there any
> > > > strong reason to disallow them?
> > 
> > Yes please!! And alone in a struct, too.
> > 
> > AFAICT, there is no mechanical/architectural reason to disallow them
> > (especially since they _can_ be constructed with some fancy tricks,
> > and they behave as expected.) My understanding is that it's disallowed
> > due to an overly strict reading of the very terse language that created
> > flexible arrays in C99.
> 
> Standard C has no such thing as a zero-size object or type, which would 
> lead to problems with a struct or union that only contains a flexible 
> array member there.

Ah-ha, okay. That root cause makes sense now.

Why are zero-sized objects missing in Standard C? Or, perhaps, the better
question is: what's needed to support the idea of a zero-sized object?

-- 
Kees Cook

Re: [wish] Flexible array members in unions

[Joseph Myers]

On Thu, 11 May 2023, Kees Cook via Gcc wrote:

> Why are zero-sized objects missing in Standard C? Or, perhaps, the better
> question is: what's needed to support the idea of a zero-sized object?

Zero-sized objects break the principle that different objects have 
different addresses, and the principle of being able to subtract pointers 
to different elements of an array.  There would also be serious C++ 
compatibility concerns, since C++ allows a struct with no members but it 
has nonzero size, unlike the GNU C extension where a struct with no 
members has size zero.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: [wish] Flexible array members in unions

[Kees Cook]

On Thu, May 11, 2023 at 09:43:49PM +0000, Joseph Myers wrote:
> On Thu, 11 May 2023, Kees Cook via Gcc wrote:
> 
> > Why are zero-sized objects missing in Standard C? Or, perhaps, the better
> > question is: what's needed to support the idea of a zero-sized object?
> 
> Zero-sized objects break the principle that different objects have 
> different addresses, and the principle of being able to subtract pointers 
> to different elements of an array.  There would also be serious C++ 
> compatibility concerns, since C++ allows a struct with no members but it 
> has nonzero size, unlike the GNU C extension where a struct with no 
> members has size zero.

Okay, understood. If this is a C-only thing, we can ignore the C++
impact. What depends on the "different objects have different addresses"
principle? And why do unions not break this -- they could point to the
same locations within the object? And don't flexible arrays already need
special handling in this regard?

-- 
Kees Cook

Re: [wish] Flexible array members in unions

[Joseph Myers]

On Thu, 11 May 2023, Kees Cook via Gcc wrote:

> Okay, understood. If this is a C-only thing, we can ignore the C++
> impact.

We're a lot more careful lately in WG14 about checking for C++ 
compatibility issues and expecting approval from the liaison group for 
anything with possible compatibility concerns for syntax in the common 
subset of C and C++.  So, no, we can't ignore the C++ impact for adding 
empty types; it would need careful consideration in the liaison group.

> What depends on the "different objects have different addresses"
> principle? And why do unions not break this -- they could point to the
> same locations within the object? And don't flexible arrays already need
> special handling in this regard?

"including a pointer to an object and a subobject at its beginning" and 
"one is a pointer to one past the end of one array object and the other is 
a pointer to the start of a different array object that happens to 
immediately follow the first array object in the address space" are both 
cases included in the semantics for comparison operators.  If you allow 
zero-size objects you get more special cases there (and quite possibly 
affect optimizations based on points-to analysis that can determine 
pointers are based on different objects, if an object is not known at 
compile time to have nonzero size).

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: [wish] Flexible array members in unions

[Alejandro Colomar]

[-- Attachment #1.1: Type: text/plain, Size: 2964 bytes --]

Hi Joseph, Kees,

On 5/12/23 00:52, Joseph Myers wrote:
> On Thu, 11 May 2023, Kees Cook via Gcc wrote:
> 
>> Okay, understood. If this is a C-only thing, we can ignore the C++
>> impact.
> 
> We're a lot more careful lately in WG14 about checking for C++ 
> compatibility issues and expecting approval from the liaison group for 
> anything with possible compatibility concerns for syntax in the common 
> subset of C and C++.  So, no, we can't ignore the C++ impact for adding 
> empty types; it would need careful consideration in the liaison group.
> 
>> What depends on the "different objects have different addresses"
>> principle? And why do unions not break this -- they could point to the
>> same locations within the object? And don't flexible arrays already need
>> special handling in this regard?
> 
> "including a pointer to an object and a subobject at its beginning" and 
> "one is a pointer to one past the end of one array object and the other is 
> a pointer to the start of a different array object that happens to 
> immediately follow the first array object in the address space" are both 
> cases included in the semantics for comparison operators.  If you allow 
> zero-size objects you get more special cases there (and quite possibly 
> affect optimizations based on points-to analysis that can determine 
> pointers are based on different objects, if an object is not known at 
> compile time to have nonzero size).

Since GNU C already supports empty structs, how about allowing that in GCC
with no intention of adding it to ISO C?  We'll see how good it behaves in
GCC, and if so suggest it for inclusion in the standard.

Why should GNU C, which allows empty structures, and de facto supports
flexible arrays in empty structs and in unions (via the empty preceeding
struct and the wrapper struct tricks, as the kernel does), shouldn't
support them officially without tricks?

Apart from violating artificial rules that disallow that use of flexible
arrays, I believe the example program I provided in the first post
doesn't violate aliasing rules or other similar rules that would result
in optimization conflicts.  Does it?

About zero-sized types, a union consisting of only flexible-array members
would effectively be a zero-sized type, so GCC should have similar issues
having this in unions and in empty structs.  So far, I don't see GCC
complaining about such horrible thing as an array of empty structs:

$ cat arr.c
#include <stdio.h>

struct s {};

struct s x[10];

int
main(void)
{
	printf("x:    %zu, %p\n", sizeof(x), &x);
	printf("x[3]: %zu, %p\n", sizeof(x[3]), &x[3]);
}
$ gcc-13 arr.c -Wall -Wextra
$ ./a.out 
x:    0, 0x55c5f6d72019
x[3]: 0, 0x55c5f6d72019



So, in GNU C land, I think it is reasonable to add this feature.

Cheers,
Alex

-- 
<http://www.alejandro-colomar.es/>
GPG key fingerprint: A9348594CE31283A826FBDD8D57633D441E25BB5

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [wish] Flexible array members in unions

[Richard Biener]

On Thu, May 11, 2023 at 11:14 PM Kees Cook via Gcc <gcc@gcc.gnu.org> wrote:
>
> On Thu, May 11, 2023 at 08:53:52PM +0000, Joseph Myers wrote:
> > On Thu, 11 May 2023, Kees Cook via Gcc wrote:
> >
> > > On Thu, May 11, 2023 at 06:29:10PM +0200, Alejandro Colomar wrote:
> > > > On 5/11/23 18:07, Alejandro Colomar wrote:
> > > > [...]
> > > > > Would you allow flexible array members in unions?  Is there any
> > > > > strong reason to disallow them?
> > >
> > > Yes please!! And alone in a struct, too.
> > >
> > > AFAICT, there is no mechanical/architectural reason to disallow them
> > > (especially since they _can_ be constructed with some fancy tricks,
> > > and they behave as expected.) My understanding is that it's disallowed
> > > due to an overly strict reading of the very terse language that created
> > > flexible arrays in C99.
> >
> > Standard C has no such thing as a zero-size object or type, which would
> > lead to problems with a struct or union that only contains a flexible
> > array member there.
>
> Ah-ha, okay. That root cause makes sense now.

Hmm. but then the workaround

struct X {
  int n;
  union u {
      char at_least_size_one;
      int iarr[];
      short sarr[];
  };
};

doesn't work either.  We could make that a GNU extension without
adverse effects?

Richard.

> Why are zero-sized objects missing in Standard C? Or, perhaps, the better
> question is: what's needed to support the idea of a zero-sized object?
>
> --
> Kees Cook

Re: [wish] Flexible array members in unions

[Jonathan Wakely]

[-- Attachment #1: Type: text/plain, Size: 1214 bytes --]

On Thu, 11 May 2023, 23:17 Kees Cook via Gcc, <gcc@gcc.gnu.org> wrote:

> On Thu, May 11, 2023 at 09:43:49PM +0000, Joseph Myers wrote:
> > On Thu, 11 May 2023, Kees Cook via Gcc wrote:
> >
> > > Why are zero-sized objects missing in Standard C? Or, perhaps, the
> better
> > > question is: what's needed to support the idea of a zero-sized object?
> >
> > Zero-sized objects break the principle that different objects have
> > different addresses, and the principle of being able to subtract
> pointers
> > to different elements of an array.  There would also be serious C++
> > compatibility concerns, since C++ allows a struct with no members but it
> > has nonzero size, unlike the GNU C extension where a struct with no
> > members has size zero.
>
> Okay, understood. If this is a C-only thing, we can ignore the C++
> impact. What depends on the "different objects have different addresses"
> principle? And why do unions not break this -- they could point to the
> same locations within the object?


You don't have two different objects with the same address in a union,
because only one of them "exists" at any time. You're reusing the storage
for one object at a time, not for all of the union members.

Re: [wish] Flexible array members in unions

[David Brown]

On 12/05/2023 08:16, Richard Biener via Gcc wrote:
> On Thu, May 11, 2023 at 11:14 PM Kees Cook via Gcc <gcc@gcc.gnu.org> wrote:
>>
>> On Thu, May 11, 2023 at 08:53:52PM +0000, Joseph Myers wrote:
>>> On Thu, 11 May 2023, Kees Cook via Gcc wrote:
>>>
>>>> On Thu, May 11, 2023 at 06:29:10PM +0200, Alejandro Colomar wrote:
>>>>> On 5/11/23 18:07, Alejandro Colomar wrote:
>>>>> [...]
>>>>>> Would you allow flexible array members in unions?  Is there any
>>>>>> strong reason to disallow them?
>>>>
>>>> Yes please!! And alone in a struct, too.
>>>>
>>>> AFAICT, there is no mechanical/architectural reason to disallow them
>>>> (especially since they _can_ be constructed with some fancy tricks,
>>>> and they behave as expected.) My understanding is that it's disallowed
>>>> due to an overly strict reading of the very terse language that created
>>>> flexible arrays in C99.
>>>
>>> Standard C has no such thing as a zero-size object or type, which would
>>> lead to problems with a struct or union that only contains a flexible
>>> array member there.
>>
>> Ah-ha, okay. That root cause makes sense now.
> 
> Hmm. but then the workaround
> 
> struct X {
>    int n;
>    union u {
>        char at_least_size_one;
>        int iarr[];
>        short sarr[];
>    };
> };
> 
> doesn't work either.  We could make that a GNU extension without
> adverse effects?
> 
> Richard.
> 

I would like and use an extension like that (for C and C++) - the 
flexible arrays would act as though they were the same size as the 
size-specific part of the union, rounding up in this case to make the 
alignments correct.

I regularly want something like :

	union ProtocolBuffer {
		struct {
			header ...
			data fields ...
		}
		uint8_t raw8[];
		uint32_t raw32[];
	}

The "raw" arrays would be used to move data around, or access it from 
communication drivers.  As C (and C++) is defined, I have to split this 
up so that the "raw" arrays can use "sizeof(ProtocolTelegram) / 4" or 
similar expressions for their size.  If flexible arrays in unions were 
allowed here, it could make my code a little neater and use more 
anonymous unions and structs to reduce unhelpful verbosity.




>> Why are zero-sized objects missing in Standard C? Or, perhaps, the better
>> question is: what's needed to support the idea of a zero-sized object?
>>
>> --
>> Kees Cook
> 

Re: [wish] Flexible array members in unions

[Qing Zhao]

> On May 12, 2023, at 2:16 AM, Richard Biener via Gcc <gcc@gcc.gnu.org> wrote:
> 
> On Thu, May 11, 2023 at 11:14 PM Kees Cook via Gcc <gcc@gcc.gnu.org> wrote:
>> 
>> On Thu, May 11, 2023 at 08:53:52PM +0000, Joseph Myers wrote:
>>> On Thu, 11 May 2023, Kees Cook via Gcc wrote:
>>> 
>>>> On Thu, May 11, 2023 at 06:29:10PM +0200, Alejandro Colomar wrote:
>>>>> On 5/11/23 18:07, Alejandro Colomar wrote:
>>>>> [...]
>>>>>> Would you allow flexible array members in unions?  Is there any
>>>>>> strong reason to disallow them?
>>>> 
>>>> Yes please!! And alone in a struct, too.
>>>> 
>>>> AFAICT, there is no mechanical/architectural reason to disallow them
>>>> (especially since they _can_ be constructed with some fancy tricks,
>>>> and they behave as expected.) My understanding is that it's disallowed
>>>> due to an overly strict reading of the very terse language that created
>>>> flexible arrays in C99.
>>> 
>>> Standard C has no such thing as a zero-size object or type, which would
>>> lead to problems with a struct or union that only contains a flexible
>>> array member there.
>> 
>> Ah-ha, okay. That root cause makes sense now.
> 
> Hmm. but then the workaround
> 
> struct X {
>  int n;
>  union u {
>      char at_least_size_one;
>      int iarr[];
>      short sarr[];
>  };
> };
> 
> doesn't work either.  We could make that a GNU extension without
> adverse effects?

I think that this might be  a very nice extension, which addresses the standard C’s restriction  on the zero-size object, and also can resolve kernel’s need. (And also other users’s similar programming need?)
And maybe it’s also possible to add such extension later to Standard C?

Similar as flexible array member in Standard C, we should limit such union as the last field of another structure.  (Since basically this union can be treated
As a flexible array member)

Qing

> 
> Richard.
> 
>> Why are zero-sized objects missing in Standard C? Or, perhaps, the better
>> question is: what's needed to support the idea of a zero-sized object?
>> 
>> --
>> Kees Cook

Re: [wish] Flexible array members in unions

[Martin Uecker]

> On Thu, May 11, 2023 at 11:14 PM Kees Cook via Gcc <gcc@gcc.gnu.org> wrote:
> >
> > On Thu, May 11, 2023 at 08:53:52PM +0000, Joseph Myers wrote:
> > > On Thu, 11 May 2023, Kees Cook via Gcc wrote:
> > >
> > > > On Thu, May 11, 2023 at 06:29:10PM +0200, Alejandro Colomar wrote:
> > > > > On 5/11/23 18:07, Alejandro Colomar wrote:
> > > > > [...]
> > > > > > Would you allow flexible array members in unions?  Is there any
> > > > > > strong reason to disallow them?
> > > >
> > > > Yes please!! And alone in a struct, too.
> > > >
> > > > AFAICT, there is no mechanical/architectural reason to disallow them
> > > > (especially since they _can_ be constructed with some fancy tricks,
> > > > and they behave as expected.) My understanding is that it's disallowed
> > > > due to an overly strict reading of the very terse language that created
> > > > flexible arrays in C99.
> > >
> > > Standard C has no such thing as a zero-size object or type, which would
> > > lead to problems with a struct or union that only contains a flexible
> > > array member there.

(I think it is fundamentally not too problematic to have zero-size
objects, although it would take some work to specify the semantics
exactly.)

But my preference would be to make structs / unions with FAM an
incomplete type which would then restrict their use (for the cases
now supported we would need backwards compatible exceptions).
We could then allow such a struct / union as the last member
of another struct / union which would make this an incomplete
type too.

We then would need a special macro (based on a builtin) instead
of sizeof to get the size, but this would be safer anyway.

In principle, an even better solution would be to allow dynamic
arrays because then it has a dynamic bound where the type with
the bound could propagate to some user. Bounds checking would
work as expected and more cases.

struct foo {
  int len;
  char buf[.len];
};

But this takes a bit more work to get right.

> >
> > Ah-ha, okay. That root cause makes sense now.
> 
> Hmm. but then the workaround
> 
> struct X {
>   int n;
>   union u {
>       char at_least_size_one;
>       int iarr[];
>       short sarr[];
>   };
> };
> 
> doesn't work either.  We could make that a GNU extension without
> adverse effects?

I think we could allow this even without the "at_least_size_one"
without a problem when allowing the use of such unions only as
a last member of some structure. Allowing it elsewhere seems
questionable anyway.

> Richard.
> 
> > Why are zero-sized objects missing in Standard C? Or, perhaps, the better
> > question is: what's needed to support the idea of a zero-sized object?

Probably a lot of convincing that it actually does not cause problems,
and is useful. Also a lot of work in making sure the standard is revised
everywhere where it is necessary. I think zero sized objects and
especially arrays are very useful also to avoid special code for corner
cases in numerical algorithms. But I think here some restrictions on
the use of the FAM will do.


Martin


 

Re: [wish] Flexible array members in unions

[Qing Zhao]

> On May 18, 2023, at 12:25 PM, Martin Uecker via Gcc <gcc@gcc.gnu.org> wrote:
> 
> 
> 
>> On Thu, May 11, 2023 at 11:14 PM Kees Cook via Gcc <gcc@gcc.gnu.org> wrote:
>>> 
>>> On Thu, May 11, 2023 at 08:53:52PM +0000, Joseph Myers wrote:
>>>> On Thu, 11 May 2023, Kees Cook via Gcc wrote:
>>>> 
>>>>> On Thu, May 11, 2023 at 06:29:10PM +0200, Alejandro Colomar wrote:
>>>>>> On 5/11/23 18:07, Alejandro Colomar wrote:
>>>>>> [...]
>>>>>>> Would you allow flexible array members in unions?  Is there any
>>>>>>> strong reason to disallow them?
>>>>> 
>>>>> Yes please!! And alone in a struct, too.
>>>>> 
>>>>> AFAICT, there is no mechanical/architectural reason to disallow them
>>>>> (especially since they _can_ be constructed with some fancy tricks,
>>>>> and they behave as expected.) My understanding is that it's disallowed
>>>>> due to an overly strict reading of the very terse language that created
>>>>> flexible arrays in C99.
>>>> 
>>>> Standard C has no such thing as a zero-size object or type, which would
>>>> lead to problems with a struct or union that only contains a flexible
>>>> array member there.
> 
> (I think it is fundamentally not too problematic to have zero-size
> objects, although it would take some work to specify the semantics
> exactly.)
> 
> But my preference would be to make structs / unions with FAM an
> incomplete type which would then restrict their use (for the cases
> now supported we would need backwards compatible exceptions).
> We could then allow such a struct / union as the last member
> of another struct / union which would make this an incomplete
> type too.

Yes, I like this approach. 
And we can make them GCC extensions first,  promote to C standard later.

My proposed patch sets (originally targeted on GCC13, now might need to target on GCC14) will
make one part of the above a GCC extension:
    Allowing the struct with FAM as the last member of another struct. (See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101832)

https://gcc.gnu.org/pipermail/gcc-patches/2023-March/614794.html
https://gcc.gnu.org/pipermail/gcc-patches/2023-March/614793.html
https://gcc.gnu.org/pipermail/gcc-patches/2023-March/614790.html

I’d like these changes going into GCC first to improve this area.

> 
> We then would need a special macro (based on a builtin) instead
> of sizeof to get the size, but this would be safer anyway.
> 
> In principle, an even better solution would be to allow dynamic
> arrays because then it has a dynamic bound where the type with
> the bound could propagate to some user. Bounds checking would
> work as expected and more cases.
> 
> struct foo {
>  int len;
>  char buf[.len];
> };
> 
> But this takes a bit more work to get right.
> 
>>> 
>>> Ah-ha, okay. That root cause makes sense now.
>> 
>> Hmm. but then the workaround
>> 
>> struct X {
>>  int n;
>>  union u {
>>      char at_least_size_one;
>>      int iarr[];
>>      short sarr[];
>>  };
>> };
>> 
>> doesn't work either.  We could make that a GNU extension without
>> adverse effects?
> 
> I think we could allow this even without the "at_least_size_one"
> without a problem when allowing the use of such unions only as
> a last member of some structure. Allowing it elsewhere seems
> questionable anyway.

Yes,  Such an union can be treated as an flexible array member 
(just multiple flexible arrays sharing the same storage).  Therefore it’s reasonable
To only allow it as the last field of a structure. 

thanks.

Qing.

> 
>> Richard.
>> 
>>> Why are zero-sized objects missing in Standard C? Or, perhaps, the better
>>> question is: what's needed to support the idea of a zero-sized object?
> 
> Probably a lot of convincing that it actually does not cause problems,
> and is useful. Also a lot of work in making sure the standard is revised
> everywhere where it is necessary. I think zero sized objects and
> especially arrays are very useful also to avoid special code for corner
> cases in numerical algorithms. But I think here some restrictions on
> the use of the FAM will do.
> 
> 
> Martin

Re: [wish] Flexible array members in unions

[Martin Uecker]

Am Donnerstag, dem 18.05.2023 um 20:59 +0000 schrieb Qing Zhao:
> 
> > On May 18, 2023, at 12:25 PM, Martin Uecker via Gcc <gcc@gcc.gnu.org> wrote:
> > 
> > 
> > 
> > > On Thu, May 11, 2023 at 11:14 PM Kees Cook via Gcc <gcc@gcc.gnu.org> wrote:
> > > > 
> > > > On Thu, May 11, 2023 at 08:53:52PM +0000, Joseph Myers wrote:
> > > > > On Thu, 11 May 2023, Kees Cook via Gcc wrote:
> > > > > 
> > > > > > On Thu, May 11, 2023 at 06:29:10PM +0200, Alejandro Colomar wrote:
> > > > > > > On 5/11/23 18:07, Alejandro Colomar wrote:
> > > > > > > [...]
> > > > > > > > Would you allow flexible array members in unions?  Is there any
> > > > > > > > strong reason to disallow them?
> > > > > > 
> > > > > > Yes please!! And alone in a struct, too.
> > > > > > 
> > > > > > AFAICT, there is no mechanical/architectural reason to disallow them
> > > > > > (especially since they _can_ be constructed with some fancy tricks,
> > > > > > and they behave as expected.) My understanding is that it's disallowed
> > > > > > due to an overly strict reading of the very terse language that created
> > > > > > flexible arrays in C99.
> > > > > 
> > > > > Standard C has no such thing as a zero-size object or type, which would
> > > > > lead to problems with a struct or union that only contains a flexible
> > > > > array member there.
> > 
> > (I think it is fundamentally not too problematic to have zero-size
> > objects, although it would take some work to specify the semantics
> > exactly.)
> > 
> > But my preference would be to make structs / unions with FAM an
> > incomplete type which would then restrict their use (for the cases
> > now supported we would need backwards compatible exceptions).
> > We could then allow such a struct / union as the last member
> > of another struct / union which would make this an incomplete
> > type too.
> 
> Yes, I like this approach. 
> And we can make them GCC extensions first,  promote to C standard later.
> 
> My proposed patch sets (originally targeted on GCC13, now might need to target on GCC14) will
> make one part of the above a GCC extension:
>     Allowing the struct with FAM as the last member of another struct. (See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101832)
> 
> https://gcc.gnu.org/pipermail/gcc-patches/2023-March/614794.html
> https://gcc.gnu.org/pipermail/gcc-patches/2023-March/614793.html
> https://gcc.gnu.org/pipermail/gcc-patches/2023-March/614790.html
> 
> I’d like these changes going into GCC first to improve this area.

Seems reasonable to me. Thanks!

Martin


> 
> > 
> > We then would need a special macro (based on a builtin) instead
> > of sizeof to get the size, but this would be safer anyway.
> > 
> > In principle, an even better solution would be to allow dynamic
> > arrays because then it has a dynamic bound where the type with
> > the bound could propagate to some user. Bounds checking would
> > work as expected and more cases.
> > 
> > struct foo {
> >  int len;
> >  char buf[.len];
> > };
> > 
> > But this takes a bit more work to get right.
> > 
> > > > 
> > > > Ah-ha, okay. That root cause makes sense now.
> > > 
> > > Hmm. but then the workaround
> > > 
> > > struct X {
> > >  int n;
> > >  union u {
> > >      char at_least_size_one;
> > >      int iarr[];
> > >      short sarr[];
> > >  };
> > > };
> > > 
> > > doesn't work either.  We could make that a GNU extension without
> > > adverse effects?
> > 
> > I think we could allow this even without the "at_least_size_one"
> > without a problem when allowing the use of such unions only as
> > a last member of some structure. Allowing it elsewhere seems
> > questionable anyway.
> 
> Yes,  Such an union can be treated as an flexible array member 
> (just multiple flexible arrays sharing the same storage).  Therefore it’s reasonable
> To only allow it as the last field of a structure. 
> 
> thanks.
> 
> Qing.
> 
> > 
> > > Richard.
> > > 
> > > > Why are zero-sized objects missing in Standard C? Or, perhaps, the better
> > > > question is: what's needed to support the idea of a zero-sized object?
> > 
> > Probably a lot of convincing that it actually does not cause problems,
> > and is useful. Also a lot of work in making sure the standard is revised
> > everywhere where it is necessary. I think zero sized objects and
> > especially arrays are very useful also to avoid special code for corner
> > cases in numerical algorithms. But I think here some restrictions on
> > the use of the FAM will do.
> > 
> > 
> > Martin
>