* [wish] Flexible array members in unions @ 2023-05-11 16:07 Alejandro Colomar 2023-05-11 16:29 ` Alejandro Colomar 0 siblings, 1 reply; 16+ messages in thread From: Alejandro Colomar @ 2023-05-11 16:07 UTC (permalink / raw) To: GCC; +Cc: Alejandro Colomar [-- Attachment #1.1: Type: text/plain, Size: 2869 bytes --] Hi! Currently, one can have pseudo-flexible array members in unions with [0] syntax, but it's not allowed with [] syntax. Here's an example of how it is possible today: struct s { ... size_t n; union { ptrdiff_t off[0]; // [n]; offsets from s->data. char data[0]; }; }; which is useful to have a structure with two (or really several) consecutive flexible arrays: one of offsets, which mark the positions of data, and another with the actual data. Below goes an example program, which works fine with GCC, and I believe rewriting it to not use the union would make it less clear, since I'd need to add casts to it. It works thanks to [0] pseudo-flexible arrays, but it doesn't compile with C99 flexible arrays. And of course, [0] arrays have issues with -fstrict-flex-arrays=3. $ cat flexi4.c #include <stddef.h> #include <stdlib.h> #include <stdio.h> #include <string.h> struct s { size_t n; union { ptrdiff_t off[0]; char data[0]; }; }; int main(void) { char *p; struct s *s; s = malloc(offsetof(struct s, off) + sizeof(ptrdiff_t) * 2 + sizeof("foobar") + sizeof("baz")); s->n = 2; p = s->data + sizeof(ptrdiff_t) * s->n; s->off[0] = p - s->data; p = stpcpy(p, "foobar") + 1; s->off[1] = p - s->data; p = stpcpy(p, "baz") + 1; puts(s->data + s->off[0]); puts(s->data + s->off[1]); free(s); } $ gcc-13 -Wall -Wextra -Werror -fanalyzer \ -fsanitize=undefined -fsanitize=address \ -D_FORTIFY_SOURCE=3 -fstrict-flex-arrays=2 \ flexi4.c $ ./a.out foobar baz $ gcc-13 -Wall -Wextra -Werror -fanalyzer \ -fsanitize=undefined -fsanitize=address \ -D_FORTIFY_SOURCE=3 -fstrict-flex-arrays=3 \ flexi4.c $ ./a.out flexi4.c:27:8: runtime error: index 0 out of bounds for type 'ptrdiff_t [*]' flexi4.c:29:8: runtime error: index 1 out of bounds for type 'ptrdiff_t [*]' flexi4.c:32:23: runtime error: index 0 out of bounds for type 'ptrdiff_t [*]' foobar flexi4.c:33:23: runtime error: index 1 out of bounds for type 'ptrdiff_t [*]' baz Would you allow flexible array members in unions? Is there any strong reason to disallow them? Currently, I get: $ gcc-13 -Wall -Wextra -fanalyzer \ -fsanitize=undefined -fsanitize=address \ -D_FORTIFY_SOURCE=3 -fstrict-flex-arrays=3 \ flexi4-true.c flexi4-true.c:9:28: error: flexible array member in union 9 | ptrdiff_t off[]; | ^~~ flexi4-true.c:10:28: error: flexible array member in union 10 | char data[]; | ^~~~ Cheers, Alex -- <http://www.alejandro-colomar.es/> GPG key fingerprint: A9348594CE31283A826FBDD8D57633D441E25BB5 [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [wish] Flexible array members in unions 2023-05-11 16:07 [wish] Flexible array members in unions Alejandro Colomar @ 2023-05-11 16:29 ` Alejandro Colomar 2023-05-11 19:07 ` Kees Cook 0 siblings, 1 reply; 16+ messages in thread From: Alejandro Colomar @ 2023-05-11 16:29 UTC (permalink / raw) To: GCC; +Cc: Kees Cook, Alejandro Colomar, Andrew Clayton, Andrew Clayton [-- Attachment #1.1: Type: text/plain, Size: 4271 bytes --] [CC += Kees, Andrew] [start of thread: <https://inbox.sourceware.org/gcc/ac2550e0-2f5c-3680-08e6-0c224d043036@gmail.com/T/#u>] On 5/11/23 18:07, Alejandro Colomar wrote: > Hi! > > Currently, one can have pseudo-flexible array members in unions with > [0] syntax, but it's not allowed with [] syntax. > > Here's an example of how it is possible today: > > struct s { > ... > > size_t n; > union { > ptrdiff_t off[0]; // [n]; offsets from s->data. > char data[0]; > }; > }; > > which is useful to have a structure with two (or really several) > consecutive flexible arrays: one of offsets, which mark the positions > of data, and another with the actual data. Below goes an example > program, which works fine with GCC, and I believe rewriting it to > not use the union would make it less clear, since I'd need to add > casts to it. > > It works thanks to [0] pseudo-flexible arrays, but it doesn't > compile with C99 flexible arrays. And of course, [0] arrays have > issues with -fstrict-flex-arrays=3. > > > $ cat flexi4.c > #include <stddef.h> > #include <stdlib.h> > #include <stdio.h> > #include <string.h> > > struct s { > size_t n; > union { > ptrdiff_t off[0]; > char data[0]; > }; > }; > > int > main(void) > { > char *p; > struct s *s; > > s = malloc(offsetof(struct s, off) + > sizeof(ptrdiff_t) * 2 + > sizeof("foobar") + sizeof("baz")); > > s->n = 2; > p = s->data + sizeof(ptrdiff_t) * s->n; > > s->off[0] = p - s->data; > p = stpcpy(p, "foobar") + 1; > s->off[1] = p - s->data; > p = stpcpy(p, "baz") + 1; > > puts(s->data + s->off[0]); > puts(s->data + s->off[1]); > > free(s); > } > $ gcc-13 -Wall -Wextra -Werror -fanalyzer \ > -fsanitize=undefined -fsanitize=address \ > -D_FORTIFY_SOURCE=3 -fstrict-flex-arrays=2 \ > flexi4.c > $ ./a.out > foobar > baz > $ gcc-13 -Wall -Wextra -Werror -fanalyzer \ > -fsanitize=undefined -fsanitize=address \ > -D_FORTIFY_SOURCE=3 -fstrict-flex-arrays=3 \ > flexi4.c > $ ./a.out > flexi4.c:27:8: runtime error: index 0 out of bounds for type 'ptrdiff_t [*]' > flexi4.c:29:8: runtime error: index 1 out of bounds for type 'ptrdiff_t [*]' > flexi4.c:32:23: runtime error: index 0 out of bounds for type 'ptrdiff_t [*]' > foobar > flexi4.c:33:23: runtime error: index 1 out of bounds for type 'ptrdiff_t [*]' > baz > > > Would you allow flexible array members in unions? Is there any > strong reason to disallow them? > > Currently, I get: > > $ gcc-13 -Wall -Wextra -fanalyzer \ > -fsanitize=undefined -fsanitize=address \ > -D_FORTIFY_SOURCE=3 -fstrict-flex-arrays=3 \ > flexi4-true.c > flexi4-true.c:9:28: error: flexible array member in union > 9 | ptrdiff_t off[]; > | ^~~ > flexi4-true.c:10:28: error: flexible array member in union > 10 | char data[]; > | ^~~~ > > Currently, the Linux kernel has to go through some hoops due to this restriction: $ grepc -tm __DECLARE_FLEX_ARRAY * include/uapi/linux/stddef.h:42: #define __DECLARE_FLEX_ARRAY(TYPE, NAME) \ struct { \ struct { } __empty_ ## NAME; \ TYPE NAME[]; \ } tools/include/uapi/linux/stddef.h:42: #define __DECLARE_FLEX_ARRAY(TYPE, NAME) \ struct { \ struct { } __empty_ ## NAME; \ TYPE NAME[]; \ } $ grep -rnB2 __DECLARE_FLEX_ARRAY * | head include/uapi/rdma/rdma_user_rxe.h-153- __u32 reserved; include/uapi/rdma/rdma_user_rxe.h-154- union { include/uapi/rdma/rdma_user_rxe.h:155: __DECLARE_FLEX_ARRAY(__u8, inline_data); include/uapi/rdma/rdma_user_rxe.h:156: __DECLARE_FLEX_ARRAY(__u8, atomic_wr); include/uapi/rdma/rdma_user_rxe.h:157: __DECLARE_FLEX_ARRAY(struct rxe_sge, sge); -- include/uapi/scsi/scsi_netlink_fc.h-53- union { include/uapi/scsi/scsi_netlink_fc.h-54- __u32 event_data; include/uapi/scsi/scsi_netlink_fc.h:55: __DECLARE_FLEX_ARRAY(__u8, event_data_flex); -- -- <http://www.alejandro-colomar.es/> GPG key fingerprint: A9348594CE31283A826FBDD8D57633D441E25BB5 [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [wish] Flexible array members in unions 2023-05-11 16:29 ` Alejandro Colomar @ 2023-05-11 19:07 ` Kees Cook 2023-05-11 20:53 ` Joseph Myers 0 siblings, 1 reply; 16+ messages in thread From: Kees Cook @ 2023-05-11 19:07 UTC (permalink / raw) To: Alejandro Colomar Cc: GCC, Alejandro Colomar, Andrew Clayton, Andrew Clayton, linux-hardening On Thu, May 11, 2023 at 06:29:10PM +0200, Alejandro Colomar wrote: > On 5/11/23 18:07, Alejandro Colomar wrote: > [...] > > Would you allow flexible array members in unions? Is there any > > strong reason to disallow them? Yes please!! And alone in a struct, too. AFAICT, there is no mechanical/architectural reason to disallow them (especially since they _can_ be constructed with some fancy tricks, and they behave as expected.) My understanding is that it's disallowed due to an overly strict reading of the very terse language that created flexible arrays in C99. > [...] > Currently, the Linux kernel has to go through some hoops due to this > restriction: > > > $ grepc -tm __DECLARE_FLEX_ARRAY * > include/uapi/linux/stddef.h:42: > #define __DECLARE_FLEX_ARRAY(TYPE, NAME) \ > struct { \ > struct { } __empty_ ## NAME; \ > TYPE NAME[]; \ > } Yes, we've had to do this as we eradicate all the fake flexible arrays in the kernel which cause endless bugs[1]. Additionally, we'll be using -fstrict-flex-arrays=3 soon to let existing array bounds checking mitigations gain coverage over trailing arrays. All of this means that we're converting a lot of code that is happily using dynamically sized arrays in unions, etc. [1] https://people.kernel.org/kees/bounded-flexible-arrays-in-c -- Kees Cook ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [wish] Flexible array members in unions 2023-05-11 19:07 ` Kees Cook @ 2023-05-11 20:53 ` Joseph Myers 2023-05-11 21:13 ` Kees Cook 0 siblings, 1 reply; 16+ messages in thread From: Joseph Myers @ 2023-05-11 20:53 UTC (permalink / raw) To: Kees Cook Cc: Alejandro Colomar, GCC, Alejandro Colomar, Andrew Clayton, Andrew Clayton, linux-hardening On Thu, 11 May 2023, Kees Cook via Gcc wrote: > On Thu, May 11, 2023 at 06:29:10PM +0200, Alejandro Colomar wrote: > > On 5/11/23 18:07, Alejandro Colomar wrote: > > [...] > > > Would you allow flexible array members in unions? Is there any > > > strong reason to disallow them? > > Yes please!! And alone in a struct, too. > > AFAICT, there is no mechanical/architectural reason to disallow them > (especially since they _can_ be constructed with some fancy tricks, > and they behave as expected.) My understanding is that it's disallowed > due to an overly strict reading of the very terse language that created > flexible arrays in C99. Standard C has no such thing as a zero-size object or type, which would lead to problems with a struct or union that only contains a flexible array member there. -- Joseph S. Myers joseph@codesourcery.com ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [wish] Flexible array members in unions 2023-05-11 20:53 ` Joseph Myers @ 2023-05-11 21:13 ` Kees Cook 2023-05-11 21:43 ` Joseph Myers 2023-05-12 6:16 ` Richard Biener 0 siblings, 2 replies; 16+ messages in thread From: Kees Cook @ 2023-05-11 21:13 UTC (permalink / raw) To: Joseph Myers Cc: Alejandro Colomar, GCC, Alejandro Colomar, Andrew Clayton, Andrew Clayton, linux-hardening On Thu, May 11, 2023 at 08:53:52PM +0000, Joseph Myers wrote: > On Thu, 11 May 2023, Kees Cook via Gcc wrote: > > > On Thu, May 11, 2023 at 06:29:10PM +0200, Alejandro Colomar wrote: > > > On 5/11/23 18:07, Alejandro Colomar wrote: > > > [...] > > > > Would you allow flexible array members in unions? Is there any > > > > strong reason to disallow them? > > > > Yes please!! And alone in a struct, too. > > > > AFAICT, there is no mechanical/architectural reason to disallow them > > (especially since they _can_ be constructed with some fancy tricks, > > and they behave as expected.) My understanding is that it's disallowed > > due to an overly strict reading of the very terse language that created > > flexible arrays in C99. > > Standard C has no such thing as a zero-size object or type, which would > lead to problems with a struct or union that only contains a flexible > array member there. Ah-ha, okay. That root cause makes sense now. Why are zero-sized objects missing in Standard C? Or, perhaps, the better question is: what's needed to support the idea of a zero-sized object? -- Kees Cook ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [wish] Flexible array members in unions 2023-05-11 21:13 ` Kees Cook @ 2023-05-11 21:43 ` Joseph Myers 2023-05-11 22:16 ` Kees Cook 2023-05-12 6:16 ` Richard Biener 1 sibling, 1 reply; 16+ messages in thread From: Joseph Myers @ 2023-05-11 21:43 UTC (permalink / raw) To: Kees Cook Cc: Alejandro Colomar, GCC, Alejandro Colomar, Andrew Clayton, Andrew Clayton, linux-hardening On Thu, 11 May 2023, Kees Cook via Gcc wrote: > Why are zero-sized objects missing in Standard C? Or, perhaps, the better > question is: what's needed to support the idea of a zero-sized object? Zero-sized objects break the principle that different objects have different addresses, and the principle of being able to subtract pointers to different elements of an array. There would also be serious C++ compatibility concerns, since C++ allows a struct with no members but it has nonzero size, unlike the GNU C extension where a struct with no members has size zero. -- Joseph S. Myers joseph@codesourcery.com ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [wish] Flexible array members in unions 2023-05-11 21:43 ` Joseph Myers @ 2023-05-11 22:16 ` Kees Cook 2023-05-11 22:52 ` Joseph Myers 2023-05-12 7:49 ` Jonathan Wakely 0 siblings, 2 replies; 16+ messages in thread From: Kees Cook @ 2023-05-11 22:16 UTC (permalink / raw) To: Joseph Myers Cc: Alejandro Colomar, GCC, Alejandro Colomar, Andrew Clayton, Andrew Clayton, linux-hardening On Thu, May 11, 2023 at 09:43:49PM +0000, Joseph Myers wrote: > On Thu, 11 May 2023, Kees Cook via Gcc wrote: > > > Why are zero-sized objects missing in Standard C? Or, perhaps, the better > > question is: what's needed to support the idea of a zero-sized object? > > Zero-sized objects break the principle that different objects have > different addresses, and the principle of being able to subtract pointers > to different elements of an array. There would also be serious C++ > compatibility concerns, since C++ allows a struct with no members but it > has nonzero size, unlike the GNU C extension where a struct with no > members has size zero. Okay, understood. If this is a C-only thing, we can ignore the C++ impact. What depends on the "different objects have different addresses" principle? And why do unions not break this -- they could point to the same locations within the object? And don't flexible arrays already need special handling in this regard? -- Kees Cook ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [wish] Flexible array members in unions 2023-05-11 22:16 ` Kees Cook @ 2023-05-11 22:52 ` Joseph Myers 2023-05-12 0:25 ` Alejandro Colomar 2023-05-12 7:49 ` Jonathan Wakely 1 sibling, 1 reply; 16+ messages in thread From: Joseph Myers @ 2023-05-11 22:52 UTC (permalink / raw) To: Kees Cook Cc: Alejandro Colomar, GCC, Alejandro Colomar, Andrew Clayton, Andrew Clayton, linux-hardening On Thu, 11 May 2023, Kees Cook via Gcc wrote: > Okay, understood. If this is a C-only thing, we can ignore the C++ > impact. We're a lot more careful lately in WG14 about checking for C++ compatibility issues and expecting approval from the liaison group for anything with possible compatibility concerns for syntax in the common subset of C and C++. So, no, we can't ignore the C++ impact for adding empty types; it would need careful consideration in the liaison group. > What depends on the "different objects have different addresses" > principle? And why do unions not break this -- they could point to the > same locations within the object? And don't flexible arrays already need > special handling in this regard? "including a pointer to an object and a subobject at its beginning" and "one is a pointer to one past the end of one array object and the other is a pointer to the start of a different array object that happens to immediately follow the first array object in the address space" are both cases included in the semantics for comparison operators. If you allow zero-size objects you get more special cases there (and quite possibly affect optimizations based on points-to analysis that can determine pointers are based on different objects, if an object is not known at compile time to have nonzero size). -- Joseph S. Myers joseph@codesourcery.com ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [wish] Flexible array members in unions 2023-05-11 22:52 ` Joseph Myers @ 2023-05-12 0:25 ` Alejandro Colomar 0 siblings, 0 replies; 16+ messages in thread From: Alejandro Colomar @ 2023-05-12 0:25 UTC (permalink / raw) To: Joseph Myers, Kees Cook Cc: GCC, Alejandro Colomar, Andrew Clayton, Andrew Clayton, linux-hardening [-- Attachment #1.1: Type: text/plain, Size: 2964 bytes --] Hi Joseph, Kees, On 5/12/23 00:52, Joseph Myers wrote: > On Thu, 11 May 2023, Kees Cook via Gcc wrote: > >> Okay, understood. If this is a C-only thing, we can ignore the C++ >> impact. > > We're a lot more careful lately in WG14 about checking for C++ > compatibility issues and expecting approval from the liaison group for > anything with possible compatibility concerns for syntax in the common > subset of C and C++. So, no, we can't ignore the C++ impact for adding > empty types; it would need careful consideration in the liaison group. > >> What depends on the "different objects have different addresses" >> principle? And why do unions not break this -- they could point to the >> same locations within the object? And don't flexible arrays already need >> special handling in this regard? > > "including a pointer to an object and a subobject at its beginning" and > "one is a pointer to one past the end of one array object and the other is > a pointer to the start of a different array object that happens to > immediately follow the first array object in the address space" are both > cases included in the semantics for comparison operators. If you allow > zero-size objects you get more special cases there (and quite possibly > affect optimizations based on points-to analysis that can determine > pointers are based on different objects, if an object is not known at > compile time to have nonzero size). Since GNU C already supports empty structs, how about allowing that in GCC with no intention of adding it to ISO C? We'll see how good it behaves in GCC, and if so suggest it for inclusion in the standard. Why should GNU C, which allows empty structures, and de facto supports flexible arrays in empty structs and in unions (via the empty preceeding struct and the wrapper struct tricks, as the kernel does), shouldn't support them officially without tricks? Apart from violating artificial rules that disallow that use of flexible arrays, I believe the example program I provided in the first post doesn't violate aliasing rules or other similar rules that would result in optimization conflicts. Does it? About zero-sized types, a union consisting of only flexible-array members would effectively be a zero-sized type, so GCC should have similar issues having this in unions and in empty structs. So far, I don't see GCC complaining about such horrible thing as an array of empty structs: $ cat arr.c #include <stdio.h> struct s {}; struct s x[10]; int main(void) { printf("x: %zu, %p\n", sizeof(x), &x); printf("x[3]: %zu, %p\n", sizeof(x[3]), &x[3]); } $ gcc-13 arr.c -Wall -Wextra $ ./a.out x: 0, 0x55c5f6d72019 x[3]: 0, 0x55c5f6d72019 So, in GNU C land, I think it is reasonable to add this feature. Cheers, Alex -- <http://www.alejandro-colomar.es/> GPG key fingerprint: A9348594CE31283A826FBDD8D57633D441E25BB5 [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [wish] Flexible array members in unions 2023-05-11 22:16 ` Kees Cook 2023-05-11 22:52 ` Joseph Myers @ 2023-05-12 7:49 ` Jonathan Wakely 1 sibling, 0 replies; 16+ messages in thread From: Jonathan Wakely @ 2023-05-12 7:49 UTC (permalink / raw) To: Kees Cook Cc: Joseph Myers, Alejandro Colomar, GCC, Alejandro Colomar, Andrew Clayton, Andrew Clayton, linux-hardening [-- Attachment #1: Type: text/plain, Size: 1214 bytes --] On Thu, 11 May 2023, 23:17 Kees Cook via Gcc, <gcc@gcc.gnu.org> wrote: > On Thu, May 11, 2023 at 09:43:49PM +0000, Joseph Myers wrote: > > On Thu, 11 May 2023, Kees Cook via Gcc wrote: > > > > > Why are zero-sized objects missing in Standard C? Or, perhaps, the > better > > > question is: what's needed to support the idea of a zero-sized object? > > > > Zero-sized objects break the principle that different objects have > > different addresses, and the principle of being able to subtract > pointers > > to different elements of an array. There would also be serious C++ > > compatibility concerns, since C++ allows a struct with no members but it > > has nonzero size, unlike the GNU C extension where a struct with no > > members has size zero. > > Okay, understood. If this is a C-only thing, we can ignore the C++ > impact. What depends on the "different objects have different addresses" > principle? And why do unions not break this -- they could point to the > same locations within the object? You don't have two different objects with the same address in a union, because only one of them "exists" at any time. You're reusing the storage for one object at a time, not for all of the union members. ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [wish] Flexible array members in unions 2023-05-11 21:13 ` Kees Cook 2023-05-11 21:43 ` Joseph Myers @ 2023-05-12 6:16 ` Richard Biener 2023-05-12 12:32 ` David Brown ` (2 more replies) 1 sibling, 3 replies; 16+ messages in thread From: Richard Biener @ 2023-05-12 6:16 UTC (permalink / raw) To: Kees Cook Cc: Joseph Myers, Alejandro Colomar, GCC, Alejandro Colomar, Andrew Clayton, Andrew Clayton, linux-hardening On Thu, May 11, 2023 at 11:14 PM Kees Cook via Gcc <gcc@gcc.gnu.org> wrote: > > On Thu, May 11, 2023 at 08:53:52PM +0000, Joseph Myers wrote: > > On Thu, 11 May 2023, Kees Cook via Gcc wrote: > > > > > On Thu, May 11, 2023 at 06:29:10PM +0200, Alejandro Colomar wrote: > > > > On 5/11/23 18:07, Alejandro Colomar wrote: > > > > [...] > > > > > Would you allow flexible array members in unions? Is there any > > > > > strong reason to disallow them? > > > > > > Yes please!! And alone in a struct, too. > > > > > > AFAICT, there is no mechanical/architectural reason to disallow them > > > (especially since they _can_ be constructed with some fancy tricks, > > > and they behave as expected.) My understanding is that it's disallowed > > > due to an overly strict reading of the very terse language that created > > > flexible arrays in C99. > > > > Standard C has no such thing as a zero-size object or type, which would > > lead to problems with a struct or union that only contains a flexible > > array member there. > > Ah-ha, okay. That root cause makes sense now. Hmm. but then the workaround struct X { int n; union u { char at_least_size_one; int iarr[]; short sarr[]; }; }; doesn't work either. We could make that a GNU extension without adverse effects? Richard. > Why are zero-sized objects missing in Standard C? Or, perhaps, the better > question is: what's needed to support the idea of a zero-sized object? > > -- > Kees Cook ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [wish] Flexible array members in unions 2023-05-12 6:16 ` Richard Biener @ 2023-05-12 12:32 ` David Brown 2023-05-15 19:58 ` Qing Zhao 2023-05-18 16:25 ` Martin Uecker 2 siblings, 0 replies; 16+ messages in thread From: David Brown @ 2023-05-12 12:32 UTC (permalink / raw) To: gcc On 12/05/2023 08:16, Richard Biener via Gcc wrote: > On Thu, May 11, 2023 at 11:14 PM Kees Cook via Gcc <gcc@gcc.gnu.org> wrote: >> >> On Thu, May 11, 2023 at 08:53:52PM +0000, Joseph Myers wrote: >>> On Thu, 11 May 2023, Kees Cook via Gcc wrote: >>> >>>> On Thu, May 11, 2023 at 06:29:10PM +0200, Alejandro Colomar wrote: >>>>> On 5/11/23 18:07, Alejandro Colomar wrote: >>>>> [...] >>>>>> Would you allow flexible array members in unions? Is there any >>>>>> strong reason to disallow them? >>>> >>>> Yes please!! And alone in a struct, too. >>>> >>>> AFAICT, there is no mechanical/architectural reason to disallow them >>>> (especially since they _can_ be constructed with some fancy tricks, >>>> and they behave as expected.) My understanding is that it's disallowed >>>> due to an overly strict reading of the very terse language that created >>>> flexible arrays in C99. >>> >>> Standard C has no such thing as a zero-size object or type, which would >>> lead to problems with a struct or union that only contains a flexible >>> array member there. >> >> Ah-ha, okay. That root cause makes sense now. > > Hmm. but then the workaround > > struct X { > int n; > union u { > char at_least_size_one; > int iarr[]; > short sarr[]; > }; > }; > > doesn't work either. We could make that a GNU extension without > adverse effects? > > Richard. > I would like and use an extension like that (for C and C++) - the flexible arrays would act as though they were the same size as the size-specific part of the union, rounding up in this case to make the alignments correct. I regularly want something like : union ProtocolBuffer { struct { header ... data fields ... } uint8_t raw8[]; uint32_t raw32[]; } The "raw" arrays would be used to move data around, or access it from communication drivers. As C (and C++) is defined, I have to split this up so that the "raw" arrays can use "sizeof(ProtocolTelegram) / 4" or similar expressions for their size. If flexible arrays in unions were allowed here, it could make my code a little neater and use more anonymous unions and structs to reduce unhelpful verbosity. >> Why are zero-sized objects missing in Standard C? Or, perhaps, the better >> question is: what's needed to support the idea of a zero-sized object? >> >> -- >> Kees Cook > ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [wish] Flexible array members in unions 2023-05-12 6:16 ` Richard Biener 2023-05-12 12:32 ` David Brown @ 2023-05-15 19:58 ` Qing Zhao 2023-05-18 16:25 ` Martin Uecker 2 siblings, 0 replies; 16+ messages in thread From: Qing Zhao @ 2023-05-15 19:58 UTC (permalink / raw) To: Richard Biener, Joseph Myers Cc: Kees Cook, Alejandro Colomar, GCC, Alejandro Colomar, Andrew Clayton, Andrew Clayton, linux-hardening > On May 12, 2023, at 2:16 AM, Richard Biener via Gcc <gcc@gcc.gnu.org> wrote: > > On Thu, May 11, 2023 at 11:14 PM Kees Cook via Gcc <gcc@gcc.gnu.org> wrote: >> >> On Thu, May 11, 2023 at 08:53:52PM +0000, Joseph Myers wrote: >>> On Thu, 11 May 2023, Kees Cook via Gcc wrote: >>> >>>> On Thu, May 11, 2023 at 06:29:10PM +0200, Alejandro Colomar wrote: >>>>> On 5/11/23 18:07, Alejandro Colomar wrote: >>>>> [...] >>>>>> Would you allow flexible array members in unions? Is there any >>>>>> strong reason to disallow them? >>>> >>>> Yes please!! And alone in a struct, too. >>>> >>>> AFAICT, there is no mechanical/architectural reason to disallow them >>>> (especially since they _can_ be constructed with some fancy tricks, >>>> and they behave as expected.) My understanding is that it's disallowed >>>> due to an overly strict reading of the very terse language that created >>>> flexible arrays in C99. >>> >>> Standard C has no such thing as a zero-size object or type, which would >>> lead to problems with a struct or union that only contains a flexible >>> array member there. >> >> Ah-ha, okay. That root cause makes sense now. > > Hmm. but then the workaround > > struct X { > int n; > union u { > char at_least_size_one; > int iarr[]; > short sarr[]; > }; > }; > > doesn't work either. We could make that a GNU extension without > adverse effects? I think that this might be a very nice extension, which addresses the standard C’s restriction on the zero-size object, and also can resolve kernel’s need. (And also other users’s similar programming need?) And maybe it’s also possible to add such extension later to Standard C? Similar as flexible array member in Standard C, we should limit such union as the last field of another structure. (Since basically this union can be treated As a flexible array member) Qing > > Richard. > >> Why are zero-sized objects missing in Standard C? Or, perhaps, the better >> question is: what's needed to support the idea of a zero-sized object? >> >> -- >> Kees Cook ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [wish] Flexible array members in unions 2023-05-12 6:16 ` Richard Biener 2023-05-12 12:32 ` David Brown 2023-05-15 19:58 ` Qing Zhao @ 2023-05-18 16:25 ` Martin Uecker 2023-05-18 20:59 ` Qing Zhao 2 siblings, 1 reply; 16+ messages in thread From: Martin Uecker @ 2023-05-18 16:25 UTC (permalink / raw) To: gcc > On Thu, May 11, 2023 at 11:14 PM Kees Cook via Gcc <gcc@gcc.gnu.org> wrote: > > > > On Thu, May 11, 2023 at 08:53:52PM +0000, Joseph Myers wrote: > > > On Thu, 11 May 2023, Kees Cook via Gcc wrote: > > > > > > > On Thu, May 11, 2023 at 06:29:10PM +0200, Alejandro Colomar wrote: > > > > > On 5/11/23 18:07, Alejandro Colomar wrote: > > > > > [...] > > > > > > Would you allow flexible array members in unions? Is there any > > > > > > strong reason to disallow them? > > > > > > > > Yes please!! And alone in a struct, too. > > > > > > > > AFAICT, there is no mechanical/architectural reason to disallow them > > > > (especially since they _can_ be constructed with some fancy tricks, > > > > and they behave as expected.) My understanding is that it's disallowed > > > > due to an overly strict reading of the very terse language that created > > > > flexible arrays in C99. > > > > > > Standard C has no such thing as a zero-size object or type, which would > > > lead to problems with a struct or union that only contains a flexible > > > array member there. (I think it is fundamentally not too problematic to have zero-size objects, although it would take some work to specify the semantics exactly.) But my preference would be to make structs / unions with FAM an incomplete type which would then restrict their use (for the cases now supported we would need backwards compatible exceptions). We could then allow such a struct / union as the last member of another struct / union which would make this an incomplete type too. We then would need a special macro (based on a builtin) instead of sizeof to get the size, but this would be safer anyway. In principle, an even better solution would be to allow dynamic arrays because then it has a dynamic bound where the type with the bound could propagate to some user. Bounds checking would work as expected and more cases. struct foo { int len; char buf[.len]; }; But this takes a bit more work to get right. > > > > Ah-ha, okay. That root cause makes sense now. > > Hmm. but then the workaround > > struct X { > int n; > union u { > char at_least_size_one; > int iarr[]; > short sarr[]; > }; > }; > > doesn't work either. We could make that a GNU extension without > adverse effects? I think we could allow this even without the "at_least_size_one" without a problem when allowing the use of such unions only as a last member of some structure. Allowing it elsewhere seems questionable anyway. > Richard. > > > Why are zero-sized objects missing in Standard C? Or, perhaps, the better > > question is: what's needed to support the idea of a zero-sized object? Probably a lot of convincing that it actually does not cause problems, and is useful. Also a lot of work in making sure the standard is revised everywhere where it is necessary. I think zero sized objects and especially arrays are very useful also to avoid special code for corner cases in numerical algorithms. But I think here some restrictions on the use of the FAM will do. Martin ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [wish] Flexible array members in unions 2023-05-18 16:25 ` Martin Uecker @ 2023-05-18 20:59 ` Qing Zhao 2023-05-19 12:08 ` Martin Uecker 0 siblings, 1 reply; 16+ messages in thread From: Qing Zhao @ 2023-05-18 20:59 UTC (permalink / raw) To: Martin Uecker, Richard Biener, Joseph Myers; +Cc: GCC, Kees Cook > On May 18, 2023, at 12:25 PM, Martin Uecker via Gcc <gcc@gcc.gnu.org> wrote: > > > >> On Thu, May 11, 2023 at 11:14 PM Kees Cook via Gcc <gcc@gcc.gnu.org> wrote: >>> >>> On Thu, May 11, 2023 at 08:53:52PM +0000, Joseph Myers wrote: >>>> On Thu, 11 May 2023, Kees Cook via Gcc wrote: >>>> >>>>> On Thu, May 11, 2023 at 06:29:10PM +0200, Alejandro Colomar wrote: >>>>>> On 5/11/23 18:07, Alejandro Colomar wrote: >>>>>> [...] >>>>>>> Would you allow flexible array members in unions? Is there any >>>>>>> strong reason to disallow them? >>>>> >>>>> Yes please!! And alone in a struct, too. >>>>> >>>>> AFAICT, there is no mechanical/architectural reason to disallow them >>>>> (especially since they _can_ be constructed with some fancy tricks, >>>>> and they behave as expected.) My understanding is that it's disallowed >>>>> due to an overly strict reading of the very terse language that created >>>>> flexible arrays in C99. >>>> >>>> Standard C has no such thing as a zero-size object or type, which would >>>> lead to problems with a struct or union that only contains a flexible >>>> array member there. > > (I think it is fundamentally not too problematic to have zero-size > objects, although it would take some work to specify the semantics > exactly.) > > But my preference would be to make structs / unions with FAM an > incomplete type which would then restrict their use (for the cases > now supported we would need backwards compatible exceptions). > We could then allow such a struct / union as the last member > of another struct / union which would make this an incomplete > type too. Yes, I like this approach. And we can make them GCC extensions first, promote to C standard later. My proposed patch sets (originally targeted on GCC13, now might need to target on GCC14) will make one part of the above a GCC extension: Allowing the struct with FAM as the last member of another struct. (See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101832) https://gcc.gnu.org/pipermail/gcc-patches/2023-March/614794.html https://gcc.gnu.org/pipermail/gcc-patches/2023-March/614793.html https://gcc.gnu.org/pipermail/gcc-patches/2023-March/614790.html I’d like these changes going into GCC first to improve this area. > > We then would need a special macro (based on a builtin) instead > of sizeof to get the size, but this would be safer anyway. > > In principle, an even better solution would be to allow dynamic > arrays because then it has a dynamic bound where the type with > the bound could propagate to some user. Bounds checking would > work as expected and more cases. > > struct foo { > int len; > char buf[.len]; > }; > > But this takes a bit more work to get right. > >>> >>> Ah-ha, okay. That root cause makes sense now. >> >> Hmm. but then the workaround >> >> struct X { >> int n; >> union u { >> char at_least_size_one; >> int iarr[]; >> short sarr[]; >> }; >> }; >> >> doesn't work either. We could make that a GNU extension without >> adverse effects? > > I think we could allow this even without the "at_least_size_one" > without a problem when allowing the use of such unions only as > a last member of some structure. Allowing it elsewhere seems > questionable anyway. Yes, Such an union can be treated as an flexible array member (just multiple flexible arrays sharing the same storage). Therefore it’s reasonable To only allow it as the last field of a structure. thanks. Qing. > >> Richard. >> >>> Why are zero-sized objects missing in Standard C? Or, perhaps, the better >>> question is: what's needed to support the idea of a zero-sized object? > > Probably a lot of convincing that it actually does not cause problems, > and is useful. Also a lot of work in making sure the standard is revised > everywhere where it is necessary. I think zero sized objects and > especially arrays are very useful also to avoid special code for corner > cases in numerical algorithms. But I think here some restrictions on > the use of the FAM will do. > > > Martin ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [wish] Flexible array members in unions 2023-05-18 20:59 ` Qing Zhao @ 2023-05-19 12:08 ` Martin Uecker 0 siblings, 0 replies; 16+ messages in thread From: Martin Uecker @ 2023-05-19 12:08 UTC (permalink / raw) To: Qing Zhao, Richard Biener, Joseph Myers; +Cc: GCC, Kees Cook Am Donnerstag, dem 18.05.2023 um 20:59 +0000 schrieb Qing Zhao: > > > On May 18, 2023, at 12:25 PM, Martin Uecker via Gcc <gcc@gcc.gnu.org> wrote: > > > > > > > > > On Thu, May 11, 2023 at 11:14 PM Kees Cook via Gcc <gcc@gcc.gnu.org> wrote: > > > > > > > > On Thu, May 11, 2023 at 08:53:52PM +0000, Joseph Myers wrote: > > > > > On Thu, 11 May 2023, Kees Cook via Gcc wrote: > > > > > > > > > > > On Thu, May 11, 2023 at 06:29:10PM +0200, Alejandro Colomar wrote: > > > > > > > On 5/11/23 18:07, Alejandro Colomar wrote: > > > > > > > [...] > > > > > > > > Would you allow flexible array members in unions? Is there any > > > > > > > > strong reason to disallow them? > > > > > > > > > > > > Yes please!! And alone in a struct, too. > > > > > > > > > > > > AFAICT, there is no mechanical/architectural reason to disallow them > > > > > > (especially since they _can_ be constructed with some fancy tricks, > > > > > > and they behave as expected.) My understanding is that it's disallowed > > > > > > due to an overly strict reading of the very terse language that created > > > > > > flexible arrays in C99. > > > > > > > > > > Standard C has no such thing as a zero-size object or type, which would > > > > > lead to problems with a struct or union that only contains a flexible > > > > > array member there. > > > > (I think it is fundamentally not too problematic to have zero-size > > objects, although it would take some work to specify the semantics > > exactly.) > > > > But my preference would be to make structs / unions with FAM an > > incomplete type which would then restrict their use (for the cases > > now supported we would need backwards compatible exceptions). > > We could then allow such a struct / union as the last member > > of another struct / union which would make this an incomplete > > type too. > > Yes, I like this approach. > And we can make them GCC extensions first, promote to C standard later. > > My proposed patch sets (originally targeted on GCC13, now might need to target on GCC14) will > make one part of the above a GCC extension: > Allowing the struct with FAM as the last member of another struct. (See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101832) > > https://gcc.gnu.org/pipermail/gcc-patches/2023-March/614794.html > https://gcc.gnu.org/pipermail/gcc-patches/2023-March/614793.html > https://gcc.gnu.org/pipermail/gcc-patches/2023-March/614790.html > > I’d like these changes going into GCC first to improve this area. Seems reasonable to me. Thanks! Martin > > > > > We then would need a special macro (based on a builtin) instead > > of sizeof to get the size, but this would be safer anyway. > > > > In principle, an even better solution would be to allow dynamic > > arrays because then it has a dynamic bound where the type with > > the bound could propagate to some user. Bounds checking would > > work as expected and more cases. > > > > struct foo { > > int len; > > char buf[.len]; > > }; > > > > But this takes a bit more work to get right. > > > > > > > > > > Ah-ha, okay. That root cause makes sense now. > > > > > > Hmm. but then the workaround > > > > > > struct X { > > > int n; > > > union u { > > > char at_least_size_one; > > > int iarr[]; > > > short sarr[]; > > > }; > > > }; > > > > > > doesn't work either. We could make that a GNU extension without > > > adverse effects? > > > > I think we could allow this even without the "at_least_size_one" > > without a problem when allowing the use of such unions only as > > a last member of some structure. Allowing it elsewhere seems > > questionable anyway. > > Yes, Such an union can be treated as an flexible array member > (just multiple flexible arrays sharing the same storage). Therefore it’s reasonable > To only allow it as the last field of a structure. > > thanks. > > Qing. > > > > > > Richard. > > > > > > > Why are zero-sized objects missing in Standard C? Or, perhaps, the better > > > > question is: what's needed to support the idea of a zero-sized object? > > > > Probably a lot of convincing that it actually does not cause problems, > > and is useful. Also a lot of work in making sure the standard is revised > > everywhere where it is necessary. I think zero sized objects and > > especially arrays are very useful also to avoid special code for corner > > cases in numerical algorithms. But I think here some restrictions on > > the use of the FAM will do. > > > > > > Martin > ^ permalink raw reply [flat|nested] 16+ messages in thread
end of thread, other threads:[~2023-05-19 12:08 UTC | newest] Thread overview: 16+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2023-05-11 16:07 [wish] Flexible array members in unions Alejandro Colomar 2023-05-11 16:29 ` Alejandro Colomar 2023-05-11 19:07 ` Kees Cook 2023-05-11 20:53 ` Joseph Myers 2023-05-11 21:13 ` Kees Cook 2023-05-11 21:43 ` Joseph Myers 2023-05-11 22:16 ` Kees Cook 2023-05-11 22:52 ` Joseph Myers 2023-05-12 0:25 ` Alejandro Colomar 2023-05-12 7:49 ` Jonathan Wakely 2023-05-12 6:16 ` Richard Biener 2023-05-12 12:32 ` David Brown 2023-05-15 19:58 ` Qing Zhao 2023-05-18 16:25 ` Martin Uecker 2023-05-18 20:59 ` Qing Zhao 2023-05-19 12:08 ` Martin Uecker
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).