[RFC] how to handle the combination of -fstrict-flex-arrays + -Warray-bounds

[RFC] how to handle the combination of -fstrict-flex-arrays + -Warray-bounds

[Qing Zhao]

Hi,

(FAM below refers to Flexible Array Members):

I need inputs on  how to handle the combination of -fstrict-flex-arrays + -Warray-bounds. 

Our initial goal is to update -Warray-bounds with multiple levels of -fstrict-flex-arrays=N 
to issue warnings according to the different levels of “N”. 
However, after detailed study, I found that this goal was very hard to be achieved.

1. -fstrict-flex-arrays and its levels

The new option -fstrict-flex-arrays has 4 levels:

level   trailing arrays
        treated as FAM

  0     [],[0],[1],[n]          	the default without option
  1     [],[0],[1]
  2     [],[0]
  3     []                      	the default when option specified without value

2. -Warray-bounds and its levels

The option -Warray-bounds currently has 2 levels:

level   trailing arrays         
        treated as FAM         

  1     [],[0],[1]			 the default when option specified without value
  2     []                     	

i.e, 
When -Warray-bounds=1, it treats [],[0],[1] as FAM, the same level as -fstrict-flex-arrays=1;
When -Warray-bounds=2, it only treat [] as FAM, the same level as -fstrict-flex-arrays=3; 

3. How to handle the combination of  -fstrict-flex-arrays and -Warray-bounds?

Question 1:  when -fstrict-flex-arrays does not present, the default is -strict-flex-arrays=0, 
                    which treats [],[0],[1],[n] as FAM, so should we update the default behavior 
                    of -Warray-bounds to treat any trailing array [n] as FAMs?

My immediate answer to Q1 is NO, we shouldn’t, that will be a big regression on -Warray-bounds, right?

Question 2:  when -fstrict-flex-arrays=N1 and -Warray-bounds=N2 present at the same time, 
                     Which one has higher priority? N1 or N2? 

-fstrict-flex-arrays=N1 controls how the compiler code generation treats the trailing arrays as FAMs, it seems
reasonable to give higher priority to N1, However, then should we completely disable the level of -Warray-bounds
N2 under such situation? 

I really don’t know what’s the best way to handle the conflict  between N1 and N2.

Can we completely cancel the 2 levels of -Warray-bounds, and always honor the level of -fstrict-flex-arrays?

Any comments or suggestion will be helpful.

thanks.

Qing

Re: [RFC] how to handle the combination of -fstrict-flex-arrays + -Warray-bounds

[Martin Sebor]

On 10/21/22 09:29, Qing Zhao wrote:
> Hi,
> 
> (FAM below refers to Flexible Array Members):
> 
> I need inputs on  how to handle the combination of -fstrict-flex-arrays + -Warray-bounds.
> 
> Our initial goal is to update -Warray-bounds with multiple levels of -fstrict-flex-arrays=N
> to issue warnings according to the different levels of “N”.
> However, after detailed study, I found that this goal was very hard to be achieved.
> 
> 1. -fstrict-flex-arrays and its levels
> 
> The new option -fstrict-flex-arrays has 4 levels:
> 
> level   trailing arrays
>          treated as FAM
> 
>    0     [],[0],[1],[n]          	the default without option
>    1     [],[0],[1]
>    2     [],[0]
>    3     []                      	the default when option specified without value
> 
> 2. -Warray-bounds and its levels
> 
> The option -Warray-bounds currently has 2 levels:
> 
> level   trailing arrays
>          treated as FAM
> 
>    1     [],[0],[1]			 the default when option specified without value
>    2     []                     	
> 
> i.e,
> When -Warray-bounds=1, it treats [],[0],[1] as FAM, the same level as -fstrict-flex-arrays=1;
> When -Warray-bounds=2, it only treat [] as FAM, the same level as -fstrict-flex-arrays=3;
> 
> 3. How to handle the combination of  -fstrict-flex-arrays and -Warray-bounds?
> 
> Question 1:  when -fstrict-flex-arrays does not present, the default is -strict-flex-arrays=0,
>                      which treats [],[0],[1],[n] as FAM, so should we update the default behavior
>                      of -Warray-bounds to treat any trailing array [n] as FAMs?
> 
> My immediate answer to Q1 is NO, we shouldn’t, that will be a big regression on -Warray-bounds, right?

Yes, it would disable -Warray-bounds in the cases where it warns
for past-the-end accesses to trailing arrays with two or more
elements.  Diagnosing those has historically (i.e., before recent
changes) been a design goal.

> 
> Question 2:  when -fstrict-flex-arrays=N1 and -Warray-bounds=N2 present at the same time,
>                       Which one has higher priority? N1 or N2?
> 
> -fstrict-flex-arrays=N1 controls how the compiler code generation treats the trailing arrays as FAMs, it seems
> reasonable to give higher priority to N1,

I tend to agree.  In other words, set N2' = min(N1, N2).

> However, then should we completely disable the level of -Warray-bounds
> N2 under such situation?
> 
> I really don’t know what’s the best way to handle the conflict  between N1 and N2.
> 
> Can we completely cancel the 2 levels of -Warray-bounds, and always honor the level of -fstrict-flex-arrays?
> 
> Any comments or suggestion will be helpful.

The recent -fstrict-flex-array changes aside, IIRC, there's only
a subtle distinction between the two -Warray-bounds levels (since
level 1 started warning on a number of instances that only level
2 used to diagnose a few releases ago).  I think that subset of
level 2 could be merged into level 1 without increasing the rate
of false positives.  Then level 2 could be assigned a new set of
potential problems to detect (such as past-the-end accesses to
trailing one-element arrays).

Martin

Re: [RFC] how to handle the combination of -fstrict-flex-arrays + -Warray-bounds

[Richard Biener]

On Sat, 22 Oct 2022, Martin Sebor wrote:

> On 10/21/22 09:29, Qing Zhao wrote:
> > Hi,
> > 
> > (FAM below refers to Flexible Array Members):
> > 
> > I need inputs on  how to handle the combination of -fstrict-flex-arrays +
> > -Warray-bounds.
> > 
> > Our initial goal is to update -Warray-bounds with multiple levels of
> > -fstrict-flex-arrays=N
> > to issue warnings according to the different levels of ?N?.
> > However, after detailed study, I found that this goal was very hard to be
> > achieved.
> > 
> > 1. -fstrict-flex-arrays and its levels
> > 
> > The new option -fstrict-flex-arrays has 4 levels:
> > 
> > level   trailing arrays
> >          treated as FAM
> > 
> >    0     [],[0],[1],[n]          	the default without option
> >    1     [],[0],[1]
> >    2     [],[0]
> >    3     []                      	the default when option specified
> >    without value
> > 
> > 2. -Warray-bounds and its levels
> > 
> > The option -Warray-bounds currently has 2 levels:
> > 
> > level   trailing arrays
> >          treated as FAM
> > 
> >    1     [],[0],[1]			 the default when option specified
> >    without value
> >    2     []                     	
> > 
> > i.e,
> > When -Warray-bounds=1, it treats [],[0],[1] as FAM, the same level as
> > -fstrict-flex-arrays=1;
> > When -Warray-bounds=2, it only treat [] as FAM, the same level as
> > -fstrict-flex-arrays=3;
> > 
> > 3. How to handle the combination of  -fstrict-flex-arrays and
> > -Warray-bounds?
> > 
> > Question 1:  when -fstrict-flex-arrays does not present, the default is
> > -strict-flex-arrays=0,
> >                      which treats [],[0],[1],[n] as FAM, so should we update
> >                      the default behavior
> >                      of -Warray-bounds to treat any trailing array [n] as
> >                      FAMs?
> > 
> > My immediate answer to Q1 is NO, we shouldn?t, that will be a big regression
> > on -Warray-bounds, right?
> 
> Yes, it would disable -Warray-bounds in the cases where it warns
> for past-the-end accesses to trailing arrays with two or more
> elements.  Diagnosing those has historically (i.e., before recent
> changes) been a design goal.
> 
> > 
> > Question 2:  when -fstrict-flex-arrays=N1 and -Warray-bounds=N2 present at
> > the same time,
> >                       Which one has higher priority? N1 or N2?
> > 
> > -fstrict-flex-arrays=N1 controls how the compiler code generation treats the
> > trailing arrays as FAMs, it seems
> > reasonable to give higher priority to N1,
> 
> I tend to agree.  In other words, set N2' = min(N1, N2).

Yes.  Or do nothing and treat them independently.  Can you check whether
it's possible to distinguish -Warray-bounds from -Warray-bounds=N?  I'd
say that explicit -Warray-bounds=N should exactly get the documented
set of diagnostis, independent of -fstrict-flex-arrays=N.

> > However, then should we completely disable the level of -Warray-bounds
> > N2 under such situation?
> > 
> > I really don?t know what?s the best way to handle the conflict  between N1
> > and N2.
> > 
> > Can we completely cancel the 2 levels of -Warray-bounds, and always honor
> > the level of -fstrict-flex-arrays?
> > 
> > Any comments or suggestion will be helpful.
> 
> The recent -fstrict-flex-array changes aside, IIRC, there's only
> a subtle distinction between the two -Warray-bounds levels (since
> level 1 started warning on a number of instances that only level
> 2 used to diagnose a few releases ago).  I think that subset of
> level 2 could be merged into level 1 without increasing the rate
> of false positives.  Then level 2 could be assigned a new set of
> potential problems to detect (such as past-the-end accesses to
> trailing one-element arrays).
> 
> Martin
> 
> 

-- 
Richard Biener <rguenther@suse.de>
SUSE Software Solutions Germany GmbH, Frankenstrasse 146, 90461 Nuernberg,
Germany; GF: Ivo Totev, Andrew Myers, Andrew McDonald, Boudien Moerman;
HRB 36809 (AG Nuernberg)

Re: [RFC] how to handle the combination of -fstrict-flex-arrays + -Warray-bounds

[Qing Zhao]

> On Oct 24, 2022, at 3:30 AM, Richard Biener <rguenther@suse.de> wrote:
> 
> On Sat, 22 Oct 2022, Martin Sebor wrote:
> 
>> On 10/21/22 09:29, Qing Zhao wrote:
>>> Hi,
>>> 
>>> (FAM below refers to Flexible Array Members):
>>> 
>>> I need inputs on  how to handle the combination of -fstrict-flex-arrays +
>>> -Warray-bounds.
>>> 
>>> Our initial goal is to update -Warray-bounds with multiple levels of
>>> -fstrict-flex-arrays=N
>>> to issue warnings according to the different levels of ?N?.
>>> However, after detailed study, I found that this goal was very hard to be
>>> achieved.
>>> 
>>> 1. -fstrict-flex-arrays and its levels
>>> 
>>> The new option -fstrict-flex-arrays has 4 levels:
>>> 
>>> level   trailing arrays
>>>         treated as FAM
>>> 
>>>   0     [],[0],[1],[n]          	the default without option
>>>   1     [],[0],[1]
>>>   2     [],[0]
>>>   3     []                      	the default when option specified
>>>   without value
>>> 
>>> 2. -Warray-bounds and its levels
>>> 
>>> The option -Warray-bounds currently has 2 levels:
>>> 
>>> level   trailing arrays
>>>         treated as FAM
>>> 
>>>   1     [],[0],[1]			 the default when option specified
>>>   without value
>>>   2     []                     	
>>> 
>>> i.e,
>>> When -Warray-bounds=1, it treats [],[0],[1] as FAM, the same level as
>>> -fstrict-flex-arrays=1;
>>> When -Warray-bounds=2, it only treat [] as FAM, the same level as
>>> -fstrict-flex-arrays=3;
>>> 
>>> 3. How to handle the combination of  -fstrict-flex-arrays and
>>> -Warray-bounds?
>>> 
>>> Question 1:  when -fstrict-flex-arrays does not present, the default is
>>> -strict-flex-arrays=0,
>>>                     which treats [],[0],[1],[n] as FAM, so should we update
>>>                     the default behavior
>>>                     of -Warray-bounds to treat any trailing array [n] as
>>>                     FAMs?
>>> 
>>> My immediate answer to Q1 is NO, we shouldn?t, that will be a big regression
>>> on -Warray-bounds, right?
>> 
>> Yes, it would disable -Warray-bounds in the cases where it warns
>> for past-the-end accesses to trailing arrays with two or more
>> elements.  Diagnosing those has historically (i.e., before recent
>> changes) been a design goal.
>> 
>>> 
>>> Question 2:  when -fstrict-flex-arrays=N1 and -Warray-bounds=N2 present at
>>> the same time,
>>>                      Which one has higher priority? N1 or N2?
>>> 
>>> -fstrict-flex-arrays=N1 controls how the compiler code generation treats the
>>> trailing arrays as FAMs, it seems
>>> reasonable to give higher priority to N1,
>> 
>> I tend to agree.  In other words, set N2' = min(N1, N2).
> 
> Yes.  Or do nothing and treat them independently.

I prefer treating them independently. 

If there is no multiple levels of -Warray-bounds, it’s safe and reasonable to control -Warray-bounds with 
different levels of -fstrict-flex-arrays=N.  However, the current -Warray-bounds already has multiple levels which
have been exposed to and been used by the end users. Changing their behavior will impact the end-users.


> Can you check whether
> it's possible to distinguish -Warray-bounds from -Warray-bounds=N?

The current difference between -Warray-bounds and -Warray-bounds=2 is:  -Warray-bounds=2 
will NOT treat 0-length arrays and 1-element arrays as FAMs. Therefore report out-of-bounds
 access to 0-lenght arrays or 1-element arrays.


>  I'd
> say that explicit -Warray-bounds=N should exactly get the documented
> set of diagnostis, independent of -fstrict-flex-arrays=N.

If we decide to make -fstrict-flex-arrays=N1 and -Warray-bounds=N2 independently.
How about -fstrict-flex-array=N and -Wstringop-overflow (-Wstringop-overread, etc)? 
Shall we control -Wstringop-overflow with -fstrict-flex-array=N?  Or treat them independently?

Qing
> 
>>> However, then should we completely disable the level of -Warray-bounds
>>> N2 under such situation?
>>> 
>>> I really don?t know what?s the best way to handle the conflict  between N1
>>> and N2.
>>> 
>>> Can we completely cancel the 2 levels of -Warray-bounds, and always honor
>>> the level of -fstrict-flex-arrays?
>>> 
>>> Any comments or suggestion will be helpful.
>> 
>> The recent -fstrict-flex-array changes aside, IIRC, there's only
>> a subtle distinction between the two -Warray-bounds levels (since
>> level 1 started warning on a number of instances that only level
>> 2 used to diagnose a few releases ago).  I think that subset of
>> level 2 could be merged into level 1 without increasing the rate
>> of false positives.  Then level 2 could be assigned a new set of
>> potential problems to detect (such as past-the-end accesses to
>> trailing one-element arrays).
>> 
>> Martin
>> 
>> 
> 
> -- 
> Richard Biener <rguenther@suse.de>
> SUSE Software Solutions Germany GmbH, Frankenstrasse 146, 90461 Nuernberg,
> Germany; GF: Ivo Totev, Andrew Myers, Andrew McDonald, Boudien Moerman;
> HRB 36809 (AG Nuernberg)

Re: [RFC] how to handle the combination of -fstrict-flex-arrays + -Warray-bounds

[Qing Zhao]

> On Oct 22, 2022, at 12:54 PM, Martin Sebor <msebor@gmail.com> wrote:
> 
> On 10/21/22 09:29, Qing Zhao wrote:
>> Hi,
>> (FAM below refers to Flexible Array Members):
>> I need inputs on  how to handle the combination of -fstrict-flex-arrays + -Warray-bounds.
>> Our initial goal is to update -Warray-bounds with multiple levels of -fstrict-flex-arrays=N
>> to issue warnings according to the different levels of “N”.
>> However, after detailed study, I found that this goal was very hard to be achieved.
>> 1. -fstrict-flex-arrays and its levels
>> The new option -fstrict-flex-arrays has 4 levels:
>> level   trailing arrays
>>         treated as FAM
>>   0     [],[0],[1],[n]          	the default without option
>>   1     [],[0],[1]
>>   2     [],[0]
>>   3     []                      	the default when option specified without value
>> 2. -Warray-bounds and its levels
>> The option -Warray-bounds currently has 2 levels:
>> level   trailing arrays
>>         treated as FAM
>>   1     [],[0],[1]			 the default when option specified without value
>>   2     []                     	
>> i.e,
>> When -Warray-bounds=1, it treats [],[0],[1] as FAM, the same level as -fstrict-flex-arrays=1;
>> When -Warray-bounds=2, it only treat [] as FAM, the same level as -fstrict-flex-arrays=3;
>> 3. How to handle the combination of  -fstrict-flex-arrays and -Warray-bounds?
>> Question 1:  when -fstrict-flex-arrays does not present, the default is -strict-flex-arrays=0,
>>                     which treats [],[0],[1],[n] as FAM, so should we update the default behavior
>>                     of -Warray-bounds to treat any trailing array [n] as FAMs?
>> My immediate answer to Q1 is NO, we shouldn’t, that will be a big regression on -Warray-bounds, right?
> 
> Yes, it would disable -Warray-bounds in the cases where it warns
> for past-the-end accesses to trailing arrays with two or more
> elements.  Diagnosing those has historically (i.e., before recent
> changes) been a design goal.
> 
>> Question 2:  when -fstrict-flex-arrays=N1 and -Warray-bounds=N2 present at the same time,
>>                      Which one has higher priority? N1 or N2?
>> -fstrict-flex-arrays=N1 controls how the compiler code generation treats the trailing arrays as FAMs, it seems
>> reasonable to give higher priority to N1,
> 
> I tend to agree.  In other words, set N2' = min(N1, N2).
> 
>> However, then should we completely disable the level of -Warray-bounds
>> N2 under such situation?
>> I really don’t know what’s the best way to handle the conflict  between N1 and N2.
>> Can we completely cancel the 2 levels of -Warray-bounds, and always honor the level of -fstrict-flex-arrays?
>> Any comments or suggestion will be helpful.
> 
> The recent -fstrict-flex-array changes aside, IIRC, there's only
> a subtle distinction between the two -Warray-bounds levels (since
> level 1 started warning on a number of instances that only level
> 2 used to diagnose a few releases ago).  

From the doc: (and I also checked the source code)

 -Warray-bounds=2
This warning level also warns about out of bounds accesses to trailing
struct members of one-element array types (@pxref{Zero Length}) and about
the intermediate results of pointer arithmetic that may yield out of bounds
values.  This warning level may give a larger number of false positives and
is deactivated by default.

As I understand, -Warray-bounds=1 (i.e., -Warray-bounds) will report out-of-bounds access to trailing arrays with two or more elements, and treat trailing arrays with 0 or 1 as FAMs;
-Warray-bounds=2 will report out-of-bounds access to trailing arrays with 0 or 1elements in addition to -Warray-bounds =1. 

Is the above understanding correct?


> I think that subset of
> level 2 could be merged into level 1 without increasing the rate
> of false positives.  Then level 2 could be assigned a new set of
> potential problems to detect (such as past-the-end accesses to
> trailing one-element arrays).

If I understand correctly, Current Level 2 already include warning about past-the-end accesses to trailing one-element arrays (and also 0-length arrays).  

Qing

> 
> Martin