* ISO C3X proposal: nonnull qualifier
@ 2021-11-15 16:01 Alejandro Colomar (man-pages)
2021-11-15 16:30 ` Alejandro Colomar (man-pages)
2021-11-15 20:18 ` Joseph Myers
0 siblings, 2 replies; 22+ messages in thread
From: Alejandro Colomar (man-pages) @ 2021-11-15 16:01 UTC (permalink / raw)
To: JeanHeyd Meneide, gcc, cfe-dev
Hi all,
I'd like to propose the following feature for ISO C (and also ISO C++).
It is based on a mix of GCC's [[gnu::nonnull]] and Clang's _Nonnull,
with a pinch of salt of mine.
I'd like to get some feedback from GCC and Clang,
before sending it as an official proposal.
BTW, since the working group is probably very busy with C2X,
I may delay sending it more than a year.
Or I may propose it first to ISO C++,
and then to ISO C.
I wrote the initial draft in the form of a manual page,
whose source code can be found here:
<https://github.com/alejandro-colomar/nonnull>
It has a Makefile to easily transform it into a PDF.
I also rendered it with cat to inline it in this email.
Cheers,
Alex
---
nonnull(3) Nxxxx nonnull(3)
NAME
nonnull - non‐null pointer
SYNOPSIS
type‐qualifier:
const
nonnull
restrict
volatile
_Atomic
DESCRIPTION
Constraints
Types other than pointer types shall not be
nonnull‐qualified.
Semantics
The properties associated with qualified types are mean‐
ingfull only for expressions that are lvalues.
If the same qualifier appears more than once in the same
specifier‐qualifier list or as declaration specifiers,
either directly or via one or more typedefs, the behavior
is the same as if it appeared only once. If other quali‐
fiers appear along with the _Atomic qualifier, the re‐
sulting type is the so‐qualified atomic type.
If an attempt is made to assign NULL to a pointer defined
with the nonnull qualifier, the behavior is undefined.
If an attempt is made to refer to a pointer with a non‐‐
nonnull‐qualified type through the use of an lvalue with
nonnull‐qualified type, the behavior is undefined.
The intended use of the nonnull and restrict qualifiers
(like the register storage class) is to promote optimiza‐
tion, and deleting all instances of the qualifier from
all preprocessing translation units composing a conform‐
ing program does not change its meaning (i.e., observable
behavior).
NOTES
These rules for nonnull are somewhat of the reverse of
const: Instead of forbidding the discarding of the quali‐
fier, we forbid the addition of the qualifier. The rea‐
son is that constant variables are a subset of variables,
and the danger is in treating a const as a variable.
Similarly, nonnull pointers are a subset of (possibly‐
NULL) pointers, but the danger is in treating possibly‐
NULL pointers as nonnull pointers.
Prior art
GCC has [[gnu::nonnull]]. Why is this better?
It can be applied more specifically in the case of point‐
ers to pointers. And, like with const, the nonnull‐ness
can be better enforced by passing the qualifier around.
However, we recognize the optimizations allowed by
[[gnu::nonnull]], and also allow them, by specifying the
behavior as undefined when the qualifier is misused, as
GCC does.
Clang has _Nonnull. Why is this better?
Clang found that using a qualifier was better than an at‐
tribute, since it allowed to more specifically apply it
to pointers to pointers. We recognize that, and also use
a qualifier.
Clang doesn't specify the behavior as being undefined.
That forbids optimizations, that would otherwise be pos‐
sible. We prefer to allow for those optimizations.
Clang considers this qualifier to be useful only as a di‐
agnostics generator. We not only allow for diagnostics
to be issued, but we have stricter rules that make it
more difficult to produce incorrect code.
Even though the language has reserved identifiers start‐
ing with underscore + uppercase for this kind of key‐
words, Clang has already used _Nonnull, and since we are
changing the meaning, it might cause problems to existing
code. So nonnull seems a better name, which hopefully is
not used by existing code, or at least it is less used.
EXAMPLES
Correct
strcpy(3) may be implemented in the following way, to
signify that it cannot accept NULL as input to any of its
arguments, and that it cannot ever return NULL either.
char *nonnull strcpy(char *nonnull restrict dest,
const char *nonnull restrict src)
{
char *d;
d = dest;
while ((*d++ = *src++) != '\0');
return dest;
}
Note that d need not be nonnull‐qualified, since possibly
being NULL is a superset of not possibly being NULL.
The following variations of the above are incorrect, for
the reasons that follow the code examples.
Incorrect
char *strcpy(char *nonnull restrict dest,
const char *nonnull restrict src)
{
char *d;
d = dest;
while ((*d++ = *src++) != '\0');
return dest;
}
Although this would be valid by itself (wouldn't have un‐
defined behavior), it forbids callers of the function
from assigning the return value to a nonnull‐qualified
pointer.
Undefined behavior
char *nonnull strcpy(char *restrict dest,
const char *nonnull restrict src)
{
char *d;
d = dest;
while ((*d++ = *src++) != '\0');
return dest;
}
This causes undefined behavior, since it assigns a non‐‐
nonnull‐qualified pointer (dest) to a qualified one (the
return value). It's also dangerous, since the user isn't
properly informed that NULL may cause undefined behavior
in the implementation of the function (dest is derefer‐
enced).
AUTHORS
Alejandro Colomar
I must thank the GCC and Clang programmers for having
provided a solid base on which I based this proposal.
SEE ALSO
N2731 ‐ 6.7.3
⟨https://gcc.gnu.org/onlinedocs/gcc/Common‐Function‐
Attributes.html#Common‐Function‐Attributes⟩
⟨https://clang.llvm.org/docs/AttributeReference.html
#nullability‐attributes⟩
C3X 2021‐11‐15 nonnull(3)
--
Alejandro Colomar
Linux man-pages comaintainer; https://www.kernel.org/doc/man-pages/
http://www.alejandro-colomar.es/
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: ISO C3X proposal: nonnull qualifier
2021-11-15 16:01 ISO C3X proposal: nonnull qualifier Alejandro Colomar (man-pages)
@ 2021-11-15 16:30 ` Alejandro Colomar (man-pages)
2021-11-15 20:18 ` Joseph Myers
1 sibling, 0 replies; 22+ messages in thread
From: Alejandro Colomar (man-pages) @ 2021-11-15 16:30 UTC (permalink / raw)
To: JeanHeyd Meneide, gcc, cfe-dev
Hi,
On 11/15/21 5:01 PM, Alejandro Colomar (man-pages) wrote:
> Hi all,
>
> I'd like to propose the following feature for ISO C (and also ISO C++).
> It is based on a mix of GCC's [[gnu::nonnull]] and Clang's _Nonnull,
> with a pinch of salt of mine.
>
> I'd like to get some feedback from GCC and Clang,
> before sending it as an official proposal.
>
> BTW, since the working group is probably very busy with C2X,
> I may delay sending it more than a year.
> Or I may propose it first to ISO C++,
> and then to ISO C.
>
> I wrote the initial draft in the form of a manual page,
> whose source code can be found here:
> <https://github.com/alejandro-colomar/nonnull>
>
> It has a Makefile to easily transform it into a PDF.
> I also rendered it with cat to inline it in this email.
I just came up with some addition to the initial draft:
$ git diff
diff --git a/nonnull.7 b/nonnull.7
index 1390b2d..75370e1 100644
--- a/nonnull.7
+++ b/nonnull.7
@@ -44,7 +44,10 @@ through the use of
an lvalue with
.BR \%nonnull -qualified
type,
-the behavior is undefined.
+the behavior is undefined,
+except if preceeding code
+can prove at compile time that the pointer will not possibly be
+.BR NULL .
.PP
The intended use of the
.B \%nonnull
@@ -212,6 +215,32 @@ since the user isn't properly informed that
may cause undefined behavior in the implementation of the function
.RI ( dest
is dereferenced).
+.SS non-nonnull-qualified to nonnull-qualified valid assignment
+.EX
+int *nonnull foo(int *p)
+{
+ if (!p)
+ exit(EXIT_FAILURE);
+
+ return p;
+}
+.EE
+.PP
+The code above is guaranteed to behave correctly,
+since the check against
+.B NULL
+guarantees that the
+(otherwise causing undefined behaviour)
+assignment is only done if
+.I p
+is not
+.BR NULL .
+Forcing the user to add casts would be dangerous,
+since casts usually disable most compiler diagnostics.
+Since this qualifier pretends to improve programs' correctness,
+that would be disastrous.
+Casts also unnecessarily make code less readable.
+Letting the compiler decide if some assignment is valid is safer.
.SH AUTHORS
Alejandro Colomar
.UR alx.manpages@gmail.com
The affected paragraphs now render as:
[
If an attempt is made to assign NULL to a pointer defined
with the nonnull qualifier, the behavior is undefined.
If an attempt is made to refer to a pointer with a non‐‐
nonnull‐qualified type through the use of an lvalue with
nonnull‐qualified type, the behavior is undefined, except
if preceeding code can prove at compile time that the
pointer will not possibly be NULL.
]
[
non‐nonnull‐qualified to nonnull‐qualified valid assignment
int *nonnull foo(int *p)
{
if (!p)
exit(EXIT_FAILURE);
return p;
}
The code above is guaranteed to behave correctly, since
the check against NULL guarantees that the (otherwise
causing undefined behaviour) assignment is only done if p
is not NULL. Forcing the user to add casts would be dan‐
gerous, since casts usually disable most compiler diag‐
nostics. Since this qualifier pretends to improve pro‐
grams' correctness, that would be disastrous. Casts also
unnecessarily make code less readable. Letting the com‐
piler decide if some assignment is valid is safer.
]
However,
I'm not sure if this imposes too much complexity for compilers,
and maybe it's better to say that it's implementation-defined.
Thanks,
Alex
--
Alejandro Colomar
Linux man-pages comaintainer; https://www.kernel.org/doc/man-pages/
http://www.alejandro-colomar.es/
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: ISO C3X proposal: nonnull qualifier
2021-11-15 16:01 ISO C3X proposal: nonnull qualifier Alejandro Colomar (man-pages)
2021-11-15 16:30 ` Alejandro Colomar (man-pages)
@ 2021-11-15 20:18 ` Joseph Myers
2021-11-15 21:09 ` Alejandro Colomar (man-pages)
1 sibling, 1 reply; 22+ messages in thread
From: Joseph Myers @ 2021-11-15 20:18 UTC (permalink / raw)
To: Alejandro Colomar (man-pages); +Cc: JeanHeyd Meneide, gcc, cfe-dev
lvalue-to-rvalue conversion loses qualifiers, which makes any rules based
on whether the RHS of an assignment was nonnull-qualified very
problematic. (The specification of restrict is exceedingly tricky and
very unlikely to be a good basis for specifying any other feature.)
I don't think a manpage is a good form for proposing a language feature.
Actual proposed normative wording for the C standard, showing all relevant
changes to all relevant subclauses, is better.
--
Joseph S. Myers
joseph@codesourcery.com
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: ISO C3X proposal: nonnull qualifier
2021-11-15 20:18 ` Joseph Myers
@ 2021-11-15 21:09 ` Alejandro Colomar (man-pages)
2021-11-15 22:17 ` Joseph Myers
2021-11-16 9:30 ` Jonathan Wakely
0 siblings, 2 replies; 22+ messages in thread
From: Alejandro Colomar (man-pages) @ 2021-11-15 21:09 UTC (permalink / raw)
To: Joseph Myers; +Cc: JeanHeyd Meneide, gcc, cfe-dev
Hi Joseph,
On 11/15/21 21:18, Joseph Myers wrote:
> lvalue-to-rvalue conversion loses qualifiers, which makes any rules based
> on whether the RHS of an assignment was nonnull-qualified very
> problematic. (The specification of restrict is exceedingly tricky and
> very unlikely to be a good basis for specifying any other feature.)
Hmm.
restrict was the closest thing to a
const-like level of safety that I could think of.
It would allow a compiler to keep track of nullness
of every pointer,
and issue appropriate diagnostics
probably better than what -fanalyzer already does.
How is restrict handling that problem of lvalue-to-rvalue already?
Can you think of any other way nonnull-ness could be passed
to nested function calls with language enforcement?
The other option would be to propose plain [[gnu::nonnull]],
which couldn't be enforced across nested function calls
(or I could't think of how yet).
Well, the simplest cases (i.e., not pointer-to-pointer)
could be detected by the compiler,
but other than that, it's impossible.
But if that's the only way,
it's better than nothing.
>
> I don't think a manpage is a good form for proposing a language feature.
> Actual proposed normative wording for the C standard, showing all relevant
> changes to all relevant subclauses, is better.
>
My intention is that the final PDF to be sent to the committee
will have those diffs.
But I have no clue of how to do that kind of things,
so for an initial draft to discuss on,
before even presenting it to the committee,
I think my "native" language for writing technical documents
will be easier.
Also,
I'm curious,
do you do those diffs usually by hand?
I mean, you can't diff(1) a PDF, can you? :)
Considering that C moves at a 10-years pace,
and we're late for C2X,
I have until around 2030 to learn how to do that :-)
Thanks!
and kind regards,
Alex
--
Alejandro Colomar
Linux man-pages comaintainer; http://www.kernel.org/doc/man-pages/
http://www.alejandro-colomar.es/
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: ISO C3X proposal: nonnull qualifier
2021-11-15 21:09 ` Alejandro Colomar (man-pages)
@ 2021-11-15 22:17 ` Joseph Myers
2021-11-15 22:35 ` Alejandro Colomar (man-pages)
2021-11-16 9:30 ` Jonathan Wakely
1 sibling, 1 reply; 22+ messages in thread
From: Joseph Myers @ 2021-11-15 22:17 UTC (permalink / raw)
To: Alejandro Colomar (man-pages); +Cc: gcc, cfe-dev
On Mon, 15 Nov 2021, Alejandro Colomar (man-pages) via Gcc wrote:
> How is restrict handling that problem of lvalue-to-rvalue already?
restrict has tricky rules about "based on" (6.7.3.1).
--
Joseph S. Myers
joseph@codesourcery.com
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: ISO C3X proposal: nonnull qualifier
2021-11-15 22:17 ` Joseph Myers
@ 2021-11-15 22:35 ` Alejandro Colomar (man-pages)
2021-11-15 22:47 ` Joseph Myers
0 siblings, 1 reply; 22+ messages in thread
From: Alejandro Colomar (man-pages) @ 2021-11-15 22:35 UTC (permalink / raw)
To: Joseph Myers; +Cc: gcc, cfe-dev, JeanHeyd Meneide
Hi Joseph,
On 11/15/21 23:17, Joseph Myers wrote:
> On Mon, 15 Nov 2021, Alejandro Colomar (man-pages) via Gcc wrote:
>
>> How is restrict handling that problem of lvalue-to-rvalue already?
>
> restrict has tricky rules about "based on" (6.7.3.1).
Hmm, I think I can "base on" that,
to define what I had in mind. :)
I'll come back to you when I have something
(or when I desist trying to come up with that something).
Cheers,
Alex
--
Alejandro Colomar
Linux man-pages comaintainer; http://www.kernel.org/doc/man-pages/
http://www.alejandro-colomar.es/
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: ISO C3X proposal: nonnull qualifier
2021-11-15 22:35 ` Alejandro Colomar (man-pages)
@ 2021-11-15 22:47 ` Joseph Myers
2021-11-16 12:34 ` Alejandro Colomar (man-pages)
0 siblings, 1 reply; 22+ messages in thread
From: Joseph Myers @ 2021-11-15 22:47 UTC (permalink / raw)
To: Alejandro Colomar (man-pages); +Cc: gcc, cfe-dev
On Mon, 15 Nov 2021, Alejandro Colomar (man-pages) via Gcc wrote:
> Hi Joseph,
>
> On 11/15/21 23:17, Joseph Myers wrote:
> > On Mon, 15 Nov 2021, Alejandro Colomar (man-pages) via Gcc wrote:
> >
> > > How is restrict handling that problem of lvalue-to-rvalue already?
> >
> > restrict has tricky rules about "based on" (6.7.3.1).
>
> Hmm, I think I can "base on" that,
> to define what I had in mind. :)
"based on" is about optimizations; I think it's even less suited to
anything relating to diagnostics than it is to optimization.
To restrict assignment between different kinds of pointers, I'd think
you'd want pointer type variants that differ in some way *other* than
qualifiers, a way that's unaffected by lvalue-to-rvalue conversion, but
that comes with its own rules on implicit conversion as if by assignment
(6.5.16.1) (though then you also need to work out what's allowed in terms
of mixing these pointer type variants in all the other operations allowing
pointers, what type results of pointer arithmetic have, etc.). And there
should surely also be some way of converting a normal pointer to this
variant with a runtime check for NULL.
Note that discussion of prior art in such a proposal should also consider
relevant prior art (for constraining possible values of a variable through
the type system) in C++ or other languages if possible.
--
Joseph S. Myers
joseph@codesourcery.com
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: ISO C3X proposal: nonnull qualifier
2021-11-15 21:09 ` Alejandro Colomar (man-pages)
2021-11-15 22:17 ` Joseph Myers
@ 2021-11-16 9:30 ` Jonathan Wakely
2021-11-16 17:13 ` [cfe-dev] " Arthur O'Dwyer
1 sibling, 1 reply; 22+ messages in thread
From: Jonathan Wakely @ 2021-11-16 9:30 UTC (permalink / raw)
To: Alejandro Colomar (man-pages); +Cc: Joseph Myers, gcc, cfe-dev
On Mon, 15 Nov 2021, 21:15 Alejandro Colomar (man-pages) <gcc@gcc.gnu.org>
wrote:
> My intention is that the final PDF to be sent to the committee
> will have those diffs.
> But I have no clue of how to do that kind of things,
> so for an initial draft to discuss on,
> before even presenting it to the committee,
> I think my "native" language for writing technical documents
> will be easier.
>
> Also,
> I'm curious,
> do you do those diffs usually by hand?
>
Yes. Just highlight text in red and green, with strike through or
underlining.
If you write the paper in LaTeX you can use macros like:
\definecolor{addclr}{rgb}{0,.6,.6}
\definecolor{remclr}{rgb}{1,0,0}
\renewcommand{\added}[1]{\textcolor{addclr}{\uline{#1}}}
\newcommand{\removed}[1]{\textcolor{remclr}{\sout{#1}}}
\renewcommand{\changed}[2]{\removed{#1}\added{#2}}
I mean, you can't diff(1) a PDF, can you? :)
>
Aside: see diffpdf, which is packaged by some distros.
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: ISO C3X proposal: nonnull qualifier
2021-11-15 22:47 ` Joseph Myers
@ 2021-11-16 12:34 ` Alejandro Colomar (man-pages)
2021-11-17 0:06 ` Alejandro Colomar (man-pages)
` (2 more replies)
0 siblings, 3 replies; 22+ messages in thread
From: Alejandro Colomar (man-pages) @ 2021-11-16 12:34 UTC (permalink / raw)
To: Joseph Myers; +Cc: gcc, cfe-dev
Hi Joseph,
On 11/15/21 23:47, Joseph Myers wrote:
> On Mon, 15 Nov 2021, Alejandro Colomar (man-pages) via Gcc wrote:
>
>> Hi Joseph,
>>
>> On 11/15/21 23:17, Joseph Myers wrote:
>>> On Mon, 15 Nov 2021, Alejandro Colomar (man-pages) via Gcc wrote:
>>>
>>>> How is restrict handling that problem of lvalue-to-rvalue already?
>>>
>>> restrict has tricky rules about "based on" (6.7.3.1).
>>
>> Hmm, I think I can "base on" that,
>> to define what I had in mind. :)
>
> "based on" is about optimizations; I think it's even less suited to
> anything relating to diagnostics than it is to optimization.
>
> To restrict assignment between different kinds of pointers, I'd think
> you'd want pointer type variants that differ in some way *other* than
> qualifiers, a way that's unaffected by lvalue-to-rvalue conversion, but
> that comes with its own rules on implicit conversion as if by assignment
> (6.5.16.1) (though then you also need to work out what's allowed in terms
> of mixing these pointer type variants in all the other operations allowing
> pointers, what type results of pointer arithmetic have, etc.). And there
> should surely also be some way of converting a normal pointer to this
> variant with a runtime check for NULL.
>
> Note that discussion of prior art in such a proposal should also consider
> relevant prior art (for constraining possible values of a variable through
> the type system) in C++ or other languages if possible.
>
The only other language that I know is C++, so I'll talk about it:
C++ added long ago something close to this: references.
However, IMO:
- They don't provide any enforced safety at all.
It's just as safe as using [[gnu::nonnull]].
See an example code below showing why.
- They unnecessarily changed pointer syntax to use value syntax.
This syntax has not been widely accepted by some C programmers,
which would have to adapt their minds considerably.
Clang's _Nonnull (and _Nullable) qualifiers (are they?):
- Do have the safety that C++ references should have had,
if they had been properly designed.
Again, see the example code below.
- They have an intuitive syntax for C programmers,
by just adding a qualifier to a pointer,
you mark it as either possibly being NULL or not.
It has the appearance of a qualifier,
and the documentation refers to it as a qualifier,
but how does it implement it clang internally?
Is it implemented as a completely different type
with some implicit conversion rules?
I don't know.
See
<https://clang.llvm.org/docs/AttributeReference.html#nullability-attributes>.
---
$ cat nonnull.c
[[gnu::returns_nonnull]]
int *foo(int *p)
{
return p;
}
/*
* No (nonnull-related) diagnostics from the following commands:
* $ gcc -Wall -Wextra -std=gnu2x -S nonnull.c
* $ clang -Weverything -std=gnu2x -S nonnull.c
*/
/*++++++++++++*/
/* Clang only */
int *_Nonnull bar(int *_Nullable p)
{
return p;
}
/*
* Clang provides safety here:
*
* $ clang -Weverything -std=gnu2x -S nonnull.c
* nonnull.c:17:9: warning: implicit conversion from nullable pointer
'int * _Nullable' to non-nullable pointer type 'int * _Nonnull'
[-Wnullable-to-nonnull-conversion]
* return p;
* ^
*/
/*++++++++++++*/
/* C++ only */
int *baz(int *p)
{
int &q = *p;
return &q;
}
/* No (nonnull-related) diagnostics from the following commands:
* $ gcc -Wall -Wextra -std=gnu++2b -S nonnull.cxx
* $ clang -Weverything -std=gnu++20 -S nonnull.cxx
*/
---
So we have the following list of prior art:
- [[gnu::nonnull]] (GCC)
- _Nonnull (Clang)
- & (references) (C++)
From which I completely dislike references,
for not adding any real benefits,
and being unnecessarily unintuitive (to C programmers).
I like from _Nonnull that it enforces diagnostics.
And I like from [[gnu::nonnull]] that it allows optimizations.
I'm trying to mix them in a good way.
Since Clang's _Nonnull is closer to what I have in mind,
I did further tests to _Nonnull,
to see what I like,
and what I dislike:
---
$ cat _Nonnull.c
#include <stdlib.h>
int *_Nonnull f(int *_Nullable p)
{
if (!p)
exit(1);
return p;
}
int *_Nonnull g(int *_Null_unspecified p)
{
return p;
}
int *_Nonnull h(int *p)
{
return p;
}
int *_Nullable i(int *_Nonnull p)
{
return p;
}
---
First of all,
I see unnecessary (probably over-engineered) qualifiers:
- _Null_unspecified seems to me the same as nothing.
If I didn't specify its nullability,
it's by definition unspecified. Right?
- _Nullable seems to me also the same as nothing.
The language allows for a pointer to be NULL,
so if you don't specify if it can or not be null,
you better stay on the safe side and consider it as nullable.
Then,
I see other problems:
- I don't get a warning from g(), nor from h(), which I'd want.
To get something like const-safety,
we need to design it so that it can be enforced;
either you guarantee that a pointer cannot be NULL (_Nonnull),
or you cannot guarantee it (not _Nonnull).
Otherwise,
[[gnu::nonnull]] would seem better to me.
So, I'd leave out of the design everything but _Nonnull,
and consider everything else as nullable by default.
- I get a warning from f().
Ideally,
a programmer should not need to cast
(casts are dangerous),
to convert a nullable pointer to a _nonnull pointer.
For that,
appropriate checks should be in the preceeding code.
Otherwise, a diagnostic should be issued.
To be on the safe side,
if a compiler has doubts,
it should diagnose.
There's some Clang document that talks about something similar.
I don't know its validity,
or if it was a draft before _Nonnull qualifiers.
<https://clang.llvm.org/docs/analyzer/developer-docs/nullability.html>
Thanks!
Alex
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [cfe-dev] ISO C3X proposal: nonnull qualifier
2021-11-16 9:30 ` Jonathan Wakely
@ 2021-11-16 17:13 ` Arthur O'Dwyer
0 siblings, 0 replies; 22+ messages in thread
From: Arthur O'Dwyer @ 2021-11-16 17:13 UTC (permalink / raw)
To: Alejandro Colomar (man-pages); +Cc: gcc, Clang Dev
On Tue, Nov 16, 2021 at 4:31 AM Jonathan Wakely via cfe-dev <
cfe-dev@lists.llvm.org> wrote:
> On Mon, 15 Nov 2021, 21:15 Alejandro Colomar (man-pages) <gcc@gcc.gnu.org>
> wrote:
>
>> Also, I'm curious, do you do those diffs usually by hand?
>>
>
> Yes. Just highlight text in red and green, with strike through or
> underlining.
>
> If you write the paper in LaTeX you can use macros like:
> \definecolor{addclr}{rgb}{0,.6,.6}
> \definecolor{remclr}{rgb}{1,0,0}
> \renewcommand{\added}[1]{\textcolor{addclr}{\uline{#1}}}
> \newcommand{\removed}[1]{\textcolor{remclr}{\sout{#1}}}
> \renewcommand{\changed}[2]{\removed{#1}\added{#2}}
>
Or, if you use Bikeshed, it's just the HTML tags <ins> and <del>. For
example:
https://github.com/Quuxplusone/draft/blob/29d1609da4182504dec7ada25b0700246bb7d86a/Makefile#L12
https://github.com/Quuxplusone/draft/blob/29d1609da4182504dec7ada25b0700246bb7d86a/d2266-implicit-move-rvalue-ref.bs#L529-L552
produces
https://quuxplusone.github.io/draft/d2266-implicit-move-rvalue-ref.html#wording
Even with Bikeshed, copy-pasting the standard text and marking up the diff
is still a manual process; but it *looks* pretty nice IMO, at both the
source level and the finished-product level.
HTH,
Arthur
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: ISO C3X proposal: nonnull qualifier
2021-11-16 12:34 ` Alejandro Colomar (man-pages)
@ 2021-11-17 0:06 ` Alejandro Colomar (man-pages)
2021-11-20 16:47 ` Ping: " Alejandro Colomar (man-pages)
2021-11-23 11:32 ` [cfe-dev] " Dmitri Gribenko
2021-11-23 11:17 ` Dmitri Gribenko
2021-12-02 20:24 ` Alejandro Colomar (man-pages)
2 siblings, 2 replies; 22+ messages in thread
From: Alejandro Colomar (man-pages) @ 2021-11-17 0:06 UTC (permalink / raw)
To: cfe-dev; +Cc: gcc, Joseph Myers
Hi,
Sorry for Clang people,
when I started this thread,
I wasn't subscribed to your list,
and some messages are not on your list.
You can find the complete thread on the GCC list:
<https://gcc.gnu.org/pipermail/gcc/2021-November/237743.html>
I have a few questions for you.
See below, please.
On 11/16/21 13:34, Alejandro Colomar (man-pages) wrote:
> $ cat _Nonnull.c
> #include <stdlib.h>
>
> int *_Nonnull f(int *_Nullable p)
> {
> if (!p)
> exit(1);
> return p;
> }
>
>
> - I get a warning from f().
> Ideally,
> a programmer should not need to cast
> (casts are dangerous),
> to convert a nullable pointer to a _Nonnull pointer.
> For that,
> appropriate checks should be in the preceeding code.
> Otherwise, a diagnostic should be issued.
> To be on the safe side,
> if a compiler has doubts,
> it should diagnose.
>
> There's some Clang document that talks about something similar.
> I don't know its validity,
> or if it was a draft before _Nonnull qualifiers.
> <https://clang.llvm.org/docs/analyzer/developer-docs/nullability.html>
That document suggests that I shouldn't get a diagnostic from f().
Why did I get a diagnostic? (I tried clang 11, 13 & 14(experimental))
Is it talking about a different nonnull attribute/qualifier?
Was it about a proposal prior to the current _Nonnull?
Why is it not in use? Was it too difficult to implement?
Do you think Clang could be improved to not warn on f()?
Thanks,
Alex
^ permalink raw reply [flat|nested] 22+ messages in thread
* Ping: ISO C3X proposal: nonnull qualifier
2021-11-17 0:06 ` Alejandro Colomar (man-pages)
@ 2021-11-20 16:47 ` Alejandro Colomar (man-pages)
2021-11-23 11:32 ` [cfe-dev] " Dmitri Gribenko
1 sibling, 0 replies; 22+ messages in thread
From: Alejandro Colomar (man-pages) @ 2021-11-20 16:47 UTC (permalink / raw)
To: cfe-dev, Arthur O'Dwyer, Joerg Sonnenberger; +Cc: gcc, Joseph Myers
Ping
On 11/17/21 01:06, Alejandro Colomar (man-pages) wrote:
> Hi,
>
> Sorry for Clang people,
> when I started this thread,
> I wasn't subscribed to your list,
> and some messages are not on your list.
> You can find the complete thread on the GCC list:
> <https://gcc.gnu.org/pipermail/gcc/2021-November/237743.html>
>
> I have a few questions for you.
> See below, please.
>
> On 11/16/21 13:34, Alejandro Colomar (man-pages) wrote:
>> $ cat _Nonnull.c
>> #include <stdlib.h>
>>
>> int *_Nonnull f(int *_Nullable p)
>> {
>> if (!p)
>> exit(1);
>> return p;
>> }
>>
>>
>> - I get a warning from f().
>> Ideally,
>> a programmer should not need to cast
>> (casts are dangerous),
>> to convert a nullable pointer to a _Nonnull pointer.
>> For that,
>> appropriate checks should be in the preceeding code.
>> Otherwise, a diagnostic should be issued.
>> To be on the safe side,
>> if a compiler has doubts,
>> it should diagnose.
>>
>> There's some Clang document that talks about something similar.
>> I don't know its validity,
>> or if it was a draft before _Nonnull qualifiers.
>> <https://clang.llvm.org/docs/analyzer/developer-docs/nullability.html>
>
> That document suggests that I shouldn't get a diagnostic from f().
> Why did I get a diagnostic? (I tried clang 11, 13 & 14(experimental))
>
>
> Is it talking about a different nonnull attribute/qualifier?
> Was it about a proposal prior to the current _Nonnull?
> Why is it not in use? Was it too difficult to implement?
>
>
> Do you think Clang could be improved to not warn on f()?
>
>
> Thanks,
> Alex
>
--
Alejandro Colomar
Linux man-pages comaintainer; http://www.kernel.org/doc/man-pages/
http://www.alejandro-colomar.es/
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [cfe-dev] ISO C3X proposal: nonnull qualifier
2021-11-16 12:34 ` Alejandro Colomar (man-pages)
2021-11-17 0:06 ` Alejandro Colomar (man-pages)
@ 2021-11-23 11:17 ` Dmitri Gribenko
2021-11-23 11:45 ` Alejandro Colomar (man-pages)
2021-12-02 20:24 ` Alejandro Colomar (man-pages)
2 siblings, 1 reply; 22+ messages in thread
From: Dmitri Gribenko @ 2021-11-23 11:17 UTC (permalink / raw)
To: Alejandro Colomar (man-pages); +Cc: Joseph Myers, gcc, cfe-dev
Hi Alejandro,
On Tue, Nov 16, 2021 at 1:34 PM Alejandro Colomar (man-pages) via
cfe-dev <cfe-dev@lists.llvm.org> wrote:
> First of all,
> I see unnecessary (probably over-engineered) qualifiers:
>
> - _Null_unspecified seems to me the same as nothing.
> If I didn't specify its nullability,
> it's by definition unspecified. Right?
>
> - _Nullable seems to me also the same as nothing.
> The language allows for a pointer to be NULL,
> so if you don't specify if it can or not be null,
> you better stay on the safe side and consider it as nullable.
_Nullable is used in conjunction with the `#pragma clang
assume_nonnull begin/end` pragma that flips the default:
```
#pragma clang assume_nonnull begin
int *global_int_ptr; // implicitly _Nonnull
#pragma clang assume_nonnull end
```
Within these pragma brackets, you need to use _Nullable to get the
opposite behavior.
The pragma itself is useful because it reduces the amount of noise the
annotations introduce. When these annotations were adopted in Apple
SDKs, it was found that in practice most pointers are non-nullable. So
if we only had _Nonnull, we would have to annotate most pointers.
Instead, Apple's SDKs bracket every header contents with this pragma,
and instead annotate nullable pointers, significantly reducing the
amount of annotations.
_Null_unspecified is a way to say that nullability is complicated due
to legacy reasons (for example, a function may return NULL under
extremely rare circumstances that most users don't care about, so we
want to allow returning NULL, while suppressing warnings at the usage
site if the user assumes that the returned pointer is non-NULL). It
might have been useful for bringing certain legacy APIs into the
annotated world with more checks enabled. But right now it is used
extremely rarely in Apple's SDKs and therefore it is unclear to me
personally whether it really pulls its weight at this point.
Dmitri
--
main(i,j){for(i=2;;i++){for(j=2;j<i;j++){if(!(i%j)){j=0;break;}}if
(j){printf("%d\n",i);}}} /*Dmitri Gribenko <gribozavr@gmail.com>*/
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [cfe-dev] ISO C3X proposal: nonnull qualifier
2021-11-17 0:06 ` Alejandro Colomar (man-pages)
2021-11-20 16:47 ` Ping: " Alejandro Colomar (man-pages)
@ 2021-11-23 11:32 ` Dmitri Gribenko
1 sibling, 0 replies; 22+ messages in thread
From: Dmitri Gribenko @ 2021-11-23 11:32 UTC (permalink / raw)
To: Alejandro Colomar (man-pages); +Cc: cfe-dev, gcc, Joseph Myers
Hi Alejandro,
On Wed, Nov 17, 2021 at 1:06 AM Alejandro Colomar (man-pages) via
cfe-dev <cfe-dev@lists.llvm.org> wrote:
> On 11/16/21 13:34, Alejandro Colomar (man-pages) wrote:
> > $ cat _Nonnull.c
> > #include <stdlib.h>
> >
> > int *_Nonnull f(int *_Nullable p)
> > {
> > if (!p)
> > exit(1);
> > return p;
> > }
> >
> >
> > - I get a warning from f().
> > Ideally,
> > a programmer should not need to cast
> > (casts are dangerous),
> > to convert a nullable pointer to a _Nonnull pointer.
> > For that,
> > appropriate checks should be in the preceeding code.
> > Otherwise, a diagnostic should be issued.
> > To be on the safe side,
> > if a compiler has doubts,
> > it should diagnose.
> >
> > There's some Clang document that talks about something similar.
> > I don't know its validity,
> > or if it was a draft before _Nonnull qualifiers.
> > <https://clang.llvm.org/docs/analyzer/developer-docs/nullability.html>
>
> That document suggests that I shouldn't get a diagnostic from f().
> Why did I get a diagnostic? (I tried clang 11, 13 & 14(experimental))
>
>
> Is it talking about a different nonnull attribute/qualifier?
> Was it about a proposal prior to the current _Nonnull?
> Why is it not in use? Was it too difficult to implement?
The false positive you're getting is from the Clang warning
-Wnullable-to-nonnull-conversion. It is a simple type-based check that
does not take dataflow information into account. In other words, the
reason for the nullability false positive in your example is identical
to the reason for the integer conversion false positive here:
```
#include <stdlib.h>
#include <stdint.h>
#include <limits.h>
uint8_t f(uint32_t x) {
if (x > UINT8_MAX)
exit(1);
return x;
}
```
warning: implicit conversion loses integer precision: 'uint32_t' (aka
'unsigned int') to 'uint8_t' (aka 'unsigned char')
[-Wimplicit-int-conversion]
This webpage https://clang.llvm.org/docs/analyzer/developer-docs/nullability.html
describes Clang Static Analyzer, is a sophisticated path-sensitive
static analysis tool. It is unfortunately often too slow to enable in
regular compilation.
> Do you think Clang could be improved to not warn on f()?
Absolutely. We can implement a dataflow-based check that takes the
flow condition into account. Flow-sensitive diagnostics should scale a
lot better than path-sensitive, and they should be fast enough to
become a compiler warning. We are currently upstreaming a dataflow
analysis framework that should make implementing such diagnostics
easier. https://lists.llvm.org/pipermail/cfe-dev/2021-October/069098.html
Dmitri
--
main(i,j){for(i=2;;i++){for(j=2;j<i;j++){if(!(i%j)){j=0;break;}}if
(j){printf("%d\n",i);}}} /*Dmitri Gribenko <gribozavr@gmail.com>*/
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [cfe-dev] ISO C3X proposal: nonnull qualifier
2021-11-23 11:17 ` Dmitri Gribenko
@ 2021-11-23 11:45 ` Alejandro Colomar (man-pages)
2021-11-23 12:45 ` Dmitri Gribenko
0 siblings, 1 reply; 22+ messages in thread
From: Alejandro Colomar (man-pages) @ 2021-11-23 11:45 UTC (permalink / raw)
To: Dmitri Gribenko; +Cc: Joseph Myers, gcc, cfe-dev
Hi Dmitry,
On 11/23/21 12:17, Dmitri Gribenko wrote:
> Hi Alejandro,
>
> On Tue, Nov 16, 2021 at 1:34 PM Alejandro Colomar (man-pages) via
> cfe-dev <cfe-dev@lists.llvm.org> wrote:
>> First of all,
>> I see unnecessary (probably over-engineered) qualifiers:
>>
>> - _Null_unspecified seems to me the same as nothing.
>> If I didn't specify its nullability,
>> it's by definition unspecified. Right?
>>
>> - _Nullable seems to me also the same as nothing.
>> The language allows for a pointer to be NULL,
>> so if you don't specify if it can or not be null,
>> you better stay on the safe side and consider it as nullable.
>
> _Nullable is used in conjunction with the `#pragma clang
> assume_nonnull begin/end` pragma that flips the default:
>
> ```
> #pragma clang assume_nonnull begin
> int *global_int_ptr; // implicitly _Nonnull
> #pragma clang assume_nonnull end
> ```
>
> Within these pragma brackets, you need to use _Nullable to get the
> opposite behavior.
>
> The pragma itself is useful because it reduces the amount of noise the
> annotations introduce. When these annotations were adopted in Apple
> SDKs, it was found that in practice most pointers are non-nullable. So
> if we only had _Nonnull, we would have to annotate most pointers.
> Instead, Apple's SDKs bracket every header contents with this pragma,
> and instead annotate nullable pointers, significantly reducing the
> amount of annotations.
That's interesting. Most of my functions also tipically are full of
[[gnu::nonnull]], so the _Nonnull default seems the best thing.
However, would that be viable in old code that relies on standard C?
I think that it would, but maybe you have more experience. Do you agree
with the following?
Let's imagine a scenario where C3X specifies that non-qualified pointers
are nonnull. And there's only a qualifier, _Nullable, to allow NULL.
Asigning _Nullable to nonnull would issue a diagnostic.
Old code will stop compiling if it uses NULL, but that can easily be
fixed by marking the pointers as _Nullable, and maybe while at that,
programmers will find a few bugs.
Compilers will have to be carefull, because memcpy() will make NULL
members of structures, so they'll need to know if that can be done or
not, and many structure members will need to be marked as _Nullable, if
the structure is expected to be bzero()ed.
Also, do you have any experience in avoiding to diagnose a _Nullable to
nonnull assignment _after_ explicitly comparing to NULL? I.e., allow
the following:
int *_Nullable p;
int *q;
if (!p)
q = p;
Thanks,
Alex
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [cfe-dev] ISO C3X proposal: nonnull qualifier
2021-11-23 11:45 ` Alejandro Colomar (man-pages)
@ 2021-11-23 12:45 ` Dmitri Gribenko
2021-12-01 22:24 ` Alejandro Colomar (man-pages)
0 siblings, 1 reply; 22+ messages in thread
From: Dmitri Gribenko @ 2021-11-23 12:45 UTC (permalink / raw)
To: Alejandro Colomar (man-pages); +Cc: Joseph Myers, gcc, cfe-dev
Hi Alejandro,
On Tue, Nov 23, 2021 at 12:45 PM Alejandro Colomar (man-pages)
<alx.manpages@gmail.com> wrote:
>
> Hi Dmitry,
>
> On 11/23/21 12:17, Dmitri Gribenko wrote:
> > Hi Alejandro,
> >
> > On Tue, Nov 16, 2021 at 1:34 PM Alejandro Colomar (man-pages) via
> > cfe-dev <cfe-dev@lists.llvm.org> wrote:
> >> First of all,
> >> I see unnecessary (probably over-engineered) qualifiers:
> >>
> >> - _Null_unspecified seems to me the same as nothing.
> >> If I didn't specify its nullability,
> >> it's by definition unspecified. Right?
> >>
> >> - _Nullable seems to me also the same as nothing.
> >> The language allows for a pointer to be NULL,
> >> so if you don't specify if it can or not be null,
> >> you better stay on the safe side and consider it as nullable.
> >
> > _Nullable is used in conjunction with the `#pragma clang
> > assume_nonnull begin/end` pragma that flips the default:
> >
> > ```
> > #pragma clang assume_nonnull begin
> > int *global_int_ptr; // implicitly _Nonnull
> > #pragma clang assume_nonnull end
> > ```
> >
> > Within these pragma brackets, you need to use _Nullable to get the
> > opposite behavior.
> >
> > The pragma itself is useful because it reduces the amount of noise the
> > annotations introduce. When these annotations were adopted in Apple
> > SDKs, it was found that in practice most pointers are non-nullable. So
> > if we only had _Nonnull, we would have to annotate most pointers.
> > Instead, Apple's SDKs bracket every header contents with this pragma,
> > and instead annotate nullable pointers, significantly reducing the
> > amount of annotations.
>
> That's interesting. Most of my functions also tipically are full of
> [[gnu::nonnull]], so the _Nonnull default seems the best thing.
>
> However, would that be viable in old code that relies on standard C?
> I think that it would, but maybe you have more experience. Do you agree
> with the following?
>
> Let's imagine a scenario where C3X specifies that non-qualified pointers
> are nonnull. And there's only a qualifier, _Nullable, to allow NULL.
> Asigning _Nullable to nonnull would issue a diagnostic.
I think C3X specifying that non-qualified pointers are nonnnull would
be a showstopper, I don't think it is likely to happen given how the
users and the committee value backward compatibility that C has
offered throughout the decades.
If I were to speculate what would happen if C3X did flip the default,
I think it would be treated by the community as a language fork.
Pre-C3X headers won't work correctly when included in C3X programs,
making incremental adoption of C3X syntax, as it was intended to be
used, impossible. Projects would likely invent a NULLABLE macro, which
would expand to _Nullable in C3X and nothing in earlier versions, to
enable an incremental transition.
That's why Clang introduced the pragma, enabling new rules to be
adopted incrementally.
> Also, do you have any experience in avoiding to diagnose a _Nullable to
> nonnull assignment _after_ explicitly comparing to NULL? I.e., allow
> the following:
>
> int *_Nullable p;
> int *q;
>
> if (!p)
> q = p;
Internally at Google we have a checker based on the dataflow analysis
framework (https://lists.llvm.org/pipermail/cfe-dev/2021-October/069098.html)
that diagnoses usages of std::optional<T>::value() not guarded by
has_value(). We are planning to upstream it after we finish
upstreaming the dataflow framework itself. Ensuring guarded usage of
std::optional<T>::value() is very similar to diagnosing dereferences
of nullable pointers. I think a lot of the experience is transferable.
However, we haven't attempted to implement a pointer nullability check
yet, so I don't yet understand all corner cases that arise in real
world software.
However, fundamentally, there are a few questions that you need to answer:
- does _Nullable create a distinct type or not? It is extremely
important when you consider C++.
- do you want nullability-related diagnostics to be mandatory, or
optional? For example, a compiler is not required to issue a
diagnostic about a program that violates the constraints of
`restrict`.
If _Nullable does not create a distinct type, and `T*` is the same
type as `T* _Nullable`, then we can't rely on the regular type system
mechanisms to issue diagnostics.
If _Nullable creates a distinct type, then according to regular C type
checking rules, you would get a warning on the `q = p` assignment
regardless of the `if (!p)` check. That's how the C type system works,
it is not flow-sensitive.
If we want the diagnostics to be fllow-sensitive like in your example
(I think it would be the best choice), then we need to add a new
flow-sensitive component to the C type system. I don't think there is
a precedent for this in C right now. I'm not sure how the committee or
implementors would react to such a proposal.
Dmitri
--
main(i,j){for(i=2;;i++){for(j=2;j<i;j++){if(!(i%j)){j=0;break;}}if
(j){printf("%d\n",i);}}} /*Dmitri Gribenko <gribozavr@gmail.com>*/
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [cfe-dev] ISO C3X proposal: nonnull qualifier
2021-11-23 12:45 ` Dmitri Gribenko
@ 2021-12-01 22:24 ` Alejandro Colomar (man-pages)
2021-12-02 0:39 ` Dmitri Gribenko
0 siblings, 1 reply; 22+ messages in thread
From: Alejandro Colomar (man-pages) @ 2021-12-01 22:24 UTC (permalink / raw)
To: Dmitri Gribenko; +Cc: Joseph Myers, gcc, cfe-dev
Hi Dmitry,
On 11/23/21 13:45, Dmitri Gribenko wrote:
>>
>> Let's imagine a scenario where C3X specifies that non-qualified pointers
>> are nonnull. And there's only a qualifier, _Nullable, to allow NULL.
>> Asigning _Nullable to nonnull would issue a diagnostic.
>
> I think C3X specifying that non-qualified pointers are nonnnull would
> be a showstopper, I don't think it is likely to happen given how the
> users and the committee value backward compatibility that C has
> offered throughout the decades.
Agreed. I even found some cases where it would cause previously-correct
code to misbehave, so changing normal pointers to mean nonnull pointers
is not possible at this point. Compilers may allow that with pragmas
anyway.
>
> If I were to speculate what would happen if C3X did flip the default,
> I think it would be treated by the community as a language fork.
Yes
> Pre-C3X headers won't work correctly when included in C3X programs,
> making incremental adoption of C3X syntax, as it was intended to be
> used, impossible. Projects would likely invent a NULLABLE macro, which
> would expand to _Nullable in C3X and nothing in earlier versions, to
> enable an incremental transition.
>
> That's why Clang introduced the pragma, enabling new rules to be
> adopted incrementally.
Let's avoid forking C :)
>
>> Also, do you have any experience in avoiding to diagnose a _Nullable to
>> nonnull assignment _after_ explicitly comparing to NULL? I.e., allow
>> the following:
>>
>> int *_Nullable p;
>> int *q;
>>
>> if (!p)
>> q = p;
>
> Internally at Google we have a checker based on the dataflow analysis
> framework (https://lists.llvm.org/pipermail/cfe-dev/2021-October/069098.html)
> that diagnoses usages of std::optional<T>::value() not guarded by
> has_value(). We are planning to upstream it after we finish
> upstreaming the dataflow framework itself. Ensuring guarded usage of
> std::optional<T>::value() is very similar to diagnosing dereferences
> of nullable pointers. I think a lot of the experience is transferable.
> However, we haven't attempted to implement a pointer nullability check
> yet, so I don't yet understand all corner cases that arise in real
> world software.
>
> However, fundamentally, there are a few questions that you need to answer:
>
> - does _Nullable create a distinct type or not? It is extremely
> important when you consider C++.
Let's talk about _Nonnull, which is more likely to be added.
I would say no. Neither const or restrict create a new type, but a
qualified version of a type. _Nonnull has a behavior somewhat in
between restrict and const.
>
> - do you want nullability-related diagnostics to be mandatory, or
> optional? For example, a compiler is not required to issue a
> diagnostic about a program that violates the constraints of
> `restrict`.
I think this should be similar to const, so a mandatory warning would be
better. But still I have doubts, because this is much more complex to
diagnose than const violations (there should be no diagnostic after an
explicit NULL check).
>
> If _Nullable does not create a distinct type, and `T*` is the same
> type as `T* _Nullable`, then we can't rely on the regular type system
> mechanisms to issue diagnostics.
>
> If _Nullable creates a distinct type, then according to regular C type
> checking rules, you would get a warning on the `q = p` assignment
> regardless of the `if (!p)` check. That's how the C type system works,
> it is not flow-sensitive.
>
> If we want the diagnostics to be fllow-sensitive like in your example
> (I think it would be the best choice), then we need to add a new
> flow-sensitive component to the C type system. I don't think there is
> a precedent for this in C right now. I'm not sure how the committee or
> implementors would react to such a proposal.
Yes, it's complex. It should be flow-sensitive. Right now there's a
precedent, I think: 'restrict'. But there are no mandatory warnings
AFAIK. This would go one step further, and would probably be the most
complex addition to the language ever. A minimal complying compiler
would need to be much more complex than ever, which is a bad thing.
Right now I have two voices in my head saying opposite things: on one
hand this would add a lot of complexity to the C language, and I don't
like that; on the other hand, this would make standard C much more safe
than it is regarding pointers, and for the plain user, it would reduce
the cognitive complexity of making sure null pointers are not passed
where they shouldn't.
GCC adding _Nonnull before the standard would be a good first step.
Cheers,
Alex
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [cfe-dev] ISO C3X proposal: nonnull qualifier
2021-12-01 22:24 ` Alejandro Colomar (man-pages)
@ 2021-12-02 0:39 ` Dmitri Gribenko
2021-12-02 1:00 ` Alejandro Colomar (man-pages)
0 siblings, 1 reply; 22+ messages in thread
From: Dmitri Gribenko @ 2021-12-02 0:39 UTC (permalink / raw)
To: Alejandro Colomar (man-pages); +Cc: Joseph Myers, gcc, cfe-dev
Hi Alejandro,
On Wed, Dec 1, 2021 at 11:24 PM Alejandro Colomar (man-pages)
<alx.manpages@gmail.com> wrote:
> On 11/23/21 13:45, Dmitri Gribenko wrote:
> > If I were to speculate what would happen if C3X did flip the default,
> > I think it would be treated by the community as a language fork.
>
> Yes
>
> > Pre-C3X headers won't work correctly when included in C3X programs,
> > making incremental adoption of C3X syntax, as it was intended to be
> > used, impossible. Projects would likely invent a NULLABLE macro, which
> > would expand to _Nullable in C3X and nothing in earlier versions, to
> > enable an incremental transition.
> >
> > That's why Clang introduced the pragma, enabling new rules to be
> > adopted incrementally.
>
> Let's avoid forking C :)
Do you consider the standard pragma `#pragma STDC FENV_ACCESS` to be a
language fork? If not, why is a pragma to control nullability of
pointers different?
Dmitri
--
main(i,j){for(i=2;;i++){for(j=2;j<i;j++){if(!(i%j)){j=0;break;}}if
(j){printf("%d\n",i);}}} /*Dmitri Gribenko <gribozavr@gmail.com>*/
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [cfe-dev] ISO C3X proposal: nonnull qualifier
2021-12-02 0:39 ` Dmitri Gribenko
@ 2021-12-02 1:00 ` Alejandro Colomar (man-pages)
0 siblings, 0 replies; 22+ messages in thread
From: Alejandro Colomar (man-pages) @ 2021-12-02 1:00 UTC (permalink / raw)
To: Dmitri Gribenko; +Cc: Joseph Myers, gcc, cfe-dev
Hi Dmitri
On 12/2/21 01:39, Dmitri Gribenko wrote:
>>> Pre-C3X headers won't work correctly when included in C3X programs,
>>> making incremental adoption of C3X syntax, as it was intended to be
>>> used, impossible. Projects would likely invent a NULLABLE macro, which
>>> would expand to _Nullable in C3X and nothing in earlier versions, to
>>> enable an incremental transition.
>>>
>>> That's why Clang introduced the pragma, enabling new rules to be
>>> adopted incrementally.
>>
>> Let's avoid forking C :)
>
> Do you consider the standard pragma `#pragma STDC FENV_ACCESS` to be a
> language fork? If not, why is a pragma to control nullability of
> pointers different?
Sorry, I put the reply at a wrong point in the quote. I didn't refer to
the pragma, but the the previous paragraph. Basically, I meant let's
not add _Nullable to C3X, and add just _Nonnull.
Cheers,
Alex
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: ISO C3X proposal: nonnull qualifier
2021-11-16 12:34 ` Alejandro Colomar (man-pages)
2021-11-17 0:06 ` Alejandro Colomar (man-pages)
2021-11-23 11:17 ` Dmitri Gribenko
@ 2021-12-02 20:24 ` Alejandro Colomar (man-pages)
2021-12-02 20:31 ` Alejandro Colomar (man-pages)
2021-12-02 20:36 ` Joseph Myers
2 siblings, 2 replies; 22+ messages in thread
From: Alejandro Colomar (man-pages) @ 2021-12-02 20:24 UTC (permalink / raw)
To: Joseph Myers; +Cc: gcc, cfe-dev
On 11/16/21 13:34, Alejandro Colomar (man-pages) wrote:
> $ cat _Nonnull.c
> #include <stdlib.h>
>
> int *_Nonnull f(int *_Nullable p)
> {
> if (!p)
> exit(1);
> return p;
> }
>
> int *_Nonnull g(int *_Null_unspecified p)
> {
> return p;
> }
>
> int *_Nonnull h(int *p)
> {
> return p;
> }
>
> int *_Nullable i(int *_Nonnull p)
> {
> return p;
> }
>
> ---
> I see other problems:
>
> - I don't get a warning from g(), nor from h(), which I'd want.
> To get something like const-safety,
> we need to design it so that it can be enforced;
> either you guarantee that a pointer cannot be NULL (_Nonnull),
> or you cannot guarantee it (not _Nonnull).
> Otherwise,
> [[gnu::nonnull]] would seem better to me.
>
> So, I'd leave out of the design everything but _Nonnull,
> and consider everything else as nullable by default.
Getting warnings from h() and g() would be easy to impose. Just forbid
implicit addition of _Nonnull qualifier.
>
> - I get a warning from f().
> Ideally,
> a programmer should not need to cast
> (casts are dangerous),
> to convert a nullable pointer to a _nonnull pointer.
> For that,
> appropriate checks should be in the preceeding code.
> Otherwise, a diagnostic should be issued.
> To be on the safe side,
> if a compiler has doubts,
> it should diagnose.
>
> There's some Clang document that talks about something similar.
> I don't know its validity,
> or if it was a draft before _Nonnull qualifiers.
> <https://clang.llvm.org/docs/analyzer/developer-docs/nullability.html>
I came with an idea that would make this QoI, so that the ISO standard
wouldn't need to force every compiler to be flow-sensitive, which would
be a bad precedent.
Implicit additions of the qualifier could be allowed to be warned
_always_ by the standard.
As an extension, quality compilers may not warn when they can see a NULL
check just before the assignment.
User code could make of the following macro, to avoid having to add
casts everywhere, which would be as bad as not having _Nonnull at all.
I got the idea from the GCC __builtin_xxx_overflow() builtins.
#if defined(__cplusplus)
#define const_cast(t, x) const_cast<t>(x)
#else
#define const_cast(t, x) ((t) (x))
#endif
#if !defined(cplusplus)
#define auto __auto_type
#endif
#define nonnull_assign(nn, p) \
({ \
auto p_ = p; \
auto nn_ = nn; \
\
if (p_ == NULL) \
*nn_ = const_cast(typeof(nn_), p_); \
\
p_ == NULL; \
})
And use it like this:
int *_Nonnull f(int *_Nullable p)
{
int *_Nonnull q;
if (nonnull_assign(&q, p))
exit(1);
return q;
}
That way there's only one place where there's a cast, so it can be
easily audited. The above would be warning-free, without requiring a
lot of complexity into the compiler.
Please, forgive the extensive use of GNU extensions in the above snippet
(and the redefinition of auto), as I forgive those who don't use them :)
So, now this can be made non-flow-sensitive, which was a big concern.
And now the biggest concern I can see is that this qualifier works
opposite to const (here discarding is allowed but not adding it), and
that is contrary to how compilers have been working for now. As far as
I could read in the standard, there's no mention to qualifiers being
dropped in an rvalue; Joseph, could you please confirm? Also, as I
already mentioned, Clang already implements this somehow without
discarding the _Nonnull qualifier, so I guess this can be done.
Cheers,
Alex
--
Alejandro Colomar
Linux man-pages comaintainer; http://www.kernel.org/doc/man-pages/
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: ISO C3X proposal: nonnull qualifier
2021-12-02 20:24 ` Alejandro Colomar (man-pages)
@ 2021-12-02 20:31 ` Alejandro Colomar (man-pages)
2021-12-02 20:36 ` Joseph Myers
1 sibling, 0 replies; 22+ messages in thread
From: Alejandro Colomar (man-pages) @ 2021-12-02 20:31 UTC (permalink / raw)
To: Joseph Myers; +Cc: gcc, cfe-dev
On 12/2/21 21:24, Alejandro Colomar (man-pages) wrote:
>
> #define nonnull_assign(nn, p) \
> ({ \
> auto p_ = p; \
> auto nn_ = nn; \
> \
> if (p_ == NULL) \
> *nn_ = const_cast(typeof(nn_), p_); \
D'oh, this check should be the opposite, of course :)
> \
> p_ == NULL; \
> })
>
And I forgot to mention, that this macro intentionally leaves the
nonnull pointer (nn) uninitialized in the case of a NULL input pointer
(p), so that the compiler can then warn about an uninitialized variable
if it's used uninitialized.
Cheers,
Alex
--
Alejandro Colomar
Linux man-pages comaintainer; http://www.kernel.org/doc/man-pages/
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: ISO C3X proposal: nonnull qualifier
2021-12-02 20:24 ` Alejandro Colomar (man-pages)
2021-12-02 20:31 ` Alejandro Colomar (man-pages)
@ 2021-12-02 20:36 ` Joseph Myers
1 sibling, 0 replies; 22+ messages in thread
From: Joseph Myers @ 2021-12-02 20:36 UTC (permalink / raw)
To: Alejandro Colomar (man-pages); +Cc: gcc, cfe-dev
On Thu, 2 Dec 2021, Alejandro Colomar (man-pages) via Gcc wrote:
> So, now this can be made non-flow-sensitive, which was a big concern. And now
> the biggest concern I can see is that this qualifier works opposite to const
> (here discarding is allowed but not adding it), and that is contrary to how
For all existing qualifiers, the rules about discarding are rules about
permitted assignments (and conversions as if by assignment) between
pointers and concern the qualifiers on pointer target types: 6.5.16.1 is
the key subclause concerning implicit conversions, and any proposal for
changes in that area needs to be precise about exactly what textual
changes are proposed to 6.5.16.1.
> compilers have been working for now. As far as I could read in the standard,
> there's no mention to qualifiers being dropped in an rvalue; Joseph, could you
> please confirm? Also, as I already mentioned, Clang already implements this
lvalue-to-rvalue conversion drops qualifiers (and _Atomic). "If the
lvalue has qualified type, the value has the unqualified version of the
type of the lvalue; additionally, if the lvalue has atomic type, the value
has the non-atomic version of the type of the lvalue; otherwise, the value
has the type of the lvalue." (6.3.2.1).
--
Joseph S. Myers
joseph@codesourcery.com
^ permalink raw reply [flat|nested] 22+ messages in thread
end of thread, other threads:[~2021-12-02 20:36 UTC | newest]
Thread overview: 22+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-11-15 16:01 ISO C3X proposal: nonnull qualifier Alejandro Colomar (man-pages)
2021-11-15 16:30 ` Alejandro Colomar (man-pages)
2021-11-15 20:18 ` Joseph Myers
2021-11-15 21:09 ` Alejandro Colomar (man-pages)
2021-11-15 22:17 ` Joseph Myers
2021-11-15 22:35 ` Alejandro Colomar (man-pages)
2021-11-15 22:47 ` Joseph Myers
2021-11-16 12:34 ` Alejandro Colomar (man-pages)
2021-11-17 0:06 ` Alejandro Colomar (man-pages)
2021-11-20 16:47 ` Ping: " Alejandro Colomar (man-pages)
2021-11-23 11:32 ` [cfe-dev] " Dmitri Gribenko
2021-11-23 11:17 ` Dmitri Gribenko
2021-11-23 11:45 ` Alejandro Colomar (man-pages)
2021-11-23 12:45 ` Dmitri Gribenko
2021-12-01 22:24 ` Alejandro Colomar (man-pages)
2021-12-02 0:39 ` Dmitri Gribenko
2021-12-02 1:00 ` Alejandro Colomar (man-pages)
2021-12-02 20:24 ` Alejandro Colomar (man-pages)
2021-12-02 20:31 ` Alejandro Colomar (man-pages)
2021-12-02 20:36 ` Joseph Myers
2021-11-16 9:30 ` Jonathan Wakely
2021-11-16 17:13 ` [cfe-dev] " Arthur O'Dwyer
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).