Problems with default _POSIX_C_SOURCE in features.h ??

Problems with default _POSIX_C_SOURCE in features.h ??

[Chris Hall]

features.h (in glibc 2.29, 31-Jan-2019) includes:
______________________________________________________________________
/* If none of the ANSI/POSIX macros are defined, or if _DEFAULT_SOURCE
    is defined, use POSIX.1-2008 (or another version depending on
    _XOPEN_SOURCE).  */
#ifdef _DEFAULT_SOURCE
# if !defined _POSIX_SOURCE && !defined _POSIX_C_SOURCE
#  define __USE_POSIX_IMPLICITLY	1
# endif
# undef  _POSIX_SOURCE
# define _POSIX_SOURCE	1
# undef  _POSIX_C_SOURCE
# define _POSIX_C_SOURCE	200809L
#endif

#if ((!defined __STRICT_ANSI__					\
       || (defined _XOPEN_SOURCE && (_XOPEN_SOURCE - 0) >= 500))	\
      && !defined _POSIX_SOURCE && !defined _POSIX_C_SOURCE)
# define _POSIX_SOURCE	1
# if defined _XOPEN_SOURCE && (_XOPEN_SOURCE - 0) < 500
#  define _POSIX_C_SOURCE	2
# elif defined _XOPEN_SOURCE && (_XOPEN_SOURCE - 0) < 600
#  define _POSIX_C_SOURCE	199506L
# elif defined _XOPEN_SOURCE && (_XOPEN_SOURCE - 0) < 700
#  define _POSIX_C_SOURCE	200112L
# else
#  define _POSIX_C_SOURCE	200809L
# endif
# define __USE_POSIX_IMPLICITLY	1
#endif
______________________________________________________________________

But this does not do what the comment says it does :-(

If _DEFAULT_SOURCE is defined, but neither POSIX_SOURCE nor
_POSIX_C_SOURCE are defined, this sets _POSIX_C_SOURCE	200809L,
no matter what _XOPEN_SOURCE is.

Bug #1 ??


Further on in features.h we have:
_______________________________________________________________________
#if defined _POSIX_C_SOURCE && _POSIX_C_SOURCE >= 2 || \
                                                  defined _XOPEN_SOURCE
# define __USE_POSIX2   1
#endif

#if defined _POSIX_C_SOURCE && (_POSIX_C_SOURCE - 0) >= 199309L
# define __USE_POSIX199309  1
#endif

#if defined _POSIX_C_SOURCE && (_POSIX_C_SOURCE - 0) >= 199506L
# define __USE_POSIX199506  1
#endif
_______________________________________________________________________

If, say, _XOPEN_SOURCE == 500 the rules say that it is not necessary to 
define _POSIX_SOURCE or _POSIX_C_SOURCE, but whatever they are set to, 
the effect as if _POSIX_C_SOURCE == 199506L (except that if 
_POSIX_C_SOURCE > 199506L, where the effect is undefined).

Should one or both of __USE_POSIX199309 and __USE_POSIX199506 be set if 
_XOPEN_SOURCE == 500 ?

Bug #2 ??


In cases where _POSIX_C_SOURCE is bigger than expected for the given 
_XOPEN_SOURCE the result does seem to be a mess -- but, hey, the 
standard says it's undefined !

Am I missing something, or should I report these as bugs ?

Chris

Re: Problems with default _POSIX_C_SOURCE in features.h ??

[Adhemerval Zanella]

On 26/03/2019 17:28, Chris Hall wrote:
> 
> features.h (in glibc 2.29, 31-Jan-2019) includes:
> ______________________________________________________________________
> /* If none of the ANSI/POSIX macros are defined, or if _DEFAULT_SOURCE
>    is defined, use POSIX.1-2008 (or another version depending on
>    _XOPEN_SOURCE).  */
> #ifdef _DEFAULT_SOURCE
> # if !defined _POSIX_SOURCE && !defined _POSIX_C_SOURCE
> #  define __USE_POSIX_IMPLICITLY    1
> # endif
> # undef  _POSIX_SOURCE
> # define _POSIX_SOURCE    1
> # undef  _POSIX_C_SOURCE
> # define _POSIX_C_SOURCE    200809L
> #endif
> 
> #if ((!defined __STRICT_ANSI__                    \
>       || (defined _XOPEN_SOURCE && (_XOPEN_SOURCE - 0) >= 500))    \
>      && !defined _POSIX_SOURCE && !defined _POSIX_C_SOURCE)
> # define _POSIX_SOURCE    1
> # if defined _XOPEN_SOURCE && (_XOPEN_SOURCE - 0) < 500
> #  define _POSIX_C_SOURCE    2
> # elif defined _XOPEN_SOURCE && (_XOPEN_SOURCE - 0) < 600
> #  define _POSIX_C_SOURCE    199506L
> # elif defined _XOPEN_SOURCE && (_XOPEN_SOURCE - 0) < 700
> #  define _POSIX_C_SOURCE    200112L
> # else
> #  define _POSIX_C_SOURCE    200809L
> # endif
> # define __USE_POSIX_IMPLICITLY    1
> #endif
> ______________________________________________________________________
> 
> But this does not do what the comment says it does :-(
> 
> If _DEFAULT_SOURCE is defined, but neither POSIX_SOURCE nor
> _POSIX_C_SOURCE are defined, this sets _POSIX_C_SOURCE    200809L,
> no matter what _XOPEN_SOURCE is.
> 
> Bug #1 ??

I think the issue is the comment itself, since on glibc doc it states:

manual/creature.texi

219 @defvr Macro _DEFAULT_SOURCE
220 @standards{GNU, (none)}
221 If you define this macro, most features are included apart from
222 X/Open, LFS and GNU extensions: the effect is to enable features from
223 the 2008 edition of POSIX, as well as certain BSD and SVID features
224 without a separate feature test macro to control them.

So my understanding is the intentio of _DEFAULT_SOURCE is indeed to
support current POSIX 2008 (or the latest release) regardless of the
_XOPEN_SOURCE value.

> 
> 
> Further on in features.h we have:
> _______________________________________________________________________
> #if defined _POSIX_C_SOURCE && _POSIX_C_SOURCE >= 2 || \
>                                                  defined _XOPEN_SOURCE
> # define __USE_POSIX2   1
> #endif
> 
> #if defined _POSIX_C_SOURCE && (_POSIX_C_SOURCE - 0) >= 199309L
> # define __USE_POSIX199309  1
> #endif
> 
> #if defined _POSIX_C_SOURCE && (_POSIX_C_SOURCE - 0) >= 199506L
> # define __USE_POSIX199506  1
> #endif
> _______________________________________________________________________
> 
> If, say, _XOPEN_SOURCE == 500 the rules say that it is not necessary to define _POSIX_SOURCE or _POSIX_C_SOURCE, but whatever they are set to, the effect as if _POSIX_C_SOURCE == 199506L (except that if _POSIX_C_SOURCE > 199506L, where the effect is undefined).
> 
> Should one or both of __USE_POSIX199309 and __USE_POSIX199506 be set if _XOPEN_SOURCE == 500 ?

My understanding is __USE_POSIX199309 refers internally to POSIX.1b Real-time
extensions and __USE_POSIX199506 to POSIX.1c Threads extensions.  Their support
are orthogonal and my understanding if since glibc supports both I see no reason
to not advertise them for _XOPEN_SOURCE == 500.

> 
> Bug #2 ??
> 
> 
> In cases where _POSIX_C_SOURCE is bigger than expected for the given _XOPEN_SOURCE the result does seem to be a mess -- but, hey, the standard says it's undefined !
> 
> Am I missing something, or should I report these as bugs ?
> 
> Chris

Re: Problems with default _POSIX_C_SOURCE in features.h ??

[Adhemerval Zanella]

On 29/03/2019 12:13, Chris Hall wrote:
> On 28/03/2019 18:56, Adhemerval Zanella wrote:
>> On 26/03/2019 17:28, Chris Hall wrote:
>>>
>>> features.h (in glibc 2.29, 31-Jan-2019) includes:
>>> ______________________________________________________________________
>>> /* If none of the ANSI/POSIX macros are defined, or if _DEFAULT_SOURCE
>>>     is defined, use POSIX.1-2008 (or another version depending on
>>>     _XOPEN_SOURCE).  */
> ...
>>> But this does not do what the comment says it does :-(
>>>
>>> If _DEFAULT_SOURCE is defined, but neither POSIX_SOURCE nor
>>> _POSIX_C_SOURCE are defined, this sets _POSIX_C_SOURCE    200809L,
>>> no matter what _XOPEN_SOURCE is.
>>>
>>> Bug #1 ??
> 
>> I think the issue is the comment itself, since on glibc doc it states:
>>
>> manual/creature.texi
>>
>> 219 @defvr Macro _DEFAULT_SOURCE
>> 220 @standards{GNU, (none)}
>> 221 If you define this macro, most features are included apart from
>> 222 X/Open, LFS and GNU extensions: the effect is to enable features from
>> 223 the 2008 edition of POSIX, as well as certain BSD and SVID features
>> 224 without a separate feature test macro to control them.
>>
>> So my understanding is the intentio of _DEFAULT_SOURCE is indeed to
>> support current POSIX 2008 (or the latest release) regardless of the
>> _XOPEN_SOURCE value.
> 
> Well, the manual doesn't actually say how _DEFAULT_SOURCE interacts with _XOPEN_SOURCE and _POSIX_C_SOURCE if one or both are defined.
> 
> I observe that _GNU_SOURCE discards any _XOPEN_SOURCE and/or _POSIX_C_SOURCE settings -- it forces 700 and 200809L, respectively. Though the manual does not specify that, either.

The manual implicit does state that:

"If you define this macro, everything is included: ISO C89, ISO C99, 
*POSIX.1*, POSIX.2, BSD, SVID, X/Open, LFS, and GNU extensions."

I give you that it might be clear that _XOPEN_SOURCE and/or 
_POSIX_C_SOURCE would be ignore to pre-defined values.

> 
> [So it is not possible to compile as if for an older _GNU_SOURCE (with, say, 600 and 200112L).  Perhaps that's really not useful ?]

My understanding it there is no concept of _GNU_SOURCE version, as for
_XOPEN_SOURCE for instance. And I am not sure how useful would be to 
start using this concept, since nowadays programs that use _GNU_SOURCE
does not set it to a particular value.  

I am not aware of any changes being planned to support POSIX.1-2017,
neither if it would require additional FTM (afaik there is no need). 
If it were the case, it might be require to discuss to how handle it 
in along with _GNU_SOURCE.

> 
> I note that:
> 
>   -D_DEFAULT_SOURCE -D_POSIX_C_SOURCE=200112L
> 
> throws away the given _POSIX_C_SOURCE and forces 200809L.  (So that's similar to _GNU_SOURCE.)
> 
> But I also note that:
> 
>   -D_DEFAULT_SOURCE -D_XOPEN_SOURCE=600
> 
> does not throw away the _XOPEN_SOURCE, and the result is _POSIX_C_SOURCE 200809L with _XOPEN_SOURCE 600, which is undefined :-(
> 
> I'm pretty sure that's a bug !

My understanding is _POSIX_C_SOURCE and _XOPEN_SOURCE are tests are
orthogonal and it will enable the most recent features you enable.  
The  -D_DEFAULT_SOURCE -D_XOPEN_SOURCE=600 will set to higher
standard you set, which is _POSIX_C_SOURCE=200809L from _DEFAULT_SOURCE.
It will define as expected:

#define __USE_POSIX 1
#define __USE_POSIX2 1
#define __USE_POSIX199309 1
#define __USE_POSIX199506 1
#define __USE_XOPEN2K 1
#define __USE_XOPEN2K8 1
#define __USE_XOPEN_EXTENDED 1
#define __USE_UNIX98 1
#define __USE_XOPEN2KXSI 1

> 
> Perhaps it is not useful to ask for _DEFAULT_SOURCE and _XOPEN_SOURCE ?

That my point: it does not make sense to set _DEFAULT_SOURCE, which will
set _POSIX_C_SOURCE=200809L, and at same time set _XOPEN_SOURCE=600, which
will set _POSIX_C_SOURCE=200112L.  

GLIBC current behaviour is just assume the most recent one, which I think
it is reasonable. Either way, I see no point or changing this semantic
(to assume lest recent one) or to dump either errors or warning for the
case of mismatched selection.

> 
> Perhaps _DEFAULT_SOURCE should act like _GNU_SOURCE and override both _POSIX_C_SOURCE *and* _XOPEN_SOURCE ?  If _XOPEN_SOURCE is defined it could either be #undef'd or forced to 700.
> 

No sure what exactly do we gain for such change, and although I think a
possible build breakage to be unlikely it is still a gratuitous change
of semantic for _DEFAULT_SOURCE.  Do we really need it (this lwn has
some history of the change https://lwn.net/Articles/590381/ and we try
to break things even it is just extra compile warning)? 

Re: Problems with default _POSIX_C_SOURCE in features.h ??

[Chris Hall]

On 29/03/2019 17:54, Adhemerval Zanella wrote:
...
> My understanding is _POSIX_C_SOURCE and _XOPEN_SOURCE are tests are
> orthogonal and it will enable the most recent features you enable.
> The  -D_DEFAULT_SOURCE -D_XOPEN_SOURCE=600 will set to higher
> standard you set, which is _POSIX_C_SOURCE=200809L from  > _DEFAULT_SOURCE.

My reading of the standards SUSv1 through SUSv4 is that _POSIX_C_SOURCE 
selects a subset of the corresponding _XOPEN_SOURCE.  So, if I ask for 
-D_XOPEN_SOURCE=600, I am asking for _POSIX_C_SOURCE=200112L with all 
the XSI additions and subtractions of SUSv3.  If I were to ask for 
-D_XOPEN_SOURCE=600 with _POSIX_C_SOURCE=200809L, I would be asking for 
the non-XSI subset of SUSv4 with all the XSI additions and subtractions 
of SUSv3.  The standard does not define the 12 mismatched combinations 
of non-XSI and XSI parts of SUSv1 through to SUSv4 (and 27 such 
combinations when SUSv5 arrives...) !!

Now, the standard says _XOPEN_SOURCE takes precedence.  Let "psx(X)" be 
the _POSIX_C_SOURCE corresponding to _XOPEN_SOURCE=X.  If I ask for 
-D_XOPEN_SOURCE=X with -D_POSIX_C_SOURCE=P then P is (effectively) 
ignored iff P <= psx(X).  That would leave (just) 6 possible 
combinations where P > psx(X) (and 10 with SUSv5).  The standard does 
not attempt to tackle the complexity -- those combinations are *undefined*.

Of course, glibc may support the undefined combinations.  The manual 
does not mention it -- though that is not a strong statement :-(

Perhaps glibc takes -D_XOPEN_SOURCE=X with -D_POSIX_C_SOURCE=P, and 
promotes the older to match the newer ?

But, as you say, for -D_DEFAULT_SOURCE -D_XOPEN_SOURCE=600:

> It will define as expected:
> 
> #define __USE_POSIX 1
> #define __USE_POSIX2 1
> #define __USE_POSIX199309 1
> #define __USE_POSIX199506 1
> #define __USE_XOPEN2K 1
> #define __USE_XOPEN2K8 1
> #define __USE_XOPEN_EXTENDED 1
> #define __USE_UNIX98 1
> #define __USE_XOPEN2KXSI 1

where __USE_XOPEN2K8XSI is not defined -- so if glibc is promoting the 
_XOPEN_SOURCE to 700, it's not obvious.

>> Perhaps it is not useful to ask for _DEFAULT_SOURCE and _XOPEN_SOURCE ?
> That my point: it does not make sense to set _DEFAULT_SOURCE, which will
> set _POSIX_C_SOURCE=200809L, and at same time set _XOPEN_SOURCE=600, which
> will set _POSIX_C_SOURCE=200112L.

Except that _DEFAULT_SOURCE is (apparently) the only way of turning on 
__USE_MISC.

....
>> Perhaps _DEFAULT_SOURCE should act like _GNU_SOURCE and override both
>> _POSIX_C_SOURCE *and* _XOPEN_SOURCE ?  If _XOPEN_SOURCE is defined it
>> could either be #undef'd or forced to 700.

> No sure what exactly do we gain for such change, and although I think a
> possible build breakage to be unlikely it is still a gratuitous change
> of semantic for _DEFAULT_SOURCE.

We agree that, given what _DEFAULT_SOURCE does, it makes no sense to 
also ask for _XOPEN_SOURCE=X, unless X matches the current latest and 
greatest _POSIX_C_SOURCE (known to glibc).

Perhaps:

   1. _DEFAULT_SOURCE is intended to exclude XSI altogether.

      The documentation suggests that is the case.

      So specifying *any* _XOPEN_SOURCE makes no sense.

      I would #undef _XOPEN_SOURCE to be consistent with _GNU_SOURCE,
      and be unambiguous.

   2. _DEFAULT_SOURCE is intended to not include XSI, but
      _XOPEN_SOURCE may be used to add same.

      The documentation does not rule this out.

      If _XOPEN_SOURCE is defined, I would force it to 700,
      to also be consistent with _GNU_SOURCE and be unambiguous.

   3. _XOPEN_SOURCE implies __USE_MISC

      In which case there is no need to use _DEFAULT_SOURCE and
      _XOPEN_SOURCE together.

      Given the "default" nature of _DEFAULT_SOURCE, I would #undef
      it if _XOPEN_SOURCE were specified.

      BUT: I note that _GNU_SOURCE => _DEFAULT_SOURCE => __USE_MISC.

           I have no idea whether that is significant.  But it does
           suggest that _XOPEN_SOURCE does *not* imply __USE_MISC.

But, yes, there could be somebody out there using (say) 
-D_XOPEN_SOURCE=600 -D_DEFAULT_SOURCE even though it makes no sense.

> Do we really need it (this lwn has
> some history of the change https://lwn.net/Articles/590381/ and we try
> to break things even it is just extra compile warning)?

Re: Problems with default _POSIX_C_SOURCE in features.h ??

[Chris Hall]

On 29/03/2019 17:54, Adhemerval Zanella wrote:
...

Following up a little further...

> ... this lwn has some history of the change 
>     https://lwn.net/Articles/590381/ ...

...I have now read the article.

I note that it talks of removing uses of __USE_MISC where it is 
redundant.  In particular, the article notes:

   "all cases where __USE_MISC is defined in glibc 2.20,
    __USE_XOPEN2K8 is also defined"

This is true because _DEFAULT_SOURCE => __USE_XOPEN2K8.

By removing __USE_MISC in these cases, it is made impossible (or at 
least very difficult) to change features.h to allow for __USE_MISC with 
an earlier POSIX_C_SOURCE.

------------------------------------------------------------------------

It is possible that my worry is a symptom of ignorance.

I have a large network application which is required to compile for more 
or less any UN*X.  I develop and test on Linux.  I would like the 
minimum of surprise when compiling and testing on other systems.

Now, FreeBSD has caused me some trouble in the past.  Recent FreeBSD 
admits to POSIX.1-2001 (more or less), but denies X/Open.  If you 
compile *without* specifying any _XXX_SOURCE macros (_POSIX_C_SOURCE, 
_XOPEN_SOURCE, POSIX_SOURCE, _ANSI_SOURCE, _C99_SOURCE and _C11_SOURCE) 
then you get:

   #define __POSIX_VISIBLE  200809
   #define __XSI_VISIBLE	   700
   #define __BSD_VISIBLE	   1
   #define __ISO_C_VISIBLE  2011
   #define __EXT1_VISIBLE   1

which appear to select everything available, including some XSI things. 
But you also get:

   #define _POSIX_VERSION   200112L
   #define _XOPEN_UNIX      -1

but _XOPEN_VERSION is not defined.

So, I figured that on Linux, with glibc, I could:

   -D_XOPEN_SOURCE=600 -D_BSD_SOURCE

and that might approximate to FreeBSD, and other UN*X which more or less 
conform to POSIX.1-2001 or are certified UNIX 03.

But late model glibc won't let me do that :-(

Perhaps I really don't need -D_BSD_SOURCE (now -D_DEFAULT_SOURCE) with 
POSIX.1-2001 ?

Perhaps I am, in any case, wasting my time ?  Perhaps the reality is 
that I cannot avoid being caught out occasionally by differences between 
systems... and I should just be grateful it's only occasionally.

If so, I might as well compile for the latest and greatest 
_XOPEN_SOURCE, and trade the advantages of recent features against the 
(greater) risk of surprise.

Of course, glibc encourages the use of _GNU_SOURCE, which selects the 
latest and greatest _XOPEN_SOURCE.  But, it also enables all the GNU 
libc specials, which I really don't want.  It also seems to select C11 
things, even if the compiler is not __STDC_VERSION__ 201112L, which I 
cannot say I like the look of, either.

I'd like a way to select the latest and greatest _XOPEN_SOURCE and/or 
_POSIX_C_SOURCE, while not selecting any GNU extensions.  Perhaps, say, 
_XOPEN_SOURCE_LATEST and/or _POSIX_SOURCE_LATEST ?

[I guess I could require users to install glibc, and surrender to the 
_GNU_SOURCE... but that doesn't feel right.]

Suggestions, anyone ?

Thanks,

Chris

Re: Problems with default _POSIX_C_SOURCE in features.h ??

[Adhemerval Zanella]

On 01/04/2019 07:11, Chris Hall wrote:
> On 29/03/2019 17:54, Adhemerval Zanella wrote:
> ...
>> My understanding is _POSIX_C_SOURCE and _XOPEN_SOURCE are tests are
>> orthogonal and it will enable the most recent features you enable.
>> The  -D_DEFAULT_SOURCE -D_XOPEN_SOURCE=600 will set to higher
>> standard you set, which is _POSIX_C_SOURCE=200809L from  > _DEFAULT_SOURCE.
> 
> My reading of the standards SUSv1 through SUSv4 is that _POSIX_C_SOURCE selects a subset of the corresponding _XOPEN_SOURCE.  So, if I ask for -D_XOPEN_SOURCE=600, I am asking for _POSIX_C_SOURCE=200112L with all the XSI additions and subtractions of SUSv3.  If I were to ask for -D_XOPEN_SOURCE=600 with _POSIX_C_SOURCE=200809L, I would be asking for the non-XSI subset of SUSv4 with all the XSI additions and subtractions of SUSv3.  The standard does not define the 12 mismatched combinations of non-XSI and XSI parts of SUSv1 through to SUSv4 (and 27 such combinations when SUSv5 arrives...) !!

Indeed POSIX states that XSI (defined by _XOPEN_SOURCE) is a superset of the
mandatory requirements for conformance to POSIX.1 (defined by _POSIX_C_SOURCE).

Now, from my understanding of POSIX.1-2001 [1] conformance it does *not* really
define the combination you use as example, -D_XOPEN_SOURCE=600 with 
_POSIX_C_SOURCE=200809L. It does not fall neither on 'Strictly Conforming POSIX 
Application' (which shall define _POSIX_C_SOURCE to be 200112L) nor on 'Strictly
Conforming XSI Application' category ('which shall define  _XOPEN_SOURCE to be 600).

Also, based on POSIX.1-2017 I would say this behaviour is now currently stated
as unspecified, since it states that:

"However, should _POSIX_C_SOURCE be set to a value greater than 200809L, the 
behavior is unspecified."

Now, I am not sure which would be best course of action on glibc for such
unspecified combinations. Current practice is to follow your description to
conditionality older XSI extension if where the case.

[1] http://pubs.opengroup.org/onlinepubs/009695399/basedefs/xbd_chap02.html
[2] http://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_chap02.html

> 
> Now, the standard says _XOPEN_SOURCE takes precedence.  Let "psx(X)" be the _POSIX_C_SOURCE corresponding to _XOPEN_SOURCE=X.  If I ask for -D_XOPEN_SOURCE=X with -D_POSIX_C_SOURCE=P then P is (effectively) ignored iff P <= psx(X).  That would leave (just) 6 possible combinations where P > psx(X) (and 10 with SUSv5).  The standard does not attempt to tackle the complexity -- those combinations are *undefined*.
> 
> Of course, glibc may support the undefined combinations.  The manual does not mention it -- though that is not a strong statement :-(
> 
> Perhaps glibc takes -D_XOPEN_SOURCE=X with -D_POSIX_C_SOURCE=P, and promotes the older to match the newer ?
> 
> But, as you say, for -D_DEFAULT_SOURCE -D_XOPEN_SOURCE=600:
> 
>> It will define as expected:
>>
>> #define __USE_POSIX 1
>> #define __USE_POSIX2 1
>> #define __USE_POSIX199309 1
>> #define __USE_POSIX199506 1
>> #define __USE_XOPEN2K 1
>> #define __USE_XOPEN2K8 1
>> #define __USE_XOPEN_EXTENDED 1
>> #define __USE_UNIX98 1
>> #define __USE_XOPEN2KXSI 1
> 
> where __USE_XOPEN2K8XSI is not defined -- so if glibc is promoting the _XOPEN_SOURCE to 700, it's not obvious.

I think the issue is -D_DEFAULT_SOURCE -D_XOPEN_SOURCE=600 from glibc
documentation states _DEFAULT_SOURCE will implicit sets _POSIX_C_SOURCE=200809L 
is also undefined (since for _POSIX_C_SOURCE=200809L it expects 
-D_XOPEN_SOURCE=700 to be a Strictly Conforming XSI Application).

Now what glibc does is:

$ gcc -Wall test.c -D_POSIX_C_SOURCE=200809L -D_XOPEN_SOURCE=600 -E -dD | grep __USE_XOPEN | grep define
#define __USE_XOPEN2K 1
#define __USE_XOPEN2K8 1
#define __USE_XOPEN 1
#define __USE_XOPEN_EXTENDED 1
#define __USE_XOPEN2K 1
#define __USE_XOPEN2KXSI 1

$ gcc -Wall test.c -D_POSIX_C_SOURCE=200809L -D_XOPEN_SOURCE=700 -E -dD | grep __USE_XOPEN | grep define
#define __USE_XOPEN2K 1
#define __USE_XOPEN2K8 1
#define __USE_XOPEN 1
#define __USE_XOPEN_EXTENDED 1
#define __USE_XOPEN2K8 1
#define __USE_XOPEN2K8XSI 1
#define __USE_XOPEN2K 1
#define __USE_XOPEN2KXSI 1

$ gcc -Wall test.c -D_DEFAULT_SOURCE  -E -dD | grep __USE_XOPEN | grep define
#define __USE_XOPEN2K 1
#define __USE_XOPEN2K8 1

$ gcc -Wall test.c -D_DEFAULT_SOURCE -D_XOPEN_SOURCE=600 -E -dD | grep __USE_XOPEN | grep define
#define __USE_XOPEN2K 1
#define __USE_XOPEN2K8 1
#define __USE_XOPEN 1
#define __USE_XOPEN_EXTENDED 1
#define __USE_XOPEN2K 1
#define __USE_XOPEN2KXSI 1

$ gcc -Wall test.c -D_DEFAULT_SOURCE -D_XOPEN_SOURCE=700 -E -dD | grep __USE_XOPEN | grep define
#define __USE_XOPEN2K 1
#define __USE_XOPEN2K8 1
#define __USE_XOPEN 1
#define __USE_XOPEN_EXTENDED 1
#define __USE_XOPEN2K8 1
#define __USE_XOPEN2K8XSI 1
#define __USE_XOPEN2K 1
#define __USE_XOPEN2KXSI 1

So my understanding is glibc is not promoting the -D_DEFAULT_SOURCE -D_XOPEN_SOURCE=600
to _XOPEN_SOURCE=700 and it is indeed behaving as the manual states which is only
setting -D_POSIX_C_SOURCE=200809L.


> 
>>> Perhaps it is not useful to ask for _DEFAULT_SOURCE and _XOPEN_SOURCE ?
>> That my point: it does not make sense to set _DEFAULT_SOURCE, which will
>> set _POSIX_C_SOURCE=200809L, and at same time set _XOPEN_SOURCE=600, which
>> will set _POSIX_C_SOURCE=200112L.
> 
> Except that _DEFAULT_SOURCE is (apparently) the only way of turning on __USE_MISC.

The _GNU_SOURCE, which also defines _DEFAULT_SOURCE, also defines __USE_MISC.

> 
> ....
>>> Perhaps _DEFAULT_SOURCE should act like _GNU_SOURCE and override both
>>> _POSIX_C_SOURCE *and* _XOPEN_SOURCE ?  If _XOPEN_SOURCE is defined it
>>> could either be #undef'd or forced to 700.
> 
>> No sure what exactly do we gain for such change, and although I think a
>> possible build breakage to be unlikely it is still a gratuitous change
>> of semantic for _DEFAULT_SOURCE.
> 
> We agree that, given what _DEFAULT_SOURCE does, it makes no sense to also ask for _XOPEN_SOURCE=X, unless X matches the current latest and greatest _POSIX_C_SOURCE (known to glibc).
> 
> Perhaps:
> 
>   1. _DEFAULT_SOURCE is intended to exclude XSI altogether.
> 
>      The documentation suggests that is the case.
> 
>      So specifying *any* _XOPEN_SOURCE makes no sense.
> 
>      I would #undef _XOPEN_SOURCE to be consistent with _GNU_SOURCE,
>      and be unambiguous.

That's not the idea, _DEFAULT_SOURCE does not *include* the XSI extension
but also does not *exclude* them.  It is still possible to include the XSI
extension if is is the intention.

> 
>   2. _DEFAULT_SOURCE is intended to not include XSI, but
>      _XOPEN_SOURCE may be used to add same.
> 
>      The documentation does not rule this out.
> 
>      If _XOPEN_SOURCE is defined, I would force it to 700,
>      to also be consistent with _GNU_SOURCE and be unambiguous.

Again, I do not think it would be safer or yield much gains to force either
_DEFAULT_SOURCE or _POSIX_C_SOURCE=200809L to set _XOPEN_SOURCE to 700.  Since
it is undefined by standard and has been defined by glibc, it will most likely
crate incompatibilities if we change it and it is not easy to create a 
compatibility layer for such cases.

> 
>   3. _XOPEN_SOURCE implies __USE_MISC
> 
>      In which case there is no need to use _DEFAULT_SOURCE and
>      _XOPEN_SOURCE together.
> 
>      Given the "default" nature of _DEFAULT_SOURCE, I would #undef
>      it if _XOPEN_SOURCE were specified.
> 
>      BUT: I note that _GNU_SOURCE => _DEFAULT_SOURCE => __USE_MISC.
> 
>           I have no idea whether that is significant.  But it does
>           suggest that _XOPEN_SOURCE does *not* imply __USE_MISC.

No, __USE_MISC is used to add *GNU* extension and tying it to _XOPEN_SOURCE
is definitely wrong. It is safe for _DEFAULT_SOURCE because it is also a
glibc extension.

> 
> But, yes, there could be somebody out there using (say) -D_XOPEN_SOURCE=600 -D_DEFAULT_SOURCE even though it makes no sense.
> 
>> Do we really need it (this lwn has
>> some history of the change https://lwn.net/Articles/590381/ and we try
>> to break things even it is just extra compile warning)?
> 
> 
> 

Re: Problems with default _POSIX_C_SOURCE in features.h ??

[Chris Hall]

On 09/04/2019 14:09, Adhemerval Zanella wrote:
> On 01/04/2019 07:11, Chris Hall wrote:
>> On 29/03/2019 17:54, Adhemerval Zanella wrote:
...
>>> My understanding is _POSIX_C_SOURCE and _XOPEN_SOURCE are tests are
>>> orthogonal and it will enable the most recent features you enable.
>>> The  -D_DEFAULT_SOURCE -D_XOPEN_SOURCE=600 will set to higher
>>> standard you set, which is _POSIX_C_SOURCE=200809L from 
>>> _DEFAULT_SOURCE.
...
> Now, I am not sure which would be best course of action on glibc for such
> unspecified combinations. Current practice is to follow your description to
> conditionality older XSI extension if where the case.

I think we are agreed that _POSIX_C_SOURCE and _XOPEN_SOURCE are not 
orthogonal.

...
> So my understanding is glibc is not promoting the -D_DEFAULT_SOURCE
> -D_XOPEN_SOURCE=600 to _XOPEN_SOURCE=700 and it is indeed behaving 
> as the manual states which is only setting -D_POSIX_C_SOURCE=200809L.

I think we are agreed that -D_DEFAULT_SOURCE -D_XOPEN_SOURCE=600 select 
an undefined combination of _POSIX_C_SOURCE and _XOPEN_SOURCE.

Setting "the 2008 edition of POSIX" is indeed what the manual says.

So asking for -D_DEFAULT_SOURCE -D_XOPEN_SOURCE=600 is to ask for 
something which is undefined.

However, glibc goes to some effort to support the selection of older 
_POSIX_C_SOURCE and/or _XOPEN_SOURCE.  But *not* with __USE_MISC and 
*not* with __USE_GNU.  I have failed to discover why that is -- though 
that is probably because I am too stupid.

...
>> Except that _DEFAULT_SOURCE is (apparently) the only way of 
>> turning on __USE_MISC.

> The _GNU_SOURCE, which also defines _DEFAULT_SOURCE, also 
> defines __USE_MISC.

True.  But I don't want GNU extensions -- so I was ignoring that.  I 
apologise for not being precise.

...
>> Perhaps:
>>
>>    1. _DEFAULT_SOURCE is intended to exclude XSI altogether.
...
> That's not the idea, _DEFAULT_SOURCE does not *include* the 
> XSI extension but also does not *exclude* them.  It is still
> possible to include the XSI extension if is is the intention.

OK, so ...

>>    2. _DEFAULT_SOURCE is intended to not include XSI, but
>>       _XOPEN_SOURCE may be used to add same.
>>
>>       The documentation does not rule this out.

... -D_DEFAULT_SOURCE -D_XOPEN_SOURCE=700 is an expected/supported 
combination (currently).

Mind you, when _POSIX_C_SOURCE=202101 (say) arrives and _DEFAULT_SOURCE 
selects it, -D_DEFAULT_SOURCE -D_XOPEN_SOURCE=700 will then be an 
undefined/unsupported combination :-(

So, to correctly select XSI with _DEFAULT_SOURCE, I have to work on my 
configure script, to check (a) what version of _POSIX_C_SOURCE will be 
selected by _DEFAULT_SOURCE, and (b) what -D_XOPEN_SOURCE=??? is 
required to (correctly) enable XSI.  Hurrah !

>>       If _XOPEN_SOURCE is defined, I would force it to 700,
>>       to also be consistent with _GNU_SOURCE and be unambiguous.

> Again, I do not think it would be safer or yield much gains to
> force either _DEFAULT_SOURCE or _POSIX_C_SOURCE=200809L to set
> _XOPEN_SOURCE to 700.  Since it is undefined by standard and 
> has been defined by glibc, it will most likely crate 
> incompatibilities if we change it and it is not easy to 
> create a compatibility layer for such cases.

Sorry, I don't understand this.

My point is: if I ask for XSI, by -D_DEFAULT_SOURCE -D_XOPEN_SOURCE=xxx, 
I think it would be helpful if _DEFAULT_SOURCE selected the latest and 
greatest _XOPEN_SOURCE, to go with the latest and greatest 
_POSIX_C_SOURCE which _DEFAULT_SOURCE  is also selecting...

...so that, when _POSIX_C_SOURCE=202101 (say) arrives, _DEFAULT_SOURCE 
would then set the corresponding _XOPEN_SOURCE=800.

I agree that anybody currently asking for (say) -D_DEFAULT_SOURCE 
-D_XOPEN_SOURCE=600 might be surprised by the new result.  Though I 
think we are agreed that what they currently get is undefined :-(

Chris

Re: Problems with default _POSIX_C_SOURCE in features.h ??

[Adhemerval Zanella]

On 09/04/2019 14:12, Chris Hall wrote:
> On 09/04/2019 14:09, Adhemerval Zanella wrote:
>> On 01/04/2019 07:11, Chris Hall wrote:
>>> On 29/03/2019 17:54, Adhemerval Zanella wrote:
> ...
>>>> My understanding is _POSIX_C_SOURCE and _XOPEN_SOURCE are tests are
>>>> orthogonal and it will enable the most recent features you enable.
>>>> The  -D_DEFAULT_SOURCE -D_XOPEN_SOURCE=600 will set to higher
>>>> standard you set, which is _POSIX_C_SOURCE=200809L from _DEFAULT_SOURCE.
> ...
>> Now, I am not sure which would be best course of action on glibc for such
>> unspecified combinations. Current practice is to follow your description to
>> conditionality older XSI extension if where the case.
> 
> I think we are agreed that _POSIX_C_SOURCE and _XOPEN_SOURCE are not orthogonal.
> 
> ...
>> So my understanding is glibc is not promoting the -D_DEFAULT_SOURCE
>> -D_XOPEN_SOURCE=600 to _XOPEN_SOURCE=700 and it is indeed behaving as the manual states which is only setting -D_POSIX_C_SOURCE=200809L.
> 
> I think we are agreed that -D_DEFAULT_SOURCE -D_XOPEN_SOURCE=600 select an undefined combination of _POSIX_C_SOURCE and _XOPEN_SOURCE.
> 
> Setting "the 2008 edition of POSIX" is indeed what the manual says.
> 
> So asking for -D_DEFAULT_SOURCE -D_XOPEN_SOURCE=600 is to ask for something which is undefined.
> 
> However, glibc goes to some effort to support the selection of older _POSIX_C_SOURCE and/or _XOPEN_SOURCE.  But *not* with __USE_MISC and *not* with __USE_GNU.  I have failed to discover why that is -- though that is probably because I am too stupid.

My understanding is because both __USE_MISC and __USE_GNU are internal flags 
not meant to be set by external programs and they can be removed/added in
future glibc releases.

> 
> ...
>>> Except that _DEFAULT_SOURCE is (apparently) the only way of turning on __USE_MISC.
> 
>> The _GNU_SOURCE, which also defines _DEFAULT_SOURCE, also defines __USE_MISC.
> 
> True.  But I don't want GNU extensions -- so I was ignoring that.  I apologise for not being precise.
> 
> ...
>>> Perhaps:
>>>
>>>    1. _DEFAULT_SOURCE is intended to exclude XSI altogether.
> ...
>> That's not the idea, _DEFAULT_SOURCE does not *include* the XSI extension but also does not *exclude* them.  It is still
>> possible to include the XSI extension if is is the intention.
> 
> OK, so ...
> 
>>>    2. _DEFAULT_SOURCE is intended to not include XSI, but
>>>       _XOPEN_SOURCE may be used to add same.
>>>
>>>       The documentation does not rule this out.
> 
> ... -D_DEFAULT_SOURCE -D_XOPEN_SOURCE=700 is an expected/supported combination (currently).
> 
> Mind you, when _POSIX_C_SOURCE=202101 (say) arrives and _DEFAULT_SOURCE selects it, -D_DEFAULT_SOURCE -D_XOPEN_SOURCE=700 will then be an undefined/unsupported combination :-(
> 
> So, to correctly select XSI with _DEFAULT_SOURCE, I have to work on my configure script, to check (a) what version of _POSIX_C_SOURCE will be selected by _DEFAULT_SOURCE, and (b) what -D_XOPEN_SOURCE=??? is required to (correctly) enable XSI.  Hurrah !

The _DEFAULT_SOURCE was added on glibc 2.19 (commit c688b4196014e016) and
explicit stated that it implies -D_BSD_SOURCE -D_SVID_SOURCE 
-D_POSIX_C_SOURCE=200809L.  Also, glibc 2.20 changed change _BSD_SOURCE and 
_SVID_SOURCE to define _DEFAULT_SOURCE and later on 2.28 take in 
consideration _ISOC11_SOURCE as well.

So _DEFAULT_SOURCE always define -D_POSIX_C_SOURCE=200809L without 
implicit _XOPEN_SOURCE from glibc 2.19 to 2.30.  Afaik there is no need to
add a POSIX 2017 check, and it would require some discussion upstream
if it where the case to see how _DEFAULT_SOURCE will play along.

> 
>>>       If _XOPEN_SOURCE is defined, I would force it to 700,
>>>       to also be consistent with _GNU_SOURCE and be unambiguous.
> 
>> Again, I do not think it would be safer or yield much gains to
>> force either _DEFAULT_SOURCE or _POSIX_C_SOURCE=200809L to set
>> _XOPEN_SOURCE to 700.  Since it is undefined by standard and has been defined by glibc, it will most likely crate incompatibilities if we change it and it is not easy to create a compatibility layer for such cases.
> 
> Sorry, I don't understand this.
> 
> My point is: if I ask for XSI, by -D_DEFAULT_SOURCE -D_XOPEN_SOURCE=xxx, I think it would be helpful if _DEFAULT_SOURCE selected the latest and greatest _XOPEN_SOURCE, to go with the latest and greatest _POSIX_C_SOURCE which _DEFAULT_SOURCE  is also selecting...
> 
> ...so that, when _POSIX_C_SOURCE=202101 (say) arrives, _DEFAULT_SOURCE would then set the corresponding _XOPEN_SOURCE=800.

But _DEFAULT_SOURCE *explicit* from its addition on glibc *does not*
set XSI neither it is intention from original commit message. What you 
are asking is a explicit breakage of the flag semantic and glibc tries 
to avoid such changes.

> 
> I agree that anybody currently asking for (say) -D_DEFAULT_SOURCE -D_XOPEN_SOURCE=600 might be surprised by the new result.  Though I think we are agreed that what they currently get is undefined :-(
> 
> Chris

The main problem in my view is there is no guarantee from glibc for which
_POSIX_C_SOURCE _DEFAULT_SOURCE will define in the future for possible new
_POSIX_C_SOURCE values. I think it is worth a discussion and I will raise
this question on libc-alpha.

Re: Problems with default _POSIX_C_SOURCE in features.h ??

[Chris Hall]

On 09/04/2019 18:42, Adhemerval Zanella wrote:
> On 09/04/2019 14:12, Chris Hall wrote:
...
>> However, glibc goes to some effort to support the selection of 
>> older _POSIX_C_SOURCE and/or _XOPEN_SOURCE.  But *not* with 
>> __USE_MISC and *not* with __USE_GNU.  I have failed to 
>> discover why that is -- though that is probably because I am
>> too stupid.

> My understanding is because both __USE_MISC and __USE_GNU are 
> internal flags not meant to be set by external programs and 
> they can be removed/added in future glibc releases.

I was trying to use "with __USE_GNU" as a short-hand for "the GNU 
extensions, as enabled by _GNU_SOURCE".

So, if I specify (say) _XOPEN_SOURCE=600, glibc appears to do its best 
to give me the facilities which were aligned with that version of the 
standard -- *without* any GNU extensions.  To enable GNU Extensions as 
well, I must add -D_GNU_SOURCE, which overrides my -D_XOPEN_SOURCE=600, 
and (currently) gives me _XOPEN_SOURCE=700.

Perhaps glibc is prepared to support the selection of an older standard 
set of features, but is not prepared to figure out which GNU Extensions 
are "compatible" with that older standard.

Perhaps it just makes no sense to ask for _XOPEN_SOURCE=600 with GNU 
Extensions.

...
> The main problem in my view is there is no guarantee from glibc for which
> _POSIX_C_SOURCE _DEFAULT_SOURCE will define in the future for possible new
> _POSIX_C_SOURCE values. I think it is worth a discussion and I will raise
> this question on libc-alpha.

My guess is that it will select the latest and greatest.

Will anyone commit to it setting _POSIX_C_SOURCE=200809L forever ??

Which leads to the problem that to enable XSI with _DEFAULT_SOURCE, you 
(currently) have to know which -D_XOPEN_SOURCE=xxx to use, and that 
depends on which version of the library is being compiled for.

Chris

Re: Problems with default _POSIX_C_SOURCE in features.h ??

[Adhemerval Zanella]

On 09/04/2019 16:00, Chris Hall wrote:
> On 09/04/2019 18:42, Adhemerval Zanella wrote:
>> On 09/04/2019 14:12, Chris Hall wrote:
> ...
>>> However, glibc goes to some effort to support the selection of older _POSIX_C_SOURCE and/or _XOPEN_SOURCE.  But *not* with __USE_MISC and *not* with __USE_GNU.  I have failed to discover why that is -- though that is probably because I am
>>> too stupid.
> 
>> My understanding is because both __USE_MISC and __USE_GNU are internal flags not meant to be set by external programs and they can be removed/added in future glibc releases.
> 
> I was trying to use "with __USE_GNU" as a short-hand for "the GNU extensions, as enabled by _GNU_SOURCE".
> 
> So, if I specify (say) _XOPEN_SOURCE=600, glibc appears to do its best to give me the facilities which were aligned with that version of the standard -- *without* any GNU extensions.  To enable GNU Extensions as well, I must add -D_GNU_SOURCE, which overrides my -D_XOPEN_SOURCE=600, and (currently) gives me _XOPEN_SOURCE=700.
> 
> Perhaps glibc is prepared to support the selection of an older standard set of features, but is not prepared to figure out which GNU Extensions are "compatible" with that older standard.
> 
> Perhaps it just makes no sense to ask for _XOPEN_SOURCE=600 with GNU Extensions.

As I said, glibc does not provide the sense of version for _GNU_SOURCE, as for
_XOPEN_SOURCE or _POSIX_C_SOURCE.  Currently you can't target a specific version
of glibc using _GNU_SOURCE, said 2.23 or 2.24.

Maybe it might an option in the future to add such facility, it would require
mapping each gnu extension to a particular version and conditionalize them
on each installed header. 

> 
> ...
>> The main problem in my view is there is no guarantee from glibc for which
>> _POSIX_C_SOURCE _DEFAULT_SOURCE will define in the future for possible new
>> _POSIX_C_SOURCE values. I think it is worth a discussion and I will raise
>> this question on libc-alpha.
> 
> My guess is that it will select the latest and greatest.
> 
> Will anyone commit to it setting _POSIX_C_SOURCE=200809L forever ??
> 
> Which leads to the problem that to enable XSI with _DEFAULT_SOURCE, you (currently) have to know which -D_XOPEN_SOURCE=xxx to use, and that depends on which version of the library is being compiled for.

As I said, there was not need to change it from original default of POSIX1.2008,
and I asked on libc-alpha which would be the idea for this flag [1].

[1] https://sourceware.org/ml/libc-alpha/2019-04/msg00239.html

Re: Problems with default _POSIX_C_SOURCE in features.h ??

[Adhemerval Zanella]

On 10/04/2019 08:21, Adhemerval Zanella wrote:
> 
> 
> On 09/04/2019 16:00, Chris Hall wrote:
>> On 09/04/2019 18:42, Adhemerval Zanella wrote:
>>> On 09/04/2019 14:12, Chris Hall wrote:
>> ...
>>> The main problem in my view is there is no guarantee from glibc for which
>>> _POSIX_C_SOURCE _DEFAULT_SOURCE will define in the future for possible new
>>> _POSIX_C_SOURCE values. I think it is worth a discussion and I will raise
>>> this question on libc-alpha.
>>
>> My guess is that it will select the latest and greatest.
>>
>> Will anyone commit to it setting _POSIX_C_SOURCE=200809L forever ??
>>
>> Which leads to the problem that to enable XSI with _DEFAULT_SOURCE, you (currently) have to know which -D_XOPEN_SOURCE=xxx to use, and that depends on which version of the library is being compiled for.
> 
> As I said, there was not need to change it from original default of POSIX1.2008,
> and I asked on libc-alpha which would be the idea for this flag [1].
> 
> [1] https://sourceware.org/ml/libc-alpha/2019-04/msg00239.html
> 

It seems the current consensus is indeed that _DEFAULT_SOURCE would map to
latest available POSIX1 standard [1]. 

[1] https://sourceware.org/ml/libc-alpha/2019-04/msg00244.html

> 
> It is possible that my worry is a symptom of ignorance.
> 
> I have a large network application which is required to compile for more or less any UN*X.  I develop and test on Linux.  I would like the minimum of surprise when compiling and testing on other systems.
> 
> Now, FreeBSD has caused me some trouble in the past.  Recent FreeBSD admits to POSIX.1-2001 (more or less), but denies X/Open.  If you compile *without* specifying any _XXX_SOURCE macros (_POSIX_C_SOURCE, _XOPEN_SOURCE, POSIX_SOURCE, _ANSI_SOURCE, _C99_SOURCE and _C11_SOURCE) then you get:
> 
>   #define __POSIX_VISIBLE  200809
>   #define __XSI_VISIBLE       700
>   #define __BSD_VISIBLE       1
>   #define __ISO_C_VISIBLE  2011
>   #define __EXT1_VISIBLE   1
> 
> which appear to select everything available, including some XSI things. But you also get:
> 
>   #define _POSIX_VERSION   200112L
>   #define _XOPEN_UNIX      -1
> 
> but _XOPEN_VERSION is not defined.
> 
> So, I figured that on Linux, with glibc, I could:
> 
>   -D_XOPEN_SOURCE=600 -D_BSD_SOURCE
> 
> and that might approximate to FreeBSD, and other UN*X which more or less conform to POSIX.1-2001 or are certified UNIX 03.
> 
> But late model glibc won't let me do that
> 
> Perhaps I really don't need -D_BSD_SOURCE (now -D_DEFAULT_SOURCE) with POSIX.1-2001 ?

My understanding is _BSD_SOURCE is a BSD-derived FTMs that glibc aims to
 only provide some compatibility support (we still lack a lot of common BSD 
common symbols, like strlcpy for instance; some kernels depends interface
are tricky to implements; and we do not really aim to be fully BSD
compatible).

It is deprecated since glibc 2.20 in favor of _DEFAULT_SOURCE and
pre-2.20 it used to enable __USE_MISC (as _DEFAULT_SOURCE currently).
It define things from 4.3BSD or System V Unix (things like some mmap
flags).

> 
> Perhaps I am, in any case, wasting my time ?  Perhaps the reality is that I cannot avoid being caught out occasionally by differences between systems... and I should just be grateful it's only occasionally.

We can't really guarantee compatibility for _BSD_SOURCE in particular
(specially because we don't aim to be fully BSD compatible). But we
usually treat _POSIC_C_SOURCE an _XOPEN_SOURCE deviations from standards
as bugs (if you find some please open a bug report).

> 
> If so, I might as well compile for the latest and greatest _XOPEN_SOURCE, and trade the advantages of recent features against the (greater) risk of surprise.
> 
> Of course, glibc encourages the use of _GNU_SOURCE, which selects the latest and greatest _XOPEN_SOURCE.  But, it also enables all the GNU libc specials, which I really don't want.  It also seems to select C11 things, even if the compiler is not __STDC_VERSION__ 201112L, which I cannot say I like the look of, either.

_GNU_SOURCE definitely does not enable C11 symbols. It relies solely on
__STDC_VERSION__ value:

include/features.h:

225 /* This is to enable the ISO C11 extension.  */
226 #if (defined _ISOC11_SOURCE \
227      || (defined __STDC_VERSION__ && __STDC_VERSION__ >= 201112L))
228 # define __USE_ISOC11   1
229 #endif
[...]
243 #ifdef __cplusplus
244 /* This is to enable compatibility for ISO C++17.  */
245 # if __cplusplus >= 201703L
246 #  define __USE_ISOC11  1
247 # endif
[...]
254 #endif

And I am not aware of any bug related to this. Are you sure we you are seeing
it or are you using a compiler version that sets __STDC_VERSION__ as default?

> 
> I'd like a way to select the latest and greatest _XOPEN_SOURCE and/or _POSIX_C_SOURCE, while not selecting any GNU extensions.  Perhaps, say, _XOPEN_SOURCE_LATEST and/or _POSIX_SOURCE_LATEST ?
> 
> [I guess I could require users to install glibc, and surrender to the _GNU_SOURCE... but that doesn't feel right.]
> 
> Suggestions, anyone ? 

AFAIK it should be exactly as POSIX recommends: -D_POSIX_C_SOURCE=value and/or -D_XOPEN_SOURCE=value.

Re: Problems with default _POSIX_C_SOURCE in features.h ??

[Florian Weimer]

* Adhemerval Zanella:

> It is deprecated since glibc 2.20 in favor of _DEFAULT_SOURCE and
> pre-2.20 it used to enable __USE_MISC (as _DEFAULT_SOURCE currently).
> It define things from 4.3BSD or System V Unix (things like some mmap
> flags).

We have also added things to __USE_MISC which are widely available
today, but were invented long after 4.3BSD or System V.

Thanks,
Florian

Re: Problems with default _POSIX_C_SOURCE in features.h ??

[Chris Hall]

On 11/04/2019 22:40, Adhemerval Zanella wrote:
> On 10/04/2019 08:21, Adhemerval Zanella wrote:
>> On 09/04/2019 16:00, Chris Hall wrote:
>>> On 09/04/2019 18:42, Adhemerval Zanella wrote:
...
>>>> The main problem in my view is there is no guarantee from glibc for which
>>>> _POSIX_C_SOURCE _DEFAULT_SOURCE will define in the future for possible new
>>>> _POSIX_C_SOURCE values. I think it is worth a discussion and I will raise
>>>> this question on libc-alpha.

>>> My guess is that it will select the latest and greatest.
>>>
>>> Will anyone commit to it setting _POSIX_C_SOURCE=200809L forever ??
>>>
>>> Which leads to the problem that to enable XSI with _DEFAULT_SOURCE, you
>>> (currently) have to know which -D_XOPEN_SOURCE=xxx to use, and that
>>> depends on which version of the library is being compiled for.

>> As I said, there was not need to change it from original default of POSIX1.2008,
>> and I asked on libc-alpha which would be the idea for this flag [1].
>>
>> [1] https://sourceware.org/ml/libc-alpha/2019-04/msg00239.html

> It seems the current consensus is indeed that _DEFAULT_SOURCE would map to
> latest available POSIX1 standard [1].
> 
> [1] https://sourceware.org/ml/libc-alpha/2019-04/msg00244.html

OK.  If I want -D_DEFAULT_SOURCE plus the corresponding latest available 
XSI (on whatever version of library my users are compiling against), 
then I observe:

   1) -D_DEFAULT_SOURCE -D_XOPEN_SOURCE=-1UL will do the trick
                                                  -- ugly, though.

   2) -D_DEFAULT_SOURCE -D_XOPEN_SOURCE=700 does the job now...

      ...but will be broken in the future.

   3) a new _DEFAULT_XOPEN_SOURCE might be cleaner ?

>> It is possible that my worry is a symptom of ignorance.
>>
>> I have a large network application which is required to compile
>> for more or less any UN*X.  I develop and test on Linux.  I 
>> would like the minimum of surprise when compiling and testing on
>> other systems.
>>
>> Now, FreeBSD has caused me some trouble in the past.  ... 
...
>> So, I figured that on Linux, with glibc, I could:
>>
>>    -D_XOPEN_SOURCE=600 -D_BSD_SOURCE
>>
>> and that might approximate to FreeBSD, and other UN*X which more
>> or less conform to POSIX.1-2001 or are certified UNIX 03.
>>
>> But late model glibc won't let me do that
...
>> Perhaps I am, in any case, wasting my time ?  Perhaps the reality
>> is that I cannot avoid being caught out occasionally by differences
>> between systems... and I should just be grateful it's only 
>> occasionally.

> We can't really guarantee compatibility for _BSD_SOURCE in particular
> (specially because we don't aim to be fully BSD compatible). But we
> usually treat _POSIC_C_SOURCE an _XOPEN_SOURCE deviations from 
> standards as bugs (if you find some please open a bug report).

That much is clear.  As I say, my hope was to minimise surprises when 
testing on BSD-like systems.

...>> Of course, glibc encourages the use of _GNU_SOURCE, which selects
>> the latest and greatest _XOPEN_SOURCE.  But, it also enables all 
>> the GNU libc specials, which I really don't want.  It also seems 
>> to select C11 things, even if the compiler is not 
>> __STDC_VERSION__ 201112L, which I cannot say I like the look 
>> of, either.

> _GNU_SOURCE definitely does not enable C11 symbols. It relies solely on
> __STDC_VERSION__ value:
> 
> include/features.h:

   190 #ifdef _GNU_SOURCE
   191 #undef  _ISOC95_SOURCE
   192 #define _ISOC95_SOURCE 1
   193 #undef  _ISOC99_SOURCE
   194 #define _ISOC99_SOURCE 1
   195 #undef  _ISOC11_SOURCE
   196 #define _ISOC11_SOURCE 1

Forcing C99 for POSIX-2008 makes some sense.  But C11 should be skipped 
if __STRICT_ANSI__ ?

...> And I am not aware of any bug related to this. Are you sure we
> you are seeing it or are you using a compiler version that sets
> __STDC_VERSION__ as default?

The effect of __USE_ISOC11 is small (8 appearances outside <features.h>).

>> I'd like a way to select the latest and greatest _XOPEN_SOURCE
>> and/or _POSIX_C_SOURCE, while not selecting any GNU extensions.
>>  Perhaps, say, _XOPEN_SOURCE_LATEST and/or _POSIX_SOURCE_LATEST ?

or _XOPEN_LATEST_SOURCE and/or _POSIX_LATEST_SOURCE ?

>> [I guess I could require users to install glibc, and surrender
>> to the _GNU_SOURCE... but that doesn't feel right.]
>>
>> Suggestions, anyone ?

> AFAIK it should be exactly as POSIX recommends: 
> -D_POSIX_C_SOURCE=value and/or -D_XOPEN_SOURCE=value.

I should have said that I'd like a way to *automatically* select the 
latest and greatest _XOPEN_SOURCE/POSIX_C_SOURCE -- like _GNU_SOURCE does.

For a given version of glibc, past and future, what should my configure 
script do to determine the value to use ?  [_GNU_SOURCE appears to 
select the latest and greatest, but that's not explicitly stated in the 
documentation.]

Or should I use -1UL ?

If I simply set -D_XOPEN_SOURCE=700, then I will need to update the 
application when the next version becomes available.

Perhaps nobody really needs an automatic way to select the latest 
available ?  [Except, as noted above, to enable XSI stuff with 
-D_DEFAULT_SOURCE.]

Thanks,

Chris

Re: Problems with default _POSIX_C_SOURCE in features.h ??

[Adhemerval Zanella]

On 12/04/2019 11:07, Chris Hall wrote:
> On 11/04/2019 22:40, Adhemerval Zanella wrote:
>> On 10/04/2019 08:21, Adhemerval Zanella wrote:
>>> On 09/04/2019 16:00, Chris Hall wrote:
>>>> On 09/04/2019 18:42, Adhemerval Zanella wrote:
> ...
>>>>> The main problem in my view is there is no guarantee from glibc for which
>>>>> _POSIX_C_SOURCE _DEFAULT_SOURCE will define in the future for possible new
>>>>> _POSIX_C_SOURCE values. I think it is worth a discussion and I will raise
>>>>> this question on libc-alpha.
> 
>>>> My guess is that it will select the latest and greatest.
>>>>
>>>> Will anyone commit to it setting _POSIX_C_SOURCE=200809L forever ??
>>>>
>>>> Which leads to the problem that to enable XSI with _DEFAULT_SOURCE, you
>>>> (currently) have to know which -D_XOPEN_SOURCE=xxx to use, and that
>>>> depends on which version of the library is being compiled for.
> 
>>> As I said, there was not need to change it from original default of POSIX1.2008,
>>> and I asked on libc-alpha which would be the idea for this flag [1].
>>>
>>> [1] https://sourceware.org/ml/libc-alpha/2019-04/msg00239.html
> 
>> It seems the current consensus is indeed that _DEFAULT_SOURCE would map to
>> latest available POSIX1 standard [1].
>>
>> [1] https://sourceware.org/ml/libc-alpha/2019-04/msg00244.html
> 
> OK.  If I want -D_DEFAULT_SOURCE plus the corresponding latest available XSI (on whatever version of library my users are compiling against), then I observe:
> 
>   1) -D_DEFAULT_SOURCE -D_XOPEN_SOURCE=-1UL will do the trick
>                                                  -- ugly, though.
> 
>   2) -D_DEFAULT_SOURCE -D_XOPEN_SOURCE=700 does the job now...
> 
>      ...but will be broken in the future.
> 
>   3) a new _DEFAULT_XOPEN_SOURCE might be cleaner ?
> 
>>> It is possible that my worry is a symptom of ignorance.
>>>
>>> I have a large network application which is required to compile
>>> for more or less any UN*X.  I develop and test on Linux.  I would like the minimum of surprise when compiling and testing on
>>> other systems.
>>>
>>> Now, FreeBSD has caused me some trouble in the past.  ... 
> ...
>>> So, I figured that on Linux, with glibc, I could:
>>>
>>>    -D_XOPEN_SOURCE=600 -D_BSD_SOURCE
>>>
>>> and that might approximate to FreeBSD, and other UN*X which more
>>> or less conform to POSIX.1-2001 or are certified UNIX 03.
>>>
>>> But late model glibc won't let me do that
> ...
>>> Perhaps I am, in any case, wasting my time ?  Perhaps the reality
>>> is that I cannot avoid being caught out occasionally by differences
>>> between systems... and I should just be grateful it's only occasionally.
> 
>> We can't really guarantee compatibility for _BSD_SOURCE in particular
>> (specially because we don't aim to be fully BSD compatible). But we
>> usually treat _POSIC_C_SOURCE an _XOPEN_SOURCE deviations from standards as bugs (if you find some please open a bug report).
> 
> That much is clear.  As I say, my hope was to minimise surprises when testing on BSD-like systems.
> 
> ...>> Of course, glibc encourages the use of _GNU_SOURCE, which selects
>>> the latest and greatest _XOPEN_SOURCE.  But, it also enables all the GNU libc specials, which I really don't want.  It also seems to select C11 things, even if the compiler is not __STDC_VERSION__ 201112L, which I cannot say I like the look of, either.
> 
>> _GNU_SOURCE definitely does not enable C11 symbols. It relies solely on
>> __STDC_VERSION__ value:
>>
>> include/features.h:
> 
>   190 #ifdef _GNU_SOURCE
>   191 #undef  _ISOC95_SOURCE
>   192 #define _ISOC95_SOURCE 1
>   193 #undef  _ISOC99_SOURCE
>   194 #define _ISOC99_SOURCE 1
>   195 #undef  _ISOC11_SOURCE
>   196 #define _ISOC11_SOURCE 1
> 
> Forcing C99 for POSIX-2008 makes some sense.  But C11 should be skipped if __STRICT_ANSI__ ?

But the features defined internally by __USE_ISOC11 will only be available
if compilers defines the correct __STDC_VERSION__.  The idea of GNU source
is to enable the latest feature in par with compiler support.

> 
> ...> And I am not aware of any bug related to this. Are you sure we
>> you are seeing it or are you using a compiler version that sets
>> __STDC_VERSION__ as default?
> 
> The effect of __USE_ISOC11 is small (8 appearances outside <features.h>).
> 
>>> I'd like a way to select the latest and greatest _XOPEN_SOURCE
>>> and/or _POSIX_C_SOURCE, while not selecting any GNU extensions.
>>>  Perhaps, say, _XOPEN_SOURCE_LATEST and/or _POSIX_SOURCE_LATEST ?
> 
> or _XOPEN_LATEST_SOURCE and/or _POSIX_LATEST_SOURCE ?

I really want to avoid *more* FTMs, since it is more permutations we need
to take care off, more tests we need to check, more documentation we need
to write and keep updated.  

> 
>>> [I guess I could require users to install glibc, and surrender
>>> to the _GNU_SOURCE... but that doesn't feel right.]
>>>
>>> Suggestions, anyone ?
> 
>> AFAIK it should be exactly as POSIX recommends: -D_POSIX_C_SOURCE=value and/or -D_XOPEN_SOURCE=value.
> 
> I should have said that I'd like a way to *automatically* select the latest and greatest _XOPEN_SOURCE/POSIX_C_SOURCE -- like _GNU_SOURCE does.
> 
> For a given version of glibc, past and future, what should my configure script do to determine the value to use ?  [_GNU_SOURCE appears to select the latest and greatest, but that's not explicitly stated in the documentation.]
> 
> Or should I use -1UL ?
> 
> If I simply set -D_XOPEN_SOURCE=700, then I will need to update the application when the next version becomes available.
> 
> Perhaps nobody really needs an automatic way to select the latest available ?  [Except, as noted above, to enable XSI stuff with -D_DEFAULT_SOURCE.]
> 
> Thanks,
> 
> Chris

My understanding is historically _DEFAULT_SOURCE was added as a compatibility
replacement over _BSD_SOURCE / _XVID_SOURCE (which were deprecated).  Since it
was meant as a replacement for already defines FTMs, adding XSI extensions 
might break programs and it is something we avoid. Unless one can actually
show that adding XSI extensions won't break things, I would rather avoid
changing _DEFAULT_SOURCE semantic. 

I think the issue is you are trying to assure compatibility using the wrong
tool: _GNU_SOURCE or _DEFAULT_SOURCE are *not* the option to assure build
compatibility over different libcs implementation.  Even we add 
_DEFAULT_XOPEN_SOURCE it would not be a complete and future-proof way to 
assure compatibility.  

My take is adding more FTMs to add the latest supported
 _POSIX_C_SOURCE and _XOPEN_SOURCE is not much gain and it would add more 
FTM permutation that would incur in just more internal complexity to handle
it.  However other maintainers might have a different view, so you might
try to raise it on libc-alpha.

Should atomic_xxx() functions reject not-_Atomic() arguments ?

[Chris Hall]

So, the C11/C18 Standard defines, for example:

   C atomic_fetch_<key>(volatile A *object, M operand);

where "An A refers to one of the atomic types."

So:

    _Atomic(uint64_t) foo ;
    uint64_t bar ;

    bar = atomic_fetch_add(&foo, 1) ;

is all by the book.

Now, the Standard also tells us that _Atomic(uint64_t) and uint64_t may 
have different sizes, representations and alignment.  So I guess:

    bar = atomic_fetch_add(&bar, 1) ;

should be an error ?

On my x86_64, gcc 9.2/glibc 2.30 do not think it is an error, and indeed 
for the x86_64 _Atomic(uint64_t) and uint64_t are identical.

But this looks like a trap for the unwary... on some machine out there, 
the Standard says 'bar = atomic_fetch_add(&bar, 1) ;' is a *mistake*.

Or will gcc/glibs throw an error on such a machine ?

Are atomic_fetch_xxx() functions broken for atomic-pointer types ?

[Chris Hall]

In Section 7.17.7.5 "The atomic_fetch and modify generic functions", the 
C11/C18 Standards say:

   1 The following operations perform arithmetic and bitwise
     computations. All of these operations are applicable to an
     object of any atomic integer type. ...

   5 NOTE The operation of the atomic_fetch and modify generic functions
          are nearly equivalent to the operation of the corresponding
          op= compound assignment operators. The only differences are
          that the compound assignment operators are not guaranteed to
          operate atomically, and the value yielded by a compound
          assignment operator is the updated value of the object,
          whereas the value returned by the atomic_fetch and modify
          generic functions is the previous value of the atomic object.

So given:

   _Atomic(uint64_t*) foo ;
   uint64_t* bar ;

   bar = atomic_fetch_add(&foo, 1) ;

why do gcc 9.2/glibc 2.30 add 1 and not 8 to the address ?

I note that <stdatomic.h> maps atomic_fetch_add() to the built-in 
__atomic_fetch_add(), which is indeed defined to work this way.

I grant that Section 7.17.7.5 is not a miracle of clarity (even by the 
standards of the Standards).  You could argue that atomic_fetch_xxx() 
are simply not defined for atomic pointer types.  But if so, *quietly* 
accepting an undefined argument and then doing something perverse with 
it, strikes me as odd.

FWIW: since the bitwise 'op=' are not defined for pointer types, it is 
also a "surprise" that atomic_fetch_and() etc. accept pointer types.

It seems to me that gcc/glibc are broken, as are the Standards.  How to 
address that at this late stage looks tricky to me :-(  Perhaps the 
Standards could have a big dollop of Implementation Defined 
retro-fitted; laid on thickly enough that might cover over the existing 
mess ?

-----

I note that <stdatomic.h> is not mentioned in the GNU C Library 
Reference Manual for version 2.31... so I'm guessing that 9 or so years 
after the C11 standard, this remains such a mess that nobody can face 
trying to fix it ?

Re: Are atomic_fetch_xxx() functions broken for atomic-pointer types ?

[Adhemerval Zanella]

On 24/02/2020 09:17, Chris Hall wrote:
> 
> I note that <stdatomic.h> is not mentioned in the GNU C Library Reference Manual for version 2.31... so I'm guessing that 9 or so years after the C11 standard, this remains such a mess that nobody can face trying to fix it ?

The stdatomic.h is provided by the compiler, the atomic operations glibc
defines are for internal use only.  So I would recommend to move this
discussion to gcc maillist.