[PATCH] manual: Document the standardized scanf flag, "m". [BZ #16376]

[PATCH] manual: Document the standardized scanf flag, "m". [BZ #16376]

[Rical Jasan]

[-- Attachment #1: Type: text/plain, Size: 808 bytes --]

Looking in stdio-common/vfscanf.c, "a" uses GNU_MALLOC, and "m" uses
POSIX_MALLOC, and there is a MALLOC which is defined to either, and
throughout the code there are several conditionals where POSIX_MALLOC is
nested beneath MALLOC (GNU_MALLOC is always MALLOC), but I'm not sure we
want to document any internal implementation differences (if they're
even significant).  The existing documentation wasn't very detailed
itself, but I still wanted to mention it.

I confirmed "m" was introduced in POSIX.1-2008 (mentioned in the Issue 7
section of CHANGE HISTORY on the s/f/scanf page).

The NEWS file says entries for bugs will be generated automatically, and
once ACK'd, I'll refactor the paragraphs before committing (tried to
make it easier to see what changed here) and provide a ChangeLog entry.

Rical

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-manual-Document-the-standardized-scanf-flag-m-.-BZ-1.patch --]
[-- Type: text/x-patch; name="0001-manual-Document-the-standardized-scanf-flag-m-.-BZ-1.patch", Size: 4531 bytes --]

From cc32f967605178358d7b0d3e6fc1d36d036e6a8f Mon Sep 17 00:00:00 2001
From: Rical Jasan <ricaljasan@pacific.net>
Date: Tue, 6 Feb 2018 21:22:27 -0800
Subject: [PATCH] manual: Document the standardized scanf flag, "m".  [BZ
 #16376]

POSIX.1-2008 introduced the optional assignment-allocation modifier,
"m", whose functionality was previously provided by the GNU extension
"a" (and still is).

	[BZ #16376]
	* manual/stdio.texi (Input Conversion Syntax)
	(String Input Conversions, Dynamic String Input): Document the
	"m" flag.
---
 manual/stdio.texi | 34 +++++++++++++++++++++-------------
 1 file changed, 21 insertions(+), 13 deletions(-)

diff --git a/manual/stdio.texi b/manual/stdio.texi
index 5d7b50c442..4db65f4e18 100644
--- a/manual/stdio.texi
+++ b/manual/stdio.texi
@@ -3440,10 +3440,9 @@ successful assignments.
 @cindex flag character (@code{scanf})
 
 @item
-An optional flag character @samp{a} (valid with string conversions only)
+An optional flag character @samp{m} (valid with string conversions only)
 which requests allocation of a buffer long enough to store the string in.
-(This is a GNU extension.)
-@xref{Dynamic String Input}.
+(There is also a GNU extension, @samp{a}.)  @xref{Dynamic String Input}.
 
 @item
 An optional decimal integer that specifies the @dfn{maximum field
@@ -3720,8 +3719,9 @@ provide the buffer, always specify a maximum field width to prevent
 overflow.}
 
 @item
-Ask @code{scanf} to allocate a big enough buffer, by specifying the
-@samp{a} flag character.  This is a GNU extension.  You should provide
+Ask @code{scanf} to allocate a big-enough buffer, by specifying the
+@samp{m} flag character (or the GNU extension, @samp{a}).
+You should provide
 an argument of type @code{char **} for the buffer address to be stored
 in.  @xref{Dynamic String Input}.
 @end itemize
@@ -3825,7 +3825,7 @@ is said about @samp{%ls} above is true for @samp{%l[}.
 
 One more reminder: the @samp{%s} and @samp{%[} conversions are
 @strong{dangerous} if you don't specify a maximum width or use the
-@samp{a} flag, because input too long would overflow whatever buffer you
+@samp{m} or @samp{a} flags, because too-long input would overflow whatever buffer you
 have provided for it.  No matter how long your buffer is, a user could
 supply input that is longer.  A well-written program reports invalid
 input with a comprehensible error message, not with a crash.
@@ -3833,18 +3833,26 @@ input with a comprehensible error message, not with a crash.
 @node Dynamic String Input
 @subsection Dynamically Allocating String Conversions
 
-A GNU extension to formatted input lets you safely read a string with no
+POSIX.1-2008 specifies an @dfn{assignment-allocation character}
+@samp{m}, valid for use with the string conversion specifiers
+@samp{s}, @samp{S}, @samp{[}, @samp{c}, and @samp{C}, which
+lets you safely read a string with no
 maximum size.  Using this feature, you don't supply a buffer; instead,
 @code{scanf} allocates a buffer big enough to hold the data and gives
-you its address.  To use this feature, write @samp{a} as a flag
-character, as in @samp{%as} or @samp{%a[0-9a-z]}.
+you its address.  To use this feature, write @samp{m} as a flag
+character; e.g., @samp{%ms} or @samp{%m[0-9a-z]}.
 
 The pointer argument you supply for where to store the input should have
 type @code{char **}.  The @code{scanf} function allocates a buffer and
-stores its address in the word that the argument points to.  You should
-free the buffer with @code{free} when you no longer need it.
+stores its address in the word that the argument points to.  When
+using the @samp{l} modifier (or equivalently, @samp{S} or @samp{C}),
+the pointer argument should have the type @code{wchar_t **}.
+
+You should free the buffer with @code{free} when you no longer need it.
+
+As a GNU extension, @samp{a} is available in place of @samp{m}.
 
-Here is an example of using the @samp{a} flag with the @samp{%[@dots{}]}
+Here is an example of using the @samp{m} flag with the @samp{%[@dots{}]}
 conversion specification to read a ``variable assignment'' of the form
 @samp{@var{variable} = @var{value}}.
 
@@ -3852,7 +3860,7 @@ conversion specification to read a ``variable assignment'' of the form
 @{
   char *variable, *value;
 
-  if (2 > scanf ("%a[a-zA-Z0-9] = %a[^\n]\n",
+  if (2 > scanf ("%m[a-zA-Z0-9] = %m[^\n]\n",
 		 &variable, &value))
     @{
       invalid_input_error ();
-- 
2.14.3

Re: [PATCH] manual: Document the standardized scanf flag, "m". [BZ #16376]

[Andreas Schwab]

On Feb 06 2018, Rical Jasan <ricaljasan@pacific.net> wrote:

> Looking in stdio-common/vfscanf.c, "a" uses GNU_MALLOC, and "m" uses
> POSIX_MALLOC, and there is a MALLOC which is defined to either, and
> throughout the code there are several conditionals where POSIX_MALLOC is
> nested beneath MALLOC (GNU_MALLOC is always MALLOC), but I'm not sure we
> want to document any internal implementation differences (if they're
> even significant).  The existing documentation wasn't very detailed
> itself, but I still wanted to mention it.

IMHO the "a" modifier should be marked as deprecated, since it conflicts
with the %a format.  I think it would be enough to mention it in a
paragraph, but nowhere else.

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

Re: [PATCH] manual: Document the standardized scanf flag, "m". [BZ #16376]

[Paul Eggert]

On 02/07/2018 12:46 AM, Andreas Schwab wrote:
> IMHO the "a" modifier should be marked as deprecated, since it conflicts
> with the %a format.  I think it would be enough to mention it in a
> paragraph, but nowhere else.

Agreed. Also, I suggest modifying the NEWS entry for glibc 1.06 that 
talks about the newly-introduced "a" modifier, so that it briefly 
mentions that the modifier is deprecated after glibc 2.28.

[PATCH] manual: Document the standardized scanf flag, "m". [BZ #16376]

[Rical Jasan]

POSIX.1-2008 introduced the optional assignment-allocation modifier,
"m", whose functionality was previously provided by the GNU extension
"a".

	[BZ #16376]
	* manual/stdio.texi (Input Conversion Syntax)
	(String Input Conversions, Dynamic String Input): Document the
	"m" flag.
---
 manual/stdio.texi | 32 ++++++++++++++++++++------------
 1 file changed, 20 insertions(+), 12 deletions(-)

diff --git a/manual/stdio.texi b/manual/stdio.texi
index 38be236991..22c338f8ea 100644
--- a/manual/stdio.texi
+++ b/manual/stdio.texi
@@ -3440,9 +3440,8 @@ successful assignments.
 @cindex flag character (@code{scanf})
 
 @item
-An optional flag character @samp{a} (valid with string conversions only)
+An optional flag character @samp{m} (valid with string conversions only)
 which requests allocation of a buffer long enough to store the string in.
-(This is a GNU extension.)
 @xref{Dynamic String Input}.
 
 @item
@@ -3720,8 +3719,8 @@ provide the buffer, always specify a maximum field width to prevent
 overflow.}
 
 @item
-Ask @code{scanf} to allocate a big enough buffer, by specifying the
-@samp{a} flag character.  This is a GNU extension.  You should provide
+Ask @code{scanf} to allocate a big-enough buffer, by specifying the
+@samp{m} flag character.  You should provide
 an argument of type @code{char **} for the buffer address to be stored
 in.  @xref{Dynamic String Input}.
 @end itemize
@@ -3825,7 +3824,7 @@ is said about @samp{%ls} above is true for @samp{%l[}.
 
 One more reminder: the @samp{%s} and @samp{%[} conversions are
 @strong{dangerous} if you don't specify a maximum width or use the
-@samp{a} flag, because input too long would overflow whatever buffer you
+@samp{m} flag, because too-long input would overflow whatever buffer you
 have provided for it.  No matter how long your buffer is, a user could
 supply input that is longer.  A well-written program reports invalid
 input with a comprehensible error message, not with a crash.
@@ -3833,18 +3832,27 @@ input with a comprehensible error message, not with a crash.
 @node Dynamic String Input
 @subsection Dynamically Allocating String Conversions
 
-A GNU extension to formatted input lets you safely read a string with no
+POSIX.1-2008 specifies an @dfn{assignment-allocation character}
+@samp{m}, valid for use with the string conversion specifiers
+@samp{s}, @samp{S}, @samp{[}, @samp{c}, and @samp{C}, which
+lets you safely read a string with no
 maximum size.  Using this feature, you don't supply a buffer; instead,
 @code{scanf} allocates a buffer big enough to hold the data and gives
-you its address.  To use this feature, write @samp{a} as a flag
-character, as in @samp{%as} or @samp{%a[0-9a-z]}.
+you its address.  To use this feature, write @samp{m} as a flag
+character; e.g., @samp{%ms} or @samp{%m[0-9a-z]}.
 
 The pointer argument you supply for where to store the input should have
 type @code{char **}.  The @code{scanf} function allocates a buffer and
-stores its address in the word that the argument points to.  You should
-free the buffer with @code{free} when you no longer need it.
+stores its address in the word that the argument points to.  When
+using the @samp{l} modifier (or equivalently, @samp{S} or @samp{C}),
+the pointer argument should have the type @code{wchar_t **}.
+
+You should free the buffer with @code{free} when you no longer need it.
+
+As a GNU extension predating @samp{m}, @samp{a} is also available, but
+its use is considered deprecated.
 
-Here is an example of using the @samp{a} flag with the @samp{%[@dots{}]}
+Here is an example of using the @samp{m} flag with the @samp{%[@dots{}]}
 conversion specification to read a ``variable assignment'' of the form
 @samp{@var{variable} = @var{value}}.
 
@@ -3852,7 +3860,7 @@ conversion specification to read a ``variable assignment'' of the form
 @{
   char *variable, *value;
 
-  if (2 > scanf ("%a[a-zA-Z0-9] = %a[^\n]\n",
+  if (2 > scanf ("%m[a-zA-Z0-9] = %m[^\n]\n",
 		 &variable, &value))
     @{
       invalid_input_error ();
-- 
2.14.3

Re: [PATCH] manual: Document the standardized scanf flag, "m". [BZ #16376]

[Zack Weinberg]

On Fri, Feb 9, 2018 at 8:07 AM, Rical Jasan <ricaljasan@pacific.net> wrote:
> POSIX.1-2008 introduced the optional assignment-allocation modifier,
> "m", whose functionality was previously provided by the GNU extension
> "a".

OK with just one tweak...

> +You should free the buffer with @code{free} when you no longer need it.
> +
> +As a GNU extension predating @samp{m}, @samp{a} is also available, but
> +its use is considered deprecated.

let's be a little more specific here:

+As a GNU extension, the modifier @samp{a} has the same effect as @samp{m}.
+This extension predates POSIX.1-2008 and is now deprecated.  Other C libraries
+may interpret e.g.@: @samp{%as} as the @samp{%a} format for reading
+floating-point numbers, followed by a literal @samp{s}.

zw

Re: [PATCH] manual: Document the standardized scanf flag, "m". [BZ #16376]

[Joseph Myers]

On Fri, 9 Feb 2018, Zack Weinberg wrote:

> > +As a GNU extension predating @samp{m}, @samp{a} is also available, but
> > +its use is considered deprecated.
> 
> let's be a little more specific here:
> 
> +As a GNU extension, the modifier @samp{a} has the same effect as @samp{m}.
> +This extension predates POSIX.1-2008 and is now deprecated.  Other C libraries
> +may interpret e.g.@: @samp{%as} as the @samp{%a} format for reading
> +floating-point numbers, followed by a literal @samp{s}.

Which glibc does in the absence of _GNU_SOURCE, since __USE_XOPEN2K is 
defined by default.  The redirection to __isoc99_scanf etc. is done if:

#if defined __USE_ISOC99 && !defined __USE_GNU \
    && (!defined __LDBL_COMPAT || !defined __REDIRECT) \
    && (defined __STRICT_ANSI__ || defined __USE_XOPEN2K)

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: [PATCH] manual: Document the standardized scanf flag, "m". [BZ #16376]

[Zack Weinberg]

On Fri, Feb 9, 2018 at 11:39 AM, Joseph Myers <joseph@codesourcery.com> wrote:
> On Fri, 9 Feb 2018, Zack Weinberg wrote:
>
>> > +As a GNU extension predating @samp{m}, @samp{a} is also available, but
>> > +its use is considered deprecated.
>>
>> let's be a little more specific here:
>>
>> +As a GNU extension, the modifier @samp{a} has the same effect as @samp{m}.
>> +This extension predates POSIX.1-2008 and is now deprecated.  Other C libraries
>> +may interpret e.g.@: @samp{%as} as the @samp{%a} format for reading
>> +floating-point numbers, followed by a literal @samp{s}.
>
> Which glibc does in the absence of _GNU_SOURCE, since __USE_XOPEN2K is
> defined by default.  The redirection to __isoc99_scanf etc. is done if:
>
> #if defined __USE_ISOC99 && !defined __USE_GNU \
>     && (!defined __LDBL_COMPAT || !defined __REDIRECT) \
>     && (defined __STRICT_ANSI__ || defined __USE_XOPEN2K)

I'm tempted to suggest that we drop the __USE_GNU - meaning that 'a'
would only be a modifier under -std=gnu89, if I'm reading that
correctly - both because it'll be easier to document, and because this
seems to be already what GCC's scanf format warnings do:

$ cat test.c
#include <stdio.h>

int main(void)
{
  char *s;
  scanf("%as", &s);
  puts(s);
  return 0;
}

$ gcc -std=gnu89 -Wformat test.c
$ gcc -std=gnu11 -Wformat test.c
test.c: In function ‘main’:
test.c:6:11: warning: format ‘%a’ expects argument of type ‘float *’,
but argument 2 has type ‘char **’ [-Wformat=]
   scanf("%as", &s);
          ~^    ~~
$ gcc -std=gnu11 -Wformat -D_GNU_SOURCE test.c
test.c: In function ‘main’:
test.c:6:11: warning: format ‘%a’ expects argument of type ‘float *’,
but argument 2 has type ‘char **’ [-Wformat=]
   scanf("%as", &s);
          ~^    ~~

$ gcc --version
gcc (Debian 7.3.0-3) 7.3.0

Re: [PATCH] manual: Document the standardized scanf flag, "m". [BZ #16376]

[Joseph Myers]

On Fri, 9 Feb 2018, Zack Weinberg wrote:

> >> +As a GNU extension, the modifier @samp{a} has the same effect as @samp{m}.
> >> +This extension predates POSIX.1-2008 and is now deprecated.  Other C libraries
> >> +may interpret e.g.@: @samp{%as} as the @samp{%a} format for reading
> >> +floating-point numbers, followed by a literal @samp{s}.
> >
> > Which glibc does in the absence of _GNU_SOURCE, since __USE_XOPEN2K is
> > defined by default.  The redirection to __isoc99_scanf etc. is done if:
> >
> > #if defined __USE_ISOC99 && !defined __USE_GNU \
> >     && (!defined __LDBL_COMPAT || !defined __REDIRECT) \
> >     && (defined __STRICT_ANSI__ || defined __USE_XOPEN2K)
> 
> I'm tempted to suggest that we drop the __USE_GNU - meaning that 'a'
> would only be a modifier under -std=gnu89, if I'm reading that

That seems reasonable to me.  (With a corresponding change to 
bits/stdio-ldbl.h to keep things consistent in the -mlong-double-64 case.)

-- 
Joseph S. Myers
joseph@codesourcery.com

[PATCH v3] manual: Document the standardized scanf flag, "m". [BZ #16376]

[Rical Jasan]

This commit documents the optional assignment-allocation modifier,
"m", introduced in POSIX.1-2008, whose functionality was previously
provided by the GNU extension "a".  Use of the GNU extension is
discouraged due to ambiguity when operating in any conformance mode
equivalent to C99/POSIX.1-2001, which introduced the "a", "A", and "F"
conversion specifiers.  (Consider a format string containing "%as".)
The additional conversion specifiers are also documented.

	[BZ #16376]
	* manual/stdio.texi (Input Conversion Syntax)
	(String Input Conversions, Dynamic String Input): Document the
	"m" flag.
	(Table of Input Conversions, Numeric Input Conversions):
	Document the "a", "A", and "F" conversion specifiers.
---
Changes in v3:

	* Discourage use of the "a" GNU extension.
	* Document the "a", "A", and "F" conversion specifiers.
---
 manual/stdio.texi | 41 +++++++++++++++++++++++++----------------
 1 file changed, 25 insertions(+), 16 deletions(-)

diff --git a/manual/stdio.texi b/manual/stdio.texi
index 38be236991..b00911620f 100644
--- a/manual/stdio.texi
+++ b/manual/stdio.texi
@@ -3440,9 +3440,8 @@ successful assignments.
 @cindex flag character (@code{scanf})
 
 @item
-An optional flag character @samp{a} (valid with string conversions only)
+An optional flag character @samp{m} (valid with string conversions only)
 which requests allocation of a buffer long enough to store the string in.
-(This is a GNU extension.)
 @xref{Dynamic String Input}.
 
 @item
@@ -3508,7 +3507,7 @@ Matches an unsigned integer written in decimal radix.
 Matches an unsigned integer written in hexadecimal radix.
 @xref{Numeric Input Conversions}.
 
-@item @samp{%e}, @samp{%f}, @samp{%g}, @samp{%E}, @samp{%G}
+@item @samp{%a}, @samp{%e}, @samp{%f}, @samp{%g}, @samp{%A}, @samp{%E}, @samp{%F}, @samp{%G}
 Matches an optionally signed floating-point number.  @xref{Numeric Input
 Conversions}.
 
@@ -3664,7 +3663,8 @@ Specifies that the argument is a @code{size_t *}.
 This modifier was introduced in @w{ISO C99}.
 @end table
 
-All of the @samp{%e}, @samp{%f}, @samp{%g}, @samp{%E}, and @samp{%G}
+All of the @samp{%a}, @samp{%e}, @samp{%f}, @samp{%g}, @samp{%A},
+@samp{%E}, @samp{%F}, and @samp{%G}
 input conversions are interchangeable.  They all match an optionally
 signed floating point number, in the same syntax as for the
 @code{strtod} function (@pxref{Parsing of Floats}).
@@ -3720,10 +3720,8 @@ provide the buffer, always specify a maximum field width to prevent
 overflow.}
 
 @item
-Ask @code{scanf} to allocate a big enough buffer, by specifying the
-@samp{a} flag character.  This is a GNU extension.  You should provide
-an argument of type @code{char **} for the buffer address to be stored
-in.  @xref{Dynamic String Input}.
+Ask @code{scanf} to allocate a large-enough buffer, by specifying the
+@samp{m} flag character.  @xref{Dynamic String Input}.
 @end itemize
 
 The @samp{%c} conversion is the simplest: it matches a fixed number of
@@ -3825,7 +3823,7 @@ is said about @samp{%ls} above is true for @samp{%l[}.
 
 One more reminder: the @samp{%s} and @samp{%[} conversions are
 @strong{dangerous} if you don't specify a maximum width or use the
-@samp{a} flag, because input too long would overflow whatever buffer you
+@samp{m} flag, because too-long input would overflow whatever buffer you
 have provided for it.  No matter how long your buffer is, a user could
 supply input that is longer.  A well-written program reports invalid
 input with a comprehensible error message, not with a crash.
@@ -3833,18 +3831,29 @@ input with a comprehensible error message, not with a crash.
 @node Dynamic String Input
 @subsection Dynamically Allocating String Conversions
 
-A GNU extension to formatted input lets you safely read a string with no
+POSIX.1-2008 specifies an @dfn{assignment-allocation character}
+@samp{m}, valid for use with the string conversion specifiers
+@samp{s}, @samp{S}, @samp{[}, @samp{c}, and @samp{C}, which
+lets you safely read a string with no
 maximum size.  Using this feature, you don't supply a buffer; instead,
 @code{scanf} allocates a buffer big enough to hold the data and gives
-you its address.  To use this feature, write @samp{a} as a flag
-character, as in @samp{%as} or @samp{%a[0-9a-z]}.
+you its address.  To use this feature, write @samp{m} as a flag
+character; e.g., @samp{%ms} or @samp{%m[0-9a-z]}.
 
 The pointer argument you supply for where to store the input should have
 type @code{char **}.  The @code{scanf} function allocates a buffer and
-stores its address in the word that the argument points to.  You should
-free the buffer with @code{free} when you no longer need it.
+stores its address in the word that the argument points to.  When
+using the @samp{l} modifier (or equivalently, @samp{S} or @samp{C}),
+the pointer argument should have the type @code{wchar_t **}.
+
+You should free the buffer with @code{free} when you no longer need it.
+
+@strong{Warning:} There was a GNU extension, @samp{a}, that predated
+@samp{m}, but its use conflicts with the @samp{a} conversion specifier
+introduced in C99 (and adopted in POSIX.1-2001), so its use is highly
+discouraged.
 
-Here is an example of using the @samp{a} flag with the @samp{%[@dots{}]}
+Here is an example of using the @samp{m} flag with the @samp{%[@dots{}]}
 conversion specification to read a ``variable assignment'' of the form
 @samp{@var{variable} = @var{value}}.
 
@@ -3852,7 +3861,7 @@ conversion specification to read a ``variable assignment'' of the form
 @{
   char *variable, *value;
 
-  if (2 > scanf ("%a[a-zA-Z0-9] = %a[^\n]\n",
+  if (2 > scanf ("%m[a-zA-Z0-9] = %m[^\n]\n",
 		 &variable, &value))
     @{
       invalid_input_error ();
-- 
2.16.1

Re: [PATCH] manual: Document the standardized scanf flag, "m". [BZ #16376]

[Rical Jasan]

On 02/08/2018 10:42 AM, Paul Eggert wrote:
> On 02/07/2018 12:46 AM, Andreas Schwab wrote:
>> IMHO the "a" modifier should be marked as deprecated, since it conflicts
>> with the %a format.  I think it would be enough to mention it in a
>> paragraph, but nowhere else.
> 
> Agreed. Also, I suggest modifying the NEWS entry for glibc 1.06 that
> talks about the newly-introduced "a" modifier, so that it briefly
> mentions that the modifier is deprecated after glibc 2.28.

I've submitted a v3 that doesn't update NEWS yet because Zack started
addressing actual deprecation of the "a" modifier, but it isn't entirely
clear yet (at least to me) what that will look like when it's done.  I
think it may be more appropriate to update the deprecation section of
NEWS for 2.28 when/if that goes in, though I could put a stub in to the
effect of, "The GNU extension scanf modifier "a" has been [documented
as] deprecated."  I'm not sure if it's appropriate to say it /has been/
deprecated just because of the manual, though perhaps that is enough;
putting a NEWS entry there is essentially the formal announcement, I
suppose.  I don't think retroactively editing the 1.06 announcement is
the way to go (I would rather just announce it in 2.28), but if others
agree that's fine/desirable, I can change my mind about that.  :)

Thanks,
Rical