[PATCH 0/1] string: Add stpecpy(3)

[PATCH 0/1] string: Add stpecpy(3)

[Alejandro Colomar]

Hi!

I recently rewrite the Linux man-pages for string-copying functions, and
just a moment ago, I released the new version.  There are some important
gaps in the string-copying library, and they should be addressed,
however it is done, but ignoring it won't solve the problem.

I know this has been suggested in the past (I did once), and has never
progressed, but I'll try to justify it as much as I can.

The gaps are:

-  No function for copying strings with truncation.  (strlcpy(3) or strscpy(9))
-  No function for catenating strings with truncation.  (strlcat(3))
-  No function for chain-copying strings with truncation.  (stpecpy(3))

   This is similar to strcpy(3)/strcat(3)/stpcpy(3), where stpcpy(3) is
   faster and more versatile than the other two, but it's also slightly
   more complex to use (only slightly).

   We wouldn't need to add the 3, but at least stpecpy(3) or both
   strlcpy(3) and strlcat(3).

   Since stpecpy(3) is significantly faster than the other two, I
   suggest at least adding stpecpy(3).  Also, strlcpy(3)/cat(3) can be
   more easily implemented in terms of stpecpy(3).

There are a few other gaps, but they are much less important, since
there are relatively good workarounds.  I don't want to overload the
discussion either, so I prefer first adding the most necessary function,
and only after that deciding if we want to support other string-copying
functions.

I had added this function this week to a new string library that I was
writing, so I already had a manual page written for it.  I'll copy it
below, to document all the details of the API.o

Cheers,

Alex


---

Alejandro Colomar (1):
  string: Add stpecpy(3)

 string/Makefile  |  1 +
 string/stpecpy.c | 39 +++++++++++++++++++++++++++++++++++++++
 string/string.h  |  7 +++++++
 3 files changed, 47 insertions(+)
 create mode 100644 string/stpecpy.c


stpecpy(3)                 Library Functions Manual                 stpecpy(3)

NAME
       stpecpy, stpecpyx - copy a string with truncation

LIBRARY
       Stp string library (libstp, pkgconf ‐‐cflags ‐‐libs libstp)

SYNOPSIS
       #include <stp/stpe/stpecpy.h>

       char *_Nullable stpecpy(char *_Nullable dst, char end[0],
                               const char *restrict src);
       char *_Nullable stpecpyx(char *_Nullable dst, char end[0],
                               const char *restrict src);

DESCRIPTION
       These functions copy the string pointer to by src, into a string at the
       buffer  pointer  to  by  dst.   If the destination buffer, limited by a
       pointer to its end —one after its last element—, isn’t large enough  to
       hold the copy, the resulting string is truncated.

       stpecpyx(3)  forces a SIGSEGV if the input is not a string, by travers‐
       ing it entirely.

       These  functions  can  be  chained  with  calls  to  stpeprintf(3)  and
       vstpeprintf(3).

       An implementation of these functions might be

           /* This code is in the public domain. */

           char *
           stpecpy(char *dst, char end[0], const char *restrict src)
           {
               char *p;

               if (dst == end || dst == NULL)
                   return dst;

               p = memccpy(dst, src, '\0', end - dst);
               if (p != NULL)
                   return p - 1;

               /* truncation detected */
               end[-1] = '\0';
               return end;
           }

           char *
           stpecpyx(char *dst, char end[0], const char *restrict src)
           {
               if (src[strlen(src)] != '\0')
                   raise(SIGSEGV);

               return stpecpy(dst, end, src);
           }

RETURN VALUE
       NULL   If dst was NULL.

       end
              •  If this call truncated.
              •  If  dst  was equal to end (a previous call to these functions
                 truncated).

       dst + strlen(dst)
              On success, these functions return a pointer to the  terminating
              null byte.

ATTRIBUTES
       For  an  explanation  of  the  terms  used in this section, see attrib‐
       utes(7).
       ┌────────────────────────────────────────────┬───────────────┬─────────┐
       │Interface                                   │ Attribute     │ Value   │
       ├────────────────────────────────────────────┼───────────────┼─────────┤
       │stpecpy(3), stpecpyx(3)                     │ Thread safety │ MT‐Safe │
       └────────────────────────────────────────────┴───────────────┴─────────┘

STANDARDS
       None.

EXAMPLES
       $ cc ./stpecpy.c $(pkgconf --cflags --libs libbsd-overlay libstp)
       $ ./a.out
       [len = 12]: Hello world!
       $

       // stpecpy.c
       #include <err.h>
       #include <stdio.h>
       #include <stdlib.h>

       #include <stp/stpecpy.h>
       #include <stp/stpeprintf.h>

       int
       main(void)
       {
           char    *p, *end;
           char    buf[BUFSIZ];
           size_t  len;

           end = buf + BUFSIZ;
           p = buf;
           p = stpecpy(p, end, "Hello, ");
           p = stpeprintf(p, end, "%d worlds", 22);
           p = stpecpy(p, end, "!");
           if (p == NULL)
               err(EXIT_FAILURE, "stpeprintf()");
           if (p == end) {
               p--;
               warnx("Truncated");
           }
           len = p - buf;
           printf("[len = %zu]: ", len);
           puts(buf);

           exit(EXIT_SUCCESS);
       }

SEE ALSO
       stpeprintf(3), string_copying(7)

libstp (unreleased)                 (date)                          stpecpy(3)

-- 
2.39.0

[PATCH 1/1] string: Add stpecpy(3)

[Alejandro Colomar]

Glibc didn't provide any function that copies a string with truncation.

It only provided strncpy(3) and stpncpy(3), which copy from a string
into a null-padded character sequence at the destination fixed-width
buffer, with truncation.

Those old functions, which don't produce a string, have been misused for
a long time as a poor-man's replacement for strlcpy(3), but doing that
is a source of bugs, since it's hard to calculate the right size that
should be passed to the function, and it's also necessary to explicitly
terminate the buffer with a null byte.  Detecting truncation is yet
another problem.

stpecpy(3), described in the new string_copying(7) manual page, is
similar to OpenBSD's strlcpy(3)/strlcat(3), but:

-  It's simpler to implement.
-  It's faster.
-  It's simpler to detect truncation.

Signed-off-by: Alejandro Colomar <alx@kernel.org>
---

Of course this is still a very early patch.  I just compiled it,
but we'd need to write tests for it.  I didn't want to do all of that
work before the discussion.  Since the source code has been copied from
libstp, I can at least say that the function works, since I already used
that library, but this still needs a lot of work to adapt to glibc, I guess.

 string/Makefile  |  1 +
 string/stpecpy.c | 39 +++++++++++++++++++++++++++++++++++++++
 string/string.h  |  7 +++++++
 3 files changed, 47 insertions(+)
 create mode 100644 string/stpecpy.c

diff --git a/string/Makefile b/string/Makefile
index 938f528b8d..95e9ebce6d 100644
--- a/string/Makefile
+++ b/string/Makefile
@@ -73,6 +73,7 @@ routines := \
   sigabbrev_np \
   sigdescr_np \
   stpcpy \
+  stpecpy \
   stpncpy \
   strcasecmp \
   strcasecmp_l \
diff --git a/string/stpecpy.c b/string/stpecpy.c
new file mode 100644
index 0000000000..e6194559ee
--- /dev/null
+++ b/string/stpecpy.c
@@ -0,0 +1,39 @@
+/* Copyright (C) 2022 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 3.0 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <https://www.gnu.org/licenses/>.  */
+
+#include <string.h>
+
+char *
+stpecpy (char *dest, char *end, const char *restrict src)
+{
+	char  *p;
+
+	if (dest == end)
+		return end;
+	if (dest == NULL)  // Allow chaining with stpeprintf(3).
+		return NULL;
+	if (dest > end)
+		__builtin_unreachable();
+
+	p = memccpy(dest, src, '\0', end - dest);
+	if (p != NULL)
+		return p - 1;
+
+	/* Truncation detected. */
+	end[-1] = '\0';
+	return end;
+}
diff --git a/string/string.h b/string/string.h
index 54dd8344de..966a8cb744 100644
--- a/string/string.h
+++ b/string/string.h
@@ -502,6 +502,13 @@ extern char *stpncpy (char *__restrict __dest,
 #endif
 
 #ifdef	__USE_GNU
+/* Copy the string SRC into a null-terminated string at DEST,
+   truncating if it would run after END.  Return a pointer to
+   the terminating null byte, or END if the string was truncated,
+   or NULL if DEST was NULL. */
+extern char *stpecpy (char *__dest, char *__end, const char *__restrict __src)
+     __THROW __nonnull ((2, 3));
+
 /* Compare S1 and S2 as strings holding name & indices/version numbers.  */
 extern int strverscmp (const char *__s1, const char *__s2)
      __THROW __attribute_pure__ __nonnull ((1, 2));
-- 
2.39.0

Re: [PATCH 1/1] string: Add stpecpy(3)

[Sam James]

[-- Attachment #1: Type: text/plain, Size: 1197 bytes --]



> On 22 Dec 2022, at 21:42, Alejandro Colomar via Libc-alpha <libc-alpha@sourceware.org> wrote:
> 
> Glibc didn't provide any function that copies a string with truncation.
> 
> It only provided strncpy(3) and stpncpy(3), which copy from a string
> into a null-padded character sequence at the destination fixed-width
> buffer, with truncation.
> 
> Those old functions, which don't produce a string, have been misused for
> a long time as a poor-man's replacement for strlcpy(3), but doing that
> is a source of bugs, since it's hard to calculate the right size that
> should be passed to the function, and it's also necessary to explicitly
> terminate the buffer with a null byte.  Detecting truncation is yet
> another problem.
> 
> stpecpy(3), described in the new string_copying(7) manual page, is
> similar to OpenBSD's strlcpy(3)/strlcat(3), but:
> 
> -  It's simpler to implement.
> -  It's faster.
> -  It's simpler to detect truncation.

Given strlcpy and strlcat are in POSIX next and therefore bar
some extraordinary event will be in glibc, I think we should
probably wait until those two land, then see if there's still
an appetite for stpecpy in glibc.

[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 358 bytes --]

Re: [PATCH 1/1] string: Add stpecpy(3)

[Alejandro Colomar]

[-- Attachment #1.1: Type: text/plain, Size: 8478 bytes --]

Hi Sam!

On 12/23/22 08:02, Sam James wrote:
> 
> 
>> On 22 Dec 2022, at 21:42, Alejandro Colomar via Libc-alpha <libc-alpha@sourceware.org> wrote:
>>
>> Glibc didn't provide any function that copies a string with truncation.
>>
>> It only provided strncpy(3) and stpncpy(3), which copy from a string
>> into a null-padded character sequence at the destination fixed-width
>> buffer, with truncation.
>>
>> Those old functions, which don't produce a string, have been misused for
>> a long time as a poor-man's replacement for strlcpy(3), but doing that
>> is a source of bugs, since it's hard to calculate the right size that
>> should be passed to the function, and it's also necessary to explicitly
>> terminate the buffer with a null byte.  Detecting truncation is yet
>> another problem.
>>
>> stpecpy(3), described in the new string_copying(7) manual page, is
>> similar to OpenBSD's strlcpy(3)/strlcat(3), but:
>>
>> -  It's simpler to implement.
>> -  It's faster.
>> -  It's simpler to detect truncation.
> 
> Given strlcpy and strlcat are in POSIX next and therefore bar
> some extraordinary event will be in glibc, I think we should
> probably wait until those two land, then see if there's still
> an appetite for stpecpy in glibc.

I disagree for the following reasons.

strlcpy(3)/strlcat(3) are designed to be _slow_, in exchange for added 
simplicity and safety.  That's what Theo told me about them.  They didn't care 
about performance.  The two performance issues are:

-  Traverse the entire input string, to make sure it's a string.  stpecpy(3) 
instead only reads what's necessary for the copy; it stops reading after truncation.

-  strlcat(3) finds the terminating null byte; that's something you already know 
where it is, with functions that return a useful pointer (mempcpy(3), stpcpy(3), 
and stpecpy(3)).

While there are many programs that may be fine with the OpenBSD functions, glibc 
should _also_ provide a way to do the operation with an optimal API.  And it's 
in line with glibc providing stpcpy(3) and mempcpy(3) as extensions (stpcpy(3) 
is now in POSIX).


The reason that triggered me wanting to add this function is seeing strncpy(3) 
used for a patch to some glibc internals themselves.  Using strlcpy(3)/cat(3) in 
glibc internals would be bad for performance; I would hope that glibc uses the 
optimal internals, even if it also provides slow functions for users.

There are probably more cases within existing code in glibc.  Just check the 
output of:

     $ grep -rn st.ncpy -A1 | grep -B1 " = '\\\\0'"


Moreover, in the Austin discussion for strlcpy(3)/cat(3), it was mentioned that 
strlcpy(3) has an interface identical to that of snprintf(3), and that "If we 
truly think that this is bad design, should we come up with a new version of 
snprintf() that also doesn't do this? I don't think so.".

Well, I do believe snprintf is also misdesigned, for the same reasons that the 
strlcpy(3) manual page states that you should use strlcpy(3) for catenating 
strings, but rather strlcat(3):

        To detect truncation, perhaps while building a pathname, something like
        the following might be used:

              char *dir, *file, pname[MAXPATHLEN];

              ...

              if (strlcpy(pname, dir, sizeof(pname)) >= sizeof(pname))
                      goto toolong;
              if (strlcat(pname, file, sizeof(pname)) >= sizeof(pname))
                      goto toolong;

        Since it is known how many  characters  were  copied  the  first  time,
        things can be sped up a bit by using a copy instead of an append:

              char *dir, *file, pname[MAXPATHLEN];
              size_t n;

              ...

              n = strlcpy(pname, dir, sizeof(pname));
              if (n >= sizeof(pname))
                      goto toolong;
              if (strlcpy(pname + n, file, sizeof(pname) ‐ n) >= sizeof(pname) ‐ n)
                      goto toolong;

        However,  one  may question the validity of such optimizations, as they
        defeat the whole purpose of strlcpy() and strlcat().  As  a  matter  of
        fact, the first version of this manual page got it wrong.

Guess what?  There's no 'cat' version of snprintf, so users are doomed to write 
buggy code when trying to use it to concatenate after some other string.  I've 
recently been investigating a lot about it, and found invocations of Undefined 
Behavior, and some milder cases of benign off-by-one (on the safe side, by luck) 
errors, in calls to snprintf(3) in several important projects:

-  NGINX Unit:
    -  Undefined Behavior:
       <https://github.com/nginx/unit/issues/795#issuecomment-1345400420>

    -  Wrong truncation detection:
       <https://github.com/nginx/unit/pull/734#discussion_r1043963527>
       <https://github.com/nginx/unit/issues/804>

-  shadow:
    -  off-by-one:
       <https://github.com/shadow-maint/shadow/pull/607>

    -  clever code that looks like a bug but it's not:
       <https://github.com/shadow-maint/shadow/issues/608>

Rather than adding some catenating variant of snprintf(3), I suggest adding a 
single function that has an interface similar to stpcpy(3) and mempcpy(3), and 
identical to stpecpy(3):  stpeprintf(3):

<http://www.alejandro-colomar.es/src/alx/alx/libstp.git/tree/src/stp/stpe/stpeprintf.c>

I implemented it in terms of vsnprintf(3), so I need to handle a return of -1, 
but if implemented from scratch in glibc, in could be written to not be limited 
to INT_MAX (although I wonder why anyone would want to copy more than INT_MAX as 
a formatted string).


Below is its manual page in libstp.

Cheers,

Alex
---

stpeprintf(3)              Library Functions Manual              stpeprintf(3)

NAME
        stpeprintf, vstpeprintf - create a formatted string with truncation

LIBRARY
        Stp string library (libstp, pkgconf ‐‐cflags ‐‐libs libstp)

SYNOPSIS
        #include <stp/stpe/stpeprintf.h>

        char *_Nullable stpeprintf(char *_Nullable dst, char end[0],
                                   const char *restrict fmt, ...);
        char *_Nullable vstpeprintf(char *_Nullable dst, char end[0],
                                   const char *restrict fmt, va_list ap);

DESCRIPTION
        These functions are almost identical to snprintf(3) and vsnprintf(3).

        The  destination  buffer  is limited by a pointer to its end —one after
        its last element— instead of a size.

        These  functions  can  be  chained  with  calls  to  stpeprintf(3)  and
        vstpeprintf(3).

RETURN VALUE
        NULL
               •  If this function failed (see ERRORS).
               •  If dst was NULL.

        end
               •  If this call truncated.
               •  If  dst  was equal to end (a previous call to these functions
                  truncated).

        dst + strlen(dst)
               On success, these functions return a pointer to the  terminating
               null byte.

ERRORS
        These functions may fail for any of the same reasons as vsnprintf(3).

ATTRIBUTES
        For  an  explanation  of  the  terms  used in this section, see attrib‐
        utes(7).
        ┌────────────────────────────────────────────┬───────────────┬─────────┐
        │Interface                                   │ Attribute     │ Value   │
        ├────────────────────────────────────────────┼───────────────┼─────────┤
        │stpeprintf(3), vstpeprintf(3)               │ Thread safety │ MT‐Safe │
        └────────────────────────────────────────────┴───────────────┴─────────┘

STANDARDS
        None.

EXAMPLES
        See stpecpy(3).

SEE ALSO
        stpecpy(3), string_copying(7)

libstp (unreleased)                 (date)                       stpeprintf(3)








-- 
<http://www.alejandro-colomar.es/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH 1/1] string: Add stpecpy(3)

[Alejandro Colomar]

[-- Attachment #1.1: Type: text/plain, Size: 1807 bytes --]



On 12/23/22 13:26, Alejandro Colomar wrote:

> Well, I do believe snprintf is also misdesigned, for the same reasons that the 
> strlcpy(3) manual page states that you should use strlcpy(3) for catenating 

typo fix:  s/should/shouldn't/

> strings, but rather strlcat(3):
> 
>         To detect truncation, perhaps while building a pathname, something like
>         the following might be used:
> 
>               char *dir, *file, pname[MAXPATHLEN];
> 
>               ...
> 
>               if (strlcpy(pname, dir, sizeof(pname)) >= sizeof(pname))
>                       goto toolong;
>               if (strlcat(pname, file, sizeof(pname)) >= sizeof(pname))
>                       goto toolong;
> 
>         Since it is known how many  characters  were  copied  the  first  time,
>         things can be sped up a bit by using a copy instead of an append:
> 
>               char *dir, *file, pname[MAXPATHLEN];
>               size_t n;
> 
>               ...
> 
>               n = strlcpy(pname, dir, sizeof(pname));
>               if (n >= sizeof(pname))
>                       goto toolong;
>               if (strlcpy(pname + n, file, sizeof(pname) ‐ n) >= sizeof(pname) ‐ n)
>                       goto toolong;
> 
>         However,  one  may question the validity of such optimizations, as they
>         defeat the whole purpose of strlcpy() and strlcat().  As  a  matter  of
>         fact, the first version of this manual page got it wrong.

-- 
<http://www.alejandro-colomar.es/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH 1/1] string: Add stpecpy(3)

[Alejandro Colomar]

[-- Attachment #1.1: Type: text/plain, Size: 2559 bytes --]

Hi Wilco,

On 12/23/22 13:26, Alejandro Colomar wrote:
>         However,  one  may question the validity of such optimizations, as they
>         defeat the whole purpose of strlcpy() and strlcat().  As  a  matter  of
>         fact, the first version of this manual page got it wrong.
> 
> Guess what?  There's no 'cat' version of snprintf, so users are doomed to write 
> buggy code when trying to use it to concatenate after some other string.  I've 
> recently been investigating a lot about it, and found invocations of Undefined 
> Behavior, and some milder cases of benign off-by-one (on the safe side, by luck) 
> errors, in calls to snprintf(3) in several important projects:
> 
> -  NGINX Unit:
>     -  Undefined Behavior:
>        <https://github.com/nginx/unit/issues/795#issuecomment-1345400420>
> 
>     -  Wrong truncation detection:
>        <https://github.com/nginx/unit/pull/734#discussion_r1043963527>
>        <https://github.com/nginx/unit/issues/804>
> 
> -  shadow:
>     -  off-by-one:
>        <https://github.com/shadow-maint/shadow/pull/607>
> 
>     -  clever code that looks like a bug but it's not:
>        <https://github.com/shadow-maint/shadow/issues/608>

And adding strlcat(3) doesn't address the issue about snprintf(3), which, as 
EdSchouten said in the Austin discussion:

"
- strlcpy() fits within the existing set of functions like a glove. strlcpy(a, 
b, n) behaves identically to snprintf(a, n, "%s", b). The return value always 
corresponds to the number of non-null bytes that would have been written. If we 
truly think that this is bad design, should we come up with a new version of 
snprintf() that also doesn't do this? I don't think so.
"

I only disagree in the last part ("I don't think so").  As I linked in my 
previous message, there have been numerous misuses of snprintf(3), due to the 
fact that it's not designed to be concatenated.  But of course there's no 
alternative, so the only way is using it, and hoping that you didn't introduce a 
bug.

Steffen (was it Nurpmeso?) in that same discussion rebated the claims about 
strlcpy(3) with performance claims that snprintf(3) is slow, but that was the 
least evil.  The real evil with snprintf(3) is that it doesn't have a 'cat' 
complement.

<https://www.austingroupbugs.net/view.php?id=986>

So, I'll send a second revision of the patch set to add stpeprintf(3).

Cheers,

Alex

-- 
<http://www.alejandro-colomar.es/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[PATCH v2 0/3] Add stpe*() functions

[Alejandro Colomar]

Hi,

This version of the patch set adds the [v]stpeprintf() functions, which
are more necessary than stpecpy(3).  snprintf(3) is the only way to
catenate formatted strings, and it's really bad for that.

stpecpy(3) has been optimized/reduced as much as I could.

Cheers,

Alex

Alejandro Colomar (3):
  string: Add stpecpy()
  stdio: Add vstpeprintf()
  stdio: Add stpeprintf()

 libio/Makefile            |  4 +--
 libio/stdio.h             | 10 ++++++++
 libio/vstpeprintf.c       | 52 +++++++++++++++++++++++++++++++++++++++
 stdio-common/Makefile     |  1 +
 stdio-common/stpeprintf.c | 32 ++++++++++++++++++++++++
 string/Makefile           |  1 +
 string/stpecpy.c          | 45 +++++++++++++++++++++++++++++++++
 string/string.h           |  7 ++++++
 8 files changed, 150 insertions(+), 2 deletions(-)
 create mode 100644 libio/vstpeprintf.c
 create mode 100644 stdio-common/stpeprintf.c
 create mode 100644 string/stpecpy.c

-- 
2.39.0

[PATCH v2 1/3] string: Add stpecpy()

[Alejandro Colomar]

This function is similar to stpcpy(3), but it tuncates the destination
string if it doesn't fit the buffer.  It's much simpler to use than
strscpy(9) or strlcpy(3), and slightly faster.

It also allows chaining with stpeprintf(3), which has the same interface
as stpecpy(3), but prints a formatted string (like snprintf(3)).

Signed-off-by: Alejandro Colomar <alx@kernel.org>
---
 string/Makefile  |  1 +
 string/stpecpy.c | 45 +++++++++++++++++++++++++++++++++++++++++++++
 string/string.h  |  7 +++++++
 3 files changed, 53 insertions(+)
 create mode 100644 string/stpecpy.c

diff --git a/string/Makefile b/string/Makefile
index 938f528b8d..95e9ebce6d 100644
--- a/string/Makefile
+++ b/string/Makefile
@@ -73,6 +73,7 @@ routines := \
   sigabbrev_np \
   sigdescr_np \
   stpcpy \
+  stpecpy \
   stpncpy \
   strcasecmp \
   strcasecmp_l \
diff --git a/string/stpecpy.c b/string/stpecpy.c
new file mode 100644
index 0000000000..6884a949a0
--- /dev/null
+++ b/string/stpecpy.c
@@ -0,0 +1,45 @@
+/* Copyright (C) 2022 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <https://www.gnu.org/licenses/>.  */
+
+#ifdef HAVE_CONFIG_H
+# include <config.h>
+#endif
+
+#include <stdbool.h>
+#include <string.h>
+
+char *
+stpecpy(char *dst, char *end, const char *restrict src)
+{
+	bool    trunc;
+	size_t  dsize, dlen, slen;
+
+	if (dst == end)
+		return end;
+	if (dst == NULL)  // Allow chaining with stpeprintf().
+		return NULL;
+	if (dst > end)
+		__builtin_unreachable();
+
+	dsize = end - dst;
+	slen = strnlen(src, dsize);
+	trunc = (slen == dsize);
+	dlen = slen - trunc;
+	dst[dlen] = '\0';
+
+	return mempcpy(dst, src, dlen) + trunc;
+}
diff --git a/string/string.h b/string/string.h
index 54dd8344de..966a8cb744 100644
--- a/string/string.h
+++ b/string/string.h
@@ -502,6 +502,13 @@ extern char *stpncpy (char *__restrict __dest,
 #endif
 
 #ifdef	__USE_GNU
+/* Copy the string SRC into a null-terminated string at DEST,
+   truncating if it would run after END.  Return a pointer to
+   the terminating null byte, or END if the string was truncated,
+   or NULL if DEST was NULL. */
+extern char *stpecpy (char *__dest, char *__end, const char *__restrict __src)
+     __THROW __nonnull ((2, 3));
+
 /* Compare S1 and S2 as strings holding name & indices/version numbers.  */
 extern int strverscmp (const char *__s1, const char *__s2)
      __THROW __attribute_pure__ __nonnull ((1, 2));
-- 
2.39.0

[PATCH v2 2/3] stdio: Add vstpeprintf()

[Alejandro Colomar]

[v]snprintf(3) is error-prone, since it doesn't allow easy catenation or
chaining.  To catenate a formatted string after an existing string, the
only possibility was to call [v]snprintf(3), adjusting the sizes and
pointers.  However, that is very error-prone, and has caused several
bugs in existing software.  I found several just in a small
investigation in some noteworthy open-source projects.

This API solves that problem by receiving a pointer to the end of the
destination buffer, so there's no recalculation involved.  It also
always returns a pointer suitable for chaining with other calls to this
function, or calls to stpecpy(3).

Signed-off-by: Alejandro Colomar <alx@kernel.org>
---
 libio/Makefile      |  4 ++--
 libio/stdio.h       |  6 ++++++
 libio/vstpeprintf.c | 52 +++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 60 insertions(+), 2 deletions(-)
 create mode 100644 libio/vstpeprintf.c

diff --git a/libio/Makefile b/libio/Makefile
index 64398ab1ee..1924f7a65c 100644
--- a/libio/Makefile
+++ b/libio/Makefile
@@ -43,8 +43,8 @@ routines	:=							      \
 									      \
 	clearerr feof ferror fileno fputc freopen fseek getc getchar	      \
 	memstream pclose putc putchar rewind setbuf setlinebuf vasprintf      \
-	iovdprintf vscanf vsnprintf obprintf fcloseall fseeko ftello	      \
-	freopen64 fseeko64 ftello64					      \
+	iovdprintf vscanf vsnprintf vstpeprintf obprintf fcloseall	      \
+	fseeko ftello freopen64 fseeko64 ftello64			      \
 									      \
 	__fbufsize __freading __fwriting __freadable __fwritable __flbf	      \
 	__fpurge __fpending __fsetlocking				      \
diff --git a/libio/stdio.h b/libio/stdio.h
index 0e0f16b464..59b8047ecc 100644
--- a/libio/stdio.h
+++ b/libio/stdio.h
@@ -384,6 +384,12 @@ extern int vsnprintf (char *__restrict __s, size_t __maxlen,
      __THROWNL __attribute__ ((__format__ (__printf__, 3, 0)));
 #endif
 
+#if __USE_GNU
+extern char *vstpeprintf (char *__dest, char *__end,
+			  const char *__restrict __fmt, __gnuc_va_list __arg)
+     __THROWNL __attribute__ ((__format__ (__printf__, 3, 0)));
+#endif
+
 #if __GLIBC_USE (LIB_EXT2)
 /* Write formatted output to a string dynamically allocated with `malloc'.
    Store the address of the string in *PTR.  */
diff --git a/libio/vstpeprintf.c b/libio/vstpeprintf.c
new file mode 100644
index 0000000000..a8becdc682
--- /dev/null
+++ b/libio/vstpeprintf.c
@@ -0,0 +1,52 @@
+/* Copyright (C) 2022 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <https://www.gnu.org/licenses/>.
+
+   As a special exception, if you link the code in this file with
+   files compiled with a GNU compiler to produce an executable,
+   that does not cause the resulting executable to be covered by
+   the GNU Lesser General Public License.  This exception does not
+   however invalidate any other reasons why the executable file
+   might be covered by the GNU Lesser General Public License.
+   This exception applies to code released by its copyright holders
+   in files containing the exception.  */
+
+#include <stdarg.h>
+#include <libioP.h>
+
+
+char *
+vstpeprintf(char *dst, char *end, const char *restrict fmt, va_list ap)
+{
+	int  dsize, len;
+
+	if (dst == end)
+		return end;
+	if (dst == NULL)
+		return NULL;
+	if (dst > end)
+		__builtin_unreachable();
+
+	dsize = end - dst;
+	len = __vsnprintf_internal(dst, dsize, fmt, ap, 0);
+
+	if (len == -1)
+		return NULL;
+	if (len >= dsize)
+		return end;
+
+	return dst + len;
+}
-- 
2.39.0

[PATCH v2 3/3] stdio: Add stpeprintf()

[Alejandro Colomar]

Add variadic-argument version of vstpeprintf(3).

Signed-off-by: Alejandro Colomar <alx@kernel.org>
---
 libio/stdio.h             |  4 ++++
 stdio-common/Makefile     |  1 +
 stdio-common/stpeprintf.c | 32 ++++++++++++++++++++++++++++++++
 3 files changed, 37 insertions(+)
 create mode 100644 stdio-common/stpeprintf.c

diff --git a/libio/stdio.h b/libio/stdio.h
index 59b8047ecc..7456ea6885 100644
--- a/libio/stdio.h
+++ b/libio/stdio.h
@@ -385,6 +385,10 @@ extern int vsnprintf (char *__restrict __s, size_t __maxlen,
 #endif
 
 #if __USE_GNU
+extern char *stpeprintf (char *__dest, char *__end,
+			 const char *__restrict __fmt, ...)
+     __THROWNL __attribute__ ((__format__ (__printf__, 3, 4)));
+
 extern char *vstpeprintf (char *__dest, char *__end,
 			  const char *__restrict __fmt, __gnuc_va_list __arg)
      __THROWNL __attribute__ ((__format__ (__printf__, 3, 0)));
diff --git a/stdio-common/Makefile b/stdio-common/Makefile
index 3e0c574ca5..d92942a86f 100644
--- a/stdio-common/Makefile
+++ b/stdio-common/Makefile
@@ -79,6 +79,7 @@ routines := \
   snprintf \
   sprintf \
   sscanf \
+  stpeprintf \
   tempnam \
   tempname \
   tmpfile \
diff --git a/stdio-common/stpeprintf.c b/stdio-common/stpeprintf.c
new file mode 100644
index 0000000000..7b953be3ee
--- /dev/null
+++ b/stdio-common/stpeprintf.c
@@ -0,0 +1,32 @@
+/* Copyright (C) 2022 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <https://www.gnu.org/licenses/>.  */
+
+#include <stdarg.h>
+#include <stdio.h>
+
+char *
+stpeprintf(char *dst, char *end, const char *restrict fmt, ...)
+{
+	char     *p;
+	va_list  ap;
+
+	va_start(ap, fmt);
+	p = vstpeprintf(dst, end, fmt, ap);
+	va_end(ap);
+
+	return p;
+}
-- 
2.39.0

Re: [PATCH v2 1/3] string: Add stpecpy()

[Alejandro Colomar]

[-- Attachment #1.1: Type: text/plain, Size: 3908 bytes --]

stpecpy(3)                 Library Functions Manual                 stpecpy(3)

NAME
        stpecpy - copy a string with truncation

LIBRARY
        stp string library (libstp, pkgconf ‐‐cflags ‐‐libs libstp)

SYNOPSIS
        #include <stp/stpe/stpecpy.h>

        char *_Nullable stpecpy(char *_Nullable dst, char end[0],
                                const char *restrict src);

DESCRIPTION
        This function copies the string pointed to by src, into a string at the
        buffer  pointed  to  by  dst.   If the destination buffer, limited by a
        pointer to its end —one after its last element—, isn’t large enough  to
        hold the copy, the resulting string is truncated.

        This   function   can  be  chained  with  calls  to  stpeprintf(3)  and
        vstpeprintf(3).

        An implementation of this function might be

            /* This code is in the public domain. */

            char *
            stpecpy(char *dst, char end[0], const char *restrict src)
            {
                char *p;

                if (dst == end || dst == NULL)
                    return dst;

                p = memccpy(dst, src, '\0', end - dst);
                if (p != NULL)
                    return p - 1;

                /* truncation detected */
                end[-1] = '\0';
                return end;
            }

RETURN VALUE
        NULL   If dst was NULL.

        end
               •  If this call truncated.
               •  If dst was equal to end (a previous  call  to  this  function
                  truncated).

        dst + strlen(dst)
               On  success,  this function returns a pointer to the terminating
               null byte.

ATTRIBUTES
        For an explanation of the terms  used  in  this  section,  see  attrib‐
        utes(7).
        ┌────────────────────────────────────────────┬───────────────┬─────────┐
        │Interface                                   │ Attribute     │ Value   │
        ├────────────────────────────────────────────┼───────────────┼─────────┤
        │stpecpy(3)                                  │ Thread safety │ MT‐Safe │
        └────────────────────────────────────────────┴───────────────┴─────────┘

STANDARDS
        None.

EXAMPLES
        $ cc ./stpecpy.c $(pkgconf --cflags --libs libbsd-overlay libstp)
        $ ./a.out
        [len = 12]: Hello world!
        $

        // stpecpy.c
        #include <err.h>
        #include <stdio.h>
        #include <stdlib.h>

        #include <stp/stpe/stpecpy.h>
        #include <stp/stpe/stpeprintf.h>

        int
        main(void)
        {
            char    *p, *end;
            char    buf[BUFSIZ];
            size_t  len;

            end = buf + BUFSIZ;
            p = buf;
            p = stpecpy(p, end, "Hello, ");
            p = stpeprintf(p, end, "%d worlds", 22);
            p = stpecpy(p, end, "!");
            if (p == NULL)
                err(EXIT_FAILURE, "stpeprintf()");
            if (p == end) {
                p--;
                warnx("Truncated");
            }
            len = p - buf;
            printf("[len = %zu]: ", len);
            puts(buf);

            exit(EXIT_SUCCESS);
        }

SEE ALSO
        stpeprintf(3), string_copying(7)

libstp (unreleased)                 (date)                          stpecpy(3)

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v2 3/3] stdio: Add stpeprintf()

[Alejandro Colomar]

[-- Attachment #1.1: Type: text/plain, Size: 2628 bytes --]

stpeprintf(3)              Library Functions Manual              stpeprintf(3)

NAME
        stpeprintf, vstpeprintf - create a formatted string with truncation

LIBRARY
        stp string library (libstp, pkgconf ‐‐cflags ‐‐libs libstp)

SYNOPSIS
        #include <stp/stpe/stpeprintf.h>

        char *_Nullable stpeprintf(char *_Nullable dst, char end[0],
                                   const char *restrict fmt, ...);
        char *_Nullable vstpeprintf(char *_Nullable dst, char end[0],
                                   const char *restrict fmt, va_list ap);

DESCRIPTION
        These functions are almost identical to snprintf(3) and vsnprintf(3).

        The  destination  buffer  is limited by a pointer to its end —one after
        its last element— instead of a size.

        These  functions  can  be  chained  with  calls  to  stpeprintf(3)  and
        vstpeprintf(3).

RETURN VALUE
        NULL
               •  If this function failed (see ERRORS).
               •  If dst was NULL.

        end
               •  If this call truncated.
               •  If  dst  was equal to end (a previous call to these functions
                  truncated).

        dst + strlen(dst)
               On success, these functions return a pointer to the  terminating
               null byte.

ERRORS
        These functions may fail for any of the same reasons as vsnprintf(3).

ATTRIBUTES
        For  an  explanation  of  the  terms  used in this section, see attrib‐
        utes(7).
        ┌────────────────────────────────────────────┬───────────────┬─────────┐
        │Interface                                   │ Attribute     │ Value   │
        ├────────────────────────────────────────────┼───────────────┼─────────┤
        │stpeprintf(3), vstpeprintf(3)               │ Thread safety │ MT‐Safe │
        └────────────────────────────────────────────┴───────────────┴─────────┘

STANDARDS
        None.

EXAMPLES
        See stpecpy(3).

SEE ALSO
        stpecpy(3), string_copying(7)

libstp (unreleased)                 (date)                       stpeprintf(3)

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v2 0/3] Add stpe*() functions

[Zack Weinberg]

On Wed, Dec 28, 2022, at 6:17 PM, Alejandro Colomar via Libc-alpha wrote:
> This version of the patch set adds the [v]stpeprintf() functions, which
> are more necessary than stpecpy(3).  snprintf(3) is the only way to
> catenate formatted strings, and it's really bad for that.

I don't necessarily _oppose_ the addition of these functions but I wonder
whether the uses you have in mind would be satisfied by open_memstream() +
fprintf().

zw

Re: [PATCH v2 0/3] Add stpe*() functions

[Alejandro Colomar]

[-- Attachment #1.1: Type: text/plain, Size: 1211 bytes --]

[please CC me]

Hi Zack,

On 12/29/22 01:01, Zack Weinberg via Libc-alpha wrote:
> On Wed, Dec 28, 2022, at 6:17 PM, Alejandro Colomar via Libc-alpha wrote:
>> This version of the patch set adds the [v]stpeprintf() functions, which
>> are more necessary than stpecpy(3).  snprintf(3) is the only way to
>> catenate formatted strings, and it's really bad for that.
> 
> I don't necessarily _oppose_ the addition of these functions but I wonder
> whether the uses you have in mind would be satisfied by open_memstream() +
> fprintf().

There are uses which could be covered by it.  However, several of the cases 
where I found dubious code around snprintf(3) (when not straight bugs) can't use 
it.  Most of the use cases of snprintf(3) were legitimately truncating; for 
example, one of them was creating a path, and anything over PATH_MAX would be an 
error.  In Nginx code, performance also matters a lot, so I guess this function 
has unnecessary overhead due to realloc(3) (although I don't know how much 
that's relevant, since snprintf(3) is already quite slow).

But it's an interesting alternative; thanks!

Cheers,

Alex

> 
> zw

-- 
<http://www.alejandro-colomar.es/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH 1/1] string: Add stpecpy(3)

[Sam James]

[-- Attachment #1: Type: text/plain, Size: 1762 bytes --]



> On 23 Dec 2022, at 12:26, Alejandro Colomar via Libc-alpha <libc-alpha@sourceware.org> wrote:
> 
> Hi Sam!
> 
> On 12/23/22 08:02, Sam James wrote:
>>> On 22 Dec 2022, at 21:42, Alejandro Colomar via Libc-alpha <libc-alpha@sourceware.org> wrote:
>>> 
>>> Glibc didn't provide any function that copies a string with truncation.
>>> 
>>> It only provided strncpy(3) and stpncpy(3), which copy from a string
>>> into a null-padded character sequence at the destination fixed-width
>>> buffer, with truncation.
>>> 
>>> Those old functions, which don't produce a string, have been misused for
>>> a long time as a poor-man's replacement for strlcpy(3), but doing that
>>> is a source of bugs, since it's hard to calculate the right size that
>>> should be passed to the function, and it's also necessary to explicitly
>>> terminate the buffer with a null byte.  Detecting truncation is yet
>>> another problem.
>>> 
>>> stpecpy(3), described in the new string_copying(7) manual page, is
>>> similar to OpenBSD's strlcpy(3)/strlcat(3), but:
>>> 
>>> -  It's simpler to implement.
>>> -  It's faster.
>>> -  It's simpler to detect truncation.
>> Given strlcpy and strlcat are in POSIX next and therefore bar
>> some extraordinary event will be in glibc, I think we should
>> probably wait until those two land, then see if there's still
>> an appetite for stpecpy in glibc.
> 
> I disagree for the following reasons.
[snip]

Hi Alex,

Thanks for your detailed and thoughtful reply. I'll reflect on your comments here
and in the rest of the thread(s) - but there's some intriguing pointers you've made.

I wasn't trying to be dismissive at all so I hope it didn't come across like that.

Thank you again!

Best,
sam


[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 358 bytes --]

Re: [PATCH 1/1] string: Add stpecpy(3)

[Alejandro Colomar]

[-- Attachment #1.1: Type: text/plain, Size: 555 bytes --]

Hey Sam!

On 12/31/22 16:13, Sam James wrote:
[...]

>>
>> I disagree for the following reasons.
> [snip]
> 
> Hi Alex,
> 
> Thanks for your detailed and thoughtful reply. I'll reflect on your comments here
> and in the rest of the thread(s) - but there's some intriguing pointers you've made.
> 
> I wasn't trying to be dismissive at all so I hope it didn't come across like that.

Ahh, no, I didn't take it bad; really :)

> 
> Thank you again!

Cheers!

Alex

> 
> Best,
> sam
> 

-- 
<http://www.alejandro-colomar.es/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]