strtol(3) setting of errno

[Alejandro Colomar <alx@kernel.org>]

[-- Attachment #1: Type: text/plain, Size: 3470 bytes --]

Hi,

We were discussing a use of strtol(3) in shadow-utils, and after reading
strtol(3) several times, I'm not sure about the exact interface of the
function.

Normally, libc functions are allowed to set errno on success, and a
caller should not inspect errno, even if it was set to 0 prior to the
call, unless the function reported an error via the return value.

However, strtol(3) is a bit special, in that it's one of the few libc
functions that report an error with a return value that is also in the
range of valid return values of the function.

So, here's how I understand the function works:

-  If the base is unsupported, return 0, and set errno to EINVAL.  Let's
   ignore this error for the rest of the question, since usually you set
   the base to something common, likely 0 or 10.

-  If no digits were found (no conversion is performed), return 0, set
   *endptr == str, and possibly set errno to EINVAL (setting errno is
   allowed but not required).

-  If the conversion would have overflowed, return LONG_MAX, and set
   errno to ERANGE.
-  If the conversion would have underflowed, return LONG_MIN, and set
   errno to ERANGE.

-  If the conversion succeeded, return the value, which may or may not
   be 0, LONG_MIN, or LONG_MAX.  And the question here is: is there any
   guarantee that strtol(3) won't set errno in this case?

I ask because the manual page says:

NOTES
     Since  strtol()  can legitimately return 0, LONG_MAX, or LONG_MIN
     (LLONG_MAX or LLONG_MIN for strtoll()) on both success and  fail‐
     ure,  the  calling program should set errno to 0 before the call,
     and then determine if an error occurred by checking whether errno
     has a nonzero value after the call.

And then in EXAMPLES:

         val = strtol(str, &endptr, base);

         /* Check for various possible errors. */

         if (errno != 0) {
             perror("strtol");
             exit(EXIT_FAILURE);
         }

         if (endptr == str) {
             fprintf(stderr, "No digits were found\n");
             exit(EXIT_FAILURE);
         }

         /* If we got here, strtol() successfully parsed a number. */

Which is consistent with a possible interpretation of what NOTES says,
but I think that may be because it is a bit ambiguous.  The example
program is my fault, because I changed that code:

	commit 93f369892aeab4d56b92962224e318f739ee2455
	Author: Alejandro Colomar <colomar.6.4.3@gmail.com>
	Date:   Wed Oct 28 10:33:08 2020 +0100

	    strtol.3: EXAMPLES: Simplify errno checking
	    
	    (No expected change in behavior,)
	    
	    Signed-off-by: Alejandro Colomar <colomar.6.4.3@gmail.com>
	    Signed-off-by: Michael Kerrisk <mtk.manpages@gmail.com>

	diff --git a/man3/strtol.3 b/man3/strtol.3
	index a436bcac4..3889ef6b5 100644
	--- a/man3/strtol.3
	+++ b/man3/strtol.3
	@@ -276,8 +276,7 @@ .SS Program source
	 
	     /* Check for various possible errors */
	 
	-    if ((errno == ERANGE && (val == LONG_MAX || val == LONG_MIN))
	-            || (errno != 0 && val == 0)) {
	+    if (errno != 0) {
		 perror("strtol");
		 exit(EXIT_FAILURE);
	     }

Now I realize that commit was probably wrong, and one needs to check
both errno and the return value to determine that the call failed.  Can
you please confirm what the correct specification of strtol(3) is?

Thanks,
Alex

-- 
<https://www.alejandro-colomar.es/>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]