strxfrm() returns an incorrect value on a short buffer

strxfrm() returns an incorrect value on a short buffer

[Tony Cook]

strxfrm() returns an incorrect value if you supply an output buffer
and that buffer is too short for the result.

With the code following:

#include <string.h>
#include <locale.h>
#include <stdio.h>

int main() {
  char xbuf[5] = "";
  char *lc = setlocale(LC_ALL, "en_AU.utf8");
  if (!lc) {
    perror("setlocale");
    return 1;
  }
  size_t sza = strxfrm(xbuf, "alphabet", sizeof(xbuf));
  printf("sz: %zd\n", sza);
  size_t szb = strxfrm(NULL, "alphabet", 0);
  printf("sz: %zd\n", szb);

  return 0;
}

On cygwin:

tony@phobos ~
$ gcc -ostrxfrmtest strxfrmtest.c

tony@phobos ~
$ ./strxfrmtest
sz: 5
sz: 20

tony@phobos ~
$ uname -a
CYGWIN_NT-6.1-WOW phobos 2.5.0(0.297/5/3) 2016-04-11 09:55 i686 Cygwin

On Linux:

tony@mars:~$ gcc -ostrxfrmtest strxfrmtest.c 
tony@mars:~$ ./strxfrmtest
sz: 26
sz: 26
tony@mars:~$ uname -a
Linux mars 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt20-1+deb8u3 (2016-01-17) x86_64 GNU/Linux

From looking at the source:

https://cygwin.com/git/gitweb.cgi?p=newlib-cygwin.git;a=blob;f=winsup/cygwin/nlsfuncs.cc;h=9dbd9b16d53094c60aa835756c967c054ced8e62;hb=HEAD#l1286

It appears that strxfrm() is just returning the size of the output
buffer on an overflow error rather than calling LCMapString() again
with cchDest set to zero to get the required buffer length that
strxfrm() is meant to return on a short buffer.

This came out of the discussion in:

https://rt.perl.org/Ticket/Display.html?id=121734

(not that I've reproduced that issue.)

Tony

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: strxfrm() returns an incorrect value on a short buffer

[Marco Atzeri]

On 12/04/2016 07:07, Tony Cook wrote:
> strxfrm() returns an incorrect value if you supply an output buffer
> and that buffer is too short for the result.
>
> With the code following:
>
> #include <string.h>
> #include <locale.h>
> #include <stdio.h>
>
> int main() {
>    char xbuf[5] = "";
>    char *lc = setlocale(LC_ALL, "en_AU.utf8");
>    if (!lc) {
>      perror("setlocale");
>      return 1;
>    }
>    size_t sza = strxfrm(xbuf, "alphabet", sizeof(xbuf));
>    printf("sz: %zd\n", sza);
>    size_t szb = strxfrm(NULL, "alphabet", 0);
>    printf("sz: %zd\n", szb);
>
>    return 0;
> }
>
> On cygwin:
>
> tony@phobos ~
> $ gcc -ostrxfrmtest strxfrmtest.c
>
> tony@phobos ~
> $ ./strxfrmtest
> sz: 5
> sz: 20
>
> tony@phobos ~
> $ uname -a
> CYGWIN_NT-6.1-WOW phobos 2.5.0(0.297/5/3) 2016-04-11 09:55 i686 Cygwin
>
> On Linux:
>
> tony@mars:~$ gcc -ostrxfrmtest strxfrmtest.c
> tony@mars:~$ ./strxfrmtest
> sz: 26
> sz: 26
> tony@mars:~$ uname -a
> Linux mars 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt20-1+deb8u3 (2016-01-17) x86_64 GNU/Linux
>
>  From looking at the source:
>
> https://cygwin.com/git/gitweb.cgi?p=newlib-cygwin.git;a=blob;f=winsup/cygwin/nlsfuncs.cc;h=9dbd9b16d53094c60aa835756c967c054ced8e62;hb=HEAD#l1286
>
> It appears that strxfrm() is just returning the size of the output
> buffer on an overflow error rather than calling LCMapString() again
> with cchDest set to zero to get the required buffer length that
> strxfrm() is meant to return on a short buffer.
>
> This came out of the discussion in:
>
> https://rt.perl.org/Ticket/Display.html?id=121734
>
> (not that I've reproduced that issue.)
>
> Tony
>

You should check the error output.
Both cygwin and Linux seem IMHO to report an insufficient buffer
and s1 is not guaranteed.

http://pubs.opengroup.org/onlinepubs/9699919799/functions/strxfrm.html

     Upon successful completion, strxfrm() [CX] [Option Start]  and 
strxfrm_l() [Option End]  shall return the length of the transformed 
string (not including the terminating NUL character). If the value 
returned is n or more, the contents of the array pointed to by s1 are 
unspecified.

     On error, strxfrm() [CX] [Option Start]  and strxfrm_l() [Option 
End]  may set errno but no return value is reserved to indicate an error.


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: strxfrm() returns an incorrect value on a short buffer

[Achim Gratz]

Tony Cook <tony <at> develop-help.com> writes:
> strxfrm() returns an incorrect value if you supply an output buffer
> and that buffer is too short for the result.

The text in the C standard is:

>>>>>
The strxfrm function returns the length of the transformed string (not
including the
terminating null character). If the value returned is n or more, the
contents of the array
pointed to by s1 are indeterminate.

EXAMPLE The value of the following expression is the size of the array
needed to hold the
transformation of the string pointed to by s.
1 + strxfrm(NULL, s, 0)
<<<<<

It doesn't really provide for an explanation of what should happen if you
start with a buffer that is too small, but from the standpoint of
defensiveness, if you are getting the size of your buffer or larger, then
you should ask again with a size of zero to get the actual minimum size
needed or try again with a larger buffer until the returned value is smaller
than the buffer size.

> It appears that strxfrm() is just returning the size of the output
> buffer on an overflow error rather than calling LCMapString() again
> with cchDest set to zero to get the required buffer length that
> strxfrm() is meant to return on a short buffer.

So, you may be expecting something that the standard doesn't explicitly
specify, although you might reasonhably invoke that Cygwin should behave
like Linux in this case.


Regards,
Achim.


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: strxfrm() returns an incorrect value on a short buffer

[Tony Cook]

On Tue, Apr 12, 2016 at 10:20:13AM +0000, Achim Gratz wrote:
> Tony Cook <tony <at> develop-help.com> writes:
> > strxfrm() returns an incorrect value if you supply an output buffer
> > and that buffer is too short for the result.
> 
> The text in the C standard is:
> 
> >>>>>
> The strxfrm function returns the length of the transformed string (not
> including the
> terminating null character). If the value returned is n or more, the
> contents of the array
> pointed to by s1 are indeterminate.
> 
> EXAMPLE The value of the following expression is the size of the array
> needed to hold the
> transformation of the string pointed to by s.
> 1 + strxfrm(NULL, s, 0)
> <<<<<
> 
> It doesn't really provide for an explanation of what should happen if you
> start with a buffer that is too small, but from the standpoint of
> defensiveness, if you are getting the size of your buffer or larger, then
> you should ask again with a size of zero to get the actual minimum size
> needed or try again with a larger buffer until the returned value is smaller
> than the buffer size.
> 
> > It appears that strxfrm() is just returning the size of the output
> > buffer on an overflow error rather than calling LCMapString() again
> > with cchDest set to zero to get the required buffer length that
> > strxfrm() is meant to return on a short buffer.
> 
> So, you may be expecting something that the standard doesn't explicitly
> specify, although you might reasonhably invoke that Cygwin should behave
> like Linux in this case.

The specification of strxfrm() in the standard doesn't special-case a
length of zero beyond allowing for s1 to be NULL.

If an implementation were permitted to return the lesser of the full
length of the transformed string and the size of the buffer the
example in the standard wouldn't return what the description says it
does.

Tony

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: strxfrm() returns an incorrect value on a short buffer

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 831 bytes --]

On Apr 12 15:07, Tony Cook wrote:
> strxfrm() returns an incorrect value if you supply an output buffer
> and that buffer is too short for the result.
> 
> With the code following:
> [...]
> It appears that strxfrm() is just returning the size of the output
> buffer on an overflow error rather than calling LCMapString() again
> with cchDest set to zero to get the required buffer length that
> strxfrm() is meant to return on a short buffer.

Thanks for the testcase.  I applied a patch

  https://sourceware.org/git/?p=newlib-cygwin.git;h=e1854211

and created new snapshots on

  https://cygwin.com/snapshots/

Please give them a try.


Thanks,
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: strxfrm() returns an incorrect value on a short buffer

[Tony Cook]

On Tue, Apr 12, 2016 at 03:30:49PM +0200, Corinna Vinschen wrote:
> On Apr 12 15:07, Tony Cook wrote:
> > strxfrm() returns an incorrect value if you supply an output buffer
> > and that buffer is too short for the result.
> > 
> > With the code following:
> > [...]
> > It appears that strxfrm() is just returning the size of the output
> > buffer on an overflow error rather than calling LCMapString() again
> > with cchDest set to zero to get the required buffer length that
> > strxfrm() is meant to return on a short buffer.
> 
> Thanks for the testcase.  I applied a patch
> 
>   https://sourceware.org/git/?p=newlib-cygwin.git;h=e1854211
> 
> and created new snapshots on
> 
>   https://cygwin.com/snapshots/
> 
> Please give them a try.

Thanks, fixed in snapshot 20160412.

Tony

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: strxfrm() returns an incorrect value on a short buffer

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1074 bytes --]

On Apr 13 10:46, Tony Cook wrote:
> On Tue, Apr 12, 2016 at 03:30:49PM +0200, Corinna Vinschen wrote:
> > On Apr 12 15:07, Tony Cook wrote:
> > > strxfrm() returns an incorrect value if you supply an output buffer
> > > and that buffer is too short for the result.
> > > 
> > > With the code following:
> > > [...]
> > > It appears that strxfrm() is just returning the size of the output
> > > buffer on an overflow error rather than calling LCMapString() again
> > > with cchDest set to zero to get the required buffer length that
> > > strxfrm() is meant to return on a short buffer.
> > 
> > Thanks for the testcase.  I applied a patch
> > 
> >   https://sourceware.org/git/?p=newlib-cygwin.git;h=e1854211
> > 
> > and created new snapshots on
> > 
> >   https://cygwin.com/snapshots/
> > 
> > Please give them a try.
> 
> Thanks, fixed in snapshot 20160412.

Thanks for your feedback,
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]