[Bug libstdc++/37475] New: codecvt::do_in/do_out functions return "ok" when the output sequence has zero length

[Bug libstdc++/37475] New: codecvt::do_in/do_out functions return "ok" when the output sequence has zero length

[tsyvarev at ispras dot ru]

The following member functions of the class codecvt<wchar_t, char, mbstate_t>

result in(stateT& state, const externT* from, const externT* from_end, const
externT*& from_next, internT* to, internT* to_limit, internT*& to_next) const

and

result out(stateT& state, const internT* from, const internT* from_end, const
internT*& from_next, externT* to, externT* to_limit, externT*& to_next) const

return "ok" if (to==to_limit) but (from < from_end), that is, when the output
sequence contains no elements but the input sequence is not empty. 

However, as appears from the description of the functions' return values
(22.2.1.5.2 p4), "partial" should be returned instead:

ok - completed the conversion
partial - not all source characters converted
error - encountered a character in [from,from_end) that it could not convert
noconv - internT and externT are the same type, and input sequence is identical
to converted sequence

Note that these functions do return "partial" if the output sequence is not
empty but still not large enough to contain all converted characters from the
input sequence, that is, if 
0< (to_limit - to) < (from_end - from).

andrew@Ubuntu:/mnt/hgfs/shared/temp/test$ g++ -Wall test.cpp && ./a.out
Calls do_out() function when size of input sequenceis 2, output - 1:
do_out() returns partial.
Calls do_out() function when size of input sequenceis 2, output - 0:
do_out() returns ok.
Calls do_in() function when size of input sequenceis 2, output - 1:
do_in() returns partial.
Calls do_in() function when size of input sequenceis 2, output - 0:
do_in() returns ok.
andrew@Ubuntu:/mnt/hgfs/shared/temp/test$ g++ --version
g++ (GCC) 4.1.2 (Ubuntu 4.1.2-0ubuntu4)
Copyright (C) 2006 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.


-- 
           Summary: codecvt::do_in/do_out functions return "ok" when the
                    output sequence has zero length
           Product: gcc
           Version: unknown
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: libstdc++
        AssignedTo: unassigned at gcc dot gnu dot org
        ReportedBy: tsyvarev at ispras dot ru


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=37475

[Bug libstdc++/37475] codecvt::do_in/do_out functions return "ok" when the output sequence has zero length

[tsyvarev at ispras dot ru]

------- Comment #1 from tsyvarev at ispras dot ru  2008-09-11 09:35 -------
Created an attachment (id=16291)
 --> (http://gcc.gnu.org/bugzilla/attachment.cgi?id=16291&action=view)
test.cpp


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=37475

[Bug libstdc++/37475] codecvt::do_in/do_out functions return "ok" when the output sequence has zero length

[paolo dot carlini at oracle dot com]

------- Comment #2 from paolo dot carlini at oracle dot com  2008-09-11 10:31 -------
http://gcc.gnu.org/ml/libstdc++/2008-09/msg00090.html


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=37475

[Bug libstdc++/37475] [DR 382] codecvt::do_in/do_out functions return "ok" when the output sequence has zero length

[paolo dot carlini at oracle dot com]

-- 

paolo dot carlini at oracle dot com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |SUSPENDED


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=37475

[Bug libstdc++/37475] [DR 382] codecvt::do_in/do_out functions return "ok" when the output sequence has zero length

[paolo dot carlini at oracle dot com]

-- 

paolo dot carlini at oracle dot com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |NEW
     Ever Confirmed|0                           |1
   Last reconfirmed|0000-00-00 00:00:00         |2008-09-11 17:39:18
               date|                            |
            Summary|codecvt::do_in/do_out       |[DR 382]
                   |functions return "ok" when  |codecvt::do_in/do_out
                   |the output sequence has zero|functions return "ok" when
                   |length                      |the output sequence has zero
                   |                            |length


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=37475

[Bug libstdc++/37475] [DR 382] codecvt::do_in/do_out functions return "ok" when the output sequence has zero length

[redi at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=37475

Jonathan Wakely <redi at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|SUSPENDED                   |NEW

--- Comment #8 from Jonathan Wakely <redi at gcc dot gnu.org> ---
I assume you mean 14.1.0

Since 382 indeed doesn't seem relevant here, let's unsuspend this.

[Bug libstdc++/37475] [DR 382] codecvt::do_in/do_out functions return "ok" when the output sequence has zero length

[kristian.spangsege at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=37475

--- Comment #7 from Kristian Spangsege <kristian.spangsege at gmail dot com> ---
This bug still exists in GCC 14.0.1. This was tested with the attacked
test_2.cpp. The actual result is `ok`, but it should have been `partial`.

Curiously, this bug does not occur when using the Cygwin or MinGW versions of
GCC. In these cases, the result is `partial` as it should be. I assume this is
because on Cygwin and MinGW, libstdc++ uses a generic locale implementation
that is different from the one ordinarily used on Linux.

[Bug libstdc++/37475] [DR 382] codecvt::do_in/do_out functions return "ok" when the output sequence has zero length

[kristian.spangsege at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=37475

--- Comment #6 from Kristian Spangsege <kristian.spangsege at gmail dot com> ---
Created attachment 58405
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=58405&action=edit
Test whether bug still exists in GCC 14.0.1

[Bug libstdc++/37475] [DR 382] codecvt::do_in/do_out functions return "ok" when the output sequence has zero length

[kristian.spangsege at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=37475

Kristian Spangsege <kristian.spangsege at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |kristian.spangsege at gmail dot co
                   |                            |m

--- Comment #5 from Kristian Spangsege <kristian.spangsege at gmail dot com> ---
This bug still exists in GCC 10.2, while libc++ (clang) appears to do the right
thing.

Specifically, when passing a nonzero amount of data (from_end > from) to
std::codecvt::out(), but an empty output buffer (to_end == to), the return
value is "ok". According to the C++ standard (C++17), the correct return value
is "partial".

This appears to happen regardless of locale. In particular, it happens in the C
locale.

As far as I can tell, http://cplusplus.github.io/LWG/lwg-closed.html#382 is not
relevant here, because the current standard text is clear (C++17), and
http://cplusplus.github.io/LWG/lwg-closed.html#382 has status NAD, which I
assume means "not a defect".

Jonathan, let me know if I am missing something.

[Bug libstdc++/37475] [DR 382] codecvt::do_in/do_out functions return "ok" when the output sequence has zero length

[redi at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=37475

--- Comment #3 from Jonathan Wakely <redi at gcc dot gnu.org> ---
Issue 382 is NAD
http://cplusplus.github.io/LWG/lwg-defects.html#2369