[Bug libc/28524] New: Conversion from ISO-2022-JP-3 with iconv may emit spurious NUL character on state reset

[Bug libc/28524] New: Conversion from ISO-2022-JP-3 with iconv may emit spurious NUL character on state reset

[npv1310 at gmail dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=28524

            Bug ID: 28524
           Summary: Conversion from ISO-2022-JP-3 with iconv may emit
                    spurious NUL character on state reset
           Product: glibc
           Version: unspecified
            Status: UNCONFIRMED
          Severity: normal
          Priority: P2
         Component: libc
          Assignee: unassigned at sourceware dot org
          Reporter: npv1310 at gmail dot com
                CC: drepper.fsp at gmail dot com
  Target Milestone: ---

Created attachment 13751
  --> https://sourceware.org/bugzilla/attachment.cgi?id=13751&action=edit
Proof of concept

Hello, while investigating CVE-2021-3326 patch [here:
https://sourceware.org/git/?p=glibc.git;a=commit;h=7d88c6142c6efc160c0ee5e4f85cde382c072888]
I've found that it is possible to force iconv() to emit spurious NUL character
by converting from ISO-2022-JP-3 encoding and resetting internal state (inbuf =
NULL). This is possible because code sequence in iconvdata/iso-2022-jp-3.c
which is responsible for resetting internal state doesn't ensure that any
character has actually been saved. The main condition [data->__statep->__count
!= ASCII_set] also picks a cases where current set is different from ASCII_set.
I've written small proof-of-concept which is attached.
In a research environment, proof-of-concept program was compiled with following
command sequence:
$ gcc iconv-issue.c -c -Wall -o /tmp/iconv-issue.o
$ gcc -o /tmp/iconv-issue -nostdlib -nostartfiles /root/glibc-build/csu/Scrt1.o
/root/glibc-build/csu/crti.o `gcc  --print-file-name=crtbeginS.o`
/tmp/iconv-issue.o
-Wl,-rpath-link=/root/glibc-build:/root/glibc-build/math:/root/glibc-build/elf:/root/glibc-build/dlfcn:/root/glibc-build/nss:/root/glibc-build/nis:/root/glibc-build/rt:/root/glibc-build/resolv:/root/glibc-build/mathvec:/root/glibc-build/support:/root/glibc-build/crypt:/root/glibc-build/nptl
/root/glibc-build/libc.so.6 `gcc  --print-file-name=crtendS.o`
/root/glibc-build/csu/crtn.o
$ env GCONV_PATH=/root/glibc-build/iconvdata
LOCPATH=/root/glibc-build/localedata LC_ALL=C
/root/glibc-build/elf/ld-linux-x86-64.so.2 --library-path
/root/glibc-build:/root/glibc-build/math:/root/glibc-build/elf:/root/glibc-build/dlfcn:/root/glibc-build/nss:/root/glibc-build/nis:/root/glibc-build/rt:/root/glibc-build/resolv:/root/glibc-build/mathvec:/root/glibc-build/support:/root/glibc-build/crypt:/root/glibc-build/nptl
/tmp/iconv-issue
Here /root/glibc-build is a build root for up-to-date version of glibc.

The last command results in a following output:
Step#1
 Input characters consumed = 3
Output characters produced = 0
Step#2
Output characters produced = 1

With glibc lacking CVE-2021-3326 fix output is normal:
Step#1
 Input characters consumed = 3
Output characters produced = 0
Step#2
Output characters produced = 0

I assess that this behavior may affect data integrity in certain use patterns
of iconv where extra data ('\0') is added.
Such NUL character may confuse programs expecting NUL as the data terminator.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug libc/28524] Conversion from ISO-2022-JP-3 with iconv may emit spurious NUL character on state reset

[fweimer at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=28524

Florian Weimer <fweimer at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
              Flags|                            |security-
                 CC|                            |fweimer at redhat dot com

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug libc/28524] Conversion from ISO-2022-JP-3 with iconv may emit spurious NUL character on state reset

[npv1310 at gmail dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=28524

--- Comment #1 from Nikita Popov <npv1310 at gmail dot com> ---
Created attachment 13756
  --> https://sourceware.org/bugzilla/attachment.cgi?id=13756&action=edit
Proposed patch

Submitted proposed patch to address this issue.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug libc/28524] Conversion from ISO-2022-JP-3 with iconv may emit spurious NUL character on state reset

[fweimer at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=28524

Florian Weimer <fweimer at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|---                         |2.35
         Resolution|---                         |FIXED
             Status|UNCONFIRMED                 |RESOLVED

--- Comment #2 from Florian Weimer <fweimer at redhat dot com> ---
Fixed for glibc 2.35 via:

commit ff012870b2c02a62598c04daa1e54632e020fd7d
Author: Nikita Popov <npv1310@gmail.com>
Date:   Tue Nov 2 13:21:42 2021 +0500

    gconv: Do not emit spurious NUL character in ISO-2022-JP-3 (bug 28524)

    Bugfix 27256 has introduced another issue:
    In conversion from ISO-2022-JP-3 encoding, it is possible
    to force iconv to emit extra NUL character on internal state reset.
    To do this, it is sufficient to feed iconv with escape sequence
    which switches active character set.
    The simplified check 'data->__statep->__count != ASCII_set'
    introduced by the aforementioned bugfix picks that case and
    behaves as if '\0' character has been queued thus emitting it.

    To eliminate this issue, these steps are taken:
    * Restore original condition
    '(data->__statep->__count & ~7) != ASCII_set'.
    It is necessary since bits 0-2 may contain
    number of buffered input characters.
    * Check that queued character is not NUL.
    Similar step is taken for main conversion loop.

    Bundled test case follows following logic:
    * Try to convert ISO-2022-JP-3 escape sequence
    switching active character set
    * Reset internal state by providing NULL as input buffer
    * Ensure that nothing has been converted.

    Signed-off-by: Nikita Popov <npv1310@gmail.com>

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug libc/28524] Conversion from ISO-2022-JP-3 with iconv may emit spurious NUL character on state reset

[sam at gentoo dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=28524

Sam James <sam at gentoo dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |sam at gentoo dot org

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug libc/28524] Conversion from ISO-2022-JP-3 with iconv may emit spurious NUL character on state reset

[siddhesh at sourceware dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=28524

Siddhesh Poyarekar <siddhesh at sourceware dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |siddhesh at sourceware dot org

--- Comment #3 from Siddhesh Poyarekar <siddhesh at sourceware dot org> ---
AFAICT the bug cannot be invoked through user input and requires iconv to be
invoked with NULL inbuf, which ought to require a separate application bug to
do so unintentionally.  Hence there's no security impact to the bug.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug libc/28524] Conversion from ISO-2022-JP-3 with iconv may emit spurious NUL character on state reset

[npv1310 at gmail dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=28524

--- Comment #4 from Nikita Popov <npv1310 at gmail dot com> ---
"the bug cannot be invoked through user input and requires iconv to be invoked
with NULL inbuf"
I never claimed opposite. I mentioned "certain use patterns" where reset
operation on iconv state should ensue. But, considering the importance of the
GLIBC project, I believe the issue in question is worth fixing.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug libc/28524] Conversion from ISO-2022-JP-3 with iconv may emit spurious NUL character on state reset

[siddhesh at sourceware dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=28524

--- Comment #5 from Siddhesh Poyarekar <siddhesh at sourceware dot org> ---
(In reply to Nikita Popov from comment #4)
> operation on iconv state should ensue. But, considering the importance of
> the GLIBC project, I believe the issue in question is worth fixing.

Agreed, and thank you for the fix!

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug libc/28524] Conversion from ISO-2022-JP-3 with iconv may emit spurious NUL character on state reset

[fweimer at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=28524

--- Comment #6 from Florian Weimer <fweimer at redhat dot com> ---
(In reply to Nikita Popov from comment #4)
> "the bug cannot be invoked through user input and requires iconv to be
> invoked with NULL inbuf"
> I never claimed opposite. I mentioned "certain use patterns" where reset
> operation on iconv state should ensue. But, considering the importance of
> the GLIBC project, I believe the issue in question is worth fixing.

Just to provide some context: We are trying to explain here why this isn't a
*security* bug (it has been flagged as a security issue elsewhere, presumably
by accident because it was a regression introduced by a security fix). Of
course it's a bug, and thank you for reporting and fixing it!

Without concrete evidence of application impact, I think this bug is just glibc
computing an incorrect result. Any bug could theoretically introduce an
application vulnerability, but we have to draw a line somewhere because
otherwise the distinction between security and non-security bugs becomes
meaningless.

As far as I understand it, this issue can only occur if the input sequence does
not return to the initial shift state, which is already partially corrupted. 
Otherwise there isn't any work left to do for the do_flush case in
iconv/skeleton.c, and the bug does not materialize at all.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug libc/28524] Conversion from ISO-2022-JP-3 with iconv may emit spurious NUL character on state reset

[bruno at clisp dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=28524

Bruno Haible <bruno at clisp dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |bruno at clisp dot org

--- Comment #7 from Bruno Haible <bruno at clisp dot org> ---
Created attachment 14901
  --> https://sourceware.org/bugzilla/attachment.cgi?id=14901&action=edit
non-roundtrip mappings in a GB18030 converter

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug libc/28524] Conversion from ISO-2022-JP-3 with iconv may emit spurious NUL character on state reset

[bruno at clisp dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=28524

Bruno Haible <bruno at clisp dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #14901|0                           |1
        is obsolete|                            |
  Attachment #14901|non-roundtrip mappings in a |Mistake
        description|GB18030 converter           |

--- Comment #8 from Bruno Haible <bruno at clisp dot org> ---
Comment on attachment 14901
  --> https://sourceware.org/bugzilla/attachment.cgi?id=14901
Mistake

Forget the added attachment; it does not belong here.

-- 
You are receiving this mail because:
You are on the CC list for the bug.