public inbox for libc-alpha@sourceware.org
 help / color / mirror / Atom feed
* [PATCH] gconv: Do not emit spurious NUL character in ISO-2022-JP-3 (bug 28524)
@ 2021-11-02  9:06 Никита Попов
  2021-11-03  4:20 ` Никита Попов
                   ` (2 more replies)
  0 siblings, 3 replies; 6+ messages in thread
From: Никита Попов @ 2021-11-02  9:06 UTC (permalink / raw)
  To: libc-alpha

[-- Attachment #1: Type: text/plain, Size: 54 bytes --]

Hello, I'm submitting a proposed patch for bug 28524.

[-- Attachment #2: 0001-gconv-Do-not-emit-spurious-NUL-character-in-ISO-2022.patch --]
[-- Type: text/x-patch, Size: 6318 bytes --]

From 2fa94ed223424fe62d1e3ef02a4b562e0e164eac Mon Sep 17 00:00:00 2001
From: Nikita Popov <npv1310@gmail.com>
Date: Tue, 2 Nov 2021 13:21:42 +0500
Subject: [PATCH] gconv: Do not emit spurious NUL character in ISO-2022-JP-3
 (bug 28524)

Bugfix 27256 has introduced another issue:
In conversion from ISO-2022-JP-3 encoding, it is possible
to force iconv to emit extra NUL character on internal state reset.
To do this, it is sufficient to feed iconv with escape sequence
which switches active character set.
The simplified check 'data->__statep->__count != ASCII_set'
introduced by the aforementioned bugfix picks that case and
behaves as if '\0' character has been queued thus emitting it.

To eliminate this issue, these steps are taken:
* Restore original condition
'(data->__statep->__count & ~7) != ASCII_set'.
It is necessary since bits 0-2 may contain
number of buffered input characters.
* Check that queued character is not NUL.
Similar step is taken for main conversion loop.

Bundled test case follows following logic:
* Try to convert ISO-2022-JP-3 escape sequence
switching active character set
* Reset internal state by providing NULL as input buffer
* Ensure that nothing has been converted.

Signed-off-by: Nikita Popov <npv1310@gmail.com>
---
 iconvdata/Makefile        |  4 ++-
 iconvdata/bug-iconv15.c   | 55 +++++++++++++++++++++++++++++++++++++++
 iconvdata/iso-2022-jp-3.c | 27 +++++++++++++------
 3 files changed, 77 insertions(+), 9 deletions(-)
 create mode 100644 iconvdata/bug-iconv15.c

diff --git a/iconvdata/Makefile b/iconvdata/Makefile
index c216f959df..f7888de29c 100644
--- a/iconvdata/Makefile
+++ b/iconvdata/Makefile
@@ -74,7 +74,7 @@ ifeq (yes,$(build-shared))
 tests = bug-iconv1 bug-iconv2 tst-loading tst-e2big tst-iconv4 bug-iconv4 \
 	tst-iconv6 bug-iconv5 bug-iconv6 tst-iconv7 bug-iconv8 bug-iconv9 \
 	bug-iconv10 bug-iconv11 bug-iconv12 tst-iconv-big5-hkscs-to-2ucs4 \
-	bug-iconv13 bug-iconv14
+	bug-iconv13 bug-iconv14 bug-iconv15
 ifeq ($(have-thread-library),yes)
 tests += bug-iconv3
 endif
@@ -327,6 +327,8 @@ $(objpfx)bug-iconv12.out: $(addprefix $(objpfx), $(gconv-modules)) \
 			  $(addprefix $(objpfx),$(modules.so))
 $(objpfx)bug-iconv14.out: $(addprefix $(objpfx), $(gconv-modules)) \
 			  $(addprefix $(objpfx),$(modules.so))
+$(objpfx)bug-iconv15.out: $(addprefix $(objpfx), $(gconv-modules)) \
+			  $(addprefix $(objpfx),$(modules.so))
 
 $(objpfx)iconv-test.out: run-iconv-test.sh \
 			 $(addprefix $(objpfx), $(gconv-modules)) \
diff --git a/iconvdata/bug-iconv15.c b/iconvdata/bug-iconv15.c
new file mode 100644
index 0000000000..4037e131ff
--- /dev/null
+++ b/iconvdata/bug-iconv15.c
@@ -0,0 +1,55 @@
+/* Bug 28524: Conversion from ISO-2022-JP-3 with iconv
+   may emit spurious NUL character on state reset.
+   Copyright (C) The GNU Toolchain Authors.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <https://www.gnu.org/licenses/>.  */
+
+#include <stddef.h>
+#include <iconv.h>
+#include <support/check.h>
+
+static int
+do_test (void)
+{
+  char in[] = "\x1b(I";
+  char *inbuf = in;
+  size_t inleft = sizeof (in) - 1;
+  char out[1];
+  char *outbuf = out;
+  size_t outleft = sizeof (out);
+  iconv_t cd;
+
+  cd = iconv_open ("UTF8", "ISO-2022-JP-3");
+  TEST_VERIFY_EXIT (cd != (iconv_t) -1);
+
+  /* First call to iconv should alter internal state.
+     Now, JISX0201_Kana_set is selected and
+     state value != ASCII_set */
+  TEST_VERIFY (iconv (cd, &inbuf, &inleft, &outbuf, &outleft) != (size_t) -1);
+
+  /* Second call shall emit spurious NUL character in unpatched glibc. */
+  TEST_VERIFY (iconv (cd, NULL, NULL, &outbuf, &outleft) != (size_t) -1);
+
+  /* No characters are expected to be produced. */
+  TEST_VERIFY (outbuf == out);
+  TEST_VERIFY (outleft == sizeof (out));
+
+  TEST_VERIFY_EXIT (iconv_close (cd) != -1);
+
+  return 0;
+}
+
+#include <support/test-driver.c>
diff --git a/iconvdata/iso-2022-jp-3.c b/iconvdata/iso-2022-jp-3.c
index 70b28ace7f..5e66d686f1 100644
--- a/iconvdata/iso-2022-jp-3.c
+++ b/iconvdata/iso-2022-jp-3.c
@@ -79,20 +79,31 @@ enum
    the output state to the initial state.  This has to be done during the
    flushing.  */
 #define EMIT_SHIFT_TO_INIT \
-  if (data->__statep->__count != ASCII_set)			      \
+  if ((data->__statep->__count & ~7) != ASCII_set)			      \
     {									      \
       if (FROM_DIRECTION)						      \
 	{								      \
-	  if (__glibc_likely (outbuf + 4 <= outend))			      \
+	  uint32_t ch = data->__statep->__count >> 6;			      \
+									      \
+	  if (__glibc_unlikely (ch != 0))				      \
 	    {								      \
-	      /* Write out the last character.  */			      \
-	      *((uint32_t *) outbuf) = data->__statep->__count >> 6;	      \
-	      outbuf += sizeof (uint32_t);				      \
-	      data->__statep->__count = ASCII_set;			\
+	      if (__glibc_likely (outbuf + 4 <= outend))		      \
+		{							      \
+		  /* Write out the last character.  */			      \
+		  put32u (outbuf, ch);					      \
+		  outbuf += 4;						      \
+		  data->__statep->__count &= 7;				      \
+		  data->__statep->__count |= ASCII_set;			      \
+		}							      \
+	      else							      \
+		/* We don't have enough room in the output buffer.  */	      \
+		status = __GCONV_FULL_OUTPUT;				      \
 	    }								      \
 	  else								      \
-	    /* We don't have enough room in the output buffer.  */	      \
-	    status = __GCONV_FULL_OUTPUT;				      \
+	    {								      \
+	      data->__statep->__count &= 7;				      \
+	      data->__statep->__count |= ASCII_set;			      \
+	    }								      \
 	}								      \
       else								      \
 	{								      \
-- 
2.17.1


^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH] gconv: Do not emit spurious NUL character in ISO-2022-JP-3 (bug 28524)
  2021-11-02  9:06 [PATCH] gconv: Do not emit spurious NUL character in ISO-2022-JP-3 (bug 28524) Никита Попов
@ 2021-11-03  4:20 ` Никита Попов
  2021-11-04 14:10 ` Florian Weimer
  2021-11-04 14:32 ` Florian Weimer
  2 siblings, 0 replies; 6+ messages in thread
From: Никита Попов @ 2021-11-03  4:20 UTC (permalink / raw)
  To: libc-alpha

Hello, are there any updates on this issue and proposed patch? I'm
looking forward to receiving valuable feedback from you. Thank you!

вт, 2 нояб. 2021 г. в 14:06, Никита Попов <npv1310@gmail.com>:
>
> Hello, I'm submitting a proposed patch for bug 28524.

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH] gconv: Do not emit spurious NUL character in ISO-2022-JP-3 (bug 28524)
  2021-11-02  9:06 [PATCH] gconv: Do not emit spurious NUL character in ISO-2022-JP-3 (bug 28524) Никита Попов
  2021-11-03  4:20 ` Никита Попов
@ 2021-11-04 14:10 ` Florian Weimer
  2021-11-04 14:32 ` Florian Weimer
  2 siblings, 0 replies; 6+ messages in thread
From: Florian Weimer @ 2021-11-04 14:10 UTC (permalink / raw)
  To: Никита
	Попов via Libc-alpha
  Cc: Никита
	Попов

* Никита Попов via Libc-alpha:

> To eliminate this issue, these steps are taken:
> * Restore original condition
> '(data->__statep->__count & ~7) != ASCII_set'.
> It is necessary since bits 0-2 may contain
> number of buffered input characters.

Right, I missed that use of __count by the framework. 8-/

> * Check that queued character is not NUL.
> Similar step is taken for main conversion loop.

And since EMIT_SHIFT_TO_INIT runs before the loop, it's necessary to
mirror that logic there.

> diff --git a/iconvdata/bug-iconv15.c b/iconvdata/bug-iconv15.c
> new file mode 100644
> index 0000000000..4037e131ff

> +  /* First call to iconv should alter internal state.
> +     Now, JISX0201_Kana_set is selected and
> +     state value != ASCII_set */

Sorry, GNU style says that comments should end with ”.  */” (period and
two spaces).

> +  TEST_VERIFY (iconv (cd, &inbuf, &inleft, &outbuf, &outleft) != (size_t) -1);

Can we add an additional test here?  I think no bytes have been added to
the output at this point, right?

> +  /* Second call shall emit spurious NUL character in unpatched glibc. */

> +  /* No characters are expected to be produced. */

Missing space after ”.” (see above).

> diff --git a/iconvdata/iso-2022-jp-3.c b/iconvdata/iso-2022-jp-3.c
> index 70b28ace7f..5e66d686f1 100644
> --- a/iconvdata/iso-2022-jp-3.c
> +++ b/iconvdata/iso-2022-jp-3.c
> @@ -79,20 +79,31 @@ enum
>     the output state to the initial state.  This has to be done during the
>     flushing.  */
>  #define EMIT_SHIFT_TO_INIT \
> -  if (data->__statep->__count != ASCII_set)			      \
> +  if ((data->__statep->__count & ~7) != ASCII_set)			      \
>      {									      \
>        if (FROM_DIRECTION)						      \
>  	{								      \
> -	  if (__glibc_likely (outbuf + 4 <= outend))			      \
> +	  uint32_t ch = data->__statep->__count >> 6;			      \
> +									      \
> +	  if (__glibc_unlikely (ch != 0))				      \
>  	    {								      \
> -	      /* Write out the last character.  */			      \
> -	      *((uint32_t *) outbuf) = data->__statep->__count >> 6;	      \
> -	      outbuf += sizeof (uint32_t);				      \
> -	      data->__statep->__count = ASCII_set;			\
> +	      if (__glibc_likely (outbuf + 4 <= outend))		      \
> +		{							      \
> +		  /* Write out the last character.  */			      \
> +		  put32u (outbuf, ch);					      \
> +		  outbuf += 4;						      \
> +		  data->__statep->__count &= 7;				      \
> +		  data->__statep->__count |= ASCII_set;			      \
> +		}							      \
> +	      else							      \
> +		/* We don't have enough room in the output buffer.  */	      \
> +		status = __GCONV_FULL_OUTPUT;				      \
>  	    }								      \
>  	  else								      \
> -	    /* We don't have enough room in the output buffer.  */	      \
> -	    status = __GCONV_FULL_OUTPUT;				      \
> +	    {								      \
> +	      data->__statep->__count &= 7;				      \
> +	      data->__statep->__count |= ASCII_set;			      \
> +	    }								      \
>  	}								      \
>        else								      \
>  	{								      \

The actual code change looks okay to me.

Thanks,
Florian


^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH] gconv: Do not emit spurious NUL character in ISO-2022-JP-3 (bug 28524)
  2021-11-02  9:06 [PATCH] gconv: Do not emit spurious NUL character in ISO-2022-JP-3 (bug 28524) Никита Попов
  2021-11-03  4:20 ` Никита Попов
  2021-11-04 14:10 ` Florian Weimer
@ 2021-11-04 14:32 ` Florian Weimer
  2021-11-04 15:03   ` Никита Попов
  2 siblings, 1 reply; 6+ messages in thread
From: Florian Weimer @ 2021-11-04 14:32 UTC (permalink / raw)
  To: Никита
	Попов via Libc-alpha
  Cc: Никита
	Попов

* Никита Попов via Libc-alpha:

> diff --git a/iconvdata/Makefile b/iconvdata/Makefile
> index c216f959df..f7888de29c 100644
> --- a/iconvdata/Makefile
> +++ b/iconvdata/Makefile

> diff --git a/iconvdata/iso-2022-jp-3.c b/iconvdata/iso-2022-jp-3.c
> index 70b28ace7f..5e66d686f1 100644
> --- a/iconvdata/iso-2022-jp-3.c
> +++ b/iconvdata/iso-2022-jp-3.c
> @@ -79,20 +79,31 @@ enum

Sorry, I forgot to mention that the patch should add

+   Copyright (C) The GNU Toolchain Authors.

to these files as well because of the DCO nature of the submission.

Thanks,
Florian


^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH] gconv: Do not emit spurious NUL character in ISO-2022-JP-3 (bug 28524)
  2021-11-04 14:32 ` Florian Weimer
@ 2021-11-04 15:03   ` Никита Попов
  2021-11-04 19:34     ` Florian Weimer
  0 siblings, 1 reply; 6+ messages in thread
From: Никита Попов @ 2021-11-04 15:03 UTC (permalink / raw)
  To: Florian Weimer; +Cc: libc-alpha

[-- Attachment #1: Type: text/plain, Size: 785 bytes --]

Hello, I got the points, sending you an adjusted patch. Regards.

чт, 4 нояб. 2021 г. в 19:32, Florian Weimer <fweimer@redhat.com>:
>
> * Никита Попов via Libc-alpha:
>
> > diff --git a/iconvdata/Makefile b/iconvdata/Makefile
> > index c216f959df..f7888de29c 100644
> > --- a/iconvdata/Makefile
> > +++ b/iconvdata/Makefile
>
> > diff --git a/iconvdata/iso-2022-jp-3.c b/iconvdata/iso-2022-jp-3.c
> > index 70b28ace7f..5e66d686f1 100644
> > --- a/iconvdata/iso-2022-jp-3.c
> > +++ b/iconvdata/iso-2022-jp-3.c
> > @@ -79,20 +79,31 @@ enum
>
> Sorry, I forgot to mention that the patch should add
>
> +   Copyright (C) The GNU Toolchain Authors.
>
> to these files as well because of the DCO nature of the submission.
>
> Thanks,
> Florian
>

[-- Attachment #2: 0001-gconv-Do-not-emit-spurious-NUL-character-in-ISO-2022.patch --]
[-- Type: text/x-patch, Size: 6993 bytes --]

From d8321b3b4399a6e1999bd0ebd1466f6235dc5630 Mon Sep 17 00:00:00 2001
From: Nikita Popov <npv1310@gmail.com>
Date: Tue, 2 Nov 2021 13:21:42 +0500
Subject: [PATCH] gconv: Do not emit spurious NUL character in ISO-2022-JP-3
 (bug 28524)

Bugfix 27256 has introduced another issue:
In conversion from ISO-2022-JP-3 encoding, it is possible
to force iconv to emit extra NUL character on internal state reset.
To do this, it is sufficient to feed iconv with escape sequence
which switches active character set.
The simplified check 'data->__statep->__count != ASCII_set'
introduced by the aforementioned bugfix picks that case and
behaves as if '\0' character has been queued thus emitting it.

To eliminate this issue, these steps are taken:
* Restore original condition
'(data->__statep->__count & ~7) != ASCII_set'.
It is necessary since bits 0-2 may contain
number of buffered input characters.
* Check that queued character is not NUL.
Similar step is taken for main conversion loop.

Bundled test case follows following logic:
* Try to convert ISO-2022-JP-3 escape sequence
switching active character set
* Reset internal state by providing NULL as input buffer
* Ensure that nothing has been converted.

Signed-off-by: Nikita Popov <npv1310@gmail.com>
---
 iconvdata/Makefile        |  5 +++-
 iconvdata/bug-iconv15.c   | 60 +++++++++++++++++++++++++++++++++++++++
 iconvdata/iso-2022-jp-3.c | 28 ++++++++++++------
 3 files changed, 84 insertions(+), 9 deletions(-)
 create mode 100644 iconvdata/bug-iconv15.c

diff --git a/iconvdata/Makefile b/iconvdata/Makefile
index c216f959df..d5507a048c 100644
--- a/iconvdata/Makefile
+++ b/iconvdata/Makefile
@@ -1,4 +1,5 @@
 # Copyright (C) 1997-2021 Free Software Foundation, Inc.
+# Copyright (C) The GNU Toolchain Authors.
 # This file is part of the GNU C Library.
 
 # The GNU C Library is free software; you can redistribute it and/or
@@ -74,7 +75,7 @@ ifeq (yes,$(build-shared))
 tests = bug-iconv1 bug-iconv2 tst-loading tst-e2big tst-iconv4 bug-iconv4 \
 	tst-iconv6 bug-iconv5 bug-iconv6 tst-iconv7 bug-iconv8 bug-iconv9 \
 	bug-iconv10 bug-iconv11 bug-iconv12 tst-iconv-big5-hkscs-to-2ucs4 \
-	bug-iconv13 bug-iconv14
+	bug-iconv13 bug-iconv14 bug-iconv15
 ifeq ($(have-thread-library),yes)
 tests += bug-iconv3
 endif
@@ -327,6 +328,8 @@ $(objpfx)bug-iconv12.out: $(addprefix $(objpfx), $(gconv-modules)) \
 			  $(addprefix $(objpfx),$(modules.so))
 $(objpfx)bug-iconv14.out: $(addprefix $(objpfx), $(gconv-modules)) \
 			  $(addprefix $(objpfx),$(modules.so))
+$(objpfx)bug-iconv15.out: $(addprefix $(objpfx), $(gconv-modules)) \
+			  $(addprefix $(objpfx),$(modules.so))
 
 $(objpfx)iconv-test.out: run-iconv-test.sh \
 			 $(addprefix $(objpfx), $(gconv-modules)) \
diff --git a/iconvdata/bug-iconv15.c b/iconvdata/bug-iconv15.c
new file mode 100644
index 0000000000..cc04bd0313
--- /dev/null
+++ b/iconvdata/bug-iconv15.c
@@ -0,0 +1,60 @@
+/* Bug 28524: Conversion from ISO-2022-JP-3 with iconv
+   may emit spurious NUL character on state reset.
+   Copyright (C) The GNU Toolchain Authors.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <https://www.gnu.org/licenses/>.  */
+
+#include <stddef.h>
+#include <iconv.h>
+#include <support/check.h>
+
+static int
+do_test (void)
+{
+  char in[] = "\x1b(I";
+  char *inbuf = in;
+  size_t inleft = sizeof (in) - 1;
+  char out[1];
+  char *outbuf = out;
+  size_t outleft = sizeof (out);
+  iconv_t cd;
+
+  cd = iconv_open ("UTF8", "ISO-2022-JP-3");
+  TEST_VERIFY_EXIT (cd != (iconv_t) -1);
+
+  /* First call to iconv should alter internal state.
+     Now, JISX0201_Kana_set is selected and
+     state value != ASCII_set.  */
+  TEST_VERIFY (iconv (cd, &inbuf, &inleft, &outbuf, &outleft) != (size_t) -1);
+
+  /* No bytes should have been added to
+     the output buffer at this point.  */
+  TEST_VERIFY (outbuf == out);
+  TEST_VERIFY (outleft == sizeof (out));
+
+  /* Second call shall emit spurious NUL character in unpatched glibc.  */
+  TEST_VERIFY (iconv (cd, NULL, NULL, &outbuf, &outleft) != (size_t) -1);
+
+  /* No characters are expected to be produced.  */
+  TEST_VERIFY (outbuf == out);
+  TEST_VERIFY (outleft == sizeof (out));
+
+  TEST_VERIFY_EXIT (iconv_close (cd) != -1);
+
+  return 0;
+}
+
+#include <support/test-driver.c>
diff --git a/iconvdata/iso-2022-jp-3.c b/iconvdata/iso-2022-jp-3.c
index 70b28ace7f..d724126545 100644
--- a/iconvdata/iso-2022-jp-3.c
+++ b/iconvdata/iso-2022-jp-3.c
@@ -1,5 +1,6 @@
 /* Conversion module for ISO-2022-JP-3.
    Copyright (C) 1998-2021 Free Software Foundation, Inc.
+   Copyright (C) The GNU Toolchain Authors.
    This file is part of the GNU C Library.
 
    The GNU C Library is free software; you can redistribute it and/or
@@ -79,20 +80,31 @@ enum
    the output state to the initial state.  This has to be done during the
    flushing.  */
 #define EMIT_SHIFT_TO_INIT \
-  if (data->__statep->__count != ASCII_set)			      \
+  if ((data->__statep->__count & ~7) != ASCII_set)			      \
     {									      \
       if (FROM_DIRECTION)						      \
 	{								      \
-	  if (__glibc_likely (outbuf + 4 <= outend))			      \
+	  uint32_t ch = data->__statep->__count >> 6;			      \
+									      \
+	  if (__glibc_unlikely (ch != 0))				      \
 	    {								      \
-	      /* Write out the last character.  */			      \
-	      *((uint32_t *) outbuf) = data->__statep->__count >> 6;	      \
-	      outbuf += sizeof (uint32_t);				      \
-	      data->__statep->__count = ASCII_set;			\
+	      if (__glibc_likely (outbuf + 4 <= outend))		      \
+		{							      \
+		  /* Write out the last character.  */			      \
+		  put32u (outbuf, ch);					      \
+		  outbuf += 4;						      \
+		  data->__statep->__count &= 7;				      \
+		  data->__statep->__count |= ASCII_set;			      \
+		}							      \
+	      else							      \
+		/* We don't have enough room in the output buffer.  */	      \
+		status = __GCONV_FULL_OUTPUT;				      \
 	    }								      \
 	  else								      \
-	    /* We don't have enough room in the output buffer.  */	      \
-	    status = __GCONV_FULL_OUTPUT;				      \
+	    {								      \
+	      data->__statep->__count &= 7;				      \
+	      data->__statep->__count |= ASCII_set;			      \
+	    }								      \
 	}								      \
       else								      \
 	{								      \
-- 
2.17.1


^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH] gconv: Do not emit spurious NUL character in ISO-2022-JP-3 (bug 28524)
  2021-11-04 15:03   ` Никита Попов
@ 2021-11-04 19:34     ` Florian Weimer
  0 siblings, 0 replies; 6+ messages in thread
From: Florian Weimer @ 2021-11-04 19:34 UTC (permalink / raw)
  To: Никита
	Попов
  Cc: libc-alpha

* Никита Попов:

> From d8321b3b4399a6e1999bd0ebd1466f6235dc5630 Mon Sep 17 00:00:00 2001
> From: Nikita Popov <npv1310@gmail.com>
> Date: Tue, 2 Nov 2021 13:21:42 +0500
> Subject: [PATCH] gconv: Do not emit spurious NUL character in ISO-2022-JP-3
>  (bug 28524)
>
> Bugfix 27256 has introduced another issue:
> In conversion from ISO-2022-JP-3 encoding, it is possible
> to force iconv to emit extra NUL character on internal state reset.
> To do this, it is sufficient to feed iconv with escape sequence
> which switches active character set.
> The simplified check 'data->__statep->__count != ASCII_set'
> introduced by the aforementioned bugfix picks that case and
> behaves as if '\0' character has been queued thus emitting it.
>
> To eliminate this issue, these steps are taken:
> * Restore original condition
> '(data->__statep->__count & ~7) != ASCII_set'.
> It is necessary since bits 0-2 may contain
> number of buffered input characters.
> * Check that queued character is not NUL.
> Similar step is taken for main conversion loop.
>
> Bundled test case follows following logic:
> * Try to convert ISO-2022-JP-3 escape sequence
> switching active character set
> * Reset internal state by providing NULL as input buffer
> * Ensure that nothing has been converted.
>
> Signed-off-by: Nikita Popov <npv1310@gmail.com>

Thanks, applied.

Florian


^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2021-11-04 19:34 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-11-02  9:06 [PATCH] gconv: Do not emit spurious NUL character in ISO-2022-JP-3 (bug 28524) Никита Попов
2021-11-03  4:20 ` Никита Попов
2021-11-04 14:10 ` Florian Weimer
2021-11-04 14:32 ` Florian Weimer
2021-11-04 15:03   ` Никита Попов
2021-11-04 19:34     ` Florian Weimer

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).