From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by sourceware.org (Postfix) with ESMTPS id 38947385D0F2 for ; Fri, 28 Oct 2022 12:54:53 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 38947385D0F2 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1666961692; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=SyDqas2cw5cYvtU/3rEewD1f/DQ9fqY9s3QjHBBPDHs=; b=Rih48U3OBNYBl0oD3pm+FhdWd01LAjx6lq7Y1yrF2O5xe5/0ps8zX8sw+b0yv0fNw1TcSq A/7QDfGcSfe6cfTibFu3RMxMnUMNfrjxHx9TACM7lZAW/RfO+9SgGrbGK3gjpr54I3G5Qc 7TW4qRFBpUCo5UKIuTwp2ZaHiATCzlo= Received: from mail-qv1-f72.google.com (mail-qv1-f72.google.com [209.85.219.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-455-kroZALRlPbiWStwF648zCw-1; Fri, 28 Oct 2022 08:54:51 -0400 X-MC-Unique: kroZALRlPbiWStwF648zCw-1 Received: by mail-qv1-f72.google.com with SMTP id em2-20020ad44f82000000b004af5338777cso2863723qvb.4 for ; Fri, 28 Oct 2022 05:54:51 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=mime-version:user-agent:content-transfer-encoding:references :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=SyDqas2cw5cYvtU/3rEewD1f/DQ9fqY9s3QjHBBPDHs=; b=qByKWuoJRxkXuloZlmNU9Hi8z9UBjqkMtjZOAVaRKQl2t+2YVvk9kqUOmK7bgV8yOu NyepViNr/XGisXMGJ56WPLjHT7J34wG2a9tOOiK/pGKCpZnV/jOVeg6PL/vOzv0RHw/B yugtW1oHEPgGuTJkMilJgo3erwuOP1s4w89rpf7R0r09EszJdIpGT9FPr5R/tR4q9vod GuN/0eNWW9dacJYL0Li4vQwarb+8EIYcX7FqZHHIiA/Y/1bCVDh2yemPnj7P5NWq5Ma6 NPXYrcy44z+ibvqBvsDnykvdiGTDd0v7XwwipG/BC1tXsx79w52sFHDVIiycjN8jjGr4 7Yyg== X-Gm-Message-State: ACrzQf2CkbPk3x/2dAqjSO2G4f3sLQG3EVA/nuINA6W3MgfxrOw9R88w M1atSQXmji63L7OD+qEdTI3jqPH7w7D2mIm00HB/SqCY9EPNPYTGRAeBo9eaFPTt1MX6T2YOCzX NB02JF9Q= X-Received: by 2002:a05:622a:1a85:b0:39b:d3cc:4714 with SMTP id s5-20020a05622a1a8500b0039bd3cc4714mr45736849qtc.310.1666961691284; Fri, 28 Oct 2022 05:54:51 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7KXZOsNIzL59KyO9MN/+cKRFgqjfwXgwO32jA3ANcSYVG8jrpnZJdJYJPbl1WlNqy39w1QpA== X-Received: by 2002:a05:622a:1a85:b0:39b:d3cc:4714 with SMTP id s5-20020a05622a1a8500b0039bd3cc4714mr45736831qtc.310.1666961691053; Fri, 28 Oct 2022 05:54:51 -0700 (PDT) Received: from t14s.localdomain (c-73-69-212-193.hsd1.ma.comcast.net. [73.69.212.193]) by smtp.gmail.com with ESMTPSA id q127-20020a378e85000000b006ee8874f5fasm2842588qkd.53.2022.10.28.05.54.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 28 Oct 2022 05:54:50 -0700 (PDT) Message-ID: <7628906a36b2bc6c580212b142d78939433dda3f.camel@redhat.com> Subject: Re: [PATCH v2 1/3] libcpp: reject codepoints above 0x10FFFF From: David Malcolm To: Ben Boeckel , gcc-patches@gcc.gnu.org Cc: jason@redhat.com, nathan@acm.org, fortran@gcc.gnu.org, gcc@gcc.gnu.org, brad.king@kitware.com, mliska@suse.cz, anlauf@gmx.de Date: Fri, 28 Oct 2022 08:54:49 -0400 In-Reply-To: <20221027231645.67623-2-ben.boeckel@kitware.com> References: <20221027231645.67623-1-ben.boeckel@kitware.com> <20221027231645.67623-2-ben.boeckel@kitware.com> User-Agent: Evolution 3.44.4 (3.44.4-1.fc36) MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spam-Status: No, score=-10.5 required=5.0 tests=BAYES_00,BODY_8BITS,DKIMWL_WL_HIGH,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,GIT_PATCH_0,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE,TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: On Thu, 2022-10-27 at 19:16 -0400, Ben Boeckel wrote: > Unicode does not support such values because they are unrepresentable > in > UTF-16. Wikipedia pointed me to RFC 3629, which was when UTF-8 introduced this restriction, whereas libcpp was implementing the higher upper limit from the earlier, superceded RFC 2279. The patch looks good to me, assuming it bootstraps and passes usual regression testing, but... >=20 > Signed-off-by: Ben Boeckel > --- > =C2=A0libcpp/ChangeLog=C2=A0 | 6 ++++++ > =C2=A0libcpp/charset.cc | 4 ++-- > =C2=A02 files changed, 8 insertions(+), 2 deletions(-) >=20 > diff --git a/libcpp/ChangeLog b/libcpp/ChangeLog > index 18d5bcceaf0..4d707277531 100644 > --- a/libcpp/ChangeLog > +++ b/libcpp/ChangeLog > @@ -1,3 +1,9 @@ > +2022-10-27=C2=A0 Ben Boeckel=C2=A0 > + > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0* include/charset.cc: Reject e= ncodings of codepoints above > 0x10FFFF. > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0UTF-16 does not support such c= odepoints and therefore all > Unicode > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0rejects such values. > + > =C2=A02022-10-19=C2=A0 Lewis Hyatt=C2=A0 ...AIUI we now put ChangeLog entries in the blurb part of the patch, so that server-side git scripts add them to the actual ChangeLog file. Does the patch pass: ./contrib/gcc-changelog/git_check_commit.py ? Thanks Dave > =C2=A0 > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0* include/cpplib.h (struc= t cpp_string): Use new > "string_length" GTY. > diff --git a/libcpp/charset.cc b/libcpp/charset.cc > index 12a398e7527..e9da6674b5f 100644 > --- a/libcpp/charset.cc > +++ b/libcpp/charset.cc > @@ -216,7 +216,7 @@ one_utf8_to_cppchar (const uchar **inbufp, size_t > *inbytesleftp, > =C2=A0=C2=A0 if (c <=3D 0x3FFFFFF && nbytes > 5) return EILSEQ; > =C2=A0 > =C2=A0=C2=A0 /* Make sure the character is valid.=C2=A0 */ > -=C2=A0 if (c > 0x7FFFFFFF || (c >=3D 0xD800 && c <=3D 0xDFFF)) return EI= LSEQ; > +=C2=A0 if (c > 0x10FFFF || (c >=3D 0xD800 && c <=3D 0xDFFF)) return EILS= EQ; > =C2=A0 > =C2=A0=C2=A0 *cp =3D c; > =C2=A0=C2=A0 *inbufp =3D inbuf; > @@ -320,7 +320,7 @@ one_utf32_to_utf8 (iconv_t bigend, const uchar > **inbufp, size_t *inbytesleftp, > =C2=A0=C2=A0 s +=3D inbuf[bigend ? 2 : 1] << 8; > =C2=A0=C2=A0 s +=3D inbuf[bigend ? 3 : 0]; > =C2=A0 > -=C2=A0 if (s >=3D 0x7FFFFFFF || (s >=3D 0xD800 && s <=3D 0xDFFF)) > +=C2=A0 if (s > 0x10FFFF || (s >=3D 0xD800 && s <=3D 0xDFFF)) > =C2=A0=C2=A0=C2=A0=C2=A0 return EILSEQ; > =C2=A0 > =C2=A0=C2=A0 rval =3D one_cppchar_to_utf8 (s, outbufp, outbytesleftp);