From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by sourceware.org (Postfix) with ESMTPS id DE43F3851536 for ; Fri, 28 Oct 2022 12:54:57 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org DE43F3851536 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1666961697; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=SyDqas2cw5cYvtU/3rEewD1f/DQ9fqY9s3QjHBBPDHs=; b=iRtT4tOAFU34Bo3JhkETTqXCZ10WnULlwCkMwfHsQ4uZWwveDd2eJdMJcsczGcXySGWq6q McP+J93o6M0HppnVWyqpX9T+f7vRdi65S3Mn789XBZw8SEVLGhQGWyFEfDwQGpxaY2y0n7 OK7+QU/qcXeoDbBOYi6VELmNCrF3wEU= Received: from mail-qk1-f200.google.com (mail-qk1-f200.google.com [209.85.222.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-612-byecF2uHPZKxNr-RIkJEDQ-1; Fri, 28 Oct 2022 08:54:56 -0400 X-MC-Unique: byecF2uHPZKxNr-RIkJEDQ-1 Received: by mail-qk1-f200.google.com with SMTP id n13-20020a05620a294d00b006cf933c40feso3642313qkp.20 for ; Fri, 28 Oct 2022 05:54:51 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=mime-version:user-agent:content-transfer-encoding:references :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=SyDqas2cw5cYvtU/3rEewD1f/DQ9fqY9s3QjHBBPDHs=; b=2rwoTfyD9Q2DAtKvdQ3u6kcqkYrKT6zMsVNsCWUA07daK5YYHVYs3nJqAU8JeANsQg c6uzgmFfQUbBxHdYFC7y28Kl347oX6bVFjChy3YWHlMCw4mxW1/6ah5n5YwGJjjZQ4DU FNZJvjBZ5Mc0Vt6CTGngbZSxYdkNrGTmuATTEjW4jjsu1Lb8Q1XnhmBT+feN//lGufWq RD5zqHHj/AZpx2FQAojOJqQFYkVh+GRiqmEUALh05B6G700QiH0tQmxX+v2eEIC+Kel/ 1L4+rjRAyZtkErD+QF47KRMDEQszpHyDjYhEKdenH2Vlofu8IgoL4SpHKOZ8DSejl+2v 1oMA== X-Gm-Message-State: ACrzQf0op92Te2GEgQkLe9JhSSvq6x1mIMRnQWh6kndPvXd7Mj25yTMd DmmcI1AE7ZsyTwyp520RO1hUS72YjelqEg6ovYllmZcGLOozjZd7CRBUy925+0HhIf42rRqfh46 EZruzfWZNdeTB X-Received: by 2002:a05:622a:1a85:b0:39b:d3cc:4714 with SMTP id s5-20020a05622a1a8500b0039bd3cc4714mr45736850qtc.310.1666961691286; Fri, 28 Oct 2022 05:54:51 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7KXZOsNIzL59KyO9MN/+cKRFgqjfwXgwO32jA3ANcSYVG8jrpnZJdJYJPbl1WlNqy39w1QpA== X-Received: by 2002:a05:622a:1a85:b0:39b:d3cc:4714 with SMTP id s5-20020a05622a1a8500b0039bd3cc4714mr45736831qtc.310.1666961691053; Fri, 28 Oct 2022 05:54:51 -0700 (PDT) Received: from t14s.localdomain (c-73-69-212-193.hsd1.ma.comcast.net. [73.69.212.193]) by smtp.gmail.com with ESMTPSA id q127-20020a378e85000000b006ee8874f5fasm2842588qkd.53.2022.10.28.05.54.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 28 Oct 2022 05:54:50 -0700 (PDT) Message-ID: <7628906a36b2bc6c580212b142d78939433dda3f.camel@redhat.com> Subject: Re: [PATCH v2 1/3] libcpp: reject codepoints above 0x10FFFF From: David Malcolm To: Ben Boeckel , gcc-patches@gcc.gnu.org Cc: jason@redhat.com, nathan@acm.org, fortran@gcc.gnu.org, gcc@gcc.gnu.org, brad.king@kitware.com, mliska@suse.cz, anlauf@gmx.de Date: Fri, 28 Oct 2022 08:54:49 -0400 In-Reply-To: <20221027231645.67623-2-ben.boeckel@kitware.com> References: <20221027231645.67623-1-ben.boeckel@kitware.com> <20221027231645.67623-2-ben.boeckel@kitware.com> User-Agent: Evolution 3.44.4 (3.44.4-1.fc36) MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spam-Status: No, score=-10.5 required=5.0 tests=BAYES_00,BODY_8BITS,DKIMWL_WL_HIGH,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,GIT_PATCH_0,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE,TXREP autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: On Thu, 2022-10-27 at 19:16 -0400, Ben Boeckel wrote: > Unicode does not support such values because they are unrepresentable > in > UTF-16. Wikipedia pointed me to RFC 3629, which was when UTF-8 introduced this restriction, whereas libcpp was implementing the higher upper limit from the earlier, superceded RFC 2279. The patch looks good to me, assuming it bootstraps and passes usual regression testing, but... >=20 > Signed-off-by: Ben Boeckel > --- > =C2=A0libcpp/ChangeLog=C2=A0 | 6 ++++++ > =C2=A0libcpp/charset.cc | 4 ++-- > =C2=A02 files changed, 8 insertions(+), 2 deletions(-) >=20 > diff --git a/libcpp/ChangeLog b/libcpp/ChangeLog > index 18d5bcceaf0..4d707277531 100644 > --- a/libcpp/ChangeLog > +++ b/libcpp/ChangeLog > @@ -1,3 +1,9 @@ > +2022-10-27=C2=A0 Ben Boeckel=C2=A0 > + > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0* include/charset.cc: Reject e= ncodings of codepoints above > 0x10FFFF. > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0UTF-16 does not support such c= odepoints and therefore all > Unicode > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0rejects such values. > + > =C2=A02022-10-19=C2=A0 Lewis Hyatt=C2=A0 ...AIUI we now put ChangeLog entries in the blurb part of the patch, so that server-side git scripts add them to the actual ChangeLog file. Does the patch pass: ./contrib/gcc-changelog/git_check_commit.py ? Thanks Dave > =C2=A0 > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0* include/cpplib.h (struc= t cpp_string): Use new > "string_length" GTY. > diff --git a/libcpp/charset.cc b/libcpp/charset.cc > index 12a398e7527..e9da6674b5f 100644 > --- a/libcpp/charset.cc > +++ b/libcpp/charset.cc > @@ -216,7 +216,7 @@ one_utf8_to_cppchar (const uchar **inbufp, size_t > *inbytesleftp, > =C2=A0=C2=A0 if (c <=3D 0x3FFFFFF && nbytes > 5) return EILSEQ; > =C2=A0 > =C2=A0=C2=A0 /* Make sure the character is valid.=C2=A0 */ > -=C2=A0 if (c > 0x7FFFFFFF || (c >=3D 0xD800 && c <=3D 0xDFFF)) return EI= LSEQ; > +=C2=A0 if (c > 0x10FFFF || (c >=3D 0xD800 && c <=3D 0xDFFF)) return EILS= EQ; > =C2=A0 > =C2=A0=C2=A0 *cp =3D c; > =C2=A0=C2=A0 *inbufp =3D inbuf; > @@ -320,7 +320,7 @@ one_utf32_to_utf8 (iconv_t bigend, const uchar > **inbufp, size_t *inbytesleftp, > =C2=A0=C2=A0 s +=3D inbuf[bigend ? 2 : 1] << 8; > =C2=A0=C2=A0 s +=3D inbuf[bigend ? 3 : 0]; > =C2=A0 > -=C2=A0 if (s >=3D 0x7FFFFFFF || (s >=3D 0xD800 && s <=3D 0xDFFF)) > +=C2=A0 if (s > 0x10FFFF || (s >=3D 0xD800 && s <=3D 0xDFFF)) > =C2=A0=C2=A0=C2=A0=C2=A0 return EILSEQ; > =C2=A0 > =C2=A0=C2=A0 rval =3D one_cppchar_to_utf8 (s, outbufp, outbytesleftp);