From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gnu.wildebeest.org (wildebeest.demon.nl [212.238.236.112]) by sourceware.org (Postfix) with ESMTPS id 6E2283858433 for ; Sat, 7 Aug 2021 15:45:59 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 6E2283858433 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=klomp.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=klomp.org Received: from reform (deer0x03.wildebeest.org [172.31.17.133]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by gnu.wildebeest.org (Postfix) with ESMTPSA id DC44D30291A9; Sat, 7 Aug 2021 17:45:55 +0200 (CEST) Received: by reform (Postfix, from userid 1000) id 686A22E81273; Sat, 7 Aug 2021 17:45:55 +0200 (CEST) From: Mark Wielaard To: gcc-rust@gcc.gnu.org Cc: simplytheother@gmail.com, Mark Wielaard Subject: [PATCH] lex: accept zero codepoints in strings Date: Sat, 7 Aug 2021 17:45:53 +0200 Message-Id: <20210807154553.441960-1-mark@klomp.org> X-Mailer: git-send-email 2.32.0 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-11.1 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_STATUS, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: gcc-rust@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: gcc-rust mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Aug 2021 15:46:01 -0000 Zero characters (codepoints) are acceptable in strings. The current Lexer::parse_string skipped such zero codepoints by accidents. The zero codepoint was also used as error/skip indicator, but that is only true if the third argument of utf8_escape_pair is true (yes, it is called pair, but is a triple). Add a testcase that checks the (sub)strings are separated by zero chars. Since we cannot slice strings yet this uses extern "C" functions, printf and memchr. --- On irc bjorn3_gh pointed out that our lexer ate embedded zero chars from strings. This fixes that issue and adds a testcase. Also on https://code.wildebeest.org/git/user/mjw/gccrs/commit/?h=str-zero gcc/rust/lex/rust-lex.cc | 2 +- .../rust/execute/torture/str-zero.rs | 26 +++++++++++++++++++ 2 files changed, 27 insertions(+), 1 deletion(-) create mode 100644 gcc/testsuite/rust/execute/torture/str-zero.rs diff --git a/gcc/rust/lex/rust-lex.cc b/gcc/rust/lex/rust-lex.cc index 0b8a8eae651..2cfbc4fb1f4 100644 --- a/gcc/rust/lex/rust-lex.cc +++ b/gcc/rust/lex/rust-lex.cc @@ -1827,7 +1827,7 @@ Lexer::parse_string (Location loc) else length += std::get<1> (utf8_escape_pair); - if (current_char32 != Codepoint (0)) + if (current_char32 != Codepoint (0) || !std::get<2> (utf8_escape_pair)) str += current_char32; // required as parsing utf8 escape only changes current_char diff --git a/gcc/testsuite/rust/execute/torture/str-zero.rs b/gcc/testsuite/rust/execute/torture/str-zero.rs new file mode 100644 index 00000000000..e7fba0d1372 --- /dev/null +++ b/gcc/testsuite/rust/execute/torture/str-zero.rs @@ -0,0 +1,26 @@ +/* { dg-output "bar foo baz foobar\n" } */ +extern "C" +{ + fn printf(s: *const i8, ...); + fn memchr(s: *const i8, c: u8, n: usize) -> *const i8; +} + +pub fn main () -> i32 +{ + let f = "%s %s %s %s\n\0"; + let s = "bar\0\ + foo\ + \x00\ + baz\u{0000}\ + foobar\0"; + let cf = f as *const str as *const i8; + let cs = s as *const str as *const i8; + unsafe + { + let cs2 = memchr (cs, b'f', 5); + let cs3 = memchr (cs2, b'b', 5); + let cs4 = memchr (cs3, b'f', 5); + printf (cf, cs, cs2, cs3, cs4); + } + 0 +} -- 2.32.0