From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by sourceware.org (Postfix) with ESMTPS id 5CA7B3858402 for ; Wed, 17 Nov 2021 22:46:12 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 5CA7B3858402 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-595-Z6ljMdaKMiaJuV6TIDZ8Pw-1; Wed, 17 Nov 2021 17:46:11 -0500 X-MC-Unique: Z6ljMdaKMiaJuV6TIDZ8Pw-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id B9CD810053FB; Wed, 17 Nov 2021 22:46:09 +0000 (UTC) Received: from t14s.localdomain.com (ovpn-113-54.phx2.redhat.com [10.3.113.54]) by smtp.corp.redhat.com (Postfix) with ESMTP id 423B35F4E0; Wed, 17 Nov 2021 22:45:27 +0000 (UTC) From: David Malcolm To: Marek Polacek Cc: Joseph Myers , Jakub Jelinek , Martin Sebor , GCC Patches , David Malcolm Subject: [committed] libcpp: escape non-ASCII source bytes in -Wbidi-chars= [PR103026] Date: Wed, 17 Nov 2021 17:45:14 -0500 Message-Id: <20211117224515.2484625-1-dmalcolm@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-13.4 required=5.0 tests=BAYES_00, DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H2, SPF_HELO_NONE, SPF_NONE, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Nov 2021 22:46:16 -0000 This flags rich_locations associated with -Wbidi-chars= so that non-ASCII bytes will be escaped when printing the source lines (using the diagnostics support I added in r12-4825-gbd5e882cf6e0def3dd1bc106075d59a303fe0d1e). In particular, this ensures that the printed source lines will be pure ASCII, and thus the visual ordering of the characters will be the same as the logical ordering. Before: Wbidi-chars-1.c: In function ‘main’: Wbidi-chars-1.c:6:43: warning: unpaired UTF-8 bidirectional control character detected [-Wbidi-chars=] 6 | /*‮ } ⁦if (isAdmin)⁩ ⁦ begin admins only */ | ^ Wbidi-chars-1.c:9:28: warning: unpaired UTF-8 bidirectional control character detected [-Wbidi-chars=] 9 | /* end admins only ‮ { ⁦*/ | ^ Wbidi-chars-11.c:6:15: warning: UTF-8 vs UCN mismatch when closing a context by "U+202C (POP DIRECTIONAL FORMATTING)" [-Wbidi-chars=] 6 | int LRE_‪_PDF_\u202c; | ^ Wbidi-chars-11.c:8:19: warning: UTF-8 vs UCN mismatch when closing a context by "U+202C (POP DIRECTIONAL FORMATTING)" [-Wbidi-chars=] 8 | int LRE_\u202a_PDF_‬_; | ^ Wbidi-chars-11.c:10:28: warning: UTF-8 vs UCN mismatch when closing a context by "U+202C (POP DIRECTIONAL FORMATTING)" [-Wbidi-chars=] 10 | const char *s1 = "LRE_‪_PDF_\u202c"; | ^ Wbidi-chars-11.c:12:33: warning: UTF-8 vs UCN mismatch when closing a context by "U+202C (POP DIRECTIONAL FORMATTING)" [-Wbidi-chars=] 12 | const char *s2 = "LRE_\u202a_PDF_‬"; | ^ After: Wbidi-chars-1.c: In function ‘main’: Wbidi-chars-1.c:6:43: warning: unpaired UTF-8 bidirectional control character detected [-Wbidi-chars=] 6 | /* } if (isAdmin) begin admins only */ | ^ Wbidi-chars-1.c:9:28: warning: unpaired UTF-8 bidirectional control character detected [-Wbidi-chars=] 9 | /* end admins only { */ | ^ Wbidi-chars-11.c:6:15: warning: UTF-8 vs UCN mismatch when closing a context by "U+202C (POP DIRECTIONAL FORMATTING)" [-Wbidi-chars=] 6 | int LRE__PDF_\u202c; | ^ Wbidi-chars-11.c:8:19: warning: UTF-8 vs UCN mismatch when closing a context by "U+202C (POP DIRECTIONAL FORMATTING)" [-Wbidi-chars=] 8 | int LRE_\u202a_PDF__; | ^ Wbidi-chars-11.c:10:28: warning: UTF-8 vs UCN mismatch when closing a context by "U+202C (POP DIRECTIONAL FORMATTING)" [-Wbidi-chars=] 10 | const char *s1 = "LRE__PDF_\u202c"; | ^ Wbidi-chars-11.c:12:33: warning: UTF-8 vs UCN mismatch when closing a context by "U+202C (POP DIRECTIONAL FORMATTING)" [-Wbidi-chars=] 12 | const char *s2 = "LRE_\u202a_PDF_"; | ^ Successfully bootstrapped & regrtested on x86_64-pc-linux-gnu. Pushed to trunk as r12-5355-g1a7f2c0774129750fdf73e9f1b78f0ce983c9ab3. libcpp/ChangeLog: PR preprocessor/103026 * lex.c (maybe_warn_bidi_on_close): Use a rich_location and call set_escape_on_output (true) on it. (maybe_warn_bidi_on_char): Likewise. Signed-off-by: David Malcolm --- libcpp/lex.c | 29 +++++++++++++++++------------ 1 file changed, 17 insertions(+), 12 deletions(-) diff --git a/libcpp/lex.c b/libcpp/lex.c index 6a4fbce6030..8290bc637cd 100644 --- a/libcpp/lex.c +++ b/libcpp/lex.c @@ -1427,9 +1427,11 @@ maybe_warn_bidi_on_close (cpp_reader *pfile, const uchar *p) const location_t loc = linemap_position_for_column (pfile->line_table, CPP_BUF_COLUMN (pfile->buffer, p)); - cpp_warning_with_line (pfile, CPP_W_BIDIRECTIONAL, loc, 0, - "unpaired UTF-8 bidirectional control character " - "detected"); + rich_location rich_loc (pfile->line_table, loc); + rich_loc.set_escape_on_output (true); + cpp_warning_at (pfile, CPP_W_BIDIRECTIONAL, &rich_loc, + "unpaired UTF-8 bidirectional control character " + "detected"); } /* We're done with this context. */ bidi::on_close (); @@ -1454,6 +1456,9 @@ maybe_warn_bidi_on_char (cpp_reader *pfile, const uchar *p, bidi::kind kind, const location_t loc = linemap_position_for_column (pfile->line_table, CPP_BUF_COLUMN (pfile->buffer, p)); + rich_location rich_loc (pfile->line_table, loc); + rich_loc.set_escape_on_output (true); + /* It seems excessive to warn about a PDI/PDF that is closing an opened context because we've already warned about the opening character. Except warn when we have a UCN x UTF-8 @@ -1462,20 +1467,20 @@ maybe_warn_bidi_on_char (cpp_reader *pfile, const uchar *p, bidi::kind kind, { if (warn_bidi == bidirectional_unpaired && bidi::current_ctx_ucn_p () != ucn_p) - cpp_warning_with_line (pfile, CPP_W_BIDIRECTIONAL, loc, 0, - "UTF-8 vs UCN mismatch when closing " - "a context by \"%s\"", bidi::to_str (kind)); + cpp_warning_at (pfile, CPP_W_BIDIRECTIONAL, &rich_loc, + "UTF-8 vs UCN mismatch when closing " + "a context by \"%s\"", bidi::to_str (kind)); } else if (warn_bidi == bidirectional_any) { if (kind == bidi::kind::PDF || kind == bidi::kind::PDI) - cpp_warning_with_line (pfile, CPP_W_BIDIRECTIONAL, loc, 0, - "\"%s\" is closing an unopened context", - bidi::to_str (kind)); + cpp_warning_at (pfile, CPP_W_BIDIRECTIONAL, &rich_loc, + "\"%s\" is closing an unopened context", + bidi::to_str (kind)); else - cpp_warning_with_line (pfile, CPP_W_BIDIRECTIONAL, loc, 0, - "found problematic Unicode character \"%s\"", - bidi::to_str (kind)); + cpp_warning_at (pfile, CPP_W_BIDIRECTIONAL, &rich_loc, + "found problematic Unicode character \"%s\"", + bidi::to_str (kind)); } } /* We're done with this context. */ -- 2.26.3