From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 28207 invoked by alias); 5 Nov 2014 16:20:00 -0000 Mailing-List: contact gcc-bugs-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Archive: List-Post: List-Help: Sender: gcc-bugs-owner@gcc.gnu.org Received: (qmail 28140 invoked by uid 55); 5 Nov 2014 16:19:56 -0000 From: "jsm28 at gcc dot gnu.org" To: gcc-bugs@gcc.gnu.org Subject: [Bug preprocessor/9449] UCNs not recognized in identifiers (c++/c99) Date: Wed, 05 Nov 2014 16:20:00 -0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: gcc X-Bugzilla-Component: preprocessor X-Bugzilla-Version: unknown X-Bugzilla-Keywords: rejects-valid X-Bugzilla-Severity: enhancement X-Bugzilla-Who: jsm28 at gcc dot gnu.org X-Bugzilla-Status: NEW X-Bugzilla-Priority: P3 X-Bugzilla-Assigned-To: unassigned at gcc dot gnu.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: http://gcc.gnu.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-SW-Source: 2014-11/txt/msg00282.txt.bz2 https://gcc.gnu.org/bugzilla/show_bug.cgi?id=9449 --- Comment #47 from Joseph S. Myers --- Author: jsm28 Date: Wed Nov 5 16:19:10 2014 New Revision: 217144 URL: https://gcc.gnu.org/viewcvs?rev=217144&root=gcc&view=rev Log: Enable -fextended-identifiers by default. As proposed at , this patch enables -fextended-identifiers by default for all standard versions including this feature (all C++ versions, C99 and above for C, but not C90 / C94 / gnu89 / preprocessing assembler). It adds a couple of tests for areas where I previously noted testsuite coverage for extended identifiers was lacking, removes -fextended-identifiers from existing tests, adds -g to various such tests to verify that extended identifiers don't break debug info generation and removes the test that was only there to verify that the feature was off by default. The current state of the feature may not correspond exactly to any particular checklist from 2004/5 (see bug 9449) of what was wanted before enabling the feature by default, but I don't think it's any worse than plenty of other features supported by default before every corner case is fully functional, and think problems can readily be fixed incrementally. The following aspects of extended identifiers could still do with more work (and should be straightforward): * C -aux-info (output should use UCNs). * ObjC -gen-decls (output should use UCNs; associated diagnostics from the ObjC front end should use extended characters or UCNs as appropriate to the locale, via using %qE or identifier_to_locale). * Use DW_AT_use_UTF8 in DWARF-3 debug info for compilation units built with extended identifiers enabled (or unconditionally). * cpplib diagnostics (outputting characters or UCNs as appropriate depending on the locale, as done for identifiers in non-cpplib diagnostics). * C++ test for UCN linking with C and extern "C". * Check GDB support / file issues for support if needed. * Actual UTF-8 in identifiers (?). (Be careful about not affecting performance for the normal fast path of lexing identifiers, if possible.) The following may be trickier: * cpplib spelling preservation (required to diagnose macro redefinition with different spellings of the same identifier in the definition or argument names; different spellings of the name of the macro itself are OK, however; also required for correct handling of multiple stringizing in C++); correct output for -d (UCNs), DWARF debug info for macros (UCNs), PCH and PCH tests. (Spelling preservation is the issue that needs fixing to remove references to corner cases in the documentation of -std=c99 and -std=c11 and in c99status.html.) The idea would be to add a second pointer to cpp_identifier that stores the original spelling (whether for extended identifiers only, or for all identifiers); this does not enlarge cpp_token because the resulting larger cpp_identifier structure is no bigger than cpp_string. * C++ translation of extended characters (including $@` and various control characters) to UCNs in phase 1 (note diagnostics thus needed, but not for C++11, for control characters in strings / character constants as those UCNs invalid); a likely implementation approach is to do translation when identifiers / strings / character constants are lexed, together with errors for stray $@` / control characters in program as not being valid UCNs in identifiers ($ only if not accepted in identifiers); note that this translation should not take place inside raw string literals. Bootstrapped with no regressions on x86_64-unknown-linux-gnu. libcpp: PR preprocessor/9449 * init.c (lang_defaults): Enable extended identifiers for C++ and C99-based standards. gcc: PR preprocessor/9449 * doc/cpp.texi (Character sets, Tokenization) (Implementation-defined behavior): Don't refer to UCNs in identifiers requiring -fextended-identifiers. * doc/cppopts.texi (-fextended-identifiers): Document as enabled by default for C99 and later and C++. * doc/invoke.texi (-std=c99, -std=c11): Don't refer to extended identifiers needing -fextended-identifiers. gcc/testsuite: PR preprocessor/9449 * lib/target-supports.exp (check_effective_target_ucn_nocache): Don't use -fextended-identifiers. * c-c++-common/cpp/normalize-3.c, c-c++-common/cpp/ucnid-2011-1.c, g++.dg/cpp/ucn-1.C, g++.dg/cpp/ucnid-1.C, g++.dg/other/ucnid-1.C, gcc.dg/cpp/normalize-1.c, gcc.dg/cpp/normalize-2.c, gcc.dg/cpp/normalize-4.c: Don't use -fextended-identifiers. * gcc.dg/cpp/ucnid-1.c: Don't use -fextended-identifiers. Use -g3. * gcc.dg/cpp/ucnid-10.c, gcc.dg/cpp/ucnid-2.c, gcc.dg/cpp/ucnid-3.c, gcc.dg/cpp/ucnid-4.c, gcc.dg/cpp/ucnid-5.c, gcc.dg/cpp/ucnid-7.c, gcc.dg/cpp/ucnid-9.c, gcc.dg/cpp/warn-normalized-1.c, gcc.dg/cpp/warn-normalized-2.c, gcc.dg/cpp/warn-normalized-3.c: Don't use -fextended-identifiers. * gcc.dg/ucnid-1.c, gcc.dg/ucnid-2.c, gcc.dg/ucnid-3.c, gcc.dg/ucnid-4.c, gcc.dg/ucnid-5.c, gcc.dg/ucnid-6.c: Don't use -fextended-identifiers. Use -g. * gcc.dg/ucnid-7.c, gcc.dg/ucnid-8.c: Don't use -fextended-identifiers. * gcc.dg/ucnid-9.c: Don't use -fextended-identifiers. Use -g. * gcc.dg/ucnid-10.c: Don't use -fextended-identifiers. * gcc.dg/ucnid-11.c, gcc.dg/ucnid-12.c: Don't use -fextended-identifiers. Use -g. * gcc.dg/ucnid-13.c: Don't use -fextended-identifiers. * gcc.dg/cpp/ucnid-8.c: Remove test. * gcc.dg/cpp/ucnid-10.c, gcc.dg/ucnid-14.c: New tests. Added: trunk/gcc/testsuite/gcc.dg/cpp/ucnid-10.c trunk/gcc/testsuite/gcc.dg/ucnid-14.c Removed: trunk/gcc/testsuite/gcc.dg/cpp/ucnid-8.c Modified: trunk/gcc/ChangeLog trunk/gcc/doc/cpp.texi trunk/gcc/doc/cppopts.texi trunk/gcc/doc/invoke.texi trunk/gcc/testsuite/ChangeLog trunk/gcc/testsuite/c-c++-common/cpp/normalize-3.c trunk/gcc/testsuite/c-c++-common/cpp/ucnid-2011-1.c trunk/gcc/testsuite/g++.dg/cpp/ucn-1.C trunk/gcc/testsuite/g++.dg/cpp/ucnid-1.C trunk/gcc/testsuite/g++.dg/other/ucnid-1.C trunk/gcc/testsuite/gcc.dg/cpp/normalize-1.c trunk/gcc/testsuite/gcc.dg/cpp/normalize-2.c trunk/gcc/testsuite/gcc.dg/cpp/normalize-4.c trunk/gcc/testsuite/gcc.dg/cpp/ucnid-1.c trunk/gcc/testsuite/gcc.dg/cpp/ucnid-2.c trunk/gcc/testsuite/gcc.dg/cpp/ucnid-3.c trunk/gcc/testsuite/gcc.dg/cpp/ucnid-4.c trunk/gcc/testsuite/gcc.dg/cpp/ucnid-5.c trunk/gcc/testsuite/gcc.dg/cpp/ucnid-7.c trunk/gcc/testsuite/gcc.dg/cpp/ucnid-9.c trunk/gcc/testsuite/gcc.dg/cpp/warn-normalized-1.c trunk/gcc/testsuite/gcc.dg/cpp/warn-normalized-2.c trunk/gcc/testsuite/gcc.dg/cpp/warn-normalized-3.c trunk/gcc/testsuite/gcc.dg/ucnid-1.c trunk/gcc/testsuite/gcc.dg/ucnid-10.c trunk/gcc/testsuite/gcc.dg/ucnid-11.c trunk/gcc/testsuite/gcc.dg/ucnid-12.c trunk/gcc/testsuite/gcc.dg/ucnid-13.c trunk/gcc/testsuite/gcc.dg/ucnid-2.c trunk/gcc/testsuite/gcc.dg/ucnid-3.c trunk/gcc/testsuite/gcc.dg/ucnid-4.c trunk/gcc/testsuite/gcc.dg/ucnid-5.c trunk/gcc/testsuite/gcc.dg/ucnid-6.c trunk/gcc/testsuite/gcc.dg/ucnid-7.c trunk/gcc/testsuite/gcc.dg/ucnid-8.c trunk/gcc/testsuite/gcc.dg/ucnid-9.c trunk/gcc/testsuite/lib/target-supports.exp trunk/libcpp/ChangeLog trunk/libcpp/init.c