From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 32548 invoked by alias); 21 Sep 2010 22:18:17 -0000 Received: (qmail 32340 invoked by uid 48); 21 Sep 2010 22:17:59 -0000 Date: Tue, 21 Sep 2010 22:18:00 -0000 Message-ID: <20100921221759.32339.qmail@sourceware.org> From: "eblake at redhat dot com" To: glibc-bugs-regex@sources.redhat.com In-Reply-To: <20100921152445.12045.eblake@redhat.com> References: <20100921152445.12045.eblake@redhat.com> Reply-To: sourceware-bugzilla@sourceware.org Subject: [Bug regex/12045] regex range semantics outside of POSIX should be documented and consistent X-Bugzilla-Reason: CC Mailing-List: contact glibc-bugs-regex-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Post: List-Help: , Sender: glibc-bugs-regex-owner@sourceware.org X-SW-Source: 2010-09/txt/msg00008.txt.bz2 ------- Additional Comments From eblake at redhat dot com 2010-09-21 22:17 ------- Actually, according to POSIX 2008, there was a requirement in older POSIX that range expressions be treated as CEO (collating element order) for all locales, but this was specifically relaxed in POSIX 2001. If glibc is going to insist on CEO ordering because of a version of POSIX two editions ago, it would be nice to see that documented. Then again, other glibc interfaces no longer comply with the stricter requirements in older POSIX that have since been relaxed (for example, whether getopt() must include an error message with "illegal" in the string). so I see no reason to tie regex to the older standard's CEO ordering either. XRAT A.9.3.5: http://www.opengroup.org/onlinepubs/9699919799/xrat/V4_xbd_chap09.html Historical implementations used native character order to interpret range expressions. The ISO POSIX-2:1993 standard instead required collating element order (CEO): the order that collating elements were specified between the order_start and order_end keywords in the LC_COLLATE category of the current locale. CEO had some advantages in portability over the native character order, but it also had some disadvantages: * CEO could not feasibly be mimicked in user code, leading to inconsistencies between POSIX matchers and matchers in popular user programs like Emacs, ksh, and Perl. * CEO caused range expressions to match accented and capitalized letters contrary to many users' expectations. For example, "[a-e]" typically matched both 'E' and 'á' but neither 'A' nor 'é' . * CEO was not consistent across implementations. In practice, CEO was often less portable than native character order. For example, it was common for the CEOs of two implementation-supplied locales to disagree, even if both locales were named "da_DK" . Because of these problems, some implementations of regular expressions continued to use native character order. Others used the collation sequence, which is more consistent with sorting than either CEO or native order, but which departs further from the traditional POSIX semantics because it generally requires "[a-e]" to match either 'A' or 'E' but not both. As a result of this kind of implementation variation, programmers who wanted to write portable regular expressions could not rely on the ISO POSIX-2:1993 standard guarantees in practice. While revising the standard, lengthy consideration was given to proposals to attack this problem by adding an API for querying the CEO to allow user-mode matchers, but none of these proposals had implementation experience and none achieved consensus. Leaving the standard alone was also considered, but rejected due to the problems described above. The current standard leaves unspecified the behavior of a range expression outside the POSIX locale. This makes it clearer that conforming applications should avoid range expressions outside the POSIX locale, and it allows implementations and compatible user-mode matchers to interpret range expressions using native order, CEO, collation sequence, or other, more advanced techniques. The concerns which led to this change were raised in IEEE PASC interpretation 1003.2 #43 and others, and related to ambiguities in the specification of how multi-character collating elements should be handled in range expressions. These ambiguities had led to multiple interpretations of the specification, in conflicting ways, which led to varying implementations. As noted above, efforts were made to resolve the differences, but no solution has been found that would be specific enough to allow for portable software while not invalidating existing implementations. -- http://sourceware.org/bugzilla/show_bug.cgi?id=12045 ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.