[Bug libc/16621] New: C.UTF-8 locales should be regarded like C w.r.t. $LANGUAGE precedence

[Bug libc/16621] New: C.UTF-8 locales should be regarded like C w.r.t. $LANGUAGE precedence

[vincent-srcware at vinc17 dot net]

https://sourceware.org/bugzilla/show_bug.cgi?id=16621

            Bug ID: 16621
           Summary: C.UTF-8 locales should be regarded like C w.r.t.
                    $LANGUAGE precedence
           Product: glibc
           Version: 2.18
            Status: NEW
          Severity: normal
          Priority: P2
         Component: libc
          Assignee: unassigned at sourceware dot org
          Reporter: vincent-srcware at vinc17 dot net
                CC: drepper.fsp at gmail dot com

Scripts tend to use LC_ALL=C.UTF-8 instead of LC_ALL=C for UTF-8 support and to
behave in a locale-independent manner. However $LANGUAGE is still taken into
account by glibc:

xvii% LANGUAGE=fr_FR LC_ALL=C.UTF-8 cp
cp: opérande de fichier manquant
Saisissez « cp --help » pour plus d'informations.
xvii% LANGUAGE=fr_FR LC_ALL=C cp
cp: missing file operand
Try 'cp --help' for more information.

Both should have output in English.

Glibc should apply the same rules with C.UTF-8 as with C locales.

Also reported in Debian:
  https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=719590

-- 
You are receiving this mail because:
You are on the CC list for the bug.
>From glibc-bugs-return-21726-listarch-glibc-bugs=sources.redhat.com@sourceware.org Fri Feb 21 12:58:16 2014
Return-Path: <glibc-bugs-return-21726-listarch-glibc-bugs=sources.redhat.com@sourceware.org>
Delivered-To: listarch-glibc-bugs@sources.redhat.com
Received: (qmail 24661 invoked by alias); 21 Feb 2014 12:58:15 -0000
Mailing-List: contact glibc-bugs-help@sourceware.org; run by ezmlm
Precedence: bulk
List-Id: <glibc-bugs.sourceware.org>
List-Subscribe: <mailto:glibc-bugs-subscribe@sourceware.org>
List-Post: <mailto:glibc-bugs@sourceware.org>
List-Help: <mailto:glibc-bugs-help@sourceware.org>, <http://sourceware.org/lists.html#faqs>
Sender: glibc-bugs-owner@sourceware.org
Delivered-To: mailing list glibc-bugs@sourceware.org
Received: (qmail 24627 invoked by uid 48); 21 Feb 2014 12:58:12 -0000
From: "schwab@linux-m68k.org" <sourceware-bugzilla@sourceware.org>
To: glibc-bugs@sourceware.org
Subject: [Bug libc/16621] C.UTF-8 locales should be regarded like C w.r.t. $LANGUAGE precedence
Date: Fri, 21 Feb 2014 12:58:00 -0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: changed
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Product: glibc
X-Bugzilla-Component: libc
X-Bugzilla-Version: 2.18
X-Bugzilla-Keywords:
X-Bugzilla-Severity: normal
X-Bugzilla-Who: schwab@linux-m68k.org
X-Bugzilla-Status: NEW
X-Bugzilla-Priority: P2
X-Bugzilla-Assigned-To: unassigned at sourceware dot org
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Flags:
X-Bugzilla-Changed-Fields:
Message-ID: <bug-16621-131-drEZvQQRkY@http.sourceware.org/bugzilla/>
In-Reply-To: <bug-16621-131@http.sourceware.org/bugzilla/>
References: <bug-16621-131@http.sourceware.org/bugzilla/>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-Bugzilla-URL: http://sourceware.org/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0
X-SW-Source: 2014-02/txt/msg00703.txt.bz2
Content-length: 234

https://sourceware.org/bugzilla/show_bug.cgi?id\x16621

--- Comment #1 from Andreas Schwab <schwab@linux-m68k.org> ---
There is no C.UTF-8 locale in glibc.

--
You are receiving this mail because:
You are on the CC list for the bug.

[Bug libc/16621] C.UTF-8 locales should be regarded like C w.r.t. $LANGUAGE precedence

[vincent-srcware at vinc17 dot net]

https://sourceware.org/bugzilla/show_bug.cgi?id=16621

--- Comment #2 from Vincent Lefèvre <vincent-srcware at vinc17 dot net> ---
(In reply to Andreas Schwab from comment #1)
> There is no C.UTF-8 locale in glibc.

That's strange, because in the Subversion mailing-list, it was regarded as
standard. Subversion works well only in UTF-8 locales, and the suggested
solution was to use C.UTF-8:
http://mail-archives.apache.org/mod_mbox/subversion-users/201307.mbox/%3C51DC54AD.7010601@wandisco.com%3E

-- 
You are receiving this mail because:
You are on the CC list for the bug.
>From glibc-bugs-return-21728-listarch-glibc-bugs=sources.redhat.com@sourceware.org Fri Feb 21 13:45:26 2014
Return-Path: <glibc-bugs-return-21728-listarch-glibc-bugs=sources.redhat.com@sourceware.org>
Delivered-To: listarch-glibc-bugs@sources.redhat.com
Received: (qmail 28239 invoked by alias); 21 Feb 2014 13:45:25 -0000
Mailing-List: contact glibc-bugs-help@sourceware.org; run by ezmlm
Precedence: bulk
List-Id: <glibc-bugs.sourceware.org>
List-Subscribe: <mailto:glibc-bugs-subscribe@sourceware.org>
List-Post: <mailto:glibc-bugs@sourceware.org>
List-Help: <mailto:glibc-bugs-help@sourceware.org>, <http://sourceware.org/lists.html#faqs>
Sender: glibc-bugs-owner@sourceware.org
Delivered-To: mailing list glibc-bugs@sourceware.org
Received: (qmail 28183 invoked by uid 48); 21 Feb 2014 13:45:20 -0000
From: "y.gribov at samsung dot com" <sourceware-bugzilla@sourceware.org>
To: glibc-bugs@sourceware.org
Subject: [Bug math/16600] Glibc floating-point tests fail on VFPv4 platforms
Date: Fri, 21 Feb 2014 13:45:00 -0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: changed
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Product: glibc
X-Bugzilla-Component: math
X-Bugzilla-Version: 2.21
X-Bugzilla-Keywords:
X-Bugzilla-Severity: normal
X-Bugzilla-Who: y.gribov at samsung dot com
X-Bugzilla-Status: WAITING
X-Bugzilla-Priority: P2
X-Bugzilla-Assigned-To: unassigned at sourceware dot org
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Flags:
X-Bugzilla-Changed-Fields:
Message-ID: <bug-16600-131-oQVOGp58UY@http.sourceware.org/bugzilla/>
In-Reply-To: <bug-16600-131@http.sourceware.org/bugzilla/>
References: <bug-16600-131@http.sourceware.org/bugzilla/>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-Bugzilla-URL: http://sourceware.org/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0
X-SW-Source: 2014-02/txt/msg00705.txt.bz2
Content-length: 475

https://sourceware.org/bugzilla/show_bug.cgi?id\x16600

--- Comment #3 from Yury Gribov <y.gribov at samsung dot com> ---
Sorry for delay, I've sent the email
(https://sourceware.org/ml/libc-alpha/2014-02/msg00605.html).

A couple of questions:

1) As for regular regeneration: should I look for announcements on some glibc
mailing lists?
2) As for this bug: are VFPv4 errors that I see expected?

--
You are receiving this mail because:
You are on the CC list for the bug.

[Bug libc/16621] C.UTF-8 locales should be regarded like C w.r.t. $LANGUAGE precedence

[fweimer at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=16621

Florian Weimer <fweimer at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
              Flags|                            |security-

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug libc/16621] C.UTF-8 locales should be regarded like C w.r.t. $LANGUAGE precedence

[ncoghlan at gmail dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=16621

Nick Coghlan <ncoghlan at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |ncoghlan at gmail dot com

--- Comment #3 from Nick Coghlan <ncoghlan at gmail dot com> ---
I have filed bug #17318 requesting the inclusion of a C.UTF-8 locale in
upstream glibc (actually prompted by
https://bugzilla.redhat.com/show_bug.cgi?id=902094, but I found this bug while
looking to see if anyone else had already made the request)

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug locale/16621] C.UTF-8 locales should be regarded like C w.r.t. $LANGUAGE precedence

[jsm28 at gcc dot gnu.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=16621

Joseph Myers <jsm28 at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
          Component|libc                        |locale

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug locale/16621] C.UTF-8 locales should be regarded like C w.r.t. $LANGUAGE precedence

[vapier at gentoo dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=16621

Mike Frysinger <vapier at gentoo dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
                 CC|                            |vapier at gentoo dot org
         Depends on|                            |17318
         Resolution|---                         |INVALID

--- Comment #4 from Mike Frysinger <vapier at gentoo dot org> ---
glibc doesn't provide a C.UTF-8, so any bug report about it makes no sense


Referenced Bugs:

https://sourceware.org/bugzilla/show_bug.cgi?id=17318
[Bug 17318] [RFE] Provide a C.UTF-8 locale by default
-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug locale/16621] C.UTF-8 locales should be regarded like C w.r.t. $LANGUAGE precedence

[vapier at gentoo dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=16621

--- Comment #6 from Mike Frysinger <vapier at gentoo dot org> ---
(In reply to Nick Coghlan from comment #5)

bugs in distros aren't really the domain of glibc upstream.  if you think the
proposal in bug 17318 has limitations or you have concerns, you should post it
there or the mailing list thread on the topic.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug locale/16621] C.UTF-8 locales should be regarded like C w.r.t. $LANGUAGE precedence

[ncoghlan at gmail dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=16621

--- Comment #7 from Nick Coghlan <ncoghlan at gmail dot com> ---
I filed #17318 because Fedora doesn't want to add C.UTF-8 independently of
upstream glibc (at least in part to avoid inconsistencies like the one reported
here).

However, I also interpret the current bug closure as categorically rejecting
the notion of treating C.UTF-8 the same as the C locale when it comes to the
LANGUAGE variable, which doesn't seem like the correct outcome.

If I've misunderstood what "CLOSED INVALID" means and the intent is for bug
#17318 to include the behaviour requested here, then yes, I would consider that
a reasonable way to resolve this issue.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug locale/16621] C.UTF-8 locales should be regarded like C w.r.t. $LANGUAGE precedence

[carlos at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=16621
Bug 16621 depends on bug 17318, which changed state.

Bug 17318 Summary: [RFE] Provide a C.UTF-8 locale by default
https://sourceware.org/bugzilla/show_bug.cgi?id=17318

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|---                         |FIXED

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug locale/16621] C.UTF-8 locales should be regarded like C w.r.t. $LANGUAGE precedence

[nl6720 at gmail dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=16621

nl6720 at gmail dot com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |nl6720 at gmail dot com

--- Comment #8 from nl6720 at gmail dot com ---
glibc 2.35 has C.UTF-8 now, so it would make make sense reopen this.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug locale/16621] C.UTF-8 locales should be regarded like C w.r.t. $LANGUAGE precedence

[vincent-srcware at vinc17 dot net]

https://sourceware.org/bugzilla/show_bug.cgi?id=16621

Vincent Lefèvre <vincent-srcware at vinc17 dot net> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|INVALID                     |---
             Status|RESOLVED                    |REOPENED

--- Comment #9 from Vincent Lefèvre <vincent-srcware at vinc17 dot net> ---
Reopening as glibc 2.35 has C.UTF-8 (comment 8).

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug locale/16621] C.UTF-8 locales should be regarded like C w.r.t. $LANGUAGE precedence

[vincent-srcware at vinc17 dot net]

https://sourceware.org/bugzilla/show_bug.cgi?id=16621

Vincent Lefèvre <vincent-srcware at vinc17 dot net> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Version|2.18                        |2.35

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug locale/16621] C.UTF-8 locales should be regarded like C w.r.t. $LANGUAGE precedence

[fweimer at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=16621

Florian Weimer <fweimer at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |fweimer at redhat dot com

--- Comment #10 from Florian Weimer <fweimer at redhat dot com> ---
C is special-cased here:

  /* Ignore LANGUAGE and its system-dependent analogon if the locale is set
     to "C" because
     1. "C" locale usually uses the ASCII encoding, and most international
        messages use non-ASCII characters. These characters get displayed
        as question marks (if using glibc's iconv()) or as invalid 8-bit
        characters (because other iconv()s refuse to convert most non-ASCII
        characters to ASCII). In any case, the output is ugly.
     2. The precise output of some programs in the "C" locale is specified
        by POSIX and should not depend on environment variables like
        "LANGUAGE" or system-dependent information.  We allow such programs
        to use gettext().  */
  if (strcmp (locale, "C") == 0)
    return locale;

It looks like the locale name is not embedded in the locale data itself, so
identifying C.UTF-8 based on its name might not be so simple here.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug locale/16621] C.UTF-8 locales should be regarded like C w.r.t. $LANGUAGE precedence

[vincent-srcware at vinc17 dot net]

https://sourceware.org/bugzilla/show_bug.cgi?id=16621

--- Comment #11 from Vincent Lefèvre <vincent-srcware at vinc17 dot net> ---
(In reply to Florian Weimer from comment #10)
> C is special-cased here:
[...]
>   if (strcmp (locale, "C") == 0)
>     return locale;
> 
> It looks like the locale name is not embedded in the locale data itself, so
> identifying C.UTF-8 based on its name might not be so simple here.

Do you mean that locale is not the string "C.UTF-8" (while setlocale() returns
the expected "C.UTF-8")?

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug locale/16621] C.UTF-8 locales should be regarded like C w.r.t. $LANGUAGE precedence

[fweimer at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=16621

--- Comment #12 from Florian Weimer <fweimer at redhat dot com> ---
(In reply to Vincent Lefèvre from comment #11)
> (In reply to Florian Weimer from comment #10)
> > C is special-cased here:
> [...]
> >   if (strcmp (locale, "C") == 0)
> >     return locale;
> > 
> > It looks like the locale name is not embedded in the locale data itself, so
> > identifying C.UTF-8 based on its name might not be so simple here.
> 
> Do you mean that locale is not the string "C.UTF-8" (while setlocale()
> returns the expected "C.UTF-8")?

There are aliases such as "C.utf8", which we would have to recognize as well.
Not doing that would make things worse, I think.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug locale/16621] C.UTF-8 locales should be regarded like C w.r.t. $LANGUAGE precedence

[vincent-srcware at vinc17 dot net]

https://sourceware.org/bugzilla/show_bug.cgi?id=16621

--- Comment #13 from Vincent Lefèvre <vincent-srcware at vinc17 dot net> ---
(In reply to Florian Weimer from comment #12)
> There are aliases such as "C.utf8", which we would have to recognize as
> well. Not doing that would make things worse, I think.

OK, though I would say that this is mainly useful for scripts, which could
always use "C.UTF-8" for better portability. BTW, Debian currently doesn't
support aliases for "C.UTF-8" (at least by default).

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug locale/16621] C.UTF-8 locales should be regarded like C w.r.t. $LANGUAGE precedence

[jwilk at jwilk dot net]

https://sourceware.org/bugzilla/show_bug.cgi?id=16621

Jakub Wilk <jwilk at jwilk dot net> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jwilk at jwilk dot net

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug locale/16621] C.UTF-8 locales should be regarded like C w.r.t. $LANGUAGE precedence

[fweimer at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=16621

Florian Weimer <fweimer at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |bruno at clisp dot org

--- Comment #14 from Florian Weimer <fweimer at redhat dot com> ---
*** Bug 29777 has been marked as a duplicate of this bug. ***

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug locale/16621] C.UTF-8 locales should be regarded like C w.r.t. $LANGUAGE precedence

[fweimer at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=16621

Florian Weimer <fweimer at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Assignee|unassigned at sourceware dot org   |fweimer at redhat dot com
             Status|REOPENED                    |ASSIGNED

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug locale/16621] C.UTF-8 locales should be regarded like C w.r.t. $LANGUAGE precedence

[fweimer at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=16621

Florian Weimer <fweimer at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|---                         |2.39

--- Comment #15 from Florian Weimer <fweimer at redhat dot com> ---
Fix pushed for 2.39:

commit 2897b231a6b71ee17d47d3d63f1112b2641a476c
Author: Bruno Haible <bruno@clisp.org>
Date:   Mon Sep 4 15:31:36 2023 +0200

    intl: Treat C.UTF-8 locale like C locale (BZ# 16621)

    The wiki page https://sourceware.org/glibc/wiki/Proposals/C.UTF-8
    says that "Setting LC_ALL=C.UTF-8 will ignore LANGUAGE just like it
    does with LC_ALL=C." This patch implements it.

    * intl/dcigettext.c (guess_category_value): Treat C.<encoding> locale
    like the C locale.

    Reviewed-by: Florian Weimer <fweimer@redhat.com>

I'm going to post my test, too.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug locale/16621] C.UTF-8 locales should be regarded like C w.r.t. $LANGUAGE precedence

[cvs-commit at gcc dot gnu.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=16621

--- Comment #16 from cvs-commit at gcc dot gnu.org <cvs-commit at gcc dot gnu.org> ---
The master branch has been updated by Florian Weimer <fw@sourceware.org>:

https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=c52c2c32db15aba8bbe1a0b4d3235f97d9c1a525

commit c52c2c32db15aba8bbe1a0b4d3235f97d9c1a525
Author: Florian Weimer <fweimer@redhat.com>
Date:   Mon Nov 20 16:03:11 2023 +0100

    intl: Add test case for bug 16621

    Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>

-- 
You are receiving this mail because:
You are on the CC list for the bug.