[Bug localedata/18408] New: Provide software utility to permit user created custom locales

[Bug localedata/18408] New: Provide software utility to permit user created custom locales

[byrnejb@harte-lyne.ca]

https://sourceware.org/bugzilla/show_bug.cgi?id=18408

            Bug ID: 18408
           Summary: Provide software utility to permit user created custom
                    locales
           Product: glibc
           Version: unspecified
            Status: NEW
          Severity: enhancement
          Priority: P2
         Component: localedata
          Assignee: unassigned at sourceware dot org
          Reporter: byrnejb@harte-lyne.ca
                CC: libc-locales at sourceware dot org
  Target Milestone: ---

A command line utility to permit user creation of and modification to valid
customised LOCALEs together with their proper installation and removal should
be provided with glibc.  The existing localedef utility is undocumented and
minimally useful for the installation of customised locales.  No useful editing
tool for locale definitions is provided at all.


See bugs: #9842 #12731 #16668

Editorial comment:

This whole situation with LOCALEs appears to me rather bizarre.  Surely the
date and time format an individual or group desire to employ on their own
computer systems should be their choice to make and not arbitrarily restrained
by the simple lack of suitable tools to do so.  Why is customising system
environmental information display made so difficult for end users to do for
themselves?

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug localedata/18408] Provide software utility to permit user created custom locales

[myllynen at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=18408

Marko Myllynen <myllynen at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |myllynen at redhat dot com

--- Comment #1 from Marko Myllynen <myllynen at redhat dot com> ---
(In reply to James B. Byrne from comment #0)
> The existing localedef utility is undocumented

It's documented in the Linux man-pages project:

http://man7.org/linux/man-pages/man1/localedef.1.html
http://man7.org/linux/man-pages/man1/locale.1.html

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug localedata/18408] Provide software utility to permit user created custom locales

[schwab@linux-m68k.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=18408

--- Comment #2 from Andreas Schwab <schwab@linux-m68k.org> ---
It's also documented in the POSIX standard.

http://pubs.opengroup.org/onlinepubs/9699919799/utilities/localedef.html

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug localedata/18408] Provide software utility to permit user created custom locales

[byrnejb@harte-lyne.ca]

https://sourceware.org/bugzilla/show_bug.cgi?id=18408

--- Comment #3 from James B. Byrne <byrnejb@harte-lyne.ca> ---
(In reply to Marko Myllynen from comment #1)
> (In reply to James B. Byrne from comment #0)
> > The existing localedef utility is undocumented
> 
> It's documented in the Linux man-pages project:
> 
> http://man7.org/linux/man-pages/man1/localedef.1.html
> http://man7.org/linux/man-pages/man1/locale.1.html

But not on RHEL6 which is what an end user, who only wants to change the date
and time format, is likely to know of or to check.

Run on a CentOS-6.6 system with all updates applied:


localedef --help
Usage: localedef [OPTION...] NAME
  or:  localedef [OPTION...] [--add-to-archive|--delete-from-archive] FILE...
  or:  localedef [OPTION...] --list-archive [FILE]
Compile locale specification

 Input Files:
  -f, --charmap=FILE         Symbolic character names defined in FILE
  -i, --inputfile=FILE       Source definitions are found in FILE
  -u, --repertoire-map=FILE  FILE contains mapping from symbolic names to UCS4
                             values
. . .

man localedef
No manual entry for localedef

What is it about this matter that prompts such utterly pointless responses? 
The issue is that date and time formats are idiosyncratic and should properly
be in the hands of end users to determine for themselves.  It is the user's
computer system and the host OS should be, as a matter of principal, as
malleable to their needs as can be made.  Yet the Glibc community seems instead
totally devoted to making changing those elements as difficult as humanly
possible.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug localedata/18408] Provide software utility to permit user created custom locales

[byrnejb@harte-lyne.ca]

https://sourceware.org/bugzilla/show_bug.cgi?id=18408

--- Comment #4 from James B. Byrne <byrnejb@harte-lyne.ca> ---
# apropos localedef
localedef: nothing appropriate

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug localedata/18408] Provide software utility to permit user created custom locales

[myllynen at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=18408

--- Comment #5 from Marko Myllynen <myllynen at redhat dot com> ---
The upstream localedef(1) manual page was contributed less than a year ago
while RHEL 6 was released in 2010 so it's no wonder it's not part of RHEL 6.
The used date and time formats are described in strftime(3).

I don't think a special locale editing tool is part of the scope for glibc as
the locale definition files can already be edited with standard utilities,
perhaps such a special tool could be something for projects like GNOME or KDE
to consider.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug localedata/18408] Provide software utility to permit user created custom locales

[fweimer at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=18408

Florian Weimer <fweimer at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |fweimer at redhat dot com

--- Comment #6 from Florian Weimer <fweimer at redhat dot com> ---
(In reply to James B. Byrne from comment #3)
> But not on RHEL6 which is what an end user, who only wants to change the
> date and time format, is likely to know of or to check.

I believe on Red Hat Enterprise Linux, glibc updates will wipe out user-defined
locales, so using this functionality might not be a good idea after all.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug localedata/18408] Provide software utility to permit user created custom locales

[byrnejb@harte-lyne.ca]

https://sourceware.org/bugzilla/show_bug.cgi?id=18408

--- Comment #7 from James B. Byrne <byrnejb@harte-lyne.ca> ---
(In reply to Florian Weimer from comment #6)
> (In reply to James B. Byrne from comment #3)
> > But not on RHEL6 which is what an end user, who only wants to change the
> > date and time format, is likely to know of or to check.
> 
> I believe on Red Hat Enterprise Linux, glibc updates will wipe out
> user-defined locales, so using this functionality might not be a good idea
> after all.

The request is for a utility to ease the production of custom locale files not
for modifications to the standard distributed locale files.  Although that
necessarily could be a consequence.  I have created several custom locales on
CentOS-6 and 7 systems and these have never been disturbed by updates to glibc.
 I cannot see how updates to glibc could have that effect as customised locales
are unknown, and therefore invisible, to the glibc update process.

I suppose if the changes to glibc involved a complete restructuring of the
directory locations then that might have the effect of disabling any customised
locales, but I cannot see that destroying them.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug localedata/18408] Provide software utility to permit user created custom locales

[carlos at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=18408

Carlos O'Donell <carlos at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |carlos at redhat dot com

--- Comment #8 from Carlos O'Donell <carlos at redhat dot com> ---
(In reply to James B. Byrne from comment #7)
> (In reply to Florian Weimer from comment #6)
> > (In reply to James B. Byrne from comment #3)
> > > But not on RHEL6 which is what an end user, who only wants to change the
> > > date and time format, is likely to know of or to check.
> > 
> > I believe on Red Hat Enterprise Linux, glibc updates will wipe out
> > user-defined locales, so using this functionality might not be a good idea
> > after all.
> 
> The request is for a utility to ease the production of custom locale files
> not for modifications to the standard distributed locale files.  Although
> that necessarily could be a consequence.  I have created several custom
> locales on CentOS-6 and 7 systems and these have never been disturbed by
> updates to glibc.  I cannot see how updates to glibc could have that effect
> as customised locales are unknown, and therefore invisible, to the glibc
> update process.
> 
> I suppose if the changes to glibc involved a complete restructuring of the
> directory locations then that might have the effect of disabling any
> customised locales, but I cannot see that destroying them.

User locales can be installed into two locations. Directly as files, where they
can be parsed, or installed into the locale archive to loaded directly into
every application for faster usage.

The general problem is that distributions, at least in Fedora and RHEL,
overwrite the locale archive when glibc is upgraded.

Either way, this is a Fedora or RHEL issue and not a generic upstream issue.

At the very least we need:

* localedef documentation.
* examples using localedef to install a locale to the locale archive.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug localedata/18408] Provide software utility to permit user created custom locales

[byrnejb@harte-lyne.ca]

https://sourceware.org/bugzilla/show_bug.cgi?id=18408

--- Comment #10 from James B. Byrne <byrnejb@harte-lyne.ca> ---
(In reply to Carlos O'Donell from comment #8)

> 
> User locales can be installed into two locations. Directly as files, where
> they can be parsed, or installed into the locale archive to loaded directly
> into every application for faster usage.
> 
> The general problem is that distributions, at least in Fedora and RHEL,
> overwrite the locale archive when glibc is upgraded.
> 
> Either way, this is a Fedora or RHEL issue and not a generic upstream issue.
> 
> At the very least we need:
> 
> * localedef documentation.
> * examples using localedef to install a locale to the locale archive.

It there any technical reason why a local custom locale archives location could
not be provided?

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug localedata/18408] Provide software utility to permit user created custom locales

[byrnejb@harte-lyne.ca]

https://sourceware.org/bugzilla/show_bug.cgi?id=18408

--- Comment #9 from James B. Byrne <byrnejb@harte-lyne.ca> ---
(In reply to Marko Myllynen from comment #5)
> The upstream localedef(1) manual page was contributed less than a year ago
> while RHEL 6 was released in 2010 so it's no wonder it's not part of RHEL 6.
> The used date and time formats are described in strftime(3).
> 
> I don't think a special locale editing tool is part of the scope for glibc
> as the locale definition files can already be edited with standard
> utilities, perhaps such a special tool could be something for projects like
> GNOME or KDE to consider.

So all anyone need do to create a custom locale file is to:

1. discover the library builder utility 'localedef'
2. discover and copy an existing file from the locales shipped with the distro
3. figure out what this sort of stuff means:
date_fmt        "<U0025><U0061><U0020><U0025><U0062><U0020><U0025><U0065>/
<U0020><U0025><U0048><U003A><U0025><U004D><U003A><U0025><U0053><U0020>/
<U0025><U005A><U0020><U0025><U0059>"
4. infer that this somehow is related to strftime
5. determine the exact strftime format field desired
6. translate string into utf-8 code points
7. manually edit the locale copy file with vi, nano, emac or equivalent and
insert the the updated utf-8 code points
8. discover through trial and error how the localedef utility must be used to 
build the custom library files from the edited custom locale definition files
9. configure the system to use the new locale files

What could be simpler?  After all, it did not take me much more than four or
five days to figure this all out on my own. I have probably forgotten a number
of other steps that were also necessary and so are not listed here.  And I was
not the first to run into this morass.  I discovered my experience was shared
by the author of Bug #985981 only later. No doubt there are many others who
either give up or lack the confidence or energy to file a bug report.

What I am reading here is that despite being the source of the LOCALE
definitions the maintainers do not think it within scope to provide a utility
to actually create one.  Am I correct?  

And yet some of the maintainers also express the opinion that the locale files
provided do not need to meet the national regulatory requirements of places for
which they do provide locales.  See bug: Bug 12731 (four years old) and Bug
9842 (six years old).  This might get addressed now, see: Bug 16668 ( over a
year old).  

But it is now 16 years since the legal requirement referred to in this reports
went into effect in Canada. It seems to me an immoderate amount of delay in
addressing a rather simple issue.  I would suggest that this is fairly
substantial evidence that something needs to be provided to manage locale
definitions that is far more accessible to the average system administrator
than the existing arcana.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug localedata/18408] Provide software utility to permit user created custom locales

[byrnejb@harte-lyne.ca]

https://sourceware.org/bugzilla/show_bug.cgi?id=18408

--- Comment #13 from James B. Byrne <byrnejb@harte-lyne.ca> ---
(In reply to Carlos O'Donell from comment #12)
> (In reply to James B. Byrne from comment #9)
> > (In reply to Marko Myllynen from comment #5)
> > > The upstream localedef(1) manual page was contributed less than a year ago
> > > while RHEL 6 was released in 2010 so it's no wonder it's not part of RHEL 6.
> > > The used date and time formats are described in strftime(3).
> > > 
> > > I don't think a special locale editing tool is part of the scope for glibc
> > > as the locale definition files can already be edited with standard
> > > utilities, perhaps such a special tool could be something for projects like
> > > GNOME or KDE to consider.
> > 
> > So all anyone need do to create a custom locale file is to:
> > 
> > 1. discover the library builder utility 'localedef'
> > 2. discover and copy an existing file from the locales shipped with the
> > distro
> > 3. figure out what this sort of stuff means:
> > date_fmt	"<U0025><U0061><U0020><U0025><U0062><U0020><U0025><U0065>/
> > <U0020><U0025><U0048><U003A><U0025><U004D><U003A><U0025><U0053><U0020>/
> > <U0025><U005A><U0020><U0025><U0059>"
> > 4. infer that this somehow is related to strftime
> > 5. determine the exact strftime format field desired
> > 6. translate string into utf-8 code points
> > 7. manually edit the locale copy file with vi, nano, emac or equivalent and
> > insert the the updated utf-8 code points
> > 8. discover through trial and error how the localedef utility must be used
> > to  build the custom library files from the edited custom locale definition
> > files
> > 9. configure the system to use the new locale files
> > 
> > What could be simpler?  After all, it did not take me much more than four or
> > five days to figure this all out on my own. I have probably forgotten a
> > number of other steps that were also necessary and so are not listed here. 
> > And I was not the first to run into this morass.  I discovered my experience
> > was shared by the author of Bug #985981 only later. No doubt there are many
> > others who either give up or lack the confidence or energy to file a bug
> > report.
> 
> I hope you're joking right? That's way too many steps. As a senior project
> developer I admit it's way too hard to create locale customizations.
> However, a tool to do so might be better created outside of glibc and as a
> frontend to localedef.
> 

The outline given is no joke. If anything I have understated the difficulty.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug localedata/18408] Provide software utility to permit user created custom locales

[carlos at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=18408

--- Comment #11 from Carlos O'Donell <carlos at redhat dot com> ---
(In reply to James B. Byrne from comment #10)
> It there any technical reason why a local custom locale archives location
> could not be provided?

Like a ~/.locale-archive loaded whenever a user runs their application? There
are no technical limits, just philosophical design issues, and security
considerations.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug localedata/18408] Provide software utility to permit user created custom locales

[carlos at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=18408

--- Comment #12 from Carlos O'Donell <carlos at redhat dot com> ---
(In reply to James B. Byrne from comment #9)
> (In reply to Marko Myllynen from comment #5)
> > The upstream localedef(1) manual page was contributed less than a year ago
> > while RHEL 6 was released in 2010 so it's no wonder it's not part of RHEL 6.
> > The used date and time formats are described in strftime(3).
> > 
> > I don't think a special locale editing tool is part of the scope for glibc
> > as the locale definition files can already be edited with standard
> > utilities, perhaps such a special tool could be something for projects like
> > GNOME or KDE to consider.
> 
> So all anyone need do to create a custom locale file is to:
> 
> 1. discover the library builder utility 'localedef'
> 2. discover and copy an existing file from the locales shipped with the
> distro
> 3. figure out what this sort of stuff means:
> date_fmt	"<U0025><U0061><U0020><U0025><U0062><U0020><U0025><U0065>/
> <U0020><U0025><U0048><U003A><U0025><U004D><U003A><U0025><U0053><U0020>/
> <U0025><U005A><U0020><U0025><U0059>"
> 4. infer that this somehow is related to strftime
> 5. determine the exact strftime format field desired
> 6. translate string into utf-8 code points
> 7. manually edit the locale copy file with vi, nano, emac or equivalent and
> insert the the updated utf-8 code points
> 8. discover through trial and error how the localedef utility must be used
> to  build the custom library files from the edited custom locale definition
> files
> 9. configure the system to use the new locale files
> 
> What could be simpler?  After all, it did not take me much more than four or
> five days to figure this all out on my own. I have probably forgotten a
> number of other steps that were also necessary and so are not listed here. 
> And I was not the first to run into this morass.  I discovered my experience
> was shared by the author of Bug #985981 only later. No doubt there are many
> others who either give up or lack the confidence or energy to file a bug
> report.

I hope you're joking right? That's way too many steps. As a senior project
developer I admit it's way too hard to create locale customizations. However, a
tool to do so might be better created outside of glibc and as a frontend to
localedef.

> What I am reading here is that despite being the source of the LOCALE
> definitions the maintainers do not think it within scope to provide a
> utility to actually create one.  Am I correct?  

No, you could argue for a CLI tool that lives with glibc and shares code to be
used to (a) spit out a template from an existing locale definition (b) let you
edit it (c) install it as a file or into locale-archive (system-wide or user).

> And yet some of the maintainers also express the opinion that the locale
> files provided do not need to meet the national regulatory requirements of
> places for which they do provide locales.  See bug: Bug 12731 (four years
> old) and Bug 9842 (six years old).  This might get addressed now, see: Bug
> 16668 ( over a year old).  

Yes, that is true. It's hard to validate this data and there are few interested
developers working on it. We went through a large spat of cleanups a few years
back where I diligently emailed 20 or 30 embassys and none got back to me. My
inclination is that legal requirements are moot at this point, users want
locales that match their customs and styles, and governments need custom
locales anyeways to meet their often stricter need.

> But it is now 16 years since the legal requirement referred to in this
> reports went into effect in Canada. It seems to me an immoderate amount of
> delay in addressing a rather simple issue.  I would suggest that this is
> fairly substantial evidence that something needs to be provided to manage
> locale definitions that is far more accessible to the average system
> administrator than the existing arcana.

I agree.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug localedata/18408] Provide software utility to permit user created custom locales

[byrnejb@harte-lyne.ca]

https://sourceware.org/bugzilla/show_bug.cgi?id=18408

--- Comment #15 from James B. Byrne <byrnejb@harte-lyne.ca> ---
(In reply to Carlos O'Donell from comment #12) 
> 
> Yes, that is true. It's hard to validate this data and there are few
> interested developers working on it. We went through a large spat of
> cleanups a few years back where I diligently emailed 20 or 30 embassys and
> none got back to me. My inclination is that legal requirements are moot at
> this point, users want locales that match their customs and styles, and
> governments need custom locales anyeways to meet their often stricter need.
>

I am not in the happy position of being able to dictate to the Canada Revenue
Agency how we will format our date time sequences when transmitting data; quite
the reverse in fact.  We solved this problem for ourselves through my efforts
but it seems to me likely that others will encounter it and it seems to me a
needless burden to place on the shoulders of others.

I point out that he Libre/Open Office developers will not provide a
straight-forward way to set a default date and time format other than through
the system locale (to do so requires creation of custom templates for each
document type).  So this is not simply an issue that affects a handful of
rarefied users. Canadian and U.S. banks also now require dates in yyyy-mm-dd
format.  What was acceptable sixteen years ago is long deprecated in practice.

The present arrangement is the absolute antithesis of the current UX buzz.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug localedata/18408] Provide software utility to permit user created custom locales

[byrnejb@harte-lyne.ca]

https://sourceware.org/bugzilla/show_bug.cgi?id=18408

--- Comment #20 from James B. Byrne <byrnejb@harte-lyne.ca> ---
(In reply to Carlos O'Donell from comment #19)
> (In reply to James B. Byrne from comment #18)
> > The L/O OO people have absolutely no intention of ever altering this.  
> 
> It is still a bad decision on their part. Having to exit LO, set the locale,
> and start the process again just to change the format is bad design.
> Alternatively they could provide an interface that let them change the
> locale for the running process (though it's MT-unsafe).
> 

One does not need to do that.  One need only format each date field as one
requires.  For each and every date field on each document.  On each and every
document one creates.  The locale issue is limited to defining the default
format.

It gets better though.  If you send an L/O Oo document to someone; and they
open it on a host with a different locale format; and the date field is not
user customised on that document; then the date format on your document changes
to the viewer's locale value.

And FOSS people still wonder why MS Word has traction in the workplace.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug localedata/18408] Provide software utility to permit user created custom locales

[carlos at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=18408

--- Comment #19 from Carlos O'Donell <carlos at redhat dot com> ---
(In reply to James B. Byrne from comment #18)
> The L/O OO people have absolutely no intention of ever altering this.  

It is still a bad decision on their part. Having to exit LO, set the locale,
and start the process again just to change the format is bad design.
Alternatively they could provide an interface that let them change the locale
for the running process (though it's MT-unsafe).

The best we can do in glibc is make locale creation easier with some kind of
CLI tool, and I'm happy to see patches for that. Firstly I'd like to see
patches to the manual to describe how to get this to work using the existing
tools though. I think that would be a worthwhile project for anyone interested
in this.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug localedata/18408] Provide software utility to permit user created custom locales

[carlos at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=18408

--- Comment #17 from Carlos O'Donell <carlos at redhat dot com> ---
(In reply to James B. Byrne from comment #15)
> (In reply to Carlos O'Donell from comment #12) 
> > 
> > Yes, that is true. It's hard to validate this data and there are few
> > interested developers working on it. We went through a large spat of
> > cleanups a few years back where I diligently emailed 20 or 30 embassys and
> > none got back to me. My inclination is that legal requirements are moot at
> > this point, users want locales that match their customs and styles, and
> > governments need custom locales anyeways to meet their often stricter need.
> >
> 
> I am not in the happy position of being able to dictate to the Canada
> Revenue Agency how we will format our date time sequences when transmitting
> data; quite the reverse in fact.  We solved this problem for ourselves
> through my efforts but it seems to me likely that others will encounter it
> and it seems to me a needless burden to place on the shoulders of others.

Agreed.

> I point out that he Libre/Open Office developers will not provide a
> straight-forward way to set a default date and time format other than
> through the system locale (to do so requires creation of custom templates
> for each document type).  So this is not simply an issue that affects a
> handful of rarefied users. Canadian and U.S. banks also now require dates in
> yyyy-mm-dd format.  What was acceptable sixteen years ago is long deprecated
> in practice.

In which case you argue that all users may need to more easily adjust their
system locales? It seems this is simply a flaw in Libre Office that needs
correcting, changing your locale to print or output documents in a particular
format is bad design. It might be a good workaround until Libre Office is fixed
though.

> The present arrangement is the absolute antithesis of the current UX buzz.

Yes.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug localedata/18408] Provide software utility to permit user created custom locales

[joseph at codesourcery dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=18408

--- Comment #21 from joseph at codesourcery dot com <joseph at codesourcery dot com> ---
On Fri, 15 May 2015, carlos at redhat dot com wrote:

> Yes, that is true. It's hard to validate this data and there are few interested
> developers working on it. We went through a large spat of cleanups a few years

localedata is certainly an area that needs someone to become the expert 
and work on cleaning up the backlog of bug reports - which would include 
researching common usage in lots of countries / languages, and documenting 
that research carefully to justify patches.  After the catch-all "libc" 
it's the area with the greatest number of open bugs.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug localedata/18408] Provide software utility to permit user created custom locales

[byrnejb@harte-lyne.ca]

https://sourceware.org/bugzilla/show_bug.cgi?id=18408

--- Comment #18 from James B. Byrne <byrnejb@harte-lyne.ca> ---
(In reply to Carlos O'Donell from comment #17)
> (In reply to James B. Byrne from comment #15)
> > I point out that he Libre/Open Office developers will not provide a
> > straight-forward way to set a default date and time format other than
> > through the system locale (to do so requires creation of custom templates
> > for each document type).  So this is not simply an issue that affects a
> > handful of rarefied users. Canadian and U.S. banks also now require dates in
> > yyyy-mm-dd format.  What was acceptable sixteen years ago is long deprecated
> > in practice.
> 
> In which case you argue that all users may need to more easily adjust their
> system locales? It seems this is simply a flaw in Libre Office that needs
> correcting, changing your locale to print or output documents in a
> particular format is bad design. It might be a good workaround until Libre
> Office is fixed though.

The L/O OO people have absolutely no intention of ever altering this.  

https://bz.apache.org/ooo/show_bug.cgi?id=30216

and

http://ask.libreoffice.org/en/question/24722/how-to-change-the-default-date-format-in-calc-to-show-4-digit-years/

asked 2013-11-02 11:41:40 +0200 

>>> You can't change the default format. It is hardcoded to the locale of
>>> the system and all of them use a 2-digit year by default. The reason
>>> is that there are simply too many default settings to allow even a
>>> modest percentage of them to be displayed via dialogs. . .

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug localedata/18408] Provide software utility to permit user created custom locales

[byrnejb@harte-lyne.ca]

https://sourceware.org/bugzilla/show_bug.cgi?id=18408

--- Comment #16 from James B. Byrne <byrnejb@harte-lyne.ca> ---
(In reply to Carlos O'Donell from comment #14)
> (In reply to James B. Byrne from comment #13)
> > The outline given is no joke. If anything I have understated the difficulty.
> 
> You wrote "What could be simpler?" That list is not simple. I assume then
> that we are both in agreement that it should be much easier.

Irony.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug localedata/18408] Provide software utility to permit user created custom locales

[carlos at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=18408

--- Comment #14 from Carlos O'Donell <carlos at redhat dot com> ---
(In reply to James B. Byrne from comment #13)
> The outline given is no joke. If anything I have understated the difficulty.

You wrote "What could be simpler?" That list is not simple. I assume then that
we are both in agreement that it should be much easier.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

Re: [Bug localedata/18408] Provide software utility to permit user created custom locales

[Keld Simonsen]

On Fri, May 15, 2015 at 04:11:40PM +0000, byrnejb@harte-lyne.ca wrote:
> https://sourceware.org/bugzilla/show_bug.cgi?id=18408
> 
> --- Comment #9 from James B. Byrne <byrnejb@harte-lyne.ca> ---
> (In reply to Marko Myllynen from comment #5)
> > The upstream localedef(1) manual page was contributed less than a year ago
> > while RHEL 6 was released in 2010 so it's no wonder it's not part of RHEL 6.
> > The used date and time formats are described in strftime(3).
> > 
> > I don't think a special locale editing tool is part of the scope for glibc
> > as the locale definition files can already be edited with standard
> > utilities, perhaps such a special tool could be something for projects like
> > GNOME or KDE to consider.
> 
> So all anyone need do to create a custom locale file is to:
> 
> 1. discover the library builder utility 'localedef'
> 2. discover and copy an existing file from the locales shipped with the distro
> 3. figure out what this sort of stuff means:
> date_fmt        "<U0025><U0061><U0020><U0025><U0062><U0020><U0025><U0065>/
> <U0020><U0025><U0048><U003A><U0025><U004D><U003A><U0025><U0053><U0020>/
> <U0025><U005A><U0020><U0025><U0059>"
> 4. infer that this somehow is related to strftime
> 5. determine the exact strftime format field desired
> 6. translate string into utf-8 code points
> 7. manually edit the locale copy file with vi, nano, emac or equivalent and
> insert the the updated utf-8 code points
> 8. discover through trial and error how the localedef utility must be used to 
> build the custom library files from the edited custom locale definition files
> 9. configure the system to use the new locale files
> 
> What could be simpler?  After all, it did not take me much more than four or
> five days to figure this all out on my own. I have probably forgotten a number
> of other steps that were also necessary and so are not listed here.  And I was
> not the first to run into this morass.  I discovered my experience was shared
> by the author of Bug #985981 only later. No doubt there are many others who
> either give up or lack the confidence or energy to file a bug report.
> 
> What I am reading here is that despite being the source of the LOCALE
> definitions the maintainers do not think it within scope to provide a utility
> to actually create one.  Am I correct?  
> 
> And yet some of the maintainers also express the opinion that the locale files
> provided do not need to meet the national regulatory requirements of places for
> which they do provide locales.  See bug: Bug 12731 (four years old) and Bug
> 9842 (six years old).  This might get addressed now, see: Bug 16668 ( over a
> year old).  
> 
> But it is now 16 years since the legal requirement referred to in this reports
> went into effect in Canada. It seems to me an immoderate amount of delay in
> addressing a rather simple issue.  I would suggest that this is fairly
> substantial evidence that something needs to be provided to manage locale
> definitions that is far more accessible to the average system administrator
> than the existing arcana.

A first start to help users could be to write up a howto on this.
Your desctiption above could be a good start to this end.
I agree that it is cumbersome, but then some things are cumbersome
in the systems area. It is also cumbersome to change the kernel behaviour or
to change things in an application. I actually thnk it is less cumbersome to 
make a custom locale than to change the other mentioned things.


Anyway if it is just to change Canadian time specs, you could copy just
the LC_TIME category from another locale that works as desired.
If you need ISO 8601 formatted dates and times in English, the i18n
locale might do.
in sh: LC_TIME=i18n

I have tried to work with Canadian SCC to have them issue official Canadian 
locales, for about 10 yoears via my friends in SCC, but that has not 
concluded in an official blessed open source set of locales (English/French).
I am still working on that, and have some new tactics. Lets see.

Best regards
Keld

[Bug localedata/18408] Provide software utility to permit user created custom locales

[keld at keldix dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=18408

--- Comment #22 from keld at keldix dot com <keld at keldix dot com> ---
On Fri, May 15, 2015 at 04:11:40PM +0000, byrnejb@harte-lyne.ca wrote:
> https://sourceware.org/bugzilla/show_bug.cgi?id=18408
> 
> --- Comment #9 from James B. Byrne <byrnejb@harte-lyne.ca> ---
> (In reply to Marko Myllynen from comment #5)
> > The upstream localedef(1) manual page was contributed less than a year ago
> > while RHEL 6 was released in 2010 so it's no wonder it's not part of RHEL 6.
> > The used date and time formats are described in strftime(3).
> > 
> > I don't think a special locale editing tool is part of the scope for glibc
> > as the locale definition files can already be edited with standard
> > utilities, perhaps such a special tool could be something for projects like
> > GNOME or KDE to consider.
> 
> So all anyone need do to create a custom locale file is to:
> 
> 1. discover the library builder utility 'localedef'
> 2. discover and copy an existing file from the locales shipped with the distro
> 3. figure out what this sort of stuff means:
> date_fmt        "<U0025><U0061><U0020><U0025><U0062><U0020><U0025><U0065>/
> <U0020><U0025><U0048><U003A><U0025><U004D><U003A><U0025><U0053><U0020>/
> <U0025><U005A><U0020><U0025><U0059>"
> 4. infer that this somehow is related to strftime
> 5. determine the exact strftime format field desired
> 6. translate string into utf-8 code points
> 7. manually edit the locale copy file with vi, nano, emac or equivalent and
> insert the the updated utf-8 code points
> 8. discover through trial and error how the localedef utility must be used to 
> build the custom library files from the edited custom locale definition files
> 9. configure the system to use the new locale files
> 
> What could be simpler?  After all, it did not take me much more than four or
> five days to figure this all out on my own. I have probably forgotten a number
> of other steps that were also necessary and so are not listed here.  And I was
> not the first to run into this morass.  I discovered my experience was shared
> by the author of Bug #985981 only later. No doubt there are many others who
> either give up or lack the confidence or energy to file a bug report.
> 
> What I am reading here is that despite being the source of the LOCALE
> definitions the maintainers do not think it within scope to provide a utility
> to actually create one.  Am I correct?  
> 
> And yet some of the maintainers also express the opinion that the locale files
> provided do not need to meet the national regulatory requirements of places for
> which they do provide locales.  See bug: Bug 12731 (four years old) and Bug
> 9842 (six years old).  This might get addressed now, see: Bug 16668 ( over a
> year old).  
> 
> But it is now 16 years since the legal requirement referred to in this reports
> went into effect in Canada. It seems to me an immoderate amount of delay in
> addressing a rather simple issue.  I would suggest that this is fairly
> substantial evidence that something needs to be provided to manage locale
> definitions that is far more accessible to the average system administrator
> than the existing arcana.

A first start to help users could be to write up a howto on this.
Your desctiption above could be a good start to this end.
I agree that it is cumbersome, but then some things are cumbersome
in the systems area. It is also cumbersome to change the kernel behaviour or
to change things in an application. I actually thnk it is less cumbersome to 
make a custom locale than to change the other mentioned things.


Anyway if it is just to change Canadian time specs, you could copy just
the LC_TIME category from another locale that works as desired.
If you need ISO 8601 formatted dates and times in English, the i18n
locale might do.
in sh: LC_TIME=i18n

I have tried to work with Canadian SCC to have them issue official Canadian 
locales, for about 10 yoears via my friends in SCC, but that has not 
concluded in an official blessed open source set of locales (English/French).
I am still working on that, and have some new tactics. Lets see.

Best regards
Keld

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug localedata/18408] Provide software utility to permit user created custom locales

[myllynen at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=18408

--- Comment #24 from Marko Myllynen <myllynen at redhat dot com> ---
(In reply to joseph@codesourcery.com from comment #21)
> 
> localedata is certainly an area that needs someone to become the expert
> ... which would include researching common usage in lots of countries /
> languages

It's easy to agree that there's been certain lack of ownership in this area (I
think it's pretty obvious given that even trivial locale patches might take
months if not years to get applied) but I'm not sure would such research
oriented approach by the glibc community be the best answer here. Since there's
already a large and active community working on these issues at Unicode CLDR
(including many major vendors of the industry) perhaps we should consider
relying on their expertise and using their data where possible (ideally in an
automated fashion).

If in some cases there would be deviations between glibc and CLDR then we could
either update glibc to match CLDR or if the glibc side looks better then report
an issue to CLDR. CLDR does not (at least yet) cover everything glibc provides
so there would still be some work required by the glibc community but probably
far less than trying do everything without CLDR.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug localedata/18408] Provide software utility to permit user created custom locales

[myllynen at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=18408

--- Comment #23 from Marko Myllynen <myllynen at redhat dot com> ---
> So all anyone need do to create a custom locale file is to:
> 
> What could be simpler?  After all, it did not take me much more than four or
> five days to figure this all out on my own. I have probably forgotten a
> number of other steps that were also necessary and so are not listed here.

I would imagine anyone interested in this area will either do a web search for
something "glibc locale" or just type "man locale." Both of these methods today
lead to the following interlinked documents which should provide all the
information needed so while there's still lots of manual work to do, at least
it should be easier to see what needs to be done.

https://sourceware.org/glibc/wiki/Locales
http://man7.org/linux/man-pages/man1/locale.1.html
http://man7.org/linux/man-pages/man5/locale.5.html
http://man7.org/linux/man-pages/man7/locale.7.html
http://man7.org/linux/man-pages/man1/localedef.1.html

The missing reference to strftime(3) was a fair point and it has now been fixed
in upstream:

http://git.kernel.org/cgit/docs/man-pages/man-pages.git/commit/?id=ea7208a6aae62dbe7af43a21654d2d269ca41957

> At the very least we need:
> 
> * localedef documentation.
> * examples using localedef to install a locale to the locale archive.

These should already be covered by localedef(1) and other pages, if you think
something is missing perhaps you could describe the issue in detail or even
provide a patch / adjust the wiki page?

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug localedata/18408] Provide software utility to permit user created custom locales

[myllynen at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=18408

--- Comment #26 from Marko Myllynen <myllynen at redhat dot com> ---
(In reply to James B. Byrne from comment #25)
> On Mon, June 22, 2015 09:12, myllynen at redhat dot com wrote:
> > 
> > These should already be covered by localedef(1) and other pages,
> > if you think something is missing perhaps you could describe the
> > issue in detail or even provide a patch / adjust the wiki page?
> 
> This is the current man page that ships with my distro, CentOS-6.  As you
> can see there is no reference to glibc or to localedef.

My point was to illustrate that things have gotten better since, I know how
unhelpful RHEL 6 / CentOS 6 locale related man pages, after all it was my main
system when I worked to have proper localedef(1) included in upstream.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug localedata/18408] Provide software utility to permit user created custom locales

[byrnejb@harte-lyne.ca]

https://sourceware.org/bugzilla/show_bug.cgi?id=18408

--- Comment #25 from James B. Byrne <byrnejb@harte-lyne.ca> ---

On Mon, June 22, 2015 09:12, myllynen at redhat dot com wrote:
> 
> These should already be covered by localedef(1) and other pages,
> if you think something is missing perhaps you could describe the
> issue in detail or even provide a patch / adjust the wiki page?
> 

This is the current man page that ships with my distro, CentOS-6.  As you can
see there is no reference to glibc or to localedef.  The point being is that
without prior knowledge of the relationship between LOCALE and these other
elements there is no clue to a researcher as to where to look further.

I searched Google quite extensively to discover the bare existence of
localedef.  But when I searched for a man page for it all I could find was some
private effort at an unofficial version, the reference to I have since
misplaced.  But, that page did point me to this:
http://pubs.opengroup.org/onlinepubs/009695399/basedefs/xbd_chap07.html#tag_07_03

Further, I note that the references you now helpfully provide are all dated as
of December 2014 or later. Which, strangely enough, postdates my original
reports on this matter by a little over two years.  One might be forgiven in
surmising that perhaps the availability of these documents in their current
form is a consequence thereof rather than a failure to initially search for
them.

However, the entire process of establishing on Linux what is fundamentally a
user preference is Byzantine to the point of ludicrous.

LOCALE(1)                                                            LOCALE(1)

NAME
       locale - Get locale-specific information.

SYNOPSIS
       locale [ -a │ -m]

       locale [ -ck ] name...

DESCRIPTION
       The locale program writes information about the current locale
       environment, or all locales, to standard output.

       When invoked without arguments, locale summarizes the current locale
       environment for each locale category defined by the LC_* environment
       variables.

       -a, --all-locales

               Write names of available locales.

       -m, --charmaps

               Write names of available charmaps.

       Output Format:

       -c, --category-name

               Write names of selected categories.

       -k, --keyword-name

               Write names and values of selected keywords.

ENVIRONMENT VARIABLES
       LC_CTYPE

               Character classification and case conversion.

       LC_COLLATE

               Collation order.

       LC_TIME

               Date and time formats.

       LC_NUMERIC

               Non-monetary numeric formats.

       LC_MONETARY

               Monetary formats.

       LC_MESSAGES

               Formats of informative and diagnostic messages and
               interactive responses.

AUTHOR
       locale is written by Ulrich Drepper for the GNU C Library.

       This manpage is written by Joel Klecker <espy@debian.org> for the
       Debian GNU/Linux system.

3rd Berkeley Distribution         March 2001                         LOCALE(1)

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug localedata/18408] Provide software utility to permit user created custom locales

[maiku.fabian at gmail dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=18408

Mike FABIAN <maiku.fabian at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |maiku.fabian at gmail dot com

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug localedata/18408] Provide software utility to permit user created custom locales

[neleai at seznam dot cz]

https://sourceware.org/bugzilla/show_bug.cgi?id=18408

Ondrej Bilka <neleai at seznam dot cz> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |neleai at seznam dot cz

--- Comment #27 from Ondrej Bilka <neleai at seznam dot cz> ---
Here what is needed is different from original request. Tool for modifying
locales wouldn't make creating new locales much easier.

A better interface would be provide overlay for changing what user needs, For
example to change time in en_US locales user would open file ~/.locale/en_US
and add

LC_TIME
d_fmt "%d %m %y"
END LC_TIME

It could run localegen as needed to rebuild ~/.locale/archive . It would also
remove installing new locales as these would be generated first time user
accessed them.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug locale/18408] Provide software utility to permit user created custom locales

[jsm28 at gcc dot gnu.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=18408

Joseph Myers <jsm28 at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
          Component|localedata                  |locale

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug locale/18408] Provide software utility to permit user created custom locales

[carlos at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=18408

Carlos O'Donell <carlos at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
          Component|core                        |locale
            Product|netresolve                  |glibc
           Severity|critical                    |enhancement

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug core/18408] Provide software utility to permit user created custom locales

[gobisha6355 at gmail dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=18408

shanjay <gobisha6355 at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
          Component|locale                      |core
            Product|glibc                       |netresolve
           Severity|enhancement                 |critical

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug locale/18408] Provide software utility to permit user created custom locales

[carlos at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=18408

Carlos O'Donell <carlos at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           See Also|                            |https://bugzilla.redhat.com
                   |                            |/show_bug.cgi?id=1031754

-- 
You are receiving this mail because:
You are on the CC list for the bug.