RE: cygwin started speaking German today

RE: cygwin started speaking German today

[Voelker, Bernhard]

[-- Attachment #1: Type: text/plain, Size: 1261 bytes --]

Charles Wilson wrote:
> On 8/31/2011 10:00 AM, Charles Wilson wrote:
> > Yes: please "restore" your system so that it's breaking, again, and then
> > send (as an attachment) the output of 'cygcheck -svr'.

cygcheck.out.gz attached.

> > Also, what are your existing Windows "Regional and Language Options" set
> > to in the Control Panel?

* tab 1: "Regional options":
  combobox "Standards and formats": "German (Germany)"
  combobox "Location"             : "Germany"
* tab 2: "Languages":
  combobox "Language used in menues and dialogs": English
* tab 3: "Advanced"
  combobox "Language for non-Unicode programs": "German (Germany)"

Maybe the last one is the reason, but I didn't change this for >1 year.

> In addition, what terminal were you using to run your 'mkdir' test
> (regular windows console, mintty, rxvt, etc); what shell (bash, tcsh,
> etc); and what were the settings IN that shell, of the following variables:

TERM=xterm  # run in blackbox WM ;-)
SHELL=/bin/bash
 
> LANG
> LC_ALL
> LC_CTYPE
> LC_MESSAGES

see cygcheck.out:
LANG = 'C.UTF-8'
(no LC_*)
 
> If the terminal you were using was mintty, then what are ITS settings
> (Options->Text: Locale + Character set).

n/a.

Have a nice day,
Berny

[-- Attachment #2: cygcheck.out.gz --]
[-- Type: application/x-gzip, Size: 11282 bytes --]

[-- Attachment #3: Type: text/plain, Size: 218 bytes --]

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

RE: cygwin started speaking German today

[Voelker, Bernhard]

Charles Wilson wrote:

>On 8/30/2011 6:18 AM, Voelker, Bernhard wrote:
>> Starting with today's update, cygwin started speaking German:
>...
>> Ok, the PC is in Germany, but none of my environment
>> variables have a 'de' inside.
>
>Hmmm.
>
>> Any hints?
>
>Try rolling back
>  libiconv, libiconv2, and libcharset
>from 1.14-1 to 1.13.1-2, and then try again.

libiconv and libcharset are not installed,
so I only rolled back libiconv2: no effect.

> If that doesn't work, also roll back
>  gettext, gettext-devel, libintl8, libgettextpo0, libasprintf0
> from 0.18.1.1-1 to 0.17-11 (while ensuring that setup doesn't RE-upgrade
> libiconv!)

only gettext is installed, the others not, but ... BINGO!

$ mkdir -v x0
mkdir: created directory `x0'

So gettext is the culprit. Need more?

Bye,
Berny

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: cygwin started speaking German today

[Charles Wilson]

On 8/31/2011 4:39 AM, Voelker, Bernhard wrote:
> only gettext is installed, the others not, but ... BINGO!
> 
> $ mkdir -v x0
> mkdir: created directory `x0'
> 
> So gettext is the culprit. Need more?

Yes: please "restore" your system so that it's breaking, again, and then
send (as an attachment) the output of 'cygcheck -svr'.

Also, what are your existing Windows "Regional and Language Options" set
to in the Control Panel?

--
Chuck



--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: cygwin started speaking German today

[Charles Wilson]

On 8/31/2011 10:00 AM, Charles Wilson wrote:
> Yes: please "restore" your system so that it's breaking, again, and then
> send (as an attachment) the output of 'cygcheck -svr'.
> 
> Also, what are your existing Windows "Regional and Language Options" set
> to in the Control Panel?

In addition, what terminal were you using to run your 'mkdir' test
(regular windows console, mintty, rxvt, etc); what shell (bash, tcsh,
etc); and what were the settings IN that shell, of the following variables:

LANG
LC_ALL
LC_CTYPE
LC_MESSAGES

If the terminal you were using was mintty, then what are ITS settings
(Options->Text: Locale + Character set).

--
Chuck

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

cygwin started speaking German today

[Voelker, Bernhard]

Starting with today's update, cygwin started speaking German:

$ mkdir -v x0
mkdir: Verzeichnis „x0“ angelegt
$ LANG=C mkdir -v x2
mkdir: created directory `x2'
$ LANG=C.UTF-8 mkdir -v x1
mkdir: Verzeichnis „x1“ angelegt

Default is LANG=C.UTF-8 here.

Ok, the PC is in Germany, but none of my environment
variables have a 'de' inside.

Any hints?

Thanks & have a nice day,
Berny

Re: cygwin started speaking German today

[Charles Wilson]

On 8/30/2011 6:18 AM, Voelker, Bernhard wrote:
> Starting with today's update, cygwin started speaking German:
...
> Ok, the PC is in Germany, but none of my environment
> variables have a 'de' inside.

Hmmm.

> Any hints?

Try rolling back
  libiconv, libiconv2, and libcharset
from 1.14-1 to 1.13.1-2, and then try again.

If that doesn't work, also roll back
  gettext, gettext-devel, libintl8, libgettextpo0, libasprintf0
from 0.18.1.1-1 to 0.17-11 (while ensuring that setup doesn't RE-upgrade
libiconv!)

Try to do this as two separate steps; that will help me know which
package is the actual culprit (if either of them actually is).

--
Chuck


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: cygwin started speaking German today

[Charles Wilson]

[-- Attachment #1: Type: text/plain, Size: 1925 bytes --]

[for the bug-gnu-gettext list, this thread started here:
http://cygwin.com/ml/cygwin/2011-08/msg00506.html
and was posted immediately after the release of gettext-0.18.1.1 for
cygwin (which had not been updated since 0.17).

On 30.08.2011 06:18, Voelker, Bernhard wrote:
> Starting with today's update, cygwin started speaking German:
> 
> $ mkdir -v x0
> mkdir: Verzeichnis „x0“ angelegt
> $ LANG=C mkdir -v x2
> mkdir: created directory `x2'
> $ LANG=C.UTF-8 mkdir -v x1
> mkdir: Verzeichnis „x1“ angelegt
> 
> Default is LANG=C.UTF-8 here.
> 
> Ok, the PC is in Germany, but none of my environment
> variables have a 'de' inside.

I can reproduce this behavior; the workaround for now is to use $LC_ALL
or $LC_CTYPE instead of $LANG.

I think I have an inkling about what is causing this problem -- but I'm
not real clear on how to fix it.  The issue is, (new) libintl implements
its own [libintl_]setlocale function, which eventually calls the cygwin
setlocale.  The old libintl did neither of those things.

Now, under the old mechanism, cygwin's setlocale() was called directly
(by the app), and cygwin::setlocale did its magic with the LC_ALL,
LC_CTYPE, and/or LANG variables.

However, *new* libintl now, I think, calls its own [libintl_]setlocale
replacement internally, and that messes up the effect of the
application's own direct-to-cygwin's setlocale() call.

I think it is because libintl_setlocale() will not allow
cygwin::setlocale() to exercise its default behavior wrt $LANG, since it
*always* calls the underlying OS setlocale with a specific LC_????
subtype, never LC_ALL -- and always with a specific locale string, never
"". (I think)

I've attached a compressed copy of the setlocale implementation, if
anybody (CALL FOR HELP) can suggest a mechanism -- or better, provide a
patch -- to make libintl_setlocale Do The Right Thing on cygwin.

--
Chuck

[-- Attachment #2: setlocale.c.xz --]
[-- Type: application/octet-stream, Size: 7828 bytes --]

[-- Attachment #3: Type: text/plain, Size: 218 bytes --]

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: cygwin started speaking German today

[Bruno Haible]

Bernhard Voelker wrote:
> > Starting with today's update, cygwin started speaking German:
> > 
> > $ mkdir -v x0
> > mkdir: Verzeichnis „x0“ angelegt
> > $ LANG=C mkdir -v x2
> > mkdir: created directory `x2'
> > $ LANG=C.UTF-8 mkdir -v x1
> > mkdir: Verzeichnis „x1“ angelegt
> > 
> > Default is LANG=C.UTF-8 here.
> > 
> > Ok, the PC is in Germany, but none of my environment
> > variables have a 'de' inside.

This is as it should be. See the NEWS entry from the gettext package:

* Runtime behaviour:
  - On MacOS X and Windows systems, <libintl.h> now extends setlocale() and
    newlocale() so that their determination of the default locale considers
    the choice the user has made in the system control panels.

'mkdir' is a GNU coreutils programs, which uses <libintl.h>, so it gets
the benefit of libintl enhancements.

After Cygwin 1.7 added working locales and defined LANG=C.UTF-8 for all users,
libintl could be extended to respect the choices the user has made in the
system control panels.

Charles Wilson writes:
> I can reproduce this behavior; the workaround for now is to use $LC_ALL
> or $LC_CTYPE instead of $LANG.

Which of the three environment variables you use, should not make a difference.
If it does, it's a bug in libintl or in the program. Can you show how to
reproduce this, please?

> I'm not real clear on how to fix it.

There is nothing to "fix". Users who don't want internationalization system-wide
can set their locale in the "Regional Settings" control panel to English.
Users who want to have a German Windows but a non-internationalized Cygwin can
set LANG=C or LC_ALL=C - exactly like POSIX specifies.

> I've attached a compressed copy of the setlocale implementation, if
> anybody (CALL FOR HELP) can suggest a mechanism -- or better, provide a
> patch -- to make libintl_setlocale Do The Right Thing on cygwin.

There is nothing to change in cygwin's setlocale implementation, nor in
libintl_setlocale. Both are POSIX compliant. The one in libintl is more
adequate for internationalized programs (given the fact that Cygwin sets
LANG to C.UTF-8 and not de_DE.UTF-8 as would be appropriate in a German
environment).

Bruno
-- 
In memoriam Elisabeth von Thadden <http://en.wikipedia.org/wiki/Elisabeth_von_Thadden>

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

RE: cygwin started speaking German today

[Voelker, Bernhard]

Bruno Haible wrote:

> > Bernhard Voelker wrote:
> > > Starting with today's update, cygwin started speaking German:
> > > 
> > > $ mkdir -v x0
> > > mkdir: Verzeichnis „x0“ angelegt
> > > $ LANG=C mkdir -v x2
> > > mkdir: created directory `x2'
> > > $ LANG=C.UTF-8 mkdir -v x1
> > > mkdir: Verzeichnis „x1“ angelegt
> > > 
> > > Default is LANG=C.UTF-8 here.
> > > 
> > > Ok, the PC is in Germany, but none of my environment
> > > variables have a 'de' inside.
> 
> This is as it should be. See the NEWS entry from the gettext package:
> 
> * Runtime behaviour:
>   - On MacOS X and Windows systems, <libintl.h> now extends setlocale() and
>     newlocale() so that their determination of the default locale considers
>     the choice the user has made in the system control panels.
> 
> 'mkdir' is a GNU coreutils programs, which uses <libintl.h>, so it gets
> the benefit of libintl enhancements.
> 
> After Cygwin 1.7 added working locales and defined LANG=C.UTF-8 for all users,
> libintl could be extended to respect the choices the user has made in the
> system control panels.
> 
> ...

I had read the NEWS but wasn't aware that the behaviour in Cygwin would
change for my system with this update.
Thanks for this comprehensive explanation!

Have a nice day,
Berny

Re: cygwin started speaking German today

[Eric Blake]

On 09/08/2011 11:46 AM, Bruno Haible wrote:
> Bernhard Voelker wrote:
>>> Starting with today's update, cygwin started speaking German:
>>>
>>> $ mkdir -v x0
>>> mkdir: Verzeichnis „x0“ angelegt
>>> $ LANG=C mkdir -v x2
>>> mkdir: created directory `x2'
>>> $ LANG=C.UTF-8 mkdir -v x1
>>> mkdir: Verzeichnis „x1“ angelegt
>>>
>>> Default is LANG=C.UTF-8 here.
>>>
>>> Ok, the PC is in Germany, but none of my environment
>>> variables have a 'de' inside.
>
> This is as it should be. See the NEWS entry from the gettext package:
>
> * Runtime behaviour:
>    - On MacOS X and Windows systems,<libintl.h>  now extends setlocale() and
>      newlocale() so that their determination of the default locale considers
>      the choice the user has made in the system control panels.

I read this as saying that if _none_ of LANG, LC_MESSAGES, or LC_ALL is 
set, then libintl is smart enough to choose the system default language. 
  But I still think that if LANG is explicitly C.UTF-8, then the user 
_has_ made an explicit language request - namely, the same language as 
for LANG=C (which is more or less English, but not always identical to 
en_US.UTF-8).

> There is nothing to change in cygwin's setlocale implementation, nor in
> libintl_setlocale. Both are POSIX compliant.

How is it POSIX compliant to ignore $LANG by using a different language 
for the C locale?  C.UTF-8 is just the C locale with a different 
charset, not carte-blanche to use the system-default language.

-- 
Eric Blake   eblake@redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: cygwin started speaking German today

[Charles Wilson]

On 9/8/2011 6:46 AM, Bruno Haible wrote:
> There is nothing to "fix". Users who don't want internationalization system-wide
> can set their locale in the "Regional Settings" control panel to English.
> Users who want to have a German Windows but a non-internationalized Cygwin can
> set LANG=C or LC_ALL=C - exactly like POSIX specifies.

But setting LANG=C.UTF-8 (and not setting any of the LC_* vars at all)
should have the same behavior as setting LC_ALL=C.UTF-8.

It does not -- and THAT's the bug.  In the former case, $LANG is ignored
and the system settings are used (in the OP's case, messages are in
German).  In the latter case, LC_ALL is respected, and (in the OP's
case) messages are in English.

Here's how to reproduce:
	cygwin-1.7.9-1
	coreutils-8.10-1
	libintl8-0.18.1.1-1
	libiconv2-1.14-1
	xterm-261-1
	bash-4.1.10-4

In MSWindow's language settings, set:

* tab 1: "Regional options":
  combobox "Standards and formats": "German (Germany)"
  combobox "Location"             : "Germany"
* tab 2: "Languages":
  combobox "Language used in menues and dialogs": English
* tab 3: "Advanced"
  combobox "Language for non-Unicode programs": "German (Germany)"

Reboot.

Then, launch a bash shell (in cmd.exe, not mintty).  Launch an Xserver
-- I actually used the XMing one
http://www.straightrunning.com/XmingNotes/ not the cygwin one -- just to
eliminate the possibility that cygwin's Xserver was exercising cygwin's
[X]setlocale() function.

Then, in the bash shell:

$ LANG=C.UTF-8 DISPLAY=127.0.0.1:0.0 xterm &
$ LC_ALL=C.UTF-8 DISPLAY=127.0.0.1:0.0 xterm &

In the first xterm:

$ mkdir -v x1
mkdir: Verzeichnis „x1“ angelegt

In the seccond xterm:

$ mkdir -v x2
mkdir: created directory `x2'

Now, it may be possible to simplify this test case, but that's what the
OP reported, so that's what I reproduced...

--
Chuck

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: cygwin started speaking German today

[Andy Koppe]

On 8 September 2011 13:04, Charles Wilson wrote:
> On 9/8/2011 6:46 AM, Bruno Haible wrote:
>> There is nothing to "fix". Users who don't want internationalization system-wide
>> can set their locale in the "Regional Settings" control panel to English.
>> Users who want to have a German Windows but a non-internationalized Cygwin can
>> set LANG=C or LC_ALL=C - exactly like POSIX specifies.
>
> But setting LANG=C.UTF-8 (and not setting any of the LC_* vars at all)
> should have the same behavior as setting LC_ALL=C.UTF-8.
>
> It does not -- and THAT's the bug.

I agree.

Further to that, however, Cygwin developers decided that the system
default locale in case neither the relevant LC_* nor LANG are set
should also be "C.UTF-8". That's what setlocale(LC_BLA, "") returns in
that case. It's rather dubious behaviour of libintl to override that.

Cygwin isn't Windows; it's a POSIX environment on top of Windows.
Taking the Regional Settings control panel into account might well
make sense, but it ought to be left to the Cygwin developers to decide
on this and implement it centrally.

Andy

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: cygwin started speaking German today

[Eric Blake]

On 09/08/2011 01:47 PM, Andy Koppe wrote:
> On 8 September 2011 13:04, Charles Wilson wrote:
>> On 9/8/2011 6:46 AM, Bruno Haible wrote:
>>> There is nothing to "fix". Users who don't want internationalization system-wide
>>> can set their locale in the "Regional Settings" control panel to English.
>>> Users who want to have a German Windows but a non-internationalized Cygwin can
>>> set LANG=C or LC_ALL=C - exactly like POSIX specifies.
>>
>> But setting LANG=C.UTF-8 (and not setting any of the LC_* vars at all)
>> should have the same behavior as setting LC_ALL=C.UTF-8.
>>
>> It does not -- and THAT's the bug.
>
> I agree.
>
> Further to that, however, Cygwin developers decided that the system
> default locale in case neither the relevant LC_* nor LANG are set
> should also be "C.UTF-8". That's what setlocale(LC_BLA, "") returns in
> that case. It's rather dubious behaviour of libintl to override that.

I'd argue that if none of LC_* or LANG are set, then 
setlocale(LC_BLA,"") should indeed return the system default, rather 
than being hard-coded to C.  POSIX intentionally left behavior as 
implementation-defined on which locale is returned when none of the 
environment variables set a locale, precisely so that systems could 
implement a sane default according to user preferences expressed in some 
other manner.

>
> Cygwin isn't Windows; it's a POSIX environment on top of Windows.
> Taking the Regional Settings control panel into account might well
> make sense, but it ought to be left to the Cygwin developers to decide
> on this and implement it centrally.

I also agree with this sentiment - if setlocale(LC_BLA, "") is not 
returning sane results (that is, if there is a system default, but 
cygwin is not honoring those defaults), then the bug should be fixed in 
cygwin, not libintl.

-- 
Eric Blake   eblake@redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: cygwin started speaking German today

[Andy Koppe]

[Resend with missing cc to gettext list.]

On 8 September 2011 13:04, Charles Wilson wrote:
> On 9/8/2011 6:46 AM, Bruno Haible wrote:
>> There is nothing to "fix". Users who don't want internationalization system-wide
>> can set their locale in the "Regional Settings" control panel to English.
>> Users who want to have a German Windows but a non-internationalized Cygwin can
>> set LANG=C or LC_ALL=C - exactly like POSIX specifies.
>
> But setting LANG=C.UTF-8 (and not setting any of the LC_* vars at all)
> should have the same behavior as setting LC_ALL=C.UTF-8.
>
> It does not -- and THAT's the bug.

I agree.

Further to that, however, Cygwin developers decided that the system
default locale in case neither the relevant LC_* nor LANG are set
should also be "C.UTF-8". That's what setlocale(LC_BLA, "") returns in
that case. It's rather dubious behaviour of libintl to override that.

Cygwin isn't Windows; it's a POSIX environment on top of Windows.
Taking the Regional Settings control panel into account might well
make sense, but it ought to be left to the Cygwin developers to decide
on this and implement it centrally.

Andy

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: cygwin started speaking German today

[Bruno Haible]

Andy Koppe wrote:
> Cygwin isn't Windows; it's a POSIX environment on top of Windows.
> Taking the Regional Settings control panel into account might well
> make sense, but it ought to be left to the Cygwin developers to decide
> on this and implement it centrally.

And it is the duty of the gettext package to provide an optimal
internationalization experience to end users for packages that use
libintl.

> Cygwin developers decided that the system
> default locale in case neither the relevant LC_* nor LANG are set
> should also be "C.UTF-8".

It's better for the end user if the POSIX default locale depends on
the "Regional settings" panel. I regret that I have not been present
at that discussion and that I had not understood the relation between
setlocale (LC_ALL, "") and the "Regional settings" panel at that time.

Bruno
-- 
In memoriam Elisabeth von Thadden <http://en.wikipedia.org/wiki/Elisabeth_von_Thadden>

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: cygwin started speaking German today

[Charles Wilson]

On 9/8/2011 5:44 PM, Bruno Haible wrote:
> Find below a patch which ought to fix this. But it has upsides and downsides.
> The upside: It treats LC_ALL, LC_$category, LANG consistently, like POSIX
> wants it.

Consistently, yes.  Correctly...no.  You said earlier:

"Users who want to have a German Windows but a non-internationalized
Cygwin can set LANG=C or LC_ALL=C - exactly like POSIX specifies."

With your patch, this is /technically/ true: if I set LANG=C exactly,
without the UTF-8 charset specifier, then yes -- I'll get english cygwin
with german windows.  But, your patch explicitly ignores "C.UTF-8" -- so
if I deliberately select the "C" locale with the "UTF-8" charset, I will
get...the german locale.

That can't be right.

Now, the reason you're ignoring "C.UTF-8" is because you want to
override cygwin's default locale setting -- which is implemented in two
ways: #1, cygwin's internal code for 'setlocale(LC_blah, "")' returns
that value, and #2, some older versions of the base-files startup
scripts (/etc/profile, /etc/skel/.*, and the like) used to set LANG or
LC_* IIRC.  However, they no longer do so.

Now, long term, I think what we will see is that some part of your
suggestions here:
http://cygwin.com/ml/cygwin/2011-09/msg00084.html
will eventually be implemented in cygwin.  When that happens, libintl
will have to change again.

Until then, what?

My suggestion for the "interim" libintl behavior is this:

If /no/ relevant env vars are set
then
	if setlocale(LC_*, "") returns C.UTF-8
	# which we know is the /current/ cygwin default locale
	then
		query Win32 API for "real" default locale
	else
		use what setlocale returns
else
	use the env var value; don't ignore 'C.UTF-8'
	# if I have explicitly set LANG=C.UTF-8 then I must really
	# really want the "C" locale, not en_US or de.

> The downside: It makes libintl_setlocale's behaviour diverge a little more from
> Cygwin's setlocale behaviour.
> Should I commit the patch or not?

I don't think so.  What do you think about the algorithm above, at least
for now, until cygwin's internal behavior is improved -- I tend to agree
with Eric:
http://cygwin.com/ml/cygwin/2011-09/msg00061.html
"I'd argue that if none of LC_* or LANG are set, then
setlocale(LC_BLA,"") should indeed return the system default, rather
than being hard-coded to C. "
and
"I also agree with this sentiment - if setlocale(LC_BLA, "") is not
returning sane results (that is, if there is a system default, but
cygwin is not honoring those defaults), then the bug should be fixed in
cygwin, not libintl."

--
Chuck

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: cygwin started speaking German today

[Andy Koppe]

On 9 September 2011 10:18, Charles Wilson wrote:
> On 9/8/2011 5:44 PM, Bruno Haible wrote:
>> Find below a patch which ought to fix this. But it has upsides and downsides.
>> The upside: It treats LC_ALL, LC_$category, LANG consistently, like POSIX
>> wants it.
>
> Consistently, yes.  Correctly...no.  You said earlier:
>
> "Users who want to have a German Windows but a non-internationalized
> Cygwin can set LANG=C or LC_ALL=C - exactly like POSIX specifies."
>
> With your patch, this is /technically/ true: if I set LANG=C exactly,
> without the UTF-8 charset specifier, then yes -- I'll get english cygwin
> with german windows.  But, your patch explicitly ignores "C.UTF-8" -- so
> if I deliberately select the "C" locale with the "UTF-8" charset, I will
> get...the german locale.
>
> That can't be right.
>
> Now, the reason you're ignoring "C.UTF-8" is because you want to
> override cygwin's default locale setting -- which is implemented in two
> ways: #1, cygwin's internal code for 'setlocale(LC_blah, "")' returns
> that value, and #2, some older versions of the base-files startup
> scripts (/etc/profile, /etc/skel/.*, and the like) used to set LANG or
> LC_* IIRC.  However, they no longer do so

Actually, that is still done, in /etc/profile.d/lang.sh. It's for the
benefit of other programs who think they know better than the system
(emacs, I'm looking at you) by analyzing LC_* and LANG themselves
instead of relying on setlocale().


> Now, long term, I think what we will see is that some part of your
> suggestions here:
> http://cygwin.com/ml/cygwin/2011-09/msg00084.html
> will eventually be implemented in cygwin.  When that happens, libintl
> will have to change again.
>
> Until then, what?
>
> My suggestion for the "interim" libintl behavior is this:
>
> If /no/ relevant env vars are set
> then
>        if setlocale(LC_*, "") returns C.UTF-8
>        # which we know is the /current/ cygwin default locale
>        then
>                query Win32 API for "real" default locale
>        else
>                use what setlocale returns
> else
>        use the env var value; don't ignore 'C.UTF-8'
>        # if I have explicitly set LANG=C.UTF-8 then I must really
>        # really want the "C" locale, not en_US or de.

Please, no. As Corinna said: "Do NOT call Windows functions in Cygwin
libraries, unless the lib is doing something very special which isn't
provided by POSIX functions.  Only call POSIX functions.  Don't mix
the Cygwin and the Windows environment.  Please leave the interfacing
to the underlying OS the sole job of Cygwin."


>> The downside: It makes libintl_setlocale's behaviour diverge a little more from
>> Cygwin's setlocale behaviour.
>> Should I commit the patch or not?
>
> I don't think so.  What do you think about the algorithm above, at least
> for now, until cygwin's internal behavior is improved -- I tend to agree
> with Eric:
> http://cygwin.com/ml/cygwin/2011-09/msg00061.html
> "I'd argue that if none of LC_* or LANG are set, then
> setlocale(LC_BLA,"") should indeed return the system default, rather
> than being hard-coded to C. "
> and
> "I also agree with this sentiment - if setlocale(LC_BLA, "") is not
> returning sane results (that is, if there is a system default, but
> cygwin is not honoring those defaults), then the bug should be fixed in
> cygwin, not libintl."

The 'C.UTF-8' default locale is not a bug, it was a deliberate design decision.

Of course a good case can be made for picking up the Windows language
in case none of LC_* and LANG are set, but there are also obvious
arguments against: translations are usually patchy, i.e. you end up
with a mix of English and translations of varying quality, which a lot
of people hate.

Futhermore, Cygwin is mostly a command line environment, with commands
and options having English names, its userbase is probably even more
geeky than that of your average Linux distribution, and Cygwin's
homepage and setup program are English-only anyway. Hence it doesn't
seem likely that defaulting to English is keeping a lot of users away.

Nevertheless, I'm on the fence on whether the default should be
changed, but I certainly agree with Eric in that any such change
should be implemented in the Cygwin DLL rather than in particular
packages.

Andy

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: cygwin started speaking German today

[Corinna Vinschen]

On Sep  9 13:33, Andy Koppe wrote:
> The 'C.UTF-8' default locale is not a bug, it was a deliberate design decision.

Exactly.  And it has been discussed a lot on the cygwin-apps mailing
list.

And above all, there *is* an official way for the user to align the
Cygwin locale with the Windows locale, see the -s and -u options
of the locale(1) command:

  http://cygwin.com/cygwin-ug-net/using-utils.html#locale

There is *no* good reason that libintl should do that over the head
of Cygwin or worse, over the head of the user.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Project Co-Leader          cygwin AT cygwin DOT com
Red Hat

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: cygwin started speaking German today

[Thorsten Kampe]

* Corinna Vinschen (Fri, 9 Sep 2011 16:59:21 +0200)
> And above all, there *is* an official way for the user to align the
> Cygwin locale with the Windows locale [...]

Misses the point. Users who chose to have a specific language 
environment most likely want to have this choice of language for all 
their applications. All localized applications should honor that and 
there should be no need to explicitly "align" anything. This is the way 
it's handled on Linux and Windows.

Thorsten


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: cygwin started speaking German today

[Eric Blake]

On 09/09/2011 08:59 AM, Corinna Vinschen wrote:
> On Sep  9 13:33, Andy Koppe wrote:
>> The 'C.UTF-8' default locale is not a bug, it was a deliberate design decision.
>
> Exactly.  And it has been discussed a lot on the cygwin-apps mailing
> list.
>
> And above all, there *is* an official way for the user to align the
> Cygwin locale with the Windows locale, see the -s and -u options
> of the locale(1) command:
>
>    http://cygwin.com/cygwin-ug-net/using-utils.html#locale

On 09/09/2011 09:09 AM, Corinna Vinschen wrote:
 >> OK, then the following four facilities are needed in Cygwin.
 >>
 >> 1) We need the name of the locale which is in effect when the user has
 >>     not specified environment variables.
 >
 > In Fedora, for instance, the fallback is what is set as system default
 > in /etc/sysconfig/i18n.
 >
 > In Cygwin the fallback is the system default set in 
/etc/profile.d/lang.sh
 > or /etc/profile.d/lang.csh.
 >
 > Why should libintl use anything else on Cygwin, but not on Linux?
 >

Given this, I think the bug is in cygwin for having base files 
/etc/profile.d/lang.{sh,csh} which hardcode LANG to C.UTF-8 instead of 
using locale -s -u to default LANG to the preferred Windows settings. 
Libintl should NOT be second-guessing an explicit setting of LANG, but 
cygwin should NOT be explicitly setting LANG to C.UTF-8 in its default 
startup scripts without any regards to the Windows settings.  Whether 
setlocale(LC_ALL,"") returns C.UTF-8 or a Windows-appropriate string 
_when LANG is undefined_ is still worth solving, but right now, an 
out-of-the-box cygwin installation _always has LANG defined_ by the 
default startup scripts.  So our first focus should be to get that 
setting of LANG fixed to honor Windows, and to teach libintl that when 
LANG is set we really meant it.

-- 
Eric Blake   eblake@redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: cygwin started speaking German today

[David Sastre]

[-- Attachment #1: Type: text/plain, Size: 2573 bytes --]

On Tue, Sep 13, 2011 at 09:45:37AM -0600, Eric Blake wrote:
> On 09/09/2011 08:59 AM, Corinna Vinschen wrote:
> >On Sep  9 13:33, Andy Koppe wrote:
> >>The 'C.UTF-8' default locale is not a bug, it was a deliberate design decision.
> >
> >Exactly.  And it has been discussed a lot on the cygwin-apps mailing
> >list.
> >
> >And above all, there *is* an official way for the user to align the
> >Cygwin locale with the Windows locale, see the -s and -u options
> >of the locale(1) command:
> >
> >   http://cygwin.com/cygwin-ug-net/using-utils.html#locale
> 
> On 09/09/2011 09:09 AM, Corinna Vinschen wrote:
> >> OK, then the following four facilities are needed in Cygwin.
> >>
> >> 1) We need the name of the locale which is in effect when the user has
> >>     not specified environment variables.
> >
> > In Fedora, for instance, the fallback is what is set as system default
> > in /etc/sysconfig/i18n.
> >
> > In Cygwin the fallback is the system default set in
> /etc/profile.d/lang.sh
> > or /etc/profile.d/lang.csh.
> >
> > Why should libintl use anything else on Cygwin, but not on Linux?
> >
> 
> Given this, I think the bug is in cygwin for having base files
> /etc/profile.d/lang.{sh,csh} which hardcode LANG to C.UTF-8 instead
> of using locale -s -u to default LANG to the preferred Windows
> settings. Libintl should NOT be second-guessing an explicit setting
> of LANG, but cygwin should NOT be explicitly setting LANG to C.UTF-8
> in its default startup scripts without any regards to the Windows
> settings.  Whether setlocale(LC_ALL,"") returns C.UTF-8 or a
> Windows-appropriate string _when LANG is undefined_ is still worth
> solving, but right now, an out-of-the-box cygwin installation
> _always has LANG defined_ by the default startup scripts.  So our
> first focus should be to get that setting of LANG fixed to honor
> Windows, and to teach libintl that when LANG is set we really meant
> it.

WRT the base-files package, would it be acceptable/does it make sense to set:

test -z "${LC_ALL:-${LC_CTYPE:-$LANG}}" && export LANG=${locale -sU}

in /etc/profile.d/lang.sh and

if ( $?LC_ALL == 0 && $?LC_CTYPE == 0 && $?LANG == 0 ) setenv LANG = `locale -sU`

in /etc/profile.d/lang.csh, both as proposed, _and_ a (possibly) commented-out

test -z "${LC_ALL:-${LC_CTYPE:-$LANG}}" && export LANG=${locale -uU}

in the skeletal .bash_profile and .profile (i.e. both system-wide and
user defined settings)?

-- 
Huella de clave primaria: AD8F BDC0 5A2C FD5F A179  60E7 F79B AB04 5299 EC56

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 230 bytes --]

Re: cygwin started speaking German today

[Ken Brown]

On 9/17/2011 4:40 PM, David Sastre wrote:
> On Tue, Sep 13, 2011 at 09:45:37AM -0600, Eric Blake wrote:
>> On 09/09/2011 08:59 AM, Corinna Vinschen wrote:
>>> On Sep  9 13:33, Andy Koppe wrote:
>>>> The 'C.UTF-8' default locale is not a bug, it was a deliberate design decision.
>>>
>>> Exactly.  And it has been discussed a lot on the cygwin-apps mailing
>>> list.
>>>
>>> And above all, there *is* an official way for the user to align the
>>> Cygwin locale with the Windows locale, see the -s and -u options
>>> of the locale(1) command:
>>>
>>>    http://cygwin.com/cygwin-ug-net/using-utils.html#locale
>>
>> On 09/09/2011 09:09 AM, Corinna Vinschen wrote:
>>>> OK, then the following four facilities are needed in Cygwin.
>>>>
>>>> 1) We need the name of the locale which is in effect when the user has
>>>>      not specified environment variables.
>>>
>>> In Fedora, for instance, the fallback is what is set as system default
>>> in /etc/sysconfig/i18n.
>>>
>>> In Cygwin the fallback is the system default set in
>> /etc/profile.d/lang.sh
>>> or /etc/profile.d/lang.csh.
>>>
>>> Why should libintl use anything else on Cygwin, but not on Linux?
>>>
>>
>> Given this, I think the bug is in cygwin for having base files
>> /etc/profile.d/lang.{sh,csh} which hardcode LANG to C.UTF-8 instead
>> of using locale -s -u to default LANG to the preferred Windows
>> settings. Libintl should NOT be second-guessing an explicit setting
>> of LANG, but cygwin should NOT be explicitly setting LANG to C.UTF-8
>> in its default startup scripts without any regards to the Windows
>> settings.  Whether setlocale(LC_ALL,"") returns C.UTF-8 or a
>> Windows-appropriate string _when LANG is undefined_ is still worth
>> solving, but right now, an out-of-the-box cygwin installation
>> _always has LANG defined_ by the default startup scripts.  So our
>> first focus should be to get that setting of LANG fixed to honor
>> Windows, and to teach libintl that when LANG is set we really meant
>> it.
>
> WRT the base-files package, would it be acceptable/does it make sense to set:
>
> test -z "${LC_ALL:-${LC_CTYPE:-$LANG}}"&&  export LANG=${locale -sU}
>
> in /etc/profile.d/lang.sh and
>
> if ( $?LC_ALL == 0&&  $?LC_CTYPE == 0&&  $?LANG == 0 ) setenv LANG = `locale -sU`
>
> in /etc/profile.d/lang.csh, both as proposed, _and_ a (possibly) commented-out
>
> test -z "${LC_ALL:-${LC_CTYPE:-$LANG}}"&&  export LANG=${locale -uU}
>
> in the skeletal .bash_profile and .profile (i.e. both system-wide and
> user defined settings)?

If you want the user-defined setting to take effect, wouldn't you have 
to omit the `test -z ...'?  LANG will already be set when .bash_profile 
is processed.

Ken


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: cygwin started speaking German today

[David Sastre]

[-- Attachment #1: Type: text/plain, Size: 3039 bytes --]

On Sat, Sep 17, 2011 at 05:49:54PM -0400, Ken Brown wrote:
> On 9/17/2011 4:40 PM, David Sastre wrote:
> >On Tue, Sep 13, 2011 at 09:45:37AM -0600, Eric Blake wrote:
> >>On 09/09/2011 08:59 AM, Corinna Vinschen wrote:
> >>>On Sep  9 13:33, Andy Koppe wrote:
> >>>>The 'C.UTF-8' default locale is not a bug, it was a deliberate design decision.
> >>>
> >>>Exactly.  And it has been discussed a lot on the cygwin-apps mailing
> >>>list.
> >>>
> >>>And above all, there *is* an official way for the user to align the
> >>>Cygwin locale with the Windows locale, see the -s and -u options
> >>>of the locale(1) command:
> >>>
> >>>   http://cygwin.com/cygwin-ug-net/using-utils.html#locale
> >>
> >>On 09/09/2011 09:09 AM, Corinna Vinschen wrote:
> >>>>OK, then the following four facilities are needed in Cygwin.
> >>>>
> >>>>1) We need the name of the locale which is in effect when the user has
> >>>>     not specified environment variables.
> >>>
> >>>In Fedora, for instance, the fallback is what is set as system default
> >>>in /etc/sysconfig/i18n.
> >>>
> >>>In Cygwin the fallback is the system default set in
> >>/etc/profile.d/lang.sh
> >>>or /etc/profile.d/lang.csh.
> >>>
> >>>Why should libintl use anything else on Cygwin, but not on Linux?
> >>>
> >>
> >>Given this, I think the bug is in cygwin for having base files
> >>/etc/profile.d/lang.{sh,csh} which hardcode LANG to C.UTF-8 instead
> >>of using locale -s -u to default LANG to the preferred Windows
> >>settings. Libintl should NOT be second-guessing an explicit setting
> >>of LANG, but cygwin should NOT be explicitly setting LANG to C.UTF-8
> >>in its default startup scripts without any regards to the Windows
> >>settings.  Whether setlocale(LC_ALL,"") returns C.UTF-8 or a
> >>Windows-appropriate string _when LANG is undefined_ is still worth
> >>solving, but right now, an out-of-the-box cygwin installation
> >>_always has LANG defined_ by the default startup scripts.  So our
> >>first focus should be to get that setting of LANG fixed to honor
> >>Windows, and to teach libintl that when LANG is set we really meant
> >>it.
> >
> >WRT the base-files package, would it be acceptable/does it make sense to set:
> >
> >test -z "${LC_ALL:-${LC_CTYPE:-$LANG}}"&&  export LANG=${locale -sU}
> >
> >in /etc/profile.d/lang.sh and
> >
> >if ( $?LC_ALL == 0&&  $?LC_CTYPE == 0&&  $?LANG == 0 ) setenv LANG = `locale -sU`
> >
> >in /etc/profile.d/lang.csh, both as proposed, _and_ a (possibly) commented-out
> >
> >test -z "${LC_ALL:-${LC_CTYPE:-$LANG}}"&&  export LANG=${locale -uU}
> >
> >in the skeletal .bash_profile and .profile (i.e. both system-wide and
> >user defined settings)?
> 
> If you want the user-defined setting to take effect, wouldn't you
> have to omit the `test -z ...'?  LANG will already be set when
> .bash_profile is processed.

Indeed, excuse the fast copy-paste. The proposal still stands.

-- 
Huella de clave primaria: AD8F BDC0 5A2C FD5F A179  60E7 F79B AB04 5299 EC56

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 230 bytes --]

Re: cygwin started speaking German today

[Corinna Vinschen]

On Sep 13 09:45, Eric Blake wrote:
> On 09/09/2011 08:59 AM, Corinna Vinschen wrote:
> >On Sep  9 13:33, Andy Koppe wrote:
> >>The 'C.UTF-8' default locale is not a bug, it was a deliberate design decision.
> >
> >Exactly.  And it has been discussed a lot on the cygwin-apps mailing
> >list.
> >
> >And above all, there *is* an official way for the user to align the
> >Cygwin locale with the Windows locale, see the -s and -u options
> >of the locale(1) command:
> >
> >   http://cygwin.com/cygwin-ug-net/using-utils.html#locale
> 
> On 09/09/2011 09:09 AM, Corinna Vinschen wrote:
> >> OK, then the following four facilities are needed in Cygwin.
> >>
> >> 1) We need the name of the locale which is in effect when the user has
> >>     not specified environment variables.
> >
> > In Fedora, for instance, the fallback is what is set as system default
> > in /etc/sysconfig/i18n.
> >
> > In Cygwin the fallback is the system default set in
> /etc/profile.d/lang.sh
> > or /etc/profile.d/lang.csh.
> >
> > Why should libintl use anything else on Cygwin, but not on Linux?
> >
> 
> Given this, I think the bug is in cygwin for having base files
> /etc/profile.d/lang.{sh,csh} which hardcode LANG to C.UTF-8 instead
> of using locale -s -u to default LANG to the preferred Windows
> settings.

Bug?  Didn't we choose C.UTF-8 after a long discussion?  Are the points
raised in this discussion invalid or outdated now?  Why?  I don't object
against using `locale -sU' in lang.sh/lang.csh, but we should not do
this without a discussion of the pros and cons.

> Libintl should NOT be second-guessing an explicit setting
> of LANG,

ACK.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Project Co-Leader          cygwin AT cygwin DOT com
Red Hat

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: cygwin started speaking German today

[Charles Wilson]

On 10/4/2011 8:28 AM, Corinna Vinschen wrote:
> On Sep 13 09:45, Eric Blake wrote:
>> Given this, I think the bug is in cygwin for having base files
>> /etc/profile.d/lang.{sh,csh} which hardcode LANG to C.UTF-8 instead
>> of using locale -s -u to default LANG to the preferred Windows
>> settings.
> 
> Bug?  Didn't we choose C.UTF-8 after a long discussion?  Are the points
> raised in this discussion invalid or outdated now?  Why?  I don't object
> against using `locale -sU' in lang.sh/lang.csh, but we should not do
> this without a discussion of the pros and cons.

IIRC, that discussion occurred before the 'locale' application (was
written|got smarter).  Sure, I think C.UTF-8 should be the "default
default" but the arguments in favor of respecting the users' own Windows
i18n settings make sense.

However, one issue is that windows basically will always have SOME
setting -- even if just "English".  Which would cause locale to report
'en_US' or something.  So you'd never actually SEE the "default default"
of C.UTF-8 take effect.  Also, would locale append '.UTF-8' (or would
lang.{sh,csh} do so?).

Maybe we should start a new thread on this topic, listing the pros and
cons as Corinna suggested.  Who wants to summarize the "con" side of the
argument?

--
Chuck

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: cygwin started speaking German today

[Corinna Vinschen]

On Oct  4 08:45, Charles Wilson wrote:
> On 10/4/2011 8:28 AM, Corinna Vinschen wrote:
> > On Sep 13 09:45, Eric Blake wrote:
> >> Given this, I think the bug is in cygwin for having base files
> >> /etc/profile.d/lang.{sh,csh} which hardcode LANG to C.UTF-8 instead
> >> of using locale -s -u to default LANG to the preferred Windows
> >> settings.
> > 
> > Bug?  Didn't we choose C.UTF-8 after a long discussion?  Are the points
> > raised in this discussion invalid or outdated now?  Why?  I don't object
> > against using `locale -sU' in lang.sh/lang.csh, but we should not do
> > this without a discussion of the pros and cons.
> 
> IIRC, that discussion occurred before the 'locale' application (was
> written|got smarter).  Sure, I think C.UTF-8 should be the "default
> default" but the arguments in favor of respecting the users' own Windows
> i18n settings make sense.

Does it?  Even if I'm running a german OS, I absolutely hate to see
german diagnostic output from gcc, and I absolutely hate certain
programs using non-ASCII chars in output.  (In)famous examples are
Unicode quoting chars rather than ' or ", or using the Unicode hyphen
character rather than -.  But that's just me.

> However, one issue is that windows basically will always have SOME
> setting -- even if just "English".  Which would cause locale to report
> 'en_US' or something.  So you'd never actually SEE the "default default"
> of C.UTF-8 take effect.  Also, would locale append '.UTF-8' (or would
> lang.{sh,csh} do so?).

That's what the -U option is for.  See `locale --help'.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Project Co-Leader          cygwin AT cygwin DOT com
Red Hat

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: cygwin started speaking German today

[Erwin Waterlander]

Corinna Vinschen schreef, Op 4-10-2011 16:29:
> Does it? Even if I'm running a german OS, I absolutely hate to see 
> german diagnostic output from gcc, and I absolutely hate certain 
> programs using non-ASCII chars in output. (In)famous examples are 
> Unicode quoting chars rather than ' or ", or using the Unicode hyphen 
> character rather than -. But that's just me.

You got used to ASCII, like all the old-timers... ;)
export LANG=C is your solution.

By the way, I noticed that with the default locale C.UTF-8 the 
nl_langinfo(CODESET) C function <langinfo.h> returns wrongly 
"ISO-8859-1", while if I set LANG to nl_NL.UTF-8 nl_langinfo(CODESET) 
returns correctly "UTF-8".

The locale command returns LC_ CTYPE="C.UTF-8" and LC_CTYPE="nl_NL.UTF-8".

-- 
Erwin


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: cygwin started speaking German today

[Corinna Vinschen]

On Oct  4 20:03, Erwin Waterlander wrote:
> Corinna Vinschen schreef, Op 4-10-2011 16:29:
> >Does it? Even if I'm running a german OS, I absolutely hate to see
> >german diagnostic output from gcc, and I absolutely hate certain
> >programs using non-ASCII chars in output. (In)famous examples are
> >Unicode quoting chars rather than ' or ", or using the Unicode
> >hyphen character rather than -. But that's just me.
> 
> You got used to ASCII, like all the old-timers... ;)
> export LANG=C is your solution.
> 
> By the way, I noticed that with the default locale C.UTF-8 the
> nl_langinfo(CODESET) C function <langinfo.h> returns wrongly
> "ISO-8859-1",

Not for me:

  $ cat > setl.c <<EOF
  #include <stdio.h>
  #include <locale.h>
  #include <langinfo.h>

  int main(int argc, char **argv)
  {
    char *loc;
    if (argc > 1)
      loc = setlocale (LC_ALL, argv[1]);
    else
      loc = setlocale (LC_CTYPE, NULL);
    printf ("locale: %s charset: %s\n", loc, nl_langinfo (CODESET));
    return 0;
  }
  EOF
  $ gcc -o setl setl.c
  $ ./setl
  locale: C charset: ANSI_X3.4-1968
  $ ./setl C
  locale: C charset: ANSI_X3.4-1968
  $ ./setl C.utf8
  locale: C.utf8 charset: UTF-8
  $ ./setl C.UTF-8
  locale: C.UTF-8 charset: UTF-8


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Project Co-Leader          cygwin AT cygwin DOT com
Red Hat

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: cygwin started speaking German today

[Erwin Waterlander]

Op 4-10-2011 20:20, Corinna Vinschen schreef:
> On Oct  4 20:03, Erwin Waterlander wrote:
>> Corinna Vinschen schreef, Op 4-10-2011 16:29:
>>> Does it? Even if I'm running a german OS, I absolutely hate to see
>>> german diagnostic output from gcc, and I absolutely hate certain
>>> programs using non-ASCII chars in output. (In)famous examples are
>>> Unicode quoting chars rather than ' or ", or using the Unicode
>>> hyphen character rather than -. But that's just me.
>> You got used to ASCII, like all the old-timers... ;)
>> export LANG=C is your solution.
>>
>> By the way, I noticed that with the default locale C.UTF-8 the
>> nl_langinfo(CODESET) C function<langinfo.h>  returns wrongly
>> "ISO-8859-1",
> Not for me:
>
>    $ cat>  setl.c<<EOF
>    #include<stdio.h>
>    #include<locale.h>
>    #include<langinfo.h>
>
>    int main(int argc, char **argv)
>    {
>      char *loc;
>      if (argc>  1)
>        loc = setlocale (LC_ALL, argv[1]);
>      else
>        loc = setlocale (LC_CTYPE, NULL);
>      printf ("locale: %s charset: %s\n", loc, nl_langinfo (CODESET));
>      return 0;
>    }
>    EOF
>    $ gcc -o setl setl.c
>    $ ./setl
>    locale: C charset: ANSI_X3.4-1968
>    $ ./setl C
>    locale: C charset: ANSI_X3.4-1968
>    $ ./setl C.utf8
>    locale: C.utf8 charset: UTF-8
>    $ ./setl C.UTF-8
>    locale: C.UTF-8 charset: UTF-8
>
>
> Corinna
>
Hi,

My program (wcd) uses gettext/libintl. Libintl is causing the effect. 
Libintl is not working properly with a locale C.UTF-8. That is a serious 
problem.

#include <stdio.h>
#include <locale.h>
#include <langinfo.h>
#include <libintl.h>

   int main(int argc, char **argv)
   {
     char *loc;
     if (argc > 1)
       loc = setlocale (LC_ALL, argv[1]);
     else
       loc = setlocale (LC_CTYPE, "");
      bindtextdomain("setl","/usr/share/locale");
      textdomain("setl");
     printf ("locale: %s charset: %s\n", loc, nl_langinfo (CODESET));
     return 0;
   }

waterlan@erwin2 ~/tmp
$ gcc -o setl setl.c -lintl

waterlan@erwin2 ~/tmp
$ echo $LANG
C.UTF-8

waterlan@erwin2 ~/tmp
$ ./setl
locale: en_US charset: ISO-8859-1

waterlan@erwin2 ~/tmp
$ locale
LANG=C.UTF-8
LC_CTYPE="C.UTF-8"
LC_NUMERIC="C.UTF-8"
LC_TIME="C.UTF-8"
LC_COLLATE="C.UTF-8"
LC_MONETARY="C.UTF-8"
LC_MESSAGES="C.UTF-8"
LC_ALL=

waterlan@erwin2 ~/tmp
$ export LANG=nl_NL.UTF-8

waterlan@erwin2 ~/tmp
$ ./setl
locale: nl_NL.UTF-8 charset: UTF-8

waterlan@erwin2 ~/tmp
$ locale
LANG=nl_NL.UTF-8
LC_CTYPE="nl_NL.UTF-8"
LC_NUMERIC="nl_NL.UTF-8"
LC_TIME="nl_NL.UTF-8"
LC_COLLATE="nl_NL.UTF-8"
LC_MONETARY="nl_NL.UTF-8"
LC_MESSAGES="nl_NL.UTF-8"
LC_ALL=


-- 
Erwin Waterlander    waterlan@xs4all.nl
Zeelsterstraat 59B,  5652 EB  Eindhoven,  The Netherlands
www: http://www.xs4all.nl/~waterlan/


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: cygwin started speaking German today

[Corinna Vinschen]

On Oct  5 18:04, Erwin Waterlander wrote:
> Op 4-10-2011 20:20, Corinna Vinschen schreef:
> >On Oct  4 20:03, Erwin Waterlander wrote:
> >>By the way, I noticed that with the default locale C.UTF-8 the
> >>nl_langinfo(CODESET) C function<langinfo.h>  returns wrongly
> >>"ISO-8859-1",
> >Not for me:
> >[...]
> 
> My program (wcd) uses gettext/libintl. Libintl is causing the
> effect. Libintl is not working properly with a locale C.UTF-8. That
> is a serious problem.

That's a bug in libintl8 0.18.1.1-1.  It does not happen with the
previous version 0.17-11.  Hopefully this gets fixed ASAP.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Project Co-Leader          cygwin AT cygwin DOT com
Red Hat

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: cygwin started speaking German today

[Erwin Waterlander]

Op 5-10-2011 18:27, Corinna Vinschen schreef:
> That's a bug in libintl8 0.18.1.1-1. It does not happen with the 
> previous version 0.17-11. Hopefully this gets fixed ASAP. Corinna 

OK, thanks.

-- 
Erwin


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: cygwin started speaking German today

[Charles Wilson]

On 10/5/2011 12:27 PM, Corinna Vinschen wrote:
> On Oct  5 18:04, Erwin Waterlander wrote:
>> Op 4-10-2011 20:20, Corinna Vinschen schreef:
>>> On Oct  4 20:03, Erwin Waterlander wrote:
>>>> By the way, I noticed that with the default locale C.UTF-8 the
>>>> nl_langinfo(CODESET) C function<langinfo.h>  returns wrongly
>>>> "ISO-8859-1",
>>> Not for me:
>>> [...]
>>
>> My program (wcd) uses gettext/libintl. Libintl is causing the
>> effect. Libintl is not working properly with a locale C.UTF-8. That
>> is a serious problem.
> 
> That's a bug in libintl8 0.18.1.1-1.  It does not happen with the
> previous version 0.17-11.  Hopefully this gets fixed ASAP.

There are two issues with libintl 0.18.1.1-1, and apparently both are
contentious.  I don't want to update libintl/gettext until both are
resolved.

One is the discovery that libintl is always doing a very expensive
"no-op" related to relocation (from "/usr/*" to "/usr/*") even when
--disable-relocation.  The discussion of potential fixes is over on the
bug-gnulib mailing list.

The other is the issue that spawned this thread, which raised questions
about how basefiles::/etc/profile.d/lang.{sh,csh} should behave, and
other related complexities.  IIRC we reached an impasse with this
subthread (and replies):
http://cygwin.com/ml/cygwin/2011-09/msg00063.html

See also the various messages in this thread, during the last day or two.

So...I'm rather stuck.  I can't fix anything if we don't have a plan for
what the desired behavior IS.  Right now, we all (except for Bruno!)
agree that $current_behavior is bad.  But how exactly to fix it -- and
whether to do so in opposition to Bruno, the actual libintl maintainer
-- is still an open question.

--
Chuck

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: cygwin started speaking German today

[Ken Brown]

On 10/5/2011 1:31 PM, Charles Wilson wrote:
> On 10/5/2011 12:27 PM, Corinna Vinschen wrote:
>> On Oct  5 18:04, Erwin Waterlander wrote:
>>> Op 4-10-2011 20:20, Corinna Vinschen schreef:
>>>> On Oct  4 20:03, Erwin Waterlander wrote:
>>>>> By the way, I noticed that with the default locale C.UTF-8 the
>>>>> nl_langinfo(CODESET) C function<langinfo.h>   returns wrongly
>>>>> "ISO-8859-1",
>>>> Not for me:
>>>> [...]
>>>
>>> My program (wcd) uses gettext/libintl. Libintl is causing the
>>> effect. Libintl is not working properly with a locale C.UTF-8. That
>>> is a serious problem.
>>
>> That's a bug in libintl8 0.18.1.1-1.  It does not happen with the
>> previous version 0.17-11.  Hopefully this gets fixed ASAP.

[...]

> The other is the issue that spawned this thread, which raised questions
> about how basefiles::/etc/profile.d/lang.{sh,csh} should behave, and
> other related complexities.  IIRC we reached an impasse with this
> subthread (and replies):
> http://cygwin.com/ml/cygwin/2011-09/msg00063.html
>
> See also the various messages in this thread, during the last day or two.
>
> So...I'm rather stuck.  I can't fix anything if we don't have a plan for
> what the desired behavior IS.  Right now, we all (except for Bruno!)
> agree that $current_behavior is bad.  But how exactly to fix it -- and
> whether to do so in opposition to Bruno, the actual libintl maintainer
> -- is still an open question.

I think you're mixing two questions that should be kept separate.  The 
first is how /etc/profile.d/lang.{sh,csh} should set LANG.  That's a 
question that the Cygwin developers and/or base-files maintainer need to 
decide.  The second is whether libintl should override Cygwin's locale 
settings.  Isn't the answer clearly no?  Why can't this be fixed (in 
opposition to Bruno, if necessary) before a final decision is made about 
/etc/profile.d/lang.{sh,csh}?

I don't recall any complaints from Cygwin users about C.UTF-8 being the 
default, but there have already been several complaints about the new 
behavior of libintl.

Ken

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: cygwin started speaking German today

[Corinna Vinschen]

On Oct  5 14:23, Ken Brown wrote:
> I don't recall any complaints from Cygwin users about C.UTF-8 being
> the default, but there have already been several complaints about
> the new behavior of libintl.

Good point.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Project Co-Leader          cygwin AT cygwin DOT com
Red Hat

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: cygwin started speaking German today

[Erwin Waterlander]

Ken Brown schreef, Op 5-10-2011 20:23:
>
> I think you're mixing two questions that should be kept separate.  The 
> first is how /etc/profile.d/lang.{sh,csh} should set LANG.  That's a 
> question that the Cygwin developers and/or base-files maintainer need 
> to decide.  The second is whether libintl should override Cygwin's 
> locale settings.  Isn't the answer clearly no?  Why can't this be 
> fixed (in opposition to Bruno, if necessary) before a final decision 
> is made about /etc/profile.d/lang.{sh,csh}?
>
> I don't recall any complaints from Cygwin users about C.UTF-8 being 
> the default, but there have already been several complaints about the 
> new behavior of libintl.
>

Indeed, these are two separate issues.

My preference is that Cygwin follows by default regional the setting of 
Windows w.r.t. language. So on my Dutch Windows I prefer nl_NL.UTF-8 
over C.UTF-8.
Second, I prefer that libintl follows Cygwin's regional settins, and not 
Windows'.

-- 
Erwin Waterlander


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: cygwin started speaking German today

[Yaakov (Cygwin/X)]

On Wed, 2011-10-05 at 14:23 -0400, Ken Brown wrote:
> I think you're mixing two questions that should be kept separate.  The 
> first is how /etc/profile.d/lang.{sh,csh} should set LANG.  That's a 
> question that the Cygwin developers and/or base-files maintainer need to 
> decide.

Agreed.  There are pros and cons both ways, and I'll leave that for
another message.

> The second is whether libintl should override Cygwin's locale 
> settings.  Isn't the answer clearly no?  Why can't this be fixed (in 
> opposition to Bruno, if necessary) before a final decision is made about 
> /etc/profile.d/lang.{sh,csh}?

Agreed.  Cygwin is a *NIX platform, and libintl needs to respect that by
not trying to set or override LANG/LC_MESSAGES.  Upstream's disagreement
with this point is completely irrelevant; it is simply not their place
to dictate to us how our platform should operate.  While it would seem
that it means carrying a patch for the foreseeable future (not that
gettext releases that often), if upstream refuses to cooperate then I
don't see that we have much of a choice.

You are also correct that this can and should be fixed ASAP, even before
the first point is decided.  +1 for patching this now.


Yaakov



--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: cygwin started speaking German today

[Christopher Faylor]

On Wed, Oct 05, 2011 at 01:50:33PM -0500, Yaakov (Cygwin/X) wrote:
>On Wed, 2011-10-05 at 14:23 -0400, Ken Brown wrote:
>> I think you're mixing two questions that should be kept separate.  The 
>> first is how /etc/profile.d/lang.{sh,csh} should set LANG.  That's a 
>> question that the Cygwin developers and/or base-files maintainer need to 
>> decide.
>
>Agreed.  There are pros and cons both ways, and I'll leave that for
>another message.
>
>> The second is whether libintl should override Cygwin's locale 
>> settings.  Isn't the answer clearly no?  Why can't this be fixed (in 
>> opposition to Bruno, if necessary) before a final decision is made about 
>> /etc/profile.d/lang.{sh,csh}?
>
>Agreed.  Cygwin is a *NIX platform, and libintl needs to respect that by
>not trying to set or override LANG/LC_MESSAGES.  Upstream's disagreement
>with this point is completely irrelevant; it is simply not their place
>to dictate to us how our platform should operate.  While it would seem
>that it means carrying a patch for the foreseeable future (not that
>gettext releases that often), if upstream refuses to cooperate then I
>don't see that we have much of a choice.
>
>You are also correct that this can and should be fixed ASAP, even before
>the first point is decided.  +1 for patching this now.

Ditto on all of the above points.

This is a free software project so, of course, every decision is up for
debate but the ultimate decision on how our project does stuff rests
with us, the maintainers of the stuff that the end-users see.  So, while
Bruno's opinion is valuable, it should not be definitive.

cgf

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: cygwin started speaking German today

[Corinna Vinschen]

Chuck, ping?

On Oct  5 14:23, Ken Brown wrote:
> On 10/5/2011 1:31 PM, Charles Wilson wrote:
> >On 10/5/2011 12:27 PM, Corinna Vinschen wrote:
> >>On Oct  5 18:04, Erwin Waterlander wrote:
> >>>Op 4-10-2011 20:20, Corinna Vinschen schreef:
> >>>>On Oct  4 20:03, Erwin Waterlander wrote:
> >>>>>By the way, I noticed that with the default locale C.UTF-8 the
> >>>>>nl_langinfo(CODESET) C function<langinfo.h>   returns wrongly
> >>>>>"ISO-8859-1",
> >>>>Not for me:
> >>>>[...]
> >>>
> >>>My program (wcd) uses gettext/libintl. Libintl is causing the
> >>>effect. Libintl is not working properly with a locale C.UTF-8. That
> >>>is a serious problem.
> >>
> >>That's a bug in libintl8 0.18.1.1-1.  It does not happen with the
> >>previous version 0.17-11.  Hopefully this gets fixed ASAP.
> 
> [...]
> 
> >The other is the issue that spawned this thread, which raised questions
> >about how basefiles::/etc/profile.d/lang.{sh,csh} should behave, and
> >other related complexities.  IIRC we reached an impasse with this
> >subthread (and replies):
> >http://cygwin.com/ml/cygwin/2011-09/msg00063.html
> >
> >See also the various messages in this thread, during the last day or two.
> >
> >So...I'm rather stuck.  I can't fix anything if we don't have a plan for
> >what the desired behavior IS.  Right now, we all (except for Bruno!)
> >agree that $current_behavior is bad.  But how exactly to fix it -- and
> >whether to do so in opposition to Bruno, the actual libintl maintainer
> >-- is still an open question.
> 
> I think you're mixing two questions that should be kept separate.
> The first is how /etc/profile.d/lang.{sh,csh} should set LANG.
> That's a question that the Cygwin developers and/or base-files
> maintainer need to decide.  The second is whether libintl should
> override Cygwin's locale settings.  Isn't the answer clearly no?
> Why can't this be fixed (in opposition to Bruno, if necessary)
> before a final decision is made about /etc/profile.d/lang.{sh,csh}?
> 
> I don't recall any complaints from Cygwin users about C.UTF-8 being
> the default, but there have already been several complaints about
> the new behavior of libintl.

This is *really* annoying behaviour.  Right now the gawk testsuite
fails, because libintl thinks it has to use the german language,
even though $LANG is set to C.UTF-8.

What's even more annoying is the fact that it's not sufficient to revert
libintl8 to 0.17-11, but you also have to revert gettext-devel to
0.17-11 *and* recompile gawk, because the 0.18.1 version redefines
setlocale to libintl_setlocale.  The effect is that applications built
against 0.18 don't run with the 0.17 DLL.  Thus, if you don't have
control over the binary (aka "normal Cygwin user"), you can not even
revert to libintl8 0.17-11, because that may break newly built
applications.

Please consider to provide a new libintl/gettext without this bug soon.


Thanks,
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Project Co-Leader          cygwin AT cygwin DOT com
Red Hat

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: cygwin started speaking German today

[Erwin Waterlander]

On 10/10/2011 7:23 PM, Corinna Vinschen wrote:
> Chuck, ping?
>
> On Oct  5 14:23, Ken Brown wrote:
>> On 10/5/2011 1:31 PM, Charles Wilson wrote:
>>> On 10/5/2011 12:27 PM, Corinna Vinschen wrote:
>>>> On Oct  5 18:04, Erwin Waterlander wrote:
>>>>> Op 4-10-2011 20:20, Corinna Vinschen schreef:
>>>>>> On Oct  4 20:03, Erwin Waterlander wrote:
>>>>>>> By the way, I noticed that with the default locale C.UTF-8 the
>>>>>>> nl_langinfo(CODESET) C function<langinfo.h>    returns wrongly
>>>>>>> "ISO-8859-1",
>>>>>> Not for me:
>>>>>> [...]
>>>>> My program (wcd) uses gettext/libintl. Libintl is causing the
>>>>> effect. Libintl is not working properly with a locale C.UTF-8. That
>>>>> is a serious problem.
>>>> That's a bug in libintl8 0.18.1.1-1.  It does not happen with the
>>>> previous version 0.17-11.  Hopefully this gets fixed ASAP.
>> [...]
>>
>>> The other is the issue that spawned this thread, which raised questions
>>> about how basefiles::/etc/profile.d/lang.{sh,csh} should behave, and
>>> other related complexities.  IIRC we reached an impasse with this
>>> subthread (and replies):
>>> http://cygwin.com/ml/cygwin/2011-09/msg00063.html
>>>
>>> See also the various messages in this thread, during the last day or two.
>>>
>>> So...I'm rather stuck.  I can't fix anything if we don't have a plan for
>>> what the desired behavior IS.  Right now, we all (except for Bruno!)
>>> agree that $current_behavior is bad.  But how exactly to fix it -- and
>>> whether to do so in opposition to Bruno, the actual libintl maintainer
>>> -- is still an open question.
>> I think you're mixing two questions that should be kept separate.
>> The first is how /etc/profile.d/lang.{sh,csh} should set LANG.
>> That's a question that the Cygwin developers and/or base-files
>> maintainer need to decide.  The second is whether libintl should
>> override Cygwin's locale settings.  Isn't the answer clearly no?
>> Why can't this be fixed (in opposition to Bruno, if necessary)
>> before a final decision is made about /etc/profile.d/lang.{sh,csh}?
>>
>> I don't recall any complaints from Cygwin users about C.UTF-8 being
>> the default, but there have already been several complaints about
>> the new behavior of libintl.
> This is *really* annoying behaviour.  Right now the gawk testsuite
> fails, because libintl thinks it has to use the german language,
> even though $LANG is set to C.UTF-8.
>
> What's even more annoying is the fact that it's not sufficient to revert
> libintl8 to 0.17-11, but you also have to revert gettext-devel to
> 0.17-11 *and* recompile gawk, because the 0.18.1 version redefines
> setlocale to libintl_setlocale.  The effect is that applications built
> against 0.18 don't run with the 0.17 DLL.  Thus, if you don't have
> control over the binary (aka "normal Cygwin user"), you can not even
> revert to libintl8 0.17-11, because that may break newly built
> applications.
>
> Please consider to provide a new libintl/gettext without this bug soon.
>
>

Indeed! My application doesn't work correctly, because the wrong 
character encoding is returned. I get ISO-8859-1 while I should get 
UTF-8. And while bypassing Cygwin's locale it doesn't even do that 
properly. Libintl returns ISO-8859-1 while my Windows' locale character 
encoding is CP1252 (which isn't the same).

-- 
Erwin

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: cygwin started speaking German today

[Charles Wilson]

On 10/10/2011 1:23 PM, Corinna Vinschen wrote:
> Chuck, ping?
> Please consider to provide a new libintl/gettext without this bug soon.

I plan to *start* that process Wednesday night. It takes many hours to
complete and validate -- so, I should have it by this weekend.

Consensus does appear to be unanimous on what to do; I just need to
review all the postings and figure out exactly /how/ to do it.

--
Chuck

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: cygwin started speaking German today

[Christopher Faylor]

On Tue, Oct 11, 2011 at 12:54:34PM -0400, Charles Wilson wrote:
>Consensus does appear to be unanimous on what to do; I just need to
>review all the postings and figure out exactly /how/ to do it.

I don't envy you.  :-)

cgf

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: cygwin started speaking German today

[Charles Wilson]

[-- Attachment #1: Type: text/plain, Size: 2137 bytes --]

On 10/11/2011 12:54 PM, Charles Wilson wrote:
> Consensus does appear to be unanimous on what to do; I just need to
> review all the postings and figure out exactly /how/ to do it.

I have uploaded the new packages.  There are three new patches:

1) modified localename.c significantly.  No longer "ignores" 
LANG=C.UTF-8; also does not try to do any parsing of the Windows I18N 
settings.  Basically, acts like linux -- if the value of the locale 
string isn't supported by the underlying setlocale() implementation, 
then it is ignored (e.g. default back to "C.UTF-8" or "C") -- libintl 
doesn't try to 'be smart' -- or to second-guess.  Also, relies on 
cygwin's glibc-like setlocale(LC_*, NULL) behavior -- which has been 
supported by newlib since the cygwin 1.5 days (even if it always 
returned "C" back then).

2) Fixes to the test suite related to the above changes.  From the 
CHECK_NOTES file:

NOTE: there were four other failures that arose because of the
       changes to localename.  These were:
          format-c-3
          format-c-4
          plural-1
          plural-2
       but the test suite was patched to avoid these failures.
       The problem is rather complicated; gettext-0.18.1.1 now has a
       libintl_setlocale function that is used by libintl clients,
       instead of the system setlocale.  The localename patch left
       that in place, but ensured that it did far less work -- was less
       obtrusive -- in its interposition between clients and the system
       setlocale.  However, some of that eliminated 'work' also allowed
       the gettext test suite to 'cheat' a bit -- it used non-existent
       locales (like 'fr' rather than 'fr_FR'), and *re-hijacked* the
       setlocale function itself.  None of that works anymore -- and
       *actual* clients are unlikely to need/want to do it -- so the
       test suite was patched to use real locales in all cases, and to
       not re-hijack.

3) Adopted Bruno's upstream changes to relocatable.c, turning off 
"expensive" relocation support in libintl.

Odds of #1 and #2 being adopted upstream are effectively nil, so...

--
Chuck


[-- Attachment #2: patch-1.txt --]
[-- Type: text/plain, Size: 2607 bytes --]

diff -u old/gettext-0.18.1.1/gettext-runtime/intl/localename.c new/gettext-0.18.1.1/gettext-runtime/intl/localename.c
--- old/gettext-0.18.1.1/gettext-runtime/intl/localename.c	2011-10-15 00:21:37.853133600 -0400
+++ new/gettext-0.18.1.1/gettext-runtime/intl/localename.c	2011-10-15 00:29:27.601133600 -0400
@@ -59,7 +59,7 @@
 # define WIN32_NATIVE
 #endif
 
-#if defined WIN32_NATIVE || defined __CYGWIN__ /* WIN32 or Cygwin */
+#if defined WIN32_NATIVE /* WIN32 */
 # define WIN32_LEAN_AND_MEAN
 # include <windows.h>
 /* List of language codes, sorted by value:
@@ -1407,7 +1407,7 @@
 #endif
 
 
-#if defined WIN32_NATIVE || defined __CYGWIN__ /* WIN32 or Cygwin */
+#if defined WIN32_NATIVE /* WIN32 */
 
 /* Canonicalize a Win32 native locale name to a Unix locale name.
    NAME is a sufficiently large buffer.
@@ -2770,8 +2770,8 @@
     setting of 'local'."
    However it does not specify the exact format.  Neither do SUSV2 and
    ISO C 99.  So we can use this feature only on selected systems (e.g.
-   those using GNU C Library).  */
-#if defined _LIBC || (defined __GLIBC__ && __GLIBC__ >= 2)
+   those using GNU C Library, or cygwin [1.5 and 1.7+]).  */
+#if defined _LIBC || (defined __GLIBC__ && __GLIBC__ >= 2) || defined __CYGWIN__
 # define HAVE_LOCALE_NULL
 #endif
 
@@ -2826,11 +2826,6 @@
          Ignore invalid LANG value set by the Terminal application.  */
       if (strcmp (retval, "UTF-8") != 0)
 #endif
-#if defined __CYGWIN__
-      /* Cygwin.
-         Ignore dummy LANG value set by ~/.profile.  */
-      if (strcmp (retval, "C.UTF-8") != 0)
-#endif
         return retval;
     }
 
@@ -2923,7 +2918,7 @@
 
 # endif
 
-# if defined WIN32_NATIVE || defined __CYGWIN__ /* WIN32 or Cygwin */
+# if defined WIN32_NATIVE /* WIN32 */
   {
     LCID lcid;
 
@@ -2933,6 +2928,23 @@
     return gl_locale_name_from_win32_LCID (lcid);
   }
 # endif
+# if defined __CYGWIN__
+  {
+    /* Rarely arrive here. This function is called only when an earlier
+     * call to gl_locale_name_posix() or gl_locale_name_environ()
+     * returned NULL.  That first function now simply delegates to
+     * setlocale (LC_*, NULL), which never fails on cygwin.  But...for
+     * completeness, or when called after gl_locale_name_environ() and
+     * none are set, go ahead and specify the cygwin default. Cheat a bit
+     * to distinguish old cygwin (1.5 and below) from new cygwin (1.7+).
+     */
+#  if PATH_MAX < 261   /* cygwin 1.5 or below */
+    return "C";
+#  else                /* PATH_MAX = 4096, cygwin 1.7 or above */
+    return "C.UTF-8";
+#  endif
+  }
+# endif
 #endif
 }
 

[-- Attachment #3: patch-2.txt --]
[-- Type: text/plain, Size: 3544 bytes --]

--- origsrc/gettext-0.18.1.1/gettext-tools/tests/format-c-3-prg.c	2010-06-06 08:49:58.000000000 -0400
+++ src/gettext-0.18.1.1/gettext-tools/tests/format-c-3-prg.c	2011-10-15 22:54:48.494133600 -0400
@@ -34,7 +34,7 @@
 /* Disable the override of setlocale that libgnuintl.h activates on MacOS X
    and Windows.  This test relies on the fake setlocale function in
    setlocale.c.  */
-#undef setlocale
+/* #undef setlocale */
 
 #define _(string) gettext (string)
 
--- origsrc/gettext-0.18.1.1/gettext-tools/tests/format-c-4-prg.c	2010-06-06 08:49:58.000000000 -0400
+++ src/gettext-0.18.1.1/gettext-tools/tests/format-c-4-prg.c	2011-10-15 21:49:23.028133600 -0400
@@ -34,7 +34,7 @@
 /* Disable the override of setlocale that libgnuintl.h activates on MacOS X
    and Windows.  This test relies on the fake setlocale function in
    setlocale.c.  */
-#undef setlocale
+/* #undef setlocale */
 
 #define _(string) gettext (string)
 
--- origsrc/gettext-0.18.1.1/gettext-tools/tests/plural-1	2010-06-06 08:49:58.000000000 -0400
+++ src/gettext-0.18.1.1/gettext-tools/tests/plural-1	2011-10-15 22:32:46.440133600 -0400
@@ -65,15 +65,15 @@ ${DIFF} fr.po.strip fr.po.un || exit 1
 tmpfiles="$tmpfiles cake.ok cake.tmp cake.out"
 : ${DIFF=diff}
 echo 'un morceau de gateau' > cake.ok
-LANGUAGE= ./cake fr 1 > cake.tmp || exit 1
+LANGUAGE= ./cake fr_FR 1 > cake.tmp || exit 1
 LC_ALL=C tr -d '\r' < cake.tmp > cake.out || exit 1
 ${DIFF} cake.ok cake.out || exit 1
 echo '2 morceaux de gateau' > cake.ok
-LANGUAGE= ./cake fr 2 > cake.tmp || exit 1
+LANGUAGE= ./cake fr_FR 2 > cake.tmp || exit 1
 LC_ALL=C tr -d '\r' < cake.tmp > cake.out || exit 1
 ${DIFF} cake.ok cake.out || exit 1
 echo '10 morceaux de gateau' > cake.ok
-LANGUAGE= ./cake fr 10 > cake.tmp || exit 1
+LANGUAGE= ./cake fr_FR 10 > cake.tmp || exit 1
 LC_ALL=C tr -d '\r' < cake.tmp > cake.out || exit 1
 ${DIFF} cake.ok cake.out || exit 1
 
--- origsrc/gettext-0.18.1.1/gettext-tools/tests/plural-1-prg.c	2010-06-06 08:49:58.000000000 -0400
+++ src/gettext-0.18.1.1/gettext-tools/tests/plural-1-prg.c	2011-10-15 23:00:57.110133600 -0400
@@ -30,7 +30,7 @@
 /* Disable the override of setlocale that libgnuintl.h activates on MacOS X
    and Windows.  This test relies on the fake setlocale function in
    setlocale.c.  */
-#undef setlocale
+/* #undef setlocale */
 
 int
 main (int argc, char *argv[])
--- origsrc/gettext-0.18.1.1/gettext-tools/tests/plural-2	2010-06-06 08:49:58.000000000 -0400
+++ src/gettext-0.18.1.1/gettext-tools/tests/plural-2	2011-10-15 22:51:09.838133600 -0400
@@ -3,10 +3,10 @@
 tmpfiles=""
 trap 'rm -fr $tmpfiles' 1 2 3 15
 
-tmpfiles="$tmpfiles ll ll.po dataout"
+tmpfiles="$tmpfiles es ll.po dataout"
 : ${MSGFMT=msgfmt}
-test -d ll || mkdir ll
-test -d ll/LC_MESSAGES || mkdir ll/LC_MESSAGES
+test -d es || mkdir es
+test -d es/LC_MESSAGES || mkdir es/LC_MESSAGES
 
 tmpfiles="$tmpfiles plural-2.data"
 cat <<EOF > plural-2.data
@@ -68,10 +68,10 @@ msgstr[7] "7"
 msgstr[8] "8"
 msgstr[9] "9"
 EOF
-  ${MSGFMT} -o ll/LC_MESSAGES/plural.mo ll.po || exit 1
+  ${MSGFMT} -o es/LC_MESSAGES/plural.mo ll.po || exit 1
   (for i in '' 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19; do
      LANGUAGE= TEXTDOMAIN=plural TEXTDOMAINDIR=. \
-       $NGETTEXT --env LC_ALL=ll X Y ${i}0 ${i}1 ${i}2 ${i}3 ${i}4 ${i}5 ${i}6 ${i}7 ${i}8 ${i}9
+       $NGETTEXT --env LC_ALL=es_ES X Y ${i}0 ${i}1 ${i}2 ${i}3 ${i}4 ${i}5 ${i}6 ${i}7 ${i}8 ${i}9
    done) > dataout
   test "$dataok" = `cat dataout` || {
     echo "Formula evaluation error for language $lang" 1>&2

[-- Attachment #4: patch-3.txt --]
[-- Type: text/plain, Size: 2493 bytes --]

diff -u old/gettext-0.18.1.1/gettext-runtime/gnulib-lib/relocatable.c new/gettext-0.18.1.1/gettext-runtime/gnulib-lib/relocatable.c
--- old/gettext-0.18.1.1/gettext-runtime/gnulib-lib/relocatable.c
+++ new/gettext-0.18.1.1/gettext-runtime/gnulib-lib/relocatable.c
@@ -85,6 +85,19 @@
 # define FILE_SYSTEM_PREFIX_LEN(P) 0
 #endif
 
+/* Whether to enable the more costly support for relocatable libraries.
+   It allows libraries to be have been installed with a different original
+   prefix than the program.  But it is quite costly, especially on Cygwin
+   platforms, see below.  Therefore we enable it by default only on native
+   Win32 platforms.  */
+#ifndef ENABLE_COSTLY_RELOCATABLE
+# if (defined _WIN32 || defined __WIN32__) && !defined __CYGWIN__
+#  define ENABLE_COSTLY_RELOCATABLE 1
+# else
+#  define ENABLE_COSTLY_RELOCATABLE 0
+# endif
+#endif
+
 /* Original installation prefix.  */
 static char *orig_prefix;
 static size_t orig_prefix_len;
@@ -154,7 +167,7 @@ set_relocation_prefix (const char *orig_prefix_arg, const char *curr_prefix_arg)
 #endif
 }
 
-#if !defined IN_LIBRARY || (defined PIC && defined INSTALLDIR)
+#if !defined IN_LIBRARY || (defined PIC && defined INSTALLDIR && ENABLE_COSTLY_RELOCATABLE)
 
 /* Convenience function:
    Computes the current installation prefix, based on the original
@@ -284,7 +297,7 @@ compute_curr_prefix (const char *orig_installprefix,
 
 #endif /* !IN_LIBRARY || PIC */
 
-#if defined PIC && defined INSTALLDIR
+#if defined PIC && defined INSTALLDIR && ENABLE_COSTLY_RELOCATABLE
 
 /* Full pathname of shared library, or NULL.  */
 static char *shared_library_fullname;
@@ -330,7 +343,9 @@ find_shared_library_fullname ()
 #if (defined __linux__ && (__GLIBC__ >= 2 || defined __UCLIBC__)) || defined __CYGWIN__
   /* Linux has /proc/self/maps. glibc 2 and uClibc have the getline()
      function.
-     Cygwin >= 1.5 has /proc/self/maps and the getline() function too.  */
+     Cygwin >= 1.5 has /proc/self/maps and the getline() function too.
+     But it is costly: ca. 0.3 ms on Linux, 3 ms on Cygwin 1.5, and 5 ms on
+     Cygwin 1.7.  */
   FILE *fp;
 
   /* Open the current process' maps file.  It describes one VMA per line.  */
@@ -403,7 +418,7 @@ get_shared_library_fullname ()
 const char *
 relocate (const char *pathname)
 {
-#if defined PIC && defined INSTALLDIR
+#if defined PIC && defined INSTALLDIR && ENABLE_COSTLY_RELOCATABLE
   static int initialized;
 
   /* Initialization code for a shared library.  */

[-- Attachment #5: Type: text/plain, Size: 218 bytes --]

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: cygwin started speaking German today

[Corinna Vinschen]

On Oct 16 14:42, Charles Wilson wrote:
> On 10/11/2011 12:54 PM, Charles Wilson wrote:
> >Consensus does appear to be unanimous on what to do; I just need to
> >review all the postings and figure out exactly /how/ to do it.
> 
> I have uploaded the new packages.  There are three new patches:
> 
> 1) modified localename.c significantly.  No longer "ignores"
> LANG=C.UTF-8; also does not try to do any parsing of the Windows
> I18N settings.  Basically, acts like linux -- if the value of the
> locale string isn't supported by the underlying setlocale()
> implementation, then it is ignored (e.g. default back to "C.UTF-8"
> or "C") -- libintl doesn't try to 'be smart' -- or to second-guess.
> Also, relies on cygwin's glibc-like setlocale(LC_*, NULL) behavior
> -- which has been supported by newlib since the cygwin 1.5 days
> (even if it always returned "C" back then).

Thanks!

> 2) Fixes to the test suite related to the above changes.
> 
> 3) Adopted Bruno's upstream changes to relocatable.c, turning off
> "expensive" relocation support in libintl.
> 
> Odds of #1 and #2 being adopted upstream are effectively nil, so...

I don't understand this.  It's very clear that the former, unfixed
behaviour is totally wrong for Cygwin, especially in terms of the
used charset.  I don't see why doing the right thing, using less special
cases for Cygwin in gettext/libintl should be unacceptable for upstream.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Project Co-Leader          cygwin AT cygwin DOT com
Red Hat

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: cygwin started speaking German today

[Charles Wilson]

On 10/17/2011 2:59 AM, Corinna Vinschen wrote:
> On Oct 16 14:42, Charles Wilson wrote:
>> 2) Fixes to the test suite related to the above changes.
>>
>> 3) Adopted Bruno's upstream changes to relocatable.c, turning off
>> "expensive" relocation support in libintl.
>>
>> Odds of #1 and #2 being adopted upstream are effectively nil, so...
>
> I don't understand this.  It's very clear that the former, unfixed
> behaviour is totally wrong for Cygwin, especially in terms of the
> used charset.  I don't see why doing the right thing, using less special
> cases for Cygwin in gettext/libintl should be unacceptable for upstream.

As for #1, I doubt Bruno would accept those changes upstream until we 
address his four points, because (I think) he is claiming that current 
(e.g. prior to my patch) libintl DOES do all of these things, and he'd 
view my change as breaking all four of them.

Bruno Haible wrote:
> OK, then the following four facilities are needed in Cygwin.
>
> 1) We need the name of the locale which is in effect when the user has
>    not specified environment variables.
>
>    Either through option a) above [*]. Programs can then do getenv ("LANG").
>    Cygwin documentation <http://www.cygwin.com/cygwin-ug-net/setup-locale.html>
>    currently says "The default locale in the absence of the aforementioned
>    locale environment variables is "C.UTF-8"." This would have to change.
>
>    Or through option b) above [**]. Programs can then peek at the return
>    value of  setlocale (LC_ALL, "").
 >
 >    Or through an API function that calls GetUserDefaultLCID() and
 >    converts that to a glibc style locale name (e.g. "zh_CN.UTF-8")
 >    or to an RFC 3066 style locale name (e.g. "zh-Hans").
 >

Option (b) has already been rejected.  Option (a) might be doable, via 
/etc/profile.d/lang.{sh,csh} + locale.exe.  But this re-opens the 
can-o-worms; what's changed since we decided that "default" user 
experience (which is distinct from cygwin::setlocale's default behavior) 
should be C.UTF-8?

> 2) We need the name of the locale of the current thread.
>
>    Either through a function newlocale(), as in POSIX.
>
>    Or through an API function that calls GetThreadLocale() and
>    converts that to a glibc style locale name (e.g. "zh_CN.UTF-8")
>    or to an RFC 3066 style locale name (e.g. "zh-Hans").
>
>    Locale per thread is mainly needed for web application servers,
>    not for GUI programs.

We need an implementation of newlocale, if we want to address this 
problem and stay "posixy".

> 3) Gettext needs the priority list of languages, if the "Regional Settings"
>    panel can specify it. MacOS X has this setting customizable, I don't know
>    whether newer Windows versions have it has well.

I don't understand this.  cygwin either does or does not support any 
give language -- and the list is pretty comprehensive, so the odds of 
"does not" is pretty low.  Unless he's talking about a per-application 
basis: "I don't have an .mo file for german, but the user has indicated 
they ALSO speak french, and I DO have an .mo file for fr_FR, so I'll use 
that -- even though LANG="de_DE"?

That seems pretty anti-posix...

> 4) Programs that do number or date/time formatting will need to access the
>    values that the user has specified. E.g. those set in
>    <http://www.sisulizer.de/_img/codepage-problems/codepage-regional.jpg>
>    <http://pc-error-free.com/blog/wp-content/uploads/2008/12/regional-settings.gif>
>    <http://www.sisulizer.de/_img/codepage-problems/w7-regions-and-languages-formats.jpg>

I guess this is talking about two different things: (1) locale.exe -s 
needs to check the win32 date/time settings when computing the proper 
value of "-c LC_TIME", and (2) *applications* would also need an ability 
to grab the *custom* date/time formatting strings from the relevant 
windows settings -- probably via a special cygwin interface.

[*] Bruno's "option a"
>   a) The system can set environment variables that reflect the regional
>      settings. For example, if the user has chosen German, Cygwin's
>      login process could set LANG to de_DE.UTF-8.
>
>      This approach is used in Linux desktops like KDE.

[**] Bruno's "option b"
>   b) The system's setlocale() function can, when the second argument is
>      the empty string and the respective environment variables don't
>      specify anything, fetch the value from the "Regional settings"
>      panel.
>
>      Cygwin could do that.


With regards to #2 *as is*...adopting those patches would break the 
tests on all other platforms.  They could be modified to use #ifdef 
__CYGWIN__ and then maybe they'd be ok -- but that depends on #1 being 
merged first.

--
Chuck

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: cygwin started speaking German today

[Corinna Vinschen]

On Oct 17 09:16, Charles Wilson wrote:
> On 10/17/2011 2:59 AM, Corinna Vinschen wrote:
> >On Oct 16 14:42, Charles Wilson wrote:
> >>2) Fixes to the test suite related to the above changes.
> >>
> >>3) Adopted Bruno's upstream changes to relocatable.c, turning off
> >>"expensive" relocation support in libintl.
> >>
> >>Odds of #1 and #2 being adopted upstream are effectively nil, so...
> >
> >I don't understand this.  It's very clear that the former, unfixed
> >behaviour is totally wrong for Cygwin, especially in terms of the
> >used charset.  I don't see why doing the right thing, using less special
> >cases for Cygwin in gettext/libintl should be unacceptable for upstream.
> 
> As for #1, I doubt Bruno would accept those changes upstream until
> we address his four points, because (I think) he is claiming that
> current (e.g. prior to my patch) libintl DOES do all of these
> things, and he'd view my change as breaking all four of them.

If he claims that, he's wrong.  Look, Cygwin was (most of the time)
running in UTF-8 mode.  Did anybody complain before that his or her
libintl running apps were using the wrong codeset when LANG was set 
to "xyz.UTF-8"?  I don't see that.  Only with 0.18.1.1 libintl
suddendly started to set the charset to ISO-8859-1 for some people,
even though LANG was set to some "xyz.UTF-8".  This is a very clear
breakage of Cygwin, and former versions of libintl did not do that.

Or, did they?  Even *if* they did that, it's stll breaking Cygwin
application's localization, so it's clearly a bug in libintl.

> 
> Bruno Haible wrote:
> >OK, then the following four facilities are needed in Cygwin.
> >
> >1) We need the name of the locale which is in effect when the user has
> >   not specified environment variables.
> >
> >   Either through option a) above [*]. Programs can then do getenv ("LANG").
> >   Cygwin documentation <http://www.cygwin.com/cygwin-ug-net/setup-locale.html>
> >   currently says "The default locale in the absence of the aforementioned
> >   locale environment variables is "C.UTF-8"." This would have to change.
> >
> >   Or through option b) above [**]. Programs can then peek at the return
> >   value of  setlocale (LC_ALL, "").
> >
> >    Or through an API function that calls GetUserDefaultLCID() and
> >    converts that to a glibc style locale name (e.g. "zh_CN.UTF-8")
> >    or to an RFC 3066 style locale name (e.g. "zh-Hans").
> >
> 
> Option (b) has already been rejected.  Option (a) might be doable,
> via /etc/profile.d/lang.{sh,csh} + locale.exe.  But this re-opens
> the can-o-worms; what's changed since we decided that "default" user
> experience (which is distinct from cygwin::setlocale's default
> behavior) should be C.UTF-8?

This is not correct.  On Linux, the default locale is "C" or "POSIX",
unless LANG or LC_xxx is set to something else.  And that only happens
if the startup script reads /etc/sysconfig/i18n and/or ~/.i18n.  Cygwin
behaves the same, except that the default locale is "C.UTF-8", unless
LANG is set to something else in /etc/profile.d/lang.sh or in some
user defined profile.

> >2) We need the name of the locale of the current thread.
> >
> >   Either through a function newlocale(), as in POSIX.
> >
> >   Or through an API function that calls GetThreadLocale() and
> >   converts that to a glibc style locale name (e.g. "zh_CN.UTF-8")
> >   or to an RFC 3066 style locale name (e.g. "zh-Hans").
> >
> >   Locale per thread is mainly needed for web application servers,
> >   not for GUI programs.
> 
> We need an implementation of newlocale, if we want to address this
> problem and stay "posixy".

Right now we don't need the locale of a single thread at all, because
Cygwin doesn't support this.  A Cygwin application must not call
SetThreadLocale since the locale will not be reflected by the POSIX
locale settings in Cygwin.

As soon as Cygwin supports per-thread locales, a newlocale function
will be available of course.  http://cygwin.com/acronyms/#SHTDI.

> >3) Gettext needs the priority list of languages, if the "Regional Settings"
> >   panel can specify it. MacOS X has this setting customizable, I don't know
> >   whether newer Windows versions have it has well.
> 
> I don't understand this.  cygwin either does or does not support any
> give language -- and the list is pretty comprehensive, so the odds
> of "does not" is pretty low.  Unless he's talking about a
> per-application basis: "I don't have an .mo file for german, but the
> user has indicated they ALSO speak french, and I DO have an .mo file
> for fr_FR, so I'll use that -- even though LANG="de_DE"?
> 
> That seems pretty anti-posix...

Yup.  You can give him the list provided by `locale -av' and `locale
-m'.  This should give him a good idea what languages and codesets are
supported by Cygwin.  It should *especially* show him that Cygwin's
default codeset for a language given by, say "LANG=de_DE" is *not* the
default Windows codepage for that language.  For instance, the default
codeset for uk_UA is KOI8-U, as on Linux, not CP1251.

> >4) Programs that do number or date/time formatting will need to access the
> >   values that the user has specified. E.g. those set in
> >   <http://www.sisulizer.de/_img/codepage-problems/codepage-regional.jpg>
> >   <http://pc-error-free.com/blog/wp-content/uploads/2008/12/regional-settings.gif>
> >   <http://www.sisulizer.de/_img/codepage-problems/w7-regions-and-languages-formats.jpg>
> 
> I guess this is talking about two different things: (1) locale.exe
> -s needs to check the win32 date/time settings when computing the
> proper value of "-c LC_TIME", and (2) *applications* would also need
> an ability to grab the *custom* date/time formatting strings from
> the relevant windows settings -- probably via a special cygwin
> interface.

The problem is that Bruno tries to impose Windows over Cygwin.  That's
not what Cygwin is about.  Why can't he accept that?

> [*] Bruno's "option a"
> >  a) The system can set environment variables that reflect the regional
> >     settings. For example, if the user has chosen German, Cygwin's
> >     login process could set LANG to de_DE.UTF-8.
> >
> >     This approach is used in Linux desktops like KDE.
> 
> [**] Bruno's "option b"
> >  b) The system's setlocale() function can, when the second argument is
> >     the empty string and the respective environment variables don't
> >     specify anything, fetch the value from the "Regional settings"
> >     panel.
> >
> >     Cygwin could do that.

That's what /etc/profile.d/lang.sh and lang.csh is about.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Project Co-Leader          cygwin AT cygwin DOT com
Red Hat

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: cygwin started speaking German today

[Corinna Vinschen]

On Oct 17 15:51, Corinna Vinschen wrote:
> On Oct 17 09:16, Charles Wilson wrote:
> The problem is that Bruno tries to impose Windows over Cygwin.  That's
> not what Cygwin is about.  Why can't he accept that?
> 
> > [*] Bruno's "option a"
> > >  a) The system can set environment variables that reflect the regional
> > >     settings. For example, if the user has chosen German, Cygwin's
> > >     login process could set LANG to de_DE.UTF-8.
> > >
> > >     This approach is used in Linux desktops like KDE.
> > 
> > [**] Bruno's "option b"
> > >  b) The system's setlocale() function can, when the second argument is
> > >     the empty string and the respective environment variables don't
> > >     specify anything, fetch the value from the "Regional settings"
> > >     panel.
> > >
> > >     Cygwin could do that.
> 
> That's what /etc/profile.d/lang.sh and lang.csh is about.

Oh, and, btw., even *if* that would be treated as a bug in Cygwin,
it's not a library's task to second guess over the head of the
underlying POSIX system, as cgf has pointed out already a while ago.

Consider libintl would do the same on Linux:  The user has set $LANG
to "C.UTF-8" and libintl would look into /etc/sysconfig/i18n and
override the user's decision.  Well, Linux applications usually use
glibc localization functions, so that won't happen, but I don't
think Bruno would have many friends in the Linux community...


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Project Co-Leader          cygwin AT cygwin DOT com
Red Hat

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: cygwin started speaking German today

[Corinna Vinschen]

On Oct  5 13:31, Charles Wilson wrote:
> The other is the issue that spawned this thread, which raised questions
> about how basefiles::/etc/profile.d/lang.{sh,csh} should behave, and
> other related complexities.  IIRC we reached an impasse with this
> subthread (and replies):
> http://cygwin.com/ml/cygwin/2011-09/msg00063.html
> 
> See also the various messages in this thread, during the last day or two.

I don't see an impasse here.  It's pretty clear that the system libintl
is talking to is Cygwin in the first place, and since Cygwin has another
language and *especially* charset setting, that should be honored by
libintl.  Everything else is just about the usage of locale -s/-u in
the profiles and that's entirely within the realms of Cygwin as well.
I don't think Bruno really has a different point of view here.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Project Co-Leader          cygwin AT cygwin DOT com
Red Hat

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: cygwin started speaking German today

[Ken Brown]

On 10/4/2011 8:45 AM, Charles Wilson wrote:
> However, one issue is that windows basically will always have SOME
> setting -- even if just "English".  Which would cause locale to report
> 'en_US' or something.  So you'd never actually SEE the "default default"
> of C.UTF-8 take effect.

You'd see it if you unset LANG and an application calls 
setlocale(LC_ALL, "").  Linux is similar.  In Fedora 14, for instance, 
the standard startup files set LANG via /etc/sysconfig/i18n; but there 
is also a "default default" of C that's used if you unset LANG.

Ken

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: cygwin started speaking German today

[Charles Wilson]

On 9/9/2011 8:33 AM, Andy Koppe wrote:
> On 9 September 2011 10:18, Charles Wilson wrote:
>> #2, some older versions of the base-files startup
>> scripts (/etc/profile, /etc/skel/.*, and the like) used to set LANG or
>> LC_* IIRC.  However, they no longer do so
> 
> Actually, that is still done, in /etc/profile.d/lang.sh. It's for the
> benefit of other programs who think they know better than the system
> (emacs, I'm looking at you) by analyzing LC_* and LANG themselves
> instead of relying on setlocale().

Ahh, I did 'grep' not 'grep -r'.  Oops.

>> If /no/ relevant env vars are set
>> then
>>        if setlocale(LC_*, "") returns C.UTF-8
>>        # which we know is the /current/ cygwin default locale
>>        then
>>                query Win32 API for "real" default locale
>>        else
>>                use what setlocale returns
>> else
>>        use the env var value; don't ignore 'C.UTF-8'
>>        # if I have explicitly set LANG=C.UTF-8 then I must really
>>        # really want the "C" locale, not en_US or de.
> 
> Please, no. As Corinna said: "Do NOT call Windows functions in Cygwin
> libraries, unless the lib is doing something very special which isn't
> provided by POSIX functions.  Only call POSIX functions.  Don't mix
> the Cygwin and the Windows environment.  Please leave the interfacing
> to the underlying OS the sole job of Cygwin."

"In the interim".  That is, until cygwin itself is prepared to DTRT,
allow libintl to pick up the slack.  That's the only way I can see to
satisfy both the mandates of cygwin (hide win32 from posix apps) and
libintl (provide the most natural i18n experience to all users
regardless of platform or language).

It boils down to the following: cygwin made a decision -- incorrectly
IMO -- to explicitly set the default language/locale to "C", rather than
allowing the system defaults, as established via the Regional Settings,
to control i18n behavior //when the user has not specifically requested
something else, for cygwin, via LANG=*//

Now, this might make sense in that it would allow old cygwin
English-only installations to *keep* being english-only, without
positive action from existing users.  But, I doubt German-speakers, with
Regional Settings set so that all pure-win32 progs speak German, would
be hampered if cygwin suddenly started speaking German, too.  Surprised,
maybe, but not hampered.

Ideally, cygwin will revisit this decision and add functionality to
"translate" the win32 regional settings into appropriate setlocale(LC_*,
"") behavior.

But...*until that time*...it makes sense as a temporary measure to allow
libintl to step in.

> The 'C.UTF-8' default locale is not a bug, it was a deliberate design decision.

If you have to hardcode a specific locale, then picking "C" -- and
forcing the UTF-8 charset -- is a good idea.  But...we don't *have* to
pick a single hardcoded locale.

> Of course a good case can be made for picking up the Windows language
> in case none of LC_* and LANG are set, but there are also obvious
> arguments against: translations are usually patchy, i.e. you end up
> with a mix of English and translations of varying quality, which a lot
> of people hate.

So...because of bugs in the the translations of program X, Y, and Z, we
should default to a "foreign" language (English) rather than expose
those poor translations, or fix them?  Isn't this just papering over the
problem?

What do the linux distributions do?  Ignore language settings chosen in
the Installer GUI, and force Chinese users to use English?  Somehow I
doubt that.

> Futhermore, Cygwin is mostly a command line environment, with commands
> and options having English names, its userbase is probably even more
> geeky than that of your average Linux distribution, and Cygwin's
> homepage and setup program are English-only anyway. Hence it doesn't
> seem likely that defaulting to English is keeping a lot of users away.

I agree, but that doesn't mean we shouldn't be sensitive to i18n
concerns.  If somebody has set Windows' Regional Settings to specify
German, then...they really ought to get German everywhere possible.  If
the apps shipped with cygwin have such bad translations that it becomes
annoying, then they can *explicitly* set LANG= to force English.

(Right now, libintl deliberately ignores LANG if it is set to "C.UTF-8".
 That's too aggressive, and needs to change; libintl should always
respect the $ENVVARS).  If cygwin continues to, by default, force an
explicit LANG= setting (e.g. /etc/profile.d/lang.sh's:
test -z "${LC_ALL:-${LC_CTYPE:-$LANG}}" && export LANG=C.UTF-8

...well, that's cygwin's choice, and libintl shouldn't try to override
/THAT/ by deliberately ignoring LANG/LC_* when they happen to be set to
the magic string "C.UTF-8".

> Nevertheless, I'm on the fence on whether the default should be
> changed, but I certainly agree with Eric in that any such change
> should be implemented in the Cygwin DLL rather than in particular
> packages.

I agree -- *eventually*.  The question is, what to do *until then*.
It's obvious the current libintl behavior needs to change. Right now, it
is too aggressive (it explicitly ignores LANG=C.UTF-8); with Bruno's
proposed change it is even MORE aggressive: it explicitly ignores
{LANG,LC_*}=C.UTF-8).

My suggestion would continue previous cygwin behavior (that is,
cygwin+libintl 0.17), *unless* we also change base-files
/etc/profile.d/lang.{sh,csh}.  I would advocate that we DO change
lang.{sh,csh}, but that's a different argument.

If cygwin's default-locale behavior is improved, at that point
/etc/profile.d/lang.{sh,csh} *must* change -- probably to something like:

test -z "${LC_ALL:-${LC_CTYPE:-$LANG}}" && export LANG=$(locale -u -U)

which IIRC does exactly what libintl does now: queries the Windows
Regional Settings. Anyway, at that time we can also further change
libintl -- that is, make libintl pure posix again.

But holding libintl hostage to cygwin misfeatures is not helpful.

--
Chuck

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: cygwin started speaking German today

[Andy Koppe]

On 9 September 2011 16:13, Charles Wilson wrote:
> On 9/9/2011 8:33 AM, Andy Koppe wrote:
>> On 9 September 2011 10:18, Charles Wilson wrote:
>>> #2, some older versions of the base-files startup
>>> scripts (/etc/profile, /etc/skel/.*, and the like) used to set LANG or
>>> LC_* IIRC.  However, they no longer do so
>>
>> Actually, that is still done, in /etc/profile.d/lang.sh. It's for the
>> benefit of other programs who think they know better than the system
>> (emacs, I'm looking at you) by analyzing LC_* and LANG themselves
>> instead of relying on setlocale().
>
> Ahh, I did 'grep' not 'grep -r'.  Oops.
>
>>> If /no/ relevant env vars are set
>>> then
>>>        if setlocale(LC_*, "") returns C.UTF-8
>>>        # which we know is the /current/ cygwin default locale
>>>        then
>>>                query Win32 API for "real" default locale
>>>        else
>>>                use what setlocale returns
>>> else
>>>        use the env var value; don't ignore 'C.UTF-8'
>>>        # if I have explicitly set LANG=C.UTF-8 then I must really
>>>        # really want the "C" locale, not en_US or de.
>>
>> Please, no. As Corinna said: "Do NOT call Windows functions in Cygwin
>> libraries, unless the lib is doing something very special which isn't
>> provided by POSIX functions.  Only call POSIX functions.  Don't mix
>> the Cygwin and the Windows environment.  Please leave the interfacing
>> to the underlying OS the sole job of Cygwin."
>
> "In the interim".  That is, until cygwin itself is prepared to DTRT,
> allow libintl to pick up the slack.

Thereby pre-empting a design decision that is duly Cygwin's. Again:
"Do NOT call Windows functions in Cygwin libraries, unless the lib is
doing something very special which isn't provided by POSIX functions.
Only call POSIX functions.  Don't mix the Cygwin and the
Windows environment.  Please leave the interfacing to the underlying
OS the sole job of Cygwin."

Besides, your proposal wouldn't work, due to the LANG=C.UTF-8 setting
in /etc/profile.d/lang.sh. And if lang.sh was changed to pick up the
Windows setting, libintl wouldn't need to muck about with Windows
APIs.

>  That's the only way I can see to
> satisfy both the mandates of cygwin (hide win32 from posix apps) and
> libintl (provide the most natural i18n experience to all users
> regardless of platform or language).

"Libintl is a library that provides native language support to
programs." It doesn that perfectly well without going against the
wishes of the platform maintainers. Users can choose a language by
setting LANG or LC_*, as documented in the Cygwin manual. In the
probably-soon-to-be-default "Cygwin Terminal", they can even choose
the locale in the options dialog.

> It boils down to the following: cygwin made a decision -- incorrectly
> IMO -- to explicitly set the default language/locale to "C", rather than
> allowing the system defaults, as established via the Regional Settings,
> to control i18n behavior //when the user has not specifically requested
> something else, for cygwin, via LANG=*//

That's certainly a valid opinion. Please take it to cygwin-developers
and try to convince Corinna to change that decision.

> Now, this might make sense in that it would allow old cygwin
> English-only installations to *keep* being english-only, without
> positive action from existing users.  But, I doubt German-speakers, with
> Regional Settings set so that all pure-win32 progs speak German, would
> be hampered if cygwin suddenly started speaking German, too.  Surprised,
> maybe, but not hampered.

True. Nevertheless there'll be complaints, and I can't remember any
demands for this. Then again, complaints would be easy enough to deal
with.

> Ideally, cygwin will revisit this decision and add functionality to
> "translate" the win32 regional settings into appropriate setlocale(LC_*,
> "") behavior.
>
> But...*until that time*...it makes sense as a temporary measure to allow
> libintl to step in.
>
>> The 'C.UTF-8' default locale is not a bug, it was a deliberate design decision.
>
> If you have to hardcode a specific locale, then picking "C" -- and
> forcing the UTF-8 charset -- is a good idea.  But...we don't *have* to
> pick a single hardcoded locale.
>
>> Of course a good case can be made for picking up the Windows language
>> in case none of LC_* and LANG are set, but there are also obvious
>> arguments against: translations are usually patchy, i.e. you end up
>> with a mix of English and translations of varying quality, which a lot
>> of people hate.
>
> So...because of bugs in the the translations of program X, Y, and Z, we
> should default to a "foreign" language (English) rather than expose
> those poor translations, or fix them?  Isn't this just papering over the
> problem?

Maybe, but that's not Cygwin's problem to solve.

> What do the linux distributions do?  Ignore language settings chosen in
> the Installer GUI, and force Chinese users to use English?  Somehow I
> doubt that.

They do indeed usually default to the local language. This prompts
questions like these from some users:

http://askubuntu.com/questions/11601/command-line-tools-in-one-language-system-in-another
http://superuser.com/questions/235764/ubuntu-errors-in-terminal-how-to-show-them-in-english-language

> holding libintl hostage to cygwin misfeatures is not helpful.

And that's not a helpful statement.

Andy

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: cygwin started speaking German today

[Corinna Vinschen]

[from vacation]

On Sep  8 12:46, Bruno Haible wrote:
> After Cygwin 1.7 added working locales and defined LANG=C.UTF-8 for all users,
> libintl could be extended to respect the choices the user has made in the
> system control panels.

That's the wrong approach.  As I wrote in an earlier message with
respect to libintl, Cygwin is not Windows but a POSIX system in the
first place.  Do NOT call Windows functions in Cygwin libraries, unless
the lib is doing something very special which isn't provided by POSIX
functions.  Only call POSIX functions.  Don't mix the Cygwin and the
Windows environment.  Please leave the interfacing to the underlying OS
the sole job of Cygwin.  This includes how the internationalization
environment is handled.  Basically, if you add a #ifdef __CYGWIN__
to your code to call Windows functions, don't.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Project Co-Leader          cygwin AT cygwin DOT com
Red Hat

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: cygwin started speaking German today

[Corinna Vinschen]

[from vacation]

On Sep  8 12:46, Bruno Haible wrote:
> After Cygwin 1.7 added working locales and defined LANG=C.UTF-8 for all users,
> libintl could be extended to respect the choices the user has made in the
> system control panels.

That's the wrong approach.  As I wrote in an earlier message with
respect to libintl, Cygwin is not Windows but a POSIX system in the
first place.  Do NOT call Windows functions in Cygwin libraries, unless
the lib is doing something very special which isn't provided by POSIX
functions.  Only call POSIX functions.  Don't mix the Cygwin and the
Windows environment.  Please leave the interfacing to the underlying OS
the sole job of Cygwin.  This includes how the internationalization
environment is handled.  Basically, if you add a #ifdef __CYGWIN__
to your code to call Windows functions, don't.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Project Co-Leader          cygwin AT cygwin DOT com
Red Hat

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: cygwin started speaking German today

[Bruno Haible]

Hello Corinna,

Corinna Vinschen wrote:
> > After Cygwin 1.7 added working locales and defined LANG=C.UTF-8 for all users,
> > libintl could be extended to respect the choices the user has made in the
> > system control panels.
> 
> That's the wrong approach.

Before discussing the technical details, let me remind the goal.

The goal of GNU gettext is to enable localization of programs on all
systems that support that. The benefit for the user is that programs feel
more "friendly". The benefit for the Free Software community is that more
users can use our programs and can contribute to their development, even if
they don't speak English.

Cygwin 1.7 has made major steps towards this goal,
  - by adding working locales,
  - by implementing various internationalization related API in cygwin.dll,
  - by choosing UTF-8 as the default encoding for locales, with automatic
    charset conversion happening in the connection to the console window.

The last cornerstone that is missing is that a user needs to do nothing to
enable the localization of messages to his/her language. It should be
automatic. Gettext 0.18.1.1 fills this gap.

> Cygwin is not Windows but a POSIX system in the first place.

POSIX explicitly allows and foresees such localization without a specific
action of the user. Quoting POSIX:2008
<http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap07.html>:

     "All implementations shall define a locale as the default locale, to be
      invoked when no environment variables are set, or set to the empty
      string.  This default locale can be the POSIX locale or any other
      implementation-defined locale.  Some implementations may provide
      facilities for local installation administrators to set the default
      locale, customizing it for each location.  POSIX:2008 does not require
      such a facility."

This "facility" for "local installation administrators" is the
"Regional Settings" control panel, in Windows (as well as in MacOS X).

So, what we need is that when the user has set his/her regional settings
to "German", and has not set any environment variables, then a program
that does

      setlocale (LC_ALL, "");

will pick up the German locale. In Cygwin 1.7 it is called "de_DE.UTF-8".

There are three ways to achieve this behaviour:

  a) The system can set environment variables that reflect the regional
     settings. For example, if the user has chosen German, Cygwin's
     login process could set LANG to de_DE.UTF-8.

     This approach is used in Linux desktops like KDE.

  b) The system's setlocale() function can, when the second argument is
     the empty string and the respective environment variables don't
     specify anything, fetch the value from the "Regional settings"
     panel.

     Cygwin could do that.

  c) Programs can call libintl_setlocale(), and libintl_setlocale can,
     when the second argument is the empty string and the respective
     environment variables don't specify anything, fetch the value
     from the "Regional settings" panel.

     This is what's implemented in gettext 0.18.1.1.

> Do NOT call Windows functions in Cygwin libraries, unless
> the lib is doing something very special which isn't provided by POSIX
> functions.  Only call POSIX functions.  Don't mix the Cygwin and the
> Windows environment.  Please leave the interfacing to the underlying OS
> the sole job of Cygwin.

OK, then the following four facilities are needed in Cygwin.

1) We need the name of the locale which is in effect when the user has
   not specified environment variables.

   Either through option a) above. Programs can then do getenv ("LANG").
   Cygwin documentation <http://www.cygwin.com/cygwin-ug-net/setup-locale.html>
   currently says "The default locale in the absence of the aforementioned
   locale environment variables is "C.UTF-8"." This would have to change.

   Or through option b) above. Programs can then peek at the return
   value of  setlocale (LC_ALL, "").

   Or through an API function that calls GetUserDefaultLCID() and
   converts that to a glibc style locale name (e.g. "zh_CN.UTF-8")
   or to an RFC 3066 style locale name (e.g. "zh-Hans").

2) We need the name of the locale of the current thread.

   Either through a function newlocale(), as in POSIX.

   Or through an API function that calls GetThreadLocale() and
   converts that to a glibc style locale name (e.g. "zh_CN.UTF-8")
   or to an RFC 3066 style locale name (e.g. "zh-Hans").

   Locale per thread is mainly needed for web application servers,
   not for GUI programs.

3) Gettext needs the priority list of languages, if the "Regional Settings"
   panel can specify it. MacOS X has this setting customizable, I don't know
   whether newer Windows versions have it has well.

4) Programs that do number or date/time formatting will need to access the
   values that the user has specified. E.g. those set in
   <http://www.sisulizer.de/_img/codepage-problems/codepage-regional.jpg>
   <http://pc-error-free.com/blog/wp-content/uploads/2008/12/regional-settings.gif>
   <http://www.sisulizer.de/_img/codepage-problems/w7-regions-and-languages-formats.jpg>

I believe all of these are available through Win32 or MUI API calls,
and decently internationalized programs will need this.

Bruno
-- 
In memoriam Elisabeth von Thadden <http://en.wikipedia.org/wiki/Elisabeth_von_Thadden>

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: cygwin started speaking German today

[Corinna Vinschen]

Hi Bruno,

On Sep  8 22:56, Bruno Haible wrote:
> > Do NOT call Windows functions in Cygwin libraries, unless
> > the lib is doing something very special which isn't provided by POSIX
> > functions.  Only call POSIX functions.  Don't mix the Cygwin and the
> > Windows environment.  Please leave the interfacing to the underlying OS
> > the sole job of Cygwin.
> 
> OK, then the following four facilities are needed in Cygwin.
> 
> 1) We need the name of the locale which is in effect when the user has
>    not specified environment variables.

In Fedora, for instance, the fallback is what is set as system default
in /etc/sysconfig/i18n.

In Cygwin the fallback is the system default set in /etc/profile.d/lang.sh
or /etc/profile.d/lang.csh. 

Why should libintl use anything else on Cygwin, but not on Linux?

If the user wants the same locale in Cygwin as in the Win32 environment,
the user can just use the `locale -u' or `locale -s' command.  That's
why it has been added.  It is not at all the task of libintl to
override the underlying OS, and in the case of Cygwin, the underlying
OS is Cygwin, not Windows.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Project Co-Leader          cygwin AT cygwin DOT com
Red Hat

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: cygwin started speaking German today

[Andy Koppe]

On 9 September 2011 16:09, Corinna Vinschen wrote:
> Hi Bruno,
>
> On Sep  8 22:56, Bruno Haible wrote:
>> > Do NOT call Windows functions in Cygwin libraries, unless
>> > the lib is doing something very special which isn't provided by POSIX
>> > functions.  Only call POSIX functions.  Don't mix the Cygwin and the
>> > Windows environment.  Please leave the interfacing to the underlying OS
>> > the sole job of Cygwin.
>>
>> OK, then the following four facilities are needed in Cygwin.
>>
>> 1) We need the name of the locale which is in effect when the user has
>>    not specified environment variables.
>
> In Fedora, for instance, the fallback is what is set as system default
> in /etc/sysconfig/i18n.
>
> In Cygwin the fallback is the system default set in /etc/profile.d/lang.sh
> or /etc/profile.d/lang.csh.

Those set LANG. There's also the hardcoded fallback to "C.UTF-8" in
setlocale() in case none of LC_ALL, LC_FOO, and LANG is set. This is
Cygwin's system default locale as per POSIX Base defs section 7.2
("All implementations shall define a locale as the default locale, to
be invoked when no environment variables are set, or set to the empty
string. This default locale can be the POSIX locale or any other
implementation-defined locale.")

Andy

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: cygwin started speaking German today

[Thorsten Kampe]

* Corinna Vinschen (Fri, 9 Sep 2011 17:09:04 +0200)
> It is not at all the task of libintl to override the underlying OS,
> and in the case of Cygwin, the underlying OS is Cygwin, not Windows.

Pardon me?
"Cygwin is: 
a collection of tools which provide a Linux look and feel environment
for Windows.

a DLL (cygwin1.dll) which acts as a Linux API layer providing
substantial Linux API functionality."

Cygwin does not have any user account management, no file system, no 
file system permissions, etc. So when did Cygwin become an operating 
system in an operating system?

Thorsten


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: cygwin started speaking German today

[Christopher Faylor]

On Sat, Sep 10, 2011 at 01:44:44PM +0200, Thorsten Kampe wrote:
>* Corinna Vinschen (Fri, 9 Sep 2011 17:09:04 +0200)
>> It is not at all the task of libintl to override the underlying OS,
>> and in the case of Cygwin, the underlying OS is Cygwin, not Windows.
>
>Pardon me?
>"Cygwin is: 
>a collection of tools which provide a Linux look and feel environment
>for Windows.
>
>a DLL (cygwin1.dll) which acts as a Linux API layer providing
>substantial Linux API functionality."
>
>Cygwin does not have any user account management, no file system, no 
>file system permissions, etc. So when did Cygwin become an operating 
>system in an operating system?

Corinna, and most of the rest of us, know full well that Cygwin is not a
real OS but that was obviously not her point.  I believe that Corinna's
point was that Cygwin distributed packages should not normally call
Windows functions.

The project's goal is to provide something that looks like UNIX to the
end-user.  That includes providing functions like open() which provide
the feel of a UNIX filesystem and functions like getpwnam() which allow
you to query user accounts.  Cygwin programs are supposed to use open()
to read files and getpw*() functions to access account information.
Ditto for rest of the UNIX api.

Yes, open() is not accessing a real-honest-to-god Cygwin filesystem and
getpwnam() is only accessing a copy of Windows account information but
WE ALL KNOW THAT.  Cygwin is emulating an OS.  That is a given.  I
assume that Corinna assumed that no one would be so pedantic as to send
email to hundreds of people to pick nits about her use of the term "OS"
in this context.  You really don't seem to have a point beyond being
disagreeably picky.

cgf

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: cygwin started speaking German today

[Thorsten Kampe]

* Christopher Faylor (Sat, 10 Sep 2011 09:49:50 -0400)
> On Sat, Sep 10, 2011 at 01:44:44PM +0200, Thorsten Kampe wrote:
> > Corinna Vinschen (Fri, 9 Sep 2011 17:09:04 +0200)
> >> It is not at all the task of libintl to override the underlying OS,
> >> and in the case of Cygwin, the underlying OS is Cygwin, not
> >> Windows.
> >
> >Pardon me?
> >"Cygwin is: 
> >a collection of tools which provide a Linux look and feel environment
> >for Windows.
> >
> >a DLL (cygwin1.dll) which acts as a Linux API layer providing
> >substantial Linux API functionality."
> >
> >Cygwin does not have any user account management, no file system, no 
> >file system permissions, etc. So when did Cygwin become an operating 
> >system in an operating system?
> 
> Corinna, and most of the rest of us, know full well that Cygwin is not
> a real OS but that was obviously not her point. I believe that
> Corinna's point was that Cygwin distributed packages should not
> normally call Windows functions.

I think everyone agrees on that. The problem is that Cygwin is 
overriding the user's localization choice done in Windows. Corinna says 
users can align their shell locale with the Windows if they want to. 
Bruno says users shouldn't have to align anything but get this behaviour 
by default (and I agree).
 
> Cygwin is emulating an OS. That is a given. I assume that Corinna
> assumed that no one would be so pedantic as to send email to hundreds
> of people to pick nits about her use of the term "OS" in this context.
> You really don't seem to have a point beyond being disagreeably picky.

My point is that Cygwin should not override Windows. Cygwin is less than 
an OS and it's more than a simple application. Nevertheless it should 
fit in the overall Windows environment and not "contradict" it.

To paraphrase Corinna: "It is not the task of Cygwin to override the 
underlying OS, and in the case of Cygwin, the underlying OS is Windows, 
not Cygwin".

Or to quote myself: "Users who chose to have a specific language 
environment most likely want to have this choice for all their 
applications."

Thorsten


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: cygwin started speaking German today

[Christopher Faylor]

On Sat, Sep 10, 2011 at 06:10:15PM +0200, Thorsten Kampe wrote:
>* Christopher Faylor (Sat, 10 Sep 2011 09:49:50 -0400)
>> On Sat, Sep 10, 2011 at 01:44:44PM +0200, Thorsten Kampe wrote:
>> > Corinna Vinschen (Fri, 9 Sep 2011 17:09:04 +0200)
>> >> It is not at all the task of libintl to override the underlying OS,
>> >> and in the case of Cygwin, the underlying OS is Cygwin, not
>> >> Windows.
>> >
>> >Pardon me?
>> >"Cygwin is: 
>> >a collection of tools which provide a Linux look and feel environment
>> >for Windows.
>> >
>> >a DLL (cygwin1.dll) which acts as a Linux API layer providing
>> >substantial Linux API functionality."
>> >
>> >Cygwin does not have any user account management, no file system, no 
>> >file system permissions, etc. So when did Cygwin become an operating 
>> >system in an operating system?
>> 
>> Corinna, and most of the rest of us, know full well that Cygwin is not
>> a real OS but that was obviously not her point. I believe that
>> Corinna's point was that Cygwin distributed packages should not
>> normally call Windows functions.
>
>I think everyone agrees on that. The problem is that Cygwin is 
>overriding the user's localization choice done in Windows. Corinna says 
>users can align their shell locale with the Windows if they want to. 
>Bruno says users shouldn't have to align anything but get this behaviour 
>by default (and I agree).

So, stick to that point and don't quibble about a perfectly
understandable choice of words.

>>Cygwin is emulating an OS.  That is a given.  I assume that Corinna
>>assumed that no one would be so pedantic as to send email to hundreds
>>of people to pick nits about her use of the term "OS" in this context.
>>You really don't seem to have a point beyond being disagreeably picky.
>
>My point is that Cygwin should not override Windows.  Cygwin is less
>than an OS and it's more than a simple application.  Nevertheless it
>should fit in the overall Windows environment and not "contradict" it.

Even if we take your point at face value it doesn't mean that Cygwin
programs will now call Windows functions to determine their locale.
Cygwin is still the "OS" in this case and programs should not be
second-guessing it by winking and saying "I know that Windows is
underneath so I'll call a Windows function to get what I need".

It's always possible that this is a misfeature of Cygwin.  If that is
the case then Cygwin can be fixed.  Having a package maintainer
(inadvertently) establish new behavior by making a new package available
which introduces new behavior in programs which use it while leaving the
old behavior for programs that don't is not the way to fix it.

This hardly seems like an urgent issue given that we don't seem to have
had many complaints about the current behavior and especially given,
IIRC, this behavior was discussed at great length when it first went in.
I'd like to suggest that we revert to the previous behavior while we
discuss the best way to handle this.  It's not fair to users to have the
rug pulled out from under them by unannounced changes.

>To paraphrase Corinna: "It is not the task of Cygwin to override the
>underlying OS, and in the case of Cygwin, the underlying OS is Windows,
>not Cygwin".

I doubt that Corinna said anything like that or at least not in this
context.  Cygwin overrides Windows all the time.  That is exactly what
it does.

>Or to quote myself: "Users who chose to have a specific language 
>environment most likely want to have this choice for all their 
>applications."

The bug report which started this was from a "user who chose to have a
specific language environment" but found that he couldn't do what he
wanted.

cgf

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple