[PATCH] ccwrap: fix build with non-english locale set

[PATCH] ccwrap: fix build with non-english locale set

[Patrick Bendorf]

/winsup/
* ccwrap: fix build with non-english locale set
---
  winsup/ccwrap | 1 +
  1 file changed, 1 insertion(+)

diff --git a/winsup/ccwrap b/winsup/ccwrap
index 7580e7a..ef83085 100755
--- a/winsup/ccwrap
+++ b/winsup/ccwrap
@@ -12,6 +12,7 @@ if ($ARGV[0] ne '++') {
      $cxx = 1;
  }
  die "$0: $ccorcxx environment variable does not exist\n" unless exists 
$ENV{$ccorcxx};
+$ENV{'LANG'} = 'C.UTF-8';
  my @compiler = split ' ', $ENV{$ccorcxx};
  if ("@ARGV" !~ / -nostdinc/o) {
      my $fd;
--
2.7.0

Re: [PATCH] ccwrap: fix build with non-english locale set

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1184 bytes --]

Hi Patrick,

On Feb 29 08:12, Patrick Bendorf wrote:
> /winsup/
> * ccwrap: fix build with non-english locale set

First of all, why fix it?  Without at least a short explanation what you
observe without this patch, this change seems arbitrary.

> ---
>  winsup/ccwrap | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/winsup/ccwrap b/winsup/ccwrap
> index 7580e7a..ef83085 100755
> --- a/winsup/ccwrap
> +++ b/winsup/ccwrap
> @@ -12,6 +12,7 @@ if ($ARGV[0] ne '++') {
>      $cxx = 1;
>  }
>  die "$0: $ccorcxx environment variable does not exist\n" unless exists
> $ENV{$ccorcxx};
> +$ENV{'LANG'} = 'C.UTF-8';
>  my @compiler = split ' ', $ENV{$ccorcxx};
>  if ("@ARGV" !~ / -nostdinc/o) {
>      my $fd;
> --
> 2.7.0

That won't work nicely for non-Cygwin build systems.  When cross
building Cygwin on, e.g., Linux, "C.UTF-8" is an unrecognized locale.
Ideally the above would test for the current build system being Cygwin
and use "C.UTF-8" on Cygwin, "C" otherwise.


Thanks,
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [PATCH] ccwrap: fix build with non-english locale set

[mail]

Hi Corinna,

Am 2016-02-29 11:33, schrieb Corinna Vinschen:
> Hi Patrick,
> 
> On Feb 29 08:12, Patrick Bendorf wrote:
>> /winsup/
>> * ccwrap: fix build with non-english locale set
> 
> First of all, why fix it?  Without at least a short explanation what 
> you
> observe without this patch, this change seems arbitrary.
> 
short explanation: after setting up cygwin on my systems the default 
locale is set to "de_DE.UTF-8". this leads to ccwrap not picking up 
certain "-isystem" arguments, which in turn leads to "stddef.h: no such 
file or directory". this breaks the build process for systems having non 
english locale.

consider the following two pastebins from a system with an english 
locale and mine. a whole bunch of "-isystem" can not be found on my 
system using german locale.
http://pastebin.com/ip5L7dZY
http://pastebin.com/wZBc2cqr

ccwrap scans the output of the first compiler invocation (line 21) for 
some specific english output on and around line 43.
output of first invocation on german locale system:
http://pastebin.com/ZZzVGReh

>> ---
>>  winsup/ccwrap | 1 +
>>  1 file changed, 1 insertion(+)
>> 
>> diff --git a/winsup/ccwrap b/winsup/ccwrap
>> index 7580e7a..ef83085 100755
>> --- a/winsup/ccwrap
>> +++ b/winsup/ccwrap
>> @@ -12,6 +12,7 @@ if ($ARGV[0] ne '++') {
>>      $cxx = 1;
>>  }
>>  die "$0: $ccorcxx environment variable does not exist\n" unless 
>> exists
>> $ENV{$ccorcxx};
>> +$ENV{'LANG'} = 'C.UTF-8';
>>  my @compiler = split ' ', $ENV{$ccorcxx};
>>  if ("@ARGV" !~ / -nostdinc/o) {
>>      my $fd;
>> --
>> 2.7.0
> 
> That won't work nicely for non-Cygwin build systems.  When cross
> building Cygwin on, e.g., Linux, "C.UTF-8" is an unrecognized locale.
> Ideally the above would test for the current build system being Cygwin
> and use "C.UTF-8" on Cygwin, "C" otherwise.
> 
thanks for pointing that out.
i changed the patch to check uname -o for cygwin string and set the 
locale to either C or C.UTF-8

---
  winsup/ccwrap | 1 +
  1 file changed, 1 insertion(+)

diff --git a/winsup/ccwrap b/winsup/ccwrap
index 7580e7a..ef83085 100755
--- a/winsup/ccwrap
+++ b/winsup/ccwrap
@@ -12,6 +12,11 @@ if ($ARGV[0] ne '++') {
      $cxx = 1;
  }
  die "$0: $ccorcxx environment variable does not exist\n" unless exists 
$ENV{$ccorcxx};
+if (`uname -o` =~ /cygwin/i) {
+    $ENV{'LANG'} = 'C.UTF-8';
+} else {
+    $ENV{'LANG'} = 'C';
+}
  my @compiler = split ' ', $ENV{$ccorcxx};
  if ("@ARGV" !~ / -nostdinc/o) {
      my $fd;
--
2.7.0

> 
> Thanks,
> Corinna

Re: [PATCH] ccwrap: fix build with non-english locale set

[Jon Turney]

On 29/02/2016 12:19, mail@patrick-bendorf.de wrote:
> +if (`uname -o` =~ /cygwin/i) {
> +    $ENV{'LANG'} = 'C.UTF-8';
> +} else {
> +    $ENV{'LANG'} = 'C';

This can just say "$ENV{'LANG'} = 'C';" right? As that has to work 
everywhere?

Re: [PATCH] ccwrap: fix build with non-english locale set

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1635 bytes --]

On Feb 29 13:19, mail@patrick-bendorf.de wrote:
> Hi Corinna,
> 
> Am 2016-02-29 11:33, schrieb Corinna Vinschen:
> >Hi Patrick,
> >
> >On Feb 29 08:12, Patrick Bendorf wrote:
> >>/winsup/
> >>* ccwrap: fix build with non-english locale set
> >
> >First of all, why fix it?  Without at least a short explanation what you
> >observe without this patch, this change seems arbitrary.
> >
> short explanation: after setting up cygwin on my systems the default locale
> is set to "de_DE.UTF-8". this leads to ccwrap not picking up certain
> "-isystem" arguments, which in turn leads to "stddef.h: no such file or
> directory". this breaks the build process for systems having non english
> locale.
> 
> consider the following two pastebins from a system with an english locale
> and mine. a whole bunch of "-isystem" can not be found on my system using
> german locale.
> http://pastebin.com/ip5L7dZY
> http://pastebin.com/wZBc2cqr
> 
> ccwrap scans the output of the first compiler invocation (line 21) for some
> specific english output on and around line 43.
> output of first invocation on german locale system:
> http://pastebin.com/ZZzVGReh
> [...]
> >Ideally the above would test for the current build system being Cygwin
> >and use "C.UTF-8" on Cygwin, "C" otherwise.
> >
> thanks for pointing that out.
> i changed the patch to check uname -o for cygwin string and set the locale
> to either C or C.UTF-8

Patch applied.


Thanks,
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [PATCH] ccwrap: fix build with non-english locale set

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 518 bytes --]

On Feb 29 12:46, Jon Turney wrote:
> On 29/02/2016 12:19, mail@patrick-bendorf.de wrote:
> >+if (`uname -o` =~ /cygwin/i) {
> >+    $ENV{'LANG'} = 'C.UTF-8';
> >+} else {
> >+    $ENV{'LANG'} = 'C';
> 
> This can just say "$ENV{'LANG'} = 'C';" right? As that has to work
> everywhere?

Uh, I missed that.  Yes, of course that should work, too.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [PATCH] ccwrap: fix build with non-english locale set

[patrick bendorf]

after some discussion on irc and the list i'm resubmitting a simpler 
version of the patch.
setting the locale on cygwin to 'C.UTF-8' is not needed, so i'm always 
setting it to 'C' which is sufficient for the build process and the most 
simple fix.

/winsup/
* ccwrap: change locale to 'C' as ccwrap searches for literal strings 
"search starts here" and "End of search list" which may be localized.
---
  winsup/ccwrap | 1 +
  1 file changed, 1 insertion(+)

diff --git a/winsup/ccwrap b/winsup/ccwrap
index 7580e7a..0c6a170 100755
--- a/winsup/ccwrap
+++ b/winsup/ccwrap
@@ -12,6 +12,7 @@ if ($ARGV[0] ne '++') {
      $cxx = 1;
  }
  die "$0: $ccorcxx environment variable does not exist\n" unless exists 
$ENV{$ccorcxx};
+$ENV{'LANG'} = 'C';
  my @compiler = split ' ', $ENV{$ccorcxx};
  if ("@ARGV" !~ / -nostdinc/o) {
      my $fd;
--
2.7.0

patrick

Am 2016-02-29 13:58, schrieb Corinna Vinschen:
> On Feb 29 12:46, Jon Turney wrote:
>> On 29/02/2016 12:19, mail@patrick-bendorf.de wrote:
>> >+if (`uname -o` =~ /cygwin/i) {
>> >+    $ENV{'LANG'} = 'C.UTF-8';
>> >+} else {
>> >+    $ENV{'LANG'} = 'C';
>> 
>> This can just say "$ENV{'LANG'} = 'C';" right? As that has to work
>> everywhere?
> 
> Uh, I missed that.  Yes, of course that should work, too.
> 
> 
> Corinna

Re: [PATCH] ccwrap: fix build with non-english locale set

[Patrick Bendorf]

sorry, now it's based on current git head. and hopefully without 
formatting issues.

/winsup/
* ccwrap: change locale to 'C' as ccwrap searches for literal strings 
"search starts here" and "End of search list" which may be localized.

---
  winsup/ccwrap | 6 +-----
  1 file changed, 1 insertion(+), 5 deletions(-)

diff --git a/winsup/ccwrap b/winsup/ccwrap
index 2f1fd3a..0c6a170 100755
--- a/winsup/ccwrap
+++ b/winsup/ccwrap
@@ -12,11 +12,7 @@ if ($ARGV[0] ne '++') {
      $cxx = 1;
  }
  die "$0: $ccorcxx environment variable does not exist\n" unless exists 
$ENV{$ccorcxx};
-if (`uname -o` =~ /cygwin/i) {
-    $ENV{'LANG'} = 'C.UTF-8';
-} else {
-    $ENV{'LANG'} = 'C';
-}
+$ENV{'LANG'} = 'C';
  my @compiler = split ' ', $ENV{$ccorcxx};
  if ("@ARGV" !~ / -nostdinc/o) {
      my $fd;
--
2.7.0

Am 29.02.2016 um 14:19 schrieb patrick bendorf:
> after some discussion on irc and the list i'm resubmitting a simpler 
> version of the patch.
> setting the locale on cygwin to 'C.UTF-8' is not needed, so i'm always 
> setting it to 'C' which is sufficient for the build process and the 
> most simple fix.
>
> /winsup/
> * ccwrap: change locale to 'C' as ccwrap searches for literal strings 
> "search starts here" and "End of search list" which may be localized.
> ---
>  winsup/ccwrap | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/winsup/ccwrap b/winsup/ccwrap
> index 7580e7a..0c6a170 100755
> --- a/winsup/ccwrap
> +++ b/winsup/ccwrap
> @@ -12,6 +12,7 @@ if ($ARGV[0] ne '++') {
>      $cxx = 1;
>  }
>  die "$0: $ccorcxx environment variable does not exist\n" unless 
> exists $ENV{$ccorcxx};
> +$ENV{'LANG'} = 'C';
>  my @compiler = split ' ', $ENV{$ccorcxx};
>  if ("@ARGV" !~ / -nostdinc/o) {
>      my $fd;
> -- 
> 2.7.0
>
> patrick
>
> Am 2016-02-29 13:58, schrieb Corinna Vinschen:
>> On Feb 29 12:46, Jon Turney wrote:
>>> On 29/02/2016 12:19, mail@patrick-bendorf.de wrote:
>>> >+if (`uname -o` =~ /cygwin/i) {
>>> >+    $ENV{'LANG'} = 'C.UTF-8';
>>> >+} else {
>>> >+    $ENV{'LANG'} = 'C';
>>>
>>> This can just say "$ENV{'LANG'} = 'C';" right? As that has to work
>>> everywhere?
>>
>> Uh, I missed that.  Yes, of course that should work, too.
>>
>>
>> Corinna

Re: [PATCH] ccwrap: fix build with non-english locale set

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1120 bytes --]

On Feb 29 14:34, Patrick Bendorf wrote:
> sorry, now it's based on current git head. and hopefully without formatting
> issues.
> 
> /winsup/
> * ccwrap: change locale to 'C' as ccwrap searches for literal strings
> "search starts here" and "End of search list" which may be localized.
> 
> ---
>  winsup/ccwrap | 6 +-----
>  1 file changed, 1 insertion(+), 5 deletions(-)
> 
> diff --git a/winsup/ccwrap b/winsup/ccwrap
> index 2f1fd3a..0c6a170 100755
> --- a/winsup/ccwrap
> +++ b/winsup/ccwrap
> @@ -12,11 +12,7 @@ if ($ARGV[0] ne '++') {
>      $cxx = 1;
>  }
>  die "$0: $ccorcxx environment variable does not exist\n" unless exists
> $ENV{$ccorcxx};
> -if (`uname -o` =~ /cygwin/i) {
> -    $ENV{'LANG'} = 'C.UTF-8';
> -} else {
> -    $ENV{'LANG'} = 'C';
> -}
> +$ENV{'LANG'} = 'C';
>  my @compiler = split ' ', $ENV{$ccorcxx};
>  if ("@ARGV" !~ / -nostdinc/o) {
>      my $fd;
> --
> 2.7.0

Patch applied.


Thanks,
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [PATCH] ccwrap: fix build with non-english locale set

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1131 bytes --]

On 02/29/2016 06:19 AM, patrick bendorf wrote:
> after some discussion on irc and the list i'm resubmitting a simpler
> version of the patch.
> setting the locale on cygwin to 'C.UTF-8' is not needed, so i'm always
> setting it to 'C' which is sufficient for the build process and the most
> simple fix.
> 
> /winsup/
> * ccwrap: change locale to 'C' as ccwrap searches for literal strings
> "search starts here" and "End of search list" which may be localized.
> ---
>  winsup/ccwrap | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/winsup/ccwrap b/winsup/ccwrap
> index 7580e7a..0c6a170 100755
> --- a/winsup/ccwrap
> +++ b/winsup/ccwrap
> @@ -12,6 +12,7 @@ if ($ARGV[0] ne '++') {
>      $cxx = 1;
>  }
>  die "$0: $ccorcxx environment variable does not exist\n" unless exists
> $ENV{$ccorcxx};
> +$ENV{'LANG'} = 'C';

This won't work if I have LC_ALL set in my environment.  If you want to
force the locale, you want to set LC_ALL (highest priority), not LANG
(lowest priority).

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: [PATCH] ccwrap: fix build with non-english locale set

[Patrick Bendorf]

[-- Attachment #1: Type: text/plain, Size: 1221 bytes --]

thanks eric.
just changed and tested it.
hopefully the last patch for this matter.

@corinna: as attachment to overcome previous problems.

patrick

Am 29.02.2016 um 16:41 schrieb Eric Blake:
> On 02/29/2016 06:19 AM, patrick bendorf wrote:
>> after some discussion on irc and the list i'm resubmitting a simpler
>> version of the patch.
>> setting the locale on cygwin to 'C.UTF-8' is not needed, so i'm always
>> setting it to 'C' which is sufficient for the build process and the most
>> simple fix.
>>
>> /winsup/
>> * ccwrap: change locale to 'C' as ccwrap searches for literal strings
>> "search starts here" and "End of search list" which may be localized.
>> ---
>>   winsup/ccwrap | 1 +
>>   1 file changed, 1 insertion(+)
>>
>> diff --git a/winsup/ccwrap b/winsup/ccwrap
>> index 7580e7a..0c6a170 100755
>> --- a/winsup/ccwrap
>> +++ b/winsup/ccwrap
>> @@ -12,6 +12,7 @@ if ($ARGV[0] ne '++') {
>>       $cxx = 1;
>>   }
>>   die "$0: $ccorcxx environment variable does not exist\n" unless exists
>> $ENV{$ccorcxx};
>> +$ENV{'LANG'} = 'C';
> This won't work if I have LC_ALL set in my environment.  If you want to
> force the locale, you want to set LC_ALL (highest priority), not LANG
> (lowest priority).
>



[-- Attachment #2: 0001-winsup.patch --]
[-- Type: text/plain, Size: 733 bytes --]

From 826fd01518760279aec8c8e494a5d5d7fde28f0b Mon Sep 17 00:00:00 2001
From: patrick bendorf <mail@patrick-bendorf.de>
Date: Mon, 29 Feb 2016 17:21:20 +0100
Subject: [PATCH] /winsup/ * ccwrap: set LC_ALL instead of LANG to 'C' to force
 locale reliably

---
 winsup/ccwrap | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/winsup/ccwrap b/winsup/ccwrap
index 0c6a170..a7d2dd1 100755
--- a/winsup/ccwrap
+++ b/winsup/ccwrap
@@ -12,7 +12,7 @@ if ($ARGV[0] ne '++') {
     $cxx = 1;
 }
 die "$0: $ccorcxx environment variable does not exist\n" unless exists $ENV{$ccorcxx};
-$ENV{'LANG'} = 'C';
+$ENV{'LC_ALL'} = 'C';
 my @compiler = split ' ', $ENV{$ccorcxx};
 if ("@ARGV" !~ / -nostdinc/o) {
     my $fd;
-- 
2.7.0

Re: [PATCH] ccwrap: fix build with non-english locale set

[Jon Turney]

On 29/02/2016 16:56, Patrick Bendorf wrote:
> thanks eric.
> just changed and tested it.
> hopefully the last patch for this matter.
>
> @corinna: as attachment to overcome previous problems.

Unfortunately, this still isn't quite right, as it forces the 2nd 
invocation of the compiler to be with LC_ALL, so localized compiler 
error messages will not be shown for actual compilation problems.

So perhaps the setting of LC_ALL should be in the implicitly forked 
block after the open('-|') ?

Re: [PATCH] ccwrap: fix build with non-english locale set

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 796 bytes --]

Patrick,

On Feb 29 17:55, Jon Turney wrote:
> On 29/02/2016 16:56, Patrick Bendorf wrote:
> >thanks eric.
> >just changed and tested it.
> >hopefully the last patch for this matter.
> >
> >@corinna: as attachment to overcome previous problems.
> 
> Unfortunately, this still isn't quite right, as it forces the 2nd invocation
> of the compiler to be with LC_ALL, so localized compiler error messages will
> not be shown for actual compilation problems.
> 
> So perhaps the setting of LC_ALL should be in the implicitly forked block
> after the open('-|') ?

are you going to look into this by any chance?


Thanks,
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [PATCH] ccwrap: fix build with non-english locale set

[Patrick Bendorf]

hi corinna,

i might find some time this weekend. this last few days were just
too much to look into the issue.

patrick

Am 04.03.2016 um 09:56 schrieb Corinna Vinschen:
> Patrick,
>
> On Feb 29 17:55, Jon Turney wrote:
>> On 29/02/2016 16:56, Patrick Bendorf wrote:
>>> thanks eric.
>>> just changed and tested it.
>>> hopefully the last patch for this matter.
>>>
>>> @corinna: as attachment to overcome previous problems.
>> Unfortunately, this still isn't quite right, as it forces the 2nd invocation
>> of the compiler to be with LC_ALL, so localized compiler error messages will
>> not be shown for actual compilation problems.
>>
>> So perhaps the setting of LC_ALL should be in the implicitly forked block
>> after the open('-|') ?
> are you going to look into this by any chance?
>
>
> Thanks,
> Corinna
>

Re: [PATCH] ccwrap: fix build with non-english locale set

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1109 bytes --]

On Mar  4 10:22, Patrick Bendorf wrote:
> hi corinna,
> 
> i might find some time this weekend. this last few days were just
> too much to look into the issue.

Cool, thanks!


Corinna


> patrick
> 
> Am 04.03.2016 um 09:56 schrieb Corinna Vinschen:
> >Patrick,
> >
> >On Feb 29 17:55, Jon Turney wrote:
> >>On 29/02/2016 16:56, Patrick Bendorf wrote:
> >>>thanks eric.
> >>>just changed and tested it.
> >>>hopefully the last patch for this matter.
> >>>
> >>>@corinna: as attachment to overcome previous problems.
> >>Unfortunately, this still isn't quite right, as it forces the 2nd invocation
> >>of the compiler to be with LC_ALL, so localized compiler error messages will
> >>not be shown for actual compilation problems.
> >>
> >>So perhaps the setting of LC_ALL should be in the implicitly forked block
> >>after the open('-|') ?
> >are you going to look into this by any chance?
> >
> >
> >Thanks,
> >Corinna
> >

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]