windres RC codepage

windres RC codepage

[Vlada Macek]

Hi,

for some Win32 program I'd like to contribute the textual RC resource
file localized to Czech language. On my WinXPPro Czech version, windres
compiles the RC file correctly (to UCS-16 I presume) and resulting EXE
file is beaving correctly.

The program is maintained by a Swedish developer and when I send him my
localized RC, he is not able to compile correct version of RES file with
windres on his Windows. I tried to set this directive at the beginning
of the file:

    LANGUAGE LANG_CZECH, 0x04

But the result is the same. The UCS-16 encoding in his RES file is
crippled and when it is linked to the EXE, the program shows some of the
Czech characters incorrectly (on my Czech Windows).

I peeked on the windres source and I'm really not sure what causes the
different behavior between Czech and Swedish Windows. What is the least
painful way for us to get this to work? Is it possible that the codepage
transcoding routines in windres are missing or not completed for these
cases? I believed that the aforementioned directive was sufficient and
the windres should transcode characters based only on this information.
Is that true?

Or is it a matter of installing Czech "locale" from Microsoft on that
Swedish Windows? But the maintainer does not wish to install on his
computer every "locale" in which his program will be localized. It would
really be an overkill.

Thanks in advance,

Vlada Macek

Re: windres RC codepage

[Aaron W. LaFramboise]

Vlada Macek wrote:

> I peeked on the windres source and I'm really not sure what causes the
> different behavior between Czech and Swedish Windows. What is the least
> painful way for us to get this to work? Is it possible that the codepage
> transcoding routines in windres are missing or not completed for these
> cases? I believed that the aforementioned directive was sufficient and
> the windres should transcode characters based only on this information.
> Is that true?
> 
> Or is it a matter of installing Czech "locale" from Microsoft on that
> Swedish Windows? But the maintainer does not wish to install on his
> computer every "locale" in which his program will be localized. It would
> really be an overkill.

There might be somebody who knows something about this specifically, but
if there isn't, it would really help to solve the problem if you could
narrow down the actual behavior.  In particular, create a minimal
testcase, and verify one case windres compiles it correctly, and one
case where it doesn't.  I'd be willing to help with that if you need
someone to run a test program.  In addition, if possible, check that
against the behavior of Microsoft's RC and see if they have the same
problem.

This sort of thing should work, but I wouldn't be suprised if there are
problems, especially if wide characters are involved.

Aaron W. LaFramboise

Re: windres RC codepage

[Aaron W. LaFramboise]

Vlada Macek wrote:

> Each of his two results (2.13.90 and 2.15.91) is different, but none is
> ok, no UCS-2 as I expect.

Well, apparently windres does some sort of codepage conversion when
compiling RC input.  It isn't trivially obvious to me why this is, so
I'll need to investigate further.

This behavior is equivilent to Microsoft's RC, so windres is doing the
right thing.  However, Microsoft's RC has a /c option to set that
codepage to Czech, or whatever, and so fix this problem.  Unfortunately,
windres does not have a similar option.

> What should be the best solution?

Ideally, someone would add a similar option to windres.  It probably
would not be difficult to add, but I can't say for sure yet.

I don't understand at all what sort of transformation of encodings is
being done.  If I use windres to compile your Czech input file (with
whatever encoding is default for US English WinXP), then decompile it
back to a RC file, the text I start with is not the same text I finish
with.  This seems very odd:

Actual Czech string: Příliš žluťoučký kůň ďábelské ódy PŘÍLIŠ ŽLUŤOUČKÝ
KŮŇ ÚPĚL ĎÁBELSKÉ ÓDY
String in my native codepage: Pøíliš žluouèký kùò ïábelské ódy PØÍLIŠ
ŽLUOUÈKÝ KÙÒ ÚPÌL ÏÁBELSKÉ ÓDY
String after conversion: Pøíli\x161 \x17eluouèký kùò ïábelské ódy PØÍLI\x160
\x17dLUOUÈKÝ KÙÒ ÚPÌL ÏÁBELSKÉ ÓDY

Anyway, I'm going to look at this a little more, and see why this
happens.  In particular, I'm going to look at the sources, and look for
a description of the binary resource representation.  If anyone else
knows more about the encoding of resources strings, speak up! :)

Aaron W. LaFramboise

Re: windres RC codepage

[Ian Lance Taylor]

"Aaron W. LaFramboise" <aaron98wiridge9@aaronwl.com> writes:

> Well, apparently windres does some sort of codepage conversion when
> compiling RC input.  It isn't trivially obvious to me why this is, so
> I'll need to investigate further.

windres does nothing intelligent when reading Unicode strings out of
an RC file, mainly because I didn't bother thinking about the issue at
all.  See binutils/winduni.c.

Ian

Re: windres RC codepage

[Aaron W. LaFramboise]

Well, my TODO list is getting quite long, and I don't have time to keep
looking at this at the moment.  I will return to this later, but this
may not be soon.

The way to fix this is to add a /c option to windres that uses a
particular charset when converting resource strings, analogous to what
Microsoft's RC has.  I suppose the way to do this would be to use
Windows routines if native, else fall back to libiconv, else fall back
to some dummy behavior.

I've filed two bug reports:

http://sources.redhat.com/bugzilla/show_bug.cgi?id=576

http://sources.redhat.com/bugzilla/show_bug.cgi?id=577

Aaron W. LaFramboise

Re: windres RC codepage

[Vlada Macek]

Aaron W. LaFramboise wrote:

>The way to fix this is to add a /c option to windres that uses a
>particular charset when converting resource strings, analogous to what
>Microsoft's RC has.  I suppose the way to do this would be to use
>Windows routines if native, else fall back to libiconv, else fall back
>to some dummy behavior.
>  
>
The -c option to explicitly define input codepage would be good. 
Nevertheless I think it may suffice to get this information from the 
LANGUAGE directive. Parameter of this directive is LCID which AFAIK 
should correspond to  country and language used in the resource file. I 
found this relation table on the web. It may be incomplete:

    http://www.microsoft.com/OpenType/otspec/lcid-cp.txt
    http://www.microsoft.com/globaldev/nlsweb/default.asp

Maybe someone can get complete table. Maybe it's contained in the 
Microsoft resource compiler reference...

\//\/\

Re: windres RC codepage

[Vlada Macek]

[-- Attachment #1.1: Type: text/plain, Size: 2415 bytes --]

Hi,

it seems my message was not delivered to the list, because it carried
ZIP attachment. My thread-leading message was successful, but the one
after Aaron's first reply weren't. So the list subscribers didn't see on
which Aaron was replying today. I'm resending that previous message as
well as the attachment that Aaron tested.

Aaron W. LaFramboise wrote:

>> There might be somebody who knows something about this
>> specifically, but if there isn't, it would really help to solve the
>> problem if you could narrow down the actual behavior. In
>> particular, create a minimal testcase, and verify one case windres
>> compiles it correctly, and one case where it doesn't. I'd be
>> willing to help with that if you need someone to run a test
>> program. In addition, if possible, check that against the behavior
>> of Microsoft's RC and see if they have the same problem.
>>
>> This sort of thing should work, but I wouldn't be suprised if there
>> are problems, especially if wide characters are involved.

Thank you for the response, Aaron!

I'm attaching files where the problem is clear. The RC file is the
source. CzechWindows is RES (COFF) compiled by me -- it has good
encoding (Total Commander' File Viewer displays all czech accents
correctly in the Unicode mode). I use Windows XP Pro Czech Version under
VMWare.

I asked the aforementioned package maintainer in Sweden to compile the
RC as well. The two results is marked SwedishWindows. Each one with
different windres version (I asked him to upgrade to my newest windres
version to be comparable). He uses Windows XP SP2 English and has the
following languages installed: English (United States), Swedish.

Each of his two results (2.13.90 and 2.15.91) is different, but none is
ok, no UCS-2 as I expect.

What should be the best solution?


--- End of forwarded e-mail ---

Ian is telling that windres is not doing any codepage translation (which
I confirmed for myself some time ago in the source). Nevertheless under
Czech Windows (codepage 1250), the RC file in this encoding is correctly
written to COFF format in UCS-2 encoding! Even when on non-Czech Windows
in Sweden, with the Czech language support post-installed, windres will
not compile correct COFF. It seems strange to me.

May I ask someone to help shed some light on it?

Thanks in advance!

-- 

\//\/\
(Sometimes credited as 1494 F8DD 6379 4CD7 E7E3 1FC9 D750 4243 1F05 9424.)


[-- Attachment #1.2: windrestest-codepage1250.tar.gz --]
[-- Type: application/x-gzip, Size: 914 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 256 bytes --]