UTF-8 compatibility between Windows and Cygwin

UTF-8 compatibility between Windows and Cygwin

[Nellis, Kenneth]

I have (BOM-less) UTF-8 text files that I can read fine in 
Cygwin, but not Windows. When I create text files in Windows 
containing non-ASCII characters, I cannot read them in 
Cygwin. I understand why, but wondering the best way to be 
able to share text files across the two environments. 

I'm pretty sure that I want to keep my Cygwin LANG=C.UTF-8 
setting, but wondering what I can do on the Windows side for 
compatibility and what are the side effects. Currently 
Windows's Command Prompt command chcp shows "Active code 
page: 437". (Is that obsolete or even relevant?)

One solution seems to be to put a BOM in each UTF-8 text 
file; then the files read fine in both environments, but 
that's not conveniently accomplished.

--Ken Nellis

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: UTF-8 compatibility between Windows and Cygwin

[Andrey Repin]

Greetings, Nellis, Kenneth!

> I have (BOM-less) UTF-8 text files that I can read fine in 
> Cygwin, but not Windows. When I create text files in Windows 
> containing non-ASCII characters, I cannot read them in 
> Cygwin. I understand why, but wondering the best way to be 
> able to share text files across the two environments. 

Please provide `od -t x1z` of the file you are referring to.

> I'm pretty sure that I want to keep my Cygwin LANG=C.UTF-8 
> setting,

Better use real language indication, and configure LC_ according to
preferences.

> but wondering what I can do on the Windows side for
> compatibility and what are the side effects.

On Windows side, you could actually use UTF-8.
Without seeing the requested dump, I can only guess, though.

> Currently
> Windows's Command Prompt command chcp shows "Active code 
> page: 437". (Is that obsolete or even relevant?)

Neither of the two, unless you consider relevance to the correct display of
the results and other console-related operations.
If you're often find yourself in a native console, I suggest you tweak LANG to
match (i.e. LANG=ru_RU.CP866 for me) and only change it to .UTF-8 for MinTTY
(it can help you with override, if you don't want to tweak your startup files).

> One solution seems to be to put a BOM in each UTF-8 text 
> file; then the files read fine in both environments, but 
> that's not conveniently accomplished.

That's hardly a solution, though. More like a workaround.


-- 
With best regards,
Andrey Repin
Thursday, May 25, 2017 18:48:43

Sorry for my terrible english...


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple