Re: Cygwin fails to utilize Unicode replacement character

[Brian Inglis <Brian.Inglis@SystematicSw.ab.ca>]

On 2018-09-03 16:15, Steven Penny wrote:
> On Mon, 3 Sep 2018 23:02:58, Corinna Vinschen wrote:
>> I can't. I only have a limited set of fonts available in the console.

Install dejavu-fonts package or just DejaVu Sans Mono font from:

	https://dejavu-fonts.github.io/Download.html
	http://sourceforge.net/projects/dejavu/files/dejavu/2.37/dejavu-fonts-ttf-2.37.tar.bz2

or see what glyph is at index 0 (.notdef)?

> http://superuser.com/questions/390933/add-font-cmd-window-choices/956818

For Windows support, from Explorer I just search all *.[ot]tf under
...CygRoot.../usr/share/fonts/ and copy into /Windows/Fonts/

>> What I just did was calling the GetFontUnicodeRanges function
>> for each font, and it turns out that none of the fonts support
>> 0xfffd "REPLACEMENT CHARACTER", but all three support 0xfffc
>> "OBJECT REPLACEMENT CHARACTER".  I expanded the testcase to check
>> for this with GetGlyphIndicesW and, lo and behold, the result
>> makes sense.

>> On the other hand, during testing I saw a 0xfffd character printed for
>> these fonts.  None of them actually supports 0xfffd, so apparently the
>> Windows console already uses replacement fonts if possible.
>> I guess I just stop here and always print 0xfffd.  I seriously doubt
>> it makes sense to add so much code just to print a single char in a
>> border case.

> this is not possible; most likely you were seeing the ".notdef glyph":
> http://docs.microsoft.com/typography/opentype/spec/recom
> for Consolas which is simlar in appearance to U+FFFD REPLACEMENT CHARACTER. The
> differnce is that if you copy the ".notdef glyph" and paste it into "Notepad" or
> similar, it will paste the proper character that couldnt be seen in the console,
> while pasting U+FFFD into "Notepad" will just paste itself.
> Expanding on the "Notepad" example, "Notepad" default font is "Lucida Console",
> which doesnt have U+FFFD either. However pasting into "Notepad" will still show
> U+FFFD properly because "Tahoma" has U+FFFD and "Notepad" can utilize composite
> font, while it appears "cmd.exe" and similar cannot.

You can use Windows font linking to use glyphs from linked fonts like:

. GNU Unifont showing bitmap glyphs for BMP code points - release 8.0.1 is
available in Cygwin package unifont-fonts - latest below is 11.0.2

	https://savannah.gnu.org/projects/unifont
	http://unifoundry.com/unifont/index.html

. Evertype Last Resort font provided by Apple showing standard representative
Unicode block glyphs with the code point in the wide glyph border

	https://www.unicode.org/policies/lastresortfont_eula.html

. SIL Fallback showing the BMP code point inside a box

	https://scripts.sil.org/cms/scripts/page.php?site_id=nrsi&id=UnicodeBMPFallbackFont

-- 
Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada

This email may be disturbing to some readers as it contains
too much technical detail. Reader discretion is advised.

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple