Python2 "narrow" build, Unicode issue in regex package

Python2 "narrow" build, Unicode issue in regex package

[Václav Haisman]

Hi.

I have recently hit an issue ([1]) with Python 2.7 and regex package
for it on Cygwin. It appears that Cygwin's Python 2.7 is so called
narrow build. This causes issues when working with Unicode code point
outside BMP, like the emoji code points in my issue.

Is there a chance Cygwin's Python could be rebuilt as a wide build?

[1] https://bitbucket.org/mrabarnett/mrab-regex/issues/241/issues-matching-unicode-code-ranges-with-p


-- 
VH

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Python2 "narrow" build, Unicode issue in regex package

[Erik Bray]

On Wed, May 24, 2017 at 10:30 AM, Václav Haisman wrote:
> Hi.
>
> I have recently hit an issue ([1]) with Python 2.7 and regex package
> for it on Cygwin. It appears that Cygwin's Python 2.7 is so called
> narrow build. This causes issues when working with Unicode code point
> outside BMP, like the emoji code points in my issue.
>
> Is there a chance Cygwin's Python could be rebuilt as a wide build?
>
> [1] https://bitbucket.org/mrabarnett/mrab-regex/issues/241/issues-matching-unicode-code-ranges-with-p

I've been bitten by this before too, and I don't know if there's a
specific policy by which Cygwin has determined the narrow build should
be used.  Though narrow builds are typical on Windows because it
translates easily to native wide character strings on Windows, whereas
using a wide build introduces significantly more overhead.

I know it's trite to answer "use a different tool", but if at all
possible you might consider switching to Python 3, which is the
future. Heck, it's really the present.  Even most of the scientific
Python community has switched over to Python 3 (well, at least the
development community has--users are understandably a little slower).
Many large corporations, such a Instagram, have switched.  And Python
2 support is ending in 2020, so the sooner the better.  I know it's a
hassle though.

Anyways, on current versions of Python 3 (I think 3.3 and above) there
is no longer a wide- versus narrow- distinction.  Instead, each string
is stored in the smallest possible representation that fits the
highest codepoint in the string.

If you need a wide character build on Cygwin you could also build it
yourself.  Just make sure to get a few Cygwin patches from
https://github.com/cygwinports/python2

Best,
Erik

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple