From: Brian Inglis <Brian.Inglis@Shaw.ca>
To: cygwin@cygwin.com
Subject: Re: raise(-1) has stopped returning an error recently
Date: Wed, 24 Nov 2021 11:01:46 -0700 [thread overview]
Message-ID: <643c1cb7-9b18-25cf-62b0-8085c8fab137@Shaw.ca> (raw)
In-Reply-To: <YZ4FGpEDDar45HC7@calimero.vinschen.de>
[-- Attachment #1: Type: text/plain, Size: 2897 bytes --]
On 2021-11-24 02:25, Corinna Vinschen via Cygwin wrote:
> On Nov 24 09:36, Duncan Roe wrote:
>> On Tue, Nov 23, 2021 at 11:18:25AM -0700, Brian Inglis wrote:
>>>> On Nov 23 19:27, Duncan Roe wrote:
>>>>> Btw to whoever maintains grep for cygwin: 'make check' should pass on
>>>>> next release (I patched out the surrogate-pair failre).
>>>
>>> I had no problems with test-raise last release.
>>
>> I don't remember having a problem with it even a few weeks ago.
>>
>>> I did with surrogate pairs but after spending too much time on all the test
>>> infrastructure around that, decided it was a low probability event, and wait
>>> until anyone complains to refer it upstream.
>>
>> I wasted time on that too. That's why I patched surrogate-pair to not do its 3rd
>> test if 'uname -s' indicates Cygwin.
>>
>> For the full story, see https://debbugs.gnu.org/cgi/bugreport.cgi?bug=27555#5
>
> What is that "permanent restriction" in Cygwin? Is that something we
> could fix or something unfixable? Did you try to debug Cygwin in terms
> of that problem? If not, could you extract a reduced, very simple
> stand-alone testcase for further debugging?
>
>>> Do Cygwin and/or Windows support surrogate pairs in UTF-8?
>
> You mean UTF-16. UTF-8 doesn't know surrogate pairs, UTF-16 does.
> Originally there was UCS-2, 16 bits, with only 65536 code points.
> However, Unicode left the BMP already with version 2.0 in 1996, so
> UTF-16 and surrogate pairs became necessary. Windows as well as Cygwin
> support them.
How does Cygwin support UTF-16 locales with surrogate pairs?
Are they the "native" locales inherited from Windows if others are not
specified e.g. UTF-8, some OEM SBCS or MBCS?
>> There are 3 tests in surrogate-pair and only the 3rd one failed. So I guess
>> surrogate pairs in UTF-8 "mostly work".
>
> UTF-16. The surrogate stuff is evil at times. Have a look at the
> __utf8_wctomb function in
> https://sourceware.org/git/?p=newlib-cygwin.git;a=blob;f=newlib/libc/stdlib/wctomb_r.c
> Lone surrogate halfs in an input stream are a problem, for instance.
Thus the confusion with grep surrogate pair tests which appear to be
running under a UTF-8 locale: see attached surrogate pair extract from
cygport --debug grep.cygport check.
Trying to rerun cygport build most tests are now "skipped test: failed
to find an adequate shell SKIP ... (exit status: 77)"! Something more
may have changed (in gnulib?) to invalidate Cygwin shell(s) in something
updated since that grep release in August, as I am getting the same
skipped tests under GitHub CI, although it could just be that something
expects say bash > 4.4 or even >= 5!
--
Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada
This email may be disturbing to some readers as it contains
too much technical detail. Reader discretion is advised.
[Data in binary units and prefixes, physical quantities in SI.]
[-- Attachment #2: grep-check-surrogate-pair.log --]
[-- Type: text/plain, Size: 5657 bytes --]
FAIL: surrogate-pair
FAIL: surrogate-pair
====================
+ initial_cwd_=$HOME/src/cygwin/grep/grep-3.7-2.x86_64/build/tests
+ testdir_prefix_
+ printf gt
+ pfx_=gt
+ mktempd_ $HOME/src/cygwin/grep/grep-3.7-2.x86_64/build/tests gt-surrogate-pair.XXXX
+ destdir_=$HOME/src/cygwin/grep/grep-3.7-2.x86_64/build/tests
+ template_=gt-surrogate-pair.XXXX
+ MAX_TRIES_=4
+ destdir_slash_=$HOME/src/cygwin/grep/grep-3.7-2.x86_64/build/tests/
+ unset TMPDIR
+ d=$HOME/src/cygwin/grep/grep-3.7-2.x86_64/build/tests/gt-surrogate-pair.QL4K
+ :
+ test -d $HOME/src/cygwin/grep/grep-3.7-2.x86_64/build/tests/gt-surrogate-pair.QL4K
+ ls -dgo $HOME/src/cygwin/grep/grep-3.7-2.x86_64/build/tests/gt-surrogate-pair.QL4K
+ perms=drwx------+ 1 0 Aug 31 01:38 $HOME/src/cygwin/grep/grep-3.7-2.x86_64/build/tests/gt-surrogate-pair.QL4K
+ :
+ echo $HOME/src/cygwin/grep/grep-3.7-2.x86_64/build/tests/gt-surrogate-pair.QL4K
+ return
+ test_dir_=$HOME/src/cygwin/grep/grep-3.7-2.x86_64/build/tests/gt-surrogate-pair.QL4K
+ cd $HOME/src/cygwin/grep/grep-3.7-2.x86_64/build/tests/gt-surrogate-pair.QL4K
+ builddir=..
+ export srcdir builddir
+ gl_init_sh_nl_=
+ IFS=
+ expr 1 + 128
+ eval trap 'Exit 129' 1
+ trap Exit 129 1
+ expr 2 + 128
+ eval trap 'Exit 130' 2
+ trap Exit 130 2
+ expr 3 + 128
+ eval trap 'Exit 131' 3
+ trap Exit 131 3
+ expr 13 + 128
+ eval trap 'Exit 141' 13
+ trap Exit 141 13
+ expr 15 + 128
+ eval trap 'Exit 143' 15
+ trap Exit 143 15
+ trap remove_tmp_ 0
+ path_prepend_ ../src
+ test 1 != 0
+ path_dir_=../src
+ abs_path_dir_=$HOME/src/cygwin/grep/grep-3.7-2.x86_64/build/tests/../src
+ PATH=$HOME/src/cygwin/grep/grep-3.7-2.x86_64/build/tests/../src:$HOME/src/cygwin/grep/grep-3.7-2.x86_64/build/src:./src:$HOME/bin:$HOME/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/sbin:...
+ create_exe_shims_ $HOME/src/cygwin/grep/grep-3.7-2.x86_64/build/tests/../src
+ return 0
+ shift
+ test 0 != 0
+ export PATH
+ require_en_utf8_locale_
+ path_prepend_ .
+ test 1 != 0
+ path_dir_=.
+ abs_path_dir_=$HOME/src/cygwin/grep/grep-3.7-2.x86_64/build/tests/.
+ PATH=$HOME/src/cygwin/grep/grep-3.7-2.x86_64/build/tests/.:$HOME/src/cygwin/grep/grep-3.7-2.x86_64/build/tests/../src:$HOME/src/cygwin/grep/grep-3.7-2.x86_64/build/src:./src:$HOME/bin:$HOME/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/sbin:...
+ create_exe_shims_ $HOME/src/cygwin/grep/grep-3.7-2.x86_64/build/tests/.
+ return 0
+ shift
+ test 0 != 0
+ export PATH
+ get-mb-cur-max en_US.UTF-8
+ require_compiled_in_MB_support
+ require_en_utf8_locale_
+ path_prepend_ .
+ test 1 != 0
+ path_dir_=.
+ abs_path_dir_=$HOME/src/cygwin/grep/grep-3.7-2.x86_64/build/tests/.
+ PATH=$HOME/src/cygwin/grep/grep-3.7-2.x86_64/build/tests/.:$HOME/src/cygwin/grep/grep-3.7-2.x86_64/build/tests/.:$HOME/src/cygwin/grep/grep-3.7-2.x86_64/build/tests/../src:$HOME/src/cygwin/grep/grep-3.7-2.x86_64/build/src:./src:$HOME/bin:$HOME/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/sbin:...
+ create_exe_shims_ $HOME/src/cygwin/grep/grep-3.7-2.x86_64/build/tests/.
+ return 0
+ shift
+ test 0 != 0
+ export PATH
+ get-mb-cur-max en_US.UTF-8
+ printf é
+ LC_ALL=en_US.UTF-8 grep [[:lower:]]
é
+ fail=0
+ printf /360/220/220/205
+ s_pair=𐐅
+ printf %s/n 𐐅
+ LC_ALL=en_US.UTF-8
+ export LC_ALL
+ returns_ 1 grep -i anything-else in
+ compare /dev/null out
+ compare_dev_null_ /dev/null out
+ test 2 = 2
+ test x/dev/null = x/dev/null
+ test -s out
+ return 0
+ return 0
+ grep . in
+ cat out
+ cat err
+ io_pair=𐐅:
+ grep --file=in in
+ fail=1
+ compare out in
+ compare_dev_null_ out in
+ test 2 = 2
+ test xout = x/dev/null
+ test xin = x/dev/null
+ return 2
+ compare_ out in
+ diff -u out in
--- out 2021-08-31 01:38:22.950752100 -0600
+++ in 2021-08-31 01:38:22.591357000 -0600
@@ -0,0 +1 @@
+𐐅
+ fail=1
+ grep --file=in -i in
+ fail=1
+ compare out in
+ compare_dev_null_ out in
+ test 2 = 2
+ test xout = x/dev/null
+ test xin = x/dev/null
+ return 2
+ compare_ out in
+ diff -u out in
--- out 2021-08-31 01:38:23.138262500 -0600
+++ in 2021-08-31 01:38:22.591357000 -0600
@@ -0,0 +1 @@
+𐐅
+ fail=1
+ grep --file=in -E in
+ fail=1
+ compare out in
+ compare_dev_null_ out in
+ test 2 = 2
+ test xout = x/dev/null
+ test xin = x/dev/null
+ return 2
+ compare_ out in
+ diff -u out in
--- out 2021-08-31 01:38:23.325773300 -0600
+++ in 2021-08-31 01:38:22.591357000 -0600
@@ -0,0 +1 @@
+𐐅
+ fail=1
+ grep --file=in -F in
+ fail=1
+ compare out in
+ compare_dev_null_ out in
+ test 2 = 2
+ test xout = x/dev/null
+ test xin = x/dev/null
+ return 2
+ compare_ out in
+ diff -u out in
--- out 2021-08-31 01:38:23.528911300 -0600
+++ in 2021-08-31 01:38:22.591357000 -0600
@@ -0,0 +1 @@
+𐐅
+ fail=1
+ grep --file=in -iE in
+ fail=1
+ compare out in
+ compare_dev_null_ out in
+ test 2 = 2
+ test xout = x/dev/null
+ test xin = x/dev/null
+ return 2
+ compare_ out in
+ diff -u out in
--- out 2021-08-31 01:38:23.716423100 -0600
+++ in 2021-08-31 01:38:22.591357000 -0600
@@ -0,0 +1 @@
+𐐅
+ fail=1
+ grep --file=in -iF in
+ fail=1
+ compare out in
+ compare_dev_null_ out in
+ test 2 = 2
+ test xout = x/dev/null
+ test xin = x/dev/null
+ return 2
+ compare_ out in
+ diff -u out in
--- out 2021-08-31 01:38:23.919559100 -0600
+++ in 2021-08-31 01:38:22.591357000 -0600
@@ -0,0 +1 @@
+𐐅
+ fail=1
+ Exit 1
+ set +e
+ exit 1
+ exit 1
+ remove_tmp_
+ __st=1
+ cleanup_
+ :
+ test = yes
+ cd $HOME/src/cygwin/grep/grep-3.7-2.x86_64/build/tests
+ chmod -R u+rwx $HOME/src/cygwin/grep/grep-3.7-2.x86_64/build/tests/gt-surrogate-pair.QL4K
+ rm -rf $HOME/src/cygwin/grep/grep-3.7-2.x86_64/build/tests/gt-surrogate-pair.QL4K
+ exit 1
FAIL surrogate-pair (exit status: 1)
next prev parent reply other threads:[~2021-11-24 18:01 UTC|newest]
Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-11-22 5:20 Duncan Roe
2021-11-22 10:25 ` Corinna Vinschen
2021-11-22 13:06 ` Corinna Vinschen
2021-11-23 8:27 ` Duncan Roe
2021-11-23 9:50 ` Corinna Vinschen
2021-11-23 18:18 ` Brian Inglis
2021-11-23 22:36 ` Duncan Roe
2021-11-24 9:25 ` Corinna Vinschen
2021-11-24 18:01 ` Brian Inglis [this message]
2021-11-25 12:54 ` Corinna Vinschen
2021-11-27 7:24 ` Brian Inglis
2021-11-28 3:04 ` Duncan Roe
2021-11-26 23:43 ` Duncan Roe
2021-11-29 10:41 ` Corinna Vinschen
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=643c1cb7-9b18-25cf-62b0-8085c8fab137@Shaw.ca \
--to=brian.inglis@shaw.ca \
--cc=cygwin@cygwin.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).