[Bug libstdc++/64467] New: [5 Regression] 28_regex/traits/char/isctype.cc and wchar_t/isctype.cc

[Bug libstdc++/64467] New: [5 Regression] 28_regex/traits/char/isctype.cc and wchar_t/isctype.cc

[hp at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64467

            Bug ID: 64467
           Summary: [5 Regression] 28_regex/traits/char/isctype.cc and
                    wchar_t/isctype.cc
           Product: gcc
           Version: unknown
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: libstdc++
          Assignee: unassigned at gcc dot gnu.org
          Reporter: hp at gcc dot gnu.org
              Host: x86_64-linux
            Target: cris-elf

These tests used to pass for cris-elf, but has feiled since a commit in the
range (217057:217069] (I'm behind on regression-bug-reporting):

Running
/tmp/hpautotest-gcc0/gcc/libstdc++-v3/testsuite/libstdc++-dg/conformance.exp
...
[...]

FAIL: 28_regex/traits/char/isctype.cc execution test
FAIL: 28_regex/traits/wchar_t/isctype.cc execution test

In libstdc++.log (compile-line only quoted for completeness):

Executing on host: /tmp/hpautotest-gcc0/cris-elf/gccobj/./gcc/xg++
-shared-libgcc -B/tmp/hpautotest-gcc0/cris-elf/gccobj/./gcc -nostdinc++
-L/tmp/hpautotest-gcc0/cris-elf/gccobj/cris-elf/libstdc++-v3/src
-L/tmp/hpautotest-gcc0/cris-elf/gccobj/cris-elf/libstdc++-v3/src/.libs
-L/tmp/hpautotest-gcc0/cris-elf/gccobj/cris-elf/libstdc++-v3/libsupc++/.libs
-nostdinc -B/tmp/hpautotest-gcc0/cris-elf/gccobj/cris-elf/newlib/ -isystem
/tmp/hpautotest-gcc0/cris-elf/gccobj/cris-elf/newlib/targ-include -isystem
/tmp/hpautotest-gcc0/gcc/newlib/libc/include
-B/tmp/hpautotest-gcc0/cris-elf/gccobj/cris-elf/libgloss/cris
-L/tmp/hpautotest-gcc0/cris-elf/gccobj/cris-elf/libgloss/libnosys
-L/tmp/hpautotest-gcc0/gcc/libgloss/cris
-B/tmp/hpautotest-gcc0/cris-elf/pre/cris-elf/bin/
-B/tmp/hpautotest-gcc0/cris-elf/pre/cris-elf/lib/ -isystem
/tmp/hpautotest-gcc0/cris-elf/pre/cris-elf/include -isystem
/tmp/hpautotest-gcc0/cris-elf/pre/cris-elf/sys-include
-B/tmp/hpautotest-gcc0/cris-elf/gccobj/cris-elf/./libgloss/cris/
-L/tmp/hpautotest-gcc0/cris-elf/gccobj/cris-elf/./libgloss/cris
-L/tmp/hpautotest-gcc0/gcc/libgloss/cris
-B/tmp/hpautotest-gcc0/cris-elf/gccobj/cris-elf/./libstdc++-v3/src/.libs
-D_GLIBCXX_ASSERT -fmessage-length=0 -ffunction-sections -fdata-sections -g -O2
-DLOCALEDIR="." -nostdinc++
-I/tmp/hpautotest-gcc0/cris-elf/gccobj/cris-elf/libstdc++-v3/include/cris-elf
-I/tmp/hpautotest-gcc0/cris-elf/gccobj/cris-elf/libstdc++-v3/include
-I/tmp/hpautotest-gcc0/gcc/libstdc++-v3/libsupc++
-I/tmp/hpautotest-gcc0/gcc/libstdc++-v3/include/backward
-I/tmp/hpautotest-gcc0/gcc/libstdc++-v3/testsuite/util
/tmp/hpautotest-gcc0/gcc/libstdc++-v3/testsuite/28_regex/traits/char/isctype.cc
  -std=gnu++11 ./libtestc++.a    -isystem
/tmp/hpautotest-gcc0/cris-elf/gccobj/cris-elf/./newlib/targ-include -isystem
/tmp/hpautotest-gcc0/gcc/newlib/libc/include
-B/tmp/hpautotest-gcc0/cris-elf/gccobj/cris-elf/./libgloss/cris/
-L/tmp/hpautotest-gcc0/cris-elf/gccobj/cris-elf/./libgloss/cris
-L/tmp/hpautotest-gcc0/gcc/libgloss/cris 
-B/tmp/hpautotest-gcc0/cris-elf/gccobj/cris-elf/./newlib/
-L/tmp/hpautotest-gcc0/cris-elf/gccobj/cris-elf/./newlib -sim3  -lm   -o
./isctype.exe    (timeout = 600)
PASS: 28_regex/traits/char/isctype.cc (test for excess errors)
assertion "!t.isctype('\n', t.lookup_classname(blank,
blank+sizeof(blank)/sizeof(blank[0])-1))" failed: file
"/tmp/hpautotest-gcc0/gcc/libstdc++-v3/testsuite/28_regex/traits/char/isctype.cc",
line 56, function: void test01()
program stopped with signal 6 (Aborted).
FAIL: 28_regex/traits/char/isctype.cc execution test

and also:

PASS: 28_regex/traits/wchar_t/isctype.cc (test for excess errors)
assertion "!t.isctype(L'\n', t.lookup_classname(blank,
blank+sizeof(blank)/sizeof(blank[0])-1))" failed: file
"/tmp/hpautotest-gcc0/gcc/libstdc++-v3/testsuite/28_regex/traits/wchar_t/isctype.cc",
line 53, function: void test01()
program stopped with signal 6 (Aborted).
FAIL: 28_regex/traits/wchar_t/isctype.cc execution test

(same fail message still at r219089).

Committer of libstdc++ changes in the indicated revision range CC:ed.

Unexpectedly, I don't see this for other non-linux and/or "newlib" targets with
libstdc++ results posted to gcc-testresults@.  There was also a commit to
gcc/ifcvt.c in this range, I'll triage the revision range to exclude that.

[Bug libstdc++/64467] [5 Regression] 28_regex/traits/char/isctype.cc and wchar_t/isctype.cc

[hp at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64467

Hans-Peter Nilsson <hp at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |NEW
   Last reconfirmed|                            |2015-01-01
                 CC|                            |redi at gcc dot gnu.org
     Ever confirmed|0                           |1

--- Comment #1 from Hans-Peter Nilsson <hp at gcc dot gnu.org> ---
(Fumble-fingered the committer-add to the CC list.)

[Bug libstdc++/64467] [5 Regression] 28_regex/traits/char/isctype.cc and wchar_t/isctype.cc

[redi at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64467

Jonathan Wakely <redi at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |ASSIGNED
           Assignee|unassigned at gcc dot gnu.org      |redi at gcc dot gnu.org

[Bug libstdc++/64467] [5 Regression] 28_regex/traits/char/isctype.cc and wchar_t/isctype.cc

[hp at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64467

--- Comment #2 from Hans-Peter Nilsson <hp at gcc dot gnu.org> ---
(In reply to Hans-Peter Nilsson from comment #0)
> I'll triage the revision range to exclude that.

It's one of 217063-217066, inclusive.

[Bug libstdc++/64467] [5 Regression] 28_regex/traits/char/isctype.cc and wchar_t/isctype.cc

[redi at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64467

--- Comment #3 from Jonathan Wakely <redi at gcc dot gnu.org> ---
Almost certainly r217066.

Is this a newlib target? I would expect to see the same failure for all newlib
targets, as I defined std::ctype_base::blank to be equal to
std::ctype_base::space for newlib (and any libc where I couldn't determine the
right bitmask for the [:blank:] character class) and the [:space:] class
includes newline.

[Bug libstdc++/64467] [5 Regression] 28_regex/traits/char/isctype.cc and wchar_t/isctype.cc

[hp at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64467

--- Comment #4 from Hans-Peter Nilsson <hp at gcc dot gnu.org> ---
(In reply to Jonathan Wakely from comment #3)
> Almost certainly r217066.
> 
> Is this a newlib target?

Yes.

> I would expect to see the same failure for all
> newlib targets, as I defined std::ctype_base::blank to be equal to
> std::ctype_base::space for newlib (and any libc where I couldn't determine
> the right bitmask for the [:blank:] character class) and the [:space:] class
> includes newline.

What I meant by "Unexpectedly, I don't see this for other non-linux and/or
"newlib" targets with libstdc++ results posted to gcc-testresults@", is that I
couldn't find that which you (and I) expected.

Though, not many posted "newlib" targets test-results include libstdc++
results.  Trying to build one myself (one where I didn't also write the port)
but alas previously trusted targets such as arm-eabi and powerpc-eabisim break
left and right (building or massive test failures).  Bah.

[Bug libstdc++/64467] [5 Regression] 28_regex/traits/char/isctype.cc and wchar_t/isctype.cc

[hp at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64467

--- Comment #5 from Hans-Peter Nilsson <hp at gcc dot gnu.org> ---
(In reply to Jonathan Wakely from comment #3)
> Almost certainly r217066.
> 
> Is this a newlib target? I would expect to see the same failure for all
> newlib targets,

I verified that this is the case for mmix-knuth-mmixware and arm-eabi too, just
to eliminate my initial suspicion that this was a target bug.

> as I defined std::ctype_base::blank to be equal to
> std::ctype_base::space for newlib (and any libc where I couldn't determine
> the right bitmask for the [:blank:] character class) and the [:space:] class
> includes newline.

Uh, src/newlib/libc/include/ctype.h:

#define _B      0200
...
#define isblank(__c) \
  __extension__ ({ __typeof__ (__c) __x = (__c);                \
        (__ctype_lookup(__x)&_B) || (int) (__x) == '\t';})

(while isspace has it's own bit, "#define _S      010")
HTH. :)

But that might not be very helpful, I'm just not sure how to help.

[Bug libstdc++/64467] [5 Regression] 28_regex/traits/char/isctype.cc and wchar_t/isctype.cc

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64467

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|---                         |5.0

[Bug libstdc++/64467] [5 Regression] 28_regex/traits/char/isctype.cc and wchar_t/isctype.cc

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64467

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Priority|P3                          |P4

[Bug libstdc++/64467] [5 Regression] 28_regex/traits/char/isctype.cc and wchar_t/isctype.cc

[redi at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64467

--- Comment #6 from Jonathan Wakely <redi at gcc dot gnu.org> ---
Like some of the BSDs, newlib provides a _B bitmask, but that does not
correspond to isblank so can't be used. The _B mask doesn't match the TAB
character, but isblank must match that. The newlib isblank macro handles '\t'
manually, but libstdc++ can't do that and only works with simple bitmasks, not
bitmasks plus extra checks.

https://gcc.gnu.org/ml/gcc-patches/2015-02/msg00059.html just adjusts the test
to account for this.

[Bug libstdc++/64467] [5 Regression] 28_regex/traits/char/isctype.cc and wchar_t/isctype.cc

[renlin at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64467

--- Comment #7 from renlin at gcc dot gnu.org ---
Author: renlin
Date: Wed Feb  4 09:24:56 2015
New Revision: 220392

URL: https://gcc.gnu.org/viewcvs?rev=220392&root=gcc&view=rev
Log:
[PATCH][libstdc++][Testsuite] isctype test fails for newlib.

libstdc++-v3/
2015-02-02  Matthew Wahab  <matthew.wahab@arm.com>

    PR libstdc++/64467
    * testsuite/28_regex/testsuiteraits/char/isctype.cc (test01): Add newlib
    special case for '\n'.
    * test01estsuite/28_regex/traits/wchar_t/isctype.cc (test01): Likewise.


Modified:
    trunk/libstdc++-v3/ChangeLog
    trunk/libstdc++-v3/testsuite/28_regex/traits/char/isctype.cc
    trunk/libstdc++-v3/testsuite/28_regex/traits/wchar_t/isctype.cc

[Bug libstdc++/64467] [5 Regression] 28_regex/traits/char/isctype.cc and wchar_t/isctype.cc

[hp at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64467

--- Comment #10 from Hans-Peter Nilsson <hp at gcc dot gnu.org> ---
(In reply to Hans-Peter Nilsson from comment #9)
> To wit, at r220506  still see:
> assertion "!t.isctype('\n', t.lookup_classname(blank,
> blank+sizeof(blank)/sizeof(blank[0])-1))" failed: file
> "/tmp/hpautotest-gcc1/gcc/libstdc++-v3/testsuite/28_regex/traits/char/
> isctype.cc", line 61, function: void test01()
> program stopped with signal 6 (Aborted).
> 
> Maybe the newlib preprocessor symbol isn't universal?

Universal regarding targets but not regarding time as it's recent-ish:

2014-09-17  Jeff Johnston  <jjohnstn@redhat.com>

        * libc/include/sys/features.h: Add __NEWLIB__ and
        __NEWLIB_MINOR__ macros.

My autotester hasn't automatically updated to a version with this change as
there hasn't been a regression-free period for gcc since 2014-05-08-20:35:45
r210245. :(

More to the point, this test being the random only requirement for the last
newlib release seems spurious.

There's a _NEWLIB_VERSION and a __NEWLIB_H__ which would work for not just the
latest newlib version.  Would it be ok to change to (suggesting) #ifdef
_NEWLIB_VERSION?  Alternatively, we have an effective-target newlib which can
be used to define a NEWLINE_IN_CLASS_BLANK which can be used for other
misfortunate targets.

[Bug libstdc++/64467] [5 Regression] 28_regex/traits/char/isctype.cc and wchar_t/isctype.cc

[redi at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64467

--- Comment #11 from Jonathan Wakely <redi at gcc dot gnu.org> ---
Is this fixed at r220682 ?

[Bug libstdc++/64467] [5 Regression] 28_regex/traits/char/isctype.cc and wchar_t/isctype.cc

[hp at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64467

Hans-Peter Nilsson <hp at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |RESOLVED
         Resolution|---                         |FIXED

--- Comment #12 from Hans-Peter Nilsson <hp at gcc dot gnu.org> ---
(In reply to Jonathan Wakely from comment #11)
> Is this fixed at r220682 ?

I think we can close this yes, as long as it's ok to have newline as isblank.

[Bug libstdc++/64467] [5 Regression] 28_regex/traits/char/isctype.cc and wchar_t/isctype.cc

[redi at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64467

--- Comment #13 from Jonathan Wakely <redi at gcc dot gnu.org> ---
In the POSIX (i.e. "C") locale it should only be space and tab, but
implementing that when the target only has 8-bits for the ctype mask and
doesn't have a blank class is very difficult.

For other locales it's OK for newline to be in the blank class.

[Bug libstdc++/64467] [5 Regression] 28_regex/traits/char/isctype.cc and wchar_t/isctype.cc

[segher at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64467

Segher Boessenkool <segher at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |REOPENED
                 CC|                            |segher at gcc dot gnu.org
         Resolution|FIXED                       |---

--- Comment #14 from Segher Boessenkool <segher at gcc dot gnu.org> ---
It now fails for my newlib target.  Disabling the isctype.cc "fix"
makes it work again.

[Bug libstdc++/64467] [5 Regression] 28_regex/traits/char/isctype.cc and wchar_t/isctype.cc

[segher at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64467

--- Comment #16 from Segher Boessenkool <segher at gcc dot gnu.org> ---
It turns out I built with config/os/generic, since I do not configure
--with-newlib.  It still is newlib though, and the dejagnu test knows
that.  Impedance mismatch, kaboom.  It also seems that isctype for
generic works better than the one for newlib?

[Bug libstdc++/64467] [5 Regression] 28_regex/traits/char/isctype.cc and wchar_t/isctype.cc

[redi at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64467

--- Comment #17 from Jonathan Wakely <redi at gcc dot gnu.org> ---
Author: redi
Date: Mon Mar  9 12:47:51 2015
New Revision: 221279

URL: https://gcc.gnu.org/viewcvs?rev=221279&root=gcc&view=rev
Log:
    PR libstdc++/64467
    * testsuite/28_regex/traits/char/isctype.cc: Don't test newline
    for newlib targets. Really fix mixed line-endings this time.

Modified:
    trunk/libstdc++-v3/ChangeLog
    trunk/libstdc++-v3/testsuite/28_regex/traits/char/isctype.cc

[Bug libstdc++/64467] [5 Regression] 28_regex/traits/char/isctype.cc and wchar_t/isctype.cc

[redi at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64467

Jonathan Wakely <redi at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|REOPENED                    |RESOLVED
         Resolution|---                         |FIXED

--- Comment #18 from Jonathan Wakely <redi at gcc dot gnu.org> ---
Fixed, I hope permanently!