[Bug modula2/108344] New: Many tests time out: isatty called in a tight loop

[Bug modula2/108344] New: Many tests time out: isatty called in a tight loop

[ro at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108344

            Bug ID: 108344
           Summary: Many tests time out: isatty called in a tight loop
           Product: gcc
           Version: 13.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: modula2
          Assignee: gaius at gcc dot gnu.org
          Reporter: ro at gcc dot gnu.org
  Target Milestone: ---
              Host: i386-pc-solaris2.11
            Target: i386-pc-solaris2.11
             Build: i386-pc-solaris2.11

I'm seeing a very weird error running the m2 testsuite on Solaris/x86: quite
a number of tests time out in compilation.  However, this only happens when
gcc is configured to use /bin/as, not with gas.  Besides, the timeouts affect
only 64-bit compiltions.  I'm seeing this with both flex 2.5.35 and 2.6.4.

Here's what I observe: for the affected tests, is in a tight loop like this:

12901:  /var/gcc/regression/master/11.4-gcc/build/gcc/cc1gm2 -fcpp-begin -E -l
 fabbd4c7 ioctl    (0x6, 0x0, 0x1032, 0x8c15c33) + 7
 08c15c87 yy_init_buffer(yy_buffer_state*, __FILE*) (0xadce188, 0xadce189,
0xfeff9778, 0xadce188, 0x0, 0xadce189) + 67
 08c15d17 yyrestart(__FILE*) (0xad29110, 0x1, 0x2000, 0xad29110, 0x14,
0xadcbb60) + 27
 08c18918 yylex() (0xad6c320, 0xfeff97f0, 0xfeff97f8, 0x8c4e16f, 0x2a, 0x1) +
2668
 08c18b9c m2flex_GetToken (0x2a, 0x1, 0xfeff9808, 0x8c4e16f, 0xadd4d58, 0x2a) +
3c
 08c4e16f M2LexBuf_GetToken (0x1, 0xfeff983c, 0xfeff9858, 0x8c347cd, 0xadd3750,
0x2a) + 3f
 08c4ebe5 M2LexBuf_OpenSource (0xadd3750, 0x2a, 0xfeff9858, 0x8c344d8) + 65
 08c347cd M2Comp_compile (0xfeffa662, 0x43, 0x36, 0xad33290) + 3fd
 08bea500 gm2_langhook_parse_file() (0x3, 0xf9e09000, 0xfeff9948, 0x92a78d7,
0x43, 0x0) + 60
 092a442d compile_file() (0x2, 0x8b3c3de, 0xfeff9948, 0x92a7e34) + 1d
 092a78dc toplev::main(int, char**) (0x1000, 0x84ee24f, 0xfae8e310, 0xa4411e1)
+ 136c
 0a4411e1 main     (0xfeff9c6c, 0xfeff99b4, 0x8be8add) + 31
 08be8a46 _start   (0x4f, 0xfeff9cee, 0xfeff9d23, 0xfeff9d2f, 0xfeff9d32,
0xfeff9d3c) + 46

That ioctl is

ioctl(6, TCGETA, 0xFEFF96E0)                    Err#25 ENOTTY

no doubt resulting from calls to isatty in the generated gm2-compiler/m2flex.c.

What's even weirder, if I run such a compilation manually, it completes in
about
half a second, and I've not yet managed to reproduce the issue with anything
but
a full make -jN -k check-m2 run.  This is pretty tedious because those 600
second
timeouts massively add up, slowing the whole test time tremendously.

I've already tried using flex -B to generate the lexer, but that didn't make
a difference, which judging from the skeleton file is no wonder.  However, when
using --never-interactive, things gets way worse: I've already several
instances of cc1gm2 running for 12+ minutes (way beyond the 600 second timeout
of runtest),
making not syscall whatsoever.  All this suggests that something is very weird
in the m2 lexer somewhere.

Here's the list of affected tests (all in gm2/iso/pass, doesn't change between
runs):

addadr1.mod
bits32c.mod
callwraptime.mod
caseiso.mod
caseiso2.mod
cast.mod
cast3.mod
castiso.mod
ChanConsts.mod
const1.mod
constreal.mod
constructor1.mod
constructor2.mod
constructor3.mod
constructor4.mod
constructor5.mod
constructor6.mod
constsize4.mod
ConvTypes.mod
delim.mod
delim2.mod
enummodule.mod
except1.mod
expproc.mod
expproc2.mod
isob.mod
isobitset.mod
isobitset2.mod
longm.mod
m.mod
proccast.mod
realbitscast.mod
set12.mod
stringchar.mod
subassign.mod
testaddindr.mod
testconv.mod
testconv2.mod
testconv3.mod
testconv4.mod
testconv5.mod
testconv6.mod
testconv7.mod
testconv8.mod
testconv9.mod
testiso.mod
testiso2.mod
testisosize.mod
testlength.mod
testlength2.mod
testlength3.mod
testlength4.mod
testlength5.mod
unbounded.mod
unbounded2.mod

[Bug modula2/108344] Many tests time out: isatty called in a tight loop

[ro at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108344

Rainer Orth <ro at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|---                         |13.0

[Bug modula2/108344] Many tests time out: isatty called in a tight loop

[ro at CeBiTec dot Uni-Bielefeld.DE]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108344

--- Comment #1 from ro at CeBiTec dot Uni-Bielefeld.DE <ro at CeBiTec dot Uni-Bielefeld.DE> ---
To gather more information, I've tried a -g3 -O0 build.  The failures
still occur, but the failure mode is slightly different: all the 64-bit
tests in gm2/iso/pass now SEGV in cc1gm2:

Core was generated by `/var/gcc/regression/master/11.4-gcc/build/gcc/cc1gm2
-fcpp-begin -E -lang-asm -'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x0b38d8cf in LINEMAPS_MACRO_USED (set=0xfaee1000) at
/vol/gcc/src/hg/master/local/gcc/../libcpp/include/line-map.h:1002
1002      return LINEMAPS_USED (set, true);

#1  0x0b38d902 in LINEMAPS_MACRO_LOWEST_LOCATION (set=0xfaee1000) at
/vol/gcc/src/hg/master/local/libcpp/include/line-map.h:1025
#2  0x0b3b3815 in linemap_location_from_macro_expansion_p (set=0xfaee1000,
location=223680) at /vol/gcc/src/hg/master/local/libcpp/line-map.cc:1360
#3  0x0b3b2f64 in linemap_lookup (set=0xfaee1000, line=223680) at
/vol/gcc/src/hg/master/local/libcpp/line-map.cc:1055
#4  0x0b3b1c5d in pure_location_p (set=0xfaee1000, loc=223680) at
/vol/gcc/src/hg/master/local/libcpp/line-map.cc:342
#5  0x0b3b1708 in get_combined_adhoc_loc (set=0xfaee1000, locus=223680,
src_range=..., data=0x0, discriminator=0) at
/vol/gcc/src/hg/master/local/libcpp/line-map.cc:190
#6  0x0a029d7e in COMBINE_LOCATION_DATA (set=0xfaee1000, loc=223680,
src_range=..., block=0x0, discriminator=0) at
/vol/gcc/src/hg/master/local/gcc/../libcpp/include/line-map.h:1067
#7  0x0b3822f8 in make_location (caret=223680, src_range=...) at
/vol/gcc/src/hg/master/local/gcc/input.cc:1194
#8  0x09491abc in m2linemap_GetLocationRange (start=2096540, end=2096540) at
/vol/gcc/src/hg/master/local/gcc/m2/gm2-gcc/m2linemap.cc:102
#9  0x094a044f in updatepos () at
/vol/gcc/src/hg/master/local/gcc/m2/m2.flex:521
#10 0x094a0968 in yywrap () at /vol/gcc/src/hg/master/local/gcc/m2/m2.flex:756
#11 0x0949e9e3 in yylex () at
/var/gcc/regression/master/11.4-gcc/build/gcc/<stdout>:1927
#12 0x094a07a7 in m2flex_GetToken () at
/vol/gcc/src/hg/master/local/gcc/m2/m2.flex:654
#13 0x094dea34 in M2LexBuf_GetToken () at m2/gm2-compiler-boot/M2LexBuf.c:958
#14 0x094dea39 in M2LexBuf_GetToken () at m2/gm2-compiler-boot/M2LexBuf.c:959
[...]
#2096552 0x094dea39 in M2LexBuf_GetToken () at
m2/gm2-compiler-boot/M2LexBuf.c:9
59
#2096553 0x094de8d7 in M2LexBuf_OpenSource (s=0xc0fa638) at
m2/gm2-compiler-boot
/M2LexBuf.c:855
#2096554 0x094bf342 in DoPass0 (s=0xc0f3550) at
m2/gm2-compiler-boot/M2Comp.c:304
#2096555 0x094befe1 in Compile (s=0xc0f3550) at
m2/gm2-compiler-boot/M2Comp.c:183
#2096556 0x094bfdae in M2Comp_compile (filename=0xfeffb441) at
m2/gm2-compiler-boot/M2Comp.c:760
#2096557 0x0949b9f5 in init_PerCompilationInit (filename=0xfeffb441
"/vol/gcc/src/hg/master/local/gcc/testsuite/gm2/iso/pass/ChanConsts.mod") at
/vol/gcc/src/hg/master/local/gcc/m2/gm2-gcc/init.cc:195
#2096558 0x09461457 in gm2_parse_input_files (filenames=0xc0907c0,
filename_count=2) at /vol/gcc/src/hg/master/local/gcc/m2/gm2-lang.cc:451
#2096559 0x09461485 in gm2_langhook_parse_file () at
/vol/gcc/src/hg/master/local/gcc/m2/gm2-lang.cc:458
#2096560 0x09c45c5c in compile_file () at
/vol/gcc/src/hg/master/local/gcc/toplev.cc:444
#2096561 0x09c48898 in do_compile (no_backend=false) at
/vol/gcc/src/hg/master/local/gcc/toplev.cc:2125
#2096562 0x09c48c11 in toplev::main (this=0xfeffa7aa, argc=79, argv=0xfeffa80c)
at /vol/gcc/src/hg/master/local/gcc/toplev.cc:2277
#2096563 0x0b328eee in main (argc=79, argv=0xfeffa80c) at
/vol/gcc/src/hg/master/local/gcc/main.cc:39

so we have unbounded recursion in M2LexBuf_GetToken.

[Bug modula2/108344] Many tests time out: isatty called in a tight loop

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108344

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|13.0                        |13.2

--- Comment #2 from Richard Biener <rguenth at gcc dot gnu.org> ---
GCC 13.1 is being released, retargeting bugs to GCC 13.2.

[Bug modula2/108344] Many tests time out: isatty called in a tight loop

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108344

--- Comment #3 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The master branch has been updated by Gaius Mulley <gaius@gcc.gnu.org>:

https://gcc.gnu.org/g:cf4dcfa6727b89362494bd49e2a28ebd10d767ce

commit r14-912-gcf4dcfa6727b89362494bd49e2a28ebd10d767ce
Author: Gaius Mulley <gaiusmod2@gmail.com>
Date:   Tue May 16 15:51:53 2023 +0100

    PR modula2/108344 disable default opening of /dev/tty

    This patch changes removes the static initialisation code for
KeyBoardLEDs.cc.
    The module is only initialised if one of the exported functions is called.
    This is useful as the module will access /dev/tty which might not be
    available.  TimerHandler.mod has also been changed to disable the scroll
    lock LED as a sign of life.

    gcc/m2/ChangeLog:

            PR modula2/108344
            * gm2-libs-coroutines/TimerHandler.mod (EnableLED): New constant.
            (Timer): Test EnableLED before switching on the scroll LED.

    libgm2/ChangeLog:

            PR modula2/108344
            * libm2cor/KeyBoardLEDs.cc (initialize_module): New function.
            (SwitchScroll): Call initialize_module.
            (SwitchNum): Call initialize_module.
            (SwitchCaps): Call initialize_module.
            (SwitchLEDs): Call initialize_module.
            (M2EXPORT): Remove initialization code.

    Signed-off-by: Gaius Mulley <gaiusmod2@gmail.com>

[Bug modula2/108344] Many tests time out: isatty called in a tight loop

[gaius at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108344

Gaius Mulley <gaius at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Last reconfirmed|                            |2023-05-16
             Status|UNCONFIRMED                 |ASSIGNED
     Ever confirmed|0                           |1

--- Comment #4 from Gaius Mulley <gaius at gcc dot gnu.org> ---
A patch has been applied which was created as a response to another bug (on the
gm2 mailing list).  [A request to not open /dev/tty by default].

[Bug modula2/108344] Many tests time out: isatty called in a tight loop

[gaius at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108344

--- Comment #5 from Gaius Mulley <gaius at gcc dot gnu.org> ---
On reflection I suspect it is doubtful this is the same bug.  However it might
reduce the timeouts when running the regression tests.

[Bug modula2/108344] Many tests time out: isatty called in a tight loop

[ro at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108344

Rainer Orth <ro at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |FIXED
             Status|ASSIGNED                    |RESOLVED

--- Comment #6 from Rainer Orth <ro at gcc dot gnu.org> ---
I hadn't seen those timeout for quite some time even before your patch
(roughly since 20230123). So it seems whatever caused this is fixed (or at
least masked).

[Bug modula2/108344] Many tests time out: isatty called in a tight loop

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108344

--- Comment #7 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The releases/gcc-13 branch has been updated by Gaius Mulley
<gaius@gcc.gnu.org>:

https://gcc.gnu.org/g:35a04220554d62d9bbe068dfa2f52d4d2d572805

commit r13-7636-g35a04220554d62d9bbe068dfa2f52d4d2d572805
Author: Gaius Mulley <gaiusmod2@gmail.com>
Date:   Fri Jul 28 22:50:41 2023 +0100

    [PATCH] PR modula2/108344 disable default opening of /dev/tty

    This patch changes removes the static initialisation code for
    KeyBoardLEDs.cc.  The module is only initialised if one of the
    exported functions is called.  This is useful as the module will
    access /dev/tty which might not be available.  TimerHandler.mod has
    also been changed to disable the scroll lock LED as a sign of life.

    gcc/m2/ChangeLog:

            PR modula2/108344
            * gm2-libs-coroutines/TimerHandler.mod (EnableLED): New
            constant.
            (Timer): Test EnableLED before switching on the scroll LED.

    libgm2/ChangeLog:

            PR modula2/108344
            * libm2cor/KeyBoardLEDs.cc (initialize_module): New
            function.
            (SwitchScroll): Call initialize_module.
            (SwitchNum): Call initialize_module.
            (SwitchCaps): Call initialize_module.
            (SwitchLEDs): Call initialize_module.
            (M2EXPORT): Remove initialization code.

    (cherry picked from commit cf4dcfa6727b89362494bd49e2a28ebd10d767ce)

    Signed-off-by: Gaius Mulley <gaiusmod2@gmail.com>