[Bug fortran/113313] New: execute_command_line hangs at run time

[Bug fortran/113313] New: execute_command_line hangs at run time

[john.harper at vuw dot ac.nz]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113313

            Bug ID: 113313
           Summary: execute_command_line hangs at run time
           Product: gcc
           Version: 13.1.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: fortran
          Assignee: unassigned at gcc dot gnu.org
          Reporter: john.harper at vuw dot ac.nz
  Target Milestone: ---

This program compiles and executes as expected with 3 other compilers (ifort,
ifx, flang). With gfortran 13.1.0 it compiles happily but prints nothing and
hangs at run time. My operating system is ubuntu1~22.04.

program test
! f2008 using execute_command_line and assuming Linux
  implicit none
  print "(A,L2)",'I am john',iam('john')
  print "(A,L2)",'I am JOHN',iam('JOHN')

contains

  logical function iam(      name)
    character(*),intent(in)::name
    integer estat
    character(len(name)+38):: cmd
    cmd = 'if [ `whoami` != "'//name//'" ]; then exit 1; fi'
    call execute_command_line(cmd,exitstat=estat)
    iam = (estat==0)
  end function iam
end program test

The other compilers all printed

I am john T
I am JOHN F

[Bug libfortran/113313] execute_command_line hangs at run time

[kargl at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113313

kargl at gcc dot gnu.org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |kargl at gcc dot gnu.org

--- Comment #1 from kargl at gcc dot gnu.org ---
Created attachment 57034
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=57034&action=edit
patch to execute_command_line.c

Code compiles and runs on FreeBSD.  That said, I see

% gfcx -o z a.f90 -g && ./z
I am john F
I am JOHN F
% valgrind ./z
==18812== Memcheck, a memory error detector
==18812== Copyright (C) 2002-2022, and GNU GPL'd, by Julian Seward et al.
==18812== Using Valgrind-3.22.0 and LibVEX; rerun with -h for copyright info
==18812== Command: ./z
==18812== 
==18812== Conditional jump or move depends on uninitialised value(s)
==18812==    at 0x4B0AD79: _gfortran_execute_command_line_i4
(execute_command_line.c:203)
==18812==    by 0x400B2A: iam.0 (a.f90:15)
==18812==    by 0x400BE1: MAIN__ (a.f90:5)
==18812==    by 0x400CEF: main (a.f90:6)
==18812== 
I am john F
==18812== Conditional jump or move depends on uninitialised value(s)
==18812==    at 0x4B0AD79: _gfortran_execute_command_line_i4
(execute_command_line.c:203)
==18812==    by 0x400B2A: iam.0 (a.f90:15)
==18812==    by 0x400C8B: MAIN__ (a.f90:6)
==18812==    by 0x400CEF: main (a.f90:6)
==18812== 
I am JOHN F
==18812== 
==18812== HEAP SUMMARY:
==18812==     in use at exit: 0 bytes in 0 blocks
==18812==   total heap usage: 26 allocs, 26 frees, 9,873 bytes allocated

Looking at the code around line 203 suggests that there may be some confusion
with setting the error status.  The attached patch allows the code to
run without the valgrind warnings.

[Bug libfortran/113313] execute_command_line hangs at run time

[john.harper at vuw dot ac.nz]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113313

--- Comment #2 from john.harper at vuw dot ac.nz ---
Thank you! You may wish to know that in my Ubuntu system the program 
runs properly if the function iam is used in an assignment statement not a 
print statement. (Fortran Discourse gave me the idea.) Revised program:

program test2
! f2008 using execute_command_line and assuming Linux
   implicit none
   logical foo(2)
   foo = [iam('john'),iam('JOHN')]
   print "(A,L2)",'I am john',foo(1)
   print "(A,L2)",'I am JOHN',foo(2)

contains

   logical function iam(      name)
     character(*),intent(in)::name
     integer estat
     character(len(name)+38):: cmd
     cmd = 'if [ `whoami` != "'//name//'" ]; then exit 1; fi'
     call execute_command_line(cmd,exitstat=estat)
     iam = (estat==0)
   end function iam
end program test2

  On Thu, 11 Jan 2024, kargl at gcc dot 
gnu.org wrote:

> Date: Thu, 11 Jan 2024 05:49:10 +0000
> From: kargl at gcc dot gnu.org <gcc-bugzilla@gcc.gnu.org>
> To: John Harper <john.harper@vuw.ac.nz>
> Subject: [Bug libfortran/113313] execute_command_line hangs at run time
> Resent-Date: Thu, 11 Jan 2024 18:49:26 +1300 (NZDT)
> Resent-From: <john.harper@vuw.ac.nz>
> 
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113313
>
> kargl at gcc dot gnu.org changed:
>
>           What    |Removed                     |Added
> ----------------------------------------------------------------------------
>                 CC|                            |kargl at gcc dot gnu.org
>
> --- Comment #1 from kargl at gcc dot gnu.org ---
> Created attachment 57034
>  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=57034&action=edit
> patch to execute_command_line.c
>
> Code compiles and runs on FreeBSD.  That said, I see
>
> % gfcx -o z a.f90 -g && ./z
> I am john F
> I am JOHN F
> % valgrind ./z
> ==18812== Memcheck, a memory error detector
> ==18812== Copyright (C) 2002-2022, and GNU GPL'd, by Julian Seward et al.
> ==18812== Using Valgrind-3.22.0 and LibVEX; rerun with -h for copyright info
> ==18812== Command: ./z
> ==18812==
> ==18812== Conditional jump or move depends on uninitialised value(s)
> ==18812==    at 0x4B0AD79: _gfortran_execute_command_line_i4
> (execute_command_line.c:203)
> ==18812==    by 0x400B2A: iam.0 (a.f90:15)
> ==18812==    by 0x400BE1: MAIN__ (a.f90:5)
> ==18812==    by 0x400CEF: main (a.f90:6)
> ==18812==
> I am john F
> ==18812== Conditional jump or move depends on uninitialised value(s)
> ==18812==    at 0x4B0AD79: _gfortran_execute_command_line_i4
> (execute_command_line.c:203)
> ==18812==    by 0x400B2A: iam.0 (a.f90:15)
> ==18812==    by 0x400C8B: MAIN__ (a.f90:6)
> ==18812==    by 0x400CEF: main (a.f90:6)
> ==18812==
> I am JOHN F
> ==18812==
> ==18812== HEAP SUMMARY:
> ==18812==     in use at exit: 0 bytes in 0 blocks
> ==18812==   total heap usage: 26 allocs, 26 frees, 9,873 bytes allocated
>
> Looking at the code around line 203 suggests that there may be some confusion
> with setting the error status.  The attached patch allows the code to
> run without the valgrind warnings.
>
> -- 
> You are receiving this mail because:
> You reported the bug.
>


-- John Harper, School of Mathematics and Statistics
Victoria Univ. of Wellington, PO Box 600, Wellington 6140, New Zealand.
e-mail john.harper@vuw.ac.nz

[Bug libfortran/113313] execute_command_line hangs at run time

[john.harper at vuw dot ac.nz]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113313

--- Comment #3 from john.harper at vuw dot ac.nz ---
I suspect that valgrind complained because I had not given an else part 
for the variable cmd. In my system the valgrind error message went away 
when I declared and evaluated cmd as follows:

    character(len(name)+50):: cmd
    cmd = 'if [ `whoami` != "'//name//'" ]; then exit 1; else exit 0; fi'

On Thu, 11 Jan 2024, john.harper at vuw dot ac.nz wrote:

> Date: Thu, 11 Jan 2024 07:03:05 +0000
> From: john.harper at vuw dot ac.nz <gcc-bugzilla@gcc.gnu.org>
> To: John Harper <john.harper@vuw.ac.nz>
> Subject: [Bug libfortran/113313] execute_command_line hangs at run time
> Resent-Date: Thu, 11 Jan 2024 20:03:22 +1300 (NZDT)
> Resent-From: <john.harper@vuw.ac.nz>
> 
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113313
>
> --- Comment #2 from john.harper at vuw dot ac.nz ---
> Thank you! You may wish to know that in my Ubuntu system the program
> runs properly if the function iam is used in an assignment statement not a
> print statement. (Fortran Discourse gave me the idea.) Revised program:
>
> program test2
> ! f2008 using execute_command_line and assuming Linux
>   implicit none
>   logical foo(2)
>   foo = [iam('john'),iam('JOHN')]
>   print "(A,L2)",'I am john',foo(1)
>   print "(A,L2)",'I am JOHN',foo(2)
>
> contains
>
>   logical function iam(      name)
>     character(*),intent(in)::name
>     integer estat
>     character(len(name)+38):: cmd
>     cmd = 'if [ `whoami` != "'//name//'" ]; then exit 1; fi'
>     call execute_command_line(cmd,exitstat=estat)
>     iam = (estat==0)
>   end function iam
> end program test2
>
>  On Thu, 11 Jan 2024, kargl at gcc dot
> gnu.org wrote:
>
>> Date: Thu, 11 Jan 2024 05:49:10 +0000
>> From: kargl at gcc dot gnu.org <gcc-bugzilla@gcc.gnu.org>
>> To: John Harper <john.harper@vuw.ac.nz>
>> Subject: [Bug libfortran/113313] execute_command_line hangs at run time
>> Resent-Date: Thu, 11 Jan 2024 18:49:26 +1300 (NZDT)
>> Resent-From: <john.harper@vuw.ac.nz>
>>
>> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113313
>>
>> kargl at gcc dot gnu.org changed:
>>
>>           What    |Removed                     |Added
>> ----------------------------------------------------------------------------
>>                 CC|                            |kargl at gcc dot gnu.org
>>
>> --- Comment #1 from kargl at gcc dot gnu.org ---
>> Created attachment 57034
>>  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=57034&action=edit
>> patch to execute_command_line.c
>>
>> Code compiles and runs on FreeBSD.  That said, I see
>>
>> % gfcx -o z a.f90 -g && ./z
>> I am john F
>> I am JOHN F
>> % valgrind ./z
>> ==18812== Memcheck, a memory error detector
>> ==18812== Copyright (C) 2002-2022, and GNU GPL'd, by Julian Seward et al.
>> ==18812== Using Valgrind-3.22.0 and LibVEX; rerun with -h for copyright info
>> ==18812== Command: ./z
>> ==18812==
>> ==18812== Conditional jump or move depends on uninitialised value(s)
>> ==18812==    at 0x4B0AD79: _gfortran_execute_command_line_i4
>> (execute_command_line.c:203)
>> ==18812==    by 0x400B2A: iam.0 (a.f90:15)
>> ==18812==    by 0x400BE1: MAIN__ (a.f90:5)
>> ==18812==    by 0x400CEF: main (a.f90:6)
>> ==18812==
>> I am john F
>> ==18812== Conditional jump or move depends on uninitialised value(s)
>> ==18812==    at 0x4B0AD79: _gfortran_execute_command_line_i4
>> (execute_command_line.c:203)
>> ==18812==    by 0x400B2A: iam.0 (a.f90:15)
>> ==18812==    by 0x400C8B: MAIN__ (a.f90:6)
>> ==18812==    by 0x400CEF: main (a.f90:6)
>> ==18812==
>> I am JOHN F
>> ==18812==
>> ==18812== HEAP SUMMARY:
>> ==18812==     in use at exit: 0 bytes in 0 blocks
>> ==18812==   total heap usage: 26 allocs, 26 frees, 9,873 bytes allocated
>>
>> Looking at the code around line 203 suggests that there may be some confusion
>> with setting the error status.  The attached patch allows the code to
>> run without the valgrind warnings.
>>
>> --
>> You are receiving this mail because:
>> You reported the bug.
>>
>
>
> -- John Harper, School of Mathematics and Statistics
> Victoria Univ. of Wellington, PO Box 600, Wellington 6140, New Zealand.
> e-mail john.harper@vuw.ac.nz
>
> -- 
> You are receiving this mail because:
> You reported the bug.
>


-- John Harper, School of Mathematics and Statistics
Victoria Univ. of Wellington, PO Box 600, Wellington 6140, New Zealand.
e-mail john.harper@vuw.ac.nz

[Bug libfortran/113313] execute_command_line hangs at run time

[jvdelisle at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113313

Jerry DeLisle <jvdelisle at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jvdelisle at gcc dot gnu.org

--- Comment #4 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> ---
I just started looking at this today. I will give the patch a spin in the next
few days and if tests OK I can commit.

John, are you able tp apply Steve's patch and try it? If not would you like to
learn how?  I can show people the ropes.  We have a MatterMost workspace you
can join and I can linkup with you there. (I will send you an invite, just let
me know.)

[Bug libfortran/113313] execute_command_line hangs at run time

[kargl at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113313

kargl at gcc dot gnu.org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Priority|P3                          |P4

--- Comment #5 from kargl at gcc dot gnu.org ---
(In reply to Jerry DeLisle from comment #4)
> I just started looking at this today. I will give the patch a spin in the
> next few days and if tests OK I can commit.
> 
> John, are you able tp apply Steve's patch and try it? If not would you like
> to learn how?  I can show people the ropes.  We have a MatterMost workspace
> you can join and I can linkup with you there. (I will send you an invite,
> just let me know.)

Jerry, there are 2 paths through the function execute_command_line()
in execute_command_line.c  One is for synchronous execution and one
is for asynchronous.  For async, exitstat is untouched.  For synchronous,
exitstat is set.

   EXITSTAT (optional) shall be a scalar of type integer with a decimal
   exponent range of at least nine.  It is an INTENT(INOUT) argument. If
   the command is executed synchronously, it is assigned the value of the
   processor-dependent exit status. Otherwise, the value of EXITSTAT is
   unchanged.

If I read the code correctly, it tries to save a copy of the input
value of exitstat, and then uses that to compare to the returned
value.  if exitstat is present, then is should simply be set to
the return value.

[Bug libfortran/113313] execute_command_line hangs at run time

[john.harper at vuw dot ac.nz]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113313

--- Comment #6 from john.harper at vuw dot ac.nz ---
I know nothing about either applying gfortran patches or MatterMost but 
I'm willing to try.

On Thu, 11 Jan 2024, jvdelisle at gcc dot gnu.org wrote:

> Date: Thu, 11 Jan 2024 20:18:36 +0000
> From: jvdelisle at gcc dot gnu.org <gcc-bugzilla@gcc.gnu.org>
> To: John Harper <john.harper@vuw.ac.nz>
> Subject: [Bug libfortran/113313] execute_command_line hangs at run time
> Resent-Date: Fri, 12 Jan 2024 09:18:48 +1300 (NZDT)
> Resent-From: <john.harper@vuw.ac.nz>
> 
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113313
>
> Jerry DeLisle <jvdelisle at gcc dot gnu.org> changed:
>
>           What    |Removed                     |Added
> ----------------------------------------------------------------------------
>                 CC|                            |jvdelisle at gcc dot gnu.org
>
> --- Comment #4 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> ---
> I just started looking at this today. I will give the patch a spin in the next
> few days and if tests OK I can commit.
>
> John, are you able tp apply Steve's patch and try it? If not would you like to
> learn how?  I can show people the ropes.  We have a MatterMost workspace you
> can join and I can linkup with you there. (I will send you an invite, just let
> me know.)
>
> -- 
> You are receiving this mail because:
> You reported the bug.
>


-- John Harper, School of Mathematics and Statistics
Victoria Univ. of Wellington, PO Box 600, Wellington 6140, New Zealand.
e-mail john.harper@vuw.ac.nz

[Bug libfortran/113313] execute_command_line hangs at run time

[sgk at troutmask dot apl.washington.edu]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113313

--- Comment #7 from Steve Kargl <sgk at troutmask dot apl.washington.edu> ---
On Thu, Jan 11, 2024 at 07:03:05AM +0000, john.harper at vuw dot ac.nz wrote:
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113313
> 
> --- Comment #2 from john.harper at vuw dot ac.nz ---
> Thank you! You may wish to know that in my Ubuntu system the program 
> runs properly if the function iam is used in an assignment statement not a 
> print statement.

Interesting.  One of the first things that the internal
implementation does is 

  /* Flush all I/O units before executing the command.  */
  flush_all_units();

libgfortran is supposedly thread-safe and looking into
flush_all_units() shows some unlocking and testing for
locks.  With 'print *, iam('john')', there may be lock
contention.

[Bug libfortran/113313] execute_command_line hangs at run time

[jvdelisle at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113313

--- Comment #8 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> ---
(In reply to Steve Kargl from comment #7)
--- snip ---
> libgfortran is supposedly thread-safe and looking into
> flush_all_units() shows some unlocking and testing for
> locks.  With 'print *, iam('john')', there may be lock
> contention.

My first thought when I observed the test case execution is locks. Have not
looked further yet.

[Bug libfortran/113313] execute_command_line hangs at run time

[john.harper at vuw dot ac.nz]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113313

--- Comment #9 from john.harper at vuw dot ac.nz ---
This variant of my test program (now called test4) hangs with gfortran 
but not with other compilers if iam is written to output_unit (like the 
original version) or to a file with a different positive unit number, but 
it does not hang if written to a file with a negative unit number obtained 
by opening it with newunit=u. Its user has those 3 choices at run time.

I hope it helps you explore this oddity of gfortran.

program test4
   use iso_fortran_env, only: output_unit
! Linux with f2008 (newunit=, execute_command_line, allocatable scalar)
   implicit none
   integer:: u = output_unit+20
   character:: ch*1, line(2)*80
   print "(A)",'Output from iam will go to unit u;', &
        ' u=output_unit if you enter a, or',&
        ' unit=u (u>0)  if you enter b, or',&
        ' newunit=u (u<0) if you enter anything else.'
   read "(A)", ch
   if(ch=='a') then
      u = output_unit
   else if(ch=='b') then
      open(   unit=u,file='test4.out')
   else
      open(newunit=u,file='test4.out')
   end if
   print "(A,I0)",'Output from iam will be written to unit ',u
   write(u,"(A,L2)")'I am john',iam('john')
   write(u,"(A,L2)")'I am JOHN',iam('JOHN')
   if(u/=output_unit) then
      rewind u
      read(u,"(A)") line
      print  "(A)", line
      close(u,status='delete')
   end if
contains

   logical function iam(      name)
     character(*),intent(in)::name
     integer estat
     character(:),allocatable:: cmd
     cmd = 'if [ `whoami` != "'//name//'" ]; then exit 1; else exit 0; fi'
     call execute_command_line(cmd,exitstat=estat)
     iam = (estat==0)
   end function iam
end program test4

-- John Harper, School of Mathematics and Statistics
Victoria Univ. of Wellington, PO Box 600, Wellington 6140, New Zealand.
e-mail john.harper@vuw.ac.nz