[Bug libfortran/48961] New: EXECUTE_COMMAND_LINE(WAIT=.false.) fails on MinGW

[Bug libfortran/48961] New: EXECUTE_COMMAND_LINE(WAIT=.false.) fails on MinGW

[thenlich at users dot sourceforge.net]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=48961

           Summary: EXECUTE_COMMAND_LINE(WAIT=.false.) fails on MinGW
           Product: gcc
           Version: 4.7.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: libfortran
        AssignedTo: unassigned@gcc.gnu.org
        ReportedBy: thenlich@users.sourceforge.net


When calling the EXECUTE_COMMAND_LINE intrinsic on Windows and setting the WAIT
argument to .false., the execution causes a runtime error.

For example:
program hello
    print *, "Hello world"
end program hello

program test_execute
    call execute_command_line("hello.exe", .false.)
end program test_execute

>test_execute
 Hello world
Fortran runtime error: Could not execute command line

Expected result: 

No runtime error should occur.

Fortran 2008:
13.7.57 EXECUTE_COMMAND_LINE (COMMAND [, WAIT, EXITSTAT, CMDSTAT, CMDMSG ])
...
3   Arguments.
...
WAIT (optional) shall be a default logical scalar. It is an INTENT (IN)
argument. If WAIT is present with the value false, and the processor supports
asynchronous execution of the command, the command is executed asynchronously;
otherwise it is executed synchronously.

5   When the command is executed synchronously, EXECUTE_COMMAND_LINE returns
after the command line has completed execution. Otherwise, EXECUTE_COMMAND_LINE
returns without waiting.

[Bug libfortran/48961] EXECUTE_COMMAND_LINE(WAIT=.false.) fails on MinGW

[burnus at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=48961

Tobias Burnus <burnus at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |burnus at gcc dot gnu.org

--- Comment #1 from Tobias Burnus <burnus at gcc dot gnu.org> 2011-05-11 15:59:45 UTC ---
Question: Does it work with .true.? I think the output is perfectly valid if
the execution itself failed (and not just the asynchronous execution).


On Linux, I would suspect that the current directory (".") is not in the search
path ("$PATH"), but I don't know how it works under windows.

Thus, please double check that (a) the executable is there, (b) it is in the
search patch, and (c) the execution works with ".true.".


In either case (wait=.true. or .false.), the "system" function is called; I do
not see a reason why it should fail for the forked version (wait=.false.) but
work for the not forked one (wait=.true.).

The source code of the function is at:
http://gcc.gnu.org/viewcvs/trunk/libgfortran/intrinsics/execute_command_line.c?view=markup

[Bug libfortran/48961] EXECUTE_COMMAND_LINE(WAIT=.false.) fails on MinGW

[thenlich at users dot sourceforge.net]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=48961

--- Comment #2 from Thomas Henlich <thenlich at users dot sourceforge.net> 2011-05-11 16:38:52 UTC ---
The program is executed, as the called program "hello.exe" prints "Hello
world". After that hello.exe returns and the runtime error occurs.

Further testing shows that it is executed synchronously, presumably because
asynchronous execution is not implemented (yet?) on Windows.

[Bug libfortran/48961] EXECUTE_COMMAND_LINE(WAIT=.false.) fails on MinGW

[burnus at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=48961

--- Comment #3 from Tobias Burnus <burnus at gcc dot gnu.org> 2011-05-11 19:19:40 UTC ---
Created attachment 24226
  --> http://gcc.gnu.org/bugzilla/attachment.cgi?id=24226
Draft patch (untested)

(In reply to comment #2)
> The program is executed, as the called program "hello.exe" prints "Hello
> world". After that hello.exe returns and the runtime error occurs.

As cmdstat is set to -2, there will be an error in set_cmdstat:

  else if (value != 0)
    runtime_error ("Could not execute command line");

The check should be > 0.


> Further testing shows that it is executed synchronously, presumably because
> asynchronous execution is not implemented (yet?) on Windows.

Indeed. That seems to be the problem. fork() does not seem to be supported. Kai
suggests to use _spawnvpe with cmd.exe/command.com

[Bug libfortran/48961] EXECUTE_COMMAND_LINE(WAIT=.false.) fails on MinGW

[burnus at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=48961

--- Comment #4 from Tobias Burnus <burnus at gcc dot gnu.org> 2011-05-11 19:52:02 UTC ---
Created attachment 24228
  --> http://gcc.gnu.org/bugzilla/attachment.cgi?id=24228
async_system - example program for MinGW by Kai

[Bug libfortran/48961] EXECUTE_COMMAND_LINE(WAIT=.false.) fails on MinGW

[burnus at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=48961

--- Comment #5 from Tobias Burnus <burnus at gcc dot gnu.org> 2011-05-11 22:27:03 UTC ---
Patch: http://gcc.gnu.org/ml/fortran/2011-05/msg00087.html

Regarding the implementation of true async on Windows: One should also remember
to update
http://gcc.gnu.org/onlinedocs/gfortran/EXECUTE_005fCOMMAND_005fLINE.html, if
one uses _spawnvpe (w/ command.exe/cmd.exe) instead of system() on Windows,
though the current wording might be sufficient.

[Bug libfortran/48961] EXECUTE_COMMAND_LINE(WAIT=.false.) fails on MinGW

[burnus at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=48961

--- Comment #6 from Tobias Burnus <burnus at gcc dot gnu.org> 2011-05-14 06:35:21 UTC ---
Author: burnus
Date: Sat May 14 06:35:18 2011
New Revision: 173748

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=173748
Log:
2011-05-14  Tobias Burnus  <burnus@net-b.de>

        PR fortran/48961
        * intrinsics/execute_command_line.c (set_cmdstat): Don't abort if
        synchronously executing with WAIT=.false.
        (execute_command_line): Fix setting of cmdstat and exitstat.


Modified:
    trunk/libgfortran/ChangeLog
    trunk/libgfortran/intrinsics/execute_command_line.c

[Bug libfortran/48961] EXECUTE_COMMAND_LINE(WAIT=.false.) fails on MinGW

[burnus at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=48961

--- Comment #7 from Tobias Burnus <burnus at gcc dot gnu.org> 2011-05-14 11:52:34 UTC ---
EXECUTE_COMMAND_LINE(WAIT=.false.) now should no longer fail on systems without
fork().

Remains to do: Implement asynchronous execution on MinGW, cf. attachment 24228.

[Bug libfortran/48961] EXECUTE_COMMAND_LINE(WAIT=.false.) fails on MinGW

[thenlich at users dot sourceforge.net]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=48961

--- Comment #8 from Thomas Henlich <thenlich at users dot sourceforge.net> 2011-05-14 12:54:26 UTC ---
Maybe I'm wrong, but I'm not sure the current implementation of asynchronous
execution is very efficient (on Unix systems):

We fork() and then call system(), which forks again and runs a shell, then the
shell exits, the forked process inside system() exits, the we exit our forked
process.

I think we might as well just fork and exec the shell directly which would save
one process creation. We also don't need the exit value from the system() call
in Fortran.

[Bug libfortran/48961] EXECUTE_COMMAND_LINE(WAIT=.false.) fails on MinGW

[burnus at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=48961

--- Comment #9 from Tobias Burnus <burnus at gcc dot gnu.org> 2011-05-14 14:00:26 UTC ---
(In reply to comment #8)
> We fork() and then call system(), which forks again and runs a shell, then the
> shell exits, the forked process inside system() exits, the we exit our forked
> process.
> 
> I think we might as well just fork and exec the shell directly which would save
> one process creation. We also don't need the exit value from the system() call
> in Fortran.

POSIX states for "system":

"The environment of the executed command shall be as if a child process were
created using fork(), and the child process invoked the sh utility using
execl() as follows:
              execl(<shell path>, "sh", "-c", command, (char *)0);
where <shell path> is an unspecified pathname for the sh utility."


Thus, for a POSIX system, one could use:
  execlp ("sh", "sh", "-c", command, (char*) 0);

However, that assumes that there is a "sh" available. That's the advantage of
"system()" that it is widely available and one avoids to specify the shell
commands.

One cannot do without the shell command as there are users, who want that using
'system("./foo > bar.dat")' works.

Thus, I do not see how one can solve this better than currently done.

[Bug libfortran/48961] EXECUTE_COMMAND_LINE(WAIT=.false.) fails on MinGW

[burnus at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=48961

--- Comment #10 from Tobias Burnus <burnus at gcc dot gnu.org> 2011-05-14 14:05:19 UTC ---
(In reply to comment #9)
>               execl(<shell path>, "sh", "-c", command, (char *)0);
> where <shell path> is an unspecified pathname for the sh utility."

> Thus, for a POSIX system, one could use:
>   execlp ("sh", "sh", "-c", command, (char*) 0);

Actually, via "confstr(_CS_PATH, buf, sizeof(buf))" one can find the path.
Nevertheless, calling "sh" assumes that one has a POSIX system; I am pretty
sure that it would fail on MinGW - even though, "system()" works there. Maybe,
one can use some configure check to know whether "sh" is available. However,
the check should be cross-compile friendly.

[Bug libfortran/48961] EXECUTE_COMMAND_LINE(WAIT=.false.) fails on MinGW

[thenlich at users dot sourceforge.net]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=48961

--- Comment #11 from Thomas Henlich <thenlich at users dot sourceforge.net> 2011-05-15 09:13:28 UTC ---
(In reply to comment #9)
> Thus, I do not see how one can solve this better than currently done.

We might call system() in a separate thread instead of a separate process which
is more efficient and would also work on Windows.

[Bug libfortran/48961] EXECUTE_COMMAND_LINE(WAIT=.false.) fails on MinGW

[thenlich at users dot sourceforge.net]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=48961

--- Comment #12 from Thomas Henlich <thenlich at users dot sourceforge.net> 2011-05-15 09:26:02 UTC ---
(In reply to comment #11)
> (In reply to comment #9)
> > Thus, I do not see how one can solve this better than currently done.
> 
> We might call system() in a separate thread instead of a separate process which
> is more efficient and would also work on Windows.

The drawback is that the calling application would not exit before system() has
returned or possibly would abort the running command.

So it seems the current implementation is not so bad after all.

[Bug libfortran/48961] EXECUTE_COMMAND_LINE(WAIT=.false.) fails on MinGW

[dominiq at lps dot ens.fr]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=48961

Dominique d'Humieres <dominiq at lps dot ens.fr> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |WAITING
   Last reconfirmed|                            |2013-06-29
     Ever confirmed|0                           |1

--- Comment #13 from Dominique d'Humieres <dominiq at lps dot ens.fr> ---
>From comment #12
> So it seems the current implementation is not so bad after all.

Could this PR be closed as FIXED?

[Bug libfortran/48961] EXECUTE_COMMAND_LINE(WAIT=.false.) fails on MinGW

[dominiq at lps dot ens.fr]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=48961

Dominique d'Humieres <dominiq at lps dot ens.fr> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|WAITING                     |RESOLVED
         Resolution|---                         |FIXED

--- Comment #14 from Dominique d'Humieres <dominiq at lps dot ens.fr> ---
>From comment #13
> Could this PR be closed as FIXED?

No answer for over 9 months and no activity for almost 3 years. Closing as
FIXED.
Please open a new PR for remaining issues.