[Bug fortran/56149] New: 64 bit gFortran-C interop hidden character argument length passed as 32 bit value

[Bug fortran/56149] New: 64 bit gFortran-C interop hidden character argument length passed as 32 bit value

[paul.laidler at ntlworld dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56149

             Bug #: 56149
           Summary: 64 bit gFortran-C interop hidden character argument
                    length passed as 32 bit value
    Classification: Unclassified
           Product: gcc
           Version: unknown
            Status: UNCONFIRMED
          Severity: enhancement
          Priority: P3
         Component: fortran
        AssignedTo: unassigned@gcc.gnu.org
        ReportedBy: paul.laidler@ntlworld.com


This is not so much a bug but a significant gFortran limitation in my project
to port ClearWin+ to gFortran. Users will not want to edit all their "printf"
calls so that format strings become null terminated. This could be avoided if
the format string length where passed as a 64 bit integer. At the moment
gFortran passes a 32 bit value and allows garbage to remain in the HIDWORD so
the argument cannot safely be distinguished from a 64 bit address.

It would appear that there is no advantage in passing a 32 bit value in this
context.

As an aside, I would also suggest that interop programming would be much
simpler if literal character strings were stored in memory as null-terminated
strings.

[Bug fortran/56149] 64 bit gFortran-C interop hidden character argument length passed as 32 bit value

[kargl at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56149

kargl at gcc dot gnu.org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Priority|P3                          |P5
                 CC|                            |kargl at gcc dot gnu.org

--- Comment #1 from kargl at gcc dot gnu.org 2013-01-30 16:22:57 UTC ---
Do you have a small self-contained example and
the command lines you use to compile the code?

The hidden string length is a 32-bit integer.
It will always be a 32-bit integer, so I do
not see how it can be confused for a 64-bit
address.

[Bug fortran/56149] 64 bit gFortran-C interop hidden character argument length passed as 32 bit value

[paul.laidler at ntlworld dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56149

--- Comment #2 from Paul Laidler <paul.laidler at ntlworld dot com> 2013-01-30 19:58:27 UTC ---
Hi

Many thanks for your reply and interest.

The ClearWin+ function winio@ (winio$ in gFortran) emulates the C function
printf that takes a format string then a variable number of arguments
depending on the content of the format. The function can be programmed in C
on the basis that all arguments are passed as 64 bit addresses until you
get to the length of the format string which has a size well below the
minimum possible address value. Any arguments after that will also be
string lengths. The format string cannot be processed to find out what
arguments are supplied until we know its length.

The 32 bit length values are passed correctly by gFortran but occupy 64
bits on  the stack. However, the HIDWORD is not currently set and will
contain whatever is left from its previous use. Nor do we have a null
terminator to calculate the length of the format string directly unless the
programmer supplies one.

Large amounts of code has been written by our users, some of whom are
looking to port to gFortran in the absence of 64 bit FTN95 (the Silverfrost
Fortran compiler). Hence the interest is porting the ClearWin+ library to a
64 bit DLL that is accessible from gFortran code.

Any help or insight that you have would be greatly appreciated.

Paul Laidler


On 30 January 2013 16:22, kargl at gcc dot gnu.org <gcc-bugzilla@gcc.gnu.org
> wrote:

>
> http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56149
>
> kargl at gcc dot gnu.org changed:
>
>            What    |Removed                     |Added
>
> ----------------------------------------------------------------------------
>            Priority|P3                          |P5
>                  CC|                            |kargl at gcc dot gnu.org
>
> --- Comment #1 from kargl at gcc dot gnu.org 2013-01-30 16:22:57 UTC ---
> Do you have a small self-contained example and
> the command lines you use to compile the code?
>
> The hidden string length is a 32-bit integer.
> It will always be a 32-bit integer, so I do
> not see how it can be confused for a 64-bit
> address.
>
> --
> Configure bugmail: http://gcc.gnu.org/bugzilla/userprefs.cgi?tab=email
> ------- You are receiving this mail because: -------
> You reported the bug.
>

[Bug fortran/56149] 64 bit gFortran-C interop hidden character argument length passed as 32 bit value

[burnus at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56149

Tobias Burnus <burnus at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |burnus at gcc dot gnu.org

--- Comment #3 from Tobias Burnus <burnus at gcc dot gnu.org> 2013-01-30 22:04:57 UTC ---
> This could be avoided if the format string length where passed as a 64 bit
> integer. [...] It would appear that there is no advantage in passing
> a 32 bit value in this context.

There is a big advantage. It's called backward compatibility. gfortran might
change to 64 bit (on systems with 64bit pointers) - but that will only happen,
when the ABI has to be broken.

This will happen when gfortran has to switch over to a new array descriptor;
that might happen for 4.9 or one release later. See also
http://gcc.gnu.org/wiki/LibgfortranAbiCleanup - third bullet item. [Cf. also PR
47552.]


> At the moment gFortran passes a 32 bit value and allows garbage to remain in
> the HIDWORD so the argument cannot safely be distinguished from a 64 bit
> address.

Is this guaranteed to work? I had the impression that stack variables could
have rather small addresses. Okay, even though should be way larger than 1000
and strings are usually shorter.


> As an aside, I would also suggest that interop programming would be much
> simpler if literal character strings were stored in memory as null-terminated
> strings.

I concur, but it might either cause problems with having character strings,
e.g., in COMMON blocks and other places where the storage size is expected to
match the string size. Or one had to copy the string all the time.

Thus, either C has to handle Fortran's version of strings - or one has to
create a C version of the string by appending the '\0'.

Note that often one does not want to do:
   call c_function( str//C_NULL_CHAR )
but rather:
   call c_function( trim(str)//C_NULL_CHAR )
Namely, one wants to remove the trailing blanks.

[Bug fortran/56149] 64 bit gFortran-C interop hidden character argument length passed as 32 bit value

[paul.laidler at ntlworld dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56149

--- Comment #4 from Paul Laidler <paul.laidler at ntlworld dot com> 2013-01-31 08:31:48 UTC ---
Many thanks for your helpful comments.
I take the point, though backwards compatibility can often be maintained by
allowing the size of the relevant integers to vary with the (32/64 bit)
context. For example, in Fortran I use an INTEGER KIIND parameter for
addresses that are either 32 bit or 64 bit depending on the context.

If you are familiar with the relevant gFortran compiler code, is there an
existing command line option to make the change (to pass 64 bit values for
the hidden length of character variables) or could an option be added? I
know that it is a big "ask" but the potential time saving for our users
would be vast.

Either way, your help and insight are much appreciated.

Paul


On 30 January 2013 22:04, burnus at gcc dot gnu.org <
gcc-bugzilla@gcc.gnu.org> wrote:

>
> http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56149
>
> Tobias Burnus <burnus at gcc dot gnu.org> changed:
>
>            What    |Removed                     |Added
>
> ----------------------------------------------------------------------------
>                  CC|                            |burnus at gcc dot gnu.org
>
> --- Comment #3 from Tobias Burnus <burnus at gcc dot gnu.org> 2013-01-30
> 22:04:57 UTC ---
> > This could be avoided if the format string length where passed as a 64
> bit
> > integer. [...] It would appear that there is no advantage in passing
> > a 32 bit value in this context.
>
> There is a big advantage. It's called backward compatibility. gfortran
> might
> change to 64 bit (on systems with 64bit pointers) - but that will only
> happen,
> when the ABI has to be broken.
>
> This will happen when gfortran has to switch over to a new array
> descriptor;
> that might happen for 4.9 or one release later. See also
> http://gcc.gnu.org/wiki/LibgfortranAbiCleanup - third bullet item. [Cf.
> also PR
> 47552.]
>
>
> > At the moment gFortran passes a 32 bit value and allows garbage to
> remain in
> > the HIDWORD so the argument cannot safely be distinguished from a 64 bit
> > address.
>
> Is this guaranteed to work? I had the impression that stack variables could
> have rather small addresses. Okay, even though should be way larger than
> 1000
> and strings are usually shorter.
>
>
> > As an aside, I would also suggest that interop programming would be much
> > simpler if literal character strings were stored in memory as
> null-terminated
> > strings.
>
> I concur, but it might either cause problems with having character strings,
> e.g., in COMMON blocks and other places where the storage size is expected
> to
> match the string size. Or one had to copy the string all the time.
>
> Thus, either C has to handle Fortran's version of strings - or one has to
> create a C version of the string by appending the '\0'.
>
> Note that often one does not want to do:
>    call c_function( str//C_NULL_CHAR )
> but rather:
>    call c_function( trim(str)//C_NULL_CHAR )
> Namely, one wants to remove the trailing blanks.
>
> --
> Configure bugmail: http://gcc.gnu.org/bugzilla/userprefs.cgi?tab=email
> ------- You are receiving this mail because: -------
> You reported the bug.
>

[Bug fortran/56149] 64 bit gFortran-C interop hidden character argument length passed as 32 bit value

[sgk at troutmask dot apl.washington.edu]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56149

--- Comment #5 from Steve Kargl <sgk at troutmask dot apl.washington.edu> 2013-01-31 15:54:56 UTC ---
On Wed, Jan 30, 2013 at 07:58:27PM +0000, paul.laidler at ntlworld dot com
wrote:
> --- Comment #2 from Paul Laidler <paul.laidler at ntlworld dot com> 2013-01-30 19:58:27 UTC ---
> 
> Many thanks for your reply and interest.
> 
> The ClearWin+ function winio@ (winio$ in gFortran) emulates the C function
> printf that takes a format string then a variable number of arguments
> depending on the content of the format. The function can be programmed in C
> on the basis that all arguments are passed as 64 bit addresses until you
> get to the length of the format string which has a size well below the
> minimum possible address value. Any arguments after that will also be
> string lengths. The format string cannot be processed to find out what
> arguments are supplied until we know its length.

My only interest here is to clearly define the issue.  With that
in mind, is winio a function written in C (or at least has a C
prototype) with the form "int winio(char *string, ...);"?

If the answer is "yes", then the issue is not related to gfortran's
implementation of the ISO C Binding feature.  The Fortran 2003 
standard specifically states that vardiac functions are not covered
under BIND(C): 

  NOTE 15.21
  The C language allows specification of a C function that can take a
  variable number of arguments (C standard, 7.15).  This standard does
  not provide a mechanism for Fortran procedures to interoperate with
  such C functions.

It seems to me that the only portable solution to your issue is to
tell your users that the proper way to call winio from Fortran is

   call winio("Some string" // C_NULL_CHAR, ...);
or
   call winio(trim(string) // C_NULL_CHAR, ...);

In your implementation of winio, you will now have a terminated 
string.  You can parse the string and not depend on the hidden
length.

As Tobias notes, changing the hidden length from a 32-bit integer
to a 64-bit integer breaks backwards compatibility.  The gfortran
developer would now need to tell everyone else that may be 
mucking around with the hidden length argument to change their
code.  Everyone with libraries and modules that use Fortran strings
would also likely need to recompile.

[Bug fortran/56149] 64 bit gFortran-C interop hidden character argument length passed as 32 bit value

[paul.laidler at ntlworld dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56149

--- Comment #6 from Paul Laidler <paul.laidler at ntlworld dot com> 2013-01-31 16:29:46 UTC ---
Yes, winio is written in C with the binding "int winio(char *string, ...)"
and it is not Fortran standard conforming.

In my opinion there should be no problem with backwards compatibility but I
am thinking of a Windows environment and have no inside knowledge of how
the C code produced by gFortran is passed on to gcc.



On 31 January 2013 15:54, sgk at troutmask dot apl.washington.edu <
gcc-bugzilla@gcc.gnu.org> wrote:

>
> http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56149
>
> --- Comment #5 from Steve Kargl <sgk at troutmask dot apl.washington.edu>
> 2013-01-31 15:54:56 UTC ---
> On Wed, Jan 30, 2013 at 07:58:27PM +0000, paul.laidler at ntlworld dot com
> wrote:
> > --- Comment #2 from Paul Laidler <paul.laidler at ntlworld dot com>
> 2013-01-30 19:58:27 UTC ---
> >
> > Many thanks for your reply and interest.
> >
> > The ClearWin+ function winio@ (winio$ in gFortran) emulates the C
> function
> > printf that takes a format string then a variable number of arguments
> > depending on the content of the format. The function can be programmed
> in C
> > on the basis that all arguments are passed as 64 bit addresses until you
> > get to the length of the format string which has a size well below the
> > minimum possible address value. Any arguments after that will also be
> > string lengths. The format string cannot be processed to find out what
> > arguments are supplied until we know its length.
>
> My only interest here is to clearly define the issue.  With that
> in mind, is winio a function written in C (or at least has a C
> prototype) with the form "int winio(char *string, ...);"?
>
> If the answer is "yes", then the issue is not related to gfortran's
> implementation of the ISO C Binding feature.  The Fortran 2003
> standard specifically states that vardiac functions are not covered
> under BIND(C):
>
>   NOTE 15.21
>   The C language allows specification of a C function that can take a
>   variable number of arguments (C standard, 7.15).  This standard does
>   not provide a mechanism for Fortran procedures to interoperate with
>   such C functions.
>
> It seems to me that the only portable solution to your issue is to
> tell your users that the proper way to call winio from Fortran is
>
>    call winio("Some string" // C_NULL_CHAR, ...);
> or
>    call winio(trim(string) // C_NULL_CHAR, ...);
>
> In your implementation of winio, you will now have a terminated
> string.  You can parse the string and not depend on the hidden
> length.
>
> As Tobias notes, changing the hidden length from a 32-bit integer
> to a 64-bit integer breaks backwards compatibility.  The gfortran
> developer would now need to tell everyone else that may be
> mucking around with the hidden length argument to change their
> code.  Everyone with libraries and modules that use Fortran strings
> would also likely need to recompile.
>
> --
> Configure bugmail: http://gcc.gnu.org/bugzilla/userprefs.cgi?tab=email
> ------- You are receiving this mail because: -------
> You reported the bug.
>

[Bug fortran/56149] 64 bit gFortran-C interop hidden character argument length passed as 32 bit value

[sgk at troutmask dot apl.washington.edu]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56149

--- Comment #7 from Steve Kargl <sgk at troutmask dot apl.washington.edu> 2013-02-01 05:21:58 UTC ---
On Thu, Jan 31, 2013 at 04:29:46PM +0000, paul.laidler at ntlworld dot com
wrote:
> --- Comment #6 from Paul Laidler <paul.laidler at ntlworld dot com> 2013-01-31 16:29:46 UTC ---
> Yes, winio is written in C with the binding "int winio(char *string, ...)"
> and it is not Fortran standard conforming.
> 
> In my opinion there should be no problem with backwards compatibility but I
> am thinking of a Windows environment and have no inside knowledge of how
> the C code produced by gFortran is passed on to gcc.
> 

Contrary to your opinion, the simply change of simply setting

Index: trans-types.c
===================================================================
--- trans-types.c       (revision 195396)
+++ trans-types.c       (working copy)
@@ -968,7 +968,7 @@ gfc_init_types (void)
   boolean_false_node = build_int_cst (boolean_type_node, 0);

   /* ??? Shouldn't this be based on gfc_index_integer_kind or so?  */
-  gfc_charlen_int_kind = 4;
+  gfc_charlen_int_kind = 8;
   gfc_charlen_type_node = gfc_get_int_type (gfc_charlen_int_kind);
 }

which gives a 64-bit integer for the hidden length parameter
yields

                === gfortran Summary ===

# of expected passes            17592
# of unexpected failures        2743
# of unexpected successes       18
# of expected failures          40
# of unresolved testcases       56
# of unsupported tests          119
/usr/home/kargl/gcc/obj4x/gcc/testsuite/gfortran/../../gfortran  version 4.8.0
20130123 (experimental) (GCC) 

and I stopped the regression test it was taking way too long to
time out failing test cases.

Quitely frankly, I can't image anyone will ever need a Fortran
string with a length of 2**31.

[Bug fortran/56149] 64 bit gFortran-C interop hidden character argument length passed as 32 bit value

[kargl at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56149

kargl at gcc dot gnu.org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|kargl at gcc dot gnu.org           |

--- Comment #8 from kargl at gcc dot gnu.org ---
Remove myself from cc list.

[Bug fortran/56149] 64 bit gFortran-C interop hidden character argument length passed as 32 bit value

[dominiq at lps dot ens.fr]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=56149

Dominique d'Humieres <dominiq at lps dot ens.fr> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |WAITING
   Last reconfirmed|                            |2015-10-09
     Ever confirmed|0                           |1

--- Comment #9 from Dominique d'Humieres <dominiq at lps dot ens.fr> ---
Is there still some interest for this PR?