[Bug fortran/61632] New: memory corruption in Fortran RTL when writing formatted data

[Bug fortran/61632] New: memory corruption in Fortran RTL when writing formatted data

[arnaud02 at users dot sourceforge.net]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61632

            Bug ID: 61632
           Summary: memory corruption in Fortran RTL when writing
                    formatted data
           Product: gcc
           Version: 4.8.2
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: fortran
          Assignee: unassigned at gcc dot gnu.org
          Reporter: arnaud02 at users dot sourceforge.net

Problem 1: using the following program:
      program p
      call ss()
      call ss()
      end program p
      subroutine ss
      CHARACTER(3), save :: ZTYP(3)
      DATA ZTYP /'XXX','YYY','ZZZ'/
      write(*,600,IOSTAT=iosa) 0.0,ZTYP
      write(*,*) 'iostat=',iosa
 600  FORMAT(1PE13.5,4X,A3)
      end subroutine ss

with gfortran 4.9.0 results in a executable that crashes:
  0.00000E+00    XXX
 iostat=        5006
  0.00000E+00    XXX

Program received signal SIGSEGV: Segmentation fault - invalid memory reference.

A run under valgrind shows:
  0.00000E+00    XXX
 iostat=        5006
  0.00000E+00    XXX
==13768== Invalid read of size 8
==13768==    at 0x414250: _gfortrani_format_error (format.c:1147)
==13768==    by 0x4047D6: require_type.part.7 (transfer.c:1158)
==13768==    by 0x406718: formatted_transfer (transfer.c:1150)
==13768==    by 0x404AE5: _gfortran_transfer_array (transfer.c:2170)

gfortran 4.7.1 is not affected but gfortran 4.8.2 and 4.9.0 are.


Problem 2: using the following program:
      program p
      call ss()
      call ss()
      end program p
      subroutine ss
      CHARACTER(3), save :: ZTYP(3)
      DATA ZTYP /'XXX','YYY','ZZZ'/
      write(*,600) 0.0,ZTYP
 600  FORMAT(1PE13.5,A3)
      end subroutine ss

with gfortran 4.9.0 results in a executable that produces:
--
  0.00000E+00XXX
At line 8 of file io2.f (unit = 6, file = 'stdout')
Fortran runtime error: Expected REAL for item 3 in formatted transfer, got
CHARACTER
(1PE13.5,A3)
   ^
--
The message is unclear as there are only 2 items. Moreover the expected type
looks incorrect. gfortran 4.6.3, 4.7.2, 4.8.2 and 4.9.0 are affected.

[Bug fortran/61632] memory corruption in Fortran RTL when writing formatted data

[dominiq at lps dot ens.fr]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61632

Dominique d'Humieres <dominiq at lps dot ens.fr> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |NEW
   Last reconfirmed|                            |2014-06-27
                 CC|                            |jvdelisle at gcc dot gnu.org
     Ever confirmed|0                           |1

--- Comment #2 from Dominique d'Humieres <dominiq at lps dot ens.fr> ---
      program p
      call ss()
      call ss()
      end program p
      subroutine ss
      CHARACTER(3), save :: ZTYP(3)
      DATA ZTYP /'XXX','YYY','ZZZ'/
      write(*,600,IOSTAT=iosa) 0.0,ZTYP
      write(*,*) 'iostat=',iosa
      if (iosa /= 0) print *, "error"
 600  FORMAT(1PE13.5,4X,A3)
      end subroutine ss

gives at run time

[Book15] f90/bug% a.out
  0.00000E+00    XXX
 iostat=        5006
 error
  0.00000E+00    XXX
 iostat=        5006
 error

which is right, but also

[Book15] f90/bug% a.out
  0.00000E+00    XXX
 iostat=        5006
 error
  0.00000E+00    XXX

Program received signal SIGSEGV: Segmentation fault - invalid memory reference.

Backtrace for this error:
#0  0x10b4c3892
#1  0x10b4c4030
#2  0x7fff8a4695a9
#3  0x10b581910
#4  0x10b58daf6
#5  0x10b58fd12
#6  0x10b58de05
#7  0x10b4b9cef
#8  0x10b4b9dfb
#9  0x10b4b9e34
Segmentation fault

which is wrong.

Note the behavior for "problem 2" is what I expect.

[Bug fortran/61632] memory corruption in Fortran RTL when writing formatted data

[dominiq at lps dot ens.fr]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61632

--- Comment #3 from Dominique d'Humieres <dominiq at lps dot ens.fr> ---
With the following reduced test

      program p
      CHARACTER(3), save :: ZTYP(3)
      DATA ZTYP /'XXX','YYY','ZZZ'/
      write(*,600,IOSTAT=iosa) 0.0,ZTYP
      if (iosta /= 0) print *, 'error'
      write(*,600,IOSTAT=iosa) 0.0,ZTYP
      if (iosta /= 0) print *, 'error'
 600  FORMAT(1PE13.5,4X,A3)
      end program p

valgrind reports

==95142== Memcheck, a memory error detector
==95142== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al.
==95142== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info
==95142== Command: a.out
==95142== 
  0.00000E+00    XXX
 error
  0.00000E+00    XXX
==95142== Invalid write of size 8
==95142==    at 0x1000DCF40: _gfortrani_format_error (in
/opt/gcc/gcc4.10w/lib/libgfortran.3.dylib)
==95142==    by 0x6F66204C4145521F: ???
==95142==    by 0x33206D6574692071: ???
==95142==    by 0x6D726F66206E691F: ???
==95142==    by 0x7274206465747460: ???
==95142==    by 0x202C726566736E60: ???
==95142==    by 0x5241484320746F66: ???
==95142==    by 0xA5245544340: ???
==95142==    by 0x32AAABA6: ???
==95142==  Address 0x7fff5f7adda9 is not stack'd, malloc'd or (recently) free'd
==95142== 
==95142== Invalid read of size 8
==95142==    at 0x1000DCF45: _gfortrani_format_error (in
/opt/gcc/gcc4.10w/lib/libgfortran.3.dylib)
==95142==    by 0x6F66204C4145521F: ???
==95142==    by 0x33206D6574692071: ???
==95142==    by 0x6D726F66206E691F: ???
==95142==    by 0x7274206465747460: ???
==95142==    by 0x202C726566736E60: ???
==95142==    by 0x5241484320746F66: ???
==95142==    by 0xA5245544340: ???
==95142==    by 0x32AAABA6: ???
==95142==  Address 0x10045166c is not stack'd, malloc'd or (recently) free'd
==95142== 
==95142== 
==95142== Process terminating with default action of signal 11 (SIGSEGV)
==95142==  General Protection Fault
==95142==    at 0x100298FC1: dyld_stub_binder (in /usr/lib/libSystem.B.dylib)
==95142==    by 0x10013502F: ??? (in /opt/gcc/gcc4.10w/lib/libgfortran.3.dylib)
==95142==    by 0x10013502F: ??? (in /opt/gcc/gcc4.10w/lib/libgfortran.3.dylib)
==95142==    by 0x6F66204C4145521F: ???
==95142==    by 0x33206D6574692071: ???
==95142==    by 0x6D726F66206E691F: ???
==95142==    by 0x7274206465747460: ???
==95142==    by 0x202C726566736E60: ???
==95142==    by 0x5241484320746F66: ???
==95142==    by 0xA5245544340: ???
==95142==    by 0x4FFFFF: ???
==95142== 
==95142== HEAP SUMMARY:
==95142==     in use at exit: 7,732 bytes in 21 blocks
==95142==   total heap usage: 23 allocs, 2 frees, 8,244 bytes allocated
==95142== 
==95142== LEAK SUMMARY:
==95142==    definitely lost: 0 bytes in 0 blocks
==95142==    indirectly lost: 0 bytes in 0 blocks
==95142==      possibly lost: 0 bytes in 0 blocks
==95142==    still reachable: 7,644 bytes in 20 blocks
==95142==         suppressed: 88 bytes in 1 blocks
==95142== Rerun with --leak-check=full to see details of leaked memory
==95142== 
==95142== For counts of detected and suppressed errors, rerun with: -v
==95142== ERROR SUMMARY: 380 errors from 2 contexts (suppressed: 0 from 0)
Segmentation fault

[Bug fortran/61632] memory corruption in Fortran RTL when writing formatted data

[jvdelisle at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61632

--- Comment #4 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> ---
Created attachment 33114
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=33114&action=edit
Patch to fix this issue.

Th eproblem was some use of pointers in a not so reliable way, calculating
offsets from the pointer values.  This uses the string lengths to calculate it
and avoids the issue completely.

[Bug fortran/61632] memory corruption in Fortran RTL when writing formatted data

[dominiq at lps dot ens.fr]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61632

--- Comment #5 from Dominique d'Humieres <dominiq at lps dot ens.fr> ---
With the patch in comment 4, the caret is shifted one character to the left and
there are some missing characters in the format string. This causes the failure
of the tests gfortran.dg/fmt_error_4.f90 and gfortran.dg/fmt_error_5.f90

Fortran runtime error: Unexpected element 'Q' in format
Q, A)                   <- with patch
   ^
(A, Q, A)               <- unpatched
    ^

Fortran runtime error: Unexpected element 'Q' in format
Q)                   <- with patch
^
(Q)                  <- unpatched
 ^

[Bug fortran/61632] memory corruption in Fortran RTL when writing formatted data

[jvdelisle at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61632

--- Comment #6 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> ---
yes, I need to adjust the offset and width calculation.  Updated patch is in
the works.

[Bug fortran/61632] memory corruption in Fortran RTL when writing formatted data

[jvdelisle at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61632

Jerry DeLisle <jvdelisle at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Assignee|unassigned at gcc dot gnu.org      |jvdelisle at gcc dot gnu.org

--- Comment #7 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> ---
Patch awaiting approval

[Bug fortran/61632] memory corruption in Fortran RTL when writing formatted data

[dominiq at lps dot ens.fr]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61632

--- Comment #8 from Dominique d'Humieres <dominiq at lps dot ens.fr> ---
Sorry! There is still a glitch: the following code

      program p
      call ss()
      call ss()
      end program p
      subroutine ss
      CHARACTER(3), save :: ZTYP(3)
      DATA ZTYP /'XXX','YYY','ZZZ'/
      write(*,600) 0.0,ZTYP
 600  FORMAT(1PE13.5,A3)
      end subroutine ss

used to give the runtime error

  0.00000E+00XXX
At line 8 of file pr61632_1.f90 (unit = 6, file = 'stdout')
Fortran runtime error: Expected REAL for item 3 in formatted transfer, got
CHARACTER
(1PE13.5,A3)
   ^

and I think the position of the caret is right. With the patch at
https://gcc.gnu.org/ml/fortran/2014-07/msg00107.html
it is

  0.00000E+00XXX
At line 8 of file pr61632_1.f90 (unit = 6, file = 'stdout')
Fortran runtime error: Expected REAL for item 3 in formatted transfer, got
CHARACTER
(1PE13.5,A3)
           ^
i.e., the caret position does not seem right. Your patch probably does not take
into account the fact that the third item of the output uses the first item of
the format (periodic "extension"). 

Otherwise I have run

make -k check-gfortran RUNTESTFLAGS="dg.exp=fmt*
--target_board=unix'{-m32,-m64}'"

without regression.

[Bug fortran/61632] memory corruption in Fortran RTL when writing formatted data

[jvdelisle at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61632

--- Comment #9 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> ---
Its not really a glitch.  In this case reversion has occurred so i can only
approximate where in the string the error occured.  I can improve on it with:

  offset = fmt->reversion_ok ? fmt->format_string_len + 2:
                   dtp->format_len - fmt->format_string_len;

I don't think I can do better for now.  Of course we want diagnostics to be
helpful, but they can not be perfect.  I would suggest we commit the patch for
now.

[Bug fortran/61632] Memory corruption on error when writing formatted data

[dominiq at lps dot ens.fr]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61632

--- Comment #10 from Dominique d'Humieres <dominiq at lps dot ens.fr> ---
The following code

      program p
      call ss0()
      end program p
      subroutine ss0
      CHARACTER(3), save :: ZTYP(3)
      DATA ZTYP /'XXX','YYY','ZZZ'/
      write(*,600,IOSTAT=iosa) 'AAA',0.0,ZTYP
 600  FORMAT(A3,1PE13.5,A3)
      print *,
      print *, 'iostat =', iosa
      end subroutine ss0

outputs

AAA  0.00000E+00XXX
YYY
 iostat =        5006

Is the printing of 'YYY' supposed to happen?

[Bug fortran/61632] Memory corruption on error when writing formatted data

[dominiq at lps dot ens.fr]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61632

--- Comment #11 from Dominique d'Humieres <dominiq at lps dot ens.fr> ---
> Its not really a glitch.  In this case reversion has occurred so i can only
> approximate where in the string the error occured.  I can improve on it with:
>
>   offset = fmt->reversion_ok ? fmt->format_string_len + 2:
> 			       dtp->format_len - fmt->format_string_len;
>
> I don't think I can do better for now.  Of course we want diagnostics to be
> helpful, but they can not be perfect.  I would suggest we commit the patch for now.

What about the following?

   if (f != NULL)
-    fmt->format_string = f->source;
+    fmt->format_string_len = strrchr (f->source, ')') - &f->source[0] + 1;

It is clumsy, but it does the trick. If accepted, I can write a test case for
it.
>From gcc-bugs-return-456318-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org Mon Jul 14 18:37:57 2014
Return-Path: <gcc-bugs-return-456318-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org>
Delivered-To: listarch-gcc-bugs@gcc.gnu.org
Received: (qmail 333 invoked by alias); 14 Jul 2014 18:37:56 -0000
Mailing-List: contact gcc-bugs-help@gcc.gnu.org; run by ezmlm
Precedence: bulk
List-Id: <gcc-bugs.gcc.gnu.org>
List-Archive: <http://gcc.gnu.org/ml/gcc-bugs/>
List-Post: <mailto:gcc-bugs@gcc.gnu.org>
List-Help: <mailto:gcc-bugs-help@gcc.gnu.org>
Sender: gcc-bugs-owner@gcc.gnu.org
Delivered-To: mailing list gcc-bugs@gcc.gnu.org
Received: (qmail 32462 invoked by uid 48); 14 Jul 2014 18:37:43 -0000
From: "manu at gcc dot gnu.org" <gcc-bugzilla@gcc.gnu.org>
To: gcc-bugs@gcc.gnu.org
Subject: [Bug c++/61803] error reports macro location first, but should not
Date: Mon, 14 Jul 2014 18:37:00 -0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: changed
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Product: gcc
X-Bugzilla-Component: c++
X-Bugzilla-Version: unknown
X-Bugzilla-Keywords:
X-Bugzilla-Severity: normal
X-Bugzilla-Who: manu at gcc dot gnu.org
X-Bugzilla-Status: NEW
X-Bugzilla-Priority: P3
X-Bugzilla-Assigned-To: unassigned at gcc dot gnu.org
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Flags:
X-Bugzilla-Changed-Fields: bug_status cf_reconfirmed_on cc component everconfirmed
Message-ID: <bug-61803-4-pNj5AWCcQK@http.gcc.gnu.org/bugzilla/>
In-Reply-To: <bug-61803-4@http.gcc.gnu.org/bugzilla/>
References: <bug-61803-4@http.gcc.gnu.org/bugzilla/>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
X-Bugzilla-URL: http://gcc.gnu.org/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0
X-SW-Source: 2014-07/txt/msg00909.txt.bz2
Content-length: 1832

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61803

Manuel López-Ibáñez <manu at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |NEW
   Last reconfirmed|                            |2014-07-14
                 CC|                            |manu at gcc dot gnu.org
          Component|c                           |c++
     Ever confirmed|0                           |1

--- Comment #1 from Manuel López-Ibáñez <manu at gcc dot gnu.org> ---
(In reply to Tom Tromey from comment #0)
> This is not really a C-specific bug but I wasn't sure
> where else to file it.

I would suggest file it against C++, just because there are more regular
contributors to C++ (Jason and Paolo) than to C (only Joseph and not nearly as
regular).

In C++, we get:

test.c:1:23: error: cannot convert ‘int*’ to ‘double*’ in return
 #define Nil ((int *) 0)
                       ^
test.c:5:10: note: in expansion of macro ‘Nil’
   return Nil;
          ^

It is strange that it points to the closing parenthesis.

> I think instead it would be better to emit the warning at the "final"
> location and then print the macro definition information as notes.

In this case yes, but this is not always the case: See PR5252. Perhaps this
should be configurable per diagnostic, with the most common case being the
default. The default of Clang is the opposite of GCC, for this very reason:

test.c:5:10: error: cannot initialize return object of type 'double *' with an
rvalue of type 'int *'
  return Nil;
         ^~~
test.c:1:13: note: expanded from macro 'Nil'
#define Nil ((int *) 0)
            ^~~~~~~~~~~
>From gcc-bugs-return-456319-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org Mon Jul 14 18:57:52 2014
Return-Path: <gcc-bugs-return-456319-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org>
Delivered-To: listarch-gcc-bugs@gcc.gnu.org
Received: (qmail 7681 invoked by alias); 14 Jul 2014 18:57:50 -0000
Mailing-List: contact gcc-bugs-help@gcc.gnu.org; run by ezmlm
Precedence: bulk
List-Id: <gcc-bugs.gcc.gnu.org>
List-Archive: <http://gcc.gnu.org/ml/gcc-bugs/>
List-Post: <mailto:gcc-bugs@gcc.gnu.org>
List-Help: <mailto:gcc-bugs-help@gcc.gnu.org>
Sender: gcc-bugs-owner@gcc.gnu.org
Delivered-To: mailing list gcc-bugs@gcc.gnu.org
Received: (qmail 7402 invoked by uid 48); 14 Jul 2014 18:57:39 -0000
From: "law at redhat dot com" <gcc-bugzilla@gcc.gnu.org>
To: gcc-bugs@gcc.gnu.org
Subject: [Bug tree-optimization/61757] [4.10 Regression] genmodes failure with enable-checking
Date: Mon, 14 Jul 2014 18:57:00 -0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: changed
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Product: gcc
X-Bugzilla-Component: tree-optimization
X-Bugzilla-Version: 4.10.0
X-Bugzilla-Keywords: build
X-Bugzilla-Severity: normal
X-Bugzilla-Who: law at redhat dot com
X-Bugzilla-Status: RESOLVED
X-Bugzilla-Priority: P1
X-Bugzilla-Assigned-To: rguenth at gcc dot gnu.org
X-Bugzilla-Target-Milestone: 4.10.0
X-Bugzilla-Flags:
X-Bugzilla-Changed-Fields: cc
Message-ID: <bug-61757-4-8rhpSOjCdd@http.gcc.gnu.org/bugzilla/>
In-Reply-To: <bug-61757-4@http.gcc.gnu.org/bugzilla/>
References: <bug-61757-4@http.gcc.gnu.org/bugzilla/>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-Bugzilla-URL: http://gcc.gnu.org/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0
X-SW-Source: 2014-07/txt/msg00910.txt.bz2
Content-length: 737

https://gcc.gnu.org/bugzilla/show_bug.cgi?ida757

Jeffrey A. Law <law at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |law at redhat dot com

--- Comment #30 from Jeffrey A. Law <law at redhat dot com> ---
Richi -- in response to your question in c#26.  I'll be looking at a handful of
DOM issues while in transit to the Cauldron.  That'll include another
evaluation of whether or not we want to thread across back edges at all.

Invalidation of equivalences after traversing a backedge has been extremely
problematical and I really question it's value in the real world.

[Bug fortran/61632] Memory corruption on error when writing formatted data

[jvdelisle at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61632

--- Comment #13 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> ---
This:

+    fmt->format_string_len = strrchr (f->source, ')') - f->source + 1;

Is taking the difference between two string pointers, ie memory addresses


This:

printf("pos 0 =%x, pos ) =%x\n",strchr (f->source, '\0'),strrchr (f->source,
')'));

Is printing the value of the pointers, the addresses.

Are you expecting something different?

[Bug fortran/61632] Memory corruption on error when writing formatted data

[dominiq at lps dot ens.fr]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61632

--- Comment #15 from Dominique d'Humieres <dominiq at lps dot ens.fr> ---
> Its sort of like Steve said earlier.  The coder is choosing to ignore an
> error condition so anything gfortran does is valid.  In this case the
> output got writen to buffer before the error occurred.  You are bypassing
> all of the normal error handling when you do this. The buffer got flushed.
>
> Its not worth the time to change any of it really.
>
> The program should exit, its invalid.

Well, I disagree with this interpretation: I did not find anywhere in the
standard that the variable set by iostat has to be used. In addition, the
following code should be valid upon Steve's rules and still prints 'YYY' (g95
does not).

[Book15] f90/bug% cat pr61632_1_red.f90
      program p
      call ss0()
      end program p
      subroutine ss0
      CHARACTER(3), save :: ZTYP(3)
      DATA ZTYP /'XXX','YYY','ZZZ'/
      write(*,600,ERR=10,IOSTAT=iosa) 'AAA',0.0,ZTYP
 600  FORMAT(A3,1PE13.5,A3)
      print *, 'success'
      return

 10   print *,
      print *, 'iostat =', iosa
      end subroutine ss0
[Book15] f90/bug% a.out
AAA  0.00000E+00XXX
YYY
 iostat =        5006

IMO this PR boils down to the fact that the I/O processing is not reset to a
clean state after an error is handled with ERR, IOSTAT, ... , as it should.

Along this line, I have found that 'parse_format' is called only once even if
'ss' is called twice.

[Bug fortran/61632] Memory corruption on error when writing formatted data

[dominiq at lps dot ens.fr]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61632

--- Comment #16 from Dominique d'Humieres <dominiq at lps dot ens.fr> ---
> This:
>
> +    fmt->format_string_len = strrchr (f->source, ')') - f->source + 1;
>
>Is taking the difference between two string pointers, ie memory addresses
>
> This:
>
> printf("pos 0 =%x, pos ) =%x\n",strchr (f->source, '\0'),strrchr (f->source, ')'));
>
> Is printing the value of the pointers, the addresses.
>
> Are you expecting something different?

Well, I am fully aware that I am C challenged (the kind who can forget two
semicolons while writing a single line!). However the above is more or less
part of what I understand.

To clarify my question, let me summarize what I understand:

(1) This PR occurs iff 'f != NULL',
(2) The beginning of f->source is the unprocessed part of the format when the
error occurs, the last character of it being the closing ')' of the format,
(3) while I was expecting f->source being

unprocessed_part\0garbage

I have examples for which I see


unprocessed_partxxx\0garbage

where xxx are some extra characters (from 1 to ~15). My question was about the
origin of these characters.

Indeed I was not happy with 'strrchr (f->source, ')')' because it could find a
')' in the garbage after '\0'. In addition, valgrind complained about it.

I have regtested and ran my own tests with the following change

   if (f != NULL)
-    fmt->format_string = f->source;
+    {
+      width = strlen (f->source);
+      for (i = width - 1; i > 0; i--)
+     {
+       if (f->source[i] == ')')
+         break;
+       width--;
+     }
+      fmt->format_string_len = width;
+    }

This makes valgrind happy without regression (I know that this will fail if the
extra characters contain a ')', so far I did not crossed this situation).

[Bug fortran/61632] Memory corruption on error when writing formatted data

[sgk at troutmask dot apl.washington.edu]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61632

--- Comment #17 from Steve Kargl <sgk at troutmask dot apl.washington.edu> ---
On Tue, Jul 15, 2014 at 09:08:44AM +0000, dominiq at lps dot ens.fr wrote:
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61632
> 
> --- Comment #15 from Dominique d'Humieres <dominiq at lps dot ens.fr> ---
> > Its sort of like Steve said earlier.  The coder is choosing to ignore an
> > error condition so anything gfortran does is valid.  In this case the
> > output got writen to buffer before the error occurred.  You are bypassing
> > all of the normal error handling when you do this. The buffer got flushed.
> >
> > Its not worth the time to change any of it really.
> >
> > The program should exit, its invalid.
> 
> Well, I disagree with this interpretation: I did not find anywhere in the
> standard that the variable set by iostat has to be used.

I did not say that iostat had to be used.  However, one can find 
things like:

9.10.1     Error conditions and the ERR= specifier

If an error condition occurs during execution of an input/output
statement, the position of the file becomes indeterminate.

9.2.3.3   File position after data transfer

If an error condition (9.10) occurred, the position of the file
is indeterminate.

If one explicitly uses IOSTAT= in an IO statement, it may be
prudent to actually check its value.

[Bug fortran/61632] Memory corruption on error when writing formatted data

[dominiq at lps dot ens.fr]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61632

--- Comment #18 from Dominique d'Humieres <dominiq at lps dot ens.fr> ---
> I did not say that iostat had to be used.  However, one can find 
> things like:
>
> 9.10.1     Error conditions and the ERR= specifier
>
> If an error condition occurs during execution of an input/output
> statement, the position of the file becomes indeterminate.
>
> 9.2.3.3   File position after data transfer
>
> If an error condition (9.10) occurred, the position of the file
> is indeterminate.
>
> If one explicitly uses IOSTAT= in an IO statement, it may be
> prudent to actually check its value.

AFAICT this does not change the problem (see the tests in comments 2, 3, and
15). Now if "the position of the file becomes indeterminate", I believe it
should be restored to a defined state after the error as been processed, either
after the jump to the error label for ERR= or after setting the value of
IOSTAT, see my comment

> Along this line, I have found that 'parse_format' is called only once even
> if 'ss' is called twice.

in comment 15.

[Bug fortran/61632] Memory corruption on error when writing formatted data

[jvdelisle at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61632

--- Comment #19 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> ---
(In reply to Dominique d'Humieres from comment #15)
> > Its sort of like Steve said earlier.  The coder is choosing to ignore an
> > error condition so anything gfortran does is valid.  In this case the
> > output got writen to buffer before the error occurred.  You are bypassing
> > all of the normal error handling when you do this. The buffer got flushed.
> >
> > Its not worth the time to change any of it really.
> >
> > The program should exit, its invalid.

I simply mean the Fortran code in the test case is invalid Fortran. Fix the
Fortran code.  Now suppose the error is a disk full.  Should the user just
ignore that and keep trying to write to the disk?

> 
> Well, I disagree with this interpretation: I did not find anywhere in the
> standard that the variable set by iostat has to be used. In addition, the
> following code should be valid upon Steve's rules and still prints 'YYY'
---snip---

Regarding the "YYY".  Why not print it? the error occurs after it is written. 
If you do not "catch" the error, the runtime leaves early, out of all the
various processing loops, and exits the program.  Whereas when the error is
caught, we continue execution in the library where we detected the error and
happily proceeds.  The transfer is directed by the format string, it went as
far as it could go. (As opposed to list directed I/O)

> 
> IMO this PR boils down to the fact that the I/O processing is not reset to a
> clean state after an error is handled with ERR, IOSTAT, ... , as it should.

The standard uses the term indeterminate.  You prefer it to be determined. OK

Person A wants the file buffers flushed, the file closed, and reopened with
status="append". Person B wants to grab the file position, clear the buffers,
rewind the file, read it back in, parse it for bad content, and signal an alarm
on the console.  The user can do these things if they want to.

> 
> Along this line, I have found that 'parse_format' is called only once even
> if 'ss' is called twice.

This is by design.  The string is parsed once for efficiency and reused for
performance purposes. Its a choice we made some time ago. The runtime checks
the string for a match to previously parsed strings.  Saves a lot of run time.
The parsed (tokenized) format is saved elsewhere and reused.

[Bug fortran/61632] Memory corruption on error when writing formatted data

[jvdelisle at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61632

Jerry DeLisle <jvdelisle at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #33114|0                           |1
        is obsolete|                            |

--- Comment #20 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> ---
Created attachment 33155
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=33155&action=edit
Revised patch

This patch borrows on Dominique's idea of using strchr to find the end of the
format string.

Tested OK on x86_64 and the various cases presented in this PR.

[Bug fortran/61632] Memory corruption on error when writing formatted data

[dominiq at lps dot ens.fr]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61632

--- Comment #22 from Dominique d'Humieres <dominiq at lps dot ens.fr> ---
> Created attachment 33160 [details]
> Updated patch taking care of valgrind error
>
> This patch includes fixing the valgrind error by allocating and setting
> the NULL byte at the end of the format string.

With this patch all my tests for this PR pass without problem: no more memory
corruption and the caret is at the right position!-).

I have two questions:

(1) my understanding of 

  offset = (j > 60) ? j - 40 : 0;

  j -= offset;
  width = dtp->format_len - offset;

is to set j to 40 and to decrease width by j-40 if j>60.

This part is now removed, should not we limit offset to something?

(2) Do you need me to package one or two of my tests for the caret position?

Indeed I'ld perfectly happy if the behavior reported in comment 10 would be at
least understood. Note that with the code

      program p
      call ss0()
      call ss()
      end program p
      subroutine ss0
      CHARACTER(3), save :: ZTYP(4)
      DATA ZTYP /'WWW','XXX','YYY','ZZZ'/
      write(10,500,err=10,iostat=iosa) 'AAA','BBB',0.0,ZTYP
 500  FORMAT(2(A3),1PE13.5,A3)
      return
 10   write(10, *)
      write(10, *) 'iostat =', iosa
      end subroutine ss0
      subroutine ss
      CHARACTER(3), save :: ZTYP(4)
      DATA ZTYP /'WWW','XXX','YYY','ZZZ'/
      write(10,600) 'AAA','BBB',0.0,ZTYP
 600  FORMAT(A3,1PE13.5,2(A3))
      end subroutine ss

gives at run time

[Book15] f90/bug% cat fort.10
AAABBB  0.00000E+00WWW

 iostat =        5006

but my mother is saying "le mieux est l'ennemi du bien" (~ the best is against
the good) so I think this should not defer the commit of the patch.

[Bug fortran/61632] Memory corruption on error when writing formatted data

[jvdelisle at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61632

--- Comment #24 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> ---
Author: jvdelisle
Date: Sun Jul 20 20:03:41 2014
New Revision: 212875

URL: https://gcc.gnu.org/viewcvs?rev=212875&root=gcc&view=rev
Log:
2014-07-20  Jerry DeLisle  <jvdelisle@gcc.gnu.org>

    PR libgfortran/61632
    * io/format.c (format_error): Avoid invalid string pointer by
    using the fortran string length values to generate error string.
    (parse_format): Allocate the null terminator for the format
    string.

Modified:
    trunk/libgfortran/ChangeLog
    trunk/libgfortran/io/format.c

[Bug fortran/61632] Memory corruption on error when writing formatted data

[jvdelisle at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61632

--- Comment #25 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> ---
Left to do: For format strings greater than 80 characters, shift the view of
where the error occurs in the format string so that it will display reasonably
well. (If the error occurs after 80 characters in the string.)

[Bug fortran/61632] Memory corruption on error when writing formatted data

[jvdelisle at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61632

Jerry DeLisle <jvdelisle at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Severity|normal                      |enhancement

--- Comment #26 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> ---
The memory corruption is resolved, Changing to enhancement.

[Bug fortran/61632] Memory corruption on error when writing formatted data

[jvdelisle at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61632

Jerry DeLisle <jvdelisle at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |ASSIGNED