[Bug libfortran/40330] New: [4.4 Regression] incorrect IO

[Bug libfortran/40330] New: [4.4 Regression] incorrect IO

[jv244 at cam dot ac dot uk]

right now, 4.4 branch is broken for CP2K, there seems to be some error in
libfortran that causes incorrect writes. The diff in the restarts looks like:

56,58c56,58
<      &FORCEFIELD
<        PARMTYPE  CHM
<        PARM_FILE_NAME ../sample_pot/water.pot
---
>    FORCEFIELD,
>  ,PARMTYPE CHM
>  ,PARM_FILE_NAME../sample_pot/water.pot

so a leading & disappears, and whitespace is being replaced by ',' 

4.3.1 and 4.4.0 work fine, 4.4 branch is broken. However, using the 4.3.1
runtime with a 4.4 compiled binary is OK, so I guess it is libgfortran. I have
no testcase yet (other than running CP2K and inspect the restart), but will
delve into it.


-- 
           Summary: [4.4 Regression] incorrect IO
           Product: gcc
           Version: 4.4.1
            Status: UNCONFIRMED
          Keywords: wrong-code
          Severity: normal
          Priority: P3
         Component: libfortran
        AssignedTo: unassigned at gcc dot gnu dot org
        ReportedBy: jv244 at cam dot ac dot uk


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=40330

[Bug libfortran/40330] [4.4 Regression] incorrect IO

[jv244 at cam dot ac dot uk]

------- Comment #1 from jv244 at cam dot ac dot uk  2009-06-03 15:46 -------
CCed the experts


-- 

jv244 at cam dot ac dot uk changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jb at gcc dot gnu dot org,
                   |                            |jvdelisle at gcc dot gnu dot
                   |                            |org


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=40330

[Bug libfortran/40330] [4.4 Regression] incorrect IO

[dominiq at lps dot ens dot fr]

------- Comment #2 from dominiq at lps dot ens dot fr  2009-06-03 15:54 -------
Did you try 4.5?


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=40330

[Bug libfortran/40330] [4.4 Regression] incorrect IO

[jv244 at cam dot ac dot uk]

------- Comment #3 from jv244 at cam dot ac dot uk  2009-06-03 16:08 -------
(In reply to comment #2)
> Did you try 4.5?
unfortunately trunk currently fails for other reasons. I'll try to look into
that afterwards.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=40330

[Bug libfortran/40330] [4.4 Regression] incorrect IO

[jv244 at cam dot ac dot uk]

------- Comment #4 from jv244 at cam dot ac dot uk  2009-06-03 16:31 -------
OK, a good hint from valgrind. The following error is present if I use 4.4.1
but not if I link against the runtime from 4.4.0:
==30712== Invalid read of size 1
==30712==    at 0x4A22DF9: strncmp (mc_replace_strmem.c:314)
==30712==    by 0x4BD83AF: _gfortrani_parse_format (format.c:188)
==30712==    by 0x4BE4B4F: data_transfer_init (transfer.c:2106)
==30712==    by 0x63CFE9: __input_section_types_MOD_section_vals_write
(input_section_types.F:2042)
==30712==    by 0x63ED60: __input_section_types_MOD_section_vals_write
(input_section_types.F:2087)
==30712==    by 0x5FB9A3: __input_cp2k_restarts_MOD_write_restart
(input_cp2k_restarts.F:175)
==30712==    by 0x764CA9: __md_util_MOD_md_output (md_util.F:61)
==30712==    by 0x764863: __md_run_MOD_qs_mol_dyn_low (md_run.F:461)
==30712==    by 0x764AFE: __md_run_MOD_qs_mol_dyn (md_run.F:142)
==30712==    by 0x40E5A6: __cp2k_runs_MOD_cp2k_run (cp2k_runs.F:329)
==30712==    by 0x404F37: __cp2k_runs_MOD_run_input (cp2k_runs.F:1095)
==30712==    by 0x403F62: MAIN__ (cp2k.F:272)
==30712==  Address 0x62b2ed0 is 0 bytes inside a block of size 640 free'd
==30712==    at 0x4A2196E: free (vg_replace_malloc.c:323)
==30712==    by 0x63D11F: __input_section_types_MOD_section_vals_write
(input_section_types.F:2042)
==30712==    by 0x63ED60: __input_section_types_MOD_section_vals_write
(input_section_types.F:2087)
==30712==    by 0x5FB9A3: __input_cp2k_restarts_MOD_write_restart
(input_cp2k_restarts.F:175)
==30712==    by 0x764CA9: __md_util_MOD_md_output (md_util.F:61)
==30712==    by 0x764863: __md_run_MOD_qs_mol_dyn_low (md_run.F:461)
==30712==    by 0x764AFE: __md_run_MOD_qs_mol_dyn (md_run.F:142)
==30712==    by 0x40E5A6: __cp2k_runs_MOD_cp2k_run (cp2k_runs.F:329)
==30712==    by 0x404F37: __cp2k_runs_MOD_run_input (cp2k_runs.F:1095)
==30712==    by 0x403F62: MAIN__ (cp2k.F:272)
==30712==    by 0x2D7E079: main (fmain.c:21)

the code at input_section_types.F:2042 looks like:

                                 
WRITE(unit=unit_nr,fmt="("//TRIM(myfmt)//",a,' ')",advance="NO")&
                                       TRIM(keyword%names(1))


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=40330

[Bug libfortran/40330] [4.4 Regression] incorrect IO

[jv244 at cam dot ac dot uk]

------- Comment #5 from jv244 at cam dot ac dot uk  2009-06-03 16:47 -------
OK, now with easy testcase, as you'll notice from the output, the loop is
important:

IMPLICIT NONE
character(len=100) :: fmt1,str1,fmt2,str2
real*8 :: r(3)=0
integer :: i,iunit

fmt1="(T2,A2,"
str1="a"
iunit=6
DO i=1,10
write(iunit,TRIM(fmt1)//"1X,3F20.10)") TRIM(str1),r(1:3)
ENDDO

END


yields with 4.4.1 under valgrind :

> gfortran test.f90 ; valgrind --tool=memcheck ./a.out
  a         0.0000000000        0.0000000000        0.0000000000
==666==
==666== Invalid read of size 1
==666==    at 0x4A22DF9: strncmp (mc_replace_strmem.c:314)
==666==    by 0x4BD83AF: _gfortrani_parse_format (format.c:188)
==666==    by 0x4BE4B4F: data_transfer_init (transfer.c:2106)
==666==    by 0x400C75: MAIN__ (in /data03/vondele/bug/io/a.out)
==666==    by 0x400D89: main (fmain.c:21)
==666==  Address 0x55d9218 is 0 bytes inside a block of size 1,152 free'd
==666==    at 0x4A2196E: free (vg_replace_malloc.c:323)
==666==    by 0x400D31: MAIN__ (in /data03/vondele/bug/io/a.out)
==666==    by 0x400D89: main (fmain.c:21)
==666==
==666== Invalid read of size 1
==666==    at 0x4A22E06: strncmp (mc_replace_strmem.c:314)
==666==    by 0x4BD83AF: _gfortrani_parse_format (format.c:188)
==666==    by 0x4BE4B4F: data_transfer_init (transfer.c:2106)
==666==    by 0x400C75: MAIN__ (in /data03/vondele/bug/io/a.out)
==666==    by 0x400D89: main (fmain.c:21)
==666==  Address 0x55d9218 is 0 bytes inside a block of size 1,152 free'd
==666==    at 0x4A2196E: free (vg_replace_malloc.c:323)
==666==    by 0x400D31: MAIN__ (in /data03/vondele/bug/io/a.out)
==666==    by 0x400D89: main (fmain.c:21)
==666==
==666== Invalid read of size 1
==666==    at 0x4A22E16: strncmp (mc_replace_strmem.c:314)
==666==    by 0x4BD83AF: _gfortrani_parse_format (format.c:188)
==666==    by 0x4BE4B4F: data_transfer_init (transfer.c:2106)
==666==    by 0x400C75: MAIN__ (in /data03/vondele/bug/io/a.out)
==666==    by 0x400D89: main (fmain.c:21)
==666==  Address 0x55d9219 is 1 bytes inside a block of size 1,152 free'd
==666==    at 0x4A2196E: free (vg_replace_malloc.c:323)
==666==    by 0x400D31: MAIN__ (in /data03/vondele/bug/io/a.out)
==666==    by 0x400D89: main (fmain.c:21)
==666==
==666== Invalid read of size 1
==666==    at 0x4A22E25: strncmp (mc_replace_strmem.c:314)
==666==    by 0x4BD83AF: _gfortrani_parse_format (format.c:188)
==666==    by 0x4BE4B4F: data_transfer_init (transfer.c:2106)
==666==    by 0x400C75: MAIN__ (in /data03/vondele/bug/io/a.out)
==666==    by 0x400D89: main (fmain.c:21)
==666==  Address 0x55d9219 is 1 bytes inside a block of size 1,152 free'd
==666==    at 0x4A2196E: free (vg_replace_malloc.c:323)
==666==    by 0x400D31: MAIN__ (in /data03/vondele/bug/io/a.out)
==666==    by 0x400D89: main (fmain.c:21)
  a         0.0000000000        0.0000000000        0.0000000000
  a         0.0000000000        0.0000000000        0.0000000000
  a         0.0000000000        0.0000000000        0.0000000000
  a         0.0000000000        0.0000000000        0.0000000000


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=40330

[Bug libfortran/40330] [4.4 Regression] incorrect IO

[jv244 at cam dot ac dot uk]

------- Comment #6 from jv244 at cam dot ac dot uk  2009-06-03 16:49 -------
compiled with trunk this is also buggy (with new bugs about dwarf2):

--690-- DWARF2 CFI reader: unhandled CFI instruction 0:10
--690-- DWARF2 CFI reader: unhandled CFI instruction 0:10
--690-- DWARF2 CFI reader: unhandled CFI instruction 0:10
==690==
==690== Conditional jump or move depends on uninitialised value(s)
==690==    at 0x400A97E: (within /lib64/ld-2.8.so)
==690==    by 0x4003882: (within /lib64/ld-2.8.so)
==690==    by 0x4013E77: (within /lib64/ld-2.8.so)
==690==    by 0x4001318: (within /lib64/ld-2.8.so)
==690==    by 0x4000A67: (within /lib64/ld-2.8.so)
==690==    by 0x0: ???
==690==    by 0x7FF000286: ???
==690==
==690== Conditional jump or move depends on uninitialised value(s)
==690==    at 0x400A987: (within /lib64/ld-2.8.so)
==690==    by 0x4003882: (within /lib64/ld-2.8.so)
==690==    by 0x4013E77: (within /lib64/ld-2.8.so)
==690==    by 0x4001318: (within /lib64/ld-2.8.so)
==690==    by 0x4000A67: (within /lib64/ld-2.8.so)
==690==    by 0x0: ???
==690==    by 0x7FF000286: ???
  a         0.0000000000        0.0000000000        0.0000000000
==690==
==690== Invalid read of size 1
==690==    at 0x4A22DF9: strncmp (mc_replace_strmem.c:314)
==690==    by 0x4BCF8C7: _gfortrani_parse_format (format.c:189)
==690==    by 0x7FEFFFB2F: ???
==690==    by 0xFFF: ???
==690==  Address 0x55d6218 is 0 bytes inside a block of size 1,152 free'd
==690==    at 0x4A2196E: free (vg_replace_malloc.c:323)
==690==    by 0x400CE4: MAIN__ (in /data03/vondele/bug/io/a.out)
==690==    by 0x400D3D: main (in /data03/vondele/bug/io/a.out)


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=40330

[Bug libfortran/40330] [4.4 Regression] incorrect IO

[jv244 at cam dot ac dot uk]

-- 

jv244 at cam dot ac dot uk changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|---                         |4.4.1


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=40330

[Bug libfortran/40330] [4.4 Regression] incorrect IO

[jb at gcc dot gnu dot org]

------- Comment #7 from jb at gcc dot gnu dot org  2009-06-03 17:39 -------
Confirmed on trunk. 

This seems to be due to the format caching stuff?

The regression on the 4.4 branch would be explained by the I/O backport
(r147887).


-- 

jb at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |NEW
     Ever Confirmed|0                           |1
   Last reconfirmed|0000-00-00 00:00:00         |2009-06-03 17:39:10
               date|                            |


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=40330

[Bug libfortran/40330] [4.4 Regression] incorrect IO

[jb at gcc dot gnu dot org]

------- Comment #8 from jb at gcc dot gnu dot org  2009-06-03 18:49 -------
Created an attachment (id=17949)
 --> (http://gcc.gnu.org/bugzilla/attachment.cgi?id=17949&action=view)
Proposed patch

This patch seems to fix it, will formally submit as soon as it regtests.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=40330

[Bug libfortran/40330] [4.4 Regression] incorrect IO

[jb at gcc dot gnu dot org]

------- Comment #9 from jb at gcc dot gnu dot org  2009-06-03 19:19 -------
Proper patch submitted.


-- 

jb at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                URL|                            |http://gcc.gnu.org/ml/fortra
                   |                            |n/2009-06/msg00040.html


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=40330

[Bug libfortran/40330] [4.4 Regression] incorrect IO

[jb at gcc dot gnu dot org]

------- Comment #10 from jb at gcc dot gnu dot org  2009-06-03 21:07 -------
Subject: Bug 40330

Author: jb
Date: Wed Jun  3 21:07:19 2009
New Revision: 148149

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=148149
Log:
PR libfortran/40330

Use heap memory for cached format strings.


Modified:
    trunk/libgfortran/ChangeLog
    trunk/libgfortran/io/format.c


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=40330

[Bug libfortran/40330] [4.4 Regression] incorrect IO

[jv244 at cam dot ac dot uk]

------- Comment #11 from jv244 at cam dot ac dot uk  2009-06-04 05:16 -------
(In reply to comment #9)
> Proper patch submitted.

Thanks, don't wait too long with 4.4 as this should make it in 4.4.1 which
could be released this month, and some further testing on the branch would be
good:
http://gcc.gnu.org/ml/gcc/2009-04/msg00564.htm


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=40330

[Bug libfortran/40330] [4.4 Regression] incorrect IO

[burnus at gcc dot gnu dot org]

------- Comment #12 from burnus at gcc dot gnu dot org  2009-06-04 05:36 -------
> Thanks, don't wait too long with 4.4 as this should make it in 4.4.1 which
> could be released this month, and some further testing on the branch would be
> good:
Planned for June 21, see (better link):
http://gcc.gnu.org/ml/gcc/2009-05/msg00538.html


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=40330

[Bug libfortran/40330] [4.4 Regression] incorrect IO

[jb at gcc dot gnu dot org]

------- Comment #13 from jb at gcc dot gnu dot org  2009-06-09 20:56 -------
Subject: Bug 40330

Author: jb
Date: Tue Jun  9 20:55:53 2009
New Revision: 148326

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=148326
Log:
PR libfortran/40330 Use heap memory for format cache

Modified:
    branches/gcc-4_4-branch/libgfortran/ChangeLog
    branches/gcc-4_4-branch/libgfortran/io/format.c


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=40330

[Bug libfortran/40330] [4.4 Regression] incorrect IO

[jb at gcc dot gnu dot org]

------- Comment #14 from jb at gcc dot gnu dot org  2009-06-09 20:57 -------
Closing as fixed.


-- 

jb at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|                            |FIXED


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=40330

[Bug libfortran/40330] [4.4 Regression] incorrect IO

[jv244 at cam dot ac dot uk]

------- Comment #15 from jv244 at cam dot ac dot uk  2009-06-09 22:14 -------
(In reply to comment #14)
> Closing as fixed.

I'm testing now 4.4 which includes your fix, but I still see CP2K's restarts
failing. This looks like a second issue, different from what you've fixed so
far. The testcase of comment #5 is working fine, with the update 4.4.1, but
CP2K runs still shows a valgrind warning:

==11664==
==11664== Invalid read of size 1
==11664==    at 0x4BE3C4C: formatted_transfer (transfer.c:874)
==11664==    by 0x6312C5: __input_section_types_MOD_section_vals_write
(input_section_types.F:2040)
==11664==    by 0x632964: __input_section_types_MOD_section_vals_write
(input_section_types.F:2083)
==11664==    by 0xF10D05: __replica_types_MOD_rep_env_create
(replica_types.F:278)
==11664==    by 0x11996B9: __vibrational_analysis_MOD_vb_anal
(vibrational_analysis.F:143)
==11664==    by 0x40E83A: __cp2k_runs_MOD_cp2k_run (cp2k_runs.F:385)
==11664==    by 0x404E93: __cp2k_runs_MOD_run_input (cp2k_runs.F:1095)
==11664==    by 0x403EBF: MAIN__ (cp2k.F:272)
==11664==    by 0x2CFF029: main (fmain.c:21)
==11664==  Address 0x6ada7c6 is 6 bytes inside a block of size 640 free'd
==11664==    at 0x4A2196E: free (vg_replace_malloc.c:323)
==11664==    by 0x631306: __input_section_types_MOD_section_vals_write
(input_section_types.F:2040)
==11664==    by 0x632964: __input_section_types_MOD_section_vals_write
(input_section_types.F:2083)
==11664==    by 0xF10D05: __replica_types_MOD_rep_env_create
(replica_types.F:278)
==11664==    by 0x11996B9: __vibrational_analysis_MOD_vb_anal
(vibrational_analysis.F:143)
==11664==    by 0x40E83A: __cp2k_runs_MOD_cp2k_run (cp2k_runs.F:385)
==11664==    by 0x404E93: __cp2k_runs_MOD_run_input (cp2k_runs.F:1095)
==11664==    by 0x403EBF: MAIN__ (cp2k.F:272)
==11664==    by 0x2CFF029: main (fmain.c:21)
==11664==
==11664== Invalid read of size 1
==11664==    at 0x4BE3C60: formatted_transfer (transfer.c:878)
==11664==    by 0x6312C5: __input_section_types_MOD_section_vals_write
(input_section_types.F:2040)
==11664==    by 0x632964: __input_section_types_MOD_section_vals_write
(input_section_types.F:2083)
==11664==    by 0xF10D05: __replica_types_MOD_rep_env_create
(replica_types.F:278)
==11664==    by 0x11996B9: __vibrational_analysis_MOD_vb_anal
(vibrational_analysis.F:143)
==11664==    by 0x40E83A: __cp2k_runs_MOD_cp2k_run (cp2k_runs.F:385)
==11664==    by 0x404E93: __cp2k_runs_MOD_run_input (cp2k_runs.F:1095)
==11664==    by 0x403EBF: MAIN__ (cp2k.F:272)
==11664==    by 0x2CFF029: main (fmain.c:21)
==11664==  Address 0x6ada7c7 is 7 bytes inside a block of size 640 free'd
==11664==    at 0x4A2196E: free (vg_replace_malloc.c:323)
==11664==    by 0x631306: __input_section_types_MOD_section_vals_write
(input_section_types.F:2040)
==11664==    by 0x632964: __input_section_types_MOD_section_vals_write
(input_section_types.F:2083)
==11664==    by 0xF10D05: __replica_types_MOD_rep_env_create
(replica_types.F:278)
==11664==    by 0x11996B9: __vibrational_analysis_MOD_vb_anal
(vibrational_analysis.F:143)
==11664==    by 0x40E83A: __cp2k_runs_MOD_cp2k_run (cp2k_runs.F:385)
==11664==    by 0x404E93: __cp2k_runs_MOD_run_input (cp2k_runs.F:1095)
==11664==    by 0x403EBF: MAIN__ (cp2k.F:272)
==11664==    by 0x2CFF029: main (fmain.c:21)

linking to the 4.3.2 runtime still solves the issue. I hope you can have a look
at what might be wrong with these traces above, I don't have a testcase other
than CP2K right now, and the code is tricky to reduce at that point.

If you happen to have CP2K around, this is the testcase for the trace:

cp2k/tests/Fist/regtest-5> valgrind --tool=memcheck
../../../exe/Linux-x86-64-gfortran/cp2k.sdbg wat_freq.inp  


-- 

jv244 at cam dot ac dot uk changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Severity|normal                      |critical
             Status|RESOLVED                    |REOPENED
         Resolution|FIXED                       |


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=40330

[Bug libfortran/40330] [4.4 Regression] incorrect IO

[jvdelisle at gcc dot gnu dot org]

------- Comment #16 from jvdelisle at gcc dot gnu dot org  2009-06-10 02:18 -------
The two locations referenced by valgrind in transfer.c is in code that is
unchanged from 4.3, so I don't think that is where the problem is.  However,
looking at the CP2K code line involved:

WRITE(unit=unit_nr,fmt="("//TRIM(myfmt)//",a,' ')",advance="NO")
TRIM(keyword%names(1))

Points to fmt or advance="NO" logic.  I am building cp2k with format caching
removed to see if we still get this error.

Joost, can you explain what the following means?

CP2K| condition FAILED at line 195
 CP2K| Abnormal program termination, stopped by process number 0
Aborted


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=40330

[Bug libfortran/40330] [4.4 Regression] incorrect IO

[jvdelisle at gcc dot gnu dot org]

------- Comment #17 from jvdelisle at gcc dot gnu dot org  2009-06-10 03:11 -------
I was able to do a regression hunt.  Going to r145209 just before the big I/O
patch eliminates the error.  I then moved forward to r145636 and confirmed the
breakage.

I then manually removed the use of the format caching in io/format.c and
confirmed that the problem goes away.  I suggest that I manually revert this
portion of the IO patches from 4.4 if we do not fix it very soon.

I plan to get ready for this partial revert on 4.4.  Then I will proceed to
crack the problem on trunk.  If we can not fix in time for 4.4.1 I will do the
revert.

Janne or Tobias, if you prefer I revert now, just let me know.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=40330

[Bug libfortran/40330] [4.4 Regression] incorrect IO

[burnus at gcc dot gnu dot org]

------- Comment #18 from burnus at gcc dot gnu dot org  2009-06-10 06:45 -------
> I was able to do a regression hunt.  Going to r145209 just before the big I/O
> patch eliminates the error.  I then moved forward to r145636 and confirmed the
> breakage.

Is 4.5 also affected?


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=40330

[Bug libfortran/40330] [4.4, 4.5 Regression] incorrect IO

[jv244 at cam dot ac dot uk]

------- Comment #19 from jv244 at cam dot ac dot uk  2009-06-10 07:22 -------
(In reply to comment #18)
> > I was able to do a regression hunt.  Going to r145209 just before the big I/O
> > patch eliminates the error.  I then moved forward to r145636 and confirmed the
> > breakage.
> 
> Is 4.5 also affected?

yes, it is (in addition to the other issues that plague trunk PR40383,
PR40332).


-- 

jv244 at cam dot ac dot uk changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|[4.4 Regression] incorrect  |[4.4, 4.5 Regression]
                   |IO                          |incorrect IO


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=40330

[Bug libfortran/40330] [4.4, 4.5 Regression] incorrect IO

[jv244 at cam dot ac dot uk]

------- Comment #20 from jv244 at cam dot ac dot uk  2009-06-10 07:24 -------
(In reply to comment #16)
> Joost, can you explain what the following means?
> 
> CP2K| condition FAILED at line 195
>  CP2K| Abnormal program termination, stopped by process number 0
> Aborted

In this case it means that the file water_1-r-0.inp is corrupted, but I guess
you've found that already. That file is the 'restart' that has just been
written by the part of the code that triggers the valgrind warnings.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=40330

[Bug libfortran/40330] [4.4, 4.5 Regression] incorrect IO

[jv244 at cam dot ac dot uk]

------- Comment #21 from jv244 at cam dot ac dot uk  2009-06-10 09:25 -------
reduced testcase:

MODULE M1
 IMPLICIT NONE
CONTAINS
 SUBROUTINE S1(I)
   INTEGER :: I,K
   CHARACTER(LEN=100) :: a,b
   write(a,'(I0,A)') I,"X"
   write(b,*) I
   write(6,FMT='('//TRIM(a)//",a,' ')", ADVANCE="NO") TRIM(b)
 END SUBROUTINE S1
END MODULE

USE M1
CALL S1(3)
CALL S1(3)
END


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=40330

[Bug libfortran/40330] [4.4, 4.5 Regression] incorrect IO

[jvdelisle at gcc dot gnu dot org]

------- Comment #22 from jvdelisle at gcc dot gnu dot org  2009-06-10 12:32 -------
Thanks for reduced test.

$ ./a.out >badfile
$ xxd badfile 
0000000: 2020 2020 2020 2020 2020 2020 2020 3320                3 
0000010: 2020 2020 2020 2020 2020 2020 2020 3300                3.

The NULL in the last byte should be a space.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=40330

[Bug libfortran/40330] [4.4, 4.5 Regression] incorrect IO

[jv244 at cam dot ac dot uk]

------- Comment #23 from jv244 at cam dot ac dot uk  2009-06-10 13:18 -------
(In reply to comment #22)
> Thanks for reduced test.
> 
> $ ./a.out >badfile
> $ xxd badfile 
> 0000000: 2020 2020 2020 2020 2020 2020 2020 3320                3 
> 0000010: 2020 2020 2020 2020 2020 2020 2020 3300                3.
> 
> The NULL in the last byte should be a space.

I guess this is try as well, in addition there are also the same valgrind
warnings about using data from a deallocated buffer:

==24239== Invalid read of size 1
==24239==    at 0x4BE3C4C: formatted_transfer (transfer.c:874)
==24239==    by 0x400D21: __m1_MOD_s1 (t.f90:9)
==24239==    by 0x400D91: MAIN__ (t.f90:15)
==24239==    by 0x400DC9: main (fmain.c:21)
==24239==  Address 0x55da8a6 is 6 bytes inside a block of size 640 free'd
==24239==    at 0x4A2196E: free (vg_replace_malloc.c:323)
==24239==    by 0x400D5E: __m1_MOD_s1 (t.f90:9)
==24239==    by 0x400D87: MAIN__ (t.f90:14)
==24239==    by 0x400DC9: main (fmain.c:21)
==24239==
==24239== Invalid read of size 1
==24239==    at 0x4BE3C60: formatted_transfer (transfer.c:878)
==24239==    by 0x400D21: __m1_MOD_s1 (t.f90:9)
==24239==    by 0x400D91: MAIN__ (t.f90:15)
==24239==    by 0x400DC9: main (fmain.c:21)
==24239==  Address 0x55da8a7 is 7 bytes inside a block of size 640 free'd
==24239==    at 0x4A2196E: free (vg_replace_malloc.c:323)
==24239==    by 0x400D5E: __m1_MOD_s1 (t.f90:9)
==24239==    by 0x400D87: MAIN__ (t.f90:14)
==24239==    by 0x400DC9: main (fmain.c:21)


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=40330

[Bug libfortran/40330] [4.4, 4.5 Regression] incorrect IO

[jb at gcc dot gnu dot org]

------- Comment #24 from jb at gcc dot gnu dot org  2009-06-10 21:17 -------
Further reduced testcase:

program pr40330
 implicit none

 call s1()
 call s1()

contains
 subroutine s1()
   character(LEN=100) :: a
   a = "(3X)"
   write(*,FMT='('//trim(a)//",a,' ')", ADVANCE="NO") "3"
 end subroutine s1
end program pr40330

The interesting thing is, if you replace the trim(a) with trim("(3X)") then it
works. Look at -fdump-tree-original to see the difference; in the second case
the format string is a constant, in the first it's not and it's deallocated,
which might be related to why we get the valgrind errors about reading from
free'd storage.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=40330

[Bug libfortran/40330] [4.4, 4.5 Regression] incorrect IO

[jvdelisle at gcc dot gnu dot org]

------- Comment #25 from jvdelisle at gcc dot gnu dot org  2009-06-11 01:29 -------
Changing priority to normal.

I have a patch that fixes the reduced test case.  Testing CP2K now.


-- 

jvdelisle at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Severity|critical                    |normal


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=40330

[Bug libfortran/40330] [4.4, 4.5 Regression] incorrect IO

[jvdelisle at gcc dot gnu dot org]

------- Comment #26 from jvdelisle at gcc dot gnu dot org  2009-06-11 02:25 -------
CP2K still fails.  However, I think I have discovered the root cause.  String
constants in formats are saved in the fnode at sting.p which is a pointer. 
When we use cached parsed string data, those constant strings are net getting
copied and saved along with the parsed format.

That "constant" string is used by write_constant_string in transfer.c.  This is
an issue with the concept because the constant string can vary with each
invocation of the IO statement, as is the case with CP2k, and it is freed at
the end of each IO operation.  The benefit of caching is lost if we have to
reparse to find the strings

This leaves the possibility of disabling the caching if a FMT_STRING token is
found while parsing the format the first time. I am going to suggest we revert
format caching from 4.4 right away and probably 4.5 if some better idea does
not pop into my head.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=40330

[Bug libfortran/40330] [4.4, 4.5 Regression] incorrect IO

[jv244 at cam dot ac dot uk]

------- Comment #27 from jv244 at cam dot ac dot uk  2009-06-11 07:04 -------
(In reply to comment #26)
> I am going to suggest we revert
> format caching from 4.4 right away

Yes, please, that gives me roughly a week to do some more testing before 4.4.1
is released.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=40330

[Bug libfortran/40330] [4.4, 4.5 Regression] incorrect IO

[jvdelisle at gcc dot gnu dot org]

------- Comment #28 from jvdelisle at gcc dot gnu dot org  2009-06-11 12:49 -------
Subject: Bug 40330

Author: jvdelisle
Date: Thu Jun 11 12:49:35 2009
New Revision: 148391

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=148391
Log:
2009-06-11  Jerry DeLisle  <jvdelisle@gcc.gnu.org>

        PR libfortran/40330
        * io/io.h: Revert format caching.
        * io/unit.c: Likewise.
        * io/transfer.c: Likewise.
        * io/format.c: Likewise.

Modified:
    branches/gcc-4_4-branch/libgfortran/ChangeLog
    branches/gcc-4_4-branch/libgfortran/io/format.c
    branches/gcc-4_4-branch/libgfortran/io/io.h
    branches/gcc-4_4-branch/libgfortran/io/transfer.c
    branches/gcc-4_4-branch/libgfortran/io/unit.c


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=40330

[Bug libfortran/40330] [4.5 Regression] incorrect IO

[jvdelisle at gcc dot gnu dot org]

------- Comment #29 from jvdelisle at gcc dot gnu dot org  2009-06-11 14:58 -------
Fixed on 4.4.1, 4.5 in review process.


-- 

jvdelisle at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|[4.4, 4.5 Regression]       |[4.5 Regression] incorrect
                   |incorrect IO                |IO


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=40330

[Bug libfortran/40330] [4.5 Regression] incorrect IO

[jv244 at cam dot ac dot uk]

------- Comment #30 from jv244 at cam dot ac dot uk  2009-06-12 05:47 -------
(In reply to comment #29)
> Fixed on 4.4.1, 4.5 in review process.

great, many thanks. CP2K now passes it >1600 tests with 4.4.1.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=40330

[Bug libfortran/40330] [4.5 Regression] incorrect IO

[jvdelisle at gcc dot gnu dot org]

------- Comment #31 from jvdelisle at gcc dot gnu dot org  2009-06-26 15:50 -------
I am working on some ideas here.  We are getting double free or corruption
related to free_format_data.  I think it is because we have two pointers to pay
attention to.  The one in the hash table and dtp->u.p.fmt which is copied. My
biggest issue is time to do this, so patience please.


-- 

jvdelisle at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         AssignedTo|unassigned at gcc dot gnu   |jvdelisle at gcc dot gnu dot
                   |dot org                     |org
             Status|REOPENED                    |ASSIGNED
   Last reconfirmed|2009-06-03 17:39:10         |2009-06-26 15:50:47
               date|                            |


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=40330

[Bug libfortran/40330] [4.5 Regression] incorrect IO

[jvdelisle at gcc dot gnu dot org]

------- Comment #32 from jvdelisle at gcc dot gnu dot org  2009-07-05 02:12 -------
I am getting close now.  I have a patch that actually passes the CP2K test case
and regression testing.  It turns out that the parse_format_list function is
called recursively so one must be careful about relaying information back up to
the callers that a FMT_STRING token is involved and so don't save the parsed
data. There is a memory leak remaining, but I know exactly what it is and what
to do about it.

The patch I am working on is really just a work around for the real problem. 
The real problem is that the pointer for literal strings in formats is
clobbered by the routine that writes it. My plan is to get the above mentioned
workaround patch finalized and then work toward eliminating this root problem
in a subsequent patch. The final fix will involve some re-constructive surgery
on the format node structure. 


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=40330

[Bug libfortran/40330] [4.5 Regression] incorrect IO

[jvdelisle at gcc dot gnu dot org]

------- Comment #33 from jvdelisle at gcc dot gnu dot org  2009-07-05 12:25 -------
Created an attachment (id=18138)
 --> (http://gcc.gnu.org/bugzilla/attachment.cgi?id=18138&action=view)
Patch ready for testing

Joost, if you can, please test this patch on all of CP2K.  Other testors
welcome.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=40330

[Bug libfortran/40330] [4.5 Regression] incorrect IO

[jvdelisle at gcc dot gnu dot org]

------- Comment #34 from jvdelisle at gcc dot gnu dot org  2009-07-09 01:20 -------
Subject: Bug 40330

Author: jvdelisle
Date: Thu Jul  9 01:20:23 2009
New Revision: 149398

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=149398
Log:
2009-07-08  Jerry DeLisle  <jvdelisle@gcc.gnu.org>

        PR libfortran/40330
        PR libfortran/40662
        * io/io.h (st_parameter_dt): Define format_not_saved bit used to signal
        whether the parsed format data was previously saved. Used to determine
        if the current format data should be freed or not.
        * io/transfer.c (st_read_done): Use the format_not_saved bit.
        (st_write_done): Likewise.
        * io/format.c (parse_format_list): Add boolean pointer to arg list.
This
        pointer is used to return status to the caller regarding whether it is
        safe to cache the parsed format data.  Currently, if a FMT_STRING token
        is encounetered, it is not safe to cache. Also, added a local boolean
        variable to hold this information as recursive calls to
        parse_format_list are made.  Remove previous save_format logic.
        (parse_format): Do not use the format caching facility if the current
        unit is an internal unit or if it is not safe to save parsed format
        data.

Modified:
    trunk/libgfortran/ChangeLog
    trunk/libgfortran/io/format.c
    trunk/libgfortran/io/io.h
    trunk/libgfortran/io/transfer.c


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=40330

[Bug libfortran/40330] [4.5 Regression] incorrect IO

[jvdelisle at gcc dot gnu dot org]

------- Comment #35 from jvdelisle at gcc dot gnu dot org  2009-07-09 01:55 -------
Subject: Bug 40330

Author: jvdelisle
Date: Thu Jul  9 01:54:47 2009
New Revision: 149399

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=149399
Log:
2009-07-08  Jerry DeLisle  <jvdelisle@gcc.gnu.org>

        PR libfortran/40330
        PR libfortran/40662
        * gfortran.dg/fmt_cache_1.f: New test.

Added:
    trunk/gcc/testsuite/gfortran.dg/fmt_cache_1.f
Modified:
    trunk/gcc/testsuite/ChangeLog


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=40330

[Bug libfortran/40330] [4.5 Regression] incorrect IO

[jvdelisle at gcc dot gnu dot org]

------- Comment #36 from jvdelisle at gcc dot gnu dot org  2009-07-09 01:59 -------
Fixed on trunk.


-- 

jvdelisle at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |RESOLVED
         Resolution|                            |FIXED


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=40330

[Bug libfortran/40330] [4.5 Regression] incorrect IO

[jv244 at cam dot ac dot uk]

------- Comment #37 from jv244 at cam dot ac dot uk  2009-07-12 14:49 -------
(In reply to comment #33)
> Joost, if you can, please test this patch on all of CP2K.  Other testors
> welcome.

A bit late, but yes, current trunk now works fine (I/O wise)


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=40330

[Bug libfortran/40330] [4.5 Regression] incorrect IO

[hjl at gcc dot gnu dot org]

------- Comment #38 from hjl at gcc dot gnu dot org  2009-07-23 17:51 -------
Subject: Bug 40330

Author: hjl
Date: Thu Jul 23 17:50:56 2009
New Revision: 150020

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=150020
Log:
2009-07-23  H.J. Lu  <hongjiu.lu@intel.com>

        Backport from mainline:
        2009-07-22  Richard Guenther  <rguenther@suse.de>

        PR c++/40799
        * g++.dg/lookup/using21.C: New testcase.

        2009-07-19  Jan Hubicka  <jh@suse.cz>

        PR tree-optimization/40676
        * gcc.c-torture/compile/pr40676.c: New testcase.

        2009-07-15  Richard Guenther  <rguenther@suse.de>

        PR middle-end/40753
        * gcc.c-torture/compile/pr40753.c: New testcase.

        2009-07-14  Dodji Seketeli  <dodji@redhat.com>

        PR debug/40705
        * g++.dg/debug/dwarf2/typedef1.C: New test.

        PR c++/40357
        * g++.dg/other/typedef3.C: New test.

        2009-07-10  Richard Guenther  <rguenther@suse.de>

        PR tree-optimization/40496
        * g++.dg/opt/pr40496.C: New testcase.

        2009-07-09  Jakub Jelinek  <jakub@redhat.com>

        PR middle-end/40692
        * gcc.c-torture/compile/pr40692.c: New test.

        2009-07-08  Jerry DeLisle  <jvdelisle@gcc.gnu.org>

        PR libfortran/40330
        PR libfortran/40662
        * gfortran.dg/fmt_cache_1.f: New test.

Added:
    branches/gcc-4_4-branch/gcc/testsuite/g++.dg/debug/dwarf2/typedef1.C
      - copied unchanged from r150018,
trunk/gcc/testsuite/g++.dg/debug/dwarf2/typedef1.C
    branches/gcc-4_4-branch/gcc/testsuite/g++.dg/lookup/using21.C
      - copied unchanged from r150018,
trunk/gcc/testsuite/g++.dg/lookup/using21.C
    branches/gcc-4_4-branch/gcc/testsuite/g++.dg/opt/pr40496.C
      - copied unchanged from r150018, trunk/gcc/testsuite/g++.dg/opt/pr40496.C
    branches/gcc-4_4-branch/gcc/testsuite/g++.dg/other/typedef3.C
      - copied unchanged from r150018,
trunk/gcc/testsuite/g++.dg/other/typedef3.C
    branches/gcc-4_4-branch/gcc/testsuite/gcc.c-torture/compile/pr40676.c
      - copied unchanged from r150018,
trunk/gcc/testsuite/gcc.c-torture/compile/pr40676.c
    branches/gcc-4_4-branch/gcc/testsuite/gcc.c-torture/compile/pr40692.c
      - copied unchanged from r150018,
trunk/gcc/testsuite/gcc.c-torture/compile/pr40692.c
    branches/gcc-4_4-branch/gcc/testsuite/gcc.c-torture/compile/pr40753.c
      - copied unchanged from r150018,
trunk/gcc/testsuite/gcc.c-torture/compile/pr40753.c
    branches/gcc-4_4-branch/gcc/testsuite/gfortran.dg/fmt_cache_1.f
      - copied unchanged from r150018,
trunk/gcc/testsuite/gfortran.dg/fmt_cache_1.f
Modified:
    branches/gcc-4_4-branch/gcc/testsuite/ChangeLog


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=40330