[Bug libfortran/56737] New: Bizarre Hollerith edit descriptor errors (valgrind shows uninitialized value in libgfortran)

[Bug libfortran/56737] New: Bizarre Hollerith edit descriptor errors (valgrind shows uninitialized value in libgfortran)

[jonathan.hogg at stfc dot ac.uk]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56737

             Bug #: 56737
           Summary: Bizarre Hollerith edit descriptor errors (valgrind
                    shows uninitialized value in libgfortran)
    Classification: Unclassified
           Product: gcc
           Version: unknown
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: libfortran
        AssignedTo: unassigned@gcc.gnu.org
        ReportedBy: jonathan.hogg@stfc.ac.uk


Created attachment 29728
  --> http://gcc.gnu.org/bugzilla/attachment.cgi?id=29728
Code to reproduce bug

With the code attached I get junk output in a real program. Simplified version
below only shows errors under valgrind. Easy workaround by transforming code to
modern Fortran that doesn't use the h edit descriptor.

Yes, the byzantine subroutine hierarchy and unuser arguments appear to be
required to reproduce the bug.

Expected test-code behaviour: Should produce valgrind-clean code. Shuold
perhaps warn that 'h' edit descriptor is obsolescent/deleted (in F95+).
Expected real-code behaviour: Shouldn't produce garbage to output.

Thanks,

Jonathan.

$gfortran -v
Using built-in specs.
COLLECT_GCC=gfortran
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/4.7/lto-wrapper
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu/Linaro
4.7.2-4ubuntu1' --with-bugurl=file:///usr/share/doc/gcc-4.7/README.Bugs
--enable-languages=c,c++,go,fortran,objc,obj-c++ --prefix=/usr
--program-suffix=-4.7 --enable-shared --enable-linker-build-id
--with-system-zlib --libexecdir=/usr/lib --without-included-gettext
--enable-threads=posix --with-gxx-include-dir=/usr/include/c++/4.7
--libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu
--enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-gnu-unique-object
--enable-plugin --enable-objc-gc --disable-werror --with-arch-32=i686
--with-tune=generic --enable-checking=release --build=x86_64-linux-gnu
--host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 4.7.2 (Ubuntu/Linaro 4.7.2-4ubuntu1)

$gfortran -g -o gfort_bug gfort_bug.f90 && valgrind ./gfort_bug
==8437== Memcheck, a memory error detector
==8437== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al.
==8437== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info
==8437== Command: ./gfort_bug
==8437== 
 fmt = (   3(1H ),6h===== ,a  12,i4,6h =====)                                   
 title1= end of level 
 level =            1
   ===== end of level   1 =====
 fmt = (   3(1H ),6h===== ,a  12,i4,6h =====)                                   
 title1= end of level 
 level =            1
==8437== Conditional jump or move depends on uninitialised value(s)
==8437==    at 0x4F0BB41: ??? (in
/usr/lib/x86_64-linux-gnu/libgfortran.so.3.0.0)
==8437==    by 0x400B51: __hsl_mc73_single_MOD_level_print (gfort_bug.f90:56)
==8437==    by 0x400B93: __hsl_mc73_single_MOD_multilevel_eig
(gfort_bug.f90:40)
==8437==    by 0x400BB9: __hsl_mc73_single_MOD_fiedler_graph (gfort_bug.f90:33)
==8437==    by 0x400BF9: __hsl_mc73_single_MOD_mc73_fiedler (gfort_bug.f90:17)
==8437==    by 0x400C27: MAIN__ (gfort_bug.f90:71)
==8437==    by 0x400C5D: main (gfort_bug.f90:62)
==8437== 
==8437== Conditional jump or move depends on uninitialised value(s)
==8437==    at 0x4F0BB1B: ??? (in
/usr/lib/x86_64-linux-gnu/libgfortran.so.3.0.0)
==8437==    by 0x400B51: __hsl_mc73_single_MOD_level_print (gfort_bug.f90:56)
==8437==    by 0x400B93: __hsl_mc73_single_MOD_multilevel_eig
(gfort_bug.f90:40)
==8437==    by 0x400BB9: __hsl_mc73_single_MOD_fiedler_graph (gfort_bug.f90:33)
==8437==    by 0x400BF9: __hsl_mc73_single_MOD_mc73_fiedler (gfort_bug.f90:17)
==8437==    by 0x400C27: MAIN__ (gfort_bug.f90:71)
==8437==    by 0x400C5D: main (gfort_bug.f90:62)
==8437== 
==8437== Syscall param write(buf) points to uninitialised byte(s)
==8437==    at 0x522C900: __write_nocancel (syscall-template.S:82)
==8437==    by 0x4F0ECEC: ??? (in
/usr/lib/x86_64-linux-gnu/libgfortran.so.3.0.0)
==8437==    by 0x4F1540E: ??? (in
/usr/lib/x86_64-linux-gnu/libgfortran.so.3.0.0)
==8437==    by 0x4F0A726: ??? (in
/usr/lib/x86_64-linux-gnu/libgfortran.so.3.0.0)
==8437==    by 0x4F0B008: _gfortran_st_write_done (in
/usr/lib/x86_64-linux-gnu/libgfortran.so.3.0.0)
==8437==    by 0x400B60: __hsl_mc73_single_MOD_level_print (gfort_bug.f90:56)
==8437==    by 0x400B93: __hsl_mc73_single_MOD_multilevel_eig
(gfort_bug.f90:40)
==8437==    by 0x400BB9: __hsl_mc73_single_MOD_fiedler_graph (gfort_bug.f90:33)
==8437==    by 0x400BF9: __hsl_mc73_single_MOD_mc73_fiedler (gfort_bug.f90:17)
==8437==    by 0x400C27: MAIN__ (gfort_bug.f90:71)
==8437==    by 0x400C5D: main (gfort_bug.f90:62)
==8437==  Address 0x5c4fc3a is 26 bytes inside a block of size 512 alloc'd
==8437==    at 0x4C2B3F8: malloc (in
/usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==8437==    by 0x4E4D488: ??? (in
/usr/lib/x86_64-linux-gnu/libgfortran.so.3.0.0)
==8437==    by 0x4F1529A: ??? (in
/usr/lib/x86_64-linux-gnu/libgfortran.so.3.0.0)
==8437==    by 0x4F0E421: ??? (in
/usr/lib/x86_64-linux-gnu/libgfortran.so.3.0.0)
==8437==    by 0x4E4AB67: ??? (in
/usr/lib/x86_64-linux-gnu/libgfortran.so.3.0.0)
==8437==    by 0x400F315: call_init.part.0 (dl-init.c:85)
==8437==    by 0x400F3EE: _dl_init (dl-init.c:52)
==8437==    by 0x40016F9: ??? (in /lib/x86_64-linux-gnu/ld-2.15.so)
==8437== 
         end of level   1 ====)
==8437== 
==8437== HEAP SUMMARY:
==8437==     in use at exit: 0 bytes in 0 blocks
==8437==   total heap usage: 29 allocs, 29 frees, 24,709 bytes allocated
==8437== 
==8437== All heap blocks were freed -- no leaks are possible
==8437== 
==8437== For counts of detected and suppressed errors, rerun with: -v
==8437== Use --track-origins=yes to see where uninitialised values come from
==8437== ERROR SUMMARY: 11 errors from 3 contexts (suppressed: 2 from 2)

[Bug libfortran/56737] [4.6/4.7/4.8/4.9 Regression] Wrong I/O result with format cache for Hollerith strings

[burnus at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56737

Tobias Burnus <burnus at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |NEW
      Known to work|                            |4.3.4, 4.4.0
           Keywords|                            |wrong-code
   Last reconfirmed|                            |2013-03-27
                 CC|                            |burnus at gcc dot gnu.org,
                   |                            |jvdelisle at gcc dot
                   |                            |gnu.org
     Ever Confirmed|0                           |1
            Summary|Bizarre Hollerith edit      |[4.6/4.7/4.8/4.9
                   |descriptor errors (valgrind |Regression] Wrong I/O
                   |shows uninitialized value   |result with format cache
                   |in libgfortran)             |for Hollerith strings
   Target Milestone|---                         |4.6.4
      Known to fail|                            |4.5.3, 4.6.3, 4.7.2, 4.8.0,
                   |                            |4.9.0

--- Comment #1 from Tobias Burnus <burnus at gcc dot gnu.org> 2013-03-27 19:32:32 UTC ---
Seems to be a regression in libgfortran. It works with gfortran 4.3 and 4.4,
but fails with gfortran 4.5 to 4.9.

(Compiling with 4.9 and running with libgfortran 4.3 works, i.e. the regression
must be in libgfortran.)

It works if one disables the format cache.



If one adds some debugging output to write_constant_string, one sees that for
the second call, the delimiter (f->u.string.p[-1]) is not 'H' - and the first
6H is also wrong.

write_constant_string: len=1: f->u.string.p=' '- delim='H'
write_constant_string: len=1: f->u.string.p=' '- delim='H'
write_constant_string: len=1: f->u.string.p=' '- delim='H'
write_constant_string: len=6: f->u.string.p='===== '- delim='h'
write_constant_string: len=6: f->u.string.p=' ====='- delim='h'
   ===== end of level   1 =====
write_constant_string: len=1: f->u.string.p=' '- delim=' '
write_constant_string: len=1: f->u.string.p=' '- delim=' '
write_constant_string: len=1: f->u.string.p=' '- delim=' '
write_constant_string: len=6: f->u.string.p='      '- delim=' '
write_constant_string: len=6: f->u.string.p=' ====='- delim=' '
         end of level   1 ====)

[Bug libfortran/56737] [4.6/4.7/4.8/4.9 Regression] Wrong I/O result with format cache for Hollerith strings

[burnus at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56737

--- Comment #2 from Tobias Burnus <burnus at gcc dot gnu.org> 2013-03-27 20:24:11 UTC ---
If one sets the breakpoint in _gfortrani_next_format, one gets for the
original, non cached version:

(gdb) p dtp->u.p.fmt.array.array[1].source
$13 = 0x7fffffffdec6 "1H ),6h===== ,a  12,i4,6h =====)", ...


For the second round, one gets:
(gdb) p dtp->u.p.fmt.array.array[1].source
$14 = 0x7fffffffdec6 ' ' <repeats 26 times>, "=====)      \f"

Thus, the "source" string is not carried over.

[Bug libfortran/56737] [4.6/4.7/4.8/4.9 Regression] Wrong I/O result with format cache for Hollerith strings

[burnus at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56737

Tobias Burnus <burnus at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |ASSIGNED
         AssignedTo|unassigned at gcc dot       |burnus at gcc dot gnu.org
                   |gnu.org                     |

--- Comment #4 from Tobias Burnus <burnus at gcc dot gnu.org> 2013-03-28 10:00:42 UTC ---
(In reply to comment #3)
> Note the comment at line 728 of format.c where we must disable format caching
> when we encounter FMT_STRING.  The problem is related to saving a pointer to a
> string rather than the string itself.  That pointer can go out of scope on
> subsequent I/O operations so caching the format data will not work.

Well, one could save the data. The string is also stored in the hash:

  u->format_hash_table[hash].key = xmalloc (dtp->format_len);
  memcpy (u->format_hash_table[hash].key, dtp->format, dtp->format_len);

Thus, instead of saving it after the parsing as key, one can duplicate the
string and store it as dtp->format before parsing the string.

That's what I do in the patch, submitted at:
http://gcc.gnu.org/ml/fortran/2013-03/msg00155.html

[Bug libfortran/56737] [4.6/4.7/4.8/4.9 Regression] Wrong I/O result with format cache for Hollerith strings

[burnus at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56737

--- Comment #5 from Tobias Burnus <burnus at gcc dot gnu.org> 2013-03-29 09:38:20 UTC ---
Author: burnus
Date: Fri Mar 29 09:37:37 2013
New Revision: 197229

URL: http://gcc.gnu.org/viewcvs?rev=197229&root=gcc&view=rev
Log:
2013-03-29  Tobias Burnus  <burnus@net-b.de>

        PR fortran/56737
        * io/format.c (parse_format): With caching, copy
        dtp->format string.
        (save_parsed_format): Use dtp->format directly without
        copying.

2013-03-29  Tobias Burnus  <burnus@net-b.de>

        PR fortran/56737
        * testsuite/gfortran.dg/fmt_cache_3.f90: New.

(Plus: Move fortran/ChangeLog item to libgfortran/ChangeLog)

Added:
    trunk/gcc/testsuite/gfortran.dg/fmt_cache_3.f90
Modified:
    trunk/gcc/fortran/ChangeLog
    trunk/gcc/testsuite/ChangeLog
    trunk/libgfortran/ChangeLog
    trunk/libgfortran/io/format.c

[Bug libfortran/56737] [4.6/4.7/4.8/4.9 Regression] Wrong I/O result with format cache for Hollerith strings

[burnus at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56737

--- Comment #6 from Tobias Burnus <burnus at gcc dot gnu.org> 2013-03-29 09:40:13 UTC ---
Author: burnus
Date: Fri Mar 29 09:39:47 2013
New Revision: 197230

URL: http://gcc.gnu.org/viewcvs?rev=197230&root=gcc&view=rev
Log:
2012-03-29  Tobias Burnus  <burnus@net-b.de>

        PR fortran/56737
        * io/format.c (parse_format_list): Also cache FMT_STRING.
        (parse_format): Update call.


Modified:
    trunk/libgfortran/ChangeLog
    trunk/libgfortran/io/format.c

[Bug libfortran/56737] [4.6/4.7/4.8/4.9 Regression] Wrong I/O result with format cache for Hollerith strings

[burnus at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56737

--- Comment #7 from Tobias Burnus <burnus at gcc dot gnu.org> 2013-04-02 15:43:19 UTC ---
Author: burnus
Date: Tue Apr  2 15:42:31 2013
New Revision: 197359

URL: http://gcc.gnu.org/viewcvs?rev=197359&root=gcc&view=rev
Log:
libgfortran/
2013-04-02  Tobias Burnus  <burnus@net-b.de>

        Backport from mainline:
        2013-03-29  Tobias Burnus  <burnus@net-b.de>

        PR fortran/56737
        * io/format.c (parse_format): With caching, copy
        dtp->format string.
        (save_parsed_format): Use dtp->format directly without
        copying.

        2012-03-29  Tobias Burnus  <burnus@net-b.de>

        PR fortran/56737
        * io/format.c (parse_format_list): Also cache FMT_STRING.
        (parse_format): Update call.

gcc/testsuite/
2013-04-02  Tobias Burnus  <burnus@net-b.de>

        Backport from mainline:
        2013-03-29  Tobias Burnus  <burnus@net-b.de>

        PR fortran/56737
        * testsuite/gfortran.dg/fmt_cache_3.f90: New.


Added:
    branches/gcc-4_8-branch/gcc/testsuite/gfortran.dg/fmt_cache_3.f90
Modified:
    branches/gcc-4_8-branch/gcc/testsuite/ChangeLog
    branches/gcc-4_8-branch/libgfortran/ChangeLog
    branches/gcc-4_8-branch/libgfortran/io/format.c

[Bug libfortran/56737] [4.6/4.7/4.8/4.9 Regression] Wrong I/O result with format cache for Hollerith strings

[rguenth at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56737

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Priority|P3                          |P4

[Bug libfortran/56737] [4.6/4.7/4.8/4.9 Regression] Wrong I/O result with format cache for Hollerith strings

[burnus at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56737

--- Comment #8 from Tobias Burnus <burnus at gcc dot gnu.org> 2013-04-03 14:02:36 UTC ---
Author: burnus
Date: Wed Apr  3 14:00:20 2013
New Revision: 197412

URL: http://gcc.gnu.org/viewcvs?rev=197412&root=gcc&view=rev
Log:
2013-04-03  Tobias Burnus  <burnus@net-b.de>

        Backport from mainline:
        2013-03-29  Tobias Burnus  <burnus@net-b.de>

        PR fortran/56737
        * io/format.c (parse_format): With caching, copy
        dtp->format string.
        (save_parsed_format): Use dtp->format directly without
        copying.

2013-04-03  Tobias Burnus  <burnus@net-b.de>

        Backport from mainline:
        2013-03-29  Tobias Burnus  <burnus@net-b.de>

        PR fortran/56737
        * testsuite/gfortran.dg/fmt_cache_3.f90: New.


Added:
    branches/gcc-4_7-branch/gcc/testsuite/gfortran.dg/fmt_cache_3.f90
Modified:
    branches/gcc-4_7-branch/gcc/testsuite/ChangeLog
    branches/gcc-4_7-branch/libgfortran/ChangeLog
    branches/gcc-4_7-branch/libgfortran/io/format.c

[Bug libfortran/56737] [4.6/4.7/4.8/4.9 Regression] Wrong I/O result with format cache for Hollerith strings

[burnus at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56737

--- Comment #9 from Tobias Burnus <burnus at gcc dot gnu.org> 2013-04-03 14:07:02 UTC ---
(In reply to comment #8)
> 2013-04-03  Tobias Burnus  <burnus@net-b.de>
> 
>         Backport from mainline:
>         2013-03-29  Tobias Burnus  <burnus@net-b.de>
> 
>         PR fortran/56737
>         * io/format.c (parse_format): With caching, copy
>         dtp->format string.
>         (save_parsed_format): Use dtp->format directly without
>         copying.

I forgot to include the following bits in the commit log - it's in the
ChangeLog and in the commit:

>        2012-03-29  Tobias Burnus  <burnus@net-b.de>
>
>        PR fortran/56737
>        * io/format.c (parse_format_list): Also cache FMT_STRING.
>        (parse_format): Update call.



So far, the issue is fixed on the trunk and the 4.7 and 4.8 branches; GCC 4.7.3
is scheduled to be released in the next days.  (Backporting to 4.6 is still
pending.)

[Bug libfortran/56737] [4.6/4.7/4.8/4.9 Regression] Wrong I/O result with format cache for Hollerith strings

[burnus at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56737

--- Comment #10 from Tobias Burnus <burnus at gcc dot gnu.org> 2013-04-04 09:32:14 UTC ---
Author: burnus
Date: Thu Apr  4 09:31:53 2013
New Revision: 197474

URL: http://gcc.gnu.org/viewcvs?rev=197474&root=gcc&view=rev
Log:
libgfortran/
2013-04-04  Tobias Burnus  <burnus@net-b.de>

        Backport from mainline:
        2013-03-29  Tobias Burnus  <burnus@net-b.de>

        PR fortran/56737
        * io/format.c (parse_format): With caching, copy
        dtp->format string.
        (save_parsed_format): Use dtp->format directly without
        copying.

        2012-03-29  Tobias Burnus  <burnus@net-b.de>

        PR fortran/56737
        * io/format.c (parse_format_list): Also cache FMT_STRING.
        (parse_format): Update call.

gcc/testsuite/
2013-04-04  Tobias Burnus  <burnus@net-b.de>

        Backport from mainline:
        2013-03-29  Tobias Burnus  <burnus@net-b.de>

        PR fortran/56737
        * testsuite/gfortran.dg/fmt_cache_3.f90: New.


Added:
    branches/gcc-4_6-branch/gcc/testsuite/gfortran.dg/fmt_cache_3.f90
Modified:
    branches/gcc-4_6-branch/gcc/testsuite/ChangeLog
    branches/gcc-4_6-branch/libgfortran/ChangeLog
    branches/gcc-4_6-branch/libgfortran/io/format.c

[Bug libfortran/56737] [4.6/4.7/4.8/4.9 Regression] Wrong I/O result with format cache for Hollerith strings

[burnus at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56737

Tobias Burnus <burnus at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |RESOLVED
         Resolution|                            |FIXED

--- Comment #11 from Tobias Burnus <burnus at gcc dot gnu.org> 2013-04-04 09:38:53 UTC ---
FIXED on the 4.9 trunk - and on the 4.6, 4.7 and 4.8 branches.

The very soon to be released 4.7.3 and 4.6.4 will contain this fix. (As will
4.8.1, but that take still a while.)

Thanks for the bug report - and sorry for the regression.

You could either upgrade to one of those, e.g. Ubuntu might provide a 4.7.3 in
a few weeks via PPA, cf. http://gcc.gnu.org/wiki/GFortranDistros
Or you build it yourself or use an unofficial binary build, cf.
http://gcc.gnu.org/wiki/GFortranBinaries
Or you replace the H9Hollerith strings by "normal" 'strings' in I/O fmts.

[Bug libfortran/56737] [4.6/4.7/4.8/4.9 Regression] Wrong I/O result with format cache for Hollerith strings

[jonathan.hogg at stfc dot ac.uk]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56737

--- Comment #12 from Jonathan Hogg <jonathan.hogg at stfc dot ac.uk> 2013-04-04 10:43:02 UTC ---
Thanks for fixing this. The code in question has been moved to use 'strings'
rather than Hollerith i/o - we aim for it to be F95 compliant anyway!

Jonathan.