[Bug libfortran/52539] New: I/O: Wrong result for UTF-8/UCS-4 list-directed and namelist read and nml write

[Bug libfortran/52539] New: I/O: Wrong result for UTF-8/UCS-4 list-directed and namelist read and nml write

[burnus at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52539

             Bug #: 52539
           Summary: I/O: Wrong result for UTF-8/UCS-4 list-directed and
                    namelist read and nml write
    Classification: Unclassified
           Product: gcc
           Version: 4.8.0
            Status: UNCONFIRMED
          Keywords: wrong-code
          Severity: normal
          Priority: P3
         Component: libfortran
        AssignedTo: unassigned@gcc.gnu.org
        ReportedBy: burnus@gcc.gnu.org
                CC: jvdelisle@gcc.gnu.org


Example using UTF-8 string with UCS-4 character variables; the string contains
(a)(ni)(hao)(b) ("a你好").

The following program prints here the following. One sees that list-directed
writing correctly works but namelist writing doesn't. (Also the fort.99
contains the correct string.)

For reading, neither list-directed reading nor namelist reading works.


Looking at list_read.c, UTF-8 seems to be unhandled. (And in nml_get_obj_data,
the "array_loop_spec ind" initialization won't survive the array descriptor
reform unmodified.)


>a你好<        ! << OK
 >a你好<       ! << OK
&NML
 STR="a",      ! << WRONG
 /
>�����XX<      ! << WRONG
 >�����XX<     ! << WRONG
&NML
 STR="a��",    ! << WRONG
 /
 >aä½<         ! << WRONG
>aä½<          ! << WRONG




character(len=3, kind=4) :: str, str2
namelist /nml/ str

str = 4_'a'//char (int (z'4F60'),4)     &
      //char (int (z'597D'), 4)//4_'b'

open(6, encoding='utf-8')
write(*, '(a)') 4_'>'//str//4_'<'
write(*, *) 4_'>'//str//4_'<'
write(*,nml=nml)

open(99, encoding='utf-8',form='formatted')
write(99, '(3a)') '&nml str = "', str, '" /'
write(99, '(a)') str
rewind(99)

str = 4_'XXXX'
str2 = 4_'YYYY'
read(99,nml=nml)
read(99, *) str2
close(99, status='delete')

write(*, '(a)') 4_'>'//str//4_'<'
write(*, *) 4_'>'//str//4_'<'
write(*,nml=nml)
write(*, *) 4_'>'//str2//4_'<'
write(*, '(a)') 4_'>'//str2//4_'<'
end

[Bug libfortran/52539] I/O: Wrong result for UTF-8/UCS-4 list-directed and namelist read and nml write

[jvdelisle at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52539

--- Comment #1 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> 2012-03-10 14:33:14 UTC ---
I remember this being discussed before.  Lets check the standard regarding
encoding for namelists.

[Bug libfortran/52539] I/O: Wrong result for UTF-8/UCS-4 list-directed and namelist read and nml write

[burnus at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52539

Tobias Burnus <burnus at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |burnus at gcc dot gnu.org

--- Comment #2 from Tobias Burnus <burnus at gcc dot gnu.org> 2012-03-10 14:52:51 UTC ---
(In reply to comment #1)
> I remember this being discussed before.  Lets check the standard regarding
> encoding for namelists.

Well, the issue already occurs for list-directed reading ("read(99, *) str2").

One finds in F2008, 10.10.3p7 for list-directed read and in F2008, 10.11.3.3p7
for namelist read:

"When the next effective item is of type character, the input form consists of
a possibly delimited sequence of zero or more rep-chars whose kind type
parameter is implied by the kind of the effective item."

That clearly implies that it supports kind=4 (UCS-4, ISO 10646) characters -
and there is also no wording that the file encoding (UTF-8) doesn't apply to
list-directed read and to namelist read/write.

[Bug libfortran/52539] I/O: Wrong result for UTF-8/UCS-4 list-directed and namelist read and nml write

[dominiq at lps dot ens.fr]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52539

Dominique d'Humieres <dominiq at lps dot ens.fr> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |NEW
   Last reconfirmed|                            |2014-01-16
     Ever confirmed|0                           |1

--- Comment #3 from Dominique d'Humieres <dominiq at lps dot ens.fr> ---
Still present at r206648.

[Bug libfortran/52539] I/O: Wrong result for UTF-8/UCS-4 list-directed and namelist read and nml write

[jvdelisle at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52539

Jerry DeLisle <jvdelisle at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Assignee|unassigned at gcc dot gnu.org      |jvdelisle at gcc dot gnu.org

--- Comment #4 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> ---
I will start working on this one.

[Bug libfortran/52539] I/O: Wrong result for UTF-8/UCS-4 list-directed and namelist read and nml write

[jvdelisle at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52539

--- Comment #5 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> ---
I believe namelist writing is working correctly.

From the given test case and not deleting the file.

$ cat fort.99 
&nml str = "a你好" /
a你好

This narrows down to the read side of things.
>From gcc-bugs-return-447788-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org Fri Mar 28 22:03:10 2014
Return-Path: <gcc-bugs-return-447788-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org>
Delivered-To: listarch-gcc-bugs@gcc.gnu.org
Received: (qmail 27985 invoked by alias); 28 Mar 2014 22:03:10 -0000
Mailing-List: contact gcc-bugs-help@gcc.gnu.org; run by ezmlm
Precedence: bulk
List-Id: <gcc-bugs.gcc.gnu.org>
List-Archive: <http://gcc.gnu.org/ml/gcc-bugs/>
List-Post: <mailto:gcc-bugs@gcc.gnu.org>
List-Help: <mailto:gcc-bugs-help@gcc.gnu.org>
Sender: gcc-bugs-owner@gcc.gnu.org
Delivered-To: mailing list gcc-bugs@gcc.gnu.org
Received: (qmail 27933 invoked by uid 55); 28 Mar 2014 22:03:06 -0000
From: "law at gcc dot gnu.org" <gcc-bugzilla@gcc.gnu.org>
To: gcc-bugs@gcc.gnu.org
Subject: [Bug target/60648] [4.9 Regression] ICE (segmentation fault) in expand_binop
Date: Fri, 28 Mar 2014 22:03:00 -0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: changed
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Product: gcc
X-Bugzilla-Component: target
X-Bugzilla-Version: 4.9.0
X-Bugzilla-Keywords: ice-on-valid-code
X-Bugzilla-Severity: normal
X-Bugzilla-Who: law at gcc dot gnu.org
X-Bugzilla-Status: ASSIGNED
X-Bugzilla-Priority: P1
X-Bugzilla-Assigned-To: law at redhat dot com
X-Bugzilla-Target-Milestone: 4.9.0
X-Bugzilla-Flags:
X-Bugzilla-Changed-Fields:
Message-ID: <bug-60648-4-oj06ko2xCs@http.gcc.gnu.org/bugzilla/>
In-Reply-To: <bug-60648-4@http.gcc.gnu.org/bugzilla/>
References: <bug-60648-4@http.gcc.gnu.org/bugzilla/>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-Bugzilla-URL: http://gcc.gnu.org/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0
X-SW-Source: 2014-03/txt/msg02657.txt.bz2
Content-length: 744

http://gcc.gnu.org/bugzilla/show_bug.cgi?id`648

--- Comment #5 from Jeffrey A. Law <law at gcc dot gnu.org> ---
Author: law
Date: Fri Mar 28 22:02:32 2014
New Revision: 208924

URL: http://gcc.gnu.org/viewcvs?rev 8924&root=gcc&view=rev
Log:
    PR target/60648
       * expr.c (do_tablejump): Use simplify_gen_binary rather than
       gen_rtx_{PLUS,MULT} to build up the address expression.

       * i386/i386.c (ix86_legitimize_address): Use copy_addr_to_reg to avoid
       creating non-canonical RTL.

       PR target/60648
       * g++.dg/pr60648.C: New test.

Added:
    trunk/gcc/testsuite/g++.dg/pr60648.C
Modified:
    trunk/gcc/ChangeLog
    trunk/gcc/config/i386/i386.c
    trunk/gcc/expr.c
    trunk/gcc/testsuite/ChangeLog

[Bug libfortran/52539] I/O: Wrong result for UTF-8/UCS-4 list-directed and namelist read and nml write

[jvdelisle at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52539

--- Comment #6 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> ---
Patch part 1:

Index: write.c
===================================================================
--- write.c    (revision 208887)
+++ write.c    (working copy)
@@ -1835,7 +1835,10 @@
               break;

         case BT_CHARACTER:
-          write_character (dtp, p, 1, obj->string_length, DELIM);
+          if (dtp->u.p.current_unit->flags.encoding == ENCODING_UTF8)
+        write_character (dtp, p, 4, obj->string_length, DELIM);
+          else
+        write_character (dtp, p, 1, obj->string_length, DELIM);
               break;

         case BT_REAL:

Results:

 Write to terminal just the strings:
>a你好<
 >a你好<
 Write to the terminal the namelist:
&NML
 STR="a你好",
 /
>From gcc-bugs-return-447796-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org Sat Mar 29 00:00:24 2014
Return-Path: <gcc-bugs-return-447796-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org>
Delivered-To: listarch-gcc-bugs@gcc.gnu.org
Received: (qmail 20091 invoked by alias); 29 Mar 2014 00:00:24 -0000
Mailing-List: contact gcc-bugs-help@gcc.gnu.org; run by ezmlm
Precedence: bulk
List-Id: <gcc-bugs.gcc.gnu.org>
List-Archive: <http://gcc.gnu.org/ml/gcc-bugs/>
List-Post: <mailto:gcc-bugs@gcc.gnu.org>
List-Help: <mailto:gcc-bugs-help@gcc.gnu.org>
Sender: gcc-bugs-owner@gcc.gnu.org
Delivered-To: mailing list gcc-bugs@gcc.gnu.org
Received: (qmail 19998 invoked by uid 48); 29 Mar 2014 00:00:18 -0000
From: "jb at gcc dot gnu.org" <gcc-bugzilla@gcc.gnu.org>
To: gcc-bugs@gcc.gnu.org
Subject: [Bug middle-end/60646] Investigate improved complex division algorithms
Date: Sat, 29 Mar 2014 00:00:00 -0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: changed
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Product: gcc
X-Bugzilla-Component: middle-end
X-Bugzilla-Version: unknown
X-Bugzilla-Keywords:
X-Bugzilla-Severity: enhancement
X-Bugzilla-Who: jb at gcc dot gnu.org
X-Bugzilla-Status: UNCONFIRMED
X-Bugzilla-Priority: P3
X-Bugzilla-Assigned-To: unassigned at gcc dot gnu.org
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Flags:
X-Bugzilla-Changed-Fields: component
Message-ID: <bug-60646-4-LIC67RnNiz@http.gcc.gnu.org/bugzilla/>
In-Reply-To: <bug-60646-4@http.gcc.gnu.org/bugzilla/>
References: <bug-60646-4@http.gcc.gnu.org/bugzilla/>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-Bugzilla-URL: http://gcc.gnu.org/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0
X-SW-Source: 2014-03/txt/msg02665.txt.bz2
Content-length: 903

http://gcc.gnu.org/bugzilla/show_bug.cgi?id`646

Janne Blomqvist <jb at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
          Component|fortran                     |middle-end

--- Comment #6 from Janne Blomqvist <jb at gcc dot gnu.org> ---
(In reply to kargl from comment #1)
> Janne,
>
> Can you be a little more specific on what you want
> to investigate?  AFAIK, the general handling of
> complex division is done be the middle-end.

Mea culpa, I remembered that the frontend was generating the code for division.
Reassigning to the middle-end then.

(In reply to Steve Kargl from comment #5)
> PS: -ffast-math uses the naive complex division, which gives
> rather amusing results.

Yes, IMHO it could be argued that even with -ffast-math the naive algorithm
shouldn't be used.

[Bug libfortran/52539] I/O: Wrong result for UTF-8/UCS-4 list-directed and namelist read and nml write

[jvdelisle at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52539

--- Comment #8 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> ---
Author: jvdelisle
Date: Sat Apr 26 21:56:48 2014
New Revision: 209829

URL: http://gcc.gnu.org/viewcvs?rev=209829&root=gcc&view=rev
Log:
2014-04-26  Jerry DeLisle  <jvdelisle@gcc.gnu>

    PR libfortran/52539
    * gfortran.dg/namelist_utf8.f90: New test.

Added:
    trunk/gcc/testsuite/gfortran.dg/namelist_utf8.f90
Modified:
    trunk/gcc/testsuite/ChangeLog

[Bug libfortran/52539] I/O: Wrong result for UTF-8/UCS-4 list-directed and namelist read and nml write

[schwab@linux-m68k.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52539

Jerry DeLisle <jvdelisle at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|---                         |FIXED

Andreas Schwab <schwab@linux-m68k.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |REOPENED
         Resolution|FIXED                       |---

--- Comment #9 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> ---
Fixed on trunk. Closing

--- Comment #10 from Andreas Schwab <schwab@linux-m68k.org> ---
Not fixed.

Breakpoint 1, MAIN__ ()
    at ../../../../gcc/gcc/testsuite/gfortran.dg/namelist_utf8.f90:21
21      if (str2 /= str) call abort
(gdb) p str2
$1 = 4_'\x61000000你好b      '

Obviously not endian safe.
>From gcc-bugs-return-449978-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org Sun Apr 27 10:49:01 2014
Return-Path: <gcc-bugs-return-449978-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org>
Delivered-To: listarch-gcc-bugs@gcc.gnu.org
Received: (qmail 1500 invoked by alias); 27 Apr 2014 10:49:01 -0000
Mailing-List: contact gcc-bugs-help@gcc.gnu.org; run by ezmlm
Precedence: bulk
List-Id: <gcc-bugs.gcc.gnu.org>
List-Archive: <http://gcc.gnu.org/ml/gcc-bugs/>
List-Post: <mailto:gcc-bugs@gcc.gnu.org>
List-Help: <mailto:gcc-bugs-help@gcc.gnu.org>
Sender: gcc-bugs-owner@gcc.gnu.org
Delivered-To: mailing list gcc-bugs@gcc.gnu.org
Received: (qmail 1143 invoked by uid 48); 27 Apr 2014 10:48:56 -0000
From: "dominiq at lps dot ens.fr" <gcc-bugzilla@gcc.gnu.org>
To: gcc-bugs@gcc.gnu.org
Subject: [Bug fortran/52251] Nonadvancing I/O and the t edit descriptor
Date: Sun, 27 Apr 2014 10:49:00 -0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: changed
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Product: gcc
X-Bugzilla-Component: fortran
X-Bugzilla-Version: 4.7.0
X-Bugzilla-Keywords: wrong-code
X-Bugzilla-Severity: normal
X-Bugzilla-Who: dominiq at lps dot ens.fr
X-Bugzilla-Status: NEW
X-Bugzilla-Priority: P3
X-Bugzilla-Assigned-To: unassigned at gcc dot gnu.org
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Flags:
X-Bugzilla-Changed-Fields: bug_status cf_reconfirmed_on everconfirmed
Message-ID: <bug-52251-4-cDWPNfYzqB@http.gcc.gnu.org/bugzilla/>
In-Reply-To: <bug-52251-4@http.gcc.gnu.org/bugzilla/>
References: <bug-52251-4@http.gcc.gnu.org/bugzilla/>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-Bugzilla-URL: http://gcc.gnu.org/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0
X-SW-Source: 2014-04/txt/msg01998.txt.bz2
Content-length: 514

http://gcc.gnu.org/bugzilla/show_bug.cgi?idR251

Dominique d'Humieres <dominiq at lps dot ens.fr> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |NEW
   Last reconfirmed|                            |2014-04-27
     Ever confirmed|0                           |1

--- Comment #4 from Dominique d'Humieres <dominiq at lps dot ens.fr> ---
Still present at r209835 (4.10).

[Bug libfortran/52539] I/O: Wrong result for UTF-8/UCS-4 list-directed and namelist read and nml write

[schwab@linux-m68k.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52539

--- Comment #11 from Andreas Schwab <schwab@linux-m68k.org> ---
Mixing push_char and push_char4 can't be right.

[Bug libfortran/52539] I/O: Wrong result for UTF-8/UCS-4 list-directed and namelist read and nml write

[schwab@linux-m68k.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52539

--- Comment #12 from Andreas Schwab <schwab@linux-m68k.org> ---
! { dg-do run }
! PR52539 UTF-8 support for namelist read and write

character(len=10, kind=4) :: str, str2
character(len=25, kind=4) :: str3

namelist /nml/ str

str = 4_'1a'//char (int (z'4F60'),4)     &
      //char (int (z'597D'), 4)//4_'b'

open(99, encoding='utf-8',form='formatted')
write(99, '(3a)') '&nml str = "', str, '" /'
write(99, '(a)') str
rewind(99)

str = 4_'XXXX'
str2 = 4_'YYYY'
read(99,nml=nml)
read(99, *) str2
if (str2 /= str) call abort
rewind(99)

read(99,'(A)') str3
if (str3 /= 4_'&nml str = "' // str // 4_'" /') call abort
read(99,'(A)') str3
if (str3 /= str) call abort

close(99, status='delete')
end

[Bug libfortran/52539] I/O: Wrong result for UTF-8/UCS-4 list-directed and namelist read and nml write

[burnus at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52539

--- Comment #13 from Tobias Burnus <burnus at gcc dot gnu.org> ---
(In reply to Andreas Schwab from comment #12)
> ! { dg-do run }
[...]

Which is essentially the following.

-str = 4_'a'//char (int (z'4F60'),4)     &
+str = 4_'1a'//char (int (z'4F60'),4)     &

If that's sufficient to make the test case pass with a different endianness, it
is fine with me.

However, I do not understand why it should make a difference.

In general, I do not understand why there is an endian problem. "a" or "1a"
should simply lead to two 4-byte characters. The "int(z'4F60'),4)" should lead
to a single character, which depending on the endianness maps to a different
glyph. All characters should be convertable to UTF-8, which has no endianness
dependence, and convertable back to the original string - which one compares
to.

[Bug libfortran/52539] I/O: Wrong result for UTF-8/UCS-4 list-directed and namelist read and nml write

[schwab@linux-m68k.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52539

--- Comment #14 from Andreas Schwab <schwab@linux-m68k.org> ---
Run it.

[Bug libfortran/52539] I/O: Wrong result for UTF-8/UCS-4 list-directed and namelist read and nml write

[burnus at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52539

--- Comment #15 from Tobias Burnus <burnus at gcc dot gnu.org> ---
(In reply to Andreas Schwab from comment #14)
> Run it.

First, Jerry, I think you really should also compare the results against the
original string.

Secondly, with the modification, I get for the original string, str and str2
(and 'open(6, encoding="UTF-8")'):
 1a你好b                5
 1a你好b                5
 愱你好b                5

Thus, the namelist read is fine - but the list-directed read is not.
>From gcc-bugs-return-449989-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org Sun Apr 27 13:08:15 2014
Return-Path: <gcc-bugs-return-449989-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org>
Delivered-To: listarch-gcc-bugs@gcc.gnu.org
Received: (qmail 22945 invoked by alias); 27 Apr 2014 13:08:14 -0000
Mailing-List: contact gcc-bugs-help@gcc.gnu.org; run by ezmlm
Precedence: bulk
List-Id: <gcc-bugs.gcc.gnu.org>
List-Archive: <http://gcc.gnu.org/ml/gcc-bugs/>
List-Post: <mailto:gcc-bugs@gcc.gnu.org>
List-Help: <mailto:gcc-bugs-help@gcc.gnu.org>
Sender: gcc-bugs-owner@gcc.gnu.org
Delivered-To: mailing list gcc-bugs@gcc.gnu.org
Received: (qmail 22881 invoked by uid 48); 27 Apr 2014 13:08:09 -0000
From: "sebastian.huber@embedded-brains.de" <gcc-bugzilla@gcc.gnu.org>
To: gcc-bugs@gcc.gnu.org
Subject: [Bug middle-end/60102] powerpc fp-bit ices at dwf_regno
Date: Sun, 27 Apr 2014 13:08:00 -0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: changed
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Product: gcc
X-Bugzilla-Component: middle-end
X-Bugzilla-Version: 4.9.0
X-Bugzilla-Keywords:
X-Bugzilla-Severity: normal
X-Bugzilla-Who: sebastian.huber@embedded-brains.de
X-Bugzilla-Status: UNCONFIRMED
X-Bugzilla-Priority: P3
X-Bugzilla-Assigned-To: unassigned at gcc dot gnu.org
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Flags:
X-Bugzilla-Changed-Fields: cc
Message-ID: <bug-60102-4-OwkuE0HoiL@http.gcc.gnu.org/bugzilla/>
In-Reply-To: <bug-60102-4@http.gcc.gnu.org/bugzilla/>
References: <bug-60102-4@http.gcc.gnu.org/bugzilla/>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-Bugzilla-URL: http://gcc.gnu.org/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0
X-SW-Source: 2014-04/txt/msg02009.txt.bz2
Content-length: 3373

http://gcc.gnu.org/bugzilla/show_bug.cgi?id`102

Sebastian Huber <sebastian.huber@embedded-brains.de> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |hjl at gcc dot gnu.org

--- Comment #5 from Sebastian Huber <sebastian.huber@embedded-brains.de> ---
With different starting points I found another commit that leads to a similar
failure:

561af01c2c9ba4c5df95348d8252896088147e32 is the first bad commit
commit 561af01c2c9ba4c5df95348d8252896088147e32
Author: hjl <hjl@138bc75d-0d04-0410-961f-82ee72b054a4>
Date:   Wed Nov 27 23:54:26 2013 +0000

    Also handle REG_XXX notes in spill_pseudos

        PR rtl-optimization/59311
        * dwarf2cfi.c (dwf_regno): Assert reg isn't pseudo register.
        * lra-spills.c (spill_pseudos): Handle REG_XXX notes.


    git-svn-id: svn+ssh://gcc.gnu.org/svn/gcc/trunk@205468
138bc75d-0d04-0410-961f-82ee72b054a4

:040000 040000 518275c5696e814bd8f7970a5914e09f0fa7f5aa
16da683b4720ab0b343b910718ad7969b0713c2d M      gcc

This is the error message in this case:

/home/sh/git-build/b-gcc-git-powerpc-eabispe/./gcc/xgcc
-B/home/sh/git-build/b-gcc-git-powerpc-eabispe/./gcc/ -nostdinc
-B/home/sh/git-build/b-gcc-git-powerpc-eabispe/powerpc-eabispe/newlib/ -isystem
/home/sh/git-build/b-gcc-git-powerpc-eabispe/powerpc-eabispe/newlib/targ-include
-isystem /home/sh/archive/gcc-git/newlib/libc/include
-B/home/sh/install-gcc-git/powerpc-eabispe/bin/
-B/home/sh/install-gcc-git/powerpc-eabispe/lib/ -isystem
/home/sh/install-gcc-git/powerpc-eabispe/include -isystem
/home/sh/install-gcc-git/powerpc-eabispe/sys-include    -g -O2 -msoft-float -O2
 -g -O2 -DIN_GCC  -DCROSS_DIRECTORY_STRUCTURE  -W -Wall -Wno-narrowing
-Wwrite-strings -Wcast-qual -Wstrict-prototypes -Wmissing-prototypes
-Wold-style-definition  -isystem ./include   -g -DIN_LIBGCC2 -fbuilding-libgcc
-fno-stack-protector -Dinhibit_libc  -I. -I. -I../../.././gcc
-I/home/sh/archive/gcc-git/libgcc -I/home/sh/archive/gcc-git/libgcc/.
-I/home/sh/archive/gcc-git/libgcc/../gcc
-I/home/sh/archive/gcc-git/libgcc/../include  -DHAVE_CC_TLS  -o _divxc3.o -MT
_divxc3.o -MD -MP -MF _divxc3.dep -DL_divxc3 -c
/home/sh/archive/gcc-git/libgcc/libgcc2.c -fvisibility=hidden -DHIDE_EXPORTS
0x406285 dwf_regno
        /home/sh/archive/gcc-git/gcc/dwarf2cfi.c:909
0x534aa4 vec<queued_reg_save, va_heap, vl_embed>::truncate(unsigned int)
        /home/sh/archive/gcc-git/gcc/vec.h:892
0x534aa4 vec<queued_reg_save, va_heap, vl_ptr>::truncate(unsigned int)
        /home/sh/archive/gcc-git/gcc/vec.h:1559
0x534aa4 dwarf2out_flush_queued_reg_saves
        /home/sh/archive/gcc-git/gcc/dwarf2cfi.c:996
0x53616f scan_trace
        /home/sh/archive/gcc-git/gcc/dwarf2cfi.c:2522
0x536b1e create_cfi_notes
        /home/sh/archive/gcc-git/gcc/dwarf2cfi.c:2565
0x536b1e execute_dwarf2_frame
        /home/sh/archive/gcc-git/gcc/dwarf2cfi.c:2925
0x536b1e execute
        /home/sh/archive/gcc-git/gcc/dwarf2cfi.c:3421
Please submit a full bug report,
with preprocessed source if appropriate.
Please include the complete backtrace with any bug report.
See <http://gcc.gnu.org/bugs.html> for instructions

In case I revert this patch on the current 4.9 branch
(5935f964089edccd96efa7f0bda85800dc0ee31d) I am able to build the
powerpc-eabispe target.

[Bug libfortran/52539] I/O: Wrong result for UTF-8/UCS-4 list-directed and namelist read and nml write

[jvdelisle at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52539

--- Comment #16 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> ---
I have to get my head around this. A user should not have to artificially
construct a string to accommodate endianess.  Also, I think I see the buffer
overflow issue Andreas spotted. I appreciate the comments.

[Bug libfortran/52539] I/O: Wrong result for UTF-8/UCS-4 list-directed and namelist read and nml write

[burnus at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52539

--- Comment #17 from Tobias Burnus <burnus at gcc dot gnu.org> ---
(In reply to Jerry DeLisle from comment #16)
> A user should not have to artificially construct a string to accommodate
> endianess.

In principle, the user doesn't have to do so. UTF-8 itself is agnostic to the
endianness of the system.

The endian issue only comes up in the source code. To solve this, we have to
handle UTF-8 when reading the source code. libcpp, which we use for the input,
handles multiple encodings. We should use find out how to use this - and use it
for 4_'strings'.

[Bug libfortran/52539] I/O: Wrong result for UTF-8/UCS-4 list-directed and namelist read and nml write

[jvdelisle at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52539

--- Comment #18 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> ---
(In reply to Tobias Burnus from comment #15)
> (In reply to Andreas Schwab from comment #14)
> > Run it.
> 
> First, Jerry, I think you really should also compare the results against the
> original string.
> 
> Secondly, with the modification, I get for the original string, str and str2
> (and 'open(6, encoding="UTF-8")'):
>  1a你好b                5
>  1a你好b                5
>  愱你好b                5
> 
> Thus, the namelist read is fine - but the list-directed read is not.

Here on x86_64 I am testing against the original string.
Breakpoint 1, MAIN__ () at test.f90:21
21    if (str2 /= str) call abort
(gdb) p str
$1 = 4_'a你好b      '
(gdb) p str2
$2 = 4_'a你好b      '
(gdb) n
22    rewind(99)
(gdb) n
24    read(99,'(A)') str3
(gdb) n
25    if (str3 /= 4_'&nml str = "' // str // 4_'" /') call abort
(gdb) p str3
$3 = 4_'&nml str = "a你好b      " /'
(gdb) n
26    read(99,*) str3
(gdb) n
27    if (str3 /= str) call abort
(gdb) p str3
$4 = 4_'a你好b', ' ' <repeats 21 times>

At line 26 is the list-directed read and line 27 is the check against the
original string.

The test case is namelist_utf8.f90.  Can someone show me the above results for
other endian.  If its aborting just do s/call abort/print *, "Fails"/

Thanks in advance.
>From gcc-bugs-return-450006-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org Sun Apr 27 18:06:43 2014
Return-Path: <gcc-bugs-return-450006-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org>
Delivered-To: listarch-gcc-bugs@gcc.gnu.org
Received: (qmail 20582 invoked by alias); 27 Apr 2014 18:06:42 -0000
Mailing-List: contact gcc-bugs-help@gcc.gnu.org; run by ezmlm
Precedence: bulk
List-Id: <gcc-bugs.gcc.gnu.org>
List-Archive: <http://gcc.gnu.org/ml/gcc-bugs/>
List-Post: <mailto:gcc-bugs@gcc.gnu.org>
List-Help: <mailto:gcc-bugs-help@gcc.gnu.org>
Sender: gcc-bugs-owner@gcc.gnu.org
Delivered-To: mailing list gcc-bugs@gcc.gnu.org
Received: (qmail 20547 invoked by uid 48); 27 Apr 2014 18:06:39 -0000
From: "schwab@linux-m68k.org" <gcc-bugzilla@gcc.gnu.org>
To: gcc-bugs@gcc.gnu.org
Subject: [Bug libfortran/52539] I/O: Wrong result for UTF-8/UCS-4 list-directed and namelist read and nml write
Date: Sun, 27 Apr 2014 18:06:00 -0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: changed
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Product: gcc
X-Bugzilla-Component: libfortran
X-Bugzilla-Version: 4.8.0
X-Bugzilla-Keywords: wrong-code
X-Bugzilla-Severity: normal
X-Bugzilla-Who: schwab@linux-m68k.org
X-Bugzilla-Status: REOPENED
X-Bugzilla-Priority: P3
X-Bugzilla-Assigned-To: jvdelisle at gcc dot gnu.org
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Flags:
X-Bugzilla-Changed-Fields:
Message-ID: <bug-52539-4-eZyl1T6LD0@http.gcc.gnu.org/bugzilla/>
In-Reply-To: <bug-52539-4@http.gcc.gnu.org/bugzilla/>
References: <bug-52539-4@http.gcc.gnu.org/bugzilla/>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-Bugzilla-URL: http://gcc.gnu.org/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0
X-SW-Source: 2014-04/txt/msg02026.txt.bz2
Content-length: 145

http://gcc.gnu.org/bugzilla/show_bug.cgi?idR539

--- Comment #19 from Andreas Schwab <schwab@linux-m68k.org> ---
Use the test case from #c12.

[Bug libfortran/52539] I/O: Wrong result for UTF-8/UCS-4 list-directed and namelist read and nml write

[jvdelisle at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52539

--- Comment #20 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> ---
Created attachment 32725
  --> http://gcc.gnu.org/bugzilla/attachment.cgi?id=32725&action=edit
New preliminary patch - please test

This patch takes a different approach. I have split up next_char and push_char
into several smaller functions to eliminate testing for ENCODING and
is_internal on every character read.

I have added two function pointers to the the gfc_unit structure and these
pointers are initialized at the beginning of the I/O statements once. I then
use wrapper functions to invoke the function pointers.  This approach keeps
everything nicely separated, is cleaner, and should have better performance.

I will probably replace the wrapper functions with simple macro expansions. 
However as it is attached it is good for some heavy testing by everyone.

I have regression tested on x86_64 and the test in Comment #12 works now.

[Bug libfortran/52539] I/O: Wrong result for UTF-8/UCS-4 list-directed and namelist read and nml write

[dominiq at lps dot ens.fr]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52539

--- Comment #21 from Dominique d'Humieres <dominiq at lps dot ens.fr> ---
> I have regression tested on x86_64 and the test in Comment #12 works now.

Confirmed on x86_64-apple-darwin13.

[Bug libfortran/52539] I/O: Wrong result for UTF-8/UCS-4 list-directed and namelist read and nml write

[jvdelisle at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52539

--- Comment #22 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> ---
One failure on NIST FM906.f

[Bug libfortran/52539] I/O: Wrong result for UTF-8/UCS-4 list-directed and namelist read and nml write

[dominiq at lps dot ens.fr]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52539

--- Comment #23 from Dominique d'Humieres <dominiq at lps dot ens.fr> ---
> One failure on NIST FM906.f

Not due to this patch. The following test

character(len=30) :: buff = ", (2.0, 3.0),,6.0D0, 2*,"
DOUBLE PRECISION AVD, BVD, CVD, DVCORR 
COMPLEX AVC, BVC, CVC, ZVCORR

read(buff, *)  AVD, AVC, BVC, BVD, CVC, CVD

end

fails with 4.9.0 and trunk

At line 5 of file FM906.f90
Fortran runtime error: Bad repeat count in item 5 of list input

I'll open a new PR for it.

[Bug libfortran/52539] I/O: Wrong result for UTF-8/UCS-4 list-directed and namelist read and nml write

[dominiq at lps dot ens.fr]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52539

--- Comment #24 from Dominique d'Humieres <dominiq at lps dot ens.fr> ---
On powerpc-apple-darwin9 with the patch in comment 20, the test
gfortran.dg/namelist_utf8.f90 fails. The following modified test

[karma] f90/bug% cat > pr52539_2_db.f90
! { dg-do run }
! PR52539 UTF-8 support for namelist read and write

character(len=10, kind=4) :: str, str2
character(len=25, kind=4) :: str3

namelist /nml/ str

str = 4_'1a'//char (int (z'4F60'),4)     &
      //char (int (z'597D'), 4)//4_'b'

open(6, encoding='utf-8')
open(99, encoding='utf-8',form='formatted')
write(99, '(3a)') '&nml str = "', str, '" /'
write(99, '(a)') str
rewind(99)

str = 4_'XXXX'
str2 = 4_'YYYY'
read(99,nml=nml)
read(99, *) str2
print *, "'", str, "'  '", str2, "'"
!if (str2 /= str) call abort
rewind(99)

read(99,'(A)') str3
print *, "'", str, "'  '", str3, "'"
if (str3 /= 4_'&nml str = "' // str // 4_'" /') call abort
read(99,'(A)') str3
if (str3 /= str) call abort

close(99, status='delete')
end

gives

 '??????????????????X      '  '1a你好b '
 '??????????????????X      '  '&nml str = "1a你好b     " /'

Program aborted. Backtrace:
#0  0xeee9b
Abort

while the file fort.99 is

&nml str = "1a你好b     " /
1a你好b
>From gcc-bugs-return-450542-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org Mon May 05 09:08:30 2014
Return-Path: <gcc-bugs-return-450542-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org>
Delivered-To: listarch-gcc-bugs@gcc.gnu.org
Received: (qmail 12427 invoked by alias); 5 May 2014 09:08:29 -0000
Mailing-List: contact gcc-bugs-help@gcc.gnu.org; run by ezmlm
Precedence: bulk
List-Id: <gcc-bugs.gcc.gnu.org>
List-Archive: <http://gcc.gnu.org/ml/gcc-bugs/>
List-Post: <mailto:gcc-bugs@gcc.gnu.org>
List-Help: <mailto:gcc-bugs-help@gcc.gnu.org>
Sender: gcc-bugs-owner@gcc.gnu.org
Delivered-To: mailing list gcc-bugs@gcc.gnu.org
Received: (qmail 12371 invoked by uid 48); 5 May 2014 09:08:23 -0000
From: "rguenth at gcc dot gnu.org" <gcc-bugzilla@gcc.gnu.org>
To: gcc-bugs@gcc.gnu.org
Subject: [Bug tree-optimization/61034] Optimizing takes too many passes
Date: Mon, 05 May 2014 09:08:00 -0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: changed
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Product: gcc
X-Bugzilla-Component: tree-optimization
X-Bugzilla-Version: 4.10.0
X-Bugzilla-Keywords: missed-optimization
X-Bugzilla-Severity: normal
X-Bugzilla-Who: rguenth at gcc dot gnu.org
X-Bugzilla-Status: ASSIGNED
X-Bugzilla-Priority: P3
X-Bugzilla-Assigned-To: rguenth at gcc dot gnu.org
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Flags:
X-Bugzilla-Changed-Fields: bug_status cf_reconfirmed_on assigned_to everconfirmed
Message-ID: <bug-61034-4-xePlgswGnh@http.gcc.gnu.org/bugzilla/>
In-Reply-To: <bug-61034-4@http.gcc.gnu.org/bugzilla/>
References: <bug-61034-4@http.gcc.gnu.org/bugzilla/>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-Bugzilla-URL: http://gcc.gnu.org/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0
X-SW-Source: 2014-05/txt/msg00234.txt.bz2
Content-length: 1480

http://gcc.gnu.org/bugzilla/show_bug.cgi?ida034

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |ASSIGNED
   Last reconfirmed|                            |2014-05-05
           Assignee|unassigned at gcc dot gnu.org      |rguenth at gcc dot gnu.org
     Ever confirmed|0                           |1

--- Comment #1 from Richard Biener <rguenth at gcc dot gnu.org> ---
You are talking about

  _73 = __builtin_malloc (16);
...
  _79 = _73;
...
  _79->count = _81;
...
  _83->count = _85;
...
  _86 = _73;
  _87 = _86->count;

note that FRE doesn't do copy-propagation at its simplification stage.
Usually the issue is that alias info isn't the best possible, and indeed:

  # PT = nonlocal escaped
  _2 = MEM[(const struct I &)c_3(D)].o;
...
  # PT = nonlocal escaped { D.2585 }
  _83 = _2;
...
  # PT = { D.2585 }
  _86 = _73;
  _87 = _86->count;

so _86->count is possibly clobbered by the store because the pointer may
point to the malloc result.

So ... as FRE figures out some pointer equivalencies it could ideally
also take the stricter points-to info into account.  It already somewhat
does this, but obviously not fully (see vn_reference_lookup_3, the
"First try to disambiguate ..." code at the beginning).

Hmm.

Confirmed (FRE is to blame).

I will have a look (at some point).

[Bug libfortran/52539] I/O: Wrong result for UTF-8/UCS-4 list-directed and namelist read and nml write

[jvdelisle at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52539

--- Comment #25 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> ---
I have managed to get to a machine on the gcc compile farm to do some further
debugging. (gcc110)

[Bug libfortran/52539] I/O: Wrong result for UTF-8/UCS-4 list-directed and namelist read and nml write

[jvdelisle at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52539

Jerry DeLisle <jvdelisle at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|jvdelisle at gcc dot gnu.org       |

--- Comment #26 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> ---
I am unable to reproduce the problem in Comment #24 on the ppc64 machine I have
access to. Its not a Mac and its running Linux based system.

Dominique: Do you have a debugger on that machine? If so, lets converse off PR
by email and will see if we can track it down.

[Bug libfortran/52539] I/O: Wrong result for UTF-8/UCS-4 list-directed and namelist read and nml write

[jvdelisle at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=52539

--- Comment #27 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> ---
Author: jvdelisle
Date: Sun May 18 02:29:27 2014
New Revision: 210574

URL: http://gcc.gnu.org/viewcvs?rev=210574&root=gcc&view=rev
Log:
2014-05-17  Jerry DeLisle  <jvdelisle@gcc.gnu>

    PR libfortran/52539
    * io/io.h (gfc_unit): New function pointers *next_char_fn_ptr
    and *push_char_fn_ptr.
    *io/list_read.c (next_char): Create macro with this name to call
    the new function pointer. Split the original next_char function
    into three new functions. (next_char_default, next_char_internal,
    next_char_utf8): New functions. (push_char): Create macro with
    this name to call new function pointer. Split the original
    push_char into three new functions. (push_char_default,
    push_char_internal, push_char4): New functions. (set_workers):
    New function to initilize the function pointers depending on the
    type of IO to be performed. (list_formatted_read_scalar): Use
    set_workers function. (finish_list_read): Likewise.
    (namelist_read): Likewise.
    (nml_get_obj_data): Use push_char_default.

Modified:
    trunk/libgfortran/ChangeLog
    trunk/libgfortran/io/io.h
    trunk/libgfortran/io/list_read.c

[Bug libfortran/52539] I/O: Wrong result for UTF-8/UCS-4 list-directed and namelist read and nml write

[jvdelisle at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=52539

--- Comment #28 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> ---
Author: jvdelisle
Date: Sun May 18 02:34:02 2014
New Revision: 210575

URL: http://gcc.gnu.org/viewcvs?rev=210575&root=gcc&view=rev
Log:
2014-05-17  Jerry DeLisle  <jvdelisle@gcc.gnu>

    PR libfortran/52539
    * gfortran.dg/namelist_utf8.f90: New test.

Modified:
    trunk/gcc/testsuite/ChangeLog
    trunk/gcc/testsuite/gfortran.dg/namelist_utf8.f90

[Bug libfortran/52539] I/O: Wrong result for UTF-8/UCS-4 list-directed and namelist read and nml write

[burnus at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=52539

--- Comment #29 from Tobias Burnus <burnus at gcc dot gnu.org> ---
Status: Patch committed to the trunk (4.10), but fails (xfailed) on
powerpc*-apple-darwin*.

[Bug libfortran/52539] I/O: Wrong result for UTF-8/UCS-4 list-directed and namelist read and nml write

[jvdelisle at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=52539

Jerry DeLisle <jvdelisle at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|REOPENED                    |RESOLVED
         Resolution|---                         |FIXED

--- Comment #30 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> ---
Closing.