libgcj/1325: StreamTokenizer doesn't handle escaped newline

libgcj/1325: StreamTokenizer doesn't handle escaped newline

[osk]

>Number:         1325
>Category:       libgcj
>Synopsis:       StreamTokenizer doesn't handle escaped newline
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    tromey
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Wed Dec 20 12:19:03 PST 2000
>Closed-Date:    Sat Aug 05 17:41:39 PDT 2000
>Last-Modified:  Sat Aug  5 17:50:00 PDT 2000
>Originator:     Oskar Liljeblad
>Release:        unknown-1.0
>Organization:
>Environment:
libgcj-snapshot-2000-07-24
>Description:
The StreamTokenizer of Sun's JDK supports escaping of
a newline (CR/LF) by putting a backslash in front of it.
This is not supported in the GCJ or Classpath
StreamTokenizer class.

The program below gives different output for GCJ and
Sun's JDK.

I did some testing and it seems that only the first
newline character is escaped, even if the stream
contains <CR><LF> newlines.
>How-To-Repeat:
import java.io.*;
class Test {
  public static void main(String[] args) throws IOException {
    StringReader r = new StringReader("\"foo\\\nbar\" after");
    StreamTokenizer st = new StreamTokenizer(r);
    System.out.println(st.nextToken()+": "+st.sval.length()+" >"+st.sval+"<");
    System.out.println(st.nextToken()+": "+st.sval.length()+" >"+st.sval+"<");
  }
}  
>Fix:
--- libjava/java/io/StreamTokenizer.java.old    Fri Aug  4 18:08:33 2000
+++ libjava/java/io/StreamTokenizer.java        Fri Aug  4 18:04:49 2000
@@ -365,6 +365,10 @@
                    break;
                  case 'v':     ch = 0xB;
                    break;
+                 case '\n':    ch = '\n';
+                   break;
+                 case '\r':    ch = '\r';
+                   break;
                  case '\"':
                  case '\'':
                  case '\\':
>Release-Note:

>Audit-Trail:

Formerly PR java.io/301

State-Changed-From-To: open->closed
State-Changed-By: tromey
State-Changed-When: Sat Aug  5 17:41:39 2000
State-Changed-Why:
    I'm checking in the patch you suggested.
    Thanks.

From: tromey@cygnus.com
To: java-gnats@sourceware.cygnus.com, osk@hem.passagen.se, tromey@cygnus.com
Cc:  
Subject: Re: java.io/301
Date: 6 Aug 2000 00:41:39 -0000

 Synopsis: StreamTokenizer doesn't handle escaped newline
 
 State-Changed-From-To: open->closed
 State-Changed-By: tromey
 State-Changed-When: Sat Aug  5 17:41:39 2000
 State-Changed-Why:
     I'm checking in the patch you suggested.
     Thanks.
 
 http://sources.redhat.com/cgi-bin/gnatsweb.pl?cmd=view&pr=301&database=java

From: tromey@sourceware.cygnus.com
To: java-gnats@sources.redhat.com
Cc:  
Subject: java.io/301
Date: 6 Aug 2000 00:49:29 -0000

 CVSROOT:	/cvs/java
 Module name:	libgcj
 Changes by:	tromey@sourceware.cygnus.com	2000-08-05 17:49:29
 
 Modified files:
 	libjava        : ChangeLog 
 	libjava/java/io: StreamTokenizer.java 
 
 Log message:
 	* java/io/StreamTokenizer.java (TT_NONE): Now private.
 	(nextToken): Handle backslashed newline.  From Oskar Liljeblad.
 	For PR java.io/301.
 
 Patches:
 http://gcc.gnu.org/cgi-bin/gcc/libjava/ChangeLog.diff?cvsroot=java&r1=1.441&r2=1.442
 http://gcc.gnu.org/cgi-bin/gcc/libjava/java/io/StreamTokenizer.java.diff?cvsroot=java&r1=1.5&r2=1.6
 
>Unformatted: