From mboxrd@z Thu Jan 1 00:00:00 1970 From: jsturm@sigma6.com To: java-gnats@sourceware.cygnus.com Subject: libgcj/1198: StreamTokenizer doesn't scan mixed token types correctly Date: Wed, 20 Dec 2000 12:21:00 -0000 Message-id: <20000123212908.27962.qmail@sourceware.cygnus.com> X-SW-Source: 2000-q4/msg00977.html List-Id: >Number: 1198 >Category: libgcj >Synopsis: StreamTokenizer doesn't scan mixed token types correctly >Confidential: no >Severity: serious >Priority: medium >Responsible: aph >State: closed >Class: sw-bug >Submitter-Id: net >Arrival-Date: Wed Dec 20 12:17:26 PST 2000 >Closed-Date: Mon Sep 04 13:35:49 PDT 2000 >Last-Modified: Mon Sep 4 13:40:00 PDT 2000 >Originator: Jeff Sturm >Release: libgcj (current) >Organization: >Environment: alphapca56-unknown-linux-gnu >Description: While trying to scan the input below: (a).(b) output of StreamTokenizer (from libgcj) doesn't match that of the JDK. >How-To-Repeat: Compile/run the attached program: gcj StreamTokenizerTest.java --main=StreamTokenizerTest -o st ./st '(a).(b)' Expected output is: ( [TT_WORD]: a ) [TT_NUMBER]: 0.0 ( [TT_WORD]: b ) [TT_EOF] >Fix: >Release-Note: >Audit-Trail: Formerly PR java.io/141 From: green@sourceware.cygnus.com To: java-gnats@sourceware.cygnus.com Cc: Subject: java.io/141 Date: 6 Mar 2000 03:48:34 -0000 CVSROOT: /cvs/java Module name: libgcj Changes by: green@sourceware.cygnus.com 00/03/05 19:48:34 Modified files: libjava/testsuite: ChangeLog Added files: libjava/testsuite/libjava.lang: PR141.java PR141.out Log message: 2000-03-05 Anthony Green * libjava.lang/PR141.java: New file. * libjava.lang/PR141.out: New file. For PR java.io/141. Patches: http://gcc.gnu.org/cgi-bin/gcc/libjava/testsuite/ChangeLog.diff?cvsroot=java&r1=1.58&r2=1.59 http://gcc.gnu.org/cgi-bin/gcc/libjava/testsuite/libjava.lang/PR141.java.diff?cvsroot=java&r1=NONE&r2=1.1 http://gcc.gnu.org/cgi-bin/gcc/libjava/testsuite/libjava.lang/PR141.out.diff?cvsroot=java&r1=NONE&r2=1.1 From: Tom Tromey To: Java Gnats Server Cc: Subject: java.io/141 Date: Wed, 9 Aug 2000 17:26:20 -0700 (PDT) I checked in a Mauve test case for this bug. Tom From: Tom Tromey To: Jeff Sturm Cc: Java Gnats Server , Warren Levy Subject: java.io/141 Date: 09 Aug 2000 18:56:11 -0600 I looked at this PR a bit more. I believe the JDK is wrong. If you follow the JLS definition of a floating point constant, `.' is not one. Alternatively, one could argue that compatibility is important here, and that the StreamTokenizer definition of a number is different from the JLS definition. I personally think that the JDK is wrong, though. Warren, what do you think? I read the JCL book on this subject but it isn't very helpful either. Tom From: Jeff Sturm To: tromey@cygnus.com Cc: Java Gnats Server , Warren Levy Subject: Re: java.io/141 Date: Fri, 11 Aug 2000 12:19:21 -0400 Tom Tromey wrote: > I believe the JDK is wrong. > If you follow the JLS definition of a floating point constant, `.' is > not one. That sounds right. > Alternatively, one could argue that compatibility is important here, > and that the StreamTokenizer definition of a number is different from > the JLS definition. Since the JCL doesn't really say how `.' should be scanned, you can make an argument either way. (I don't really want `.' to be scanned as a number, but I can live with it.) IIRC an exception was getting thrown when I tried to scan some source code with StreamTokenizer. That made StreamTokenizer useless for me. I don't think nextToken() should ever throw an exception other than IOException. Perhaps I should have stated that in the bug report. -- Jeff Sturm jeff.sturm@appnet.com Responsible-Changed-From-To: tromey->warrenl Responsible-Changed-By: tromey Responsible-Changed-When: Sat Aug 19 18:25:20 2000 Responsible-Changed-Why: Warren, this is yours. Currently StreamTokenizer throws NumberFormatException for `.', but I think it shouldn't. From: tromey@cygnus.com To: java-gnats@sourceware.cygnus.com, jsturm@sigma6.com, tromey@cygnus.com, warrenl@cygnus.com Cc: Subject: Re: java.io/141 Date: 20 Aug 2000 01:25:20 -0000 Synopsis: StreamTokenizer doesn't scan mixed token types correctly Responsible-Changed-From-To: tromey->warrenl Responsible-Changed-By: tromey Responsible-Changed-When: Sat Aug 19 18:25:20 2000 Responsible-Changed-Why: Warren, this is yours. Currently StreamTokenizer throws NumberFormatException for `.', but I think it shouldn't. http://sources.redhat.com/cgi-bin/gnatsweb.pl?cmd=view&pr=141&database=java Responsible-Changed-From-To: warrenl->aph Responsible-Changed-By: tromey Responsible-Changed-When: Mon Sep 4 13:35:49 2000 Responsible-Changed-Why: Andrew fixed it. State-Changed-From-To: open->closed State-Changed-By: tromey State-Changed-When: Mon Sep 4 13:35:49 2000 State-Changed-Why: Andrew checked in a patch for this bug. From: tromey@cygnus.com To: aph@cygnus.com, java-gnats@sourceware.cygnus.com, jsturm@sigma6.com, warrenl@cygnus.com Cc: Subject: Re: java.io/141 Date: 4 Sep 2000 20:35:49 -0000 Synopsis: StreamTokenizer doesn't scan mixed token types correctly Responsible-Changed-From-To: warrenl->aph Responsible-Changed-By: tromey Responsible-Changed-When: Mon Sep 4 13:35:49 2000 Responsible-Changed-Why: Andrew fixed it. State-Changed-From-To: open->closed State-Changed-By: tromey State-Changed-When: Mon Sep 4 13:35:49 2000 State-Changed-Why: Andrew checked in a patch for this bug. http://sources.redhat.com/cgi-bin/gnatsweb.pl?cmd=view&pr=141&database=java >Unformatted: ----gnatsweb-attachment---- Content-Type: application/octet-stream; name="StreamTokenizerTest.java" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="StreamTokenizerTest.java" aW1wb3J0IGphdmEuaW8uSU9FeGNlcHRpb247Ci8vaW1wb3J0IGphdmEuaW8uU3RyZWFtVG9rZW5p emVyOwppbXBvcnQgamF2YS5pby5TdHJpbmdSZWFkZXI7CgpwdWJsaWMgY2xhc3MgU3RyZWFtVG9r ZW5pemVyVGVzdCB7CglwdWJsaWMgc3RhdGljIHZvaWQgdGVzdChTdHJpbmcgc3RyKSB0aHJvd3Mg SU9FeGNlcHRpb24gewoJCVN0cmluZ1JlYWRlciByZWFkZXIgPSBuZXcgU3RyaW5nUmVhZGVyKHN0 cik7CgkJU3RyZWFtVG9rZW5pemVyIHN0ID0gbmV3IFN0cmVhbVRva2VuaXplcihyZWFkZXIpOwoJ CWludCB0ID0gMDsKCgkJZG8gewoJCQl0ID0gc3QubmV4dFRva2VuKCk7CgoJCQlzd2l0Y2ggKHQp IHsKCQkJCWNhc2UgU3RyZWFtVG9rZW5pemVyLlRUX1dPUkQ6CgkJCQkJU3lzdGVtLm91dC5wcmlu dGxuKCJbVFRfV09SRF06ICIgKyBzdC5zdmFsKTsKCQkJCQlicmVhazsKCQkJCWNhc2UgU3RyZWFt VG9rZW5pemVyLlRUX05VTUJFUjoKCQkJCQlTeXN0ZW0ub3V0LnByaW50bG4oIltUVF9OVU1CRVJd OiAiICsgc3QubnZhbCk7CgkJCQkJYnJlYWs7CgkJCQljYXNlIFN0cmVhbVRva2VuaXplci5UVF9F T0w6CgkJCQkJU3lzdGVtLm91dC5wcmludGxuKCJbVFRfRU9MXSIpOwoJCQkJCWJyZWFrOwoJCQkJ Y2FzZSBTdHJlYW1Ub2tlbml6ZXIuVFRfRU9GOgoJCQkJCVN5c3RlbS5vdXQucHJpbnRsbigiW1RU X0VPRl0iKTsKCQkJCQlicmVhazsKCQkJCWRlZmF1bHQ6CgkJCQkJU3lzdGVtLm91dC5wcmludGxu KChjaGFyKXQpOwoJCQkJCWJyZWFrOwoJCQl9CgkJfSB3aGlsZSAodCAhPSBTdHJlYW1Ub2tlbml6 ZXIuVFRfRU9GKTsKCX0KCglwdWJsaWMgc3RhdGljIHZvaWQgbWFpbihTdHJpbmdbXSBhcmdzKSB7 CgkJZm9yIChpbnQgbiA9IDA7IG4gPCBhcmdzLmxlbmd0aDsgbisrKSB7CgkJCXRyeSB7CgkJCQl0 ZXN0KGFyZ3Nbbl0pOwoJCQl9IGNhdGNoIChUaHJvd2FibGUgdCkgewoJCQkJdC5wcmludFN0YWNr VHJhY2UoKTsKCQkJfQoJCX0KCX0KfQo=