From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 16581 invoked by alias); 27 Jul 2006 13:08:31 -0000 Received: (qmail 16572 invoked by uid 22791); 27 Jul 2006 13:08:30 -0000 X-Spam-Check-By: sourceware.org Received: from outmail128150.authsmtp.com (HELO outmail128150.authsmtp.com) (62.13.128.150) by sourceware.org (qpsmtpd/0.31) with ESMTP; Thu, 27 Jul 2006 13:08:26 +0000 Received: from [192.168.1.32] (host217-37-65-246.in-addr.btopenworld.com [217.37.65.246]) (authenticated bits=0) by squirrel.dmpriest.net.uk (8.13.6/8.13.6/Kp) with ESMTP id k6RD8Ifh083669; Thu, 27 Jul 2006 14:08:19 +0100 (BST) Message-ID: <44C8BAFF.6080001@object-refinery.com> Date: Thu, 27 Jul 2006 13:08:00 -0000 From: David Gilbert User-Agent: Mozilla Thunderbird 1.0.8 (X11/20060502) MIME-Version: 1.0 To: Roman Kennke CC: classpath@gnu.org, mauve-discuss@sourceware.org Subject: Re: Testing JDK bugs? References: <44C88C1B.6000408@kennke.org> In-Reply-To: <44C88C1B.6000408@kennke.org> Content-Type: text/plain; charset=ISO-8859-15; format=flowed Content-Transfer-Encoding: 7bit X-Server-Quench: f96f561b-1d70-11db-b770-001185d377ca X-AuthRoute: OCdyYgsUB1ZZRRob BmULDStKRB85DhpG AxIIMU5ALVQIUwlL IwBRKV9FNUAYWVBB XzMTWE0aUlpuIyYp dAhRbQNNY05EXw1g UUEHQFNVCgZvHxoE GBwaVAZwchpGNiV2 BzJqJyYDXEx5c0B0 DE1WHWtIM25mPmUZ URJFdldTcB5Lf0sX dwJ4U3MQYWUGZ3Jl E1RsYDs4Kw9Semxp TxoRLFQdCWQnPXY2 Wh8ZSl1z X-Authentic-SMTP: 61633132333134.squirrel.dmpriest.net.uk:199/Kp X-Report-SPAM: If SPAM / abuse - report it at: http://www.authsmtp.com/abuse X-Virus-Status: No virus detected - but ensure you scan with your own anti-virus system! X-IsSubscribed: yes Mailing-List: contact mauve-discuss-help@sourceware.org; run by ezmlm Precedence: bulk List-Subscribe: List-Archive: List-Post: List-Help: , Sender: mauve-discuss-owner@sourceware.org X-SW-Source: 2006-q3/txt/msg00010.txt.bz2 Roman Kennke wrote: > Hi lists, hi David (you wrote most of the tests I'm gonna talk about..) > > While trying to clean up some Mauve failures I came upon a couple of > tests that fail on JDK because they test strictly against the spec > where the JDK isn't as strict. This is mostly bounds checking, where > the spec says throws BadLocationException if invalid or similar. I > think the JDK simply doesn't perform explicit checks in the Content > implementations. See for example the tests for GapContent and > StringContent. > > Now I am not sure how to handle this. I've commented these tests out > locally, simply to avoid clutter in the Mauve output. The question is > how to interpret the spec. Adding the throws BadLocationException does > mean (to me) that the impl may or may not throw a > BadLocationException, but the application should be prepared to deal > with it anyways. Moreover, the throws BadLocationException is > specified in the interface. The implementations are not required to > throw the BadLocationException if they decide to deal with wrong input > themselves. For instance, the GapContent implementation (ours as well > as JDK) can very well handle Position outside the range, because it > only calculates offsets. > > The situation gets worse. There are a number of tests both in Mauve > and in the Intel testsuite that actually test the JDK behaviour of > _not_ throwing the BLE, sometimes indirectly (via a Document impl or > so). So we can't get to fully PASS with Mauve. > We should decide if we want to test strict spec compliance or > reference impl compatibility. So far the decisions in GNU Classpath > have been made in favour of (bug-) compatibility over strict spec > compliance, so I think we should do the same for Mauve. > > Anyway, I think we should either disable the spec-compliance checks or > the RI-compatibility checks or both in Mauve so that we have at least > a chance to reach 100%. > > Any opinions on that? > > /Roman Hi Roman, The theory is easy: Mauve should test AN implementation against THE spec. The reality is...well, you know already what the reality is. Send me the names of the tests that are causing problems, and I'll try to recode them to pass on Sun's JRE 1.5.0, otherwise I'll file a bug report with Sun. I may recode some tests with two paths, so I can set a system property (say) and check for strict spec compliance if I want or need to. I never heard back from Sun about the GapContent bug I filed in January. I filed a few other bug reports since then, with little to show for it: http://www.object-refinery.com/jre/bugs/ ...but with 100,000 bug reports in the first half of 2006, I guess they're busy! Regards, Dave