Re: Updating Mauve tags

[Meskauskas Audrius <audriusa@bluewin.ch>]

The Sun's swing did have as many as 6514 bugs through the history, 
despite many of them are fixed
now. Together with the new features and improvements, each new major 
release inevitably brings
some regressions that are later fixed. J2SE 5.0 
<http://java.sun.com/j2se/1.5.0/download.jsp> already had six updates in 
the past. They do make some
mistakes, same as we do, same as any java development group inevitably 
does. This may be especially
true for tests that were succeeding in the past but fail with the newest 
version.

Because of that reason I would not suggest any automated removal or 
inactivation of the tests just
because they fail on Sun's or any other java implementation. If the 
implemented behavior clearly
mismatches the Sun's API specification, this is probably just a bug that 
is likely to be fixed - probably in the
next minor release.

Some tests can be invalid and can be inactivated or removed (probably 
better altered, making them valid).
However I think that this work (a difficult work) must be done manually. 
The typical reason can be if we find
the similar problem in the Sun's bug reports and the Sun states that 
this is not a bug or it will never be fixed,
or if we realize that the former author of the test clearly 
misinterprets the Sun's API standard.

If needed, we could probably simply have and use the list of tests that 
are known to pass with the final
releases of the JDK 1.2, 1.3 and 1.4 (for the 1.5, such list would need 
the regular updates).

Audrius.

Roman Kennke wrote:

>Hi Mark,
>
>  
>
>>>I see that we have a concept of tags in Mauve. That is a collection of
>>>keys at the top of each test class. This way we can filter the tests.
>>>ATM we have tags for the JDK versions like JDK1.4 JDK1.3 and so on and a
>>>couple of other tags. However, it seems that they are not maintained in
>>>a usable way, so most people simply include every tag that they can
>>>think of (that is what's done in batch_run for example) to run all
>>>tests.
>>>      
>>>
>>Why do you feel they aren't maintained in a usable way?
>>    
>>
>
>This was caused by a misunderstanding of the usage/meaning of those
>tags. I was thinking that when a test has the tag JDK1.x, that this test
>is meant to PASS under a JDK1.x-ish JDK. As Michael and others have
>pointed out on IRC this is not the case. If I want to test a JDK1.3-sh
>(for example) environment I should include JDK1.0 JDK1.1 JDK1.2 and
>JDK1.3 tags in my keys.
>
>The problem that I am seeing is when a test that is written to PASS
>under 1.4 fails under 1.5. There are lots of those tests in the
>testsuite for the javax.swing package. So my plan would have been to tag
>all tests that pass under JDK1.5 with the 1.5 tag and those that don't
>only with JDK1.4 or whatever is ok. Since the tags are not meant to be
>used that way, maybe we can do it different. Could we extend the
>choose-classes script to detect !JDK1.x tags in the tag header of java
>source files and don't include the test in a JDK1.x test run?
>
>/Roman
>  
>