[Q] Number of unit tests in Mauve and of assertions in Classpath?

[Q] Number of unit tests in Mauve and of assertions in Classpath?

[Markus Gälli]

Hi,

if I understand correct, Mauve is a test suite for several Java 
libraries / implementations.

I am curious: Do the java implementers like Classpath use assertions in 
their code or "only" your tests?
Especially I'd like to have some numbers of the latest versions (which 
one would that be?) of Mauve + Classpath:

- How many tests are in Mauve?
- How many run green for Classpath?
- How many assertions (like pre- postconditions/ invariants) are there 
in Classpath?

Thanks a lot.

Cheers,

Markus

Re: [Q] Number of unit tests in Mauve and of assertions in Classpath?

[Stephen Crawley]

Markus,

You wrote:
> if I understand correct, Mauve is a test suite for several Java 
> libraries / implementations.

Correct.

> I am curious: Do the java implementers like Classpath use assertions in 
> their code or "only" your tests?

Neither Classpath or Mauve uses Java assertions!  Both are designed to
be compilable without relying on Java language extensions from JDK 1.4.

> Especially I'd like to have some numbers of the latest versions (which 
> one would that be?) of Mauve + Classpath:
> 
> - How many tests are in Mauve?

You can count the Mauve testcase files yourself.  The testcase files
that are compiled and run in a specific regression depends on the 'keys'
file you use.  Each testcase file typically contains a number of
individual subtests; i.e. calls to a 'harness.check(..)' method.

For the classpath/kissme regression, I'm not sure how many testcase
files get included.  The regression runs over 10000 subtests, and
roughly 145 are currently failing.

It is worth noting that a significant number of subtests fail when
you run them on a Sun JDK.  Sometimes the testcase is at fault, and
sometimes the fault lies in the JDK.

> - How many run green for Classpath?

What do you mean by "green"?

> - How many assertions (like pre- postconditions/ invariants) are there 
> in Classpath?

See above.  Neither Mauve or Classpath use Java assertions.

-- Steve

Re: [Q] Number of unit tests in Mauve and of assertions in Classpath?

[Markus Gälli]

Hi Stephen,

thanks a lot for that very fast and good answers.
>> I am curious: Do the java implementers like Classpath use assertions 
>> in
>> their code or "only" your tests?
> Neither Classpath or Mauve uses Java assertions!  Both are designed to
> be compilable without relying on Java language extensions from JDK 1.4.
Ah, ok. Sure. Petty though. I am just comparing the number of per- 
postconditions used
with the number of unit tests in some big java case studies.
>> Especially I'd like to have some numbers of the latest versions (which
>> one would that be?) of Mauve + Classpath:
(snip)
> For the classpath/kissme regression, I'm not sure how many testcase
> files get included.  The regression runs over 10000 subtests, and
> roughly 145 are currently failing.
I guess this 145 are not independent, meaning, if you fixed one, many 
others would be fixed too. Right?

I am just writing a paper stating that it makes sense to sort failing 
tests by size of covered methods beginning with the smallest.
My motivation for asking here, was that if you had assertions (which I 
surely understand that you don't have), more of the failing
tests would fail at the most specific assertion, making my sorting a 
bit more useless.
But hey, you just gave me a wonderful argument: In some big and 
important projects, assertions are even not an option!
> It is worth noting that a significant number of subtests fail when
> you run them on a Sun JDK.  Sometimes the testcase is at fault, and
> sometimes the fault lies in the JDK.
>> - How many run green for Classpath?
> What do you mean by "green"?
Running without failure. So my next questions:

- Do you think sorting failing unit tests would be helpful?
- Does Mauve sort failing unit tests?
- If not, how could it be done?
- Do you know any (as responsive :-) java open source community with 
some big project and which
uses both, JDK 1.4 (thus theoretically assertions) and unit tests?

Thanks a lot - and cheers

Markus

Re: [Q] Number of unit tests in Mauve and of assertions in Classpath?

[Sascha Brawer]

Markus Gälli <markus.gaelli@iam.unibe.ch> wrote on Thu, 11 Mar 2004 09:
19:30 +0100:

>> roughly 145 [Mauve tests] are currently failing.
>I guess this 145 are not independent, meaning, if you fixed one, many
>others would be fixed too. Right?

In some cases, there exist multiple tests for the same functionality. For
example the tests for solving quadratic and cubic equations, where the
testee is called with a bunch of somewhat random test equations. You'll
have the same situation with any tests that are feeding random input data
into a function and check whether the result is as expected. Any failures
would be sort of mutually dependent: As soon as you fix any of these
failures, the other failures are likely to disappear immediately (because
it's usually the same code that is failing). Stated in more abstract
terms, the "fixing order" is only a partial, not a total ordering on tests.

>I am just writing a paper stating that it makes sense to sort failing
>tests by size of covered methods beginning with the smallest.

Interesting... Why would this be an improvement -- because you assume
that smaller methods get invoked by larger ones, and the smaller methods
would be the specific point of failure? In that case, couldn't you just
sort the failed tests based on a (post-order) traversal of the observed
call graph? Or, this may be easier to obtain, based on the total number
of instructions/bytecodes executed by the testee and its callees? Well, I
guess the reason for sorting by method size will be stated in your paper.
Would you mind posting a link to your paper on this list when you've
published it? Thanks in advance!

>My motivation for asking here, was that if you had assertions (which I 
>surely understand that you don't have), more of the failing
>tests would fail at the most specific assertion, making my sorting a
>bit more useless.

This seems like a reasonable assumption, but I wouldn't know how to
falsify it.

>But hey, you just gave me a wonderful argument: In some big and
>important projects, assertions are even not an option!

Well, I guess we will eventually use assertions in Classpath. There's no
law saying we must not use 1.4 language features. The reason why we
haven't been using assertions in the past is the lack of support by free
Java compilers. Which would be easy enough to fix (a quick-and-dirty fix
would be just to ignore the assert statement).

>> It is worth noting that a significant number of subtests fail when
>> you run them on a Sun JDK.  Sometimes the testcase is at fault, and
>> sometimes the fault lies in the JDK.
>>> - How many run green for Classpath?
>> What do you mean by "green"?
>Running without failure.

Some other test harnesses allow to express which tests are known to fail
in what environment (XFAIL). IMHO, that would be quite useful for Mauve,
especially because of the failures in the Sun reference implementation.
Nobody has done this yet, but anyone is welcome to contribute an
implementation. :-)

>[...]
>- Does Mauve sort failing unit tests?
>- If not, how could it be done?

Mauve simply takes a large text file that lists the names of test
classes, loads the respective class (which must implement the
gnu.testlet.Testlet interface) and invokes its "test" method. So the
"sorting" is just the ordering in the text file. If you sorted this list
in any particular way, the output (= failing tests) would be sorted in
the same order.

>- Do you know any (as responsive :-) java open source community with
>some big project and which
>uses both, JDK 1.4 (thus theoretically assertions) and unit tests?

I guess most JUnit users would fall into that category, but I
unfortunately don't know any specific examples.

Best regards,

-- Sascha

Sascha Brawer, brawer@dandelis.ch, http://www.dandelis.ch/people/brawer/

Re: [Q] Number of unit tests in Mauve and of assertions in Classpath?

[Markus Gälli]

Hi Sascha,

>>> roughly 145 [Mauve tests] are currently failing.
>> I guess this 145 are not independent, meaning, if you fixed one, many
>> others would be fixed too. Right?
>
> In some cases, there exist multiple tests for the same functionality.  
> For
> example the tests for solving quadratic and cubic equations, where the
> testee is called with a bunch of somewhat random test equations. You'll
> have the same situation with any tests that are feeding random input  
> data
> into a function and check whether the result is as expected. Any  
> failures
> would be sort of mutually dependent: As soon as you fix any of these
> failures, the other failures are likely to disappear immediately  
> (because
> it's usually the same code that is failing). Stated in more abstract
> terms, the "fixing order" is only a partial, not a total ordering on  
> tests.
Right. In my experience many tests are on the same level also. There  
you basically don't
care with which you begin, as any exception will do. I think that you  
can get more
out of the ones which form an inclusion hierarchy of coverage sets  
meaning
that one test is concerned with a more abstract view than the other but  
both
running the same inner parts.
>
>> I am just writing a paper stating that it makes sense to sort failing
>> tests by size of covered methods beginning with the smallest.
>
> Interesting... Why would this be an improvement -- because you assume
> that smaller methods get invoked by larger ones, and the smaller  
> methods
> would be the specific point of failure? In that case, couldn't you just
> sort the failed tests based on a (post-order) traversal of the observed
> call graph? Or, this may be easier to obtain, based on the total number
> of instructions/bytecodes executed by the testee and its callees?
That was what I meant. I hope I am not that sloppy in my paper... ;-)
It should have been: "The size of the sets of covered method  
signatures".
> Well, I
> guess the reason for sorting by method size will be stated in your  
> paper.
> Would you mind posting a link to your paper on this list when you've
> published it? Thanks in advance!
You can read an old version here:
http://www.iam.unibe.ch/~scg/Archive/Papers/ 
Gael03bPartialOrderingTestsByCoverageSets.pdf
Will hopefully publish a published version Real Soon Now (TM)
>> My motivation for asking here, was that if you had assertions (which I
>> surely understand that you don't have), more of the failing
>> tests would fail at the most specific assertion, making my sorting a
>> bit more useless.
>
> This seems like a reasonable assumption, but I wouldn't know how to
> falsify it.
>
>> But hey, you just gave me a wonderful argument: In some big and
>> important projects, assertions are even not an option!
>
> Well, I guess we will eventually use assertions in Classpath. There's  
> no
> law saying we must not use 1.4 language features. The reason why we
> haven't been using assertions in the past is the lack of support by  
> free
> Java compilers. Which would be easy enough to fix (a quick-and-dirty  
> fix
> would be just to ignore the assert statement).
Difficult to introduce that to Object? Could be made painless (in  
Smalltalk at least)
See: http://www.smalltalkchronicles.net/edition2-1/st_compiler.htm

(snip)
>
> Mauve simply takes a large text file that lists the names of test
> classes, loads the respective class (which must implement the
> gnu.testlet.Testlet interface) and invokes its "test" method. So the
> "sorting" is just the ordering in the text file. If you sorted this  
> list
> in any particular way, the output (= failing tests) would be sorted in
> the same order.
>
>> - Do you know any (as responsive :-) java open source community with
>> some big project and which
>> uses both, JDK 1.4 (thus theoretically assertions) and unit tests?
>
> I guess most JUnit users would fall into that category, but I
> unfortunately don't know any specific examples.
>
Thanks a lot.

Cheers,

Markus

Re: [Q] Number of unit tests in Mauve and of assertions in Classpath?

[PureNative Software]

Sascha,

Regarding your question:

> >> - Do you know any (as responsive :-) java open source community with
> >> some big project and which
> >> uses both, JDK 1.4 (thus theoretically assertions) and unit tests?

Although not a free (as in GNU) software project, NewJ Library for C++, an
independent 100% native implementation of the core Java API, uses both unit
tests and assertions. The test cases are our C++ version of Mauve, which
make use of the gnu::testlet::Testlet and the rest of the testing framework
just like the Java version. The assertions are actually built right into the
Core J2 Library implementation, which may be compiled in or out for debug or
release builds. As you might expect, the use of both unit tests (like Mauve)
and assertions find more coding or logic errors than either one alone.
Furthermore, unit tests and assertions serve slightly different purposes and
are applicable to slightly different circumstances.

For example, unit tests serve to verify that the implementation of the
library meets the agreed-upon specification. These test cases often validate
method inputs with expected outputs. And test cases serve this purpose
rather well if they are broad enough. Often, such test cases probe boundary
conditions, but it is difficult for them to test all possible inputs,
whether valid or invalid, that may be passed to a method in a real-world
situation. This is where assertions come in. Assertions are always there (if
it has been opted to enable them) so they can validate input and outputs for
any (and every) real-world application. Furthermore, they can test things
that test cases cannot, such as, validating inaccessible members and
performing internal consistency checks. We have found that using both test
cases and assertions gives us a higher level of confidence in the quality of
our library than using one testing technique alone.

Regards,
Vargas
http://www.pure-native.com/newj.html

----- Original Message -----
From: "Markus Gälli" <markus.gaelli@iam.unibe.ch>
To: "Sascha Brawer" <brawer@dandelis.ch>
Cc: "Stephen Crawley" <crawley@dstc.edu.au>;
<mauve-discuss@sources.redhat.com>
Sent: Thursday, March 11, 2004 5:51 AM
Subject: Re: [Q] Number of unit tests in Mauve and of assertions in
Classpath?


> Hi Sascha,
>
> >>> roughly 145 [Mauve tests] are currently failing.
> >> I guess this 145 are not independent, meaning, if you fixed one, many
> >> others would be fixed too. Right?
> >
> > In some cases, there exist multiple tests for the same functionality.
> > For
> > example the tests for solving quadratic and cubic equations, where the
> > testee is called with a bunch of somewhat random test equations. You'll
> > have the same situation with any tests that are feeding random input
> > data
> > into a function and check whether the result is as expected. Any
> > failures
> > would be sort of mutually dependent: As soon as you fix any of these
> > failures, the other failures are likely to disappear immediately
> > (because
> > it's usually the same code that is failing). Stated in more abstract
> > terms, the "fixing order" is only a partial, not a total ordering on
> > tests.
> Right. In my experience many tests are on the same level also. There
> you basically don't
> care with which you begin, as any exception will do. I think that you
> can get more
> out of the ones which form an inclusion hierarchy of coverage sets
> meaning
> that one test is concerned with a more abstract view than the other but
> both
> running the same inner parts.
> >
> >> I am just writing a paper stating that it makes sense to sort failing
> >> tests by size of covered methods beginning with the smallest.
> >
> > Interesting... Why would this be an improvement -- because you assume
> > that smaller methods get invoked by larger ones, and the smaller
> > methods
> > would be the specific point of failure? In that case, couldn't you just
> > sort the failed tests based on a (post-order) traversal of the observed
> > call graph? Or, this may be easier to obtain, based on the total number
> > of instructions/bytecodes executed by the testee and its callees?
> That was what I meant. I hope I am not that sloppy in my paper... ;-)
> It should have been: "The size of the sets of covered method
> signatures".
> > Well, I
> > guess the reason for sorting by method size will be stated in your
> > paper.
> > Would you mind posting a link to your paper on this list when you've
> > published it? Thanks in advance!
> You can read an old version here:
> http://www.iam.unibe.ch/~scg/Archive/Papers/
> Gael03bPartialOrderingTestsByCoverageSets.pdf
> Will hopefully publish a published version Real Soon Now (TM)
> >> My motivation for asking here, was that if you had assertions (which I
> >> surely understand that you don't have), more of the failing
> >> tests would fail at the most specific assertion, making my sorting a
> >> bit more useless.
> >
> > This seems like a reasonable assumption, but I wouldn't know how to
> > falsify it.
> >
> >> But hey, you just gave me a wonderful argument: In some big and
> >> important projects, assertions are even not an option!
> >
> > Well, I guess we will eventually use assertions in Classpath. There's
> > no
> > law saying we must not use 1.4 language features. The reason why we
> > haven't been using assertions in the past is the lack of support by
> > free
> > Java compilers. Which would be easy enough to fix (a quick-and-dirty
> > fix
> > would be just to ignore the assert statement).
> Difficult to introduce that to Object? Could be made painless (in
> Smalltalk at least)
> See: http://www.smalltalkchronicles.net/edition2-1/st_compiler.htm
>
> (snip)
> >
> > Mauve simply takes a large text file that lists the names of test
> > classes, loads the respective class (which must implement the
> > gnu.testlet.Testlet interface) and invokes its "test" method. So the
> > "sorting" is just the ordering in the text file. If you sorted this
> > list
> > in any particular way, the output (= failing tests) would be sorted in
> > the same order.
> >
> >> - Do you know any (as responsive :-) java open source community with
> >> some big project and which
> >> uses both, JDK 1.4 (thus theoretically assertions) and unit tests?
> >
> > I guess most JUnit users would fall into that category, but I
> > unfortunately don't know any specific examples.
> >
> Thanks a lot.
>
> Cheers,
>
> Markus
>

Re: [Q] Number of unit tests in Mauve and of assertions in Classpath?

[Markus Gälli]

Hi Vargas,

> Regarding your question:
>
>>>> - Do you know any (as responsive :-) java open source community with
>>>> some big project and which
>>>> uses both, JDK 1.4 (thus theoretically assertions) and unit tests?
>
> Although not a free (as in GNU) software project, NewJ Library for 
> C++, an
> independent 100% native implementation of the core Java API, uses both 
> unit
> tests and assertions. The test cases are our C++ version of Mauve, 
> which
> make use of the gnu::testlet::Testlet and the rest of the testing 
> framework
> just like the Java version. The assertions are actually built right 
> into the
> Core J2 Library implementation, which may be compiled in or out for 
> debug or
> release builds. As you might expect, the use of both unit tests (like 
> Mauve)
> and assertions find more coding or logic errors than either one alone.
> Furthermore, unit tests and assertions serve slightly different 
> purposes and
> are applicable to slightly different circumstances.
Thanks for that info.

Maybe I am asking for too much here, but I would be very curious to 
know the

- #lines of code(NewJ Library),
- #assertions(NewJ Library),
- #unit tests and
- #lines of code(unit tests)
-#coverage of the unit tests
-#assertions not executed by any unit test
- over several versions?

So if you happen to be able to answer only some of this questions, this 
would be great.

Cheers,

Markus

Re: [Q] Number of unit tests in Mauve and of assertions in Classpath?

[Byron Vargas]

Markus,

Here are some stats for you.

NewJ Library is made up of the Pie Library, which is the foundation layer
for automated object management and approximating Java language features in
C++, and the Core J2 Library, which implements the core Java APIs like lang,
io, util, etc. in C++. Both libraries combined consist of the following:

Total lines: 100757
Comment lines: 22542 (22.372638702% of total)
Source code lines: 53245 (52.844963074% of total)
Blank lines: 24970 (24.782396317% of total)

Number of assertions in Pie Library: 43
Number of assertions in Core J2 Library (java.* packages): 441
Number of assertions in Core J2 Library (private packages): 221

Additionally, the code base includes debug-only sections which take
advantage of conditional compilation inherent in C++.

Number of units tests, coverage, lines: Same as the Java version of Mauve
for the packages and classes we support.

"Assertions not executed by any unit test": We have no quantitative
information on that.

By "over several versions," what should I understand that to mean?
(a) By "over several versions" of NewJ Library, like 0.1, 0.2, 0.3. If so,
that would be very difficult to provide to you.
(b) By "over several versions" of the Java API or specification. If so, NewJ
Library supports 1.2 and some 1.4 features like assertions. Everything is in
the same code base, compile-in or compile-out at build time, so it's all the
same metrics.

For more detailed information, download the trial version from
pure-native.com, read the NewJ Developer's Guide (they're quite detailed and
only around 50 pages), and peruse the included header files and samples.

Hope this helps.

Regards,
Vargas
http://www.pure-native.com/newj.html

----- Original Message -----
From: "Markus Gälli" <markus.gaelli@iam.unibe.ch>
To: "PureNative Software" <info@pure-native.com>
Cc: "Sascha Brawer" <brawer@dandelis.ch>; "Stephen Crawley"
<crawley@dstc.edu.au>; <mauve-discuss@sources.redhat.com>
Sent: Monday, March 15, 2004 8:21 AM
Subject: Re: [Q] Number of unit tests in Mauve and of assertions in
Classpath?


> Hi Vargas,
>
> > Regarding your question:
> >
> >>>> - Do you know any (as responsive :-) java open source community with
> >>>> some big project and which
> >>>> uses both, JDK 1.4 (thus theoretically assertions) and unit tests?
> >
> > Although not a free (as in GNU) software project, NewJ Library for
> > C++, an
> > independent 100% native implementation of the core Java API, uses both
> > unit
> > tests and assertions. The test cases are our C++ version of Mauve,
> > which
> > make use of the gnu::testlet::Testlet and the rest of the testing
> > framework
> > just like the Java version. The assertions are actually built right
> > into the
> > Core J2 Library implementation, which may be compiled in or out for
> > debug or
> > release builds. As you might expect, the use of both unit tests (like
> > Mauve)
> > and assertions find more coding or logic errors than either one alone.
> > Furthermore, unit tests and assertions serve slightly different
> > purposes and
> > are applicable to slightly different circumstances.
> Thanks for that info.
>
> Maybe I am asking for too much here, but I would be very curious to
> know the
>
> - #lines of code(NewJ Library),
> - #assertions(NewJ Library),
> - #unit tests and
> - #lines of code(unit tests)
> -#coverage of the unit tests
> -#assertions not executed by any unit test
> - over several versions?
>
> So if you happen to be able to answer only some of this questions, this
> would be great.
>
> Cheers,
>
> Markus
>

Re: [Q] Number of unit tests in Mauve and of assertions in Classpath?

[Markus Gälli]

Hi Byron,

> Here are some stats for you.

Thanks a lot for that detailed description. This is really of some use 
for me, I just
could not include it in my paper as the submission deadline was one 
hour ago.

I'd be curious, if you could also provide me with a version name of 
NewJLibrary,
where this data holds.

As I was not successful to download Mauve (I tried the ftp version,
but the server seems to be down, then my OS-X stopped without throwing 
any exception...),
I would also like to know how many assertions are called inside the 
Mauve tests,
maybe you or someone else can answer this question too? Then I stop 
bothering you,
promised :-)

Thanks again, this can be of some great value in one of the next papers 
I want to write,
I'll submit the old one as soon as I get it accepted somewhere.

Cheers,

Markus
>
> NewJ Library is made up of the Pie Library, which is the foundation 
> layer
> for automated object management and approximating Java language 
> features in
> C++, and the Core J2 Library, which implements the core Java APIs like 
> lang,
> io, util, etc. in C++. Both libraries combined consist of the 
> following:
>
> Total lines: 100757
> Comment lines: 22542 (22.372638702% of total)
> Source code lines: 53245 (52.844963074% of total)
> Blank lines: 24970 (24.782396317% of total)
>
> Number of assertions in Pie Library: 43
> Number of assertions in Core J2 Library (java.* packages): 441
> Number of assertions in Core J2 Library (private packages): 221
>
> Additionally, the code base includes debug-only sections which take
> advantage of conditional compilation inherent in C++.
>
> Number of units tests, coverage, lines: Same as the Java version of 
> Mauve
> for the packages and classes we support.
>
> "Assertions not executed by any unit test": We have no quantitative
> information on that.
>
> By "over several versions," what should I understand that to mean?
> (a) By "over several versions" of NewJ Library, like 0.1, 0.2, 0.3. If 
> so,
> that would be very difficult to provide to you.
> (b) By "over several versions" of the Java API or specification. If 
> so, NewJ
> Library supports 1.2 and some 1.4 features like assertions. Everything 
> is in
> the same code base, compile-in or compile-out at build time, so it's 
> all the
> same metrics.
>
> For more detailed information, download the trial version from
> pure-native.com, read the NewJ Developer's Guide (they're quite 
> detailed and
> only around 50 pages), and peruse the included header files and 
> samples.
>
> Hope this helps.
>
> Regards,
> Vargas
> http://www.pure-native.com/newj.html
>
> ----- Original Message -----
> From: "Markus Gälli" <markus.gaelli@iam.unibe.ch>
> To: "PureNative Software" <info@pure-native.com>
> Cc: "Sascha Brawer" <brawer@dandelis.ch>; "Stephen Crawley"
> <crawley@dstc.edu.au>; <mauve-discuss@sources.redhat.com>
> Sent: Monday, March 15, 2004 8:21 AM
> Subject: Re: [Q] Number of unit tests in Mauve and of assertions in
> Classpath?
>
>
>> Hi Vargas,
>>
>>> Regarding your question:
>>>
>>>>>> - Do you know any (as responsive :-) java open source community 
>>>>>> with
>>>>>> some big project and which
>>>>>> uses both, JDK 1.4 (thus theoretically assertions) and unit tests?
>>>
>>> Although not a free (as in GNU) software project, NewJ Library for
>>> C++, an
>>> independent 100% native implementation of the core Java API, uses 
>>> both
>>> unit
>>> tests and assertions. The test cases are our C++ version of Mauve,
>>> which
>>> make use of the gnu::testlet::Testlet and the rest of the testing
>>> framework
>>> just like the Java version. The assertions are actually built right
>>> into the
>>> Core J2 Library implementation, which may be compiled in or out for
>>> debug or
>>> release builds. As you might expect, the use of both unit tests (like
>>> Mauve)
>>> and assertions find more coding or logic errors than either one 
>>> alone.
>>> Furthermore, unit tests and assertions serve slightly different
>>> purposes and
>>> are applicable to slightly different circumstances.
>> Thanks for that info.
>>
>> Maybe I am asking for too much here, but I would be very curious to
>> know the
>>
>> - #lines of code(NewJ Library),
>> - #assertions(NewJ Library),
>> - #unit tests and
>> - #lines of code(unit tests)
>> -#coverage of the unit tests
>> -#assertions not executed by any unit test
>> - over several versions?
>>
>> So if you happen to be able to answer only some of this questions, 
>> this
>> would be great.
>>
>> Cheers,
>>
>> Markus
>>
>

Re: [Q] Number of unit tests in Mauve and of assertions in Classpath?

[PureNative Software]

Markus,

Too bad I counldn't get you the data sooner...

The data is from NewJ Library version 0.5 (development snapshot) for the
Win32 platform using Visual C++ 6.0.

I'm not sure how many testing assertions (not to be confused with Java 1.4
assert) are inside Mauve. It would be easy to determine if you grep'd
through the Mauve source code, if you could successfully download it...

Regards,
Vargas

----- Original Message -----
From: "Markus Gälli" <markus.gaelli@iam.unibe.ch>
To: "Byron Vargas" <byron.vargas@worldpost.com>
Cc: "Sascha Brawer" <brawer@dandelis.ch>; "Stephen Crawley"
<crawley@dstc.edu.au>; <mauve-discuss@sources.redhat.com>
Sent: Tuesday, March 16, 2004 7:55 AM
Subject: Re: [Q] Number of unit tests in Mauve and of assertions in
Classpath?


Hi Byron,

> Here are some stats for you.

Thanks a lot for that detailed description. This is really of some use
for me, I just
could not include it in my paper as the submission deadline was one
hour ago.

I'd be curious, if you could also provide me with a version name of
NewJLibrary,
where this data holds.

As I was not successful to download Mauve (I tried the ftp version,
but the server seems to be down, then my OS-X stopped without throwing
any exception...),
I would also like to know how many assertions are called inside the
Mauve tests,
maybe you or someone else can answer this question too? Then I stop
bothering you,
promised :-)

Thanks again, this can be of some great value in one of the next papers
I want to write,
I'll submit the old one as soon as I get it accepted somewhere.

Cheers,

Markus
>
> NewJ Library is made up of the Pie Library, which is the foundation
> layer
> for automated object management and approximating Java language
> features in
> C++, and the Core J2 Library, which implements the core Java APIs like
> lang,
> io, util, etc. in C++. Both libraries combined consist of the
> following:
>
> Total lines: 100757
> Comment lines: 22542 (22.372638702% of total)
> Source code lines: 53245 (52.844963074% of total)
> Blank lines: 24970 (24.782396317% of total)
>
> Number of assertions in Pie Library: 43
> Number of assertions in Core J2 Library (java.* packages): 441
> Number of assertions in Core J2 Library (private packages): 221
>
> Additionally, the code base includes debug-only sections which take
> advantage of conditional compilation inherent in C++.
>
> Number of units tests, coverage, lines: Same as the Java version of
> Mauve
> for the packages and classes we support.
>
> "Assertions not executed by any unit test": We have no quantitative
> information on that.
>
> By "over several versions," what should I understand that to mean?
> (a) By "over several versions" of NewJ Library, like 0.1, 0.2, 0.3. If
> so,
> that would be very difficult to provide to you.
> (b) By "over several versions" of the Java API or specification. If
> so, NewJ
> Library supports 1.2 and some 1.4 features like assertions. Everything
> is in
> the same code base, compile-in or compile-out at build time, so it's
> all the
> same metrics.
>
> For more detailed information, download the trial version from
> pure-native.com, read the NewJ Developer's Guide (they're quite
> detailed and
> only around 50 pages), and peruse the included header files and
> samples.
>
> Hope this helps.
>
> Regards,
> Vargas
> http://www.pure-native.com/newj.html
>
> ----- Original Message -----
> From: "Markus Gälli" <markus.gaelli@iam.unibe.ch>
> To: "PureNative Software" <info@pure-native.com>
> Cc: "Sascha Brawer" <brawer@dandelis.ch>; "Stephen Crawley"
> <crawley@dstc.edu.au>; <mauve-discuss@sources.redhat.com>
> Sent: Monday, March 15, 2004 8:21 AM
> Subject: Re: [Q] Number of unit tests in Mauve and of assertions in
> Classpath?
>
>
>> Hi Vargas,
>>
>>> Regarding your question:
>>>
>>>>>> - Do you know any (as responsive :-) java open source community
>>>>>> with
>>>>>> some big project and which
>>>>>> uses both, JDK 1.4 (thus theoretically assertions) and unit tests?
>>>
>>> Although not a free (as in GNU) software project, NewJ Library for
>>> C++, an
>>> independent 100% native implementation of the core Java API, uses
>>> both
>>> unit
>>> tests and assertions. The test cases are our C++ version of Mauve,
>>> which
>>> make use of the gnu::testlet::Testlet and the rest of the testing
>>> framework
>>> just like the Java version. The assertions are actually built right
>>> into the
>>> Core J2 Library implementation, which may be compiled in or out for
>>> debug or
>>> release builds. As you might expect, the use of both unit tests (like
>>> Mauve)
>>> and assertions find more coding or logic errors than either one
>>> alone.
>>> Furthermore, unit tests and assertions serve slightly different
>>> purposes and
>>> are applicable to slightly different circumstances.
>> Thanks for that info.
>>
>> Maybe I am asking for too much here, but I would be very curious to
>> know the
>>
>> - #lines of code(NewJ Library),
>> - #assertions(NewJ Library),
>> - #unit tests and
>> - #lines of code(unit tests)
>> -#coverage of the unit tests
>> -#assertions not executed by any unit test
>> - over several versions?
>>
>> So if you happen to be able to answer only some of this questions,
>> this
>> would be great.
>>
>> Cheers,
>>
>> Markus
>>
>

Re: [Q] Number of unit tests in Mauve and of assertions in Classpath?

[PureNative Software]

Markus,

Here are some stats for you.

NewJ Library is made up of the Pie Library, which is the foundation layer
for automated object management and approximating Java language features in
C++, and the Core J2 Library, which implements the core Java APIs like lang,
io, util, etc. in C++. Both libraries combined consist of the following:

Total lines: 100757
Comment lines: 22542 (22.372638702% of total)
Source code lines: 53245 (52.844963074% of total)
Blank lines: 24970 (24.782396317% of total)

Number of assertions in Pie Library: 43
Number of assertions in Core J2 Library (java.* packages): 441
Number of assertions in Core J2 Library (private packages): 221

Additionally, the code base includes debug-only sections which take
advantage of conditional compilation inherent in C++.

Number of units tests, coverage, lines: Same as the Java version of Mauve
for the packages and classes we support.

"Assertions not executed by any unit test": We have no quantitative
information on that.

By "over several versions," what should I understand that to mean?
(a) By "over several versions" of NewJ Library, like 0.1, 0.2, 0.3. If so,
that would be very difficult to provide to you.
(b) By "over several versions" of the Java API or specification. If so, NewJ
Library supports 1.2 and some 1.4 features like assertions. Everything is in
the same code base, compile-in or compile-out at build time, so it's all the
same metrics.

For more detailed information, download the trial version from
pure-native.com, read the NewJ Developer's Guide (they're quite detailed and
only around 50 pages), and peruse the included header files and samples.

Hope this helps.

Regards,
Vargas
http://www.pure-native.com/newj.html

----- Original Message -----
From: "Markus Gälli" <markus.gaelli@iam.unibe.ch>
To: "PureNative Software" <info@pure-native.com>
Cc: "Sascha Brawer" <brawer@dandelis.ch>; "Stephen Crawley"
<crawley@dstc.edu.au>; <mauve-discuss@sources.redhat.com>
Sent: Monday, March 15, 2004 8:21 AM
Subject: Re: [Q] Number of unit tests in Mauve and of assertions in
Classpath?


> Hi Vargas,
>
> > Regarding your question:
> >
> >>>> - Do you know any (as responsive :-) java open source community with
> >>>> some big project and which
> >>>> uses both, JDK 1.4 (thus theoretically assertions) and unit tests?
> >
> > Although not a free (as in GNU) software project, NewJ Library for
> > C++, an
> > independent 100% native implementation of the core Java API, uses both
> > unit
> > tests and assertions. The test cases are our C++ version of Mauve,
> > which
> > make use of the gnu::testlet::Testlet and the rest of the testing
> > framework
> > just like the Java version. The assertions are actually built right
> > into the
> > Core J2 Library implementation, which may be compiled in or out for
> > debug or
> > release builds. As you might expect, the use of both unit tests (like
> > Mauve)
> > and assertions find more coding or logic errors than either one alone.
> > Furthermore, unit tests and assertions serve slightly different
> > purposes and
> > are applicable to slightly different circumstances.
> Thanks for that info.
>
> Maybe I am asking for too much here, but I would be very curious to
> know the
>
> - #lines of code(NewJ Library),
> - #assertions(NewJ Library),
> - #unit tests and
> - #lines of code(unit tests)
> -#coverage of the unit tests
> -#assertions not executed by any unit test
> - over several versions?
>
> So if you happen to be able to answer only some of this questions, this
> would be great.
>
> Cheers,
>
> Markus
>