* [Q] Number of unit tests in Mauve and of assertions in Classpath? @ 2004-03-11 6:28 Markus Gälli 2004-03-11 6:49 ` Stephen Crawley 0 siblings, 1 reply; 11+ messages in thread From: Markus Gälli @ 2004-03-11 6:28 UTC (permalink / raw) To: mauve-discuss Hi, if I understand correct, Mauve is a test suite for several Java libraries / implementations. I am curious: Do the java implementers like Classpath use assertions in their code or "only" your tests? Especially I'd like to have some numbers of the latest versions (which one would that be?) of Mauve + Classpath: - How many tests are in Mauve? - How many run green for Classpath? - How many assertions (like pre- postconditions/ invariants) are there in Classpath? Thanks a lot. Cheers, Markus ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [Q] Number of unit tests in Mauve and of assertions in Classpath? 2004-03-11 6:28 [Q] Number of unit tests in Mauve and of assertions in Classpath? Markus Gälli @ 2004-03-11 6:49 ` Stephen Crawley 2004-03-11 8:19 ` Markus Gälli 0 siblings, 1 reply; 11+ messages in thread From: Stephen Crawley @ 2004-03-11 6:49 UTC (permalink / raw) To: Markus Gälli; +Cc: mauve-discuss, crawley Markus, You wrote: > if I understand correct, Mauve is a test suite for several Java > libraries / implementations. Correct. > I am curious: Do the java implementers like Classpath use assertions in > their code or "only" your tests? Neither Classpath or Mauve uses Java assertions! Both are designed to be compilable without relying on Java language extensions from JDK 1.4. > Especially I'd like to have some numbers of the latest versions (which > one would that be?) of Mauve + Classpath: > > - How many tests are in Mauve? You can count the Mauve testcase files yourself. The testcase files that are compiled and run in a specific regression depends on the 'keys' file you use. Each testcase file typically contains a number of individual subtests; i.e. calls to a 'harness.check(..)' method. For the classpath/kissme regression, I'm not sure how many testcase files get included. The regression runs over 10000 subtests, and roughly 145 are currently failing. It is worth noting that a significant number of subtests fail when you run them on a Sun JDK. Sometimes the testcase is at fault, and sometimes the fault lies in the JDK. > - How many run green for Classpath? What do you mean by "green"? > - How many assertions (like pre- postconditions/ invariants) are there > in Classpath? See above. Neither Mauve or Classpath use Java assertions. -- Steve ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [Q] Number of unit tests in Mauve and of assertions in Classpath? 2004-03-11 6:49 ` Stephen Crawley @ 2004-03-11 8:19 ` Markus Gälli 2004-03-11 9:34 ` Sascha Brawer 0 siblings, 1 reply; 11+ messages in thread From: Markus Gälli @ 2004-03-11 8:19 UTC (permalink / raw) To: Stephen Crawley; +Cc: mauve-discuss Hi Stephen, thanks a lot for that very fast and good answers. >> I am curious: Do the java implementers like Classpath use assertions >> in >> their code or "only" your tests? > Neither Classpath or Mauve uses Java assertions! Both are designed to > be compilable without relying on Java language extensions from JDK 1.4. Ah, ok. Sure. Petty though. I am just comparing the number of per- postconditions used with the number of unit tests in some big java case studies. >> Especially I'd like to have some numbers of the latest versions (which >> one would that be?) of Mauve + Classpath: (snip) > For the classpath/kissme regression, I'm not sure how many testcase > files get included. The regression runs over 10000 subtests, and > roughly 145 are currently failing. I guess this 145 are not independent, meaning, if you fixed one, many others would be fixed too. Right? I am just writing a paper stating that it makes sense to sort failing tests by size of covered methods beginning with the smallest. My motivation for asking here, was that if you had assertions (which I surely understand that you don't have), more of the failing tests would fail at the most specific assertion, making my sorting a bit more useless. But hey, you just gave me a wonderful argument: In some big and important projects, assertions are even not an option! > It is worth noting that a significant number of subtests fail when > you run them on a Sun JDK. Sometimes the testcase is at fault, and > sometimes the fault lies in the JDK. >> - How many run green for Classpath? > What do you mean by "green"? Running without failure. So my next questions: - Do you think sorting failing unit tests would be helpful? - Does Mauve sort failing unit tests? - If not, how could it be done? - Do you know any (as responsive :-) java open source community with some big project and which uses both, JDK 1.4 (thus theoretically assertions) and unit tests? Thanks a lot - and cheers Markus ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [Q] Number of unit tests in Mauve and of assertions in Classpath? 2004-03-11 8:19 ` Markus Gälli @ 2004-03-11 9:34 ` Sascha Brawer 2004-03-11 10:51 ` Markus Gälli 0 siblings, 1 reply; 11+ messages in thread From: Sascha Brawer @ 2004-03-11 9:34 UTC (permalink / raw) To: Stephen Crawley, Markus Gälli; +Cc: mauve-discuss Markus Gälli <markus.gaelli@iam.unibe.ch> wrote on Thu, 11 Mar 2004 09: 19:30 +0100: >> roughly 145 [Mauve tests] are currently failing. >I guess this 145 are not independent, meaning, if you fixed one, many >others would be fixed too. Right? In some cases, there exist multiple tests for the same functionality. For example the tests for solving quadratic and cubic equations, where the testee is called with a bunch of somewhat random test equations. You'll have the same situation with any tests that are feeding random input data into a function and check whether the result is as expected. Any failures would be sort of mutually dependent: As soon as you fix any of these failures, the other failures are likely to disappear immediately (because it's usually the same code that is failing). Stated in more abstract terms, the "fixing order" is only a partial, not a total ordering on tests. >I am just writing a paper stating that it makes sense to sort failing >tests by size of covered methods beginning with the smallest. Interesting... Why would this be an improvement -- because you assume that smaller methods get invoked by larger ones, and the smaller methods would be the specific point of failure? In that case, couldn't you just sort the failed tests based on a (post-order) traversal of the observed call graph? Or, this may be easier to obtain, based on the total number of instructions/bytecodes executed by the testee and its callees? Well, I guess the reason for sorting by method size will be stated in your paper. Would you mind posting a link to your paper on this list when you've published it? Thanks in advance! >My motivation for asking here, was that if you had assertions (which I >surely understand that you don't have), more of the failing >tests would fail at the most specific assertion, making my sorting a >bit more useless. This seems like a reasonable assumption, but I wouldn't know how to falsify it. >But hey, you just gave me a wonderful argument: In some big and >important projects, assertions are even not an option! Well, I guess we will eventually use assertions in Classpath. There's no law saying we must not use 1.4 language features. The reason why we haven't been using assertions in the past is the lack of support by free Java compilers. Which would be easy enough to fix (a quick-and-dirty fix would be just to ignore the assert statement). >> It is worth noting that a significant number of subtests fail when >> you run them on a Sun JDK. Sometimes the testcase is at fault, and >> sometimes the fault lies in the JDK. >>> - How many run green for Classpath? >> What do you mean by "green"? >Running without failure. Some other test harnesses allow to express which tests are known to fail in what environment (XFAIL). IMHO, that would be quite useful for Mauve, especially because of the failures in the Sun reference implementation. Nobody has done this yet, but anyone is welcome to contribute an implementation. :-) >[...] >- Does Mauve sort failing unit tests? >- If not, how could it be done? Mauve simply takes a large text file that lists the names of test classes, loads the respective class (which must implement the gnu.testlet.Testlet interface) and invokes its "test" method. So the "sorting" is just the ordering in the text file. If you sorted this list in any particular way, the output (= failing tests) would be sorted in the same order. >- Do you know any (as responsive :-) java open source community with >some big project and which >uses both, JDK 1.4 (thus theoretically assertions) and unit tests? I guess most JUnit users would fall into that category, but I unfortunately don't know any specific examples. Best regards, -- Sascha Sascha Brawer, brawer@dandelis.ch, http://www.dandelis.ch/people/brawer/ ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [Q] Number of unit tests in Mauve and of assertions in Classpath? 2004-03-11 9:34 ` Sascha Brawer @ 2004-03-11 10:51 ` Markus Gälli 2004-03-15 12:46 ` PureNative Software 0 siblings, 1 reply; 11+ messages in thread From: Markus Gälli @ 2004-03-11 10:51 UTC (permalink / raw) To: Sascha Brawer; +Cc: Stephen Crawley, mauve-discuss Hi Sascha, >>> roughly 145 [Mauve tests] are currently failing. >> I guess this 145 are not independent, meaning, if you fixed one, many >> others would be fixed too. Right? > > In some cases, there exist multiple tests for the same functionality. > For > example the tests for solving quadratic and cubic equations, where the > testee is called with a bunch of somewhat random test equations. You'll > have the same situation with any tests that are feeding random input > data > into a function and check whether the result is as expected. Any > failures > would be sort of mutually dependent: As soon as you fix any of these > failures, the other failures are likely to disappear immediately > (because > it's usually the same code that is failing). Stated in more abstract > terms, the "fixing order" is only a partial, not a total ordering on > tests. Right. In my experience many tests are on the same level also. There you basically don't care with which you begin, as any exception will do. I think that you can get more out of the ones which form an inclusion hierarchy of coverage sets meaning that one test is concerned with a more abstract view than the other but both running the same inner parts. > >> I am just writing a paper stating that it makes sense to sort failing >> tests by size of covered methods beginning with the smallest. > > Interesting... Why would this be an improvement -- because you assume > that smaller methods get invoked by larger ones, and the smaller > methods > would be the specific point of failure? In that case, couldn't you just > sort the failed tests based on a (post-order) traversal of the observed > call graph? Or, this may be easier to obtain, based on the total number > of instructions/bytecodes executed by the testee and its callees? That was what I meant. I hope I am not that sloppy in my paper... ;-) It should have been: "The size of the sets of covered method signatures". > Well, I > guess the reason for sorting by method size will be stated in your > paper. > Would you mind posting a link to your paper on this list when you've > published it? Thanks in advance! You can read an old version here: http://www.iam.unibe.ch/~scg/Archive/Papers/ Gael03bPartialOrderingTestsByCoverageSets.pdf Will hopefully publish a published version Real Soon Now (TM) >> My motivation for asking here, was that if you had assertions (which I >> surely understand that you don't have), more of the failing >> tests would fail at the most specific assertion, making my sorting a >> bit more useless. > > This seems like a reasonable assumption, but I wouldn't know how to > falsify it. > >> But hey, you just gave me a wonderful argument: In some big and >> important projects, assertions are even not an option! > > Well, I guess we will eventually use assertions in Classpath. There's > no > law saying we must not use 1.4 language features. The reason why we > haven't been using assertions in the past is the lack of support by > free > Java compilers. Which would be easy enough to fix (a quick-and-dirty > fix > would be just to ignore the assert statement). Difficult to introduce that to Object? Could be made painless (in Smalltalk at least) See: http://www.smalltalkchronicles.net/edition2-1/st_compiler.htm (snip) > > Mauve simply takes a large text file that lists the names of test > classes, loads the respective class (which must implement the > gnu.testlet.Testlet interface) and invokes its "test" method. So the > "sorting" is just the ordering in the text file. If you sorted this > list > in any particular way, the output (= failing tests) would be sorted in > the same order. > >> - Do you know any (as responsive :-) java open source community with >> some big project and which >> uses both, JDK 1.4 (thus theoretically assertions) and unit tests? > > I guess most JUnit users would fall into that category, but I > unfortunately don't know any specific examples. > Thanks a lot. Cheers, Markus ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [Q] Number of unit tests in Mauve and of assertions in Classpath? 2004-03-11 10:51 ` Markus Gälli @ 2004-03-15 12:46 ` PureNative Software 2004-03-15 13:21 ` Markus Gälli 0 siblings, 1 reply; 11+ messages in thread From: PureNative Software @ 2004-03-15 12:46 UTC (permalink / raw) To: Sascha Brawer, Markus Gälli; +Cc: Stephen Crawley, mauve-discuss Sascha, Regarding your question: > >> - Do you know any (as responsive :-) java open source community with > >> some big project and which > >> uses both, JDK 1.4 (thus theoretically assertions) and unit tests? Although not a free (as in GNU) software project, NewJ Library for C++, an independent 100% native implementation of the core Java API, uses both unit tests and assertions. The test cases are our C++ version of Mauve, which make use of the gnu::testlet::Testlet and the rest of the testing framework just like the Java version. The assertions are actually built right into the Core J2 Library implementation, which may be compiled in or out for debug or release builds. As you might expect, the use of both unit tests (like Mauve) and assertions find more coding or logic errors than either one alone. Furthermore, unit tests and assertions serve slightly different purposes and are applicable to slightly different circumstances. For example, unit tests serve to verify that the implementation of the library meets the agreed-upon specification. These test cases often validate method inputs with expected outputs. And test cases serve this purpose rather well if they are broad enough. Often, such test cases probe boundary conditions, but it is difficult for them to test all possible inputs, whether valid or invalid, that may be passed to a method in a real-world situation. This is where assertions come in. Assertions are always there (if it has been opted to enable them) so they can validate input and outputs for any (and every) real-world application. Furthermore, they can test things that test cases cannot, such as, validating inaccessible members and performing internal consistency checks. We have found that using both test cases and assertions gives us a higher level of confidence in the quality of our library than using one testing technique alone. Regards, Vargas http://www.pure-native.com/newj.html ----- Original Message ----- From: "Markus Gälli" <markus.gaelli@iam.unibe.ch> To: "Sascha Brawer" <brawer@dandelis.ch> Cc: "Stephen Crawley" <crawley@dstc.edu.au>; <mauve-discuss@sources.redhat.com> Sent: Thursday, March 11, 2004 5:51 AM Subject: Re: [Q] Number of unit tests in Mauve and of assertions in Classpath? > Hi Sascha, > > >>> roughly 145 [Mauve tests] are currently failing. > >> I guess this 145 are not independent, meaning, if you fixed one, many > >> others would be fixed too. Right? > > > > In some cases, there exist multiple tests for the same functionality. > > For > > example the tests for solving quadratic and cubic equations, where the > > testee is called with a bunch of somewhat random test equations. You'll > > have the same situation with any tests that are feeding random input > > data > > into a function and check whether the result is as expected. Any > > failures > > would be sort of mutually dependent: As soon as you fix any of these > > failures, the other failures are likely to disappear immediately > > (because > > it's usually the same code that is failing). Stated in more abstract > > terms, the "fixing order" is only a partial, not a total ordering on > > tests. > Right. In my experience many tests are on the same level also. There > you basically don't > care with which you begin, as any exception will do. I think that you > can get more > out of the ones which form an inclusion hierarchy of coverage sets > meaning > that one test is concerned with a more abstract view than the other but > both > running the same inner parts. > > > >> I am just writing a paper stating that it makes sense to sort failing > >> tests by size of covered methods beginning with the smallest. > > > > Interesting... Why would this be an improvement -- because you assume > > that smaller methods get invoked by larger ones, and the smaller > > methods > > would be the specific point of failure? In that case, couldn't you just > > sort the failed tests based on a (post-order) traversal of the observed > > call graph? Or, this may be easier to obtain, based on the total number > > of instructions/bytecodes executed by the testee and its callees? > That was what I meant. I hope I am not that sloppy in my paper... ;-) > It should have been: "The size of the sets of covered method > signatures". > > Well, I > > guess the reason for sorting by method size will be stated in your > > paper. > > Would you mind posting a link to your paper on this list when you've > > published it? Thanks in advance! > You can read an old version here: > http://www.iam.unibe.ch/~scg/Archive/Papers/ > Gael03bPartialOrderingTestsByCoverageSets.pdf > Will hopefully publish a published version Real Soon Now (TM) > >> My motivation for asking here, was that if you had assertions (which I > >> surely understand that you don't have), more of the failing > >> tests would fail at the most specific assertion, making my sorting a > >> bit more useless. > > > > This seems like a reasonable assumption, but I wouldn't know how to > > falsify it. > > > >> But hey, you just gave me a wonderful argument: In some big and > >> important projects, assertions are even not an option! > > > > Well, I guess we will eventually use assertions in Classpath. There's > > no > > law saying we must not use 1.4 language features. The reason why we > > haven't been using assertions in the past is the lack of support by > > free > > Java compilers. Which would be easy enough to fix (a quick-and-dirty > > fix > > would be just to ignore the assert statement). > Difficult to introduce that to Object? Could be made painless (in > Smalltalk at least) > See: http://www.smalltalkchronicles.net/edition2-1/st_compiler.htm > > (snip) > > > > Mauve simply takes a large text file that lists the names of test > > classes, loads the respective class (which must implement the > > gnu.testlet.Testlet interface) and invokes its "test" method. So the > > "sorting" is just the ordering in the text file. If you sorted this > > list > > in any particular way, the output (= failing tests) would be sorted in > > the same order. > > > >> - Do you know any (as responsive :-) java open source community with > >> some big project and which > >> uses both, JDK 1.4 (thus theoretically assertions) and unit tests? > > > > I guess most JUnit users would fall into that category, but I > > unfortunately don't know any specific examples. > > > Thanks a lot. > > Cheers, > > Markus > ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [Q] Number of unit tests in Mauve and of assertions in Classpath? 2004-03-15 12:46 ` PureNative Software @ 2004-03-15 13:21 ` Markus Gälli 2004-03-16 3:04 ` PureNative Software 2004-03-16 3:04 ` Byron Vargas 0 siblings, 2 replies; 11+ messages in thread From: Markus Gälli @ 2004-03-15 13:21 UTC (permalink / raw) To: PureNative Software; +Cc: Sascha Brawer, Stephen Crawley, mauve-discuss Hi Vargas, > Regarding your question: > >>>> - Do you know any (as responsive :-) java open source community with >>>> some big project and which >>>> uses both, JDK 1.4 (thus theoretically assertions) and unit tests? > > Although not a free (as in GNU) software project, NewJ Library for > C++, an > independent 100% native implementation of the core Java API, uses both > unit > tests and assertions. The test cases are our C++ version of Mauve, > which > make use of the gnu::testlet::Testlet and the rest of the testing > framework > just like the Java version. The assertions are actually built right > into the > Core J2 Library implementation, which may be compiled in or out for > debug or > release builds. As you might expect, the use of both unit tests (like > Mauve) > and assertions find more coding or logic errors than either one alone. > Furthermore, unit tests and assertions serve slightly different > purposes and > are applicable to slightly different circumstances. Thanks for that info. Maybe I am asking for too much here, but I would be very curious to know the - #lines of code(NewJ Library), - #assertions(NewJ Library), - #unit tests and - #lines of code(unit tests) -#coverage of the unit tests -#assertions not executed by any unit test - over several versions? So if you happen to be able to answer only some of this questions, this would be great. Cheers, Markus ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [Q] Number of unit tests in Mauve and of assertions in Classpath? 2004-03-15 13:21 ` Markus Gälli @ 2004-03-16 3:04 ` PureNative Software 2004-03-16 3:04 ` Byron Vargas 1 sibling, 0 replies; 11+ messages in thread From: PureNative Software @ 2004-03-16 3:04 UTC (permalink / raw) To: Markus Gälli; +Cc: Sascha Brawer, Stephen Crawley, mauve-discuss Markus, Here are some stats for you. NewJ Library is made up of the Pie Library, which is the foundation layer for automated object management and approximating Java language features in C++, and the Core J2 Library, which implements the core Java APIs like lang, io, util, etc. in C++. Both libraries combined consist of the following: Total lines: 100757 Comment lines: 22542 (22.372638702% of total) Source code lines: 53245 (52.844963074% of total) Blank lines: 24970 (24.782396317% of total) Number of assertions in Pie Library: 43 Number of assertions in Core J2 Library (java.* packages): 441 Number of assertions in Core J2 Library (private packages): 221 Additionally, the code base includes debug-only sections which take advantage of conditional compilation inherent in C++. Number of units tests, coverage, lines: Same as the Java version of Mauve for the packages and classes we support. "Assertions not executed by any unit test": We have no quantitative information on that. By "over several versions," what should I understand that to mean? (a) By "over several versions" of NewJ Library, like 0.1, 0.2, 0.3. If so, that would be very difficult to provide to you. (b) By "over several versions" of the Java API or specification. If so, NewJ Library supports 1.2 and some 1.4 features like assertions. Everything is in the same code base, compile-in or compile-out at build time, so it's all the same metrics. For more detailed information, download the trial version from pure-native.com, read the NewJ Developer's Guide (they're quite detailed and only around 50 pages), and peruse the included header files and samples. Hope this helps. Regards, Vargas http://www.pure-native.com/newj.html ----- Original Message ----- From: "Markus Gälli" <markus.gaelli@iam.unibe.ch> To: "PureNative Software" <info@pure-native.com> Cc: "Sascha Brawer" <brawer@dandelis.ch>; "Stephen Crawley" <crawley@dstc.edu.au>; <mauve-discuss@sources.redhat.com> Sent: Monday, March 15, 2004 8:21 AM Subject: Re: [Q] Number of unit tests in Mauve and of assertions in Classpath? > Hi Vargas, > > > Regarding your question: > > > >>>> - Do you know any (as responsive :-) java open source community with > >>>> some big project and which > >>>> uses both, JDK 1.4 (thus theoretically assertions) and unit tests? > > > > Although not a free (as in GNU) software project, NewJ Library for > > C++, an > > independent 100% native implementation of the core Java API, uses both > > unit > > tests and assertions. The test cases are our C++ version of Mauve, > > which > > make use of the gnu::testlet::Testlet and the rest of the testing > > framework > > just like the Java version. The assertions are actually built right > > into the > > Core J2 Library implementation, which may be compiled in or out for > > debug or > > release builds. As you might expect, the use of both unit tests (like > > Mauve) > > and assertions find more coding or logic errors than either one alone. > > Furthermore, unit tests and assertions serve slightly different > > purposes and > > are applicable to slightly different circumstances. > Thanks for that info. > > Maybe I am asking for too much here, but I would be very curious to > know the > > - #lines of code(NewJ Library), > - #assertions(NewJ Library), > - #unit tests and > - #lines of code(unit tests) > -#coverage of the unit tests > -#assertions not executed by any unit test > - over several versions? > > So if you happen to be able to answer only some of this questions, this > would be great. > > Cheers, > > Markus > ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [Q] Number of unit tests in Mauve and of assertions in Classpath? 2004-03-15 13:21 ` Markus Gälli 2004-03-16 3:04 ` PureNative Software @ 2004-03-16 3:04 ` Byron Vargas 2004-03-16 12:55 ` Markus Gälli 1 sibling, 1 reply; 11+ messages in thread From: Byron Vargas @ 2004-03-16 3:04 UTC (permalink / raw) To: Markus Gälli; +Cc: Sascha Brawer, Stephen Crawley, mauve-discuss Markus, Here are some stats for you. NewJ Library is made up of the Pie Library, which is the foundation layer for automated object management and approximating Java language features in C++, and the Core J2 Library, which implements the core Java APIs like lang, io, util, etc. in C++. Both libraries combined consist of the following: Total lines: 100757 Comment lines: 22542 (22.372638702% of total) Source code lines: 53245 (52.844963074% of total) Blank lines: 24970 (24.782396317% of total) Number of assertions in Pie Library: 43 Number of assertions in Core J2 Library (java.* packages): 441 Number of assertions in Core J2 Library (private packages): 221 Additionally, the code base includes debug-only sections which take advantage of conditional compilation inherent in C++. Number of units tests, coverage, lines: Same as the Java version of Mauve for the packages and classes we support. "Assertions not executed by any unit test": We have no quantitative information on that. By "over several versions," what should I understand that to mean? (a) By "over several versions" of NewJ Library, like 0.1, 0.2, 0.3. If so, that would be very difficult to provide to you. (b) By "over several versions" of the Java API or specification. If so, NewJ Library supports 1.2 and some 1.4 features like assertions. Everything is in the same code base, compile-in or compile-out at build time, so it's all the same metrics. For more detailed information, download the trial version from pure-native.com, read the NewJ Developer's Guide (they're quite detailed and only around 50 pages), and peruse the included header files and samples. Hope this helps. Regards, Vargas http://www.pure-native.com/newj.html ----- Original Message ----- From: "Markus Gälli" <markus.gaelli@iam.unibe.ch> To: "PureNative Software" <info@pure-native.com> Cc: "Sascha Brawer" <brawer@dandelis.ch>; "Stephen Crawley" <crawley@dstc.edu.au>; <mauve-discuss@sources.redhat.com> Sent: Monday, March 15, 2004 8:21 AM Subject: Re: [Q] Number of unit tests in Mauve and of assertions in Classpath? > Hi Vargas, > > > Regarding your question: > > > >>>> - Do you know any (as responsive :-) java open source community with > >>>> some big project and which > >>>> uses both, JDK 1.4 (thus theoretically assertions) and unit tests? > > > > Although not a free (as in GNU) software project, NewJ Library for > > C++, an > > independent 100% native implementation of the core Java API, uses both > > unit > > tests and assertions. The test cases are our C++ version of Mauve, > > which > > make use of the gnu::testlet::Testlet and the rest of the testing > > framework > > just like the Java version. The assertions are actually built right > > into the > > Core J2 Library implementation, which may be compiled in or out for > > debug or > > release builds. As you might expect, the use of both unit tests (like > > Mauve) > > and assertions find more coding or logic errors than either one alone. > > Furthermore, unit tests and assertions serve slightly different > > purposes and > > are applicable to slightly different circumstances. > Thanks for that info. > > Maybe I am asking for too much here, but I would be very curious to > know the > > - #lines of code(NewJ Library), > - #assertions(NewJ Library), > - #unit tests and > - #lines of code(unit tests) > -#coverage of the unit tests > -#assertions not executed by any unit test > - over several versions? > > So if you happen to be able to answer only some of this questions, this > would be great. > > Cheers, > > Markus > ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [Q] Number of unit tests in Mauve and of assertions in Classpath? 2004-03-16 3:04 ` Byron Vargas @ 2004-03-16 12:55 ` Markus Gälli 2004-03-16 17:15 ` PureNative Software 0 siblings, 1 reply; 11+ messages in thread From: Markus Gälli @ 2004-03-16 12:55 UTC (permalink / raw) To: Byron Vargas; +Cc: Sascha Brawer, Stephen Crawley, mauve-discuss Hi Byron, > Here are some stats for you. Thanks a lot for that detailed description. This is really of some use for me, I just could not include it in my paper as the submission deadline was one hour ago. I'd be curious, if you could also provide me with a version name of NewJLibrary, where this data holds. As I was not successful to download Mauve (I tried the ftp version, but the server seems to be down, then my OS-X stopped without throwing any exception...), I would also like to know how many assertions are called inside the Mauve tests, maybe you or someone else can answer this question too? Then I stop bothering you, promised :-) Thanks again, this can be of some great value in one of the next papers I want to write, I'll submit the old one as soon as I get it accepted somewhere. Cheers, Markus > > NewJ Library is made up of the Pie Library, which is the foundation > layer > for automated object management and approximating Java language > features in > C++, and the Core J2 Library, which implements the core Java APIs like > lang, > io, util, etc. in C++. Both libraries combined consist of the > following: > > Total lines: 100757 > Comment lines: 22542 (22.372638702% of total) > Source code lines: 53245 (52.844963074% of total) > Blank lines: 24970 (24.782396317% of total) > > Number of assertions in Pie Library: 43 > Number of assertions in Core J2 Library (java.* packages): 441 > Number of assertions in Core J2 Library (private packages): 221 > > Additionally, the code base includes debug-only sections which take > advantage of conditional compilation inherent in C++. > > Number of units tests, coverage, lines: Same as the Java version of > Mauve > for the packages and classes we support. > > "Assertions not executed by any unit test": We have no quantitative > information on that. > > By "over several versions," what should I understand that to mean? > (a) By "over several versions" of NewJ Library, like 0.1, 0.2, 0.3. If > so, > that would be very difficult to provide to you. > (b) By "over several versions" of the Java API or specification. If > so, NewJ > Library supports 1.2 and some 1.4 features like assertions. Everything > is in > the same code base, compile-in or compile-out at build time, so it's > all the > same metrics. > > For more detailed information, download the trial version from > pure-native.com, read the NewJ Developer's Guide (they're quite > detailed and > only around 50 pages), and peruse the included header files and > samples. > > Hope this helps. > > Regards, > Vargas > http://www.pure-native.com/newj.html > > ----- Original Message ----- > From: "Markus Gälli" <markus.gaelli@iam.unibe.ch> > To: "PureNative Software" <info@pure-native.com> > Cc: "Sascha Brawer" <brawer@dandelis.ch>; "Stephen Crawley" > <crawley@dstc.edu.au>; <mauve-discuss@sources.redhat.com> > Sent: Monday, March 15, 2004 8:21 AM > Subject: Re: [Q] Number of unit tests in Mauve and of assertions in > Classpath? > > >> Hi Vargas, >> >>> Regarding your question: >>> >>>>>> - Do you know any (as responsive :-) java open source community >>>>>> with >>>>>> some big project and which >>>>>> uses both, JDK 1.4 (thus theoretically assertions) and unit tests? >>> >>> Although not a free (as in GNU) software project, NewJ Library for >>> C++, an >>> independent 100% native implementation of the core Java API, uses >>> both >>> unit >>> tests and assertions. The test cases are our C++ version of Mauve, >>> which >>> make use of the gnu::testlet::Testlet and the rest of the testing >>> framework >>> just like the Java version. The assertions are actually built right >>> into the >>> Core J2 Library implementation, which may be compiled in or out for >>> debug or >>> release builds. As you might expect, the use of both unit tests (like >>> Mauve) >>> and assertions find more coding or logic errors than either one >>> alone. >>> Furthermore, unit tests and assertions serve slightly different >>> purposes and >>> are applicable to slightly different circumstances. >> Thanks for that info. >> >> Maybe I am asking for too much here, but I would be very curious to >> know the >> >> - #lines of code(NewJ Library), >> - #assertions(NewJ Library), >> - #unit tests and >> - #lines of code(unit tests) >> -#coverage of the unit tests >> -#assertions not executed by any unit test >> - over several versions? >> >> So if you happen to be able to answer only some of this questions, >> this >> would be great. >> >> Cheers, >> >> Markus >> > ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [Q] Number of unit tests in Mauve and of assertions in Classpath? 2004-03-16 12:55 ` Markus Gälli @ 2004-03-16 17:15 ` PureNative Software 0 siblings, 0 replies; 11+ messages in thread From: PureNative Software @ 2004-03-16 17:15 UTC (permalink / raw) To: Markus Gälli; +Cc: Sascha Brawer, Stephen Crawley, mauve-discuss Markus, Too bad I counldn't get you the data sooner... The data is from NewJ Library version 0.5 (development snapshot) for the Win32 platform using Visual C++ 6.0. I'm not sure how many testing assertions (not to be confused with Java 1.4 assert) are inside Mauve. It would be easy to determine if you grep'd through the Mauve source code, if you could successfully download it... Regards, Vargas ----- Original Message ----- From: "Markus Gälli" <markus.gaelli@iam.unibe.ch> To: "Byron Vargas" <byron.vargas@worldpost.com> Cc: "Sascha Brawer" <brawer@dandelis.ch>; "Stephen Crawley" <crawley@dstc.edu.au>; <mauve-discuss@sources.redhat.com> Sent: Tuesday, March 16, 2004 7:55 AM Subject: Re: [Q] Number of unit tests in Mauve and of assertions in Classpath? Hi Byron, > Here are some stats for you. Thanks a lot for that detailed description. This is really of some use for me, I just could not include it in my paper as the submission deadline was one hour ago. I'd be curious, if you could also provide me with a version name of NewJLibrary, where this data holds. As I was not successful to download Mauve (I tried the ftp version, but the server seems to be down, then my OS-X stopped without throwing any exception...), I would also like to know how many assertions are called inside the Mauve tests, maybe you or someone else can answer this question too? Then I stop bothering you, promised :-) Thanks again, this can be of some great value in one of the next papers I want to write, I'll submit the old one as soon as I get it accepted somewhere. Cheers, Markus > > NewJ Library is made up of the Pie Library, which is the foundation > layer > for automated object management and approximating Java language > features in > C++, and the Core J2 Library, which implements the core Java APIs like > lang, > io, util, etc. in C++. Both libraries combined consist of the > following: > > Total lines: 100757 > Comment lines: 22542 (22.372638702% of total) > Source code lines: 53245 (52.844963074% of total) > Blank lines: 24970 (24.782396317% of total) > > Number of assertions in Pie Library: 43 > Number of assertions in Core J2 Library (java.* packages): 441 > Number of assertions in Core J2 Library (private packages): 221 > > Additionally, the code base includes debug-only sections which take > advantage of conditional compilation inherent in C++. > > Number of units tests, coverage, lines: Same as the Java version of > Mauve > for the packages and classes we support. > > "Assertions not executed by any unit test": We have no quantitative > information on that. > > By "over several versions," what should I understand that to mean? > (a) By "over several versions" of NewJ Library, like 0.1, 0.2, 0.3. If > so, > that would be very difficult to provide to you. > (b) By "over several versions" of the Java API or specification. If > so, NewJ > Library supports 1.2 and some 1.4 features like assertions. Everything > is in > the same code base, compile-in or compile-out at build time, so it's > all the > same metrics. > > For more detailed information, download the trial version from > pure-native.com, read the NewJ Developer's Guide (they're quite > detailed and > only around 50 pages), and peruse the included header files and > samples. > > Hope this helps. > > Regards, > Vargas > http://www.pure-native.com/newj.html > > ----- Original Message ----- > From: "Markus Gälli" <markus.gaelli@iam.unibe.ch> > To: "PureNative Software" <info@pure-native.com> > Cc: "Sascha Brawer" <brawer@dandelis.ch>; "Stephen Crawley" > <crawley@dstc.edu.au>; <mauve-discuss@sources.redhat.com> > Sent: Monday, March 15, 2004 8:21 AM > Subject: Re: [Q] Number of unit tests in Mauve and of assertions in > Classpath? > > >> Hi Vargas, >> >>> Regarding your question: >>> >>>>>> - Do you know any (as responsive :-) java open source community >>>>>> with >>>>>> some big project and which >>>>>> uses both, JDK 1.4 (thus theoretically assertions) and unit tests? >>> >>> Although not a free (as in GNU) software project, NewJ Library for >>> C++, an >>> independent 100% native implementation of the core Java API, uses >>> both >>> unit >>> tests and assertions. The test cases are our C++ version of Mauve, >>> which >>> make use of the gnu::testlet::Testlet and the rest of the testing >>> framework >>> just like the Java version. The assertions are actually built right >>> into the >>> Core J2 Library implementation, which may be compiled in or out for >>> debug or >>> release builds. As you might expect, the use of both unit tests (like >>> Mauve) >>> and assertions find more coding or logic errors than either one >>> alone. >>> Furthermore, unit tests and assertions serve slightly different >>> purposes and >>> are applicable to slightly different circumstances. >> Thanks for that info. >> >> Maybe I am asking for too much here, but I would be very curious to >> know the >> >> - #lines of code(NewJ Library), >> - #assertions(NewJ Library), >> - #unit tests and >> - #lines of code(unit tests) >> -#coverage of the unit tests >> -#assertions not executed by any unit test >> - over several versions? >> >> So if you happen to be able to answer only some of this questions, >> this >> would be great. >> >> Cheers, >> >> Markus >> > ^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2004-03-16 17:15 UTC | newest] Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2004-03-11 6:28 [Q] Number of unit tests in Mauve and of assertions in Classpath? Markus Gälli 2004-03-11 6:49 ` Stephen Crawley 2004-03-11 8:19 ` Markus Gälli 2004-03-11 9:34 ` Sascha Brawer 2004-03-11 10:51 ` Markus Gälli 2004-03-15 12:46 ` PureNative Software 2004-03-15 13:21 ` Markus Gälli 2004-03-16 3:04 ` PureNative Software 2004-03-16 3:04 ` Byron Vargas 2004-03-16 12:55 ` Markus Gälli 2004-03-16 17:15 ` PureNative Software
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).