How to test fail conditions in TestCase?

How to test fail conditions in TestCase?

[Phil Muldoon]

Hi

I've been added "expected normal" results to test cases as normal, but 
I've just started "expected fail" conditions for the test case of 
CorefileByteBuffer. I was reminded again, when writing previous tests, I 
was unsure of how to write these properly. Here is what I do now.

For a normal, expected pass situation, something like:

    CorefileByteBuffer coreBuffer = new CorefileByteBuffer(new 
File(Config.getPkgDataDir (),
                                                                    
"test-core"));

    // Set the corefile carat to memory address 0x00170000L
    coreBuffer.position(0x00170000L);
    assertEquals("Peek a byte at 0x00170000",0x7f,coreBuffer.get());
    assertEquals("Peek a byte at 0x00170001",0x45,coreBuffer.get());
    assertEquals("Peek a byte at 0x00170002",0x4c,coreBuffer.get());
    assertEquals("Peek a byte at 0x00170003",0x46,coreBuffer.get());
    assertEquals("Peek a byte at 0x00170004",0x01,coreBuffer.get());


But for an expected fail (ie an exception should be thrown or an error 
condition detected) something like:

    // Test reading at position 0. In this core file there is no
    // segment at position 0 (in our case for this bytebuffer, the
    // position within the buffer should equal the memory address access.)
    // This should always fail!

    coreBuffer.position(0);
    try
    {
      assertEquals("Peek a byte at 0x00170000",0x7f,coreBuffer.get());
    }
    catch (RuntimeException e)
    {
      return;
    }
    fail(".get() read at position 0 should have failed but didn't!");
  }
 

Is that the "right" way to do above? Is there a "right" way?

Regards

Phil

Re: How to test fail conditions in TestCase?

[Kris Van Hees]

From the JUnit 3.8.2 FAQ (prior to the introduction of annotations that
only work with Java 1.5.0 and up):

> How do I write a test that passes when an expected exception is thrown?
> 
> Catch the exception within the test method. If it isn't thrown, call the
> fail() method to signal the failure of the test.
> 
> The following is an example test that passes when the expected
> IndexOutOfBoundsException is raised:
> 
>     public void testIndexOutOfBoundsException() {
> 		
>         ArrayList emptyList = new ArrayList();
> 		
>         try {
> 	
>             Object o = emptyList.get(0);
> 	
>             fail("Should raise an IndexOutOfBoundsException");
> 	
>         } catch (IndexOutOfBoundsException expected) {
>             assertTrue(true);
>         }
>     }
      
So, yes, you are right :)

On Mon, Apr 16, 2007 at 01:55:18PM -0500, Phil Muldoon wrote:
> Hi
> 
> I've been added "expected normal" results to test cases as normal, but 
> I've just started "expected fail" conditions for the test case of 
> CorefileByteBuffer. I was reminded again, when writing previous tests, I 
> was unsure of how to write these properly. Here is what I do now.
> 
> For a normal, expected pass situation, something like:
> 
>    CorefileByteBuffer coreBuffer = new CorefileByteBuffer(new 
> File(Config.getPkgDataDir (),
>                                                                    
> "test-core"));
> 
>    // Set the corefile carat to memory address 0x00170000L
>    coreBuffer.position(0x00170000L);
>    assertEquals("Peek a byte at 0x00170000",0x7f,coreBuffer.get());
>    assertEquals("Peek a byte at 0x00170001",0x45,coreBuffer.get());
>    assertEquals("Peek a byte at 0x00170002",0x4c,coreBuffer.get());
>    assertEquals("Peek a byte at 0x00170003",0x46,coreBuffer.get());
>    assertEquals("Peek a byte at 0x00170004",0x01,coreBuffer.get());
> 
> 
> But for an expected fail (ie an exception should be thrown or an error 
> condition detected) something like:
> 
>    // Test reading at position 0. In this core file there is no
>    // segment at position 0 (in our case for this bytebuffer, the
>    // position within the buffer should equal the memory address access.)
>    // This should always fail!
> 
>    coreBuffer.position(0);
>    try
>    {
>      assertEquals("Peek a byte at 0x00170000",0x7f,coreBuffer.get());
>    }
>    catch (RuntimeException e)
>    {
>      return;
>    }
>    fail(".get() read at position 0 should have failed but didn't!");
>  }
> 
> 
> Is that the "right" way to do above? Is there a "right" way?
> 
> Regards
> 
> Phil

Re: How to test fail conditions in TestCase?

[Mark Wielaard]

Hi Phil,

On Mon, 2007-04-16 at 13:55 -0500, Phil Muldoon wrote:
> But for an expected fail (ie an exception should be thrown or an error 
> condition detected) something like:
> 
>     // Test reading at position 0. In this core file there is no
>     // segment at position 0 (in our case for this bytebuffer, the
>     // position within the buffer should equal the memory address access.)
>     // This should always fail!
> 
>     coreBuffer.position(0);
>     try
>     {
>       assertEquals("Peek a byte at 0x00170000",0x7f,coreBuffer.get());
>     }
>     catch (RuntimeException e)
>     {
>       return;
>     }
>     fail(".get() read at position 0 should have failed but didn't!");
>   }
>  
> 
> Is that the "right" way to do above? Is there a "right" way?

The asserEquals here feels weird since you don't expect it to be
actually executed (since coreBuffer.get() should raise an exception).
Personally I like to use an explicit control variable that gets checked
as follows:

  boolean runtimeExceptionRaised = false;
  coreBuffer.position(0);
  try
    {
      coreBuffer.get();
    }
  catch (RuntimeException e)
    {
      runtimeExceptionRaised = true;
    }
  assertTrue(".get() read at position 0 raised runtime exeception",
             runtimeExceptionRaised);

Cheers,

Mark

Re: How to test fail conditions in TestCase?

[Phil Muldoon]

Mark Wielaard wrote:
>
> The asserEquals here feels weird since you don't expect it to be
> actually executed (since coreBuffer.get() should raise an exception).
> Personally I like to use an explicit control variable that gets checked
> as follows:
>   


Yours and Kris's statement basically agree, but differ in 
implementation, so I'll go forward with those recommendation. I need to 
brush of my Junit reference as I completely forgot the use of AssertTrue 
in situations like these.

Thanks!

Regards

Phil