Re: [PATCH] asan unit tests from llvm lit-test

[Jakub Jelinek <jakub@redhat.com>]

On Wed, Nov 28, 2012 at 12:03:27PM +0100, Jakub Jelinek wrote:
> Then the test would be run once without ASAN_DIE_IF in environment (or =0),
> that would produce output full of
> EXPECT_DEATH1 AddressSanitizer:.*heap-use-after-free EXPECT_DEATHEND1
> ...
> which tcl could parse, and figure from it that it should run the test
> again 156 or how many times, with ASAN_DIE_IF from 1 to 156, and at each
> iteration try to match the output against the regexp for that iteration.
> Then you'd essentially just have to tweak a few lines at the start of the
> test, includes, first few lines in main and that would be it.

That said, I find it very undesirable to put that many tests into one large
one, especially if it triggers undefined behavior like:
ATTRIBUTE_NO_ADDRESS_SAFETY_ANALYSIS
static void NoAddressSafety() {
  char *foo = new char[10];
  Ident(foo)[10] = 0;
  delete [] foo;
}

TEST(AddressSanitizer, AttributeNoAddressSafetyTest) {
  Ident(NoAddressSafety)();
}

As soon as you corrupt malloc state, anything can happen.  Things like
this should be verified by a compile only test that no instrumentation calls
have been added.

Looking at the test, there aren't just EXPECT_DEATH kinds of tests, and
running everything not guarded with EXPECT_DEATH macro many times might
be too expensive.  So perhaps it could run each TEST as a separate process,
by first just printing all test names that sould be run, then running
them one by one by asking for it in env, and for tests that would print
EPXECT_DEATHX ... EXPECT_DEATHENDX run that particular test again with
the requested EXPECT_DEATH counter.
Or run everything except EXPECT_DEATH macros first, and in EXPECT_DEATH
printouts print not just some counter, but also name of the current test,
and then when rerunning for some particular EXPECT_DEATH just run the
corresponding TEST and not all others.  Still, it is a couple of dozens
of lines in the test (defining the macros) and a little more than that
in tcl.

	Jakub