Need help building and executing specific run tests for libstdc++

Need help building and executing specific run tests for libstdc++

[Jakob Hasse]

Hello, I'm stuck building and running the tests for gcc, in particular the run-tests (not compile tests) for just one test case I added myself.

I found an issue in libstdc++ (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=105387) and now want to write a test for it to prove that the patch I created works and that the libstdc++ code breaks without the patch. I added the test application into libstdc++-v3/testsuite/abi/. Now how do I run this very application and via the test suite and how do I confirm that it successfully ran?

I've been looking into the libstdc++ test doc page: libstdc++-v3/doc/html/manual/test.html in the repository, but it is still not clear to me how to do the aforementioned task. After successfully building the entire gcc, I stayed in the build directory and tried this: make check-c++ RUNTESTFLAGS="libstdc++.exp". But all I got was a bunch of output without any of the files in libstdc++-v3/testsuite/abi/ listed.

How do I run the tests (with stdout output, because using gdb is another story) for a particular .cc file in any of the directories in libstdc++-v3/testsuite/ ?

Thanks,
Jakob

Re: Need help building and executing specific run tests for libstdc++

[Jonathan Wakely]

On Thu, 28 Apr 2022 at 12:37, Jakob Hasse via Gcc-help
<gcc-help@gcc.gnu.org> wrote:
>
> Hello, I'm stuck building and running the tests for gcc, in particular the run-tests (not compile tests) for just one test case I added myself.
>
> I found an issue in libstdc++ (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=105387) and now want to write a test for it to prove that the patch I created works and that the libstdc++ code breaks without the patch. I added the test application into libstdc++-v3/testsuite/abi/.

It should be in testsuite/18_support

> Now how do I run this very application and via the test suite and how do I confirm that it successfully ran?
>
> I've been looking into the libstdc++ test doc page: libstdc++-v3/doc/html/manual/test.html in the repository, but it is still not clear to me how to do the aforementioned task.

https://gcc.gnu.org/onlinedocs/libstdc++/manual/test.html#test.run
should be clear, no?

> After successfully building the entire gcc, I stayed in the build directory and tried this: make check-c++ RUNTESTFLAGS="libstdc++.exp". But all I got was a bunch of output without any of the files in libstdc++-v3/testsuite/abi/ listed.

That's not what the docs say to do though.

You want conformance.exp not libstdc++.exp and it's easier to run it
from the $target/libstdc++-v3 directory not the top-level build
directory.

The test output goes to the logs, not to stdout.

>
> How do I run the tests (with stdout output, because using gdb is another story) for a particular .cc file in any of the directories in libstdc++-v3/testsuite/ ?

make check RUNTESTFLAGS=conformance.exp=yourfile.cc

You should not rely on stdout output, because that can't be automated.
You should #include <testsuite_hooks.h> and use the VERIFY macro,
which is equivalent to the standard 'assert' macro.

The results can be checked in the testsuite/libstdc++.log file.