From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 93783 invoked by alias); 7 Jun 2016 07:37:20 -0000 Mailing-List: contact libc-alpha-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: libc-alpha-owner@sourceware.org Received: (qmail 93769 invoked by uid 89); 7 Jun 2016 07:37:19 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-2.4 required=5.0 tests=AWL,BAYES_00,RCVD_IN_DNSWL_LOW autolearn=ham version=3.3.2 spammy=inh, harness, canned, 2033 X-HELO: mail-qg0-f53.google.com X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:to:from:subject:organization:message-id:date :user-agent:mime-version:content-transfer-encoding; bh=lvnVlEdvLNsmLbpm7ce2NbHCuB4AA2KzixYHL5YPRas=; b=dFAO5K92x3BFRu49o/TPmv2edGAl/Qv16vIRWZ/2kNbbEeNKmVUXCTh82g0NXxviSb yGTN/XWAyQSt/wT6fLKFldwEL42jGduRXihBaje3k5nLwjw/cAgQipHAODsoTqvZVRdC UrN2OY+ei4t249Nfw4Ozw52IExdoqI/Hh1RPbIxN9P7fI75INPrhRgSxTN1drbbLG0sy CV45TCE5Du7AY1ydBqel47y15DSbGz6oG/DLu3LzQEKXHV94c2biWoX6KimTXBPPF64W J+l9vwCaOMCLSLMLXDsPwAevP0fwx/PkagBrZYsVp/TmW7r91f9ZT3w1CdaQG3nf3lP2 5dtA== X-Gm-Message-State: ALyK8tKb2WMUY0b+GnXhjmt408Z5heFubxPKzbSl9OroXtzWpqpSMhskCPO7cXqI13sFyvB4 X-Received: by 10.140.29.201 with SMTP id b67mr18669807qgb.77.1465285027104; Tue, 07 Jun 2016 00:37:07 -0700 (PDT) To: GNU C Library , "Joseph S. Myers" From: Carlos O'Donell Subject: [PATCH] Add compile testing to glibc test framework. Message-ID: <575679A0.4090209@redhat.com> Date: Tue, 07 Jun 2016 07:37:00 -0000 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.7.1 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-SW-Source: 2016-06/txt/msg00162.txt.bz2 The goal of this patch is to add compile testing to the glibc test framework. The changes allow you to add C source files that the test framework can compile and verify the compiler exits with a zero exit status for PASS. While a non-zero exit status by the compiler indicates a FAIL. The support for UNSUPPORTED is handled in the normal way for other tests that are conditionally disabled (can't be run and thus can't exit with code 77). Only C source files are currently supported because that's all I needed to support today, but it could be extended to compile any type of source target. I wanted to approach the work incrementally. The test is being compiled under the glibc test framework, which is as close as we currently get to building real user applications. For example I use the test to compile various glibc/linux header order combinations e.g. cat sysdeps/unix/sysv/linux/tst-ipv6-compat1.c #include #include int main (void) { return 0; } git diff sysdeps/unix/sysv/linux/Makefile diff --git a/sysdeps/unix/sysv/linux/Makefile b/sysdeps/unix/sysv/linux/Makefile index c57575f..dad556c 100644 --- a/sysdeps/unix/sysv/linux/Makefile +++ b/sysdeps/unix/sysv/linux/Makefile @@ -203,3 +203,9 @@ tests += tst-setgetname tst-align-clone tst-getpid1 \ tst-thread-affinity-pthread tst-thread-affinity-pthread2 \ tst-thread-affinity-sched endif + +ifeq ($(subdir),inet) +# Add tests for Linux and glibc header compatibility. +tests-compile += tst-ipv6-compat1 tst-ipv6-compat2 tst-ipv6-compat3 +TEST_SRC_DIR = ../sysdeps/unix/sysv/linux/ +endif Previously these kinds of tests would result in the testsuite failing to run to completion because compiler errors are treated as harness failures. I tried a variety of solutions to implement this, but settled on the present set of two variables: tests-compile and TESTS_SRC_DIR as a way to avoid needing anything like sysd-rules to find the test sources. I did not want to unduly slow down build test cycles. The static pattern rule is pretty concise and optimal. Thoughts? -- Cheers, Carlos. 2015-06-06 Carlos O'Donell * Rules [ifeq ($(run-built-tests),no)] (tests): Add $(tests-compile). [ifneq ($(run-built-tests),no)] (tests): Add $(tests-compile:%=$(objpfx)%.out). (tests): Add $(tests-compile) to merge result list. [ifeq ($(build-programs),yes)] (binaries-all-tests): Add $(tests-compile). [ifneq "$(strip $(tests-compile))" ""] (tc-OUTPUT_OPTION-file): Define. (tc-OUTPUT_OPTION): Define. (tc-compile-mkdeps-flags): Define. (tc-compile.c): Define. ($(tests-compile:%=$(objpfx)%.out)): Define static pattern rule. diff --git a/Rules b/Rules index 8306d36..e92a9ff 100644 --- a/Rules +++ b/Rules @@ -91,10 +91,10 @@ else others: $(addprefix $(objpfx),$(extra-objs)) endif ifeq ($(run-built-tests),no) -tests: $(addprefix $(objpfx),$(tests) $(test-srcs)) $(tests-special) +tests: $(addprefix $(objpfx),$(tests) $(test-srcs)) $(tests-compile) $(tests-special) xtests: tests $(xtests-special) else -tests: $(tests:%=$(objpfx)%.out) $(tests-special) +tests: $(tests:%=$(objpfx)%.out) $(tests-compile:%=$(objpfx)%.out) $(tests-special) xtests: tests $(xtests:%=$(objpfx)%.out) $(xtests-special) endif @@ -102,7 +102,7 @@ tests-special-notdir = $(patsubst $(objpfx)%, %, $(tests-special)) xtests-special-notdir = $(patsubst $(objpfx)%, %, $(xtests-special)) tests: $(..)scripts/merge-test-results.sh -s $(objpfx) $(subdir) \ - $(sort $(tests) $(tests-special-notdir:.out=)) \ + $(sort $(tests) $(tests-special-notdir:.out=) $(tests-compile)) \ > $(objpfx)subdir-tests.sum xtests: $(..)scripts/merge-test-results.sh -s $(objpfx) $(subdir) \ @@ -111,7 +111,7 @@ xtests: ifeq ($(build-programs),yes) binaries-all-notests = $(others) $(sysdep-others) -binaries-all-tests = $(tests) $(xtests) $(test-srcs) +binaries-all-tests = $(tests) $(xtests) $(test-srcs) $(tests-compile) binaries-all = $(binaries-all-notests) $(binaries-all-tests) binaries-static-notests = $(others-static) binaries-static-tests = $(tests-static) $(xtests-static) @@ -198,6 +198,45 @@ $(objpfx)%.out: /dev/null $(objpfx)% # Make it 2nd arg for canned sequence. $(make-test-out) > $@; \ $(evaluate-test) +# tests-compile provides a way for developers to test compiling a singular +# C source file under the environment of the glibc build. This is useful +# when testing system invariants like compatibility between headers files +# provided by the linux kernel headers (--with-headers). While it is +# arguable that such tests, as invariants, should go in configure scripts, +# that would lead to a large number of unruly tests in configure.ac. +# Having the compatibility tests as part of the regression testsuite is far +# more manageable from a maintenance perspective. +# +# There are three things you need to do to add a compilation test to glibc: +# * Create the source file to compile. +# - It should compile without error in order for the test to PASS. +# - It should compile with error in order for the test to FAIL. +# +# * Add the test name to 'tests-compile' +# - e.g. +# ifeq ($(subdir),inet) +# tests-compile += tst-ipv6-compat +# ... +# +# * Set 'TEST_SRC_DIR' to the relative path to the test sources (no trailing +# slash): +# ... +# TEST_SRC_DIR = ../sysdeps/unix/sysv/linux +# endif +# +# You will now have an additional test that is compiled during 'make check', +# with compiler output to $(objpfx)/.out as you would expect for +# a normal test. +ifneq "$(strip $(tests-compile))" "" +tc-OUTPUT_OPTION-file = $(objpfx)$(notdir $(@:%.out=%.o)) +tc-OUTPUT_OPTION = -o $(tc-OUTPUT_OPTION-file) +tc-compile-mkdep-flags = -MD -MP -MF $(tc-OUTPUT_OPTION-file).dt -MT $(tc-OUTPUT_OPTION-file) +tc-compile.c = $(CC) $(TEST_SRC_DIR)/$(notdir $(@:%.out=%.c)) -c $(CFLAGS) $(CPPFLAGS) +$(tests-compile:%=$(objpfx)%.out): $(objpfx)%.out : $(TEST_SRC_DIR)/%.c Makefile + $(tc-compile.c) $(tc-OUTPUT_OPTION) $(tc-compile-mkdep-flags) \ + >& $@; $(evaluate-test) +endif + # tests-unsupported lists tests that we will not try to build at all in # this configuration. Note this runs every time because it does not # actually create its target. The dependency on Makefile is meant to