From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from relay6-d.mail.gandi.net (relay6-d.mail.gandi.net [217.70.183.198]) by sourceware.org (Postfix) with ESMTPS id C81F23858436 for ; Wed, 10 Nov 2021 10:38:53 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org C81F23858436 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=seketeli.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=seketeli.org Received: (Authenticated sender: dodji@seketeli.org) by relay6-d.mail.gandi.net (Postfix) with ESMTPSA id 227E3C0010; Wed, 10 Nov 2021 10:38:51 +0000 (UTC) Received: by localhost (Postfix, from userid 1000) id 366E258010A; Tue, 9 Nov 2021 15:39:33 +0100 (CET) From: Dodji Seketeli To: tangmeng Cc: libabigail@sourceware.org Subject: Re: [PATCH v3] make abicompat test happy Organization: Me, myself and I References: <20211109122759.23613-1-tangmeng@uniontech.com> X-Operating-System: Fedora 36 X-URL: http://www.seketeli.net/~dodji Date: Tue, 09 Nov 2021 15:39:33 +0100 In-Reply-To: <20211109122759.23613-1-tangmeng@uniontech.com> (tangmeng@uniontech.com's message of "Tue, 9 Nov 2021 20:27:59 +0800") Message-ID: <87wnlh75qy.fsf@seketeli.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Status: No, score=-9.1 required=5.0 tests=BAYES_00, DATE_IN_PAST_12_24, GIT_PATCH_0, JMQ_SPF_NEUTRAL, KAM_DMARC_STATUS, RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H2, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: libabigail@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Mailing list of the Libabigail project List-Unsubscribe: , List-Archive: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Nov 2021 10:38:55 -0000 Hello, tangmeng a =C3=A9crit: [...] > 3. The runtestabicompat command itself is executed successfully, so it > should return 0, which should be distinguished from the test result. Actually, we want to return a non-zero value when any of the individual tests fail. This is because "make check" runs all the tests of the regression test suite and it looks at the return value of each of the runtestXXXX programs to detect if it failed or not. So we really need to be non-zero when a test case fails. I fixed it and applied the patch below. Thanks! Cheers, commit b7ba0fe8f5976251cf38f9abf249acdacdcf626b Author: tangmeng Date: Tue Nov 9 20:27:59 2021 +0800 test-abicompat: Make the test output more pleasant =20=20=20=20 When testing with runtestabicompat, the following problems were found: 1. abicompat tested multiple scenarios, but the last result was used as the basis for the return value of the command. 2. For multiple test scenarios, the execution results cannot be known after the test, which is easy to cause confusion. =20=20=20=20 * test/test-abicompat.cc (main): make test output more pleasant. =20=20=20=20 Signed-off-by: tangmeng Signed-off-by: Dodji Seketeli diff --git a/tests/test-abicompat.cc b/tests/test-abicompat.cc index 49f1d5d4..a1339eef 100644 --- a/tests/test-abicompat.cc +++ b/tests/test-abicompat.cc @@ -27,6 +27,7 @@ =20 using std::string; using std::cerr; +using std::cout; =20 struct InOutSpec { @@ -206,12 +207,13 @@ main() using abigail::tools_utils::ensure_parent_dir_created; using abigail::tools_utils::abidiff_status; =20 - bool is_ok =3D true; + unsigned int cnt_total =3D 0, cnt_passed =3D 0, cnt_failed =3D 0; string in_app_path, in_lib1_path, in_lib2_path, suppression_path, - abicompat_options, ref_report_path, out_report_path, abicompat, cmd; + abicompat_options, ref_report_path, out_report_path, abicompat, cmd, d= iffcmd; =20 for (InOutSpec* s =3D in_out_specs; s->in_app_path; ++s) { + bool is_ok =3D true; in_app_path =3D string(get_src_dir()) + "/tests/" + s->in_app_path; in_lib1_path =3D string(get_src_dir()) + "/tests/" + s->in_lib1_path; if (s->in_lib2_path && strcmp(s->in_lib2_path, "")) @@ -253,13 +255,37 @@ main() =20 if (abicompat_ok) { - cmd =3D "diff -u " + ref_report_path + " " + out_report_path; - if (system(cmd.c_str())) + diffcmd =3D "diff -u " + ref_report_path + " " + out_report_path; + if (system(diffcmd.c_str())) is_ok =3D false; } else is_ok =3D false; + + if (is_ok) + { + cout << BRIGHT_YELLOW_COLOR + << "Test Passed:" + << DEFAULT_TERMINAL_COLOR + << cmd + << std::endl; + cnt_passed++; + } + else + { + cout << BRIGHT_RED_COLOR + << "Test Failed:" + << DEFAULT_TERMINAL_COLOR + << cmd + << std::endl; + cnt_failed++; + } + cnt_total++; } + cout << "Summary: " << cnt_total << " tested!" + << " Test Passed: " << cnt_passed + << ", Test Failed: " << cnt_failed + << ".\n"; =20 - return !is_ok; + return cnt_failed; } diff --git a/tests/test-utils.h b/tests/test-utils.h index 46d9b785..5596edc6 100644 --- a/tests/test-utils.h +++ b/tests/test-utils.h @@ -9,6 +9,10 @@ #include "config.h" #include =20 +#define BRIGHT_YELLOW_COLOR "\e[1;33m" +#define BRIGHT_RED_COLOR "\e[1;31m" +#define DEFAULT_TERMINAL_COLOR "\033[0m" + namespace abigail { namespace tests --=20 Dodji