From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm1-x32a.google.com (mail-wm1-x32a.google.com [IPv6:2a00:1450:4864:20::32a]) by sourceware.org (Postfix) with ESMTPS id 0E021385C303 for ; Tue, 3 Oct 2023 10:37:24 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 0E021385C303 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=gmail.com Received: by mail-wm1-x32a.google.com with SMTP id 5b1f17b1804b1-406589e5765so7458915e9.0 for ; Tue, 03 Oct 2023 03:37:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1696329442; x=1696934242; darn=gcc.gnu.org; h=content-transfer-encoding:mime-version:message-id:references :in-reply-to:subject:cc:to:from:date:from:to:cc:subject:date :message-id:reply-to; bh=piwGnVJCUpABJ8QD3K+uITWeYEaTOM4aKzYwGI9QtQg=; b=hfcO5yftIUNpJLfurd/fnbaoMNXovnoWl7CyFZZK0oiIRDjvqr19zv0rzUOoEZ3uQX mMNP0jrs2DJUZ6RhCie1X1e7Xf7t0o0JG1l4JslrAihczT+lRrOLZI1T7Gb2Dzr8tLZS FhJvvCL3CBFy9vPNZKWHF1hBatnrhMJf8zUL6tKbcLhJaXamEwGraNN9MMCY8NZ64tCn qtTaP4ycPISZ1m2EcQnfdGf0oYg7ccACOb5pXhonfsyDWHqtfc5QT4uUTi4LrdZBlCjO LpMGLuDMbklFY20jK6VOgz8HCawiLcbjBaLOADXIsXZG3CjgHRwQ1m2WDs62Q7pQg3rG dhaw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696329442; x=1696934242; h=content-transfer-encoding:mime-version:message-id:references :in-reply-to:subject:cc:to:from:date:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=piwGnVJCUpABJ8QD3K+uITWeYEaTOM4aKzYwGI9QtQg=; b=FzSeuyPztVQ3usbvQzOa8mMnTLP8SqbuKfsVvtb51gfY67p9I/Y/QXkKB/AfhRuyzE /zmM15izR0+UQxAE5e/Upscw1rWvJk2geBR2x/do2jI1ZSTs7CRJ0Xv6fSNoQ2kkaKVH 3N33qEXVP4oaykSxmeqOn9CfDw+SWWQslRmYgRS/2ouhr+l91NeRcVNE53lVwHaGnthC JfWgiaHdYnamP1D+4wpxtlc0FXbxikY1t1lFJb3p9AU23e7pckzw3Yw4dhHAJ6fk7ALQ MQDSJm0uyojjk2WpCOVc4bWv8bXOk8cyKFQ1A2NoACJQRAES2aeu9YYTV4I245pQx5jn 9fHA== X-Gm-Message-State: AOJu0YwBXQ9sFgz6n0X25DFxhAstaYC0g/vEbeUrqKohZkBCmr3V2fdG b2LhgbP8SKfM/bJ9uI07mnyAUy6/5TL2SQ== X-Google-Smtp-Source: AGHT+IEHRqGGb4hDd8AFJjUCrt9cwf4xhpyzZhnkFY07HayBGVAefzpXDwrOlOxvZxGcih6cU8mwfg== X-Received: by 2002:a05:600c:218d:b0:3fb:c9f4:150e with SMTP id e13-20020a05600c218d00b003fbc9f4150emr12084524wme.14.1696329442198; Tue, 03 Oct 2023 03:37:22 -0700 (PDT) Received: from [127.0.0.1] (80-110-214-113.static.upcbusiness.at. [80.110.214.113]) by smtp.gmail.com with ESMTPSA id p25-20020a7bcc99000000b004063ea92492sm972351wma.22.2023.10.03.03.37.21 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 03 Oct 2023 03:37:21 -0700 (PDT) Date: Tue, 03 Oct 2023 12:37:20 +0200 From: rep.dot.nop@gmail.com To: Maxim Kuvyrkov CC: Maxim Kuvyrkov via Gcc-patches , Jeff Law , Diego Novillo , Doug Evans Subject: =?US-ASCII?Q?Re=3A_=5BPATCH_01/12=5D_=5Bcontrib=5D_validate=5F?= =?US-ASCII?Q?failures=2Epy=3A_Avoid_testsuite_aliasing?= In-Reply-To: <5D44DB95-5D01-4D4A-AA36-499079DBA92D@linaro.org> References: <20230602152052.1874860-1-maxim.kuvyrkov@linaro.org> <20230602152052.1874860-2-maxim.kuvyrkov@linaro.org> <20230926174604.4266b7c7@nbbrfq.loc> <5D44DB95-5D01-4D4A-AA36-499079DBA92D@linaro.org> Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Status: No, score=-3.3 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: On 27 September 2023 16:47:27 CEST, Maxim Kuvyrkov wrote: >Hi Bernhard, > >Thanks, I meant to fix this, but forgot=2E np=2E >The underlying problem here is that we want to detect which sub-testsuite= s had failures=2E Current regex doesn't match go's case because there is n= o "=2E=2E=2E" at the end: "Running foo" vs "Running foo =2E=2E=2E" =2E > >My preferred way of fixing this is to make go's testsuite print out "=2E= =2E=2E" =2E We have a similar patch for glibc [1]=2E > >[1] https://sourceware=2Eorg/pipermail/libc-alpha/2023-June/148702=2Ehtml Which asks: ---8<--- >> WDYT? >=20 > I looked at the gcc-testresults mailing list, and there appear no > =3D=3D=3D =E2=80=A6 failures =3D=3D=3D lines at all? What was the motiv= ation for adding it > in the first place? The only motivation is that it looks like a nice header for the following = FAILs=2E What's your preference for the line -- drop it entirely or print = out: =3D=3D=3D glibc failures =3D=3D=3D no unexpected failures ? ---8<--- I'd drop the above entirely if there are no failures, it's pretty superflu= ous, isn't it=2E And concerning gotools and the missing trailing "=2Eexp =2E=2E=2E", I gues= s it's fine to add that to streamline the gotools output to all the other e= xisting sum output=2E TIA, > >-- >Maxim Kuvyrkov >https://www=2Elinaro=2Eorg > >> On Sep 26, 2023, at 19:46, Bernhard Reutner-Fischer wrote: >>=20 >> Hi Maxim! >>=20 >> On Mon, 5 Jun 2023 18:06:25 +0400 >> Maxim Kuvyrkov via Gcc-patches wrote: >>=20 >>>> On Jun 3, 2023, at 19:17, Jeff Law wrote: >>>>=20 >>>> On 6/2/23 09:20, Maxim Kuvyrkov via Gcc-patches wrote: =20 >>>>> This patch adds tracking of current testsuite "tool" and "exp" >>>>> to the processing of =2Esum files=2E This avoids aliasing between >>>>> tests from different testsuites with same name+description=2E >>>>> E=2Eg=2E, this is necessary for testsuite/c-c++-common, which is ran >>>>> for both gcc and g++ "tools"=2E >>>>> This patch changes manifest format from =2E=2E=2E >>>>> >>>>> FAIL: gcc_test >>>>> FAIL: g++_test >>>>> >>>>> =2E=2E=2E to =2E=2E=2E >>>>> >>>>> =3D=3D=3D gcc tests =3D=3D=3D >>>>> Running gcc/foo=2Eexp =2E=2E=2E >>>>> FAIL: gcc_test >>>>> =3D=3D=3D gcc Summary =3D=3D >>>>> =3D=3D=3D g++ tests =3D=3D=3D >>>>> Running g++/bar=2Eexp =2E=2E=2E >>>>> FAIL: g++_test >>>>> =3D=3D=3D g++ Summary =3D=3D >>>>> =2E >>>>> The new format uses same formatting as DejaGnu's =2Esum files >>>>> to specify which "tool" and "exp" the test belongs to=2E =20 >>>> I think the series is fine=2E You're not likely to hear from Diego o= r Doug I suspect, I don't think either are involved in GNU stuff anymore=2E >>>>=20 >>>=20 >>> Thanks, Jeff=2E I'll wait for a couple of days and will merge if ther= e are no new comments=2E >>=20 >> Maxim, may i ask you to have a look at the following problem, please? >>=20 >> ISTM that your exp code does not work as expected for go, maybe you >> forgot to test the changes with go enabled? >>=20 >> Ever since your changes in summer i see the following: >>=20 >> gcc-14=2Emine$ /scratch/src/gcc-14=2Emine/contrib/testsuite-management/= validate_failures=2Epy --clean_build =2E=2E/gcc-14=2Eorig/ >> Getting actual results from build directory =2E >> =2E/gcc/testsuite/go/go=2Esum >> =2E/gcc/testsuite/gcc/gcc=2Esum >> =2E/gcc/testsuite/objc/objc=2Esum >> =2E/gcc/testsuite/jit/jit=2Esum >> =2E/gcc/testsuite/gdc/gdc=2Esum >> =2E/gcc/testsuite/gnat/gnat=2Esum >> =2E/gcc/testsuite/ada/acats/acats=2Esum >> =2E/gcc/testsuite/g++/g++=2Esum >> =2E/gcc/testsuite/obj-c++/obj-c++=2Esum >> =2E/gcc/testsuite/rust/rust=2Esum >> =2E/gcc/testsuite/gfortran/gfortran=2Esum >> =2E/x86_64-pc-linux-gnu/libgomp/testsuite/libgomp=2Esum >> =2E/x86_64-pc-linux-gnu/libphobos/testsuite/libphobos=2Esum >> =2E/x86_64-pc-linux-gnu/libstdc++-v3/testsuite/libstdc++=2Esum >> =2E/x86_64-pc-linux-gnu/libffi/testsuite/libffi=2Esum >> =2E/x86_64-pc-linux-gnu/libitm/testsuite/libitm=2Esum >> =2E/x86_64-pc-linux-gnu/libgo/libgo=2Esum >> =2E/x86_64-pc-linux-gnu/libatomic/testsuite/libatomic=2Esum >> =2E/gotools/gotools=2Esum >> =2Esum file seems to be broken: tool=3D"gotools", exp=3D"None", summary= _line=3D"FAIL: TestScript" >> Traceback (most recent call last): >> File "/scratch/src/gcc-14=2Emine/contrib/testsuite-management/validate= _failures=2Epy", line 732, in >> retval =3D Main(sys=2Eargv) >> File "/scratch/src/gcc-14=2Emine/contrib/testsuite-management/validate= _failures=2Epy", line 721, in Main >> retval =3D CompareBuilds() >> File "/scratch/src/gcc-14=2Emine/contrib/testsuite-management/validate= _failures=2Epy", line 622, in CompareBuilds >> actual =3D GetResults(sum_files) >> File "/scratch/src/gcc-14=2Emine/contrib/testsuite-management/validate= _failures=2Epy", line 466, in GetResults >> build_results=2Eupdate(ParseSummary(sum_fname)) >> File "/scratch/src/gcc-14=2Emine/contrib/testsuite-management/validate= _failures=2Epy", line 405, in ParseSummary >> result =3D result_set=2EMakeTestResult(line, ordinal) >> File "/scratch/src/gcc-14=2Emine/contrib/testsuite-management/validate= _failures=2Epy", line 239, in MakeTestResult >> return TestResult(summary_line, ordinal, >> File "/scratch/src/gcc-14=2Emine/contrib/testsuite-management/validate= _failures=2Epy", line 151, in __init__ >> raise >> RuntimeError: No active exception to reraise >>=20 >>=20 >> The problem seems to be that gotools=2Esum does not mention any "=2Eexp= " >> files=2E >>=20 >> $ grep "Running " gotools/gotools=2Esum=20 >> Running cmd/go >> Running runtime >> Running cgo >> Running carchive >> Running cmd/vet >> Running embed >> $ grep -c "\=2Eexp" gotools/gotools=2Esum=20 >> 0 >>=20 >> The =2Esum files looks like this: >> ---8<--- >> Test Run By foo on Tue Sep 26 14:46:48 CEST 2023 >> Native configuration is x86_64-foo-linux-gnu >>=20 >> =3D=3D=3D gotools tests =3D=3D=3D >>=20 >> Running cmd/go >> UNTESTED: TestAccidentalGitCheckout >> PASS: TestAlwaysLinkSysoFiles >> =2E=2E=2E >> UNTESTED: TestParallelTest >> FAIL: TestScript >> =2E=2E=2E >> ---8<--- >>=20 >> May i ask you to have a look, please? >>=20 >> TIA, > >