From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by sourceware.org (Postfix) with ESMTP id 39946386F81C for ; Wed, 13 Jan 2021 14:08:18 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 39946386F81C Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-499-au9aLhUEMXKKjxboPUfF5g-1; Wed, 13 Jan 2021 09:08:16 -0500 X-MC-Unique: au9aLhUEMXKKjxboPUfF5g-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 148708145E0; Wed, 13 Jan 2021 14:08:15 +0000 (UTC) Received: from ovpn-112-159.phx2.redhat.com (ovpn-112-159.phx2.redhat.com [10.3.112.159]) by smtp.corp.redhat.com (Postfix) with ESMTP id 745E45D9EF; Wed, 13 Jan 2021 14:08:14 +0000 (UTC) Message-ID: <6e09fa48585819fdbc06988a7a3f58bf22e03f28.camel@redhat.com> Subject: Re: [PATCH] Add pytest for a GCOV test-case From: David Malcolm To: Rainer Orth , Martin =?UTF-8?Q?Li=C5=A1ka?= Cc: Jeff Law , gcc-patches@gcc.gnu.org Date: Wed, 13 Jan 2021 09:08:13 -0500 In-Reply-To: References: <05b7e58dc023fbea33a051a7800a214c838feeb4.camel@redhat.com> <9021ed74-22d4-5f2a-c60f-4e0bc63e48b6@redhat.com> User-Agent: Evolution 3.36.5 (3.36.5-1.fc32) MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-7.7 required=5.0 tests=BAYES_00, DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Jan 2021 14:08:28 -0000 On Wed, 2021-01-13 at 14:38 +0100, Rainer Orth wrote: > Hi Martin, > > > On 1/6/21 12:36 AM, Jeff Law wrote: > > > unresolved "could not find python interpreter $testcase" in > > > run-gcov-pytest if you find the right magic in the output of your > > > spawn. > > > > Achieved that with the updated patch. > > > > Ready for master? > > unfortunately, your patch has a large number of problems: > > * On targets where run-gcov-pytest decides that pytest isn't > available > (incorrectly in some cases), mail-report.log is cluttered with > > UNRESOLVED: could not find Python interpreter and (or) pytest module > for pr98273.C > > I fear you've been misled by David and Jeff here: UNRESOLVED isn't > appropriate for cases like this. Please read the DejaGnu manual > for > the semantics of the various test outcomes. If anything (we often > just silently skip testcases that cannot be run on some target), > use > UNSUPPORTED instead. This may be lack of sleep on my part, but looking back at the emails, I said that this case should be "UNSUPPORTED", but it looks like Martin misread it as "UNRESOLVED", and I didn't correct him. Sorry. > * Besides, the test outcomes are not generic message facilities but > are > supposed to follow a common format: > > : [] > > with the pathname to the test relative to (in this case) > gcc/testsuite. In this case, this might be something like > > UNSUPPORTED: g++.dg/gcov/pr98273.C run-gcov-pytest > > Currently, you don't have the pathname in run-gcov-pytest, though. > > * If we now have an (even optional) dependency on python/pytest, this > (with the exact versions and use) needs to be documented in > install.texi. Agreed. [...snip various other issues...] > > On top of all this, I wonder why you insist on a particular Python > version here: I tried your single testcase and it PASSes just as > well > with Python 2.7!? One reason I'm asking is that Solaris 11.3 > bundles > both Python 2.7 and 3.4, but (unlike Linux and Solaris 11.4) don't > have /usr/bin/python3, just python (which is 2.7), python2.7, and > python3.4. Not that it matters too much, but you should be aware > of > the issue. In particular, given the differences between Python 2 and Python 3 I think it's a good idea to be explicit about the versions of Python that we expect i.e. are such tests coded to the common subset of Python 2 and 3, or just to Python 3? Given that they're intended to be optional, I suggest just Python 3. (I would like to be able to add other Python-based scripts to the DejaGnu suite e.g. to verify JSON output, hence I'd prefer to not have to maintain Python 2 compatibility) [...snip...] Dave