From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from wout3-smtp.messagingengine.com (wout3-smtp.messagingengine.com [64.147.123.19]) by sourceware.org (Postfix) with ESMTPS id 2BC9D396E455 for ; Tue, 10 Aug 2021 21:15:32 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 2BC9D396E455 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=serhei.io Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=serhei.io Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailout.west.internal (Postfix) with ESMTP id 482493200940; Tue, 10 Aug 2021 17:15:29 -0400 (EDT) Received: from imap21 ([10.202.2.71]) by compute2.internal (MEProxy); Tue, 10 Aug 2021 17:15:29 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=serhei.io; h= mime-version:message-id:in-reply-to:references:date:from:to :subject:content-type; s=fm2; bh=ZDjRXY/IAChL/qbXTC7KDexpfqu6I+B SUc62Iu3G5CM=; b=mG8SxEmngds2gYj9EybLD89TjcmENHCUSsBrLiM4BpVixAN pWkvhjfZtFFDfRzgvvtRi75Zgl+Lc91LNc7AiCeNHYmeB3hZp+7GUqdyqUXJKvqY zu/+mLdd3YbaFlkG4c/1i7xrf6qdO9JlOAwgu9ajpDe64GaM6xzk33JWPmWvXtYh XPfwPwftvCg3dMnjX2jPGdP30ybG8Gw9SIO7Rri3etJGuxsjgHfHsLsmU6ZqEU9D YzQpvvof6WJIBz+N4QoQlEfNFDAtraZ2y0wCSdfze1snXeB07IvCYnRxMVrR9JvP NKi/acBNlnBRoTG7zEfqP1vz2+XVbFBnMJ9dRMw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; bh=ZDjRXY /IAChL/qbXTC7KDexpfqu6I+BSUc62Iu3G5CM=; b=iTaYdGzRw9SOUxKx52Xf32 aaQVVOKkZL9vSEkuJ8i2xOpv09Zy9Hp+HGDreVP+obG1/9xHO+2MEuCKaetwJQT6 ascrUj4htcUkn4FHojSsVjQnFZrmm9EN0aN43MHf8nv2GBaoGOT+NZmiNCgxj+Ub 7QE2dgMHtr4frBiwRyqisfxBluAJsL6dDKm1AfwPESInzVq1nBzZWAsR4uAz78S2 rF7WpIDIlGE5QgnUuIDwGTWmBIAUQvea7TRC4B0ii3YtaHBdmocCkJsN4C6+j20V FlOlCxpEAMXneoEqMV3tCOvIKiZozE5kSh90RLoI+R/GysDrF0ca5s+Behzs+KeA == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvtddrjeelgdduieduucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepofgfggfkjghffffhvffutgesthdtredtreertdenucfhrhhomhepfdfuvghr hhgvihcuofgrkhgrrhhovhdfuceomhgvsehsvghrhhgvihdrihhoqeenucggtffrrghtth gvrhhnpeelveffgfeiuedtgfevffefffeuleejhfdvgfetvddvtdeglefhhfejkeetudeg hfenucffohhmrghinhepshhouhhrtggvfigrrhgvrdhorhhgpdhsvghrhhgvihdrihhone cuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepmhgvsehs vghrhhgvihdrihho X-ME-Proxy: Received: by mailuser.nyi.internal (Postfix, from userid 501) id 5FA8E51C0060; Tue, 10 Aug 2021 17:15:28 -0400 (EDT) X-Mailer: MessagingEngine.com Webmail Interface User-Agent: Cyrus-JMAP/3.5.0-alpha0-552-g2afffd2709-fm-20210805.001-g2afffd27 Mime-Version: 1.0 Message-Id: In-Reply-To: References: Date: Tue, 10 Aug 2021 17:15:02 -0400 From: "Serhei Makarov" To: "Keith Seitz" , Bunsen Subject: Re: Unusual test_outcome_map keys Content-Type: text/plain X-Spam-Status: No, score=-3.5 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, JMQ_SPF_NEUTRAL, RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: bunsen@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Bunsen mailing list List-Unsubscribe: , List-Archive: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Aug 2021 21:15:42 -0000 On Fri, Aug 6, 2021, at 3:35 PM, Keith Seitz via Bunsen wrote: > Hi, > > While working on fixing up the gdb.sum parser, I've encountered > `test_outcome_map', defined in common/parse_dejagnu.py. > > This is defined: > > test_outcome_map = {'PASS':'PASS', 'XPASS':'XPASS', 'IPASS':'IPASS', > 'FAIL':'FAIL', 'KFAIL':'KFAIL', 'XFAIL':'XFAIL', > 'ERROR: tcl error sourcing':'ERROR', > 'UNTESTED':'UNTESTED', 'UNSUPPORTED':'UNSUPPORTED', > 'UNRESOLVED':'UNRESOLVED'} > > Can you fill me in on these two unusual keys? Sorry for the delay, I had to dig up a bit of backstory on this. I thought I saw IPASS in some test results, but I wasn't able to dig up any indication of where, so I may be wrong. It is certainly true that KPASS should be included in addition to KFAIL. I took the liberty of patching accordingly: https://sourceware.org/git/?p=bunsen.git;a=commit;h=763c2037ea61eafdf61df09ef8516278dd1ad12f I also changed 'ERROR: tcl error sourcing':'ERROR' to 'ERROR':'ERROR' although I'm less sure of this part. There may be some SystemTap testcases where the program under observation prints 'ERROR: ' and that shouldn't be treated as an error outcome for the testcase. But that could be a SystemTap-specific nit and I would solve it by using the method you suggest below to edit test_outcome_map for the SystemTap parser only. > What's the preferred method for adding members to this map? > > In GDB, we have a special "DUPLICATE" outcome, and right now my plan > is to simply add definitions for this (and any other additions) to our > own gdb.parse_dejagnu_sum function, e.g., > > test_outcome_map['DUPLICATE'] = 'DUPLICATE' > > I'm not a huge fan of modifying global data like this, but since > Bunsen doesn't run over multiple tools at the same time, this shouldn't > be an issue. No objections to either method for extending the table. In my patch, I opted to just add 'DUPLICATE' to test_outcome_map since it's extremely unlikely to interfere with other testsuites. -- All the best, Serhei http://serhei.io