From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR02-AM5-obe.outbound.protection.outlook.com (mail-eopbgr00040.outbound.protection.outlook.com [40.107.0.40]) by sourceware.org (Postfix) with ESMTPS id 70168385AE65; Thu, 9 Jun 2022 09:38:20 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 70168385AE65 ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass; b=DFrso6iik7nsHxR1+W9DaExfX8QDuAugMa1LAAZhjO9utM+Acp/meXRQFiBLNS/fodBIzJbcBFNAehML1zeQCv44jAevH/uGpDcaDB4gmo64SzPYIFlv6IwSFXz5UnrxoY5h84CqvI/2Xa9MBc3l5DUqtmbELy+Bw5XqMoT586GuPESui+H8u8Nobi0bBKdsefgAQmLolcspCHURSp8/u7BVMsKjSpdxIIB+oa5PazJLgRawZxUsTWVY9aMlhwlKEy+E3duKR9Ab6g9hkB/QQjKYtePaoZliRbIBGs8RSYDLv+EgEJDI7+3tRa4klxxSD8rM+FKg/ifhfoNEDK6NBA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=H1UWWvOYXsbeDfwQb8TTlMfXhDg7E8k+Xr68mRmSUmQ=; b=kF50GccX/2FysFIh4FsTHYrkj1Id1LfvAcQpMonIZWPjbo4QBn3bHJDVjyTaIBGyIy1Hl7uXn77JesgOR4Ykfu3cU3b9hXtkEt72ucWaeNfCyovprfYmAkI/VvEHQwruNw6heNB4rzIjzq/uZ+gnP5uTpH7/BccYn2uZKALn2jnWfTTrul/sYn65q6tERwtM4CgwpKGct3EODomELnyobsrlh1gY1xPscrdIfk1q1vtPnKPsyI7a4DUZxXH3JXtrDDv75yCKYRHfm9Wc62WOXcLOypFfIOZqXIJLelt/RklEOV8Ji2hvXt61dMSvv9wrmkfqx5MxwxE1BMJ0QRW7WA== ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is 63.35.35.123) smtp.rcpttodomain=sourceware.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com; arc=pass (0 oda=1 ltdi=1 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com] dmarc=[1,1,header.from=arm.com]) Received: from DU2P250CA0021.EURP250.PROD.OUTLOOK.COM (2603:10a6:10:231::26) by AM6PR08MB4359.eurprd08.prod.outlook.com (2603:10a6:20b:b9::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5332.13; Thu, 9 Jun 2022 09:38:11 +0000 Received: from DBAEUR03FT016.eop-EUR03.prod.protection.outlook.com (2603:10a6:10:231:cafe::ee) by DU2P250CA0021.outlook.office365.com (2603:10a6:10:231::26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5332.12 via Frontend Transport; Thu, 9 Jun 2022 09:38:11 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; pr=C Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by DBAEUR03FT016.mail.protection.outlook.com (100.127.142.204) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5332.12 via Frontend Transport; Thu, 9 Jun 2022 09:38:10 +0000 Received: ("Tessian outbound e40990bc24d7:v120"); Thu, 09 Jun 2022 09:38:10 +0000 X-CheckRecipientChecked: true X-CR-MTA-CID: 61f0d95179af41d2 X-CR-MTA-TID: 64aa7808 Received: from 9015dc7cf0ef.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 7C616E73-81DB-44C2-BA04-6844DDCCA035.1; Thu, 09 Jun 2022 09:38:03 +0000 Received: from EUR05-VI1-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 9015dc7cf0ef.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Thu, 09 Jun 2022 09:38:03 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=D+8svhcQo6qtQIqp+I6iXrexcgejRoN7iZdDYbm85OEKGDyG0SR1mqkDHAsNEpQjEpgo5SRsKPhEg26BCHS1f1Tl5ZEAHPI1r6gFhxNenXYBslxR8bljmN0zYLU90ZRKg3VY/99m6Yw7XoojoDOHVh4iLahQxwBAkehJc5liVQjM2K8nKIzpU4iCBrGSeCJeMpOo55d0HL1lbaqmQ4iNNrrrNSevu9EIUGCR87qLEHj5a8dSsP/zupgulgw9WKhLfXUzeuD2wVfS0JdxSCuOPlHv20NfGDeIApoFfh63HMgSnOvc41IkC8Boedxz2fHajXFEUIqEPTGBHJ1HgdpKOw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=H1UWWvOYXsbeDfwQb8TTlMfXhDg7E8k+Xr68mRmSUmQ=; b=nLPO9Y9iuvANxyvtAssWR8wUP14OhR8Pyxt+QANkOgF4Cwb4oJY2k2Fi8td1HTs3oTjO/OHn1JvcR+5plT7IBjqAJQTx7/VHG7gSL/RPdGIW73I4aOMg+R2LuHjna1nfEQYEOdYcMmdpnj9+2VbFC7lg/kOPXPP5PuZxtcx1R/NTx2QGKz+f7YrjiarfrYPEn5PvAhNpri2H8Y6i6jz/52AB/S/KLIvLOlt0ZYC9uq0PJOYtNa3IVmP+wJa596JhlshSwGNNZ+OdctBY/Jhb2zju4iepfo23N9i+fLRwYhan7l7WAaWUCAHSeXsQ2J3pa6NkbiO9THNCtzsYHdflkw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none Authentication-Results-Original: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; Received: from VI1PR08MB3919.eurprd08.prod.outlook.com (2603:10a6:803:c4::31) by DB7PR08MB2988.eurprd08.prod.outlook.com (2603:10a6:5:18::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5314.13; Thu, 9 Jun 2022 09:38:01 +0000 Received: from VI1PR08MB3919.eurprd08.prod.outlook.com ([fe80::d9c0:539c:a641:5735]) by VI1PR08MB3919.eurprd08.prod.outlook.com ([fe80::d9c0:539c:a641:5735%2]) with mapi id 15.20.5332.013; Thu, 9 Jun 2022 09:38:01 +0000 Message-ID: <2281be8e-8920-7ba5-9e96-ffd713462b9c@arm.com> Date: Thu, 9 Jun 2022 10:37:58 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.9.1 Subject: Re: [builder] gdb_check_step: remove gdb.gdb/selftest.exp Content-Language: en-US To: Mark Wielaard , buildbot@sourceware.org Cc: gdb@sourceware.org References: <20220608185138.675964-1-mark@klomp.org> From: Luis Machado In-Reply-To: <20220608185138.675964-1-mark@klomp.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-ClientProxiedBy: LO4P265CA0167.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:312::12) To VI1PR08MB3919.eurprd08.prod.outlook.com (2603:10a6:803:c4::31) MIME-Version: 1.0 X-MS-Office365-Filtering-Correlation-Id: 766d5ca4-91b8-4343-f340-08da49fbc459 X-MS-TrafficTypeDiagnostic: DB7PR08MB2988:EE_|DBAEUR03FT016:EE_|AM6PR08MB4359:EE_ X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: Xpt0zDKg8LYmUhbgO6ZitwrZnjP33xRrRQSnBGenYUeENME3gq2xov/BiD8O/QvqC+fSp3IO8gWpOhOu2l/KMLUJoK/suHtx1nbKs1t94gxNAhGcyjRMFEydspHW/gC3zcLxfyIbpcR0za+LKjIAE6rgotXRKciPuaPjbS1p1tflBGYS3jiWrV1D92bdc4ouqwN6LmKrlFGYSIIo7vlj++SfBvaD0omwBkHSFw0jHsqbwCZyvj0F4DbjzeTgXwtUEG3efHrwFmBxw2eSy5hsFWCiQb9E0hYFQkNJ0e5u5Iw5OyYiSkO0QxNc4I4399yu/jLsxGCqNBI58ApDy6wGVQiM3CQb9GVvdV099IG8FwM56oa5HH43xLM74ECgHSd8nnpAeqRXIEthtmN6Qo9UJiYiu8Z+JFG+zomumG7+DYnBXNDOjQOaNlwA2SSpdmzT1qWzfBKp7NXjWuTpGoNXCmWB2l6tdpd6//Ug8G5hJ/ACDwmE8IG3RO7Y+nEMNhMUw9gEw7MFdEPmX5mz7x20tjl41yialSThM+rM8GlzMo3gT0M0gy6I/+az3aaN4uE6g8x8Zmolq0vpNttpFWQnLcKTtd7ZL93s9AngRGAB0/wtAQzUU/+J8Xtgp+U9VAAAD4mMtpkf+H+T1LejTKR1UQihcePi/gom3TZrvSDN7WLsKiSxqL30YO+6GTPntpfz3tEy9KQDskOWZuo4uuq4B0R3T5vAx01ETTPAvZzC7GfMR0ZtOk8rzHf8xyInUzMw1c6cp8H/8bE2oxhR120ff+7xoU5f9dqjyjJpLhMJaJmQh4T2xd39SI6nh8+i+bxrvU39N33LL4avcpCCtFOy1pb1/3fk8e1Sm4MKyuahdws= X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:VI1PR08MB3919.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(4636009)(366004)(86362001)(8936002)(508600001)(186003)(36756003)(83380400001)(2906002)(5660300002)(31686004)(316002)(31696002)(38100700002)(4326008)(84970400001)(6666004)(44832011)(6506007)(53546011)(6512007)(26005)(66946007)(66476007)(66556008)(8676002)(966005)(6486002)(2616005)(45980500001)(43740500002); DIR:OUT; SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB7PR08MB2988 Original-Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: DBAEUR03FT016.eop-EUR03.prod.protection.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: 4478c19a-09c6-4bed-a35f-08da49fbbeda X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 7qtMB6cwOPCw86YLXMHVis0CblFbBl17zaTpSg9grp4/o3y70YipiyJ52uQgp0PPG7f8fItEpWOdB/3hsnlFDOy4VHO3NTzPbr0UHIw8YGl1UVcrYc/6YtMv4SOPl1vBze9AOX/w+BBJOwGR6qfa0qkqKbjBR9SIcLVI8O9HicJm+ozGQDkkHvweyjhqiA7wMPQ6BjhYYleL83XZuERAE3CS2YAq68TE23XF56aQHPWm3Z/5GZTu6F1jIpnMDplesrz1TtNs7ov7g5Sl+5729JSlaemUkRWWfBq/7JaRXEGcDa66i+2OrD96b+KCo5nmRg3vFxaIvCp/x4oGqT8yqOBbdV6uBDrQzIqjX/NtwtZSgePyNpyB/dDzVK6y3FWrjny8nj55js70NvlFu7TDt8TZklr5qo+M3NZm+d6zYhhPkxJN9UHN5+91Z6MiOhIhnBS9fI8HpkvRQFc46A1j9Dg7CJc00PuY58deDeaggwl0zEbb0MV1f5sqw4LkFWRspwUFU5R5qXmI6D6wZjgcoWBX9KcFULQVHqTPD1TvNPENCUeoZ4lRb7XHM4BZveVoYQjrEtEm8PsBJV0wOLe9ZXewx1eUkEYXmPMbtvcZL47dRe+l/2Q0Dkpaxc7MGLRpLEIUYvr7bfufqNGG9jKDdyInD50oz26T+9As0xlvYGc5PoCyUYcCO0OICQ4nnIA25v/Hw9hJwaLYM/DSOdz0khER7fOStM3RTEhumtfDCTmxlXk29vg5QDVwl9w5L04ok2TvLNW4c6sEbdQFpJtgNLdqqlVK6zNmPbB1nw69Ivk= X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(13230001)(4636009)(46966006)(40470700004)(36840700001)(86362001)(6506007)(6512007)(31696002)(6666004)(26005)(2906002)(53546011)(44832011)(84970400001)(356005)(81166007)(6486002)(966005)(508600001)(36860700001)(82310400005)(83380400001)(70586007)(8936002)(2616005)(186003)(40460700003)(336012)(47076005)(450100002)(36756003)(5660300002)(8676002)(31686004)(4326008)(316002)(70206006)(43740500002); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Jun 2022 09:38:10.6194 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 766d5ca4-91b8-4343-f340-08da49fbc459 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: DBAEUR03FT016.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB4359 X-Spam-Status: No, score=-13.2 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, FORGED_SPF_HELO, GIT_PATCH_0, KAM_DMARC_NONE, NICE_REPLY_A, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_NONE, TXREP, T_SCC_BODY_TEXT_LINE, UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gdb@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gdb mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Jun 2022 09:38:22 -0000 On 6/8/22 19:51, Mark Wielaard wrote: > Hi, > > I just committed the following update to builder: > > selftest.exp is bad (flaky), just keep the gdb and gdbserver > unittest.exp for the CI runs. > > builder.sourceware.org does two builds for gdb. quick CI builds that > just build gdb and gdb-server and run a small amount of tests that are > known to reliably PASS. If any of this fails it will sent email to the > gdb-testers mailinglist and the author of the commit. > https://builder.sourceware.org/buildbot/#/builders?tags=gdb > > It also does full builds for binutils-gdb as a whole plus the full > testsuite both against gcc and clang. These do record all logs in > bunsendb but doesn't sent emails: > https://builder.sourceware.org/buildbot/#/builders?tags=binutils-gdb > > With the removal of the selftest.exp there are only two unittest.exp > tests (one for gdb and one for gdbserver). This does seem very > minimal. Could someone suggest stable, known good passing tests that > should be added because when they do start failing it really is a > regression that should be flagged? > > Note that the intention is to also allow a trybot that runs the same > set of CI tests so it can be used as pre-commit check. > > Cheers, > > Mark > > P.S. There were same bad failing emails today. Our apologies, that was > a bunsen upload typo. > > --- > builder/master.cfg | 1 - > 1 file changed, 1 deletion(-) > > diff --git a/builder/master.cfg b/builder/master.cfg > index 852322f..af4f65f 100644 > --- a/builder/master.cfg > +++ b/builder/master.cfg > @@ -1446,7 +1446,6 @@ gdb_check_step = steps.Test( > util.Interpolate('-j%(prop:ncpus)s'), > 'check-gdb', > ("TESTS= " > - "gdb.gdb/selftest.exp " > "gdb.gdb/unittest.exp " > "gdb.server/unittest.exp ")], > name='make check-gdb', I always use gdb.base/break.exp as a good smoke test. If that one fails, then things are really broken. I think gdb.base/break*.exp should make a good smoke test list. We just need to exclude gdb.base/break-interp.exp, which is problematic on some targets. If you also want to exercise gdbserver, then run with native-gdbserver and native-extended-gdbserver boards. For example: make check-gdb TESTS="gdb.base/break*.exp" RUNTESTFLAGS="--target_board=native-gdbserver" -j$(nproc).