From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx07-00178001.pphosted.com (mx07-00178001.pphosted.com [185.132.182.106]) by sourceware.org (Postfix) with ESMTPS id BDC633838020 for ; Thu, 1 Jul 2021 11:37:37 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org BDC633838020 Received: from pps.filterd (m0241204.ppops.net [127.0.0.1]) by mx07-00178001.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 161Baf6K023492; Thu, 1 Jul 2021 13:37:36 +0200 Received: from beta.dmz-eu.st.com (beta.dmz-eu.st.com [164.129.1.35]) by mx07-00178001.pphosted.com with ESMTP id 39h1xqbu96-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 01 Jul 2021 13:37:36 +0200 Received: from euls16034.sgp.st.com (euls16034.sgp.st.com [10.75.44.20]) by beta.dmz-eu.st.com (STMicroelectronics) with ESMTP id 6333A10002A; Thu, 1 Jul 2021 13:37:35 +0200 (CEST) Received: from Webmail-eu.st.com (sfhdag2node3.st.com [10.75.127.6]) by euls16034.sgp.st.com (STMicroelectronics) with ESMTP id 5404E21ED4E; Thu, 1 Jul 2021 13:37:35 +0200 (CEST) Received: from [10.211.11.200] (10.75.127.48) by SFHDAG2NODE3.st.com (10.75.127.6) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 1 Jul 2021 13:37:33 +0200 Subject: Re: [RFC] Proposal for hosting GDB CI builds To: Luis Machado , "gdb@sourceware.org" References: <7bfae273-3887-30c8-dc65-94d5b177db56@linaro.org> From: Christophe LYON Message-ID: <585368bc-4365-d8de-cc40-9bb2dcae94e6@foss.st.com> Date: Thu, 1 Jul 2021 13:37:29 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.11.0 MIME-Version: 1.0 In-Reply-To: <7bfae273-3887-30c8-dc65-94d5b177db56@linaro.org> Content-Type: text/plain; charset="utf-8"; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US X-Originating-IP: [10.75.127.48] X-ClientProxiedBy: SFHDAG1NODE2.st.com (10.75.127.2) To SFHDAG2NODE3.st.com (10.75.127.6) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391, 18.0.790 definitions=2021-07-01_07:2021-06-30, 2021-07-01 signatures=0 X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, NICE_REPLY_A, RCVD_IN_DNSWL_LOW, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: gdb@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gdb mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Jul 2021 11:37:39 -0000 Hi Luis, On 30/06/2021 18:46, Luis Machado via Gdb wrote: > Hi, > > This proposal comes as an attempt to revive the GDB CI builds, given > the previous one (buildbot) is no longer being maintained by Sergio > (thanks a lot for keeping it working for all these years by the way). > > CI GDB builds are a great help for spotting regressions without having > to do the tedious and time-consuming work of running the GDB testsuite > for each git revision, for each architecture and reading through > hundreds of lines of summaries. If a regression is spotted, then one > needs to bisect to find the culprit. This isn't great, specially for > architectures without great availability of desktop hardware. > > Ideally, for each commit, we'd run full builds to validate the state > of the tree, but we're not quite there yet. So meanwhile, having some > level of automation to get the builds done without manual intervention > sounds like a reasonable step forward. > > From previous IRC conversations, it seems to be a consensus that > availability of processing power is not a problem. It is reasonably > easy to find hardware to do some builds. The most difficult resource > to find is manpower to setup the CI infrastructure and keep it running. > > With the above said, I've discussed this internally at Linaro and we > can spare some manpower to setup and maintain an isolated > Linaro-hosted Jenkins instance for GDB CI builds. As you could guess, you have a big +1 from me ;-) > > Linaro can take care of providing builders and build jobs for ARM. > Other architectures would be handled by their respective contributors. > Those contributors can write jobs and plug builders as needed. > > Setting up new jobs doesn't require the use of the web interface. It > can be done with yaml files in a git repo. It is reasonably simple. > > You can see an example of the Linaro CI here: https://ci.linaro.org/ > > Also, a GDB job for aarch64: https://ci.linaro.org/job/tcwg-gdb/ > > And also the summaries for GDB testsuite runs: > https://ci.linaro.org/job/tcwg-compare-results/13968/artifact/artifacts/logs/0-report.html > > Of course, this effort only makes sense if the community is OK with > using Jenkins as the CI mechanism and if it actually sees value in > having a system like this in place. > > Thoughts? Questions? From the reports you shared above, it seems there are lots of tests with "random" results (or rather lots of differences between two runs). My fear is that reports will soon get ignored if there are too many false regressions. But I hope that making such variations public will help fix the root cause. Thanks, Christophe