From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pg1-x52b.google.com (mail-pg1-x52b.google.com [IPv6:2607:f8b0:4864:20::52b]) by sourceware.org (Postfix) with ESMTPS id D44ED383582E for ; Thu, 1 Jul 2021 11:50:10 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org D44ED383582E Received: by mail-pg1-x52b.google.com with SMTP id a2so5827791pgi.6 for ; Thu, 01 Jul 2021 04:50:10 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=dSMTOi0E+9JDgow4v9u+ks2JRUz8ZHpzZQh2siivExg=; b=SbJNvxPjdrcesadH4W72GsRQR/XSUsbeS9/CW/0To54RKMglfU8QxMETBJz4/xzcy1 UrrO6PZlDGJIyzJugOC+T8u1bKpIIPFHJgYPqA2ivdEOUVvMh4n/HQTeeiTutY5oBnD3 6fXFpuYaCaXi6P63vhHvJqqXMw1a3E6H3NTWVRVF0NToKuOGFHTyTNnyL+c4d5agXNnq xBtzcd34tIxDAJfevNG2a1iHJpODs3BUU4R5hLvm2NAhP+UAHmXftDHiFQ8gDcMvsOFF VpwYgviI1AczJWNhCb8mmT7MI5gZ7w/AyUX1nYv1dWPplszUXZUJ+gpwDJ5sPg0CGy9s jIbQ== X-Gm-Message-State: AOAM530vTtjzjsklbvV69f/sinqw2SLDJLsD4xwVBrnT7Mba0bdrftmY 4T22uDgZKL7HTRUM8shuuuBIcqVpxhNFTg== X-Google-Smtp-Source: ABdhPJwchiRewTFiPQ6d+6VjH8yyf7PpI28gaZxSlMivUvrPOIWhNKmKL3AxGYr9DfXXKauRZ4+rAg== X-Received: by 2002:a62:788b:0:b029:300:21b3:d630 with SMTP id t133-20020a62788b0000b029030021b3d630mr41475554pfc.77.1625140210006; Thu, 01 Jul 2021 04:50:10 -0700 (PDT) Received: from ?IPv6:2804:7f0:4841:1e6a:d86b:249c:eb1d:ab69? ([2804:7f0:4841:1e6a:d86b:249c:eb1d:ab69]) by smtp.gmail.com with ESMTPSA id b18sm9711544pjq.2.2021.07.01.04.50.08 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 01 Jul 2021 04:50:09 -0700 (PDT) Subject: Re: [RFC] Proposal for hosting GDB CI builds To: Christophe LYON , "gdb@sourceware.org" References: <7bfae273-3887-30c8-dc65-94d5b177db56@linaro.org> <585368bc-4365-d8de-cc40-9bb2dcae94e6@foss.st.com> From: Luis Machado Message-ID: Date: Thu, 1 Jul 2021 08:50:07 -0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.11.0 MIME-Version: 1.0 In-Reply-To: <585368bc-4365-d8de-cc40-9bb2dcae94e6@foss.st.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-4.9 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, NICE_REPLY_A, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: gdb@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gdb mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Jul 2021 11:50:13 -0000 Hi Christophe, On 7/1/21 8:37 AM, Christophe LYON wrote: > Hi Luis, > > On 30/06/2021 18:46, Luis Machado via Gdb wrote: >> Hi, >> >> This proposal comes as an attempt to revive the GDB CI builds, given >> the previous one (buildbot) is no longer being maintained by Sergio >> (thanks a lot for keeping it working for all these years by the way). >> >> CI GDB builds are a great help for spotting regressions without having >> to do the tedious and time-consuming work of running the GDB testsuite >> for each git revision, for each architecture and reading through >> hundreds of lines of summaries. If a regression is spotted, then one >> needs to bisect to find the culprit. This isn't great, specially for >> architectures without great availability of desktop hardware. >> >> Ideally, for each commit, we'd run full builds to validate the state >> of the tree, but we're not quite there yet. So meanwhile, having some >> level of automation to get the builds done without manual intervention >> sounds like a reasonable step forward. >> >> From previous IRC conversations, it seems to be a consensus that >> availability of processing power is not a problem. It is reasonably >> easy to find hardware to do some builds. The most difficult resource >> to find is manpower to setup the CI infrastructure and keep it running. >> >> With the above said, I've discussed this internally at Linaro and we >> can spare some manpower to setup and maintain an isolated >> Linaro-hosted Jenkins instance for GDB CI builds. > > > As you could guess, you have a big +1 from me ;-) > > >> >> Linaro can take care of providing builders and build jobs for ARM. >> Other architectures would be handled by their respective contributors. >> Those contributors can write jobs and plug builders as needed. >> >> Setting up new jobs doesn't require the use of the web interface. It >> can be done with yaml files in a git repo. It is reasonably simple. >> >> You can see an example of the Linaro CI here: https://ci.linaro.org/ >> >> Also, a GDB job for aarch64: https://ci.linaro.org/job/tcwg-gdb/ >> >> And also the summaries for GDB testsuite runs: >> https://ci.linaro.org/job/tcwg-compare-results/13968/artifact/artifacts/logs/0-report.html >> >> >> Of course, this effort only makes sense if the community is OK with >> using Jenkins as the CI mechanism and if it actually sees value in >> having a system like this in place. >> >> Thoughts? Questions? > > From the reports you shared above, it seems there are lots of tests > with "random" results (or rather lots of differences between two runs). > My fear is that reports will soon get ignored if there are too many > false regressions. But I hope that making such variations public will > help fix the root cause. Unfortunately that's always been the nature of GDB's testsuite. It has improved over the year, but there are still problematic tests. We will certainly have to put those into an exception list when reporting the summaries, at least at an initial stage. > > Thanks, > > > Christophe > >