Re: Status of the AArch* builders

[Sergio Durigan Junior <sergiodj@redhat.com>]

On Wednesday, October 24 2018, Alan Hayward wrote:

>> On 17 Oct 2018, at 19:45, Sergio Durigan Junior <sergiodj@redhat.com> wrote:
>> 
>>> For now, I have disabled the email notifications.  IME there's always
>>> something that we need to tweak in the first days to make sure that the
>>> builders are running fine.  So if you could just keep an eye on the
>>> builds and make sure that everything is OK, I'd appreciate.
>>> 
>>> After a few days have passed, I will enable the email notifications.
>>> Regular test failures will be sent to the gdb-testers@ ml, and breakages
>>> will be sent to gdb-patches@.
>> 
>> For the record, I've now enabled the e-mail notifications for the
>> Aarch64 builders.  I've also added them to the list of Try builders, so
>> it's possible to submit try jobs to them.
>> 
>
> Currently the AArch64 builders keep going back to failed regressions. This is
> mostly due to *all* the gdb.threads and gdb.server tests being racy on AArch64
> Ubuntu. Keeping it down to 16 threads has reduced the frequency, but not
> removed it.
> I’m currently trying to fix up as many of the AArch64 test failures across the
> whole test suite as I can. I suspect fixing the threaded issue is going to
> take a while longer.

TBH, all builders suffer from this problem.  They all have racy tests,
and even though I tried to implement a system to detect such tests and
exclude them from the reports that are sent to gdb-testers, they still
sneak in the reports.  That's the main (and sole?) reason why
gdb-testers is currently impossible to follow, and people don't really
read it.

> In the meantime is it possible to update just the AArch64 buildbot scripts so
> they don’t take into account the thread/server tests for regressions? Either
> remove the tests before running, or grep them out of the results. Not sure if
> that’s possible with the way it’s set up (and not sure where to look). This
> would then give us confidence with the rest of the tests. They can be
> reinstated when stable again.

I think it can be done, but it's not so trivial, and I'm busy with other
stuff right now :-/.  Sorry about that.

I'll try to take a look at this problem during the weekend.  BTW, the
configuration files for the GDB BuildBot live at:

  https://git.sergiodj.net/gdb-buildbot.git/

Thanks,

-- 
Sergio
GPG key ID: 237A 54B1 0287 28BF 00EF  31F4 D0EB 7628 65FC 5E36
Please send encrypted e-mail if possible
http://sergiodj.net/