Arm binutils buildbot workers

Arm binutils buildbot workers

[Christophe Lyon]

Hi,

After the GDB builbots, we'd like to add binutils workers for arm64 and 
armhf.  They will be running in Ubuntu containers like the GDB ones:
https://builder.sourceware.org/buildbot/#/builders?tags=gdb

At this stage, I'm not sure whether there's interest in having both 
ubuntu-20.04 and 22.04?

Other question related to binutils: I've noticed there are several 
buildbot flavors, with and without --enable-targets=all.  The machine is 
powerful so we can probably offer to use --enable-targets=all.   Does it 
make sense?

What's your opinion?  Or maybe it would be a better use of the resource 
as a try-bot?

Thanks,

Christophe

Re: Arm binutils buildbot workers

[Mark Wielaard]

Hi Christophe,

On Mon, Jul 11, 2022 at 12:54:53PM +0200, Christophe Lyon via Binutils wrote:
> After the GDB builbots, we'd like to add binutils workers for arm64 and
> armhf.  They will be running in Ubuntu containers like the GDB ones:
> https://builder.sourceware.org/buildbot/#/builders?tags=gdb
> 
> At this stage, I'm not sure whether there's interest in having both
> ubuntu-20.04 and 22.04?

If ubuntu-20.04 is an LTS release it might be interesting to have. Is
it an LTS release for Arm (I am not an Ubuntu user and don't know if
they officially support something other than x86_64 as LTS release)?
If not I think just having the latest is better.

> Other question related to binutils: I've noticed there are several buildbot
> flavors, with and without --enable-targets=all.  The machine is powerful so
> we can probably offer to use --enable-targets=all.   Does it make sense?

Yes for 64bit, unclear for 32bit systems. As far as I understand for
32bit systems --enable-targets=all is broken, but only for sim. So we
might have to configure with --enable-targets=all --disable-sim

> What's your opinion?  Or maybe it would be a better use of the resource as a
> try-bot?

Could it be both? The idea behind to user try branches is that you can
run the buildbot builders as if doing a "real" build. There are not
many people using the try branches at the moment, so it isn't really
that much more work (also the configuration is simply adding an
identifical builder to the try-scheduler).

Cheers,

Mark

Re: Arm binutils buildbot workers

[Richard Earnshaw]

On 17/07/2022 16:09, Mark Wielaard wrote:
> If ubuntu-20.04 is an LTS release it might be interesting to have. Is
> it an LTS release for Arm (I am not an Ubuntu user and don't know if
> they officially support something other than x86_64 as LTS release)?
> If not I think just having the latest is better.

For Ubuntu, all .04 releases in even years are LTS releases.  Arm (and 
aarch64) have been supported for several years now.

Re: Arm binutils buildbot workers

[Christophe Lyon]

On 7/17/22 17:09, Mark Wielaard wrote:
> Hi Christophe,
> 
> On Mon, Jul 11, 2022 at 12:54:53PM +0200, Christophe Lyon via Binutils wrote:
>> After the GDB builbots, we'd like to add binutils workers for arm64 and
>> armhf.  They will be running in Ubuntu containers like the GDB ones:
>> https://builder.sourceware.org/buildbot/#/builders?tags=gdb
>>
>> At this stage, I'm not sure whether there's interest in having both
>> ubuntu-20.04 and 22.04?
> 
> If ubuntu-20.04 is an LTS release it might be interesting to have. Is
> it an LTS release for Arm (I am not an Ubuntu user and don't know if
> they officially support something other than x86_64 as LTS release)?
> If not I think just having the latest is better.
> 
As Richard said, both are LTS and support aarch64 and armhf targets.
Let's use 22.04.

>> Other question related to binutils: I've noticed there are several buildbot
>> flavors, with and without --enable-targets=all.  The machine is powerful so
>> we can probably offer to use --enable-targets=all.   Does it make sense?
> 
> Yes for 64bit, unclear for 32bit systems. As far as I understand for
> 32bit systems --enable-targets=all is broken, but only for sim. So we
> might have to configure with --enable-targets=all --disable-sim
> 

I've just tried to use the configure/make options I found in master.cfg 
for binutils_factory_target_all:
configure --enable-gold --enable-shared --enable-targets=all
make all-gas all-ld all-binutils all-gold
make check-gas check-ld check-binutils

which completes on both 32 and 64 bit systems (aarch64 and armhf).
There are some FAILs in the results, though, but no problem with sim. I 
think it is a problem with gdb only?

>> What's your opinion?  Or maybe it would be a better use of the resource as a
>> try-bot?
> 
> Could it be both? The idea behind to user try branches is that you can
> run the buildbot builders as if doing a "real" build. There are not
> many people using the try branches at the moment, so it isn't really
> that much more work (also the configuration is simply adding an
> identifical builder to the try-scheduler).
> 
Sure. IIUC, it's just a matter of an additional *_try_*_builder entry in 
master.cfg, and they can share the same worker?

Thanks,

Christophe


> Cheers,
> 
> Mark
> 

Re: Arm binutils buildbot workers

[Mark Wielaard]

Hi Christophe,

On Mon, Jul 18, 2022 at 06:39:42PM +0200, Christophe Lyon wrote:
> I've just tried to use the configure/make options I found in master.cfg for
> binutils_factory_target_all:
> configure --enable-gold --enable-shared --enable-targets=all
> make all-gas all-ld all-binutils all-gold
> make check-gas check-ld check-binutils
> 
> which completes on both 32 and 64 bit systems (aarch64 and armhf).
> There are some FAILs in the results, though, but no problem with sim. I
> think it is a problem with gdb only?

OK great. I guess it is not a problem for just building
binutils/gas/ld because that doesn't drag in sim. But when building
gdb it might also build sim?

> > Could it be both? The idea behind to user try branches is that you can
> > run the buildbot builders as if doing a "real" build. There are not
> > many people using the try branches at the moment, so it isn't really
> > that much more work (also the configuration is simply adding an
> > identifical builder to the try-scheduler).
> > 
> Sure. IIUC, it's just a matter of an additional *_try_*_builder entry in
> master.cfg, and they can share the same worker?

Yes. The try-builder get triggered by a different schedule (one that
listens to the git users/hacker/try- branches). But uses the same
buildfactory and the same worker as the normal builder.

The worker can also be the same as the workers now used for gdb. So
you don't need to create new containers unless you want to.

Cheers,

Mark

Re: Arm binutils buildbot workers

[Christophe Lyon]

On 7/19/22 23:29, Mark Wielaard wrote:
> Hi Christophe,
> 
> On Mon, Jul 18, 2022 at 06:39:42PM +0200, Christophe Lyon wrote:
>> I've just tried to use the configure/make options I found in master.cfg for
>> binutils_factory_target_all:
>> configure --enable-gold --enable-shared --enable-targets=all
>> make all-gas all-ld all-binutils all-gold
>> make check-gas check-ld check-binutils
>>
>> which completes on both 32 and 64 bit systems (aarch64 and armhf).
>> There are some FAILs in the results, though, but no problem with sim. I
>> think it is a problem with gdb only?
> 
> OK great. I guess it is not a problem for just building
> binutils/gas/ld because that doesn't drag in sim. But when building
> gdb it might also build sim?
Exactly.


>>> Could it be both? The idea behind to user try branches is that you can
>>> run the buildbot builders as if doing a "real" build. There are not
>>> many people using the try branches at the moment, so it isn't really
>>> that much more work (also the configuration is simply adding an
>>> identifical builder to the try-scheduler).
>>>
>> Sure. IIUC, it's just a matter of an additional *_try_*_builder entry in
>> master.cfg, and they can share the same worker?
> 
> Yes. The try-builder get triggered by a different schedule (one that
> listens to the git users/hacker/try- branches). But uses the same
> buildfactory and the same worker as the normal builder.
Nice.

> The worker can also be the same as the workers now used for gdb. So
> you don't need to create new containers unless you want to.
OK, so for instance, I could re-start the existing containers, giving 
them more CPUs (say 8), keep ncpus/maxcpus to 4 in the workers 
definitions, but increase max_builds to 2, so that for instance we can 
run binutils and gdb jobs on the same worker at the same time?

Do binutils and gdb cohabit well when it comes to testing them in 
parallel?  I guess it should be the case, but in practice gdb has proven 
to have many tests with random results :-(


> 
> Cheers,
> 
> Mark

Re: Arm binutils buildbot workers

[Luis Machado]

On 7/20/22 08:32, Christophe Lyon via Binutils wrote:
> 
> 
> On 7/19/22 23:29, Mark Wielaard wrote:
>> Hi Christophe,
>>
>> On Mon, Jul 18, 2022 at 06:39:42PM +0200, Christophe Lyon wrote:
>>> I've just tried to use the configure/make options I found in master.cfg for
>>> binutils_factory_target_all:
>>> configure --enable-gold --enable-shared --enable-targets=all
>>> make all-gas all-ld all-binutils all-gold
>>> make check-gas check-ld check-binutils
>>>
>>> which completes on both 32 and 64 bit systems (aarch64 and armhf).
>>> There are some FAILs in the results, though, but no problem with sim. I
>>> think it is a problem with gdb only?
>>
>> OK great. I guess it is not a problem for just building
>> binutils/gas/ld because that doesn't drag in sim. But when building
>> gdb it might also build sim?
> Exactly.
> 
> 
>>>> Could it be both? The idea behind to user try branches is that you can
>>>> run the buildbot builders as if doing a "real" build. There are not
>>>> many people using the try branches at the moment, so it isn't really
>>>> that much more work (also the configuration is simply adding an
>>>> identifical builder to the try-scheduler).
>>>>
>>> Sure. IIUC, it's just a matter of an additional *_try_*_builder entry in
>>> master.cfg, and they can share the same worker?
>>
>> Yes. The try-builder get triggered by a different schedule (one that
>> listens to the git users/hacker/try- branches). But uses the same
>> buildfactory and the same worker as the normal builder.
> Nice.
> 
>> The worker can also be the same as the workers now used for gdb. So
>> you don't need to create new containers unless you want to.
> OK, so for instance, I could re-start the existing containers, giving them more CPUs (say 8), keep ncpus/maxcpus to 4 in the workers definitions, but increase max_builds to 2, so that for instance we can run binutils and gdb jobs on the same worker at the same time?
> 
> Do binutils and gdb cohabit well when it comes to testing them in parallel?  I guess it should be the case, but in practice gdb has proven to have many tests with random results :-(

That should work fine. Most of the instability in GDB is due to timeouts/output ordering issues. It shouldn't get worse if you're running binutils tests in parallel.