proposed patch: gcc-fullest builder to build+check multilib -m32/-m64

proposed patch: gcc-fullest builder to build+check multilib -m32/-m64

[Frank Ch. Eigler]

Hi -

Proposing here instead of pushing because ... i'm not sure quoting is
quite right, so want to give a heads up for some trial & error.

diff --git a/builder/master.cfg b/builder/master.cfg
index cfa7f13ad683..43b893e51b49 100644
--- a/builder/master.cfg
+++ b/builder/master.cfg
@@ -4074,7 +4074,8 @@ c['builders'].append(gcc_fedora_ppc64le_builder)
 #c['builders'].append(gcc_debian_testing_x86_64_builder)
 
 def gcc_full_build_factory_gen(extra_configure_args=None,
-                               extra_setup_step=None):
+                               extra_setup_step=None,
+                               extra_check_args=None):
         gcc_full_build_factory = util.BuildFactory()
         gcc_full_build_factory.addStep(gcc_build_git_step)
         gcc_full_build_factory.addStep(gcc_rm_build_step)
@@ -4100,10 +4101,12 @@ def gcc_full_build_factory_gen(extra_configure_args=None,
                 haltOnFailure=True))
         # We want parallelism to get through this as quickly as possible.
         # Even if that means bunsen gets some parallel/duplicate log files
+        make_check_command = "make check -k -j%(prop:maxcpus)s "
+        if extra_check_args:
+                make_check_command += extra_check_args # NB: just string concat inside the sh -c
         gcc_full_build_factory.addStep(steps.Test(
                 workdir='gcc-build',
-                command=util.Interpolate(
-                        "make check -k -j%(prop:maxcpus)s || true"),
+                command=util.Interpolate(make_check_command + " || true"),
                 name='make check',
                 timeout=7200, # 2 hours
                 haltOnFailure=False, flunkOnFailure=False))
@@ -4128,11 +4131,13 @@ gcc_fullest_debian_amd64_builder = util.BuilderConfig(
                     readContainerFile('debian-stable')},
         workernames=big_vm_workers,
         tags=["gcc-full", "debian", "x86_64"],
-        factory=gcc_full_build_factory_gen([
+        factory=gcc_full_build_factory_gen(extra_configure_args=[
                 "--enable-checking=yes,extra,rtl",
                 "--enable-languages=all",
                 "--enable-host-shared",
-        ]))
+                "--enable-multilib-list=m32,m64" # multilib!
+        ],extra_check_args=r"RUNTESTFLAGS=--target_board=unix\\{,-m32\\}") # NB: quoting
+        )
 c['builders'].append(gcc_fullest_debian_amd64_builder)
 
 gcc_full_debian_amd64_builder = util.BuilderConfig(

Re: proposed patch: gcc-fullest builder to build+check multilib -m32/-m64

[Mark Wielaard]

Hi Frank,

On Fri, Sep 27, 2024 at 01:43:32PM -0400, Frank Ch. Eigler wrote:
> Proposing here instead of pushing because ... i'm not sure quoting is
> quite right, so want to give a heads up for some trial & error.

O, fun. python, tcl and shellstring escapes...

> diff --git a/builder/master.cfg b/builder/master.cfg
> index cfa7f13ad683..43b893e51b49 100644
> --- a/builder/master.cfg
> +++ b/builder/master.cfg
> @@ -4074,7 +4074,8 @@ c['builders'].append(gcc_fedora_ppc64le_builder)
>  #c['builders'].append(gcc_debian_testing_x86_64_builder)
>  
>  def gcc_full_build_factory_gen(extra_configure_args=None,
> -                               extra_setup_step=None):
> +                               extra_setup_step=None,
> +                               extra_check_args=None):
>          gcc_full_build_factory = util.BuildFactory()
>          gcc_full_build_factory.addStep(gcc_build_git_step)
>          gcc_full_build_factory.addStep(gcc_rm_build_step)
> @@ -4100,10 +4101,12 @@ def gcc_full_build_factory_gen(extra_configure_args=None,
>                  haltOnFailure=True))
>          # We want parallelism to get through this as quickly as possible.
>          # Even if that means bunsen gets some parallel/duplicate log files
> +        make_check_command = "make check -k -j%(prop:maxcpus)s "
> +        if extra_check_args:
> +                make_check_command += extra_check_args # NB: just string concat inside the sh -c
>          gcc_full_build_factory.addStep(steps.Test(
>                  workdir='gcc-build',
> -                command=util.Interpolate(
> -                        "make check -k -j%(prop:maxcpus)s || true"),
> +                command=util.Interpolate(make_check_command + " || true"),
>                  name='make check',
>                  timeout=7200, # 2 hours
>                  haltOnFailure=False, flunkOnFailure=False))

OK, this part should work.

> @@ -4128,11 +4131,13 @@ gcc_fullest_debian_amd64_builder = util.BuilderConfig(
>                      readContainerFile('debian-stable')},
>          workernames=big_vm_workers,
>          tags=["gcc-full", "debian", "x86_64"],
> -        factory=gcc_full_build_factory_gen([
> +        factory=gcc_full_build_factory_gen(extra_configure_args=[
>                  "--enable-checking=yes,extra,rtl",
>                  "--enable-languages=all",
>                  "--enable-host-shared",
> -        ]))
> +                "--enable-multilib-list=m32,m64" # multilib!
> +        ],extra_check_args=r"RUNTESTFLAGS=--target_board=unix\\{,-m32\\}") # NB: quoting
> +        )

OK, adding argument names makes sense.  () and [] seems to match.  But
then...  You are using a python "raw string literal". So no escape
sequences nside the " quotes. But still a double \ because one will be
eaten by the shell?

I must admit I don't fully know what you are expecting at the shell
level or how precicely this comes out. Sorry.

Cheers,

Mark

Re: proposed patch: gcc-fullest builder to build+check multilib -m32/-m64

[Frank Ch. Eigler]

Hi -

> OK, adding argument names makes sense.  () and [] seems to match.  But
> then...  You are using a python "raw string literal". So no escape
> sequences nside the " quotes. But still a double \ because one will be
> eaten by the shell?
> 
> I must admit I don't fully know what you are expecting at the shell
> level or how precicely this comes out. Sorry.

The dejagnu code expects to see \{ characters, so python should eat nothing,
the shell forked by python should eat one (\\{ -> \{), leaving \{ for expect.
That's the theory, but I'm not easily able to test this locally. ;)

- FChE

Re: proposed patch: gcc-fullest builder to build+check multilib -m32/-m64

[Mark Wielaard]

On Sun, Sep 29, 2024 at 04:09:08PM -0400, Frank Ch. Eigler wrote:
> Hi -
> 
> > OK, adding argument names makes sense.  () and [] seems to match.  But
> > then...  You are using a python "raw string literal". So no escape
> > sequences nside the " quotes. But still a double \ because one will be
> > eaten by the shell?
> > 
> > I must admit I don't fully know what you are expecting at the shell
> > level or how precicely this comes out. Sorry.
> 
> The dejagnu code expects to see \{ characters, so python should eat nothing,
> the shell forked by python should eat one (\\{ -> \{), leaving \{ for expect.
> That's the theory, but I'm not easily able to test this locally. ;)

Could you try with os.system? Don't know if that us exactly what
happens, but it might help. Something like:

python3 -c 'import os; os.system ("make check -k -j12 " + r"RUNTESTFLAGS=--target_board=unix\\{,-m32\\}" + " || true")'

Cheers,

Mark

Re: proposed patch: gcc-fullest builder to build+check multilib -m32/-m64

[Frank Ch. Eigler]

Hi -

> Could you try with os.system? Don't know if that us exactly what
> happens, but it might help. Something like:
> 
> python3 -c 'import os; os.system ("make check -k -j12 " + r"RUNTESTFLAGS=--target_board=unix\\{,-m32\\}" + " || true")'

Genius!
OK, it works with an additional \, i.e.,   \\\{

- FChE