CI scripts

CI scripts

[Jan Vrany]

Hi Simon,

about a month ago when you helped me with use-after-free fix
you wrote:

> I ran the patch through my CI job, looks good.

I'm wondering whether the CI scripts for your job are available?
I'd like to setup a similar job on my CI to avoid similar problems
in future.

I'm especially interested in "configure ..." and "make check ..." 
commands.

Thanks!

Jan

Re: CI scripts

[Simon Marchi]

On 1/12/23 07:56, Jan Vrany via Gdb wrote:
> Hi Simon,
> 
> about a month ago when you helped me with use-after-free fix
> you wrote:
> 
>> I ran the patch through my CI job, looks good.
> 
> I'm wondering whether the CI scripts for your job are available?
> I'd like to setup a similar job on my CI to avoid similar problems
> in future.
> 
> I'm especially interested in "configure ..." and "make check ..." 
> commands.
> 
> Thanks!
> 
> Jan
> 

Hi Jan,

Yes, it's on Jenkins, the job description (it uses Jenkins Job Builder)
is here:

https://github.com/simark/lttng-ci/blob/master/jobs/binutils-gdb.yaml

The build script is runs is here (which is where the commands you are
intereted in are):

https://github.com/simark/lttng-ci/blob/master/scripts/binutils-gdb/build.sh

The job itself is hosted here:

https://ci.lttng.org/view/GDB/job/binutils-gdb_master_linuxbuild/

It's hosted on my employer's infrastructure, hence why it's on the LTTng
CI.

It was green for a while (I filter out pre-existing known failures /
flaky tests in the testsuite, hoping to fix those little by little).  I
usually try to keep it that way by catching regressions early and
notifying people, but it got broken by various commits during the
holidays, so I have to play catch up now.

Simon

Re: CI scripts

[Jan Vrany]

Hi Simon,

On Thu, 2023-01-12 at 08:55 -0500, Simon Marchi wrote:
>
> On 1/12/23 07:56, Jan Vrany via Gdb wrote:
> > Hi Simon,
> >
> > about a month ago when you helped me with use-after-free fix
> > you wrote:
> >
> > > I ran the patch through my CI job, looks good.
> >
> > I'm wondering whether the CI scripts for your job are available?
> > I'd like to setup a similar job on my CI to avoid similar problems
> > in future.
> >
> > I'm especially interested in "configure ..." and "make check ..."
> > commands.
> >
> > Thanks!
> >
> > Jan
> >
>
> Hi Jan,
>
> Yes, it's on Jenkins, the job description (it uses Jenkins Job Builder)
> is here:
>
> https://github.com/simark/lttng-ci/blob/master/jobs/binutils-gdb.yaml
>
> The build script is runs is here (which is where the commands you are
> intereted in are):
>
> https://github.com/simark/lttng-ci/blob/master/scripts/binutils-gdb/build.sh
>

Perfect, thanks a lot! I managed to setup a job on my CI. I just call your
script to build GDB and run and process test.

There's one little thing - the script uses `--with-guile=guile-2.2`. My build
nodes have no guile-2.2, only `guile` which is an alias for `guile-3.0` so I
have to patch the script to use just `--with-guile`.


> The job itself is hosted here:
>
> https://ci.lttng.org/view/GDB/job/binutils-gdb_master_linuxbuild/

Thanks! Interestingly, my build of 1b1be68b9 (using your script) shows
only 6 failures wheres "your" [1] build shows 72750 failures...

Jan

[1]: https://ci.lttng.org/view/GDB/job/binutils-gdb_master_linuxbuild/1044/

>
> It's hosted on my employer's infrastructure, hence why it's on the LTTng
> CI.
>
> It was green for a while (I filter out pre-existing known failures /
> flaky tests in the testsuite, hoping to fix those little by little).  I
> usually try to keep it that way by catching regressions early and
> notifying people, but it got broken by various commits during the
> holidays, so I have to play catch up now.
>
> Simon

Re: CI scripts

[Simon Marchi]

On 1/18/23 07:09, Jan Vrany via Gdb wrote:
> Hi Simon,
> 
> On Thu, 2023-01-12 at 08:55 -0500, Simon Marchi wrote:
>>
>> On 1/12/23 07:56, Jan Vrany via Gdb wrote:
>>> Hi Simon,
>>>
>>> about a month ago when you helped me with use-after-free fix
>>> you wrote:
>>>
>>>> I ran the patch through my CI job, looks good.
>>>
>>> I'm wondering whether the CI scripts for your job are available?
>>> I'd like to setup a similar job on my CI to avoid similar problems
>>> in future.
>>>
>>> I'm especially interested in "configure ..." and "make check ..."
>>> commands.
>>>
>>> Thanks!
>>>
>>> Jan
>>>
>>
>> Hi Jan,
>>
>> Yes, it's on Jenkins, the job description (it uses Jenkins Job Builder)
>> is here:
>>
>> https://github.com/simark/lttng-ci/blob/master/jobs/binutils-gdb.yaml
>>
>> The build script is runs is here (which is where the commands you are
>> intereted in are):
>>
>> https://github.com/simark/lttng-ci/blob/master/scripts/binutils-gdb/build.sh
>>
> 
> Perfect, thanks a lot! I managed to setup a job on my CI. I just call your
> script to build GDB and run and process test.
> 
> There's one little thing - the script uses `--with-guile=guile-2.2`. My build
> nodes have no guile-2.2, only `guile` which is an alias for `guile-3.0` so I
> have to patch the script to use just `--with-guile`.

Oh, I'll update to just --with-guile and let the system pick the default
guile.

I just noticed I provided you links to my personal fork, which is not up
to date.  The files on the real repo are more up to date:

https://github.com/lttng/lttng-ci/blob/master/scripts/binutils-gdb/build.sh

Notably, this version generates a gdb.fail.sum, which contains just the
tests that caused the build to be considered a failure.  So it's only
FAILs/UNRESOLVEDs/XPASSes/DUPLICATEs, after filtering out the known
pre-existing ones.

>> The job itself is hosted here:
>>
>> https://ci.lttng.org/view/GDB/job/binutils-gdb_master_linuxbuild/
> 
> Thanks! Interestingly, my build of 1b1be68b9 (using your script) shows
> only 6 failures wheres "your" [1] build shows 72750 failures...

I don't know, if you use -D_GLIBCXX_DEBUG (like the script does), you
should see loads of failures in the C++ tests.  I reported the problem
here, and it should go back to normal once Tom provides a fix.

https://inbox.sourceware.org/gdb-patches/871qnt2bob.fsf@tromey.com/T/#mceb9a4e339cd209f7b6d8d361ac5b4523d71635c

I have some local modifications to the result parsing script, which I
need to clean up and push, so maybe there's a bit of difference in how
the results are analyzed.

Simon

Re: CI scripts

[Jan Vrany]

On Wed, 2023-01-18 at 08:36 -0500, Simon Marchi wrote:
>
> On 1/18/23 07:09, Jan Vrany via Gdb wrote:
> > Hi Simon,
> >
> > On Thu, 2023-01-12 at 08:55 -0500, Simon Marchi wrote:
> > >
> > > On 1/12/23 07:56, Jan Vrany via Gdb wrote:
> > > > Hi Simon,
> > > >
> > > > about a month ago when you helped me with use-after-free fix
> > > > you wrote:
> > > >
> > > > > I ran the patch through my CI job, looks good.
> > > >
> > > > I'm wondering whether the CI scripts for your job are available?
> > > > I'd like to setup a similar job on my CI to avoid similar problems
> > > > in future.
> > > >
> > > > I'm especially interested in "configure ..." and "make check ..."
> > > > commands.
> > > >
> > > > Thanks!
> > > >
> > > > Jan
> > > >
> > >
> > > Hi Jan,
> > >
> > > Yes, it's on Jenkins, the job description (it uses Jenkins Job Builder)
> > > is here:
> > >
> > > https://github.com/simark/lttng-ci/blob/master/jobs/binutils-gdb.yaml
> > >
> > > The build script is runs is here (which is where the commands you are
> > > intereted in are):
> > >
> > > https://github.com/simark/lttng-ci/blob/master/scripts/binutils-gdb/build.sh
> > >
> >
> > Perfect, thanks a lot! I managed to setup a job on my CI. I just call your
> > script to build GDB and run and process test.
> >
> > There's one little thing - the script uses `--with-guile=guile-2.2`. My build
> > nodes have no guile-2.2, only `guile` which is an alias for `guile-3.0` so I
> > have to patch the script to use just `--with-guile`.
>
> Oh, I'll update to just --with-guile and let the system pick the default
> guile.

Thanks!

>
> I just noticed I provided you links to my personal fork, which is not up
> to date.  The files on the real repo are more up to date:
>
> https://github.com/lttng/lttng-ci/blob/master/scripts/binutils-gdb/build.sh

Thanks, I have updated my job to get build.sh from this repo.

>
> Notably, this version generates a gdb.fail.sum, which contains just the
> tests that caused the build to be considered a failure.  So it's only
> FAILs/UNRESOLVEDs/XPASSes/DUPLICATEs, after filtering out the known
> pre-existing ones.
>
> > > The job itself is hosted here:
> > >
> > > https://ci.lttng.org/view/GDB/job/binutils-gdb_master_linuxbuild/
> >
> > Thanks! Interestingly, my build of 1b1be68b9 (using your script) shows
> > only 6 failures wheres "your" [1] build shows 72750 failures...
>
> I don't know, if you use -D_GLIBCXX_DEBUG (like the script does), you
> should see loads of failures in the C++ tests.  I reported the problem
> here, and it should go back to normal once Tom provides a fix.
>
> https://inbox.sourceware.org/gdb-patches/871qnt2bob.fsf@tromey.com/T/#mceb9a4e339cd209f7b6d8d361ac5b4523d71635c
>
> I have some local modifications to the result parsing script, which I
> need to clean up and push, so maybe there's a bit of difference in how
> the results are analyzed.

I wrote the job so it uses the exact same script as you're using so yes,
I do use -D_GLIBCXX_DEBUG.

After updating the repo from your fork to upstream I do see loads of failures
as well. Once fixed, it should get back to normal for me too.

Thanks! 

Jan

Re: CI scripts

[Simon Marchi]

> After updating the repo from your fork to upstream I do see loads of failures
> as well. Once fixed, it should get back to normal for me too.

Almost!

Before that I had these failures:

FAIL: gdb.dap/basic-dap.exp: disassemble one instruction success
FAIL: gdb.dap/basic-dap.exp: instructions in disassemble output

They would be fixed by that series:

https://inbox.sourceware.org/gdb-patches/20230106185729.42372-1-simon.marchi@efficios.com/

And often some gdb.threads/detach-step-over.exp internal errors.  Since
gdb.threads/detach-step-over.exp is a new-ish test, I'd like to avoid
marking it as "known fail", and actually fix the problem.  The goal is
for that list to shrink, not grow :).

Simon

Re: CI scripts

[Frank Ch. Eigler]

Simon Marchi via Gdb <gdb@sourceware.org> writes:

> [...]
>> I'm wondering whether the CI scripts for your job are available?
>> I'd like to setup a similar job on my CI to avoid similar problems
>> in future.
>> [...]
> Yes, it's on Jenkins, the job description (it uses Jenkins Job Builder)
> is here:
>
> https://github.com/simark/lttng-ci/blob/master/jobs/binutils-gdb.yaml
> [...]
> https://github.com/simark/lttng-ci/blob/master/scripts/binutils-gdb/build.sh

(I assume y'all are also aware of https://builder.sourceware.org/, which
runs several gdb build/test variants.)

> It was green for a while (I filter out pre-existing known failures /
> flaky tests in the testsuite, hoping to fix those little by little).  [...]

Yeah, that's an interesting problem.  We're also addressing it within
the bunsen testsuite data analysis gadget also running here on
sourceware, but hopefully with no hard-coded "false-results" lists
required.

- FChE