Need Help with nss test failure caught by Arm Pre-Commit CI tester

Need Help with nss test failure caught by Arm Pre-Commit CI tester

[Arjun Shankar]

Hi Maxim,

It appears that my patch series triggers a CI failure on the Arm tester:

FAIL: nss/tst-initgroups1
FAIL: nss/tst-initgroups2

These are files that were moved, and were earlier at:
grp/tst-initgroups1
grp/tst-initgroups2

Here's a log from CI run on the last patch in the series:
https://ci.linaro.org/job/tcwg_glibc_check--master-arm-precommit/786/artifact/artifacts/artifacts.precommit/notify/mail-body.txt

Last patch in the series is here:
https://patchwork.sourceware.org/project/glibc/patch/20231002130150.1497733-17-arjun@redhat.com/

During today's patch review meeting, we were speculating that this
might be an expected containerised test failure that might have been
earlier suppressed by an allow-list in the aarch* testers' CI -- and
that the failure is now seen because the files moved (via this patch
series), therefore needing an allow-list update.

Can you confirm that this is the case? If yes, let's update the
allow-list, perhaps adding to the list before I push my patches so
that the failures aren't triggered if/when the CI runs against master?

Thanks!
-- 
Arjun Shankar
he/him/his

Re: Need Help with nss test failure caught by Arm Pre-Commit CI tester

[Maxim Kuvyrkov]

> On Oct 23, 2023, at 19:56, Arjun Shankar <arjun@redhat.com> wrote:
> 
> Hi Maxim,
> 
> It appears that my patch series triggers a CI failure on the Arm tester:
> 
> FAIL: nss/tst-initgroups1
> FAIL: nss/tst-initgroups2
> 
> These are files that were moved, and were earlier at:
> grp/tst-initgroups1
> grp/tst-initgroups2
> 
> Here's a log from CI run on the last patch in the series:
> https://ci.linaro.org/job/tcwg_glibc_check--master-arm-precommit/786/artifact/artifacts/artifacts.precommit/notify/mail-body.txt
> 
> Last patch in the series is here:
> https://patchwork.sourceware.org/project/glibc/patch/20231002130150.1497733-17-arjun@redhat.com/
> 
> During today's patch review meeting, we were speculating that this
> might be an expected containerised test failure that might have been
> earlier suppressed by an allow-list in the aarch* testers' CI -- and
> that the failure is now seen because the files moved (via this patch
> series), therefore needing an allow-list update.
> 
> Can you confirm that this is the case?

Hi Arjun,

The baseline [expected] fails used in testing are in [1] -- they are linked in the above report.

[1] https://ci.linaro.org/job/tcwg_glibc_check--master-arm-precommit/786/artifact/artifacts/artifacts.precommit/sumfiles/xfails.xfail/*view*/

And, indeed, they show that grp/tst-initgroups[12] were failing before your patch:
===
Running glibc:grp ...
FAIL: grp/tst-initgroups1 
FAIL: grp/tst-initgroups2 
===

So it's not a real regression.

> If yes, let's update the
> allow-list, perhaps adding to the list before I push my patches so
> that the failures aren't triggered if/when the CI runs against master?

The baseline fails are generated automatically by post-commit testing.  When you will merge your patch our CI will detect the new failures as a regression and will email you.  But right after reporting the "regression" the post-commit CI will automatically include the new FAILs in the expected list, and testing will continue as expected.

Does this answer your question?

Kind regards,

--
Maxim Kuvyrkov
https://www.linaro.org

Re: Need Help with nss test failure caught by Arm Pre-Commit CI tester

[Arjun Shankar]

Hi Maxim,

> The baseline [expected] fails used in testing are in [1] -- they are linked in the above report.
>
> [1] https://ci.linaro.org/job/tcwg_glibc_check--master-arm-precommit/786/artifact/artifacts/artifacts.precommit/sumfiles/xfails.xfail/*view*/

> And, indeed, they show that grp/tst-initgroups[12] were failing before your patch:
> ===
> Running glibc:grp ...
> FAIL: grp/tst-initgroups1
> FAIL: grp/tst-initgroups2
> ===

Thanks! I see that now.

> The baseline fails are generated automatically by post-commit testing.  When you will merge your patch our CI will detect the new failures as a regression and will email you.  But right after reporting the "regression" the post-commit CI will automatically include the new FAILs in the expected list, and testing will continue as expected.
>
> Does this answer your question?

Yes it does. Thank you!

-- 
Arjun Shankar
he/him/his