Re: [Linaro-TCWG-CI] glibc patch #75959: FAIL: 1 regressions

Re: [Linaro-TCWG-CI] glibc patch #75959: FAIL: 1 regressions

[Siddhesh Poyarekar]

On 2023-09-14 16:32, ci_notify@linaro.org wrote:
> Dear contributor, our automatic CI has detected problems related to your patch(es).  Please find some details below.  If you have any questions, please follow up on linaro-toolchain@lists.linaro.org mailing list, Libera's #linaro-tcwg channel, or ping your favourite Linaro toolchain developer on the usual project channel.
> 
> In CI config tcwg_glibc_check/master-arm after:
> 
>    | glibc patch https://patchwork.sourceware.org/patch/75959
>    | Author: Siddhesh Poyarekar <siddhesh@sourceware.org>
>    | Date:   Thu Sep 14 06:13:02 2023 -0400
>    |
>    |     getaddrinfo: Fix use after free in getcanonname (CVE-2023-4806)
>    |
>    |     When an NSS plugin only implements the _gethostbyname2_r and
>    |     _getcanonname_r callbacks, getaddrinfo could use memory that was freed
>    |     during tmpbuf resizing, through h_name in a previous query response.
>    |     Fix this by copying h_name over and freeing it at the end.
>    |
>    | ... 3 lines of the commit log omitted.
>    | ... applied on top of baseline commit:
>    | 803f4073cc Add MOVE_MOUNT_BENEATH from Linux 6.5 to sys/mount.h
> 
> FAIL: 1 regressions
> 
> regressions.sum:
> 		=== glibc tests ===
> 
> Running glibc:nss ...
> FAIL: nss/tst-nss-gai-hv2-canonname
> 
> 		=== Results Summary ===
> 
> You can find the failure logs in *.log.1.xz files in
>   - https://ci.linaro.org/job/tcwg_glibc_check--master-arm-precommit/703/artifact/artifacts/artifacts.precommit/00-sumfiles/ .
> The full lists of regressions and progressions are in
>   - https://ci.linaro.org/job/tcwg_glibc_check--master-arm-precommit/703/artifact/artifacts/artifacts.precommit/notify/ .
> The list of [ignored] baseline and flaky failures are in
>   - https://ci.linaro.org/job/tcwg_glibc_check--master-arm-precommit/703/artifact/artifacts/artifacts.precommit/sumfiles/xfails.xfail .
> 
> 
> 
> -----------------8<--------------------------8<--------------------------8<--------------------------
> The information below can be used to reproduce a debug environment:
> 
> Current build   : https://ci.linaro.org/job/tcwg_glibc_check--master-arm-precommit/703/artifact/artifacts
> Reference build : https://ci.linaro.org/job/tcwg_glibc_check--master-arm-build/612/artifact/artifacts

Hello,

I'm looking at the logs and all it has is:

original exit status 127
running post-clean rsync

for the new test.  It looks like other NSS tests also fail in the same 
way.  Is this a known issue on arm?

Thanks,
Sid

Re: [Linaro-TCWG-CI] glibc patch #75959: FAIL: 1 regressions

[Andreas Schwab]

On Sep 14 2023, Siddhesh Poyarekar wrote:

> I'm looking at the logs and all it has is:
>
> original exit status 127
> running post-clean rsync
>
> for the new test.

127 is command not found in the shell.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

Re: [Linaro-TCWG-CI] glibc patch #75959: FAIL: 1 regressions

[Maxim Kuvyrkov]

Hi All,

As Siddhesh has pointed out, 32-bit ARM has other similar nss/* tests failing with "original exit status 127".  We do not have these failures for 64-bit AArch64.

There seems to be a problem unrelated to Siddhesh's patch.
Adhemerval, would you please investigate what's causing it?  I see plenty of fails in AArch32 results that do not appear in AArch64 results.  See the non-flaky entries at the bottom of [1] and [2].

For background, both aarch64 and aarch32 testing is done in docker containers with very similar-looking rootfs images constructed from the same Dockerfile template [3].  Therefore, it's unlikely that we forgot to include a tool or binary in the aarch32 image.

[1] https://ci.linaro.org/job/tcwg_glibc_check--master-arm-build/lastStableBuild/artifact/artifacts/sumfiles/xfails.xfail/*view*/

[2] https://ci.linaro.org/job/tcwg_glibc_check--master-aarch64-build/lastStableBuild/artifact/artifacts/sumfiles/xfails.xfail/*view*/

[3] https://git.linaro.org/ci/dockerfiles.git/tree/tcwg-base/Dockerfile.in

--
Maxim Kuvyrkov
https://www.linaro.org

> On Sep 15, 2023, at 10:16, Andreas Schwab <schwab@linux-m68k.org> wrote:
> 
> On Sep 14 2023, Siddhesh Poyarekar wrote:
> 
>> I'm looking at the logs and all it has is:
>> 
>> original exit status 127
>> running post-clean rsync
>> 
>> for the new test.
> 
> 127 is command not found in the shell.
> 
> -- 
> Andreas Schwab, schwab@linux-m68k.org
> GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
> "And now for something completely different."

Re: [Linaro-TCWG-CI] glibc patch #75959: FAIL: 1 regressions

[Andreas Schwab]

On Sep 15 2023, Maxim Kuvyrkov wrote:

> For background, both aarch64 and aarch32 testing is done in docker containers with very similar-looking rootfs images constructed from the same Dockerfile template [3].  Therefore, it's unlikely that we forgot to include a tool or binary in the aarch32 image.

But it could be of the wrong architecture.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

Re: [Linaro-TCWG-CI] glibc patch #75959: FAIL: 1 regressions

[Carlos O'Donell]

On 9/14/23 17:04, Siddhesh Poyarekar wrote:
> 
> 
> On 2023-09-14 16:32, ci_notify@linaro.org wrote:
>> Dear contributor, our automatic CI has detected problems related to your patch(es).  Please find some details below.  If you have any questions, please follow up on linaro-toolchain@lists.linaro.org mailing list, Libera's #linaro-tcwg channel, or ping your favourite Linaro toolchain developer on the usual project channel.
>>
>> In CI config tcwg_glibc_check/master-arm after:
>>
>>    | glibc patch https://patchwork.sourceware.org/patch/75959
>>    | Author: Siddhesh Poyarekar <siddhesh@sourceware.org>
>>    | Date:   Thu Sep 14 06:13:02 2023 -0400
>>    |
>>    |     getaddrinfo: Fix use after free in getcanonname (CVE-2023-4806)
>>    |
>>    |     When an NSS plugin only implements the _gethostbyname2_r and
>>    |     _getcanonname_r callbacks, getaddrinfo could use memory that was freed
>>    |     during tmpbuf resizing, through h_name in a previous query response.
>>    |     Fix this by copying h_name over and freeing it at the end.
>>    |
>>    | ... 3 lines of the commit log omitted.
>>    | ... applied on top of baseline commit:
>>    | 803f4073cc Add MOVE_MOUNT_BENEATH from Linux 6.5 to sys/mount.h
>>
>> FAIL: 1 regressions
>>
>> regressions.sum:
>>         === glibc tests ===
>>
>> Running glibc:nss ...
>> FAIL: nss/tst-nss-gai-hv2-canonname
>>
>>         === Results Summary ===
>>
>> You can find the failure logs in *.log.1.xz files in
>>   - https://ci.linaro.org/job/tcwg_glibc_check--master-arm-precommit/703/artifact/artifacts/artifacts.precommit/00-sumfiles/ .
>> The full lists of regressions and progressions are in
>>   - https://ci.linaro.org/job/tcwg_glibc_check--master-arm-precommit/703/artifact/artifacts/artifacts.precommit/notify/ .
>> The list of [ignored] baseline and flaky failures are in
>>   - https://ci.linaro.org/job/tcwg_glibc_check--master-arm-precommit/703/artifact/artifacts/artifacts.precommit/sumfiles/xfails.xfail .
>>
>>
>>
>> -----------------8<--------------------------8<--------------------------8<--------------------------
>> The information below can be used to reproduce a debug environment:
>>
>> Current build   : https://ci.linaro.org/job/tcwg_glibc_check--master-arm-precommit/703/artifact/artifacts
>> Reference build : https://ci.linaro.org/job/tcwg_glibc_check--master-arm-build/612/artifact/artifacts
> 
> Hello,
> 
> I'm looking at the logs and all it has is:
> 
> original exit status 127
> running post-clean rsync
> 
> for the new test.  It looks like other NSS tests also fail in the same way.  Is this a known issue on arm?

I saw this too when I was doing the review for the patch.

And what I noticed was this:

Running glibc:nss ...
FAIL: nss/tst-nss-compat1 
FAIL: nss/tst-nss-db-endgrent 
FAIL: nss/tst-nss-db-endpwent 
FAIL: nss/tst-nss-files-hosts-getent 
FAIL: nss/tst-nss-files-hosts-long 
FAIL: nss/tst-nss-files-hosts-v4mapped 
FAIL: nss/tst-nss-gai-actions 
FAIL: nss/tst-nss-test3 
FAIL: nss/tst-reload1 
FAIL: nss/tst-reload2 

Which means...

 75 tests-container := \
 76   tst-nss-compat1 \
 77   tst-nss-db-endgrent \
 78   tst-nss-db-endpwent \
 79   tst-nss-files-hosts-long \
 80   tst-nss-files-hosts-v4mapped \
 81   tst-nss-gai-actions \
 82   tst-nss-test3 \
 83   tst-reload1 \
 84   tst-reload2 \
 85 # tests-container

... all the container tests fail.

Note that tst-nss-files-hosts-getent is *not* a container test,
but it does use support_become_root() and xchroot() which means
it is one of the containerized tests using variant test infrastructure.

So this looked like either an infrastructure issue or permission
issues with running those tests.

I did my own testing on x86_64 and i686 and it was clean.

-- 
Cheers,
Carlos.