Re: [Linaro-TCWG-CI] glibc-2.39.9000-340-g176671f604: FAIL: 1 regressions on aarch64

Re: [Linaro-TCWG-CI] glibc-2.39.9000-340-g176671f604: FAIL: 1 regressions on aarch64

[Florian Weimer]

* ci notify:

> In glibc_check master-aarch64 after:
>
>   | commit glibc-2.39.9000-340-g176671f604
>   | Author: Carlos Llamas <cmllamas@google.com>
>   | Date:   Tue Jun 18 10:56:34 2024 +0200
>   | 
>   |     linux: add definitions for hugetlb page size encodings
>   |     
>   |     A desired hugetlb page size can be encoded in the flags parameter of
>   |     system calls such as mmap() and shmget(). The Linux UAPI headers have
>   |     included explicit definitions for these encodings since v4.14.
>   |     
>   |     This patch adds these definitions that are used along with MAP_HUGETLB
>   | ... 14 lines of the commit log omitted.
>
> FAIL: 1 regressions
>
> regressions.sum:
> 		=== glibc tests ===
>
> Running glibc:misc ...
> FAIL: misc/tst-mman-consts

This must be the error I saw elsewhere:

| First source:
| #define _GNU_SOURCE 1
| #include <sys/mman.h>
| 
| 
| Second source:
| #define _GNU_SOURCE 1
| #include <linux/mman.h>
| 
| 
| Only in first source: MAP_ABOVE4G
| Different values for MAP_HUGE_16GB: 2281701376 != -2013265920

The reason is that the kernel changed the definition of MAP_HUGE_16GB.
Older UAPI headers use an undefined expression which is treated by
compilers as a signed int, hence t he discrepancy.

This was fixed on the kernel side in this commit:

commit 710bb68c2e3a24512e2d2bae470960d7488e97b1
Author: Matthias Goergens <matthias.goergens@gmail.com>
Date:   Mon Sep 5 11:19:04 2022 +0800

    hugetlb_encode.h: fix undefined behaviour (34 << 26)
    
    Left-shifting past the size of your datatype is undefined behaviour in C.
    The literal 34 gets the type `int`, and that one is not big enough to be
    left shifted by 26 bits.
    
    An `unsigned` is long enough (on any machine that has at least 32 bits for
    their ints.)
    
    For uniformity, we mark all the literals as unsigned.  But it's only
    really needed for HUGETLB_FLAG_ENCODE_16GB.
    
    Thanks to Randy Dunlap for an initial review and suggestion.
    
    Link: https://lkml.kernel.org/r/20220905031904.150925-1-matthias.goergens@gmail.com
    Signed-off-by: Matthias Goergens <matthias.goergens@gmail.com>
    Acked-by: Randy Dunlap <rdunlap@infradead.org>
    Cc: Mike Kravetz <mike.kravetz@oracle.com>
    Cc: Muchun Song <songmuchun@bytedance.com>
    Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

Not sure what to do about this on the glibc side.  Can we waive this in
the CI, or should try to fix up the glibc test?

Thanks,
Florian

Re: [Linaro-TCWG-CI] glibc-2.39.9000-340-g176671f604: FAIL: 1 regressions on aarch64

[Carlos Llamas]

On Tue, Jun 18, 2024 at 06:37:29PM +0200, Florian Weimer wrote:
> | First source:
> | #define _GNU_SOURCE 1
> | #include <sys/mman.h>
> | 
> | 
> | Second source:
> | #define _GNU_SOURCE 1
> | #include <linux/mman.h>
> | 
> | 
> | Only in first source: MAP_ABOVE4G
> | Different values for MAP_HUGE_16GB: 2281701376 != -2013265920
> 
> The reason is that the kernel changed the definition of MAP_HUGE_16GB.
> Older UAPI headers use an undefined expression which is treated by
> compilers as a signed int, hence t he discrepancy.
> 

Oops, sorry I never tested this against older kernel headers.

> This was fixed on the kernel side in this commit:
> 
> commit 710bb68c2e3a24512e2d2bae470960d7488e97b1
> Author: Matthias Goergens <matthias.goergens@gmail.com>
> Date:   Mon Sep 5 11:19:04 2022 +0800
> 
>     hugetlb_encode.h: fix undefined behaviour (34 << 26)
>     
>     Left-shifting past the size of your datatype is undefined behaviour in C.
>     The literal 34 gets the type `int`, and that one is not big enough to be
>     left shifted by 26 bits.
>     
>     An `unsigned` is long enough (on any machine that has at least 32 bits for
>     their ints.)
>     
>     For uniformity, we mark all the literals as unsigned.  But it's only
>     really needed for HUGETLB_FLAG_ENCODE_16GB.
>     
>     Thanks to Randy Dunlap for an initial review and suggestion.
>     
>     Link: https://lkml.kernel.org/r/20220905031904.150925-1-matthias.goergens@gmail.com
>     Signed-off-by: Matthias Goergens <matthias.goergens@gmail.com>
>     Acked-by: Randy Dunlap <rdunlap@infradead.org>
>     Cc: Mike Kravetz <mike.kravetz@oracle.com>
>     Cc: Muchun Song <songmuchun@bytedance.com>
>     Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
> 

Sounds like this kernel commit should have been backported to stable but
never was. The glibc test caught the issue. I'll go ahead and backport
the fix to the linux stable branches.

> Not sure what to do about this on the glibc side.  Can we waive this in
> the CI, or should try to fix up the glibc test?

Hopefully this can be done until the fix propagates to the older
kernel's UAPI.

Thanks
Carlos Llamas

Re: [Linaro-TCWG-CI] glibc-2.39.9000-340-g176671f604: FAIL: 1 regressions on aarch64

[Carlos Llamas]

On Tue, Jun 18, 2024 at 06:37:29PM +0200, Florian Weimer wrote:
> The reason is that the kernel changed the definition of MAP_HUGE_16GB.
> Older UAPI headers use an undefined expression which is treated by
> compilers as a signed int, hence t he discrepancy.
> 
> This was fixed on the kernel side in this commit:
> 
> commit 710bb68c2e3a24512e2d2bae470960d7488e97b1
> Author: Matthias Goergens <matthias.goergens@gmail.com>
> Date:   Mon Sep 5 11:19:04 2022 +0800
> 
>     hugetlb_encode.h: fix undefined behaviour (34 << 26)
>     
>     Left-shifting past the size of your datatype is undefined behaviour in C.
>     The literal 34 gets the type `int`, and that one is not big enough to be
>     left shifted by 26 bits.
>     
>     An `unsigned` is long enough (on any machine that has at least 32 bits for
>     their ints.)
>     
>     For uniformity, we mark all the literals as unsigned.  But it's only
>     really needed for HUGETLB_FLAG_ENCODE_16GB.
>     
>     Thanks to Randy Dunlap for an initial review and suggestion.
>     
>     Link: https://lkml.kernel.org/r/20220905031904.150925-1-matthias.goergens@gmail.com
>     Signed-off-by: Matthias Goergens <matthias.goergens@gmail.com>
>     Acked-by: Randy Dunlap <rdunlap@infradead.org>
>     Cc: Mike Kravetz <mike.kravetz@oracle.com>
>     Cc: Muchun Song <songmuchun@bytedance.com>
>     Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
> 

This patch is now queued up for stable trees, from v4.19 and up.

Thanks,
Carlos Llamas

Re: [Linaro-TCWG-CI] glibc-2.39.9000-340-g176671f604: FAIL: 1 regressions on aarch64

[Adhemerval Zanella Netto]

On 18/06/24 13:37, Florian Weimer wrote:
> * ci notify:
> 
>> In glibc_check master-aarch64 after:
>>
>>   | commit glibc-2.39.9000-340-g176671f604
>>   | Author: Carlos Llamas <cmllamas@google.com>
>>   | Date:   Tue Jun 18 10:56:34 2024 +0200
>>   | 
>>   |     linux: add definitions for hugetlb page size encodings
>>   |     
>>   |     A desired hugetlb page size can be encoded in the flags parameter of
>>   |     system calls such as mmap() and shmget(). The Linux UAPI headers have
>>   |     included explicit definitions for these encodings since v4.14.
>>   |     
>>   |     This patch adds these definitions that are used along with MAP_HUGETLB
>>   | ... 14 lines of the commit log omitted.
>>
>> FAIL: 1 regressions
>>
>> regressions.sum:
>> 		=== glibc tests ===
>>
>> Running glibc:misc ...
>> FAIL: misc/tst-mman-consts
> 
> This must be the error I saw elsewhere:
> 
> | First source:
> | #define _GNU_SOURCE 1
> | #include <sys/mman.h>
> | 
> | 
> | Second source:
> | #define _GNU_SOURCE 1
> | #include <linux/mman.h>
> | 
> | 
> | Only in first source: MAP_ABOVE4G
> | Different values for MAP_HUGE_16GB: 2281701376 != -2013265920
> 
> The reason is that the kernel changed the definition of MAP_HUGE_16GB.
> Older UAPI headers use an undefined expression which is treated by
> compilers as a signed int, hence t he discrepancy.
> 
> This was fixed on the kernel side in this commit:
> 
> commit 710bb68c2e3a24512e2d2bae470960d7488e97b1
> Author: Matthias Goergens <matthias.goergens@gmail.com>
> Date:   Mon Sep 5 11:19:04 2022 +0800
> 
>     hugetlb_encode.h: fix undefined behaviour (34 << 26)
>     
>     Left-shifting past the size of your datatype is undefined behaviour in C.
>     The literal 34 gets the type `int`, and that one is not big enough to be
>     left shifted by 26 bits.
>     
>     An `unsigned` is long enough (on any machine that has at least 32 bits for
>     their ints.)
>     
>     For uniformity, we mark all the literals as unsigned.  But it's only
>     really needed for HUGETLB_FLAG_ENCODE_16GB.
>     
>     Thanks to Randy Dunlap for an initial review and suggestion.
>     
>     Link: https://lkml.kernel.org/r/20220905031904.150925-1-matthias.goergens@gmail.com
>     Signed-off-by: Matthias Goergens <matthias.goergens@gmail.com>
>     Acked-by: Randy Dunlap <rdunlap@infradead.org>
>     Cc: Mike Kravetz <mike.kravetz@oracle.com>
>     Cc: Muchun Song <songmuchun@bytedance.com>
>     Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
> 
> Not sure what to do about this on the glibc side.  Can we waive this in
> the CI, or should try to fix up the glibc test?

At least for Linaro CI it won't trigger any additional issues, the 
misc/tst-mman-consts will be marked as flaky tests on additional tests.
I think we can waive this for now.