glibc 2.31: hard freeze on Friday, 31st Jan

glibc 2.31: hard freeze on Friday, 31st Jan

[Siddhesh Poyarekar]

Hi,

It looks like testing during freeze has gone without incident and we're
nearing the release date.  I'll begin release testing tomorrow, so
please do not commit anything after Friday noon UTC.  That is, even if I
approve your patch for master and it fails to go in before noon UTC
tomorrow, it will have to wait until master reopens.  NEWS patches are
an exception of course; I'll just fix up and commit if they are not
pushed in time when I review changes for more newsworthy items.

If everything goes to plan, I will have finished release testing and cut
a release by Saturday night.

Thanks,
Siddhesh

Re: glibc 2.31: hard freeze on Friday, 31st Jan

[Carlos O'Donell]

On 1/30/20 10:15 AM, Siddhesh Poyarekar wrote:
> Hi,
> 
> It looks like testing during freeze has gone without incident and we're
> nearing the release date.  I'll begin release testing tomorrow, so
> please do not commit anything after Friday noon UTC.  That is, even if I
> approve your patch for master and it fails to go in before noon UTC
> tomorrow, it will have to wait until master reopens.  NEWS patches are
> an exception of course; I'll just fix up and commit if they are not
> pushed in time when I review changes for more newsworthy items.
> 
> If everything goes to plan, I will have finished release testing and cut
> a release by Saturday night.

Siddhesh,

Thank you for running the release!

The Fedora glibc team is largely done testing with no incidents.

The release is looking good.

Florian,

Is there anything else that you think needs cleaning up before release?

-- 
Cheers,
Carlos.

Re: glibc 2.31: hard freeze on Friday, 31st Jan

[Florian Weimer]

* Carlos O'Donell:

> Florian,
>
> Is there anything else that you think needs cleaning up before release?

No, I think we are good.

Thanks,
Florian

Re: glibc 2.31: hard freeze on Friday, 31st Jan

[Adhemerval Zanella]

On 30/01/2020 19:33, Florian Weimer wrote:
> * Carlos O'Donell:
> 
>> Florian,
>>
>> Is there anything else that you think needs cleaning up before release?
> 
> No, I think we are good.
> 
> Thanks,
> Florian
> 

Based on the extended asm and local register variable usage with GCC on
the 'powerpc Linux scv support and scv system call ABI proposal' I think
we hit a nasty latent sparc issue I am seeing at least on sparc32 tests
(it clobbers the syscalls result and make some calls to clock_nanosleep
to not act as a cancellation entrypoint).

I am working on a solution, do you think we should push it after the
release branch is created or can we hold a day or two?

Re: glibc 2.31: hard freeze on Friday, 31st Jan

[Florian Weimer]

* Adhemerval Zanella:

> Based on the extended asm and local register variable usage with GCC on
> the 'powerpc Linux scv support and scv system call ABI proposal' I think
> we hit a nasty latent sparc issue I am seeing at least on sparc32 tests
> (it clobbers the syscalls result and make some calls to clock_nanosleep
> to not act as a cancellation entrypoint).
>
> I am working on a solution, do you think we should push it after the
> release branch is created or can we hold a day or two?

I expect it will take considerable time for the GCC developers to agree
on the actual semantics of local register variables.  I do not think we
should hold the glibc release for it.  We can backport fixes as needed.

Thanks,
Florian

Re: glibc 2.31: hard freeze on Friday, 31st Jan

[Adhemerval Zanella]

On 31/01/2020 09:12, Florian Weimer wrote:
> * Adhemerval Zanella:
> 
>> Based on the extended asm and local register variable usage with GCC on
>> the 'powerpc Linux scv support and scv system call ABI proposal' I think
>> we hit a nasty latent sparc issue I am seeing at least on sparc32 tests
>> (it clobbers the syscalls result and make some calls to clock_nanosleep
>> to not act as a cancellation entrypoint).
>>
>> I am working on a solution, do you think we should push it after the
>> release branch is created or can we hold a day or two?
> 
> I expect it will take considerable time for the GCC developers to agree
> on the actual semantics of local register variables.  I do not think we
> should hold the glibc release for it.  We can backport fixes as needed.

I am seeing intermittent failures on some cancellation tests as reported
on wiki:

FAIL: nptl/tst-cancel18
FAIL: nptl/tst-cancel22
FAIL: nptl/tst-cancel23
FAIL: nptl/tst-cancel4
FAIL: nptl/tst-cancel5
FAIL: nptl/tst-cancel7
FAIL: nptl/tst-cancelx18
FAIL: nptl/tst-cancelx4
FAIL: nptl/tst-cancelx5
FAIL: nptl/tst-cancelx7

And based on the thread I do think we are doing some not fully supported
on sparc that we can mitigate with more concise syscall code. In any case
if you think this is not worth, we can backport it for sure.

Re: glibc 2.31: hard freeze on Friday, 31st Jan

[Florian Weimer]

* Adhemerval Zanella:

> On 31/01/2020 09:12, Florian Weimer wrote:
>> * Adhemerval Zanella:
>> 
>>> Based on the extended asm and local register variable usage with GCC on
>>> the 'powerpc Linux scv support and scv system call ABI proposal' I think
>>> we hit a nasty latent sparc issue I am seeing at least on sparc32 tests
>>> (it clobbers the syscalls result and make some calls to clock_nanosleep
>>> to not act as a cancellation entrypoint).
>>>
>>> I am working on a solution, do you think we should push it after the
>>> release branch is created or can we hold a day or two?
>> 
>> I expect it will take considerable time for the GCC developers to agree
>> on the actual semantics of local register variables.  I do not think we
>> should hold the glibc release for it.  We can backport fixes as needed.
>
> I am seeing intermittent failures on some cancellation tests as reported
> on wiki:
>
> FAIL: nptl/tst-cancel18
> FAIL: nptl/tst-cancel22
> FAIL: nptl/tst-cancel23
> FAIL: nptl/tst-cancel4
> FAIL: nptl/tst-cancel5
> FAIL: nptl/tst-cancel7
> FAIL: nptl/tst-cancelx18
> FAIL: nptl/tst-cancelx4
> FAIL: nptl/tst-cancelx5
> FAIL: nptl/tst-cancelx7
>
> And based on the thread I do think we are doing some not fully supported
> on sparc that we can mitigate with more concise syscall code. In any case
> if you think this is not worth, we can backport it for sure.

I think local register variables are currently ill-defined on all
architectures.  Getting past that will take a lot of time.

I think the sparc32 assembler would actually be okay if GCC actually
implemented the new model (where local register variables reside in
specific registers if used in asm operands, but behavior like ordinary
local variables otherwise).

Thanks,
Florian

Re: glibc 2.31: hard freeze on Friday, 31st Jan

[Adhemerval Zanella]

On 31/01/2020 09:49, Florian Weimer wrote:
> * Adhemerval Zanella:
> 
>> On 31/01/2020 09:12, Florian Weimer wrote:
>>> * Adhemerval Zanella:
>>>
>>>> Based on the extended asm and local register variable usage with GCC on
>>>> the 'powerpc Linux scv support and scv system call ABI proposal' I think
>>>> we hit a nasty latent sparc issue I am seeing at least on sparc32 tests
>>>> (it clobbers the syscalls result and make some calls to clock_nanosleep
>>>> to not act as a cancellation entrypoint).
>>>>
>>>> I am working on a solution, do you think we should push it after the
>>>> release branch is created or can we hold a day or two?
>>>
>>> I expect it will take considerable time for the GCC developers to agree
>>> on the actual semantics of local register variables.  I do not think we
>>> should hold the glibc release for it.  We can backport fixes as needed.
>>
>> I am seeing intermittent failures on some cancellation tests as reported
>> on wiki:
>>
>> FAIL: nptl/tst-cancel18
>> FAIL: nptl/tst-cancel22
>> FAIL: nptl/tst-cancel23
>> FAIL: nptl/tst-cancel4
>> FAIL: nptl/tst-cancel5
>> FAIL: nptl/tst-cancel7
>> FAIL: nptl/tst-cancelx18
>> FAIL: nptl/tst-cancelx4
>> FAIL: nptl/tst-cancelx5
>> FAIL: nptl/tst-cancelx7
>>
>> And based on the thread I do think we are doing some not fully supported
>> on sparc that we can mitigate with more concise syscall code. In any case
>> if you think this is not worth, we can backport it for sure.
> 
> I think local register variables are currently ill-defined on all
> architectures.  Getting past that will take a lot of time.

My understanding from the thread discussion was that some our assumptions
about local register variables was not in par with gcc documentation. 
What I am not sure is if the gcc documentation made it more clear or
actually changed its semantic over the releases.

> 
> I think the sparc32 assembler would actually be okay if GCC actually
> implemented the new model (where local register variables reside in
> specific registers if used in asm operands, but behavior like ordinary
> local variables otherwise).

Again my understanding from Segher Boessenkool explanation is GCC does
provide such semantic for local register variables.  I am not fully sure
what is causing sparc32 issues here, but nonetheless it is still a
*regression* that I would prefer to not release the glibc with it.  

I am currently testing a fix where I simplify a bit the sparc kernel
ABI call by not use 'g1' as the error indication, but rather using
the default Linux syscall ABI where negative value between -1 and
-4096 indicates an error (as for majority of architectures do).

Re: glibc 2.31: hard freeze on Friday, 31st Jan

[Siddhesh Poyarekar]

On 31/01/20 17:03, Adhemerval Zanella wrote:
> Based on the extended asm and local register variable usage with GCC on
> the 'powerpc Linux scv support and scv system call ABI proposal' I think
> we hit a nasty latent sparc issue I am seeing at least on sparc32 tests
> (it clobbers the syscalls result and make some calls to clock_nanosleep
> to not act as a cancellation entrypoint).
> 
> I am working on a solution, do you think we should push it after the
> release branch is created or can we hold a day or two?

I see that this is documented in the 2.31 wiki, I think that's a
reasonable compromise given the stage we're in.  Lets stik to schedule
and backport once there's consensus on the fix.

Siddhesh

Re: glibc 2.31: hard freeze on Friday, 31st Jan

[Carlos O'Donell]

On 1/31/20 10:06 AM, Siddhesh Poyarekar wrote:
> On 31/01/20 17:03, Adhemerval Zanella wrote:
>> Based on the extended asm and local register variable usage with GCC on
>> the 'powerpc Linux scv support and scv system call ABI proposal' I think
>> we hit a nasty latent sparc issue I am seeing at least on sparc32 tests
>> (it clobbers the syscalls result and make some calls to clock_nanosleep
>> to not act as a cancellation entrypoint).
>>
>> I am working on a solution, do you think we should push it after the
>> release branch is created or can we hold a day or two?
> 
> I see that this is documented in the 2.31 wiki, I think that's a
> reasonable compromise given the stage we're in.  Lets stik to schedule
> and backport once there's consensus on the fix.

Agreed.

The sparc32 issue, while a regression, is not a blocker for the release.

We should document the regression and alert downstream that a fix is in progress.

-- 
Cheers,
Carlos.

Re: glibc 2.31: hard freeze on Friday, 31st Jan

[Romain Naour]

Hi All,

Le 30/01/2020 à 16:15, Siddhesh Poyarekar a écrit :
> Hi,
> 
> It looks like testing during freeze has gone without incident and we're
> nearing the release date.  I'll begin release testing tomorrow, so
> please do not commit anything after Friday noon UTC.  That is, even if I
> approve your patch for master and it fails to go in before noon UTC
> tomorrow, it will have to wait until master reopens.  NEWS patches are
> an exception of course; I'll just fix up and commit if they are not
> pushed in time when I review changes for more newsworthy items.
> 
> If everything goes to plan, I will have finished release testing and cut
> a release by Saturday night.

The risc64 port needs at least a kernel headers >= 5.0

Otherwise glibc fail to build with:

../sysdeps/unix/sysv/linux/riscv/rv64/arch-syscall.h:193:33: error: expected
declaration specifiers or '...' before numeric constant
  193 | #define __NR_riscv_flush_icache 259
      |                                 ^~~
In file included from ../sysdeps/unix/sysv/linux/riscv/flush-icache.c:25:
/home/naourr/buildroot/test/toolchain/host/riscv64-buildroot-linux-gnu/sysroot/usr/include/asm/syscalls.h:29:36:
error: unknown type name 'sys_riscv_flush_icache'
   29 | __SYSCALL(__NR_riscv_flush_icache, sys_riscv_flush_icache)
      |                                    ^~~~~~~~~~~~~~~~~~~~~~

See: https://gitlab.com/kubu93/toolchains-builder/-/jobs/422726962


There are other issues related to glibc 2.31 that need to be fixed in other
software:

The last released version of Busybox needs to be patched to remove stime()

https://git.busybox.net/busybox/commit/?id=d3539be8f27b8cbfdfee460fe08299158f08bcd9


GCC 9.2.0 (and probably older gcc version) needs to be fixed due to an issue
with libsanitizer:

https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=ce9568e9e9cf6094be30e748821421e703754ffc

https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=75003cdd23c310ec385344e8040d490e8dd6d2be

Maybe this is something to add in the glibc wiki or release note?

Best regards,
Romain

> 
> Thanks,
> Siddhesh
> 

Re: glibc 2.31: hard freeze on Friday, 31st Jan

[Siddhesh Poyarekar]

On 01/02/20 13:53, Romain Naour wrote:
> The risc64 port needs at least a kernel headers >= 5.0
> 
> Otherwise glibc fail to build with:
> 
> ../sysdeps/unix/sysv/linux/riscv/rv64/arch-syscall.h:193:33: error: expected
> declaration specifiers or '...' before numeric constant
>   193 | #define __NR_riscv_flush_icache 259
>       |                                 ^~~
> In file included from ../sysdeps/unix/sysv/linux/riscv/flush-icache.c:25:
> /home/naourr/buildroot/test/toolchain/host/riscv64-buildroot-linux-gnu/sysroot/usr/include/asm/syscalls.h:29:36:
> error: unknown type name 'sys_riscv_flush_icache'
>    29 | __SYSCALL(__NR_riscv_flush_icache, sys_riscv_flush_icache)
>       |                                    ^~~~~~~~~~~~~~~~~~~~~~
> 
> See: https://gitlab.com/kubu93/toolchains-builder/-/jobs/422726962
> 

Thanks for testing.  We have a NEWS item that declares that we don't
need the latest Linux kernel headers to build anymore, which appears to
be not true for riscv.  I'll fix that NEWS item up like so:

* It is no longer necessary to have recent Linux kernel headers to build
  working (non-stub) system call wrappers on all architectures except
  64-bit RISC-V.  64-bit RISC-V requires a minimum kernel headers
  version of 5.0.

> 
> There are other issues related to glibc 2.31 that need to be fixed in other
> software:
> 
> The last released version of Busybox needs to be patched to remove stime()
> 
> https://git.busybox.net/busybox/commit/?id=d3539be8f27b8cbfdfee460fe08299158f08bcd9
> 

This is covered by the NEWS.

> GCC 9.2.0 (and probably older gcc version) needs to be fixed due to an issue
> with libsanitizer:
> 
> https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=ce9568e9e9cf6094be30e748821421e703754ffc
> 
> https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=75003cdd23c310ec385344e8040d490e8dd6d2be
> 
> Maybe this is something to add in the glibc wiki or release note?
> 

I have added this to the release wiki page.

Thanks,
Siddhesh

Re: glibc 2.31: hard freeze on Friday, 31st Jan

[Florian Weimer]

* Romain Naour:

> Hi All,
>
> Le 30/01/2020 à 16:15, Siddhesh Poyarekar a écrit :
>> Hi,
>> 
>> It looks like testing during freeze has gone without incident and we're
>> nearing the release date.  I'll begin release testing tomorrow, so
>> please do not commit anything after Friday noon UTC.  That is, even if I
>> approve your patch for master and it fails to go in before noon UTC
>> tomorrow, it will have to wait until master reopens.  NEWS patches are
>> an exception of course; I'll just fix up and commit if they are not
>> pushed in time when I review changes for more newsworthy items.
>> 
>> If everything goes to plan, I will have finished release testing and cut
>> a release by Saturday night.
>
> The risc64 port needs at least a kernel headers >= 5.0
>
> Otherwise glibc fail to build with:
>
> ../sysdeps/unix/sysv/linux/riscv/rv64/arch-syscall.h:193:33: error: expected
> declaration specifiers or '...' before numeric constant
>   193 | #define __NR_riscv_flush_icache 259
>       |                                 ^~~
> In file included from ../sysdeps/unix/sysv/linux/riscv/flush-icache.c:25:
> /home/naourr/buildroot/test/toolchain/host/riscv64-buildroot-linux-gnu/sysroot/usr/include/asm/syscalls.h:29:36:
> error: unknown type name 'sys_riscv_flush_icache'
>    29 | __SYSCALL(__NR_riscv_flush_icache, sys_riscv_flush_icache)
>       |                                    ^~~~~~~~~~~~~~~~~~~~~~
>
> See: https://gitlab.com/kubu93/toolchains-builder/-/jobs/422726962

This is quite ocnfusing.  Why doesn't expand __SYSCALL to nothing in
this context?

Ah, I see, this header wasn't originally written as a UAPI header.
Can you test if it works to drop these lines?

#if __has_include (<asm/syscalls.h>)
# include <asm/syscalls.h>
#else
# include <asm/unistd.h>
#endif

Including <sys/syscall.h> should now be sufficient.

Re: glibc 2.31: hard freeze on Friday, 31st Jan

[Romain Naour]

Hi Florian,

Le 01/02/2020 à 22:25, Florian Weimer a écrit :
> * Romain Naour:
> 
>> Hi All,
>>
>> Le 30/01/2020 à 16:15, Siddhesh Poyarekar a écrit :
>>> Hi,
>>>
>>> It looks like testing during freeze has gone without incident and we're
>>> nearing the release date.  I'll begin release testing tomorrow, so
>>> please do not commit anything after Friday noon UTC.  That is, even if I
>>> approve your patch for master and it fails to go in before noon UTC
>>> tomorrow, it will have to wait until master reopens.  NEWS patches are
>>> an exception of course; I'll just fix up and commit if they are not
>>> pushed in time when I review changes for more newsworthy items.
>>>
>>> If everything goes to plan, I will have finished release testing and cut
>>> a release by Saturday night.
>>
>> The risc64 port needs at least a kernel headers >= 5.0
>>
>> Otherwise glibc fail to build with:
>>
>> ../sysdeps/unix/sysv/linux/riscv/rv64/arch-syscall.h:193:33: error: expected
>> declaration specifiers or '...' before numeric constant
>>   193 | #define __NR_riscv_flush_icache 259
>>       |                                 ^~~
>> In file included from ../sysdeps/unix/sysv/linux/riscv/flush-icache.c:25:
>> /home/naourr/buildroot/test/toolchain/host/riscv64-buildroot-linux-gnu/sysroot/usr/include/asm/syscalls.h:29:36:
>> error: unknown type name 'sys_riscv_flush_icache'
>>    29 | __SYSCALL(__NR_riscv_flush_icache, sys_riscv_flush_icache)
>>       |                                    ^~~~~~~~~~~~~~~~~~~~~~
>>
>> See: https://gitlab.com/kubu93/toolchains-builder/-/jobs/422726962
> 
> This is quite ocnfusing.  Why doesn't expand __SYSCALL to nothing in
> this context?
> 
> Ah, I see, this header wasn't originally written as a UAPI header.
> Can you test if it works to drop these lines?
> 
> #if __has_include (<asm/syscalls.h>)
> # include <asm/syscalls.h>
> #else
> # include <asm/unistd.h>
> #endif
> 
> Including <sys/syscall.h> should now be sufficient.
> 

Indeed it is sufficient to fix this issue.

Thanks!

Best regards,
Romain