glibc 2.27: 3 weeks till release

glibc 2.27: 3 weeks till release

[Dmitry V. Levin]

[-- Attachment #1: Type: text/plain, Size: 1385 bytes --]

Hi,

We have 3 weeks till the planned release date.

* There are no release blockers listed in the release page[1], that is,
either we have no blockers or we haven't recognized them yet.
I suppose the latter is more likely explanation, so please
put them on the page.

* There are 4 features listed as desirable in the release page:

+ Month names in alternative grammatical case

Very little has changed for the last two weeks, but Carlos volunteered
to review this series.

+ C11 threads

This seems to be waiting for review for many weeks.
Are we still pursuing this?

+ Istvan Kurucsai's malloc security patches

Likewise, are we still pursuing this?

+ RISC-V port

The shared part has been committed, all the rest doesn't affect other
architectures.

* There are issues not listed in the release page that change ABI:

+ Implement allocate_once [2]

This is close to be committed.

+ libidn2-based IDNA implementation [3]

This qualifies as security bug fixes.

+ Regression caused by setjmp changes [4]

There is going to be another ABI change if these changes are reverted.

[1] https://sourceware.org/glibc/wiki/Release/2.27
[2] https://sourceware.org/ml/libc-alpha/2018-01/msg00288.html
[3] https://sourceware.org/ml/libc-alpha/2018-01/msg00335.html
[4] https://sourceware.org/ml/libc-alpha/2018-01/msg00268.html


-- 
ldv

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: glibc 2.27: 3 weeks till release

[Carlos O'Donell]

On 01/10/2018 07:44 PM, Dmitry V. Levin wrote:
> Hi,
> 
> We have 3 weeks till the planned release date.
> 
> * There are no release blockers listed in the release page[1], that is,
> either we have no blockers or we haven't recognized them yet.
> I suppose the latter is more likely explanation, so please
> put them on the page.
> 
> * There are 4 features listed as desirable in the release page:
> 
> + Month names in alternative grammatical case
> 
> Very little has changed for the last two weeks, but Carlos volunteered
> to review this series.

I have completed this review. The only remaining issue left is for
Rafal to add a few more tests, and explain the missing ABALTMON_*
definitions under _GNU_SOURCE. I think we'll get this checked in tomorrow.

> + C11 threads
> 
> This seems to be waiting for review for many weeks.
> Are we still pursuing this?

I will review this next.

> + Istvan Kurucsai's malloc security patches
> 
> Likewise, are we still pursuing this?

I don't have time to review these. If someone else would like to review
them they can.

> + RISC-V port
> 
> The shared part has been committed, all the rest doesn't affect other
> architectures.
> 
> * There are issues not listed in the release page that change ABI:
> 
> + Implement allocate_once [2]
> 
> This is close to be committed.

I reviewed this and it's complete, I have no real objections, only
some questions about the failure semantics. I expect we'll get this
committed tomorrow.

> + libidn2-based IDNA implementation [3]
> 
> This qualifies as security bug fixes.

I will be reviewing this after the C11 threads review.

> + Regression caused by setjmp changes [4]
> 
> There is going to be another ABI change if these changes are reverted.

Florian, Would you be able to review these changes while I review yours?
 
> [1] https://sourceware.org/glibc/wiki/Release/2.27
> [2] https://sourceware.org/ml/libc-alpha/2018-01/msg00288.html
> [3] https://sourceware.org/ml/libc-alpha/2018-01/msg00335.html
> [4] https://sourceware.org/ml/libc-alpha/2018-01/msg00268.html
 

-- 
Cheers,
Carlos.

Re: glibc 2.27: 3 weeks till release

[Joseph Myers]

On Thu, 11 Jan 2018, Dmitry V. Levin wrote:

> + RISC-V port
> 
> The shared part has been committed, all the rest doesn't affect other
> architectures.

To be precise:

* There are additions to architecture lists in documentation 
<https://sourceware.org/ml/libc-alpha/2017-12/msg00915.html>, which affect 
shared files, don't seem appropriate to apply without the port itself, but 
should be sufficiently safe.

* There are config.h.in additions 
<https://sourceware.org/ml/libc-alpha/2017-12/msg00916.html> that also 
don't seem appropriate without the port itself but should be safe.

* There are additions of RISC-V configurations to build-many-glibcs.py 
<https://sourceware.org/ml/libc-alpha/2017-12/msg00922.html> but that's 
inherently safe as build-many-glibcs.py is not part of the normal build / 
install process at all.

* There may be documentation for __riscv_flush_icache, requested but not 
yet posted <https://sourceware.org/ml/libc-alpha/2018-01/msg00093.html>.

* There may be ldconfig.h and cache.c flag additions for ldconfig support 
for multiple RISC-V ABIs, requested but not yet posted 
<https://sourceware.org/ml/libc-alpha/2018-01/msg00008.html>.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: glibc 2.27: 3 weeks till release

[Zack Weinberg]

On Wed, Jan 10, 2018 at 10:44 PM, Dmitry V. Levin <ldv@altlinux.org> wrote:
> We have 3 weeks till the planned release date.
...
> * There are 4 features listed as desirable in the release page:
...

I have added the patch removing libio.h from the installed stdio.h
from the desirable-feature list.  Florian raised a serious concern
with the status quo (libio.h still installed and its contents still
visible via stdio.h, but #warns if used directly) in
https://sourceware.org/ml/libc-alpha/2018-01/msg00242.html ; I think
we need to make an explicit decision whether to push forward and drop
libio.h from public use in 2.27 or back out
https://sourceware.org/ml/libc-alpha/2017-12/msg00869.html and try
again next release cycle.

I apologize for pushing this issue so late in the cycle but I am
concerned that, if we don't drop libio.h in 2.27, we'll have to defer
stdio improvements that depend on dropping it until 2.29.

zw

Re: glibc 2.27: 3 weeks till release

[Dmitry V. Levin]

[-- Attachment #1: Type: text/plain, Size: 284 bytes --]

On Thu, Jan 11, 2018 at 06:44:41AM +0300, Dmitry V. Levin wrote:
> * There are issues not listed in the release page that change ABI:

And there is also a patch that deprecates <libio.h> and <_G_config.h>:
https://sourceware.org/ml/libc-alpha/2018-01/msg00248.html


-- 
ldv

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: glibc 2.27: 3 weeks till release

[Zack Weinberg]

On Thu, Jan 11, 2018 at 9:45 AM, Dmitry V. Levin <ldv@altlinux.org> wrote:
> On Thu, Jan 11, 2018 at 06:44:41AM +0300, Dmitry V. Levin wrote:
>> * There are issues not listed in the release page that change ABI:

I don't seem to have received the message you're quoting here and it's
not in the archive either.

> And there is also a patch that deprecates <libio.h> and <_G_config.h>:
> https://sourceware.org/ml/libc-alpha/2018-01/msg00248.html

This patch obviously changes the set of supported APIs, but it should
not change the *ABI* of any library (unlike several earlier iterations
of patches to this end - please consider all of those withdrawn).

zw

Re: glibc 2.27: 3 weeks till release

[Dmitry V. Levin]

[-- Attachment #1: Type: text/plain, Size: 883 bytes --]

On Thu, Jan 11, 2018 at 09:51:52AM -0500, Zack Weinberg wrote:
> On Thu, Jan 11, 2018 at 9:45 AM, Dmitry V. Levin <ldv@altlinux.org> wrote:
> > On Thu, Jan 11, 2018 at 06:44:41AM +0300, Dmitry V. Levin wrote:
> >> * There are issues not listed in the release page that change ABI:
> 
> I don't seem to have received the message you're quoting here and it's
> not in the archive either.

I bet you have received it because I've seen your reply to that message. :)

> > And there is also a patch that deprecates <libio.h> and <_G_config.h>:
> > https://sourceware.org/ml/libc-alpha/2018-01/msg00248.html
> 
> This patch obviously changes the set of supported APIs, but it should
> not change the *ABI* of any library (unlike several earlier iterations
> of patches to this end - please consider all of those withdrawn).

Sure, thanks for clarification.


-- 
ldv

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: glibc 2.27: 3 weeks till release

[Zack Weinberg]

On Thu, Jan 11, 2018 at 9:57 AM, Dmitry V. Levin <ldv@altlinux.org> wrote:
> On Thu, Jan 11, 2018 at 09:51:52AM -0500, Zack Weinberg wrote:
>> On Thu, Jan 11, 2018 at 9:45 AM, Dmitry V. Levin <ldv@altlinux.org> wrote:
>> > On Thu, Jan 11, 2018 at 06:44:41AM +0300, Dmitry V. Levin wrote:
>> >> * There are issues not listed in the release page that change ABI:
>>
>> I don't seem to have received the message you're quoting here and it's
>> not in the archive either.
>
> I bet you have received it because I've seen your reply to that message. :)

Doh! I read it at least twice and managed not to see the quoted
sentence both times.

zw

Re: glibc 2.27: 3 weeks till release

[Rafal Luzynski]

11.01.2018 04:44 "Dmitry V. Levin" <ldv@altlinux.org> wrote:
> [...]
> * There are issues not listed in the release page that change ABI:
>
> + Implement allocate_once [2]
>
> This is close to be committed.
>
> + libidn2-based IDNA implementation [3]
>
> This qualifies as security bug fixes.
>
> + Regression caused by setjmp changes [4]
>
> There is going to be another ABI change if these changes are reverted.
>
> [1] https://sourceware.org/glibc/wiki/Release/2.27
> [2] https://sourceware.org/ml/libc-alpha/2018-01/msg00288.html
> [3] https://sourceware.org/ml/libc-alpha/2018-01/msg00335.html
> [4] https://sourceware.org/ml/libc-alpha/2018-01/msg00268.html

Shouldn't we consider my patches for alternative month names as ABI
change as well? Just to reiterate:

* nl_langinfo() family:
  + the meaning of MON_x and ABMON_x constants has been changed,
  + new constants: ALTMON_x and _NL_ABALTMON_x have been added, they
    work as their appropriate MON_x and ABMON_x variants previously;
* strftime() family:
  + the meaning of %B, %b, and %h has been changed (because of the
    above change),
  + new format specifiers: %OB, %Ob, %Oh have been added and they
    work as %B, %b, and %h previously;
* strptime() family:
  + the new format specifiers: %OB, %Ob, %Oh are valid,
  + %B, %b, %h, %OB, %Ob, %Oh accept all forms of month names,
    including the new ones.

There is no backward compatibility provided.  It has been discussed
and decided that this is a bug fix and providing the backward
compatibility causes more problems with almost no benefit.
That means that the change will be visible also in old binaries
running with the new glibc.

Note that the negative impact of this change is smaller than the
negative impact of the current bug.

Regards,

Rafal

Re: glibc 2.27: 3 weeks till release

[Carlos O'Donell]

On 01/11/2018 08:03 AM, Rafal Luzynski wrote:
> 11.01.2018 04:44 "Dmitry V. Levin" <ldv@altlinux.org> wrote:
>> [...]
>> * There are issues not listed in the release page that change ABI:
>>
>> + Implement allocate_once [2]
>>
>> This is close to be committed.
>>
>> + libidn2-based IDNA implementation [3]
>>
>> This qualifies as security bug fixes.
>>
>> + Regression caused by setjmp changes [4]
>>
>> There is going to be another ABI change if these changes are reverted.
>>
>> [1] https://sourceware.org/glibc/wiki/Release/2.27
>> [2] https://sourceware.org/ml/libc-alpha/2018-01/msg00288.html
>> [3] https://sourceware.org/ml/libc-alpha/2018-01/msg00335.html
>> [4] https://sourceware.org/ml/libc-alpha/2018-01/msg00268.html
> 
> Shouldn't we consider my patches for alternative month names as ABI
> change as well? Just to reiterate:
> 
> * nl_langinfo() family:
>   + the meaning of MON_x and ABMON_x constants has been changed,

This is a API change, but not an ABI change.

>   + new constants: ALTMON_x and _NL_ABALTMON_x have been added, they
>     work as their appropriate MON_x and ABMON_x variants previously;

This is an addition to the ABI, there are new constants with values,
and yes, that is an ABI change, but not a negative one IMO, and we
should get your work committed in this cycle.

> * strftime() family:
>   + the meaning of %B, %b, and %h has been changed (because of the
>     above change),
>   + new format specifiers: %OB, %Ob, %Oh have been added and they
>     work as %B, %b, and %h previously;

This is an API change, but not an ABI change.

> * strptime() family:
>   + the new format specifiers: %OB, %Ob, %Oh are valid,
>   + %B, %b, %h, %OB, %Ob, %Oh accept all forms of month names,
>     including the new ones.

Likewise.

> There is no backward compatibility provided.  It has been discussed
> and decided that this is a bug fix and providing the backward
> compatibility causes more problems with almost no benefit.
> That means that the change will be visible also in old binaries
> running with the new glibc.

Correct. That is not an ABI change though.

> Note that the negative impact of this change is smaller than the
> negative impact of the current bug.

Agreed.

-- 
Cheers,
Carlos.

Re: glibc 2.27: 3 weeks till release

[Florian Weimer]

On 01/12/2018 05:15 AM, Carlos O'Donell wrote:
> On 01/11/2018 08:03 AM, Rafal Luzynski wrote:
>> Shouldn't we consider my patches for alternative month names as ABI
>> change as well? Just to reiterate:
>>
>> * nl_langinfo() family:
>>    + the meaning of MON_x and ABMON_x constants has been changed,

> This is a API change, but not an ABI change.

What's the impact on statically linked applications?  Will they continue 
to be able to read both kinds of locale data?

Thanks,
Florian

Re: glibc 2.27: 3 weeks till release

[Rafal Luzynski]

12.01.2018 08:46 Florian Weimer <fweimer@redhat.com> wrote:
> On 01/12/2018 05:15 AM, Carlos O'Donell wrote:
> > On 01/11/2018 08:03 AM, Rafal Luzynski wrote:
> >> Shouldn't we consider my patches for alternative month names as ABI
> >> change as well? Just to reiterate:
> >>
> >> * nl_langinfo() family:
> >> + the meaning of MON_x and ABMON_x constants has been changed,
>
> > This is a API change, but not an ABI change.
>
> What's the impact on statically linked applications? Will they continue
> to be able to read both kinds of locale data?

I have not tried this so I don't know, I assume they will not be able.

Is it likely that an application linked statically against the new glibc
will read the old locale data and there will be no chance to rebuild
them from sources, or vice versa?  Note that the old locale data sources
are valid, the new features are optional.

Regards,

Rafal

Re: glibc 2.27: 3 weeks till release

[Joseph Myers]

On Fri, 12 Jan 2018, Rafal Luzynski wrote:

> > What's the impact on statically linked applications? Will they continue
> > to be able to read both kinds of locale data?
> 
> I have not tried this so I don't know, I assume they will not be able.
> 
> Is it likely that an application linked statically against the new glibc
> will read the old locale data and there will be no chance to rebuild
> them from sources, or vice versa?  Note that the old locale data sources
> are valid, the new features are optional.

We've previously accepted some incompatibility of compiled locale files 
with old glibc versions (and thus old statically linked binaries).  Note 
the following in the NEWS file for 2.19:

* Binary locale files now only depend on the endianness of the system for
  which they are generated and not on other properties of that system.  As a
  consequence, binary files generated with new localedef may be incompatible
  with old versions of the GNU C Library, and binary files generated with
  old localedef may be incompatible with this version of the GNU C Library,
  in the following circumstances:

  + Locale files may be incompatible on m68k systems.

  + Locale archive files (but not separate files for individual locales) may
    be incompatible on systems where plain "char" is signed.

If the present change affects binary locale compatibility, such a note 
will be needed in the "Deprecated and removed features, and other changes 
affecting compatibility:" section of NEWS for 2.27.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: glibc 2.27: 3 weeks till release

[Rafal Luzynski]

12.01.2018 18:14 Joseph Myers <joseph@codesourcery.com> wrote:
>
>
> On Fri, 12 Jan 2018, Rafal Luzynski wrote:
>
> > > What's the impact on statically linked applications? Will they continue
> > > to be able to read both kinds of locale data?
> >
> > I have not tried this so I don't know, I assume they will not be able.
> >
> > Is it likely that an application linked statically against the new glibc
> > will read the old locale data and there will be no chance to rebuild
> > them from sources, or vice versa? Note that the old locale data sources
> > are valid, the new features are optional.
>
> We've previously accepted some incompatibility of compiled locale files
> with old glibc versions (and thus old statically linked binaries). Note
> the following in the NEWS file for 2.19:
>
> [ ...cut to save the space... ]
>
> If the present change affects binary locale compatibility, such a note
> will be needed in the "Deprecated and removed features, and other changes
> affecting compatibility:" section of NEWS for 2.27.

I think that we still need this change in NEWS, don't we?  Will anybody
please write and commit this?  I have already proved that my English skills
are insufficient. :-(

Regards,

Rafal

Re: glibc 2.27: 3 weeks till release

[Carlos O'Donell]

On 01/22/2018 03:58 PM, Rafal Luzynski wrote:
> 12.01.2018 18:14 Joseph Myers <joseph@codesourcery.com> wrote:
>>
>>
>> On Fri, 12 Jan 2018, Rafal Luzynski wrote:
>>
>>>> What's the impact on statically linked applications? Will they continue
>>>> to be able to read both kinds of locale data?
>>>
>>> I have not tried this so I don't know, I assume they will not be able.
>>>
>>> Is it likely that an application linked statically against the new glibc
>>> will read the old locale data and there will be no chance to rebuild
>>> them from sources, or vice versa? Note that the old locale data sources
>>> are valid, the new features are optional.
>>
>> We've previously accepted some incompatibility of compiled locale files
>> with old glibc versions (and thus old statically linked binaries). Note
>> the following in the NEWS file for 2.19:
>>
>> [ ...cut to save the space... ]
>>
>> If the present change affects binary locale compatibility, such a note
>> will be needed in the "Deprecated and removed features, and other changes
>> affecting compatibility:" section of NEWS for 2.27.
> 
> I think that we still need this change in NEWS, don't we?  Will anybody
> please write and commit this?  I have already proved that my English skills
> are insufficient. :-(

Added to release notes for the wiki:
https://sourceware.org/glibc/wiki/Release/2.27#Statically_compiled_applications_using_locales

Added to NEWS.

commit 4378b735efba4a9de3cc9406a9492a823ff2fbf4 (HEAD -> master, origin/master, origin/HEAD)
Author: Carlos O'Donell <carlos@systemhalted.org>
Date:   Tue Jan 23 12:20:01 2018 -0800

    NEWS: Document static application consequenes of %OB/%Ob.

~~~
...
  This feature will cause existing statically compiled applications
  to fail to load locales and fall back to the builtin C/POSIX locales.
  See notes below for other changes affecting compatibility.

Deprecated and removed features, and other changes affecting compatibility:

* Statically compiled applications attempting to load locales compiled for the
  GNU C Library version 2.27 will fail and fall back to the builtin C/POSIX
  locale.  The reason for this is that the addition of the new "%OB" and "%ob",
  support for two grammatical forms of the month names, also extends the locale
  data binary format.  Static applications needing locale support must be
  recompiled to match the runtime and data they are deployed with. In some
  distributions there is an upgrade window where dynamically linked applications
  may use a new library but the old locale data and also fall back to the
  builtin C/POSIX locales; restarting the application process is sufficient to
  fix this.
~~~

-- 
Cheers,
Carlos.

Re: glibc 2.27: 3 weeks till release

[Rafal Luzynski]

23.01.2018 21:23 Carlos O'Donell <carlos@redhat.com> wrote:
>
>
> On 01/22/2018 03:58 PM, Rafal Luzynski wrote:
> > [...]
> > I think that we still need this change in NEWS, don't we? Will anybody
> > please write and commit this? I have already proved that my English skills
> > are insufficient. :-(
>
> Added to release notes for the wiki:
> https://sourceware.org/glibc/wiki/Release/2.27#Statically_compiled_applications_using_locales
>
> Added to NEWS.
>
> commit 4378b735efba4a9de3cc9406a9492a823ff2fbf4 (HEAD -> master,
> origin/master, origin/HEAD)
> Author: Carlos O'Donell <carlos@systemhalted.org>
> Date: Tue Jan 23 12:20:01 2018 -0800
>
> NEWS: Document static application consequenes of %OB/%Ob.

Thank you, Carlos.  Minor typos fixed.

Regards,

Rafal

Re: glibc 2.27: 3 weeks till release

[Carlos O'Donell]

On 01/11/2018 11:46 PM, Florian Weimer wrote:
> On 01/12/2018 05:15 AM, Carlos O'Donell wrote:
>> On 01/11/2018 08:03 AM, Rafal Luzynski wrote:
>>> Shouldn't we consider my patches for alternative month names as
>>> ABI change as well? Just to reiterate:
>>> 
>>> * nl_langinfo() family: + the meaning of MON_x and ABMON_x
>>> constants has been changed,
> 
>> This is a API change, but not an ABI change.
> 
> What's the impact on statically linked applications?  Will they
> continue to be able to read both kinds of locale data?

Officially speaking, statically linked applications which are not
recompiled to the new runtime are unsupported, everything else is
just an extension of the inevitable day when your static binary
fails to work with the installed data. The key is to provide a
fallback, in this case your setlocale() call would fail and you'd
get only C/POSIX builtins.

-- 
Cheers,
Carlos.

Re: glibc 2.27: 3 weeks till release

[Florian Weimer]

On 01/17/2018 11:36 PM, Carlos O'Donell wrote:
> On 01/11/2018 11:46 PM, Florian Weimer wrote:
>> On 01/12/2018 05:15 AM, Carlos O'Donell wrote:
>>> On 01/11/2018 08:03 AM, Rafal Luzynski wrote:
>>>> Shouldn't we consider my patches for alternative month names as
>>>> ABI change as well? Just to reiterate:
>>>>
>>>> * nl_langinfo() family: + the meaning of MON_x and ABMON_x
>>>> constants has been changed,
>>
>>> This is a API change, but not an ABI change.
>>
>> What's the impact on statically linked applications?  Will they
>> continue to be able to read both kinds of locale data?
> 
> Officially speaking, statically linked applications which are not
> recompiled to the new runtime are unsupported, everything else is
> just an extension of the inevitable day when your static binary
> fails to work with the installed data. The key is to provide a
> fallback, in this case your setlocale() call would fail and you'd
> get only C/POSIX builtins.

But this might have to change.  Static linking (with PIE) appears to be 
one of the more attractive ways to reduce indirect branches.

Is there an easy way to change the locale data encoding so that 
statically linked binaries will simply ignore the extra bits introduced 
by the ALTMON change?

Thanks,
Florian

Re: glibc 2.27: 3 weeks till release

[Rafal Luzynski]

18.01.2018 12:21 Florian Weimer <fweimer@redhat.com> wrote:
>
>
> On 01/17/2018 11:36 PM, Carlos O'Donell wrote:
> > [...]
> > Officially speaking, statically linked applications which are not
> > recompiled to the new runtime are unsupported, everything else is
> > just an extension of the inevitable day when your static binary
> > fails to work with the installed data. The key is to provide a
> > fallback, in this case your setlocale() call would fail and you'd
> > get only C/POSIX builtins.
>
> But this might have to change. Static linking (with PIE) appears to be
> one of the more attractive ways to reduce indirect branches.
>
> Is there an easy way to change the locale data encoding so that
> statically linked binaries will simply ignore the extra bits introduced
> by the ALTMON change?

My knowledge about static linking (with or without PIE) is kinda limited
but I think that in order to make the binary locale data backward and
forward compatible some sanity tests in locale/loadlocale.c would have to
be relaxed.  For example, LC_TIME should have 159 entries but 111 entries
should be also OK because ALTMON arrays are optional and we could copy the
missing entries from MON arrays.  At the moment my patches do this when
converting the locale data source code into the binary version but not
when loading the binary data.  Now the sanity tests check if the LC_TIME
size is exactly 159 (that's after my patches are applied, 111 in the
currently existing version) and reject the locale data if this is different.
Of course, this change is impossible for the statically linked binaries
which already exist so this is not helpful for you.

I think that if you need to run existing binaries statically linked
against glibc on the future new Linux systems with new locale data
and you want localization to work as previously then the easiest
solution is to deliver the old locale data in the binary version and
put them in a directory pointed to by LOCPATH environment variable.
For the same reasons the same was true whenever the locale data size
was changed in the past.  I have not tested this solution, though.

Regards,

Rafal

Re: glibc 2.27: 3 weeks till release

[Florian Weimer]

On 01/18/2018 12:59 PM, Rafal Luzynski wrote:
> My knowledge about static linking (with or without PIE) is kinda limited
> but I think that in order to make the binary locale data backward and
> forward compatible some sanity tests in locale/loadlocale.c would have to
> be relaxed.  For example, LC_TIME should have 159 entries but 111 entries
> should be also OK because ALTMON arrays are optional and we could copy the
> missing entries from MON arrays.  At the moment my patches do this when
> converting the locale data source code into the binary version but not
> when loading the binary data.  Now the sanity tests check if the LC_TIME
> size is exactly 159 (that's after my patches are applied, 111 in the
> currently existing version) and reject the locale data if this is different.
> Of course, this change is impossible for the statically linked binaries
> which already exist so this is not helpful for you.

Yes, the final part is the typical problem.  I have thought about this 
some more and I don't think we need to preserve static binary 
compatibility at this point.  In the future, when we improve static 
linking in general, we should consider relaxing the consistency checks 
and perhaps change the way we add new locale data to accommodate 
existing binaries.

But as I said, currently, this doesn't have to be taken into consideration.

Thanks,
Florian

Re: glibc 2.27: 3 weeks till release

[Joseph Myers]

On Thu, 18 Jan 2018, Florian Weimer wrote:

> Yes, the final part is the typical problem.  I have thought about this some
> more and I don't think we need to preserve static binary compatibility at this
> point.  In the future, when we improve static linking in general, we should
> consider relaxing the consistency checks and perhaps change the way we add new
> locale data to accommodate existing binaries.

Improving static binary compatibility would also mean ensuring the things 
that load .so modules work reliably even when the installed .so modules 
are from a newer libc version.  That's NSS, and character set conversions 
loading gconv modules, at least.

Is ld.so.cache compatibility between different glibc versions ever a 
consideration?  I decided not, when reviewing the RISC-V patches (i.e., 
there was no need to ask for 32-bit and 64-bit libraries to be identified 
separately in the cache now, even though that would be needed if in future 
Linux supports RV32I binaries on RV64I systems, because it would be OK to 
change the flag values incompatibly at that future point).

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: glibc 2.27: 3 weeks till release

[Florian Weimer]

On 01/18/2018 02:31 PM, Joseph Myers wrote:
> On Thu, 18 Jan 2018, Florian Weimer wrote:
> 
>> Yes, the final part is the typical problem.  I have thought about this some
>> more and I don't think we need to preserve static binary compatibility at this
>> point.  In the future, when we improve static linking in general, we should
>> consider relaxing the consistency checks and perhaps change the way we add new
>> locale data to accommodate existing binaries.
> 
> Improving static binary compatibility would also mean ensuring the things
> that load .so modules work reliably even when the installed .so modules
> are from a newer libc version.  That's NSS, and character set conversions
> loading gconv modules, at least.

Right.  I plan to find alternate ways to implement NSS/gconv for 
statically linked binaries.  That's obviously not going to happen over 
night, but my plan is to get rid of *all* internal uses of static 
dlopen, so that we can remove all support for it.

Thanks,
Florian

Re: glibc 2.27: 3 weeks till release

[Carlos O'Donell]

On 01/10/2018 07:44 PM, Dmitry V. Levin wrote:
> + libidn2-based IDNA implementation [3]
> 
> This qualifies as security bug fixes.

It is my opinion that this change should be deferred until the 2.28
branch opens after the release.

While I agree that moving IDNA to libidn2 solves a *lot* of problems
it may also introduce subtle differences that might cause application
regressions and we will want time to fix those.

I suggest that we, Fedora, as a downstream, adopt this patch ourselves
and begin testing it immediately in Fedora Rawhide. If everything
goes well we should check it into 2.28 when it opens or shortly after,
and that will give us the broadest testing window.

-- 
Cheers,
Carlos.

Re: glibc 2.27: 3 weeks till release

[Adhemerval Zanella]

[-- Attachment #1.1: Type: text/plain, Size: 1956 bytes --]

On 11/01/2018 01:44, Dmitry V. Levin wrote:
> Hi,
> 
> We have 3 weeks till the planned release date.
> 
> * There are no release blockers listed in the release page[1], that is,
> either we have no blockers or we haven't recognized them yet.
> I suppose the latter is more likely explanation, so please
> put them on the page.
> 
> * There are 4 features listed as desirable in the release page:
> 
> + Month names in alternative grammatical case
> 
> Very little has changed for the last two weeks, but Carlos volunteered
> to review this series.
> 
> + C11 threads
> 
> This seems to be waiting for review for many weeks.
> Are we still pursuing this?
> 
> + Istvan Kurucsai's malloc security patches
> 
> Likewise, are we still pursuing this?
> 
> + RISC-V port
> 
> The shared part has been committed, all the rest doesn't affect other
> architectures.
> 
> * There are issues not listed in the release page that change ABI:
> 
> + Implement allocate_once [2]
> 
> This is close to be committed.
> 
> + libidn2-based IDNA implementation [3]
> 
> This qualifies as security bug fixes.
> 
> + Regression caused by setjmp changes [4]
> 
> There is going to be another ABI change if these changes are reverted.
> 
> [1] https://sourceware.org/glibc/wiki/Release/2.27
> [2] https://sourceware.org/ml/libc-alpha/2018-01/msg00288.html
> [3] https://sourceware.org/ml/libc-alpha/2018-01/msg00335.html
> [4] https://sourceware.org/ml/libc-alpha/2018-01/msg00268.html

Recent LLVM has added support for _Float128 [1] and its intrinsics [2] 
used in glibc headers.  However it is currently failing on architectures
where long double is already a ieee float 128 (aarch64 for instance).
Szabolcs has opened BZ#22700 [3] and I think we should add it as blocker
for 2.27.

[1] https://reviews.llvm.org/D40673
[2] https://reviews.llvm.org/rL321948
[3] https://sourceware.org/bugzilla/show_bug.cgi?id=22700


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: glibc 2.27: 3 weeks till release

[Dmitry V. Levin]

[-- Attachment #1: Type: text/plain, Size: 852 bytes --]

On Fri, Jan 12, 2018 at 10:54:01AM -0200, Adhemerval Zanella wrote:
[...]
> Recent LLVM has added support for _Float128 [1] and its intrinsics [2] 
> used in glibc headers.  However it is currently failing on architectures
> where long double is already a ieee float 128 (aarch64 for instance).
> Szabolcs has opened BZ#22700 [3] and I think we should add it as blocker
> for 2.27.
> 
> [1] https://reviews.llvm.org/D40673
> [2] https://reviews.llvm.org/rL321948
> [3] https://sourceware.org/bugzilla/show_bug.cgi?id=22700

According to the last link, clang has temporarily reverted the problematic
definition.  Although it would be desirable to have bits/floatn.h
compilable on these architectures by future clang releases and I encourage
everybody involved to make this happen, it doesn't look like a blocker for
2.27.


-- 
ldv

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: glibc 2.27: 3 weeks till release

[Adhemerval Zanella]

[-- Attachment #1.1: Type: text/plain, Size: 1036 bytes --]



On 18/01/2018 13:30, Dmitry V. Levin wrote:
> On Fri, Jan 12, 2018 at 10:54:01AM -0200, Adhemerval Zanella wrote:
> [...]
>> Recent LLVM has added support for _Float128 [1] and its intrinsics [2] 
>> used in glibc headers.  However it is currently failing on architectures
>> where long double is already a ieee float 128 (aarch64 for instance).
>> Szabolcs has opened BZ#22700 [3] and I think we should add it as blocker
>> for 2.27.
>>
>> [1] https://reviews.llvm.org/D40673
>> [2] https://reviews.llvm.org/rL321948
>> [3] https://sourceware.org/bugzilla/show_bug.cgi?id=22700
> 
> According to the last link, clang has temporarily reverted the problematic
> definition.  Although it would be desirable to have bits/floatn.h
> compilable on these architectures by future clang releases and I encourage
> everybody involved to make this happen, it doesn't look like a blocker for
> 2.27.
> 
> 

Right, Szabolcz has updated the task after I brought it to this thread.
I agree this is not a blocker anymore.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]