glibc 2.24 -- Release blockers

glibc 2.24 -- Release blockers

[Adhemerval Zanella]

Hi all,

Now that we proposed and defined current release blockers, I would like
to move forward and either commit them or move to 2.25.  The idea is, as
in previous releases, to have a somewhat large time window to allow 
multiple arch and distro maintainers to test the upcoming release.

I plan to announce hard freeze by the start of next week, so we have a
quite narrow time window to sort out all the current release blockers.
The idea is to have all of them either committed or postponed to 2.25.

The current release blockers are listed in release wiki [1] below and I am
listing them with some comments:

  * [BZ #20139] Don't allow configure with not supporting AVX512 assembler 
    w/o --disable-avx512 [2]

    This seems to fix possible broken build depending of binutils and target
    hardware. I tend to see this a arch specific issue where the arch 
    maintainer should have the final answer.

  * [BZ #20309] X86-64: Properly align stack in _dl_tlsdesc_dynamic [3]

    This seems to be an important ABI fix and look ok to me.  I think also
    the arch maintainer have the final word.

  * Revised^2: deprecate major/minor/makedev in sys/types.h [4]

    This is a 4 patch set.  I see from previous iterations that some
    developers would like to include it in 2.24. 

    First patch is already commit. 

    Second patch seems ok with just the version inclusion and abifiles update
    seems to be wrong.  There is not compat symbol being added and both old
    an new version seems to be produce the same output.  I will reply to
    original thread.

    Patch 3 also seems most straightforward and I think is ok for upstream.

    Patch 4 also seems ok. 

    I am plan to check all the patches (with second one being adjusted to
    not add any version in Version or abilist).

  * Refactor Linux raise implementation (BZ#15368) [5]

    I think I addressed all the issue from previous revisions and I would like
    to commit it before hard release.

  * Remove __ASSUME_OFF_DIFF_OFF64 definition [6]

    This is another cleanup that I would like to push on 2.24 mainly to get rid
    of superfluous __ASSUME_OFF_DIFF_OFF64 define.  I see it is mostly safe, since
    the __OFF_T_MATCHES_OFF64_T have the same meaning.

    I would also like to push it before hard freeze.

  * Add pretty printers for the NPTL lock types [7]

    I do not have a strong opinion about this one, but I do see some developers
    would like to push this forward on 2.24.  Since it is not a non-intrusive
    and tests are already returning UNSUPPORTED, so I think it is safe for testing.
    I will let the reviewers of the thread device or not if the patch is ok for
    inclusion.

  * [BZ #13165] New condition variable [9]

    I would like to move forward on this one.  Based on Torvald feedback is has been
    tested on fedora rawride I think it is ok for inclusion.

    I will also check on the architectures I have access.

  * [BZ #20024] Fixed vector sincos/sincosf ABI [10]

    This is another ABI fix that I think should be up to arch maintainer the final
    word.
    
  * malloc: Remove malloc_get_state, malloc_set_state [11]

    I plan to test and review this patch as well, but I think is should be safe
    due its limited usage by programs.
    

So I would like for inputs of current blocker and if should move any of them to
2.25 (I see we have a narrow window for hard freeze).

Florian, could you also handle the security bugs (NEWS addition, CVE identifiers,
etc.) since you already have some experience from previous releases?


[1] https://sourceware.org/glibc/wiki/Release/2.24
[2] https://sourceware.org/ml/libc-alpha/2016-06/msg01280.html
[3] https://sourceware.org/ml/libc-alpha/2016-06/msg01209.html
[4] https://sourceware.org/ml/libc-alpha/2016-05/msg00274.html
[5] http://patchwork.sourceware.org/patch/13220/
[6] http://patchwork.sourceware.org/patch/13448/
[7] https://sourceware.org/ml/libc-alpha/2016-06/msg01126.html
[8] https://sourceware.org/ml/libc-alpha/2016-05/msg00605.html
[9] https://sourceware.org/ml/libc-alpha/2016-05/msg00605.html
[10] https://sourceware.org/ml/libc-alpha/2016-07/msg00008.html
[11] https://sourceware.org/ml/libc-alpha/2016-06/msg00905.html

Re: glibc 2.24 -- Release blockers

[Zack Weinberg]

On Thu, Jul 7, 2016 at 3:50 PM, Adhemerval Zanella
<adhemerval.zanella@linaro.org> wrote:
>
>   * Revised^2: deprecate major/minor/makedev in sys/types.h [4]
...
>     Second patch seems ok with just the version inclusion and abifiles update
>     seems to be wrong.  There is not compat symbol being added and both old
>     an new version seems to be produce the same output.  I will reply to
>     original thread.

The addition of version GLIBC_2.24 to all libc.abilist files is
unnecessary now, because it has already happened (with the addition of
quick_exit).

The changes to misc/Versions only make a difference for non-Linux
configurations -- but, IIRC, they are necessary in that case.  Notice
that arm/nacl/libc.abilist adds symbols as well as version GLIBC_2.24.
That should still happen.

zw

Re: glibc 2.24 -- Release blockers

[Adhemerval Zanella]

On 07/07/2016 17:22, Zack Weinberg wrote:
> On Thu, Jul 7, 2016 at 3:50 PM, Adhemerval Zanella
> <adhemerval.zanella@linaro.org> wrote:
>>
>>   * Revised^2: deprecate major/minor/makedev in sys/types.h [4]
> ...
>>     Second patch seems ok with just the version inclusion and abifiles update
>>     seems to be wrong.  There is not compat symbol being added and both old
>>     an new version seems to be produce the same output.  I will reply to
>>     original thread.
> 
> The addition of version GLIBC_2.24 to all libc.abilist files is
> unnecessary now, because it has already happened (with the addition of
> quick_exit).
> 
> The changes to misc/Versions only make a difference for non-Linux
> configurations -- but, IIRC, they are necessary in that case.  Notice
> that arm/nacl/libc.abilist adds symbols as well as version GLIBC_2.24.
> That should still happen.

Yes, I noted that since I remind about nacl.

Re: glibc 2.24 -- Release blockers

[Torvald Riegel]

On Thu, 2016-07-07 at 16:50 -0300, Adhemerval Zanella wrote:
>   * [BZ #13165] New condition variable [9]
> 
>     I would like to move forward on this one.  Based on Torvald feedback is has been
>     tested on fedora rawride I think it is ok for inclusion.

Testing on Rawhide was the plan, but hasn't happened yet (unless Carlos
did so in the meantime without telling me).
I think we should be able to test in Rawhide from now to the end of the
testing phase, and we could easily revert the patch in case a problem
shows up.  But I'll have to talk to Carlos to confirm this.

>     I will also check on the architectures I have access.

Thanks.  Please pick the second version of it (sorry for not marking it
as v2 in the subject):
https://sourceware.org/ml/libc-alpha/2016-06/msg00554.html

Re: glibc 2.24 -- Release blockers

[Adhemerval Zanella]

On 08/07/2016 04:20, Torvald Riegel wrote:
> On Thu, 2016-07-07 at 16:50 -0300, Adhemerval Zanella wrote:
>>   * [BZ #13165] New condition variable [9]
>>
>>     I would like to move forward on this one.  Based on Torvald feedback is has been
>>     tested on fedora rawride I think it is ok for inclusion.
> 
> Testing on Rawhide was the plan, but hasn't happened yet (unless Carlos
> did so in the meantime without telling me).
> I think we should be able to test in Rawhide from now to the end of the
> testing phase, and we could easily revert the patch in case a problem
> shows up.  But I'll have to talk to Carlos to confirm this.

Alright, seems reasonable.

> 
>>     I will also check on the architectures I have access.
> 
> Thanks.  Please pick the second version of it (sorry for not marking it
> as v2 in the subject):
> https://sourceware.org/ml/libc-alpha/2016-06/msg00554.html
> 

I am testing this version along with the atomic one which implements
'atomic_fetch_xor_release' [1] (with a fix to replace
atomic_compare_and_exchange_bool_rel) and everything seems ok. 

I have tested in two other architectures you haven't tested (s390 and
armhf) and I saw no regressions.  I am also planning to check with some
tests from LTP that uses pthread_cond_* on x86, powerpc64le, s390 and
arm.


[1] http://patchwork.sourceware.org/patch/12522/

Re: glibc 2.24 -- Release blockers

[Adhemerval Zanella]

I forgot to add in the previous email my initial idea for the dates
regarding 2.24 release.

I would like to start hard freeze by July 12th (Tue), it will give
us some time to settle (either commit or drop) the remaining release
blockers.

Based on previous releases, I would like to follow with 4 weeks for
testing and set the release date to Aug 2th.  Depending of testing
discussion I think it might be feasible to short it to just 3 weeks.

On 07/07/2016 16:50, Adhemerval Zanella wrote:
> Hi all,
> 
> Now that we proposed and defined current release blockers, I would like
> to move forward and either commit them or move to 2.25.  The idea is, as
> in previous releases, to have a somewhat large time window to allow 
> multiple arch and distro maintainers to test the upcoming release.
> 
> I plan to announce hard freeze by the start of next week, so we have a
> quite narrow time window to sort out all the current release blockers.
> The idea is to have all of them either committed or postponed to 2.25.
> 
> The current release blockers are listed in release wiki [1] below and I am
> listing them with some comments:
> 
>   * [BZ #20139] Don't allow configure with not supporting AVX512 assembler 
>     w/o --disable-avx512 [2]
> 
>     This seems to fix possible broken build depending of binutils and target
>     hardware. I tend to see this a arch specific issue where the arch 
>     maintainer should have the final answer.
> 
>   * [BZ #20309] X86-64: Properly align stack in _dl_tlsdesc_dynamic [3]
> 
>     This seems to be an important ABI fix and look ok to me.  I think also
>     the arch maintainer have the final word.
> 
>   * Revised^2: deprecate major/minor/makedev in sys/types.h [4]
> 
>     This is a 4 patch set.  I see from previous iterations that some
>     developers would like to include it in 2.24. 
> 
>     First patch is already commit. 
> 
>     Second patch seems ok with just the version inclusion and abifiles update
>     seems to be wrong.  There is not compat symbol being added and both old
>     an new version seems to be produce the same output.  I will reply to
>     original thread.
> 
>     Patch 3 also seems most straightforward and I think is ok for upstream.
> 
>     Patch 4 also seems ok. 
> 
>     I am plan to check all the patches (with second one being adjusted to
>     not add any version in Version or abilist).
> 
>   * Refactor Linux raise implementation (BZ#15368) [5]
> 
>     I think I addressed all the issue from previous revisions and I would like
>     to commit it before hard release.
> 
>   * Remove __ASSUME_OFF_DIFF_OFF64 definition [6]
> 
>     This is another cleanup that I would like to push on 2.24 mainly to get rid
>     of superfluous __ASSUME_OFF_DIFF_OFF64 define.  I see it is mostly safe, since
>     the __OFF_T_MATCHES_OFF64_T have the same meaning.
> 
>     I would also like to push it before hard freeze.
> 
>   * Add pretty printers for the NPTL lock types [7]
> 
>     I do not have a strong opinion about this one, but I do see some developers
>     would like to push this forward on 2.24.  Since it is not a non-intrusive
>     and tests are already returning UNSUPPORTED, so I think it is safe for testing.
>     I will let the reviewers of the thread device or not if the patch is ok for
>     inclusion.
> 
>   * [BZ #13165] New condition variable [9]
> 
>     I would like to move forward on this one.  Based on Torvald feedback is has been
>     tested on fedora rawride I think it is ok for inclusion.
> 
>     I will also check on the architectures I have access.
> 
>   * [BZ #20024] Fixed vector sincos/sincosf ABI [10]
> 
>     This is another ABI fix that I think should be up to arch maintainer the final
>     word.
>     
>   * malloc: Remove malloc_get_state, malloc_set_state [11]
> 
>     I plan to test and review this patch as well, but I think is should be safe
>     due its limited usage by programs.
>     
> 
> So I would like for inputs of current blocker and if should move any of them to
> 2.25 (I see we have a narrow window for hard freeze).
> 
> Florian, could you also handle the security bugs (NEWS addition, CVE identifiers,
> etc.) since you already have some experience from previous releases?
> 
> 
> [1] https://sourceware.org/glibc/wiki/Release/2.24
> [2] https://sourceware.org/ml/libc-alpha/2016-06/msg01280.html
> [3] https://sourceware.org/ml/libc-alpha/2016-06/msg01209.html
> [4] https://sourceware.org/ml/libc-alpha/2016-05/msg00274.html
> [5] http://patchwork.sourceware.org/patch/13220/
> [6] http://patchwork.sourceware.org/patch/13448/
> [7] https://sourceware.org/ml/libc-alpha/2016-06/msg01126.html
> [8] https://sourceware.org/ml/libc-alpha/2016-05/msg00605.html
> [9] https://sourceware.org/ml/libc-alpha/2016-05/msg00605.html
> [10] https://sourceware.org/ml/libc-alpha/2016-07/msg00008.html
> [11] https://sourceware.org/ml/libc-alpha/2016-06/msg00905.html
> 

Re: glibc 2.24 -- Release blockers

[Andreas Schwab]

Here is the result of rebuilding (a big part of) openSUSE Tumbleweed
against glibc 2.24:

https://build.opensuse.org/project/monitor?utf8=%E2%9C%93&commit=Filter%3A&failed=1&pkgname=&repo_a=1&repo_f=1&repo_p=1&repo_rebuild=1&repo_s=1&arch_aarch64=1&arch_armv6l=1&arch_armv7l=1&arch_i586=1&arch_ppc=1&arch_ppc64=1&arch_ppc64le=1&arch_s390x=1&arch_x86_64=1&project=home%3AAndreas_Schwab%3Aglibc&defaults=0

openmpi and emacs fail to build due to the malloc changes.

The kernel fails to build due to commit 94e73c9.

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

Re: glibc 2.24 -- Release blockers

[Florian Weimer]

On 07/13/2016 02:31 PM, Andreas Schwab wrote:
> Here is the result of rebuilding (a big part of) openSUSE Tumbleweed
> against glibc 2.24:
>
> https://build.opensuse.org/project/monitor?utf8=%E2%9C%93&commit=Filter%3A&failed=1&pkgname=&repo_a=1&repo_f=1&repo_p=1&repo_rebuild=1&repo_s=1&arch_aarch64=1&arch_armv6l=1&arch_armv7l=1&arch_i586=1&arch_ppc=1&arch_ppc64=1&arch_ppc64le=1&arch_s390x=1&arch_x86_64=1&project=home%3AAndreas_Schwab%3Aglibc&defaults=0
>
> openmpi and emacs fail to build due to the malloc changes.

Thanks for doing this.

I downloaded

https://build.opensuse.org/public/build/home:Andreas_Schwab:glibc/rebuild/x86_64/openmpi/_log

and it seems that openmpi rebuilds its own copy of ptmalloc with the 
system malloc.h header file and fails if the latter does not define 
__malloc_initialize_hook.  Considering that this symbol has been 
deprecated for a long time, I would say this is openmpi's fault.

I don't see logs for an emacs failure.

Florian

Re: glibc 2.24 -- Release blockers

[Andreas Schwab]

Florian Weimer <fweimer@redhat.com> writes:

> I don't see logs for an emacs failure.

https://build.opensuse.org/package/live_build_log/home:Andreas_Schwab:glibc/emacs/p/ppc64
https://build.opensuse.org/package/live_build_log/home:Andreas_Schwab:glibc/emacs/p/ppc64le

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

Re: glibc 2.24 -- Release blockers

[Florian Weimer]

On 07/13/2016 03:03 PM, Andreas Schwab wrote:
> Florian Weimer <fweimer@redhat.com> writes:
>
>> I don't see logs for an emacs failure.
>
> https://build.opensuse.org/package/live_build_log/home:Andreas_Schwab:glibc/emacs/p/ppc64
> https://build.opensuse.org/package/live_build_log/home:Andreas_Schwab:glibc/emacs/p/ppc64le

Apparently, ppc64 has very aggressive ASLR, and the built-in malloc in 
Emacs assumes that brk will hand out addresses which are close to the 
.data section in the executable: It has a direct mapping from addresses 
to blocks to block information, and if there is a large gap between 
.data and the heap, the block array is huge.

This was not encountered before because Emacs unexec does not support 
64-bit AIX, and that is probably the only ppc64 target where Emacs would 
use its internal malloc.

ppc64le probably has the same issue, but the latest upstream release 
lacks unexec support, so I can't test it there.

A workaround is to have lots and lots of memory.

I don't know if there is anything on the glibc side that we can 
reasonably do about this matter.  I don't want to back out the removal 
of __malloc_initialize_hook from <malloc.h> (which triggers the switch 
to the internal malloc).

Comments?

Florian

Re: glibc 2.24 -- Release blockers

[Jeff Law]

On 07/13/2016 11:09 AM, Florian Weimer wrote:
> On 07/13/2016 03:03 PM, Andreas Schwab wrote:
>> Florian Weimer <fweimer@redhat.com> writes:
>>
>>> I don't see logs for an emacs failure.
>>
>> https://build.opensuse.org/package/live_build_log/home:Andreas_Schwab:glibc/emacs/p/ppc64
>>
>> https://build.opensuse.org/package/live_build_log/home:Andreas_Schwab:glibc/emacs/p/ppc64le
>>
>
> Apparently, ppc64 has very aggressive ASLR, and the built-in malloc in
> Emacs assumes that brk will hand out addresses which are close to the
> .data section in the executable: It has a direct mapping from addresses
> to blocks to block information, and if there is a large gap between
> .data and the heap, the block array is huge.
>
> This was not encountered before because Emacs unexec does not support
> 64-bit AIX, and that is probably the only ppc64 target where Emacs would
> use its internal malloc.
>
> ppc64le probably has the same issue, but the latest upstream release
> lacks unexec support, so I can't test it there.
I thought upstream emacs fixed the various unexec issues for ppc64le 
that fell out of the various RELRO changes.

Jeff

Re: glibc 2.24 -- Release blockers

[Florian Weimer]

On 07/13/2016 07:15 PM, Jeff Law wrote:

>> Apparently, ppc64 has very aggressive ASLR, and the built-in malloc in
>> Emacs assumes that brk will hand out addresses which are close to the
>> .data section in the executable: It has a direct mapping from addresses
>> to blocks to block information, and if there is a large gap between
>> .data and the heap, the block array is huge.
>>
>> This was not encountered before because Emacs unexec does not support
>> 64-bit AIX, and that is probably the only ppc64 target where Emacs would
>> use its internal malloc.
>>
>> ppc64le probably has the same issue, but the latest upstream release
>> lacks unexec support, so I can't test it there.

> I thought upstream emacs fixed the various unexec issues for ppc64le
> that fell out of the various RELRO changes.

There has not been an Emacs release since then.  I deliberately tested 
the latest upstream release (24.5).

Florian

Re: glibc 2.24 -- Release blockers

[Dan Horák]

On Wed, 13 Jul 2016 19:09:19 +0200
Florian Weimer <fweimer@redhat.com> wrote:

> On 07/13/2016 03:03 PM, Andreas Schwab wrote:
> > Florian Weimer <fweimer@redhat.com> writes:
> >
> >> I don't see logs for an emacs failure.
> >
> > https://build.opensuse.org/package/live_build_log/home:Andreas_Schwab:glibc/emacs/p/ppc64
> > https://build.opensuse.org/package/live_build_log/home:Andreas_Schwab:glibc/emacs/p/ppc64le
> 
> Apparently, ppc64 has very aggressive ASLR, and the built-in malloc
> in Emacs assumes that brk will hand out addresses which are close to
> the .data section in the executable: It has a direct mapping from
> addresses to blocks to block information, and if there is a large gap
> between .data and the heap, the block array is huge.
> 
> This was not encountered before because Emacs unexec does not support 
> 64-bit AIX, and that is probably the only ppc64 target where Emacs
> would use its internal malloc.
> 
> ppc64le probably has the same issue, but the latest upstream release 
> lacks unexec support, so I can't test it there.
> 
> A workaround is to have lots and lots of memory.

I guess it explains also emacs build failure in rawhide -
http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=3535700


		Dan
 
> I don't know if there is anything on the glibc side that we can 
> reasonably do about this matter.  I don't want to back out the
> removal of __malloc_initialize_hook from <malloc.h> (which triggers
> the switch to the internal malloc).
> 
> Comments?
> 
> Florian

Re: glibc 2.24 -- Release blockers

[Andreas Schwab]

Florian Weimer <fweimer@redhat.com> writes:

> I don't know if there is anything on the glibc side that we can reasonably
> do about this matter.  I don't want to back out the removal of
> __malloc_initialize_hook from <malloc.h> (which triggers the switch to the
> internal malloc).

Apparently it's not ready for the world yet.

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

Re: glibc 2.24 -- Release blockers

[Torvald Riegel]

On Thu, 2016-07-14 at 10:23 +0200, Andreas Schwab wrote:
> Florian Weimer <fweimer@redhat.com> writes:
> 
> > I don't know if there is anything on the glibc side that we can reasonably
> > do about this matter.  I don't want to back out the removal of
> > __malloc_initialize_hook from <malloc.h> (which triggers the switch to the
> > internal malloc).
> 
> Apparently it's not ready for the world yet.

It rather seems to be the case that a part of the world (ie, emacs)
still hasn't managed to not use internals of glibc, or fixed its own
work-arounds.

Re: glibc 2.24 -- Release blockers

[Andreas Schwab]

Torvald Riegel <triegel@redhat.com> writes:

> On Thu, 2016-07-14 at 10:23 +0200, Andreas Schwab wrote:
>> Florian Weimer <fweimer@redhat.com> writes:
>> 
>> > I don't know if there is anything on the glibc side that we can reasonably
>> > do about this matter.  I don't want to back out the removal of
>> > __malloc_initialize_hook from <malloc.h> (which triggers the switch to the
>> > internal malloc).
>> 
>> Apparently it's not ready for the world yet.
>
> It rather seems to be the case that a part of the world (ie, emacs)
> still hasn't managed to not use internals of glibc, or fixed its own
> work-arounds.

Yes, but that needs to be resolved before the release.

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

Re: glibc 2.24 -- Release blockers

[Torvald Riegel]

On Thu, 2016-07-14 at 10:32 +0200, Andreas Schwab wrote:
> Torvald Riegel <triegel@redhat.com> writes:
> 
> > On Thu, 2016-07-14 at 10:23 +0200, Andreas Schwab wrote:
> >> Florian Weimer <fweimer@redhat.com> writes:
> >> 
> >> > I don't know if there is anything on the glibc side that we can reasonably
> >> > do about this matter.  I don't want to back out the removal of
> >> > __malloc_initialize_hook from <malloc.h> (which triggers the switch to the
> >> > internal malloc).
> >> 
> >> Apparently it's not ready for the world yet.
> >
> > It rather seems to be the case that a part of the world (ie, emacs)
> > still hasn't managed to not use internals of glibc, or fixed its own
> > work-arounds.
> 
> Yes, but that needs to be resolved before the release.

IIRC, Florian started this several months ago, so it seems emacs should
have had enough time to catch up by now.  Also, bugs in their own malloc
replacement hardly seem like something we'd be responsible for.

Re: glibc 2.24 -- Release blockers

[Andreas Schwab]

Torvald Riegel <triegel@redhat.com> writes:

> On Thu, 2016-07-14 at 10:32 +0200, Andreas Schwab wrote:
>> Torvald Riegel <triegel@redhat.com> writes:
>> 
>> > On Thu, 2016-07-14 at 10:23 +0200, Andreas Schwab wrote:
>> >> Florian Weimer <fweimer@redhat.com> writes:
>> >> 
>> >> > I don't know if there is anything on the glibc side that we can reasonably
>> >> > do about this matter.  I don't want to back out the removal of
>> >> > __malloc_initialize_hook from <malloc.h> (which triggers the switch to the
>> >> > internal malloc).
>> >> 
>> >> Apparently it's not ready for the world yet.
>> >
>> > It rather seems to be the case that a part of the world (ie, emacs)
>> > still hasn't managed to not use internals of glibc, or fixed its own
>> > work-arounds.
>> 
>> Yes, but that needs to be resolved before the release.
>
> IIRC, Florian started this several months ago, so it seems emacs should
> have had enough time to catch up by now.  Also, bugs in their own malloc
> replacement hardly seem like something we'd be responsible for.

We are responsible for removing a supported interface.

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

Re: glibc 2.24 -- Release blockers

[Florian Weimer]

On 07/14/2016 10:41 AM, Andreas Schwab wrote:
> Torvald Riegel <triegel@redhat.com> writes:
>> IIRC, Florian started this several months ago, so it seems emacs should
>> have had enough time to catch up by now.  Also, bugs in their own malloc
>> replacement hardly seem like something we'd be responsible for.
>
> We are responsible for removing a supported interface.

The interface was deprecated in 2011.  Existing binaries continue to 
work, which is something we did not think possible in the past.

Five years should have been plenty of time for Emacs to adjust.  The 
problem was raised repeatedly on the emacs-devel list (not just this year).

Emacs 25 will still use the deprecated __malloc_initialize_hook if it 
can, significantly reducing test coverage for the src/gmalloc.c 
allocator.  (The unexec code does not work on many non-Linux 64 bit 
platforms, so the 64-bit issue with that allocator went unnoticed so far.)

I'm not sure about the way forward.  This Emacs dependency blocks any 
non-trivial change to glibc malloc, and we have security and performance 
issues to address.

Thanks,
Florian

Re: glibc 2.24 -- Release blockers

[Torvald Riegel]

On Thu, 2016-07-14 at 10:41 +0200, Andreas Schwab wrote:
> Torvald Riegel <triegel@redhat.com> writes:
> 
> > On Thu, 2016-07-14 at 10:32 +0200, Andreas Schwab wrote:
> >> Torvald Riegel <triegel@redhat.com> writes:
> >> 
> >> > On Thu, 2016-07-14 at 10:23 +0200, Andreas Schwab wrote:
> >> >> Florian Weimer <fweimer@redhat.com> writes:
> >> >> 
> >> >> > I don't know if there is anything on the glibc side that we can reasonably
> >> >> > do about this matter.  I don't want to back out the removal of
> >> >> > __malloc_initialize_hook from <malloc.h> (which triggers the switch to the
> >> >> > internal malloc).
> >> >> 
> >> >> Apparently it's not ready for the world yet.
> >> >
> >> > It rather seems to be the case that a part of the world (ie, emacs)
> >> > still hasn't managed to not use internals of glibc, or fixed its own
> >> > work-arounds.
> >> 
> >> Yes, but that needs to be resolved before the release.
> >
> > IIRC, Florian started this several months ago, so it seems emacs should
> > have had enough time to catch up by now.  Also, bugs in their own malloc
> > replacement hardly seem like something we'd be responsible for.
> 
> We are responsible for removing a supported interface.

AFAIU, the remaining emacs failure is a bug in *emacs' own* malloc
implementation.  While it seems that what we do triggers this problem,
it's pretty clearly a bug they should have fixed and will have to fix
anyway.

I can't remember other cases of us not making reasonable changes because
not making them hides bugs in other projects.  Why would emacs be an
exception?

Re: glibc 2.24 -- Release blockers

[Andreas Schwab]

Florian Weimer <fweimer@redhat.com> writes:

> I'm not sure about the way forward.  This Emacs dependency blocks any
> non-trivial change to glibc malloc, and we have security and performance
> issues to address.

And openmpi fails to build, too.  Pulling the trigger isn't the way
forward.  Working with the affected projects to resolve the issues is.
Glibc isn't just a random library, it is the central part of GNU/Linux.

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

Re: glibc 2.24 -- Release blockers

[Andreas Schwab]

Torvald Riegel <triegel@redhat.com> writes:

> I can't remember other cases of us not making reasonable changes because
> not making them hides bugs in other projects.  Why would emacs be an
> exception?

It isn't an exception.  It is an unresolved problem.

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

Re: glibc 2.24 -- Release blockers

[Torvald Riegel]

On Thu, 2016-07-14 at 10:50 +0200, Florian Weimer wrote:
> On 07/14/2016 10:41 AM, Andreas Schwab wrote:
> > Torvald Riegel <triegel@redhat.com> writes:
> >> IIRC, Florian started this several months ago, so it seems emacs should
> >> have had enough time to catch up by now.  Also, bugs in their own malloc
> >> replacement hardly seem like something we'd be responsible for.
> >
> > We are responsible for removing a supported interface.
> 
> The interface was deprecated in 2011.  Existing binaries continue to 
> work, which is something we did not think possible in the past.
> 
> Five years should have been plenty of time for Emacs to adjust.  The 
> problem was raised repeatedly on the emacs-devel list (not just this year).
> 
> Emacs 25 will still use the deprecated __malloc_initialize_hook if it 
> can, significantly reducing test coverage for the src/gmalloc.c 
> allocator.  (The unexec code does not work on many non-Linux 64 bit 
> platforms, so the 64-bit issue with that allocator went unnoticed so far.)
> 
> I'm not sure about the way forward.  This Emacs dependency blocks any 
> non-trivial change to glibc malloc, and we have security and performance 
> issues to address.

So, Emacs had plenty of time to fix this, was notified repeatedly, we
helped them as far as they could, and they still don't do anything on
their side?

If we have to choose between security and performance projects for
almost all programs (except Emacs) vs. making it easier for Emacs to
pretend that the world around them doesn't move, then I'm sure that I'd
pick the former.

Re: glibc 2.24 -- Release blockers

[Torvald Riegel]

On Thu, 2016-07-14 at 10:59 +0200, Andreas Schwab wrote:
> Torvald Riegel <triegel@redhat.com> writes:
> 
> > I can't remember other cases of us not making reasonable changes because
> > not making them hides bugs in other projects.  Why would emacs be an
> > exception?
> 
> It isn't an exception.  It is an unresolved problem.

If there are no cases that are similar, and we generally treat
user-program bugs differently, it is an exception.  Do you have examples
in other programs that we treated similarly?

Re: glibc 2.24 -- Release blockers

[Torvald Riegel]

On Thu, 2016-07-14 at 10:57 +0200, Andreas Schwab wrote:
> Florian Weimer <fweimer@redhat.com> writes:
> 
> > I'm not sure about the way forward.  This Emacs dependency blocks any
> > non-trivial change to glibc malloc, and we have security and performance
> > issues to address.
> 
> And openmpi fails to build, too.  Pulling the trigger isn't the way
> forward.  Working with the affected projects to resolve the issues is.
> Glibc isn't just a random library, it is the central part of GNU/Linux.

openmpi fails because its replacement for our allocator doesn't build.
IOW, it doesn't want our stuff, and fails to do it's own thing.

Furthermore, I'd guess that they don't want our malloc because they
assume ptmalloc to be faster.  Which will stay that way until we can
finally remove the symbol that has been deprecated for years.
Thus, if we don't move forward, we're not making it any better for
openmpi either because we're not doing something about the root
problem. 

Re: glibc 2.24 -- Release blockers

[Andreas Schwab]

Torvald Riegel <triegel@redhat.com> writes:

> So, Emacs had plenty of time to fix this, was notified repeatedly, we
> helped them as far as they could, and they still don't do anything on
> their side?

It is not "they", it is "we and they".

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

Re: glibc 2.24 -- Release blockers

[Andreas Schwab]

Torvald Riegel <triegel@redhat.com> writes:

> openmpi fails because its replacement for our allocator doesn't build.

It fails because we removed a documented interface.

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

Re: glibc 2.24 -- Release blockers

[Torvald Riegel]

On Thu, 2016-07-14 at 11:17 +0200, Andreas Schwab wrote:
> Torvald Riegel <triegel@redhat.com> writes:
> 
> > So, Emacs had plenty of time to fix this, was notified repeatedly, we
> > helped them as far as they could, and they still don't do anything on
> > their side?
> 
> It is not "they", it is "we and they".

If you're trying to say that both we and they should work on solving
this, then I agree.  But it seems that we did whereas they didn't.
If you're trying to say something less, please be a little less cryptic.

Re: glibc 2.24 -- Release blockers

[Florian Weimer]

On 07/14/2016 11:09 AM, Torvald Riegel wrote:
> On Thu, 2016-07-14 at 10:59 +0200, Andreas Schwab wrote:
>> Torvald Riegel <triegel@redhat.com> writes:
>>
>>> I can't remember other cases of us not making reasonable changes because
>>> not making them hides bugs in other projects.  Why would emacs be an
>>> exception?
>>
>> It isn't an exception.  It is an unresolved problem.
>
> If there are no cases that are similar, and we generally treat
> user-program bugs differently, it is an exception.  Do you have examples
> in other programs that we treated similarly?

The story around memcpy@@GLIBC_2.14 is similar.  We added an 
optimization to a new symbol version only.  This kept old and buggy 
binaries working, but you could no longer copy new binaries to an older, 
pre-2.14 system and run it there (which usually works, although it is 
technically unsupported).  So in this case, we rated compatibility with 
buggy applications higher than user convenience.

I'm not sure if we have ever done something like that to hide bugs in 
applications which were compiled from source after a glibc update.

Florian

Re: glibc 2.24 -- Release blockers

[Torvald Riegel]

On Thu, 2016-07-14 at 11:30 +0200, Andreas Schwab wrote:
> Torvald Riegel <triegel@redhat.com> writes:
> 
> > openmpi fails because its replacement for our allocator doesn't build.
> 
> It fails because we removed a documented interface.

... which has been deprecated for 5 years, and is an interface to glibc
internals.

You seem to ignore the bigger picture here.  What is your suggestion for
improving our malloc, your assessment of when we will be able to do
anything in this space, and estimation of the (dis)advantages this will
bring *across all of glibc's users*?  How many years do you want to wait
until we can remove a deprecated interface?

Re: glibc 2.24 -- Release blockers

[Torvald Riegel]

On Thu, 2016-07-14 at 11:53 +0200, Torvald Riegel wrote:
> On Thu, 2016-07-14 at 11:17 +0200, Andreas Schwab wrote:
> > Torvald Riegel <triegel@redhat.com> writes:
> > 
> > > So, Emacs had plenty of time to fix this, was notified repeatedly, we
> > > helped them as far as they could, and they still don't do anything on
> > > their side?
> > 
> > It is not "they", it is "we and they".
> 
> If you're trying to say that both we and they should work on solving
> this, then I agree.  But it seems that we did whereas they didn't.
> If you're trying to say something less, please be a little less cryptic.

s/less/else/

Re: glibc 2.24 -- Release blockers

[Andreas Schwab]

Torvald Riegel <triegel@redhat.com> writes:

> How many years do you want to wait until we can remove a deprecated
> interface?

Do not wait, do something about it.

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

Re: glibc 2.24 -- Release blockers

[Torvald Riegel]

On Thu, 2016-07-14 at 12:06 +0200, Andreas Schwab wrote:
> Torvald Riegel <triegel@redhat.com> writes:
> 
> > How many years do you want to wait until we can remove a deprecated
> > interface?
> 
> Do not wait, do something about it.

More detail and fewer vague general statements, please.

Are you saying that we can do something that will not make us postpone
Florian's change?  If so, what is it?

If we postpone for whatever reason, we do wait, effectively, for this
other action to take effect.  How long do you estimate do we have to
wait?

Re: glibc 2.24 -- Release blockers

[Andreas Schwab]

Torvald Riegel <triegel@redhat.com> writes:

> Are you saying that we can do something that will not make us postpone
> Florian's change?  If so, what is it?

Help making the failures go away.

> If we postpone for whatever reason, we do wait, effectively, for this
> other action to take effect.  How long do you estimate do we have to
> wait?

Again, do not wait.

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

Re: glibc 2.24 -- Release blockers

[Sinny Kumari]

On Wed, Jul 13, 2016 at 10:52 PM, Dan Horák <dan@danny.cz> wrote:
> On Wed, 13 Jul 2016 19:09:19 +0200
> Florian Weimer <fweimer@redhat.com> wrote:
>
>> On 07/13/2016 03:03 PM, Andreas Schwab wrote:
>> > Florian Weimer <fweimer@redhat.com> writes:
>> >
>> >> I don't see logs for an emacs failure.
>> >
>> > https://build.opensuse.org/package/live_build_log/home:Andreas_Schwab:glibc/emacs/p/ppc64
>> > https://build.opensuse.org/package/live_build_log/home:Andreas_Schwab:glibc/emacs/p/ppc64le
>>
>> Apparently, ppc64 has very aggressive ASLR, and the built-in malloc
>> in Emacs assumes that brk will hand out addresses which are close to
>> the .data section in the executable: It has a direct mapping from
>> addresses to blocks to block information, and if there is a large gap
>> between .data and the heap, the block array is huge.
>>
>> This was not encountered before because Emacs unexec does not support
>> 64-bit AIX, and that is probably the only ppc64 target where Emacs
>> would use its internal malloc.
>>
>> ppc64le probably has the same issue, but the latest upstream release
>> lacks unexec support, so I can't test it there.
>>
>> A workaround is to have lots and lots of memory.
>
> I guess it explains also emacs build failure in rawhide -
> http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=3535700

Indeed, it affected emacs build on ppc64(le) . emacs build failure on
ppc64(le) was giving message
"Memory exhausted--use M-x save-some-buffers then exit and restart Emacs".
Later on, building emacs (tried on ppc64 machine locally) with
increased memory (~15GB) succeeded.


-- 
http://sinny.io/

Re: glibc 2.24 -- Release blockers

[Torvald Riegel]

On Thu, 2016-07-14 at 12:40 +0200, Andreas Schwab wrote:
> Torvald Riegel <triegel@redhat.com> writes:
> 
> > Are you saying that we can do something that will not make us postpone
> > Florian's change?  If so, what is it?
> 
> Help making the failures go away.
> 
> > If we postpone for whatever reason, we do wait, effectively, for this
> > other action to take effect.  How long do you estimate do we have to
> > wait?
> 
> Again, do not wait.

So you're in fact not against shipping Florian's change, provided that
we do what exactly?:

Know that openmpi can be fixed?
Have a patch ready for openmpi?  Have it committed upstream?  Have it be
part of an official openmpi release (though that would probably result
in postponing on our side, thus effectively waiting)?

And similarly for Emacs...

Please be specific if you're suggesting something.  You seem to think
that there is a better way than for us to simply ship Florian's change;
if so, you need to describe this alternative in some detail and argue
why it's better.

Re: glibc 2.24 -- Release blockers

[Andreas Schwab]

Torvald Riegel <triegel@redhat.com> writes:

> Know that openmpi can be fixed?
> Have a patch ready for openmpi?  Have it committed upstream?  Have it be
> part of an official openmpi release (though that would probably result
> in postponing on our side, thus effectively waiting)?

In that order.

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

Re: glibc 2.24 -- Release blockers

[Torvald Riegel]

On Thu, 2016-07-14 at 12:54 +0200, Andreas Schwab wrote:
> Torvald Riegel <triegel@redhat.com> writes:
> 
> > Know that openmpi can be fixed?
> > Have a patch ready for openmpi?  Have it committed upstream?  Have it be
> > part of an official openmpi release (though that would probably result
> > in postponing on our side, thus effectively waiting)?
> 
> In that order.

Yeah, obviously.  You still haven't answer my other questions, though
(eg, what you'd like to suggest as a precondition to including Florian's
changes in a glibc release).
If you can't suggest a concrete plan and assess the likely results of it
(as well as when they would take effect), it's hard to act on this and
simply shipping the changes now seems the best option we have.

Re: glibc 2.24 -- Release blockers

[Paul Eggert]

On 07/13/2016 07:09 PM, Florian Weimer wrote:
> Apparently, ppc64 has very aggressive ASLR, and the built-in malloc in 
> Emacs assumes that brk will hand out addresses which are close to the 
> .data section in the executable: It has a direct mapping from 
> addresses to blocks to block information, and if there is a large gap 
> between .data and the heap, the block array is huge.

Is this the main problem with Emacs and draft-glibc-2.24 malloc? If so, 
can you suggest a patch to Emacs, or a patch to glibc, that will work 
around the problem? For example, can Emacs disable the aggressive ASLR 
somehow? I assume there is a relatively simple fix to Emacs or to glibc 
that could work around this specific problem. If it is a small and 
well-contained change to Emacs, I will advocate that it be squeezed into 
the next Emacs release.

Please bear in mind that I don't have easy access to that platform, and 
that my Internet connection is spotty right now so I can't do a lot of 
debugging myself.

Re: glibc 2.24 -- Release blockers

[Florian Weimer]

On 07/14/2016 12:59 PM, Paul Eggert wrote:
> On 07/13/2016 07:09 PM, Florian Weimer wrote:
>> Apparently, ppc64 has very aggressive ASLR, and the built-in malloc in
>> Emacs assumes that brk will hand out addresses which are close to the
>> .data section in the executable: It has a direct mapping from
>> addresses to blocks to block information, and if there is a large gap
>> between .data and the heap, the block array is huge.
>
> Is this the main problem with Emacs and draft-glibc-2.24 malloc? If so,
> can you suggest a patch to Emacs, or a patch to glibc, that will work
> around the problem? For example, can Emacs disable the aggressive ASLR
> somehow? I assume there is a relatively simple fix to Emacs or to glibc
> that could work around this specific problem.

If my analysis is right, src/gmalloc.c assumes that it can allocate an 
array which contains three words for every BLOCKSIZE (4096) bytes of the 
heap.  This assumption is just not valid on 64-bit architectures. 
Increasing BLOCKSIZE will help somewhat, but this looks rather like a 
core issue with the allocation algorithm.

GDB does this to disable randomization:

       errno = 0;
       personality_orig = personality (0xffffffff);
       if (errno == 0 && !(personality_orig & ADDR_NO_RANDOMIZE))
         {
           personality_set = 1;
           personality (personality_orig | ADDR_NO_RANDOMIZE);
         }
       if (errno != 0 || (personality_set
                          && !(personality (0xffffffff) & 
ADDR_NO_RANDOMIZE)))
         warning (_("Error disabling address space randomization: %s"),
                  safe_strerror (errno));

(0xffffffff should really be -1.)

I do not know if we can do the above early enough before glibc calls 
sbrk internally.  Emacs may have to re-exec itself to apply this 
successfully.  I'm not sure if this approach will be more reliable than 
any kludge that is put into gmalloc.

Florian

Re: glibc 2.24 -- Release blockers

[Florian Weimer]

On 07/14/2016 12:46 PM, Sinny Kumari wrote:

> Indeed, it affected emacs build on ppc64(le) . emacs build failure on
> ppc64(le) was giving message
> "Memory exhausted--use M-x save-some-buffers then exit and restart Emacs".
> Later on, building emacs (tried on ppc64 machine locally) with
> increased memory (~15GB) succeeded.

I expect that the compiled emacs binary will occasionally need ~12 GiB 
of RAM as well.  Could you check if this is the case?

And then run Emacs under “setarch --addr-no-randomize”, to see if it 
makes a difference.

Thanks,
Florian

Re: glibc 2.24 -- Release blockers

[Sinny Kumari]

On Thu, Jul 14, 2016 at 5:00 PM, Florian Weimer <fweimer@redhat.com> wrote:
> On 07/14/2016 12:46 PM, Sinny Kumari wrote:
>
>> Indeed, it affected emacs build on ppc64(le) . emacs build failure on
>> ppc64(le) was giving message
>> "Memory exhausted--use M-x save-some-buffers then exit and restart Emacs".
>> Later on, building emacs (tried on ppc64 machine locally) with
>> increased memory (~15GB) succeeded.
>
>
> I expect that the compiled emacs binary will occasionally need ~12 GiB of
> RAM as well.  Could you check if this is the case?

Yeah, you are right. Running emacs on even a small c file(20 lines)
uses ~12.2 GB
memory

> And then run Emacs under “setarch --addr-no-randomize”, to see if it makes a
> difference.

when I ran emacs under setarch it uses very reasonable amount of memory (few MB)
Command I used was "setarch ppc64 --addr-no-randomize  emacs foo.c"

Re: glibc 2.24 -- Release blockers

[Andreas Schwab]

Paul Eggert <eggert@cs.ucla.edu> writes:

> Is this the main problem with Emacs and draft-glibc-2.24 malloc? If so,
> can you suggest a patch to Emacs, or a patch to glibc, that will work
> around the problem? For example, can Emacs disable the aggressive ASLR
> somehow? I assume there is a relatively simple fix to Emacs or to glibc
> that could work around this specific problem. If it is a small and
> well-contained change to Emacs, I will advocate that it be squeezed into
> the next Emacs release.

Backporting the hybrid-malloc changes should be enough.

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

Re: glibc 2.24 -- Release blockers

[Richard Earnshaw (lists)]

On 14/07/16 13:16, Sinny Kumari wrote:
>> I expect that the compiled emacs binary will occasionally need ~12 GiB of
>> > RAM as well.  Could you check if this is the case?
> Yeah, you are right. Running emacs on even a small c file(20 lines)
> uses ~12.2 GB
> memory
> 

Maybe it's finally time that emacs was renamed as egacs[1]

:-)

R.

[1] Eight Mega^WGiga-Bytes And Constantly Swapping

Re: glibc 2.24 -- Release blockers

[Paul Eggert]

On 07/14/2016 02:23 PM, Andreas Schwab wrote:
> Backporting the hybrid-malloc changes should be enough.

Thanks for the suggestion. I will look into that when I get the time 
(not now, as I have other things to attend to).

If it's easy, could you please give a few details about which 
hybrid-malloc changes you mean? My first thought is to use all the 
hybrid-malloc code in Emacs master, except that 'configure' should be 
modified to use the code only if it detects that it is using a glibc 
version that lacks the recently-removed external symbol. If you have a 
better idea in mind please let us know.

Re: glibc 2.24 -- Release blockers

[Florian Weimer]

On 07/14/2016 03:19 PM, Paul Eggert wrote:
> On 07/14/2016 02:23 PM, Andreas Schwab wrote:
>> Backporting the hybrid-malloc changes should be enough.
>
> Thanks for the suggestion. I will look into that when I get the time
> (not now, as I have other things to attend to).
>
> If it's easy, could you please give a few details about which
> hybrid-malloc changes you mean? My first thought is to use all the
> hybrid-malloc code in Emacs master, except that 'configure' should be
> modified to use the code only if it detects that it is using a glibc
> version that lacks the recently-removed external symbol. If you have a
> better idea in mind please let us know.

You should remove the Doug Lea malloc detection completely, to increase 
test coverage for the new code.  Otherwise, it's difficult to see that 
how we can get the test coverage we want.

If you do not use glibc-internal __ symbols, this approach should work 
with older glibc releases, too.

Thanks,
Florian

Re: glibc 2.24 -- Release blockers

[Florian Weimer]

On 07/14/2016 02:16 PM, Sinny Kumari wrote:
> On Thu, Jul 14, 2016 at 5:00 PM, Florian Weimer <fweimer@redhat.com> wrote:
>> On 07/14/2016 12:46 PM, Sinny Kumari wrote:
>>
>>> Indeed, it affected emacs build on ppc64(le) . emacs build failure on
>>> ppc64(le) was giving message
>>> "Memory exhausted--use M-x save-some-buffers then exit and restart Emacs".
>>> Later on, building emacs (tried on ppc64 machine locally) with
>>> increased memory (~15GB) succeeded.
>>
>>
>> I expect that the compiled emacs binary will occasionally need ~12 GiB of
>> RAM as well.  Could you check if this is the case?
>
> Yeah, you are right. Running emacs on even a small c file(20 lines)
> uses ~12.2 GB
> memory
>
>> And then run Emacs under “setarch --addr-no-randomize”, to see if it makes a
>> difference.
>
> when I ran emacs under setarch it uses very reasonable amount of memory (few MB)
> Command I used was "setarch ppc64 --addr-no-randomize  emacs foo.c"

Thanks, this is helpful.  It seems that Emacs (or more precisely, the 
glibc startup code) does not call brk very early, so it might be 
feasible to call the personality function from __malloc_initialize_hook.

This is just a fallback option if the hybrid malloc changes cannot be 
applied for some reason.

Florian

Re: glibc 2.24 -- Release blockers

[Andreas Schwab]

Paul Eggert <eggert@cs.ucla.edu> writes:

> If it's easy, could you please give a few details about which
> hybrid-malloc changes you mean?

This is what I was trying but doesn't work yet:

https://build.opensuse.org/package/view_file/home:Andreas_Schwab:glibc:emacs/emacs/gmalloc.patch?expand=1

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

Re: glibc 2.24 -- Release blockers

[Paul Eggert]

On 07/14/2016 03:55 PM, Andreas Schwab wrote:
> This is what I was trying but doesn't work yet:
>
> https://build.opensuse.org/package/view_file/home:Andreas_Schwab:glibc:emacs/emacs/gmalloc.patch?expand=1

That looks like it would work eventually, but is probably too large to 
squeeze into the Emacs 25 release as we're pretty late in the 25 release 
process.

Does Emacs master (i.e., post-25 branch) work with glibc when the 
deprecated malloc symbols are removed? If so, that would be a reasonable 
target. But this wouldn't happen until the major Emacs release after 
Emacs 25.

Re: glibc 2.24 -- Release blockers

[Paul Eggert]

On 07/14/2016 03:51 PM, Florian Weimer wrote:
> You should remove the Doug Lea malloc detection completely

I expect such an approach is too much to expect to fold into Emacs 25, 
which is close to release. We could try something along those lines for 
the next major Emacs release, but that won't be for a while.

Re: glibc 2.24 -- Release blockers

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 815 bytes --]

On 07/14/2016 01:27 PM, Florian Weimer wrote:
> GDB does this to disable randomization: 

Thanks. Emacs disables ASLR by invoking the 'setfattr -n user.pax.flags 
-v er' shell command on the Emacs executable before running it ('paxctl 
+a' on older systems). Does this approach not work on ppc64? If not, 
what shell command would work?

As a fallback, Emacs calls personality (PER_LINUX32 | ADDR_NO_RANDOMIZE) 
early on. Perhaps the PER_LINUX32 persona does not work on ppc64? If so, 
please try the attached patch against the emacs-25 branch, on ppc64 and 
ppc64le; this causes Emacs to simply turn on the ADDR_NO_RANDOMIZE flag 
instead. If this doesn't work, perhaps we need to migrate this 
personality-flag-setting into alloc.c's malloc_initialize_hook function, 
so that it operates before 'main' starts up.

[-- Attachment #2: 0001-Port-to-glibc-2.24-pre-release-ppc64.patch --]
[-- Type: text/x-patch, Size: 4237 bytes --]

From 86bbd2df5a8d8833fe6565bfdf5730fb4f429102 Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@cs.ucla.edu>
Date: Fri, 15 Jul 2016 02:20:13 +0200
Subject: [PATCH] Port to glibc 2.24 (pre-release) + ppc64

Inspired by a suggestion by Florian Weimer in:
https://sourceware.org/ml/libc-alpha/2016-07/msg00425.html
* configure.ac (HAVE_PERSONALITY_ADDR_NO_RANDOMIZE):
Rename from HAVE_PERSONALITY_LINUX32, and check for
ADDR_NO_RANDOMIZE (the crucial thing) instead of for LINUX32.
All uses changed.
* src/emacs.c (main) [HAVE_PERSONALITY_ADDR_NO_RANDOMIZE]:
Use ADDR_NO_RANDOMIZE from personality.h rather than inventing the
flag ourselves.  Just set that flag, rather than also setting the
persona.
---
 admin/CPP-DEFINES |  2 +-
 configure.ac      | 20 +++++++++++---------
 src/emacs.c       | 29 +++++++++++++++--------------
 3 files changed, 27 insertions(+), 24 deletions(-)

diff --git a/admin/CPP-DEFINES b/admin/CPP-DEFINES
index 796b57d..d404dee 100644
--- a/admin/CPP-DEFINES
+++ b/admin/CPP-DEFINES
@@ -237,7 +237,7 @@ HAVE_NET_IF_DL_H
 HAVE_NET_IF_H
 HAVE_NLIST_H
 HAVE_OTF_GET_VARIATION_GLYPHS
-HAVE_PERSONALITY_LINUX32
+HAVE_PERSONALITY_ADDR_NO_RANDOMIZE
 HAVE_PNG
 HAVE_PNG_H
 HAVE_POSIX_MEMALIGN
diff --git a/configure.ac b/configure.ac
index 678e98e..9da23d1 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1615,15 +1615,17 @@ AC_CHECK_HEADERS_ONCE(
   sys/resource.h
   sys/utsname.h pwd.h utmp.h util.h)
 
-AC_MSG_CHECKING(if personality LINUX32 can be set)
-AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[#include <sys/personality.h>]], [[personality (PER_LINUX32)]])],
-               emacs_cv_personality_linux32=yes,
-	       emacs_cv_personality_linux32=no)
-AC_MSG_RESULT($emacs_cv_personality_linux32)
-
-if test $emacs_cv_personality_linux32 = yes; then
-  AC_DEFINE(HAVE_PERSONALITY_LINUX32, 1,
-            [Define to 1 if personality LINUX32 can be set.])
+AC_CACHE_CHECK([if personality ADDR_NO_RANDOMIZE flag exists],
+  [emacs_cv_personality_addr_no_randomize],
+  [AC_COMPILE_IFELSE(
+     [AC_LANG_PROGRAM([[#include <sys/personality.h>]],
+		      [[personality (personality (0xffffffff)
+				     | ADDR_NO_RANDOMIZE)]])],
+     [emacs_cv_personality_addr_no_randomize=yes],
+     [emacs_cv_personality_addr_no_randomize=no])])
+if test $emacs_cv_personality_addr_no_randomize = yes; then
+  AC_DEFINE([HAVE_PERSONALITY_ADDR_NO_RANDOMIZE], [1],
+            [Define to 1 if personality flag ADDR_NO_RANDOMIZE exists.])
 fi
 
 # Note that Solaris has sys/sysinfo.h which defines struct
diff --git a/src/emacs.c b/src/emacs.c
index 5c187e7..fcf68a3 100644
--- a/src/emacs.c
+++ b/src/emacs.c
@@ -106,7 +106,7 @@ extern void moncontrol (int mode);
 #include <sys/resource.h>
 #endif
 
-#ifdef HAVE_PERSONALITY_LINUX32
+#ifdef HAVE_PERSONALITY_ADDR_NO_RANDOMIZE
 #include <sys/personality.h>
 #endif
 
@@ -784,24 +784,25 @@ main (int argc, char **argv)
   dumping = !initialized && (strcmp (argv[argc - 1], "dump") == 0
 			     || strcmp (argv[argc - 1], "bootstrap") == 0);
 
-#ifdef HAVE_PERSONALITY_LINUX32
+#ifdef HAVE_PERSONALITY_ADDR_NO_RANDOMIZE
   if (dumping && ! getenv ("EMACS_HEAP_EXEC"))
     {
-      /* Set this so we only do this once.  */
-      xputenv ("EMACS_HEAP_EXEC=true");
-
-      /* A flag to turn off address randomization which is introduced
-         in linux kernel shipped with fedora core 4 */
-#define ADD_NO_RANDOMIZE 0x0040000
-      personality (PER_LINUX32 | ADD_NO_RANDOMIZE);
-#undef  ADD_NO_RANDOMIZE
+      /* Disable address randomization if possible, as it interferes
+	 with dumping.  */
+      int pers = personality (0xffffffff);
+      if (0 <= pers && (pers & ADDR_NO_RANDOMIZE) == 0
+	  && 0 <= personality (pers | ADDR_NO_RANDOMIZE))
+	{
+	  /* Set this so we only do this once.  */
+	  xputenv ("EMACS_HEAP_EXEC=true");
 
-      execvp (argv[0], argv);
+	  execvp (argv[0], argv);
 
-      /* If the exec fails, try to dump anyway.  */
-      emacs_perror (argv[0]);
+	  /* If the exec fails, try to dump anyway.  */
+	  emacs_perror (argv[0]);
+	}
     }
-#endif /* HAVE_PERSONALITY_LINUX32 */
+#endif
 
 #if defined (HAVE_SETRLIMIT) && defined (RLIMIT_STACK) && !defined (CYGWIN)
   /* Extend the stack space available.  Don't do that if dumping,
-- 
2.5.5

Re: glibc 2.24 -- Release blockers

[Florian Weimer]

On 07/15/2016 01:23 AM, Paul Eggert wrote:
> On 07/14/2016 03:51 PM, Florian Weimer wrote:
>> You should remove the Doug Lea malloc detection completely
>
> I expect such an approach is too much to expect to fold into Emacs 25,
> which is close to release. We could try something along those lines for
> the next major Emacs release, but that won't be for a while.

In this case, I suggest *not* to revert the __malloc_initialize_hook 
removal because if we put it back in, Emacs will not use its internal 
malloc, and we get no additional test coverage compared to we had before 
the removal.

Florian

Re: glibc 2.24 -- Release blockers

[Paul Eggert]

On 07/15/2016 10:35 AM, Florian Weimer wrote:
>> I expect such an approach is too much to expect to fold into Emacs 25,
>> which is close to release. We could try something along those lines for
>> the next major Emacs release, but that won't be for a while.
>
> In this case, I suggest *not* to revert the __malloc_initialize_hook 
> removal because if we put it back in, Emacs will not use its internal 
> malloc, and we get no additional test coverage compared to we had 
> before the removal.

By "test coverage" do you mean some testing by a buildbot that builds 
and runs Emacs with bleeding-edge glibc? If so, we shouldn't need to 
remove __malloc_initialize_hook to test this; all that should be needed 
is './configure emacs_cv_var_doug_lea_malloc=no', which causes Emacs 
'configure' to pretend that __malloc_initialize_hook does not exist.

How about the following more-conservative approach instead?

1. Restore __malloc_initialize_hook in glibc.

2. Try to get Emacs 25 to work even when configured with './configure 
emacs_cv_var_doug_lea_malloc=no'. I'm hoping that something like the 
patch I proposed in 
<https://sourceware.org/ml/libc-alpha/2016-07/msg00458.html> is enough 
to do this, and will be accepted by the Emacs maintainers this close to 
a release.

3. If (2) is too ambitious for Emacs 25, get it to work with Emacs 26. 
This should be quite doable, as essentially the same thing is already 
working with Cygwin.

4. Once a stable version of Emacs is published with either (2) or (3), 
then remove __malloc_initialize_hook from glibc.

Re: glibc 2.24 -- Release blockers

[Florian Weimer]

On 07/15/2016 02:45 AM, Paul Eggert wrote:
> On 07/14/2016 01:27 PM, Florian Weimer wrote:
>> GDB does this to disable randomization:
>
> Thanks. Emacs disables ASLR by invoking the 'setfattr -n user.pax.flags
> -v er' shell command on the Emacs executable before running it ('paxctl
> +a' on older systems). Does this approach not work on ppc64? If not,
> what shell command would work?

I have never seen these commands before.  On mainline Linux, you need to 
use setarch (perhaps from a shell script wrapper), and this calls 
personality internally.

Sinny wrapped the build with setarch on ppc64, and it worked.

> As a fallback, Emacs calls personality (PER_LINUX32 | ADDR_NO_RANDOMIZE)
> early on. Perhaps the PER_LINUX32 persona does not work on ppc64? If so,
> please try the attached patch against the emacs-25 branch, on ppc64 and
> ppc64le; this causes Emacs to simply turn on the ADDR_NO_RANDOMIZE flag
> instead. If this doesn't work, perhaps we need to migrate this
> personality-flag-setting into alloc.c's malloc_initialize_hook function,
> so that it operates before 'main' starts up.

The problem is that this happens at dump time only, while the issue 
occurs when running the dumped Emacs.  Furthermore, the personality 
system call (or at least ADDR_NO_RANDOMIZE) only applies to new process 
images, it does not affected the randomization decisions made at load 
time for the current process.

Thanks,
Florian

Re: glibc 2.24 -- Release blockers

[Paul Eggert]

On 07/14/2016 12:46 PM, Sinny Kumari wrote:
>> I guess it explains also emacs build failure in rawhide -
>> >http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=3535700
> Indeed, it affected emacs build on ppc64(le) . emacs build failure on
> ppc64(le) was giving message
> "Memory exhausted--use M-x save-some-buffers then exit and restart Emacs".

The Emacs build procedure is supposed to work around this sort of 
problem by running a command like this:

setfattr -n user.pax.flags -v er temacs

immediately after building temacs. Is there some way to tell whether the 
setfattr command was not installed correctly on that rawhide build? 
Unfortunately the URL above doesn't seem to let me look at the build log.

Re: glibc 2.24 -- Release blockers

[Florian Weimer]

On 07/15/2016 12:12 PM, Paul Eggert wrote:
> On 07/15/2016 10:35 AM, Florian Weimer wrote:
>>> I expect such an approach is too much to expect to fold into Emacs 25,
>>> which is close to release. We could try something along those lines for
>>> the next major Emacs release, but that won't be for a while.
>>
>> In this case, I suggest *not* to revert the __malloc_initialize_hook
>> removal because if we put it back in, Emacs will not use its internal
>> malloc, and we get no additional test coverage compared to we had
>> before the removal.
>
> By "test coverage" do you mean some testing by a buildbot that builds
> and runs Emacs with bleeding-edge glibc? If so, we shouldn't need to
> remove __malloc_initialize_hook to test this; all that should be needed
> is './configure emacs_cv_var_doug_lea_malloc=no', which causes Emacs
> 'configure' to pretend that __malloc_initialize_hook does not exist.

I meant testing by Emacs developers and users who use pre-release builds.

The build barely exercises all the libraries Emacs linked into the 
process image.

> How about the following more-conservative approach instead?
>
> 1. Restore __malloc_initialize_hook in glibc.
>
> 2. Try to get Emacs 25 to work even when configured with './configure
> emacs_cv_var_doug_lea_malloc=no'. I'm hoping that something like the
> patch I proposed in
> <https://sourceware.org/ml/libc-alpha/2016-07/msg00458.html> is enough
> to do this, and will be accepted by the Emacs maintainers this close to
> a release.
>
> 3. If (2) is too ambitious for Emacs 25, get it to work with Emacs 26.
> This should be quite doable, as essentially the same thing is already
> working with Cygwin.
>
> 4. Once a stable version of Emacs is published with either (2) or (3),
> then remove __malloc_initialize_hook from glibc.

What worries me is that your proposal still ignores the deprecation of 
__malloc_initialize_hook.  Deprecation means “stop using this 
interface”.  Yet you advocate to keep using it indefinitely, relying on 
glibc pulling trigger at some unspecified point in the future, rather 
than a new Emacs release (which would be much more predictable).  I 
don't see how this ensures progress towards a solution.

We already thought we were at step 4.

If Emacs does not actually exercises the gmalloc code before it is 
force-activated by a glibc change, we won't know which other bugs are 
lurking there.  For example, gmalloc does not interpose 
malloc_usable_size or __libc_memalign, which could cause issues for the 
libraries it links to.  Or the allocator behavior may trigger 
performance issues with some of the graphics libraries.

Florian

Re: glibc 2.24 -- Release blockers

[Dmitry V. Levin]

[-- Attachment #1: Type: text/plain, Size: 917 bytes --]

On Fri, Jul 15, 2016 at 12:52:33PM +0200, Florian Weimer wrote:
> On 07/15/2016 02:45 AM, Paul Eggert wrote:
> >On 07/14/2016 01:27 PM, Florian Weimer wrote:
> >>GDB does this to disable randomization:
> >
> >Thanks. Emacs disables ASLR by invoking the 'setfattr -n user.pax.flags
> >-v er' shell command on the Emacs executable before running it ('paxctl
> >+a' on older systems). Does this approach not work on ppc64? If not,
> >what shell command would work?
> 
> I have never seen these commands before.  On mainline Linux, you need to 
> use setarch (perhaps from a shell script wrapper), and this calls 
> personality internally.

One has to use personality(personality(0xffffffff)|ADDR_NO_RANDOMIZE)
approach as implemented in GDB, a simple shell script wrapper cannot
implement this.

user.pax.flags is a PaX specific, mainline linux kernel doesn't recognize
this attribute.


-- 
ldv

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: glibc 2.24 -- Release blockers

[Dmitry V. Levin]

[-- Attachment #1: Type: text/plain, Size: 774 bytes --]

On Thu, Jul 14, 2016 at 01:27:54PM +0200, Florian Weimer wrote:
[...]
> GDB does this to disable randomization:
> 
>       errno = 0;
>       personality_orig = personality (0xffffffff);
>       if (errno == 0 && !(personality_orig & ADDR_NO_RANDOMIZE))
>         {
>           personality_set = 1;
>           personality (personality_orig | ADDR_NO_RANDOMIZE);
>         }
>       if (errno != 0 || (personality_set
>                          && !(personality (0xffffffff) & 
> ADDR_NO_RANDOMIZE)))
>         warning (_("Error disabling address space randomization: %s"),
>                  safe_strerror (errno));
> 
> (0xffffffff should really be -1.)

0xffffffff is more portable, see e.g. commit glibc-2.22-637-ge0043e1
for details.


-- 
ldv

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: glibc 2.24 -- Release blockers

[Florian Weimer]

On 07/15/2016 01:37 PM, Dmitry V. Levin wrote:
> On Thu, Jul 14, 2016 at 01:27:54PM +0200, Florian Weimer wrote:
> [...]
>> GDB does this to disable randomization:
>>
>>       errno = 0;
>>       personality_orig = personality (0xffffffff);
>>       if (errno == 0 && !(personality_orig & ADDR_NO_RANDOMIZE))
>>         {
>>           personality_set = 1;
>>           personality (personality_orig | ADDR_NO_RANDOMIZE);
>>         }
>>       if (errno != 0 || (personality_set
>>                          && !(personality (0xffffffff) &
>> ADDR_NO_RANDOMIZE)))
>>         warning (_("Error disabling address space randomization: %s"),
>>                  safe_strerror (errno));
>>
>> (0xffffffff should really be -1.)
>
> 0xffffffff is more portable, see e.g. commit glibc-2.22-637-ge0043e1
> for details.

Right, it's also documented in the manual page.

Thanks,
Florian

Re: glibc 2.24 -- Release blockers

[Florian Weimer]

On 07/15/2016 01:32 PM, Dmitry V. Levin wrote:
> On Fri, Jul 15, 2016 at 12:52:33PM +0200, Florian Weimer wrote:
>> On 07/15/2016 02:45 AM, Paul Eggert wrote:
>>> On 07/14/2016 01:27 PM, Florian Weimer wrote:
>>>> GDB does this to disable randomization:
>>>
>>> Thanks. Emacs disables ASLR by invoking the 'setfattr -n user.pax.flags
>>> -v er' shell command on the Emacs executable before running it ('paxctl
>>> +a' on older systems). Does this approach not work on ppc64? If not,
>>> what shell command would work?
>>
>> I have never seen these commands before.  On mainline Linux, you need to
>> use setarch (perhaps from a shell script wrapper), and this calls
>> personality internally.
>
> One has to use personality(personality(0xffffffff)|ADDR_NO_RANDOMIZE)
> approach as implemented in GDB, a simple shell script wrapper cannot
> implement this.

Would you please elaborate?

In my testing, the ADDR_NO_RANDOMIZE bit is inherited by subprocesses 
(which is a problem in itself, of course, because it disables hardening 
in network helpers used by Emacs).

Thanks,
Florian

Re: glibc 2.24 -- Release blockers

[Dmitry V. Levin]

[-- Attachment #1: Type: text/plain, Size: 1356 bytes --]

On Fri, Jul 15, 2016 at 01:47:33PM +0200, Florian Weimer wrote:
> On 07/15/2016 01:32 PM, Dmitry V. Levin wrote:
> >On Fri, Jul 15, 2016 at 12:52:33PM +0200, Florian Weimer wrote:
> >>On 07/15/2016 02:45 AM, Paul Eggert wrote:
> >>>On 07/14/2016 01:27 PM, Florian Weimer wrote:
> >>>>GDB does this to disable randomization:
> >>>
> >>>Thanks. Emacs disables ASLR by invoking the 'setfattr -n user.pax.flags
> >>>-v er' shell command on the Emacs executable before running it ('paxctl
> >>>+a' on older systems). Does this approach not work on ppc64? If not,
> >>>what shell command would work?
> >>
> >>I have never seen these commands before.  On mainline Linux, you need to
> >>use setarch (perhaps from a shell script wrapper), and this calls
> >>personality internally.
> >
> >One has to use personality(personality(0xffffffff)|ADDR_NO_RANDOMIZE)
> >approach as implemented in GDB, a simple shell script wrapper cannot
> >implement this.
> 
> Would you please elaborate?

I mean setarch(8) is not versatile enough, it just rewrites personality
flags.  For example,

$ strace -qq -epersonality \
  setarch linux64 --sticky-timeouts setarch linux64 --addr-no-randomize true
personality(PER_LINUX|STICKY_TIMEOUTS)  = 0 (PER_LINUX)
personality(PER_LINUX|ADDR_NO_RANDOMIZE) = 0x4000000 (PER_LINUX|STICKY_TIMEOUTS)


-- 
ldv

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: glibc 2.24 -- Release blockers

[Florian Weimer]

On 07/15/2016 02:14 PM, Dmitry V. Levin wrote:

> I mean setarch(8) is not versatile enough, it just rewrites personality
> flags.  For example,
>
> $ strace -qq -epersonality \
>   setarch linux64 --sticky-timeouts setarch linux64 --addr-no-randomize true
> personality(PER_LINUX|STICKY_TIMEOUTS)  = 0 (PER_LINUX)
> personality(PER_LINUX|ADDR_NO_RANDOMIZE) = 0x4000000 (PER_LINUX|STICKY_TIMEOUTS)

Ah, you mean that setarch is not cumulative and removes personality 
flags which have not been specified?

Thanks,
Florian

Re: glibc 2.24 -- Release blockers

[Dmitry V. Levin]

[-- Attachment #1: Type: text/plain, Size: 661 bytes --]

On Fri, Jul 15, 2016 at 02:23:37PM +0200, Florian Weimer wrote:
> On 07/15/2016 02:14 PM, Dmitry V. Levin wrote:
> 
> >I mean setarch(8) is not versatile enough, it just rewrites personality
> >flags.  For example,
> >
> >$ strace -qq -epersonality \
> >  setarch linux64 --sticky-timeouts setarch linux64 --addr-no-randomize 
> >  true
> >personality(PER_LINUX|STICKY_TIMEOUTS)  = 0 (PER_LINUX)
> >personality(PER_LINUX|ADDR_NO_RANDOMIZE) = 0x4000000 
> >(PER_LINUX|STICKY_TIMEOUTS)
> 
> Ah, you mean that setarch is not cumulative and removes personality 
> flags which have not been specified?

Yes, setarch is not cumulative.


-- 
ldv

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: glibc 2.24 -- Release blockers

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 684 bytes --]

On 07/15/2016 12:52 PM, Florian Weimer wrote:
> The problem is that this happens at dump time only, while the issue 
> occurs when running the dumped Emacs. Furthermore, the personality 
> system call (or at least ADDR_NO_RANDOMIZE) only applies to new 
> process images, it does not affected the randomization decisions made 
> at load time for the current process.

I wrote a revised patch to emacs-25 (attached) which I hope addresses 
the former issue, by disabling randomization in both the undumped and 
dumped Emacs when Emacs uses its own allocator. Emacs already addresses 
the latter issue by re-execing itself when it changes personalities, so 
this shouldn't be an issue.

[-- Attachment #2: 0001-Port-to-glibc-2.24-pre-release-ppc64.patch --]
[-- Type: text/x-patch, Size: 5060 bytes --]

From 6c7e7b23abe495da5ac83be6cf8e714d3160af45 Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@cs.ucla.edu>
Date: Fri, 15 Jul 2016 13:07:09 +0200
Subject: [PATCH] Port to glibc 2.24 (pre-release) + ppc64

Inspired by a suggestion by Florian Weimer in:
https://sourceware.org/ml/libc-alpha/2016-07/msg00425.html
* configure.ac (HAVE_PERSONALITY_ADDR_NO_RANDOMIZE):
Rename from HAVE_PERSONALITY_LINUX32, and check for
ADDR_NO_RANDOMIZE (the crucial thing) instead of for LINUX32.
All uses changed.
* src/emacs.c (main) [HAVE_PERSONALITY_ADDR_NO_RANDOMIZE]:
Use ADDR_NO_RANDOMIZE from personality.h rather than inventing the
flag ourselves.  Just set that flag, rather than also setting the
persona.  Do all this earlier, so as to avoid problems with calls
to brk in the interim.  When doing it, avoid functions like
putenv that may allocate memory.
---
 admin/CPP-DEFINES |  2 +-
 configure.ac      | 20 +++++++++++---------
 src/emacs.c       | 55 ++++++++++++++++++++++++++++++++-----------------------
 3 files changed, 44 insertions(+), 33 deletions(-)

diff --git a/admin/CPP-DEFINES b/admin/CPP-DEFINES
index 796b57d..d404dee 100644
--- a/admin/CPP-DEFINES
+++ b/admin/CPP-DEFINES
@@ -237,7 +237,7 @@ HAVE_NET_IF_DL_H
 HAVE_NET_IF_H
 HAVE_NLIST_H
 HAVE_OTF_GET_VARIATION_GLYPHS
-HAVE_PERSONALITY_LINUX32
+HAVE_PERSONALITY_ADDR_NO_RANDOMIZE
 HAVE_PNG
 HAVE_PNG_H
 HAVE_POSIX_MEMALIGN
diff --git a/configure.ac b/configure.ac
index 678e98e..9da23d1 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1615,15 +1615,17 @@ AC_CHECK_HEADERS_ONCE(
   sys/resource.h
   sys/utsname.h pwd.h utmp.h util.h)
 
-AC_MSG_CHECKING(if personality LINUX32 can be set)
-AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[#include <sys/personality.h>]], [[personality (PER_LINUX32)]])],
-               emacs_cv_personality_linux32=yes,
-	       emacs_cv_personality_linux32=no)
-AC_MSG_RESULT($emacs_cv_personality_linux32)
-
-if test $emacs_cv_personality_linux32 = yes; then
-  AC_DEFINE(HAVE_PERSONALITY_LINUX32, 1,
-            [Define to 1 if personality LINUX32 can be set.])
+AC_CACHE_CHECK([if personality ADDR_NO_RANDOMIZE flag exists],
+  [emacs_cv_personality_addr_no_randomize],
+  [AC_COMPILE_IFELSE(
+     [AC_LANG_PROGRAM([[#include <sys/personality.h>]],
+		      [[personality (personality (0xffffffff)
+				     | ADDR_NO_RANDOMIZE)]])],
+     [emacs_cv_personality_addr_no_randomize=yes],
+     [emacs_cv_personality_addr_no_randomize=no])])
+if test $emacs_cv_personality_addr_no_randomize = yes; then
+  AC_DEFINE([HAVE_PERSONALITY_ADDR_NO_RANDOMIZE], [1],
+            [Define to 1 if personality flag ADDR_NO_RANDOMIZE exists.])
 fi
 
 # Note that Solaris has sys/sysinfo.h which defines struct
diff --git a/src/emacs.c b/src/emacs.c
index 5c187e7..901bc0c 100644
--- a/src/emacs.c
+++ b/src/emacs.c
@@ -106,7 +106,7 @@ extern void moncontrol (int mode);
 #include <sys/resource.h>
 #endif
 
-#ifdef HAVE_PERSONALITY_LINUX32
+#ifdef HAVE_PERSONALITY_ADDR_NO_RANDOMIZE
 #include <sys/personality.h>
 #endif
 
@@ -674,6 +674,37 @@ main (int argc, char **argv)
 
   stack_base = &dummy;
 
+  dumping = !initialized && (strcmp (argv[argc - 1], "dump") == 0
+			     || strcmp (argv[argc - 1], "bootstrap") == 0);
+
+#ifdef HAVE_PERSONALITY_ADDR_NO_RANDOMIZE
+
+  /* True if address randomization interferes with memory allocaiton.  */
+# ifdef HYBRID_MALLOC
+  bool disable_aslr = dumping;
+# elif defined SYSTEM_MALLOC || defined DOUG_LEA_MALLOC
+  bool disable_aslr = false;
+# else
+  bool disable_aslr = true;
+# endif
+
+  if (disable_aslr)
+    {
+      int pers = personality (0xffffffff);
+      if (! (pers & ADDR_NO_RANDOMIZE)
+	  && 0 <= personality (pers | ADDR_NO_RANDOMIZE))
+	{
+	  /* Address randomization was enabled, but is now disabled.
+	     Re-execute Emacs to get a clean slate.  */
+	  execvp (argv[0], argv);
+
+	  /* If the exec fails, warn the user and then try without a
+	     clean slate.  */
+	  fprintf (stderr, "%s: %s\n", argv[0], strerror (errno));
+	}
+    }
+#endif
+
 #ifndef CANNOT_DUMP
   might_dump = !initialized;
 #endif
@@ -781,28 +812,6 @@ main (int argc, char **argv)
         }
     }
 
-  dumping = !initialized && (strcmp (argv[argc - 1], "dump") == 0
-			     || strcmp (argv[argc - 1], "bootstrap") == 0);
-
-#ifdef HAVE_PERSONALITY_LINUX32
-  if (dumping && ! getenv ("EMACS_HEAP_EXEC"))
-    {
-      /* Set this so we only do this once.  */
-      xputenv ("EMACS_HEAP_EXEC=true");
-
-      /* A flag to turn off address randomization which is introduced
-         in linux kernel shipped with fedora core 4 */
-#define ADD_NO_RANDOMIZE 0x0040000
-      personality (PER_LINUX32 | ADD_NO_RANDOMIZE);
-#undef  ADD_NO_RANDOMIZE
-
-      execvp (argv[0], argv);
-
-      /* If the exec fails, try to dump anyway.  */
-      emacs_perror (argv[0]);
-    }
-#endif /* HAVE_PERSONALITY_LINUX32 */
-
 #if defined (HAVE_SETRLIMIT) && defined (RLIMIT_STACK) && !defined (CYGWIN)
   /* Extend the stack space available.  Don't do that if dumping,
      since some systems (e.g. DJGPP) might define a smaller stack
-- 
2.5.5

Re: glibc 2.24 -- Release blockers

[Paul Eggert]

On 07/15/2016 01:17 PM, Florian Weimer wrote:
> On 07/15/2016 12:12 PM, Paul Eggert wrote:
>> On 07/15/2016 10:35 AM, Florian Weimer wrote:
>>>> I expect such an approach is too much to expect to fold into Emacs 25,
>>>> which is close to release. We could try something along those lines 
>>>> for
>>>> the next major Emacs release, but that won't be for a while.
>>>
>>> In this case, I suggest *not* to revert the __malloc_initialize_hook
>>> removal because if we put it back in, Emacs will not use its internal
>>> malloc, and we get no additional test coverage compared to we had
>>> before the removal.
>>
>> By "test coverage" do you mean some testing by a buildbot that builds
>> and runs Emacs with bleeding-edge glibc? If so, we shouldn't need to
>> remove __malloc_initialize_hook to test this; all that should be needed
>> is './configure emacs_cv_var_doug_lea_malloc=no', which causes Emacs
>> 'configure' to pretend that __malloc_initialize_hook does not exist.
>
> I meant testing by Emacs developers and users who use pre-release builds.
>
> The build barely exercises all the libraries Emacs linked into the 
> process image.
>
>> How about the following more-conservative approach instead?
>>
>> 1. Restore __malloc_initialize_hook in glibc.
>>
>> 2. Try to get Emacs 25 to work even when configured with './configure
>> emacs_cv_var_doug_lea_malloc=no'. I'm hoping that something like the
>> patch I proposed in
>> <https://sourceware.org/ml/libc-alpha/2016-07/msg00458.html> is enough
>> to do this, and will be accepted by the Emacs maintainers this close to
>> a release.
>>
>> 3. If (2) is too ambitious for Emacs 25, get it to work with Emacs 26.
>> This should be quite doable, as essentially the same thing is already
>> working with Cygwin.
>>
>> 4. Once a stable version of Emacs is published with either (2) or (3),
>> then remove __malloc_initialize_hook from glibc.
>
> What worries me is that your proposal still ignores the deprecation of 
> __malloc_initialize_hook.

OK, then let's add a new step 3.5, as follows.

3.5.  Call the result of step (2) or (3) version N. (Presumably N will 
be 25 or 26.) In Emacs version N+1, disable use of the system 
__malloc_initialize_hook. That is, Emacs will no longer set or use the 
system __malloc_initialize_hook.

This should address the deprecation concern.

> We already thought we were at step 4.

Clearly we were wrong. :-)

Re: glibc 2.24 -- Release blockers

[Sinny Kumari]

On Fri, Jul 15, 2016 at 6:42 PM, Paul Eggert <eggert@cs.ucla.edu> wrote:
> On 07/15/2016 12:52 PM, Florian Weimer wrote:
>>
>> The problem is that this happens at dump time only, while the issue occurs
>> when running the dumped Emacs. Furthermore, the personality system call (or
>> at least ADDR_NO_RANDOMIZE) only applies to new process images, it does not
>> affected the randomization decisions made at load time for the current
>> process.
>
>
> I wrote a revised patch to emacs-25 (attached) which I hope addresses the
> former issue, by disabling randomization in both the undumped and dumped
> Emacs when Emacs uses its own allocator. Emacs already addresses the latter
> issue by re-execing itself when it changes personalities, so this shouldn't
> be an issue.

After applying this patch in emacs-25.0.95, it builds and run fine on ppc64(le),
Fedora rawhide. It no longer consumes memory in GB !
For reference, scratch build link is
http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=3538651

Re: glibc 2.24 -- Release blockers

[Sinny Kumari]

On Fri, Jul 15, 2016 at 7:49 PM, Sinny Kumari <ksinny@gmail.com> wrote:
> On Fri, Jul 15, 2016 at 6:42 PM, Paul Eggert <eggert@cs.ucla.edu> wrote:
>> On 07/15/2016 12:52 PM, Florian Weimer wrote:
>>>
>>> The problem is that this happens at dump time only, while the issue occurs
>>> when running the dumped Emacs. Furthermore, the personality system call (or
>>> at least ADDR_NO_RANDOMIZE) only applies to new process images, it does not
>>> affected the randomization decisions made at load time for the current
>>> process.
>>
>>
>> I wrote a revised patch to emacs-25 (attached) which I hope addresses the
>> former issue, by disabling randomization in both the undumped and dumped
>> Emacs when Emacs uses its own allocator. Emacs already addresses the latter
>> issue by re-execing itself when it changes personalities, so this shouldn't
>> be an issue.
>
> After applying this patch in emacs-25.0.95, it builds and run fine on ppc64(le),
> Fedora rawhide. It no longer consumes memory in GB !
> For reference, scratch build link is
> http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=3538651

Also, output of "cat /proc/self/personality" inside emacs shell
contains value 00040000

Re: glibc 2.24 -- Release blockers

[Florian Weimer]

On 07/15/2016 04:28 PM, Sinny Kumari wrote:
> On Fri, Jul 15, 2016 at 7:49 PM, Sinny Kumari <ksinny@gmail.com> wrote:
>> On Fri, Jul 15, 2016 at 6:42 PM, Paul Eggert <eggert@cs.ucla.edu> wrote:
>>> On 07/15/2016 12:52 PM, Florian Weimer wrote:
>>>>
>>>> The problem is that this happens at dump time only, while the issue occurs
>>>> when running the dumped Emacs. Furthermore, the personality system call (or
>>>> at least ADDR_NO_RANDOMIZE) only applies to new process images, it does not
>>>> affected the randomization decisions made at load time for the current
>>>> process.
>>>
>>>
>>> I wrote a revised patch to emacs-25 (attached) which I hope addresses the
>>> former issue, by disabling randomization in both the undumped and dumped
>>> Emacs when Emacs uses its own allocator. Emacs already addresses the latter
>>> issue by re-execing itself when it changes personalities, so this shouldn't
>>> be an issue.
>>
>> After applying this patch in emacs-25.0.95, it builds and run fine on ppc64(le),
>> Fedora rawhide. It no longer consumes memory in GB !
>> For reference, scratch build link is
>> http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=3538651
>
> Also, output of "cat /proc/self/personality" inside emacs shell
> contains value 00040000

Thanks!

This means that new processes spawned by Emacs will have ASLR disable.

Paul, I think the patch should call personality again after it is active 
for the Emacs process, to disable it for subprocesses.

Florian

Re: glibc 2.24 -- Release blockers

[Torvald Riegel]

On Fri, 2016-07-15 at 15:24 +0200, Paul Eggert wrote:
> On 07/15/2016 01:17 PM, Florian Weimer wrote:
> > On 07/15/2016 12:12 PM, Paul Eggert wrote:
> >> On 07/15/2016 10:35 AM, Florian Weimer wrote:
> >>>> I expect such an approach is too much to expect to fold into Emacs 25,
> >>>> which is close to release. We could try something along those lines 
> >>>> for
> >>>> the next major Emacs release, but that won't be for a while.
> >>>
> >>> In this case, I suggest *not* to revert the __malloc_initialize_hook
> >>> removal because if we put it back in, Emacs will not use its internal
> >>> malloc, and we get no additional test coverage compared to we had
> >>> before the removal.
> >>
> >> By "test coverage" do you mean some testing by a buildbot that builds
> >> and runs Emacs with bleeding-edge glibc? If so, we shouldn't need to
> >> remove __malloc_initialize_hook to test this; all that should be needed
> >> is './configure emacs_cv_var_doug_lea_malloc=no', which causes Emacs
> >> 'configure' to pretend that __malloc_initialize_hook does not exist.
> >
> > I meant testing by Emacs developers and users who use pre-release builds.
> >
> > The build barely exercises all the libraries Emacs linked into the 
> > process image.
> >
> >> How about the following more-conservative approach instead?
> >>
> >> 1. Restore __malloc_initialize_hook in glibc.
> >>
> >> 2. Try to get Emacs 25 to work even when configured with './configure
> >> emacs_cv_var_doug_lea_malloc=no'. I'm hoping that something like the
> >> patch I proposed in
> >> <https://sourceware.org/ml/libc-alpha/2016-07/msg00458.html> is enough
> >> to do this, and will be accepted by the Emacs maintainers this close to
> >> a release.
> >>
> >> 3. If (2) is too ambitious for Emacs 25, get it to work with Emacs 26.
> >> This should be quite doable, as essentially the same thing is already
> >> working with Cygwin.
> >>
> >> 4. Once a stable version of Emacs is published with either (2) or (3),
> >> then remove __malloc_initialize_hook from glibc.
> >
> > What worries me is that your proposal still ignores the deprecation of 
> > __malloc_initialize_hook.
> 
> OK, then let's add a new step 3.5, as follows.
> 
> 3.5.  Call the result of step (2) or (3) version N. (Presumably N will 
> be 25 or 26.) In Emacs version N+1, disable use of the system 
> __malloc_initialize_hook. That is, Emacs will no longer set or use the 
> system __malloc_initialize_hook.
> 
> This should address the deprecation concern.
> 
> > We already thought we were at step 4.
> 
> Clearly we were wrong. :-)

Unless I misunderstand your proposal, this wouldn't change anything
regarding Emacs currently using a symbol that has been deprecated.

Re: glibc 2.24 -- Release blockers

[Paul Eggert]

On 07/15/2016 06:00 PM, Torvald Riegel wrote:
>> 3.5.  Call the result of step (2) or (3) version N. (Presumably N will
>> >be 25 or 26.) In Emacs version N+1, disable use of the system
>> >__malloc_initialize_hook. That is, Emacs will no longer set or use the
>> >system __malloc_initialize_hook.
>> >
>> >This should address the deprecation concern.
>> >
>>> > >We already thought we were at step 4.
>> >
>> >Clearly we were wrong. :-)
> Unless I misunderstand your proposal, this wouldn't change anything
> regarding Emacs currently using a symbol that has been deprecated.
>

The idea is that after step 3.5 is executed, Emacs won't be using 
deprecated malloc-related symbols. Emacs may supply its own malloc 
implementation that provides the deprecated functionality, presumably 
using Emacs-specific names, but Emacs won't be using glibc's 
__malloc_initialize_hook, malloc_set_state, etc. As I understand it, 
this should satisfy Florian's concern.

Re: glibc 2.24 -- Release blockers

[Torvald Riegel]

On Fri, 2016-07-15 at 19:04 +0200, Paul Eggert wrote:
> On 07/15/2016 06:00 PM, Torvald Riegel wrote:
> >> 3.5.  Call the result of step (2) or (3) version N. (Presumably N will
> >> >be 25 or 26.) In Emacs version N+1, disable use of the system
> >> >__malloc_initialize_hook. That is, Emacs will no longer set or use the
> >> >system __malloc_initialize_hook.
> >> >
> >> >This should address the deprecation concern.
> >> >
> >>> > >We already thought we were at step 4.
> >> >
> >> >Clearly we were wrong. :-)
> > Unless I misunderstand your proposal, this wouldn't change anything
> > regarding Emacs currently using a symbol that has been deprecated.
> >
> 
> The idea is that after step 3.5 is executed, Emacs won't be using 
> deprecated malloc-related symbols. Emacs may supply its own malloc 
> implementation that provides the deprecated functionality, presumably 
> using Emacs-specific names, but Emacs won't be using glibc's 
> __malloc_initialize_hook, malloc_set_state, etc. As I understand it, 
> this should satisfy Florian's concern.

But this still had the unspecified N release number.

Re: glibc 2.24 -- Release blockers

[Paul Eggert]

On 07/15/2016 08:41 PM, Torvald Riegel wrote:
> But this still had the unspecified N release number.
N will eventually arrive. We all want N to be as small as possible, but 
there are practical limits.

Re: glibc 2.24 -- Release blockers

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 495 bytes --]

On 07/15/2016 04:19 PM, Sinny Kumari wrote:
> After applying this patch in emacs-25.0.95, it builds and run fine on ppc64(le),
> Fedora rawhide. It no longer consumes memory in GB !

Thanks. Unfortunately the patch fails on other platforms, including 
Fedora 23 x86-64. So please try the attached, more-conservative patch 
instead. I have installed this into the Emacs master branch; it should 
also work in the emacs-25 branch, but I'd like further testing before I 
start advocating for that.

[-- Attachment #2: 0001-Port-to-glibc-2.24-pre-release-ppc64.patch --]
[-- Type: text/x-patch, Size: 4225 bytes --]

From 2921f054ef3a2dca9a2cc48bbe956751a49b9d19 Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@cs.ucla.edu>
Date: Fri, 15 Jul 2016 13:07:09 +0200
Subject: [PATCH] Port to glibc 2.24 (pre-release) + ppc64

Inspired by a suggestion by Florian Weimer in:
https://sourceware.org/ml/libc-alpha/2016-07/msg00425.html
* configure.ac (HAVE_PERSONALITY_ADDR_NO_RANDOMIZE):
Rename from HAVE_PERSONALITY_LINUX32, and check for
ADDR_NO_RANDOMIZE (the crucial thing) instead of for LINUX32.
All uses changed.
* src/emacs.c (main) [HAVE_PERSONALITY_ADDR_NO_RANDOMIZE]:
Use ADDR_NO_RANDOMIZE from personality.h rather than inventing the
flag ourselves.  Just set that flag, rather than also setting the
persona.  When doing it, avoid functions like putenv that may
allocate memory.
---
 admin/CPP-DEFINES |  2 +-
 configure.ac      | 20 +++++++++++---------
 src/emacs.c       | 30 ++++++++++++++----------------
 3 files changed, 26 insertions(+), 26 deletions(-)

diff --git a/admin/CPP-DEFINES b/admin/CPP-DEFINES
index c7ec8ce..5e6146b 100644
--- a/admin/CPP-DEFINES
+++ b/admin/CPP-DEFINES
@@ -233,7 +233,7 @@ HAVE_NET_IF_DL_H
 HAVE_NET_IF_H
 HAVE_NLIST_H
 HAVE_OTF_GET_VARIATION_GLYPHS
-HAVE_PERSONALITY_LINUX32
+HAVE_PERSONALITY_ADDR_NO_RANDOMIZE
 HAVE_PNG
 HAVE_PNG_H
 HAVE_POSIX_MEMALIGN
diff --git a/configure.ac b/configure.ac
index dd1af5b..c94ecb6 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1643,15 +1643,17 @@ AC_DEFUN
   sys/resource.h
   sys/utsname.h pwd.h utmp.h util.h)
 
-AC_MSG_CHECKING(if personality LINUX32 can be set)
-AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[#include <sys/personality.h>]], [[personality (PER_LINUX32)]])],
-               emacs_cv_personality_linux32=yes,
-	       emacs_cv_personality_linux32=no)
-AC_MSG_RESULT($emacs_cv_personality_linux32)
-
-if test $emacs_cv_personality_linux32 = yes; then
-  AC_DEFINE(HAVE_PERSONALITY_LINUX32, 1,
-            [Define to 1 if personality LINUX32 can be set.])
+AC_CACHE_CHECK([for ADDR_NO_RANDOMIZE],
+  [emacs_cv_personality_addr_no_randomize],
+  [AC_COMPILE_IFELSE(
+     [AC_LANG_PROGRAM([[#include <sys/personality.h>]],
+		      [[personality (personality (0xffffffff)
+				     | ADDR_NO_RANDOMIZE)]])],
+     [emacs_cv_personality_addr_no_randomize=yes],
+     [emacs_cv_personality_addr_no_randomize=no])])
+if test $emacs_cv_personality_addr_no_randomize = yes; then
+  AC_DEFINE([HAVE_PERSONALITY_ADDR_NO_RANDOMIZE], [1],
+            [Define to 1 if personality flag ADDR_NO_RANDOMIZE exists.])
 fi
 
 # Note that Solaris has sys/sysinfo.h which defines struct
diff --git a/src/emacs.c b/src/emacs.c
index bb85733..b221984 100644
--- a/src/emacs.c
+++ b/src/emacs.c
@@ -112,7 +112,7 @@ extern void moncontrol (int mode);
 #include <sys/resource.h>
 #endif
 
-#ifdef HAVE_PERSONALITY_LINUX32
+#ifdef HAVE_PERSONALITY_ADDR_NO_RANDOMIZE
 #include <sys/personality.h>
 #endif
 
@@ -796,24 +796,22 @@ main (int argc, char **argv)
   dumping = !initialized && (strcmp (argv[argc - 1], "dump") == 0
 			     || strcmp (argv[argc - 1], "bootstrap") == 0);
 
-#ifdef HAVE_PERSONALITY_LINUX32
-  if (dumping && ! getenv ("EMACS_HEAP_EXEC"))
+#ifdef HAVE_PERSONALITY_ADDR_NO_RANDOMIZE
+  if (dumping)
     {
-      /* Set this so we only do this once.  */
-      xputenv ("EMACS_HEAP_EXEC=true");
-
-      /* A flag to turn off address randomization which is introduced
-         in linux kernel shipped with fedora core 4 */
-#define ADD_NO_RANDOMIZE 0x0040000
-      personality (PER_LINUX32 | ADD_NO_RANDOMIZE);
-#undef  ADD_NO_RANDOMIZE
-
-      execvp (argv[0], argv);
+      int pers = personality (0xffffffff);
+      if (! (pers & ADDR_NO_RANDOMIZE)
+	  && 0 <= personality (pers | ADDR_NO_RANDOMIZE))
+	{
+	  /* Address randomization was enabled, but is now disabled.
+	     Re-execute Emacs to get a clean slate.  */
+	  execvp (argv[0], argv);
 
-      /* If the exec fails, try to dump anyway.  */
-      emacs_perror (argv[0]);
+	  /* If the exec fails, warn and then try without a clean slate.  */
+	  perror (argv[0]);
+	}
     }
-#endif /* HAVE_PERSONALITY_LINUX32 */
+#endif
 
 #if defined (HAVE_SETRLIMIT) && defined (RLIMIT_STACK) && !defined (CYGWIN)
   /* Extend the stack space available.  Don't do that if dumping,
-- 
2.5.5

Re: glibc 2.24 -- Release blockers

[Paul Eggert]

On 07/15/2016 04:33 PM, Florian Weimer wrote:
>
> This means that new processes spawned by Emacs will have ASLR disable.

Yes, on second thought that's too drastic. The patch I just now 
installed into master (and sent via email to this list) doesn't do that. 
It disables ASLR only in temacs, not in ordinary Emacs invocations. 
(Enabling ASLR in ordinary Emacs broke the Emacs build on Fedora 23 
x86-64.) This is how Emacs has worked for many years; although you 
indicated that it might not work with bleeding-edge glibc in some 
situations, I don't understand why not, and would like to understand it 
better before changing this aspect of Emacs..

Re: glibc 2.24 -- Release blockers

[Zack Weinberg]

On Fri, Jul 15, 2016 at 4:00 PM, Paul Eggert <eggert@cs.ucla.edu> wrote:
> On 07/15/2016 04:33 PM, Florian Weimer wrote:
>>
>> This means that new processes spawned by Emacs will have ASLR disable.
>
> Yes, on second thought that's too drastic. The patch I just now installed
> into master (and sent via email to this list) doesn't do that. It disables
> ASLR only in temacs, not in ordinary Emacs invocations. (Enabling ASLR in
> ordinary Emacs broke the Emacs build on Fedora 23 x86-64.) This is how Emacs
> has worked for many years; although you indicated that it might not work
> with bleeding-edge glibc in some situations, I don't understand why not, and
> would like to understand it better before changing this aspect of Emacs..

I don't have affected hardware to test this on, but the phenomenon
seems pretty clear to me.  Emacs' bundled malloc assumes that the data
segment is contiguous with, or at least close to, the memory region
controlled by brk().  This is not true with ASLR enabled, on certain
architectures (e.g. ppc64).

That assumption is a fundamental design constraint, rather than
something done to support dumping.  Therefore, if the bundled malloc
is used in the "ordinary Emacs", ASLR must be disabled in the
"ordinary Emacs"; disabling it in temacs is not enough.  It looks like
HYBRID_MALLOC mode would avoid the problem, but from upthread, that
mode isn't ready for prime time yet...?

zw

Re: glibc 2.24 -- Release blockers

[Sinny Kumari]

On Sat, Jul 16, 2016 at 1:26 AM, Paul Eggert <eggert@cs.ucla.edu> wrote:
> On 07/15/2016 04:19 PM, Sinny Kumari wrote:
>>
>> After applying this patch in emacs-25.0.95, it builds and run fine on
>> ppc64(le),
>> Fedora rawhide. It no longer consumes memory in GB !
>
>
> Thanks. Unfortunately the patch fails on other platforms, including Fedora
> 23 x86-64. So please try the attached, more-conservative patch instead. I
> have installed this into the Emacs master branch; it should also work in the
> emacs-25 branch, but I'd like further testing before I start advocating for
> that.

Sadly, after applying this patch, build fails for Fedora rawhide on
both ppc64 and
ppc64le due to Memory exhausted problem.

koji scratch build link -
http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=3540001
Direct link to build log of ppc64 -
http://ppc.koji.fedoraproject.org/kojifiles/work/tasks/2/3540002/build.log

Re: glibc 2.24 -- Release blockers

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 1028 bytes --]

On 07/18/2016 08:49 AM, Sinny Kumari wrote:
> Sadly, after applying this patch, build fails for Fedora rawhide on
> both ppc64 and
> ppc64le due to Memory exhausted problem.
>
> koji scratch build link -
> http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=3540001
> Direct link to build log of ppc64 -
> http://ppc.koji.fedoraproject.org/kojifiles/work/tasks/2/3540002/build.log

Thanks for checking. I thought about it for a bit and came up with the 
attached Emacs patch instead, which works for me on x86-64 and x86 
Fedora 23. Can you please try it on ppc64? I assume __PPC64__ is 
defined, if not please feel to substitute the proper symbol and please 
let me know what that symbol is.

This patch applies to Emacs's emacs-25 branch; if it works I plan to 
port it to Emacs's master branch. The patch does not implement Florian's 
idea of re-enabling ASLR before Emacs execs a child process, as I want 
to keep the emacs-25 patch as simple as possible. However, I do plan to 
try Florian's idea in the master branch.


[-- Attachment #2: 0001-Port-to-glibc-2.24-pre-release-ppc64.patch --]
[-- Type: text/x-patch, Size: 5043 bytes --]

From da7966b5870fc65897bda5adaa403bd5e5740238 Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@cs.ucla.edu>
Date: Tue, 19 Jul 2016 15:23:14 +0200
Subject: [PATCH] Port to glibc 2.24 (pre-release) + ppc64

Inspired by a suggestion by Florian Weimer in:
https://sourceware.org/ml/libc-alpha/2016-07/msg00425.html
* configure.ac (HAVE_PERSONALITY_ADDR_NO_RANDOMIZE):
Rename from HAVE_PERSONALITY_LINUX32, and check for
ADDR_NO_RANDOMIZE (the crucial thing) instead of for LINUX32.
All uses changed.
* src/emacs.c (main) [HAVE_PERSONALITY_ADDR_NO_RANDOMIZE]:
Use ADDR_NO_RANDOMIZE from personality.h rather than inventing the
flag ourselves.  Just set that flag, rather than also setting the
persona.  Do all this earlier, so as to avoid problems with calls
to brk in the interim.  When doing it, avoid functions like
putenv that may allocate memory.  Special case for __PPC64__,
which needs ASLR disabled in dumped Emacs too.
---
 admin/CPP-DEFINES |  2 +-
 configure.ac      | 20 +++++++++++---------
 src/emacs.c       | 53 ++++++++++++++++++++++++++++++-----------------------
 3 files changed, 42 insertions(+), 33 deletions(-)

diff --git a/admin/CPP-DEFINES b/admin/CPP-DEFINES
index 796b57d..d404dee 100644
--- a/admin/CPP-DEFINES
+++ b/admin/CPP-DEFINES
@@ -237,7 +237,7 @@ HAVE_NET_IF_DL_H
 HAVE_NET_IF_H
 HAVE_NLIST_H
 HAVE_OTF_GET_VARIATION_GLYPHS
-HAVE_PERSONALITY_LINUX32
+HAVE_PERSONALITY_ADDR_NO_RANDOMIZE
 HAVE_PNG
 HAVE_PNG_H
 HAVE_POSIX_MEMALIGN
diff --git a/configure.ac b/configure.ac
index 678e98e..9da23d1 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1615,15 +1615,17 @@ AC_CHECK_HEADERS_ONCE(
   sys/resource.h
   sys/utsname.h pwd.h utmp.h util.h)
 
-AC_MSG_CHECKING(if personality LINUX32 can be set)
-AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[#include <sys/personality.h>]], [[personality (PER_LINUX32)]])],
-               emacs_cv_personality_linux32=yes,
-	       emacs_cv_personality_linux32=no)
-AC_MSG_RESULT($emacs_cv_personality_linux32)
-
-if test $emacs_cv_personality_linux32 = yes; then
-  AC_DEFINE(HAVE_PERSONALITY_LINUX32, 1,
-            [Define to 1 if personality LINUX32 can be set.])
+AC_CACHE_CHECK([if personality ADDR_NO_RANDOMIZE flag exists],
+  [emacs_cv_personality_addr_no_randomize],
+  [AC_COMPILE_IFELSE(
+     [AC_LANG_PROGRAM([[#include <sys/personality.h>]],
+		      [[personality (personality (0xffffffff)
+				     | ADDR_NO_RANDOMIZE)]])],
+     [emacs_cv_personality_addr_no_randomize=yes],
+     [emacs_cv_personality_addr_no_randomize=no])])
+if test $emacs_cv_personality_addr_no_randomize = yes; then
+  AC_DEFINE([HAVE_PERSONALITY_ADDR_NO_RANDOMIZE], [1],
+            [Define to 1 if personality flag ADDR_NO_RANDOMIZE exists.])
 fi
 
 # Note that Solaris has sys/sysinfo.h which defines struct
diff --git a/src/emacs.c b/src/emacs.c
index 5c187e7..b0e5a05 100644
--- a/src/emacs.c
+++ b/src/emacs.c
@@ -106,7 +106,7 @@ extern void moncontrol (int mode);
 #include <sys/resource.h>
 #endif
 
-#ifdef HAVE_PERSONALITY_LINUX32
+#ifdef HAVE_PERSONALITY_ADDR_NO_RANDOMIZE
 #include <sys/personality.h>
 #endif
 
@@ -674,6 +674,35 @@ main (int argc, char **argv)
 
   stack_base = &dummy;
 
+  dumping = !initialized && (strcmp (argv[argc - 1], "dump") == 0
+			     || strcmp (argv[argc - 1], "bootstrap") == 0);
+
+#ifdef HAVE_PERSONALITY_ADDR_NO_RANDOMIZE
+
+  /* True if address randomization interferes with memory allocaiton.  */
+# ifdef __PPC64__
+  bool disable_aslr = true;
+# else
+  bool disable_aslr = dumping;
+# endif
+
+  if (disable_aslr)
+    {
+      int pers = personality (0xffffffff);
+      if (! (pers & ADDR_NO_RANDOMIZE)
+	  && 0 <= personality (pers | ADDR_NO_RANDOMIZE))
+	{
+	  /* Address randomization was enabled, but is now disabled.
+	     Re-execute Emacs to get a clean slate.  */
+	  execvp (argv[0], argv);
+
+	  /* If the exec fails, warn the user and then try without a
+	     clean slate.  */
+	  fprintf (stderr, "%s: %s\n", argv[0], strerror (errno));
+	}
+    }
+#endif
+
 #ifndef CANNOT_DUMP
   might_dump = !initialized;
 #endif
@@ -781,28 +810,6 @@ main (int argc, char **argv)
         }
     }
 
-  dumping = !initialized && (strcmp (argv[argc - 1], "dump") == 0
-			     || strcmp (argv[argc - 1], "bootstrap") == 0);
-
-#ifdef HAVE_PERSONALITY_LINUX32
-  if (dumping && ! getenv ("EMACS_HEAP_EXEC"))
-    {
-      /* Set this so we only do this once.  */
-      xputenv ("EMACS_HEAP_EXEC=true");
-
-      /* A flag to turn off address randomization which is introduced
-         in linux kernel shipped with fedora core 4 */
-#define ADD_NO_RANDOMIZE 0x0040000
-      personality (PER_LINUX32 | ADD_NO_RANDOMIZE);
-#undef  ADD_NO_RANDOMIZE
-
-      execvp (argv[0], argv);
-
-      /* If the exec fails, try to dump anyway.  */
-      emacs_perror (argv[0]);
-    }
-#endif /* HAVE_PERSONALITY_LINUX32 */
-
 #if defined (HAVE_SETRLIMIT) && defined (RLIMIT_STACK) && !defined (CYGWIN)
   /* Extend the stack space available.  Don't do that if dumping,
      since some systems (e.g. DJGPP) might define a smaller stack
-- 
2.5.5

Re: glibc 2.24 -- Release blockers

[Paul Eggert]

On 07/15/2016 11:19 PM, Zack Weinberg wrote:
> Emacs' bundled malloc assumes that the data
> segment is contiguous with, or at least close to, the memory region
> controlled by brk().  This is not true with ASLR enabled, on certain
> architectures (e.g. ppc64).

Unfortunately Emacs with glibc 2.22 malloc seems to be making the 
opposite assumption, at least on Fedora 23 x86-64. That is, if I disable 
ASLR in an ordinary (dumped) Emacs with glibc 2.22, Emacs stops working 
(I don't know why). This leads me to be leery of disabling ASLR in an 
ordinary Emacs, except for platforms like ppc64 where it seems to be 
necessary. The patch I just sent in therefore has a special case for 
__PPC64__. Quite a hack, but without knowing the problem better I don't 
know what other hack to try.

> It looks like
> HYBRID_MALLOC mode would avoid the problem, but from upthread, that
> mode isn't ready for prime time yet...?


Yes, my sense is that HYBRID_MALLOC should work but it is not ready for 
emacs-25. It is already enabled in the master branch and we should be 
able to get it to work with ppc64 too, at least assuming the patch I 
just emailed here works on emacs-25.

Re: glibc 2.24 -- Release blockers

[Sinny Kumari]

On Tue, Jul 19, 2016 at 8:32 PM, Paul Eggert <eggert@cs.ucla.edu> wrote:
> On 07/18/2016 08:49 AM, Sinny Kumari wrote:
>>
>> Sadly, after applying this patch, build fails for Fedora rawhide on
>> both ppc64 and
>> ppc64le due to Memory exhausted problem.
>>
>> koji scratch build link -
>> http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=3540001
>> Direct link to build log of ppc64 -
>> http://ppc.koji.fedoraproject.org/kojifiles/work/tasks/2/3540002/build.log
>
>
> Thanks for checking. I thought about it for a bit and came up with the
> attached Emacs patch instead, which works for me on x86-64 and x86 Fedora
> 23. Can you please try it on ppc64? I assume __PPC64__ is defined, if not
> please feel to substitute the proper symbol and please let me know what that
> symbol is.

Yes, __PPC64__ macro is defined.
Applied this patch and emacs builds fine for Fedora rawhide [1] and
Fedora 24 [2] for
ppc64(le) arches.
Ran updated emacs on Fedora rawhide, ppc64 box and it uses few megabytes memory.
Checked content of  /proc/self/personality inside emacs shell and it has value
00040000.

[1] http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=3544685
[2] http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=3544697

Re: glibc 2.24 -- Release blockers

[Paul Eggert]

Thanks for checking it. I have pushed a patch into Emacs master which 
incorporates this idea plus Florian's suggestion about restoring 
randomization in child processes; please give it a try.

Also, I have filed a bug report and proposed patch for the emacs-25 
branch here:

http://bugs.gnu.org/24033

and will try to convince the Emacs maintainers that the patch is worth 
installing into emacs-25 even though it's pretty late in emacs-25's 
release schedule.

Re: glibc 2.24 -- Release blockers

[Sinny Kumari]

On Wed, Jul 20, 2016 at 1:37 PM, Paul Eggert <eggert@cs.ucla.edu> wrote:
> Thanks for checking it. I have pushed a patch into Emacs master which
> incorporates this idea plus Florian's suggestion about restoring
> randomization in child processes; please give it a try.

So far, didn't succeeded to test patch pushed in master branch.
At first trying to reproduce memory issue by building source tar obtained
from commit bf5ddded70c11edaf3514b25da27fc71cfb8e965 and
098d29af10a869e65a36804579f16edc49aabcfa . But didn't see memory
issue in those commits.
I think, this is because ./configure gives following output:

 Should Emacs use the GNU version of malloc?             no (only
before dumping)
 Should Emacs use a relocating allocator for buffers?    no

> Also, I have filed a bug report and proposed patch for the emacs-25 branch
> here:
>
> http://bugs.gnu.org/24033

Thanks Paul for proposing it for emacs-25. I tested final patch proposed at
http://debbugs.gnu.org/cgi/bugreport.cgi?bug=24033#11 and it works on
ppc64(le) architectures.

> and will try to convince the Emacs maintainers that the patch is worth
> installing into emacs-25 even though it's pretty late in emacs-25's release
> schedule.

Glad to see that fix will be included in emacs-25.

Re: glibc 2.24 -- Release blockers

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 681 bytes --]

On 07/20/2016 09:03 AM, Sinny Kumari wrote:
> Applied this patch and emacs builds fine for Fedora rawhide [1] and
> Fedora 24 [2] for
> ppc64(le) arches.

Thanks for checking. I have one more thing to ask. As can be seen in 
<http:// bugs.gnu.org/24033>, an Emacs maintainer had some qualms about 
the size of the emacs-25 patch I sent you earlier, so I came up with the 
attached, more-conservative patch instead. Can you please try the 
revised patch on your ppc64 platform with draft glibc 2.24? If it works 
for you, I have the OK to install this patch into the emacs-25 branch, 
and I hope this resolves the pressing compatibility issues between draft 
glibc 2.24 and Emacs.


[-- Attachment #2: 0001-Port-to-glibc-2.24-pre-release-ppc64.patch --]
[-- Type: text/x-patch, Size: 2199 bytes --]

From 37bc57cf803f2ab0b7a7844730ebc17eb43e942d Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@cs.ucla.edu>
Date: Tue, 19 Jul 2016 15:23:14 +0200
Subject: [PATCH] Port to glibc 2.24 (pre-release) + ppc64

Backport from master (Bug#24033).
Inspired by a suggestion by Florian Weimer in:
https://sourceware.org/ml/libc-alpha/2016-07/msg00425.html
* src/emacs.c (main) [__PPC64__]:
Special case for __PPC64__, which needs ASLR disabled in
dumped Emacs too.
---
 src/emacs.c | 24 ++++++++++++++++++++++--
 1 file changed, 22 insertions(+), 2 deletions(-)

diff --git a/src/emacs.c b/src/emacs.c
index 5c187e7..2480dfc 100644
--- a/src/emacs.c
+++ b/src/emacs.c
@@ -674,6 +674,26 @@ main (int argc, char **argv)
 
   stack_base = &dummy;
 
+#if defined HAVE_PERSONALITY_LINUX32 && defined __PPC64__
+  /* This code partly duplicates the HAVE_PERSONALITY_LINUX32 code
+     below.  This duplication is planned to be fixed in a later
+     Emacs release.  */
+# define ADD_NO_RANDOMIZE 0x0040000
+  int pers = personality (0xffffffff);
+  if (! (pers & ADD_NO_RANDOMIZE)
+      && 0 <= personality (pers | ADD_NO_RANDOMIZE))
+    {
+      /* Address randomization was enabled, but is now disabled.
+	 Re-execute Emacs to get a clean slate.  */
+      execvp (argv[0], argv);
+
+      /* If the exec fails, warn the user and then try without a
+	 clean slate.  */
+      perror (argv[0]);
+    }
+# undef ADD_NO_RANDOMIZE
+#endif
+
 #ifndef CANNOT_DUMP
   might_dump = !initialized;
 #endif
@@ -784,7 +804,7 @@ main (int argc, char **argv)
   dumping = !initialized && (strcmp (argv[argc - 1], "dump") == 0
 			     || strcmp (argv[argc - 1], "bootstrap") == 0);
 
-#ifdef HAVE_PERSONALITY_LINUX32
+#if defined HAVE_PERSONALITY_LINUX32 && !defined __PPC64__
   if (dumping && ! getenv ("EMACS_HEAP_EXEC"))
     {
       /* Set this so we only do this once.  */
@@ -801,7 +821,7 @@ main (int argc, char **argv)
       /* If the exec fails, try to dump anyway.  */
       emacs_perror (argv[0]);
     }
-#endif /* HAVE_PERSONALITY_LINUX32 */
+#endif
 
 #if defined (HAVE_SETRLIMIT) && defined (RLIMIT_STACK) && !defined (CYGWIN)
   /* Extend the stack space available.  Don't do that if dumping,
-- 
2.5.5

Re: glibc 2.24 -- Release blockers

[Paul Eggert]

On 07/21/2016 12:44 PM, Sinny Kumari wrote:
> Thanks Paul for proposing it for emacs-25. I tested final patch proposed at
> http://debbugs.gnu.org/cgi/bugreport.cgi?bug=24033#11  and it works on
> ppc64(le) architectures.

Ah, sorry, our emails crossed paths. Thanks for checking, and I'll 
install the patch into emacs-25.

Re: glibc 2.24 -- Release blockers

[Aurelien Jarno]

On 2016-07-21 13:25, Paul Eggert wrote:
> On 07/21/2016 12:44 PM, Sinny Kumari wrote:
> > Thanks Paul for proposing it for emacs-25. I tested final patch proposed at
> > http://debbugs.gnu.org/cgi/bugreport.cgi?bug=24033#11  and it works on
> > ppc64(le) architectures.
> 
> Ah, sorry, our emails crossed paths. Thanks for checking, and I'll install
> the patch into emacs-25.

Thanks for doing so. I got a look at the emacs git, and found your patch
to backport to emacs-24. However as said on the wiki [1], it is also
necessary to compile src/gmalloc.c with -ffreestanding. I have done that
and it indeed works fine. I have found it is necessary to do so starting
with GCC 5, however I haven't found any corresponding in the emacs git.
Do you have any pointer to an upstream patch?

Aurelien

[1] https://sourceware.org/glibc/wiki/Release/2.24#Known_Issues

-- 
Aurelien Jarno                          GPG: 4096R/1DDD8C9B
aurelien@aurel32.net                 http://www.aurel32.net

Re: glibc 2.24 -- Release blockers

[Andreas Schwab]

On So, Aug 07 2016, Aurelien Jarno <aurelien@aurel32.net> wrote:

> Thanks for doing so. I got a look at the emacs git, and found your patch
> to backport to emacs-24. However as said on the wiki [1], it is also
> necessary to compile src/gmalloc.c with -ffreestanding. I have done that
> and it indeed works fine. I have found it is necessary to do so starting
> with GCC 5, however I haven't found any corresponding in the emacs git.
> Do you have any pointer to an upstream patch?

Do you mean commit 4b1436b?

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: glibc 2.24 -- Release blockers

[Paul Eggert]

Aurelien Jarno wrote:
> I got a look at the emacs git, and found your patch
> to backport to emacs-24. However as said on the wiki [1], it is also
> necessary to compile src/gmalloc.c with -ffreestanding. I have done that
> and it indeed works fine.

Is that still really needed with the emacs-25 branch? As Andreas notes, it has a 
different way of avoiding the calloc issue. What happens if you build the 
emacs-25 branch without using -ffreestanding?

Re: glibc 2.24 -- Release blockers

[Aurelien Jarno]

On 2016-08-07 14:19, Andreas Schwab wrote:
> On So, Aug 07 2016, Aurelien Jarno <aurelien@aurel32.net> wrote:
> 
> > Thanks for doing so. I got a look at the emacs git, and found your patch
> > to backport to emacs-24. However as said on the wiki [1], it is also
> > necessary to compile src/gmalloc.c with -ffreestanding. I have done that
> > and it indeed works fine. I have found it is necessary to do so starting
> > with GCC 5, however I haven't found any corresponding in the emacs git.
> > Do you have any pointer to an upstream patch?
> 
> Do you mean commit 4b1436b?

Thanks for the hint, the wiki talked about -ffreestanding, so I was
looking for a commit containing it.

Aurelien

-- 
Aurelien Jarno                          GPG: 4096R/1DDD8C9B
aurelien@aurel32.net                 http://www.aurel32.net

Re: glibc 2.24 -- Release blockers

[Aurelien Jarno]

On 2016-08-07 12:35, Paul Eggert wrote:
> Aurelien Jarno wrote:
> > I got a look at the emacs git, and found your patch
> > to backport to emacs-24. However as said on the wiki [1], it is also
> > necessary to compile src/gmalloc.c with -ffreestanding. I have done that
> > and it indeed works fine.
> 
> Is that still really needed with the emacs-25 branch? As Andreas notes, it
> has a different way of avoiding the calloc issue. What happens if you build
> the emacs-25 branch without using -ffreestanding?

The emacs-25 branch is probably fine, I was searching the commit to
backport to emacs-24 and was looking for a commit with -ffreestanding.
The emacs-25 branch is indeed fine.

Aurelien

-- 
Aurelien Jarno                          GPG: 4096R/1DDD8C9B
aurelien@aurel32.net                 http://www.aurel32.net