[Bug malloc/6527] Malloc alignment insufficient for PowerPC

[Bug malloc/6527] Malloc alignment insufficient for PowerPC

[jsm28 at gcc dot gnu.org]

http://sourceware.org/bugzilla/show_bug.cgi?id=6527

Joseph Myers <jsm28 at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
          Component|libc                        |malloc

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug malloc/6527] Malloc alignment insufficient for PowerPC

[siddhesh at redhat dot com]

http://sourceware.org/bugzilla/show_bug.cgi?id=6527

Siddhesh Poyarekar <siddhesh at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
                 CC|                            |siddhesh at redhat dot com
         Resolution|                            |FIXED

--- Comment #2 from Siddhesh Poyarekar <siddhesh at redhat dot com> 2012-10-23 08:00:41 UTC ---
This patch is already in. Closing this as fixed.

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug malloc/6527] Malloc alignment insufficient for PowerPC

[jsm28 at gcc dot gnu.org]

http://sourceware.org/bugzilla/show_bug.cgi?id=6527

Joseph Myers <jsm28 at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |REOPENED
         Resolution|FIXED                       |

--- Comment #3 from Joseph Myers <jsm28 at gcc dot gnu.org> 2012-10-23 12:46:03 UTC ---
This is *not* fixed.  The correct alignment is enabled only for !SHLIB_COMPAT
(libc, GLIBC_2_0, GLIBC_2_16) - that is, for new ports (x32).  No-one has yet
done the compatibility work required for new binaries on powerpc32 to get the
correct alignment without breaking existing emacs binaries.

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug malloc/6527] Malloc alignment insufficient for PowerPC

[jsm28 at gcc dot gnu.org]

http://sourceware.org/bugzilla/show_bug.cgi?id=6527

--- Comment #4 from Joseph Myers <jsm28 at gcc dot gnu.org> 2013-01-09 21:22:04 UTC ---
I think in fact such a change should be made for 32-bit x86 as well, if we can
find a way to do it without breaking existing emacs binaries, because the
alignment requirement of _Decimal128 is 16-byte on 32-bit x86, and decimal
floating-point is defined in terms of standard types added to the C set.

(32-bit x86 would need a separate MALLOC_ALIGNMENT definition because
_Decimal128 isn't a type malloc.c considers when working out the required
alignment, and most architectures don't support it.)

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug malloc/6527] [powerpc] Malloc alignment insufficient for PowerPC

[jsm28 at gcc dot gnu.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=6527

Joseph Myers <jsm28 at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|Malloc alignment            |[powerpc] Malloc alignment
                   |insufficient for PowerPC    |insufficient for PowerPC

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug malloc/6527] [powerpc] Malloc alignment insufficient for PowerPC

[jouko.orava at iki dot fi]

https://sourceware.org/bugzilla/show_bug.cgi?id=6527

Jouko Orava <jouko.orava at iki dot fi> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jouko.orava at iki dot fi

--- Comment #5 from Jouko Orava <jouko.orava at iki dot fi> ---
Insufficient alignment in malloc()/calloc() pointers is now plaguing SSE4 and
AVX support on x86-64, too; see GCC bugzilla bugs #55916 and related #60088 for
examples. GCC does provide the __BIGGEST_ALIGNMENT__ preprocessor macro, and
future glibc might prefer to use that (rounded up to a power of two) for
alignment, to avoid these issues.

Due to existing malloc_get_state/malloc_set_state users (emacs),
we cannot just change the alignment.

Currently, existing malloc_get_state dumps contain versions 1 to 4, major
version number being zero. Ignoring the difficulties in implementation, we
could use a specific major state version number for new states, with minor
version number indicating the native/minimum alignment shift (log2 of
MALLOC_ALIGNMENT). Then, the case of zero major, 1 to 4 minor state,
becomes just a special case; alignment being the current one used for that
architecture in glibc.

As to the implementation, the only sane way I can think of is to add tests with
unlikely jumps to a slow path to the beginning of every public API call,
in case the "current alignment" is not the one optimized for at glibc compile
time.

(Note that this would allow an user to use a recompiled glibc with different
 optimal alignment, without breaking existing state dumps.)

Is there anything in this approach that makes it obviously/immediately
unacceptable?

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug malloc/6527] [powerpc] Malloc alignment insufficient for PowerPC

[joseph at codesourcery dot com]

http://sourceware.org/bugzilla/show_bug.cgi?id=6527

--- Comment #6 from joseph at codesourcery dot com <joseph at codesourcery dot com> ---
I don't think malloc should try to provide alignment for vector types, 
only C standard types and types from C standard extensions defined in a 
way that makes them types for which malloc should provide alignment; other 
functions such as aligned_alloc should be used when bigger alignment is 
required.  That leaves powerpc (long double) and 32-bit x86 (_Decimal128, 
__float128 / future standard _Float128) as the only affected cases, I 
think.

The most recently discussed suggestion for dealing with malloc_get_state / 
malloc_set_state is that malloc_set_state should mark all previous 
allocations as unfreeable (so attempts to free a previous allocation are 
no-ops), limiting the amount malloc needs to do with the state generated 
by malloc with different alignment.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug malloc/6527] [powerpc] Malloc alignment insufficient for PowerPC

[jouko.orava at iki dot fi]

https://sourceware.org/bugzilla/show_bug.cgi?id=6527

--- Comment #7 from Jouko Orava <jouko.orava at iki dot fi> ---
(In reply to joseph@codesourcery.com from comment #6)
> I don't think malloc should try to provide alignment for vector types,

Pity. It causes surprising segfaults for existing code,
and few programmers seem to understand the cause or mechanism.

On affected architectures, modifying malloc() is the best option;
the amount of existing code that assumes malloc() returns a pointer
suitable for even vector types, is just too large to manage otherwise.

I guess it is best if I take the easy way out, and instead of
trying to fix the issue, I shall work around it. I can trivially
provide a small library to override the glibc allocator, providing
sufficiently aligned pointers even for vector types.

GNU Fortran already uses malloc()/calloc() wrappers, so it should
be easy to patch separately at the GCC end (although, to be honest,
I don't think that will be acceptable to GCC folks, either).

> The most recently discussed suggestion for dealing with malloc_get_state / 
> malloc_set_state is that malloc_set_state should mark all previous 
> allocations as unfreeable

I'm sure emacs users will love that feature.

I'd thought that a slow path that uses e.g. separate metadata pool, so that it
can trivially (but slowly) support any alignment, would have been a more
sensible approach. After all, most users I know would prefer slower operation
to out of memory errors. Most Linux distributions do not have user limits that
would stop a memory-hogging userspace application from triggering the OOM
killer. (Or didn't it come up what would happen if the malloc_get_state() -
malloc_set_state() cycle would be repeated a few times?)

Anyway, thank you for your time, and for reading my overly verbose messages;
I'm sorry I couldn't be of more use with respect to this bug.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug malloc/6527] [powerpc] Malloc alignment insufficient for PowerPC

[joseph at codesourcery dot com]

http://sourceware.org/bugzilla/show_bug.cgi?id=6527

--- Comment #8 from joseph at codesourcery dot com <joseph at codesourcery dot com> ---
On Fri, 7 Feb 2014, jouko.orava at iki dot fi wrote:

> > The most recently discussed suggestion for dealing with malloc_get_state / 
> > malloc_set_state is that malloc_set_state should mark all previous 
> > allocations as unfreeable
> 
> I'm sure emacs users will love that feature.
> 
> I'd thought that a slow path that uses e.g. separate metadata pool, so that it
> can trivially (but slowly) support any alignment, would have been a more
> sensible approach. After all, most users I know would prefer slower operation
> to out of memory errors. Most Linux distributions do not have user limits that
> would stop a memory-hogging userspace application from triggering the OOM
> killer. (Or didn't it come up what would happen if the malloc_get_state() -
> malloc_set_state() cycle would be repeated a few times?)

It was asserted that malloc_set_state is only called very early in emacs 
startup and no significant amount of the previously allocated memory gets 
freed anyway.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug malloc/6527] [powerpc] Malloc alignment insufficient for PowerPC

[jouko.orava at iki dot fi]

https://sourceware.org/bugzilla/show_bug.cgi?id=6527

Jouko Orava <jouko.orava at iki dot fi> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|jouko.orava at iki dot fi          |

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug malloc/6527] [powerpc] Malloc alignment insufficient for PowerPC

[alisdairm at me dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=6527

Alisdair Meredith <alisdairm at me dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |alisdairm at me dot com

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug malloc/6527] [powerpc] Malloc alignment insufficient for PowerPC

[guillaume at morinfr dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=6527

Guillaume Morin <guillaume at morinfr dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |guillaume at morinfr dot org

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug malloc/6527] [powerpc] Malloc alignment insufficient for PowerPC

[azanella at linux dot vnet.ibm.com]

https://sourceware.org/bugzilla/show_bug.cgi?id=6527

Adhemerval Zanella Netto <azanella at linux dot vnet.ibm.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |azanella at linux dot vnet.ibm.com

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug malloc/6527] [powerpc] Malloc alignment insufficient for PowerPC

[joost.vandevondele at mat dot ethz.ch]

https://sourceware.org/bugzilla/show_bug.cgi?id=6527

Joost VandeVondele <joost.vandevondele at mat dot ethz.ch> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |joost.vandevondele at mat dot ethz
                   |                            |.ch

--- Comment #9 from Joost VandeVondele <joost.vandevondele at mat dot ethz.ch> ---
(In reply to joseph@codesourcery.com from comment #6)
> I don't think malloc should try to provide alignment for vector types, 
> only C standard types and types from C standard extensions defined in a 
> way that makes them types for which malloc should provide alignment; other 
> functions such as aligned_alloc should be used when bigger alignment is 
> required.  That leaves powerpc (long double) and 32-bit x86 (_Decimal128, 
> __float128 / future standard _Float128) as the only affected cases, I 
> think.

but now that gcc is autovectorizing code, it might lead to binaries with
results that randomly change depending on the alignment of the pointer returned
by malloc. See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64247

-- 
You are receiving this mail because:
You are on the CC list for the bug.

Re: [Bug malloc/6527] [powerpc] Malloc alignment insufficient for PowerPC

[Ondřej Bílka]

On Thu, Dec 11, 2014 at 08:30:13PM +0000, joost.vandevondele at mat dot ethz.ch wrote:
> --- Comment #9 from Joost VandeVondele <joost.vandevondele at mat dot ethz.ch> ---
> (In reply to joseph@codesourcery.com from comment #6)
> > I don't think malloc should try to provide alignment for vector types, 
> > only C standard types and types from C standard extensions defined in a 
> > way that makes them types for which malloc should provide alignment; other 
> > functions such as aligned_alloc should be used when bigger alignment is 
> > required.  That leaves powerpc (long double) and 32-bit x86 (_Decimal128, 
> > __float128 / future standard _Float128) as the only affected cases, I 
> > think.
> 
> but now that gcc is autovectorizing code, it might lead to binaries with
> results that randomly change depending on the alignment of the pointer returned
> by malloc. See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64247
> 
As I wrote in link that is not problem of malloc. Results there vary
because compiler can reassociate floating operations in fortran code.
If you compiled with different gcc version or copied array to one 
misaligned by 4 bytes you would also get diffent result.