[Bug libc/5553] If mmap fails malloc may assert.

[Bug libc/5553] If mmap fails malloc may assert.

[fweimer at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=5553

Florian Weimer <fweimer at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
              Flags|                            |security-

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug libc/5553] If mmap fails malloc may assert.

[pasky at suse dot cz]

------- Additional Comments From pasky at suse dot cz  2010-02-25 10:30 -------
Thanks. I have provided a changelog entry, but you are right that I forgot to
update the year.

-- 


http://sourceware.org/bugzilla/show_bug.cgi?id=5553

------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.

[Bug libc/5553] If mmap fails malloc may assert.

[drepper at redhat dot com]

------- Additional Comments From drepper at redhat dot com  2010-02-24 23:45 -------
I've applied the patch.  But you failed to update the copyright year and
provided no ChangeLog entry.

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|REOPENED                    |RESOLVED
         Resolution|                            |FIXED


http://sourceware.org/bugzilla/show_bug.cgi?id=5553

------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.

[Bug libc/5553] If mmap fails malloc may assert.

[pasky at suse dot cz]

------- Additional Comments From pasky at suse dot cz  2010-02-24 11:35 -------
Created an attachment (id=4624)
 --> (http://sourceware.org/bugzilla/attachment.cgi?id=4624&action=view)
proposed patch


-- 


http://sourceware.org/bugzilla/show_bug.cgi?id=5553

------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.

[Bug libc/5553] If mmap fails malloc may assert.

[pasky at suse dot cz]

------- Additional Comments From pasky at suse dot cz  2010-02-24 11:33 -------
valloc() implementation has the same problem.

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |REOPENED
         Resolution|FIXED                       |


http://sourceware.org/bugzilla/show_bug.cgi?id=5553

------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.

[Bug libc/5553] If mmap fails malloc may assert.

[drow at sources dot redhat dot com]

------- Additional Comments From drow at sources dot redhat dot com  2008-01-09 20:49 -------
Thanks.  Is your change to memalign correct?

       ar_ptr = arena_get2(ar_ptr->next ? ar_ptr : 0, bytes);
+      (void)mutex_unlock(&ar_ptr->mutex);
       if(ar_ptr) {
         p = _int_memalign(ar_ptr, alignment, bytes);
         (void)mutex_unlock(&ar_ptr->mutex);

if (!ar_ptr), segfault, else double unlock.

-- 


http://sourceware.org/bugzilla/show_bug.cgi?id=5553

------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.

[Bug libc/5553] If mmap fails malloc may assert.

[drepper at redhat dot com]

------- Additional Comments From drepper at redhat dot com  2008-01-09 20:35 -------
The patch is correct but incomplete.  I've checked in the complete version.

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|                            |FIXED


http://sourceware.org/bugzilla/show_bug.cgi?id=5553

------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.

[Bug libc/5553] If mmap fails malloc may assert.

[drow at sources dot redhat dot com]

------- Additional Comments From drow at sources dot redhat dot com  2008-01-08 17:00 -------
Created an attachment (id=2187)
 --> (http://sourceware.org/bugzilla/attachment.cgi?id=2187&action=view)
Proposed fix.

Sorry about the broken bug; I hit enter accidentally.

The failing assertion is this one:
  assert(!victim || chunk_is_mmapped(mem2chunk(victim)) ||
	 ar_ptr == arena_for_chunk(mem2chunk(victim)));

GDB's bigcore.c testcase triggers this assertion on several PowerPC systems I
tested.  It starts by a malloc too large for the system to satisfy; when
_int_malloc fails, malloc creates and tries a new arena.  This arena is saved
as the default arena for the main thread so future allocations come from that
arena instead of the main one.

Later the test tries a malloc which can be met by mmap.  Eventually mmap
returns ENOMEM after a number of similar allocations:

mmap(NULL, 134221824, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0xa810c000
mmap(NULL, 134221824, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0xb010d000
mmap(NULL, 134221824, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
-1 ENOMEM (Cannot alloca
te memory)
mmap(NULL, 134221824, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
-1 ENOMEM (Cannot alloca
te memory)
brk(0x180c1000) 			= 0x180c1000

I do not know why brk succeeded (another seven times, all 0x8000000 bytes) when
mmap failed.  But the result is a non-mmapped chunk allocated from the main
arena.	The assert checks the thread's specific arena and fails.  Updating
ar_ptr fixes the failure.

Patch attached.


-- 


http://sourceware.org/bugzilla/show_bug.cgi?id=5553

------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.