From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 28524 invoked by alias); 7 Sep 2006 15:50:48 -0000 Received: (qmail 28504 invoked by uid 22791); 7 Sep 2006 15:50:48 -0000 X-Spam-Check-By: sourceware.org Received: from sunsite.ms.mff.cuni.cz (HELO sunsite.mff.cuni.cz) (195.113.15.26) by sourceware.org (qpsmtpd/0.31) with ESMTP; Thu, 07 Sep 2006 15:50:40 +0000 Received: from sunsite.mff.cuni.cz (sunsite.mff.cuni.cz [127.0.0.1]) by sunsite.mff.cuni.cz (8.13.1/8.13.1) with ESMTP id k87FoaPS019382; Thu, 7 Sep 2006 17:50:36 +0200 Received: (from jj@localhost) by sunsite.mff.cuni.cz (8.13.1/8.13.1/Submit) id k87FoaQT019379; Thu, 7 Sep 2006 17:50:36 +0200 Date: Thu, 07 Sep 2006 15:50:00 -0000 From: Jakub Jelinek To: Ulrich Drepper Cc: Glibc hackers Subject: [PATCH] Fix malloc with really large sizes [BZ #2775] Message-ID: <20060907155035.GY4556@sunsite.mff.cuni.cz> Reply-To: Jakub Jelinek Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.1i Mailing-List: contact libc-hacker-help@sourceware.org; run by ezmlm Precedence: bulk List-Subscribe: List-Archive: List-Post: List-Help: , Sender: libc-hacker-owner@sourceware.org X-SW-Source: 2006-09/txt/msg00015.txt.bz2 Hi! As the BZ#2775 testcase shows (not sure if it is appropriate for make check, as it uses many threads to force initial thread to allocate on a different arena than main_arena), we sometimes call grow_heap to grow a heap, but the size is so large that the diff is negative (and in that case grow_heap acts as shrink_heap and unlike growing e.g. assumes the diff is already a multiple of page size). The grow_heap change is just trying to be really safe, if e.g. old heap size is 1MB and diff is e.g. 0x7ff01000 on 32-bit 4KB pagesize arch, then we could easily create heap bigger than HEAP_MAX_SIZE. 2006-09-07 Jakub Jelinek [BZ #2775] * malloc/malloc.c (sYSMALLOc): Only call grow_heap if (long) (MINSIZE + nb - old_size) is positive. * malloc/arena.c (grow_heap): When growing bail even if new_size is negative. --- libc/malloc/arena.c.jj 2006-09-07 16:46:50.000000000 +0200 +++ libc/malloc/arena.c 2006-09-07 17:35:38.000000000 +0200 @@ -712,7 +712,7 @@ grow_heap(h, diff) heap_info *h; long di if(diff >= 0) { diff = (diff + page_mask) & ~page_mask; new_size = (long)h->size + diff; - if(new_size > HEAP_MAX_SIZE) + if((unsigned long) new_size > (unsigned long) HEAP_MAX_SIZE) return -1; if(mprotect((char *)h + h->size, diff, PROT_READ|PROT_WRITE) != 0) return -2; --- libc/malloc/malloc.c.jj 2006-09-04 16:42:01.000000000 +0200 +++ libc/malloc/malloc.c 2006-09-07 17:39:59.000000000 +0200 @@ -2970,7 +2970,8 @@ static Void_t* sYSMALLOc(nb, av) INTERNA /* First try to extend the current heap. */ old_heap = heap_for_ptr(old_top); old_heap_size = old_heap->size; - if (grow_heap(old_heap, MINSIZE + nb - old_size) == 0) { + if ((long) (MINSIZE + nb - old_size) > 0 + && grow_heap(old_heap, MINSIZE + nb - old_size) == 0) { av->system_mem += old_heap->size - old_heap_size; arena_mem += old_heap->size - old_heap_size; #if 0 Jakub