From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 23138 invoked by alias); 27 Apr 2009 09:12:09 -0000 Received: (qmail 23120 invoked by uid 22791); 27 Apr 2009 09:12:08 -0000 X-SWARE-Spam-Status: No, hits=-2.3 required=5.0 tests=AWL,BAYES_00 X-Spam-Check-By: sourceware.org Received: from sunsite.ms.mff.cuni.cz (HELO sunsite.mff.cuni.cz) (195.113.15.26) by sourceware.org (qpsmtpd/0.43rc1) with ESMTP; Mon, 27 Apr 2009 09:12:03 +0000 Received: from sunsite.mff.cuni.cz (localhost.localdomain [127.0.0.1]) by sunsite.mff.cuni.cz (8.13.8/8.13.8) with ESMTP id n3R9MZFQ002957; Mon, 27 Apr 2009 11:22:35 +0200 Received: (from jakub@localhost) by sunsite.mff.cuni.cz (8.13.8/8.13.8/Submit) id n3R9MXBK002955; Mon, 27 Apr 2009 11:22:33 +0200 Date: Mon, 27 Apr 2009 09:12:00 -0000 From: Jakub Jelinek To: Ulrich Drepper Cc: Glibc hackers Subject: [PATCH] Fix localedef Message-ID: <20090427092233.GC16681@sunsite.ms.mff.cuni.cz> Reply-To: Jakub Jelinek Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.2i Mailing-List: contact libc-hacker-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: libc-hacker-owner@sourceware.org X-SW-Source: 2009-04/txt/msg00007.txt.bz2 Hi! We still have a bunch of issues in localedef: 1) MAP_ANON without MAP_PRIVATE or MAP_SHARED is MAP_FAILED/EINVAL, so we actually never reserved anything 2) enlarge_archive, if the reserved region size isn't big enough, now munmaps the old region and mmaps the whole file elsewhere, but doesn't update the head pointer, which now points to the munmaped area 3) unfortunately, it seems kernel rejects mremap MREMAP_MAYMOVE|MREMAP_FIXED if new_addr == addr (-1/EINVAL): /* Check if the location we're moving into overlaps the * old location at all, and fail if it does. */ if ((new_addr <= addr) && (new_addr+new_len) > addr) goto out; if ((addr <= new_addr) && (addr+old_len) > new_addr) goto out; This patch fixes 1) by oring in MAP_PRIVATE, 2) by head = ah->addr, 3) by replacing mremap with mmap64 (for speed I'm just mmaping starting at end of previous mmapping rounded down to page boundary) and in addition to this it adds the RESERVE_MMAP_SIZE stuff to enlarge_archive, as e.g. when --add-to-archive hundreds of locales into an empty locale-archive this makes quite a significant difference. 2009-04-27 Jakub Jelinek * locale/programs/locarchive.c (create_archive): Add MAP_PRIVATE to MAP_ANON in PROT_NONE mmap64 call. (open_archive): Likewise. (file_data_available_p): Use mmap64 instead of mremap. (enlarge_archive): Likewise. Update head if ah->addr changed. Attempt to reserve address space after mmap64 region. --- libc/locale/programs/locarchive.c.jj 2009-04-27 10:44:42.000000000 +0200 +++ libc/locale/programs/locarchive.c 2009-04-27 10:45:42.000000000 +0200 @@ -134,8 +134,8 @@ create_archive (const char *archivefname size_t reserved = RESERVE_MMAP_SIZE; int xflags = 0; if (total < reserved - && ((p = mmap64 (NULL, reserved, PROT_NONE, MAP_ANON, -1, 0)) - != MAP_FAILED)) + && ((p = mmap64 (NULL, reserved, PROT_NONE, MAP_PRIVATE | MAP_ANON, + -1, 0)) != MAP_FAILED)) xflags = MAP_FIXED; else { @@ -259,10 +259,16 @@ file_data_available_p (struct locarhandl if (st.st_size > ah->reserved) return false; - void *p = mremap (ah->addr, ah->mmaped, st.st_size, - MREMAP_FIXED | MREMAP_MAYMOVE, ah->addr); + const size_t pagesz = getpagesize (); + size_t start = ah->mmaped & ~(pagesz - 1); + void *p = mmap64 (ah->addr + start, st.st_size - start, + PROT_READ | PROT_WRITE, MAP_SHARED | MAP_FIXED, + ah->fd, start); if (p == MAP_FAILED) - return false; + { + ah->mmaped = start; + return false; + } ah->mmaped = st.st_size; return true; @@ -312,14 +318,15 @@ enlarge_archive (struct locarhandle *ah, error (EXIT_FAILURE, errno, _("cannot map locale archive file")); if (st.st_size < ah->reserved) - ah->addr = mremap (ah->addr, ah->mmaped, st.st_size, - MREMAP_MAYMOVE | MREMAP_FIXED, ah->addr); + ah->addr = mmap64 (ah->addr, st.st_size, PROT_READ | PROT_WRITE, + MAP_SHARED | MAP_FIXED, ah->fd, 0); else { munmap (ah->addr, ah->reserved); ah->addr = mmap64 (NULL, st.st_size, PROT_READ | PROT_WRITE, MAP_SHARED, ah->fd, 0); ah->reserved = st.st_size; + head = ah->addr; } if (ah->addr == MAP_FAILED) goto enomap; @@ -384,8 +391,22 @@ enlarge_archive (struct locarhandle *ah, error (EXIT_FAILURE, errval, _("cannot resize archive file")); } + /* To prepare for enlargements of the mmaped area reserve some + address space. */ + size_t reserved = RESERVE_MMAP_SIZE; + int xflags = 0; + if (total < reserved + && ((p = mmap64 (NULL, reserved, PROT_NONE, MAP_PRIVATE | MAP_ANON, + -1, 0)) != MAP_FAILED)) + xflags = MAP_FIXED; + else + { + p = NULL; + reserved = total; + } + /* Map the header and all the administration data structures. */ - p = mmap64 (NULL, total, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0); + p = mmap64 (p, total, PROT_READ | PROT_WRITE, MAP_SHARED | xflags, fd, 0); if (p == MAP_FAILED) { int errval = errno; @@ -404,7 +425,7 @@ enlarge_archive (struct locarhandle *ah, new_ah.mmaped = total; new_ah.addr = p; new_ah.fd = fd; - new_ah.reserved = total; + new_ah.reserved = reserved; /* Walk through the hash name hash table to find out what data is still referenced and transfer it into the new file. */ @@ -593,8 +614,8 @@ open_archive (struct locarhandle *ah, bo int xflags = 0; void *p; if (st.st_size < reserved - && ((p = mmap64 (NULL, RESERVE_MMAP_SIZE, PROT_NONE, MAP_ANON, -1, 0)) - != MAP_FAILED)) + && ((p = mmap64 (NULL, reserved, PROT_NONE, MAP_PRIVATE | MAP_ANON, + -1, 0)) != MAP_FAILED)) xflags = MAP_FIXED; else { Jakub