From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 2829 invoked by alias); 4 Jun 2019 13:49:08 -0000 Mailing-List: contact cygwin-help@cygwin.com; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: cygwin-owner@cygwin.com Mail-Followup-To: cygwin@cygwin.com Received: (qmail 2821 invoked by uid 89); 4 Jun 2019 13:49:08 -0000 Authentication-Results: sourceware.org; auth=none X-Spam-SWARE-Status: No, score=-0.9 required=5.0 tests=AWL,BAYES_00,FREEMAIL_FROM,RCVD_IN_DNSWL_NONE,SPF_PASS,UNSUBSCRIBE_BODY autolearn=no version=3.3.1 spammy=arise X-HELO: mail-oi1-f178.google.com Received: from mail-oi1-f178.google.com (HELO mail-oi1-f178.google.com) (209.85.167.178) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Tue, 04 Jun 2019 13:49:06 +0000 Received: by mail-oi1-f178.google.com with SMTP id v186so1306467oie.5 for ; Tue, 04 Jun 2019 06:49:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=xqWX0/UBSCI+NDpXAoBu6eEEr7rXPx9auuyThFn6vUo=; b=JglsRLRjnpN2rzTjIrOh4Al34eqqlRc7HFC+5L2Ii00hMEzMXcolhJnYvIG03B9mZQ TtxMVmfmG3hvmT/ZA9Bv0UlDaCkCt/VXRdFo7E3DxASK0r/BiN7lZ7FADZJXk4wVKHeM 5QmTZ3Cf0frtmSRdE2PW/HsZVhs7Orw7sH1QOzckOxF92ulyA/iWKNqhfpSuSOE+mGY2 RkxbG8ecr4yHS9gfGHMu4Pivsf5OEIgBhhFpldp3pWr5SZCTu8RclmAkXA0E+CvwjtJQ sxQjIC+/soolmLl2oY4VZ9GmPL9OmC50oEiNnkyGTjUQSSzqtMDNgdgetcOGXXd+oypD q10Q== MIME-Version: 1.0 References: <20190603115456.GG3437@calimero.vinschen.de> <20190604131836.GS3437@calimero.vinschen.de> In-Reply-To: <20190604131836.GS3437@calimero.vinschen.de> From: Stanislav Kascak Date: Tue, 04 Jun 2019 13:49:00 -0000 Message-ID: Subject: Re: possible problem with memory allocation using calloc/mmap/munmap To: cygwin@cygwin.com, Stanislav Kascak Content-Type: text/plain; charset="UTF-8" X-SW-Source: 2019-06/txt/msg00024.txt.bz2 > > > > It seems that when mmap() is called with length argument exceeding > > > > size of file, only memory to fit that file is allocated. munmap() > > > > however frees the full specified length. Since (at least on my > > > > computer) big chunk of memory allocated by calloc() is located after > > > > mmap() allocation, munmap() frees even memory of that calloc(). > > > > > > Ken's right. Due to the differences between mapping files on Windows > > > vs. Unix, Cygwin can't map beyond the file size + the remainder of the > > > last page. Cygwin tries to workaround that on 32 bit by allocating > > > an anonymous mapping following the file mapping to keep the range free > > > from other mappings. But on 64 bit this workaround doesn't work anymore > > > because the OS is missing an (undocumented) flag which allows to > > > create mappings on 4K boundaries, rather than just on 64K boundaries. > > > > > > I know this situation is unsatisfying, but I have no easy workaround > > > to allow this. Cygwin could add the anonymous mapping on the next > > > 64K boundary on 64 bit, but that would result in a hole in the mapping > > > which seemed like a rather bad idea when porting mmap to 64 bit. > > > > > > Ken's also right that munmap is doing the right thing here. If > > > anything's wrong, it's mmap's workaround for mappings beyond the file > > > length. If only 64 bit would allow 4K-aligned mappings :( > > > > Thanks for the answer. It is appreciated. > > I understand the problem and difficulty to resolve it. Maybe returning > > an error from mmap (and putting a comment to code for its reason) > > would be sufficient. mmap caller could just adjust requested > > allocation size to file size. Without error, caller has no way of > > knowing memory was not allocated and segfault is then thrown in an > > unrelated memory segment which makes the root cause hard to track > > down. But, I do not know all the implication that could result from > > that, so evaluation of this approach is up to you. > > Given that most of the required code already exists for 32 bit systems > (except under WOW64, suffering the same problem as the 64 bit WIndows > environment), I hacked a bit on this code this morning and I got your > testcase running fine. The idea being that after a successful mmap the > expectation that a matching munmap does *not* unmap unrelated mappings > is valid. > > In more depth, here's what Cygwin does on 32 bit, assuming a file size > of 100 bytes and a mapping request of 256K: > > First Cygwin mmaps the file. This results in a 4K mapping in Windows: > > file: |-- 100b --| > > mapping: |-- 4K --....--| > > Next Cygwin adds another mapping to fill up the range up to the next > 64K allocation granularity boundary: > > |-- file 4K --|-- filler 60K --| > > Eventually Cygwin adds another mapping to fullfill the entire mapping > request: > > |-- file 4K --|-- filler 60K --|-- filler 192K --| > > The problem on WOW64 and real 64 bit is that it's impossible to map > the first filler. However, this area in the VM will *never* be > allocated by other application functions due to the allocation > granularity of 64K! > > So my workaround for 64 bit and WOW64 is to just skip allocating the > first filler: > > |-- file 4K --|-- THE VOID 60K --|-- filler 192K --| > > The advantage is now that the following munmap of 256K will only > unmap the map for the file and the filler, but not the region you > calloced before, which formerly was accidentally mapped to the > filler region. This just can't happen anymore now. > > Would that be feasible? If so I can push my patch and create a > developer snapshot for testing. Two questions arise when I'm thinking about workaround solution: - what happens if caller tries to write to |-- THE VOID 60K --|. Since this is unallocated, would there be a segfault? - is it possible that some subsequent mem alloc request would return region from |-- THE VOID 60K --| which could again cause segfault after munmap? Stanislav Kascak -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple