Re: possible problem with memory allocation using calloc/mmap/munmap

[Corinna Vinschen <corinna-cygwin@cygwin.com>]

[-- Attachment #1: Type: text/plain, Size: 3722 bytes --]

On Jun  4 11:38, Stanislav Kascak wrote:
> > > It seems that when mmap() is called with length argument exceeding
> > > size of file, only memory to fit that file is allocated. munmap()
> > > however frees the full specified length. Since (at least on my
> > > computer) big chunk of memory allocated by calloc() is located after
> > > mmap() allocation, munmap() frees even memory of that calloc().
> >
> > Ken's right.  Due to the differences between mapping files on Windows
> > vs. Unix, Cygwin can't map beyond the file size + the remainder of the
> > last page.  Cygwin tries to workaround that on 32 bit by allocating
> > an anonymous mapping following the file mapping to keep the range free
> > from other mappings.  But on 64 bit this workaround doesn't work anymore
> > because the OS is missing an (undocumented) flag which allows to
> > create mappings on 4K boundaries, rather than just on 64K boundaries.
> >
> > I know this situation is unsatisfying, but I have no easy workaround
> > to allow this.  Cygwin could add the anonymous mapping on the next
> > 64K boundary on 64 bit, but that would result in a hole in the mapping
> > which seemed like a rather bad idea when porting mmap to 64 bit.
> >
> > Ken's also right that munmap is doing the right thing here.  If
> > anything's wrong, it's mmap's workaround for mappings beyond the file
> > length.  If only 64 bit would allow 4K-aligned mappings :(
> 
> Thanks for the answer. It is appreciated.
> I understand the problem and difficulty to resolve it. Maybe returning
> an error from mmap (and putting a comment to code for its reason)
> would be sufficient. mmap caller could just adjust requested
> allocation size to file size. Without error, caller has no way of
> knowing memory was not allocated and segfault is then thrown in an
> unrelated memory segment which makes the root cause hard to track
> down. But, I do not know all the implication that could result from
> that, so evaluation of this approach is up to you.

Given that most of the required code already exists for 32 bit systems
(except under WOW64, suffering the same problem as the 64 bit WIndows
environment), I hacked a bit on this code this morning and I got your
testcase running fine.  The idea being that after a successful mmap the
expectation that a matching munmap does *not* unmap unrelated mappings
is valid.

In more depth, here's what Cygwin does on 32 bit, assuming a file size
of 100 bytes and a mapping request of 256K:

First Cygwin mmaps the file.  This results in a 4K mapping in Windows:

 file:    |-- 100b --|

 mapping: |-- 4K --....--|

Next Cygwin adds another mapping to fill up the range up to the next
64K allocation granularity boundary:

 |-- file 4K --|-- filler 60K --|

Eventually Cygwin adds another mapping to fullfill the entire mapping
request:

 |-- file 4K --|-- filler 60K --|-- filler 192K --|

The problem on WOW64 and real 64 bit is that it's impossible to map
the first filler.  However, this area in the VM will *never* be
allocated by other application functions due to the allocation
granularity of 64K!

So my workaround for 64 bit and WOW64 is to just skip allocating the
first filler:

 |-- file 4K --|-- THE VOID 60K --|-- filler 192K --|

The advantage is now that the following munmap of 256K will only
unmap the map for the file and the filler, but not the region you
calloced before, which formerly was accidentally mapped to the
filler region.  This just can't happen anymore now.

Would that be feasible?  If so I can push my patch and create a
developer snapshot for testing.


Thanks,
Corinna

-- 
Corinna Vinschen
Cygwin Maintainer

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]