GGC_QUIRE_SIZE change

GGC_QUIRE_SIZE change

[Richard Kenner]

It's causing problems for me on DEC Unix 4.0C.  When I have a program that
uses sufficient memory (in this case it's in Ada), the mmap fails with
ENOSPACE.  I've spent a lot of time in the debugger and everything looks
OK, but it does fail.  Setting GGC_QUIRE_SIZE to 1 "fixes" it.

Does anybody have any idea what's going on?

Re: GGC_QUIRE_SIZE change

[Zack Weinberg]

On Tue, Aug 29, 2000 at 10:18:04AM -0400, Richard Kenner wrote:
> It's causing problems for me on DEC Unix 4.0C.  When I have a program that
> uses sufficient memory (in this case it's in Ada), the mmap fails with
> ENOSPACE.  I've spent a lot of time in the debugger and everything looks
> OK, but it does fail.  Setting GGC_QUIRE_SIZE to 1 "fixes" it.
> 
> Does anybody have any idea what's going on?

How much memory has been allocated already when it fails?  Do you
think it would work to fall back to 1-page allocations if a 16-page
mmap() returns ENOSPACE?

zw

Re: GGC_QUIRE_SIZE change

[Richard Kenner]

    How much memory has been allocated already when it fails?  

About 13 MB, with it saying that about 6 MB is mapped.

    Do you think it would work to fall back to 1-page allocations if a
    16-page mmap() returns ENOSPACE?

I don't think so.  I think there is something fundamentally wrong here:
allocating 128KB shouldn't be an issue.  Interestingly enough, when the
mmap fails, GDB is no longer able to call any functions, claiming it
couldn't read the address that corresponds to _start.  But I can read it
with x/i just fine.  So something is messing up badly, though I have no
idea what.

If I change the macro from 16 to 1, things work fine.

Re: GGC_QUIRE_SIZE change

[Zack Weinberg]

On Tue, Aug 29, 2000 at 01:25:20PM -0400, Richard Kenner wrote:
> 
>     Do you think it would work to fall back to 1-page allocations if a
>     16-page mmap() returns ENOSPACE?
> 
> I don't think so.  I think there is something fundamentally wrong here:
> allocating 128KB shouldn't be an issue.  Interestingly enough, when the
> mmap fails, GDB is no longer able to call any functions, claiming it
> couldn't read the address that corresponds to _start.  But I can read it
> with x/i just fine.  So something is messing up badly, though I have no
> idea what.
> 
> If I change the macro from 16 to 1, things work fine.

This is beginning to sound like an OS bug.  Could you try compiling
this test program with and without -DGGC_QUIRE_SIZE=1 and see what it
prints when run?  On my system I get

$ ./test-16
pagesize=4096 blocksize=65536
failed after 32753 allocations (524048 pages); Cannot allocate memory
$ ./test-1
pagesize=4096 blocksize=4096
failed after 524046 allocations (524046 pages); Cannot allocate memory

-- in both cases, failure only after allocating 2GB of address space.
Note that it doesn't touch the allocated memory, so it may work when
GCC wouldn't.

zw

#include <sys/types.h>
#include <sys/mman.h>
#include <fcntl.h>
#include <unistd.h>
#include <stdio.h>
#include <string.h>
#include <errno.h>

#ifndef MAP_FAILED
#define MAP_FAILED -1
#endif

#ifndef MAP_ANON
#ifdef MAP_ANONYMOUS
#define MAP_ANON MAP_ANONYMOUS 
#else
#define MAP_ANON 0
#endif
#endif

#ifndef GGC_QUIRE_SIZE
#define GGC_QUIRE_SIZE 16
#endif

int
main(void)
{
    char *page;
    size_t count;
    size_t blocksize;
    int fd;

#if MAP_ANON
    fd = -1;
#else
    fd = open("/dev/zero", O_RDWR);
#endif
    blocksize = getpagesize() * GGC_QUIRE_SIZE;
    printf("pagesize=%zd blocksize=%zd\n", getpagesize(), blocksize);

    for(count = 1; ; count++)
    {
	page = mmap(0, blocksize, PROT_READ|PROT_WRITE,
		    MAP_PRIVATE|MAP_ANON, fd, 0);
	if(page == MAP_FAILED)
	{
	    const char *err = strerror(errno);
	    printf("failed after %zd allocations (%zd pages); %s\n",
		   count, count * blocksize / getpagesize(), err);
	    return 0;
	}
    }
}

Re: GGC_QUIRE_SIZE change

[Richard Kenner]

    This is beginning to sound like an OS bug.  Could you try compiling
    this test program with and without -DGGC_QUIRE_SIZE=1 and see what it
    prints when run?  

When I get this, it runs until swap space fills, so this is definitely
very different from the behavior seen with GCC.

Re: GGC_QUIRE_SIZE change

[Zack Weinberg]

On Tue, Aug 29, 2000 at 02:52:35PM -0400, Richard Kenner wrote:
>     This is beginning to sound like an OS bug.  Could you try compiling
>     this test program with and without -DGGC_QUIRE_SIZE=1 and see what it
>     prints when run?  
> 
> When I get this, it runs until swap space fills, so this is definitely
> very different from the behavior seen with GCC.

Try adding a

  *page = 1;

after the check to see if mmap failed.

zw

Re: GGC_QUIRE_SIZE change

[Richard Kenner]

    Try adding a

      *page = 1;

    after the check to see if mmap failed.

Same thing.  Of course that only touches 1/16th of the pages, but I don't
think the issue is amount of allocated memory, but instead has something
to do with address space ranges, though I have no idea of in what way
exactly, of course.

Re: GGC_QUIRE_SIZE change

[Zack Weinberg]

On Tue, Aug 29, 2000 at 03:06:06PM -0400, Richard Kenner wrote:
>     Try adding a
> 
>       *page = 1;
> 
>     after the check to see if mmap failed.
> 
> Same thing.  Of course that only touches 1/16th of the pages, but I don't
> think the issue is amount of allocated memory, but instead has something
> to do with address space ranges, though I have no idea of in what way
> exactly, of course.

Hm, perhaps it has to do with allocating memory with malloc at the
same time.

Do you have strace or equivalent on that system?  I'd be interested to
see if this is the brk region colliding with the mmap region.

[Why they are anywhere near each other on a 64-bit platform is the
next question...]

zw

Re: GGC_QUIRE_SIZE change

[Richard Kenner]

    Do you have strace or equivalent on that system?  I'd be interested to
    see if this is the brk region colliding with the mmap region.

Yes, I do.  I've never used it, so tell me what to do.

It will take an hour or so for me to get back to where things were
when it was failing.

Re: GGC_QUIRE_SIZE change

[Zack Weinberg]

On Tue, Aug 29, 2000 at 03:21:18PM -0400, Richard Kenner wrote:
>     Do you have strace or equivalent on that system?  I'd be interested to
>     see if this is the brk region colliding with the mmap region.
> 
> Yes, I do.  I've never used it, so tell me what to do.

First do

strace /bin/ls

That will tell you what strace calls the brk and mmap system calls.
(They aren't always brk and mmap.)

Then trace the failing compiler, asking for just those calls:

strace -o trace.out -e trace=brk,mmap,munmap ./cc1 <arguments>

Replace cc1 by whichever compiler you're using, and brk,mmap,munmap
with whatever they're called on your system.  Send me trace.out.

> It will take an hour or so for me to get back to where things were
> when it was failing.

Ok, I need to go get lunch anyhow.

zw