[Bug go/60931] New: libgo has issues when page size is not 4k

[Bug go/60931] New: libgo has issues when page size is not 4k

[anton at samba dot org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=60931

            Bug ID: 60931
           Summary: libgo has issues when page size is not 4k
           Product: gcc
           Version: 4.10.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: go
          Assignee: ian at airs dot com
          Reporter: anton at samba dot org

Created attachment 32659
  --> http://gcc.gnu.org/bugzilla/attachment.cgi?id=32659&action=edit
Bump page size to 64kB

We are seeing random failures with go programs on a 64kB page size ppc64 box.
It looks like garbage collection issues - sometimes we SEGV in timer code,
sometimes we SEGV in the code that wraps a kernel read syscall. If I prevent
the garbage collector from running, the programs work.

The libgo malloc hard codes the page size so I wrote a quick hack to bump this
(and a few other dependent variables). This makes the problem go away, but we
will need to come up with a better way to do this at runtime.

[Bug go/60931] libgo has issues when page size is not 4k

[pinskia at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=60931

--- Comment #1 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
This is going to be true on AARCH64 also where most distros are going to be
using 64k pages (some might use 4k pages if they also support AARCH32).  MIPS
has many different page sizes too (4k, 8k, 16k, 32k, and 64k).  So hard coding
the page size seems wrong, maybe you should call getpagesize instead.

[Bug go/60931] libgo has issues when page size is not 4k

[anton at samba dot org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=60931

--- Comment #2 from Anton Blanchard <anton at samba dot org> ---
I agree, but when I tried this I found a few places that expect PageSize to be
a compile time constant so it is not as trivial as I had hoped.

[Bug go/60931] libgo has issues when page size is not 4k

[ian at airs dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=60931

--- Comment #3 from Ian Lance Taylor <ian at airs dot com> ---
It would be extremely helpful if you could find a test case that can recreate
this problem with some reliability.  There is no obvious dependency on the
system page size in libgo.  The PageSize constant is the unit that the memory
allocator deals in, and should have no direct relationship to the system page
size.  I believe that there is a bug, but we need to track it down.

If you set the environment variable GOGC=1 the garbage collector will run much
more frequently; perhaps that will help get a reproducible test case.

[Bug go/60931] libgo has issues when page size is not 4k

[anton at samba dot org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=60931

Anton Blanchard <anton at samba dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #32659|0                           |1
        is obsolete|                            |

--- Comment #4 from Anton Blanchard <anton at samba dot org> ---
Created attachment 32669
  --> http://gcc.gnu.org/bugzilla/attachment.cgi?id=32669&action=edit
Don't use madvise(DONT_NEED) on sub pages

[Bug go/60931] libgo has issues when page size is not 4k

[anton at samba dot org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=60931

--- Comment #5 from Anton Blanchard <anton at samba dot org> ---
I think I see it:

19112 madvise(0xc211030000, 4096, MADV_DONTNEED) = 0

That 4kB madvise(MADV_DONTNEED) gets rounded up to the system page size of 64kB
and we end up covering still in use memory.

The following patch fixes it for me, but it just ignores any sub pages. We
should keep them around so later calls have a chance at consolidating regions
up to a system page size.

[Bug go/60931] libgo has issues when page size is not 4k

[jakub at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=60931

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jakub at gcc dot gnu.org

--- Comment #6 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
Perhaps it would be better instead of not doing the madvise at all if start or
length isn't page aligned round the start to the next page boundary and end to
the previous page boundary and madvise if the rounded end is above the rounded
start.

[Bug go/60931] libgo has issues when page size is not 4k

[ian at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=60931

--- Comment #9 from ian at gcc dot gnu.org <ian at gcc dot gnu.org> ---
Author: ian
Date: Fri Apr 25 04:29:07 2014
New Revision: 209777

URL: http://gcc.gnu.org/viewcvs?rev=209777&root=gcc&view=rev
Log:
    PR go/60931

runtime: Fix garbage collector issue with non 4kB system page size

The go garbage collector tracks memory in terms of 4kB pages.
Most of the code checks getpagesize() at runtime and does the
right thing.

On a 64kB ppc64 box I see SEGVs in long running processes
which has been diagnosed as a bug in scavengelist.
scavengelist does a madvise(MADV_DONTNEED) without rounding
the arguments to the system page size.  A strace of one of the
failures shows the problem:

madvise(0xc211030000, 4096, MADV_DONTNEED) = 0

The kernel rounds the length up to 64kB and we mark 60kB of
valid data as no longer needed.

Round start up to a system page and end down before calling
madvise.

Modified:
    trunk/libgo/runtime/mheap.c

[Bug go/60931] libgo has issues when page size is not 4k

[ian at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=60931

--- Comment #8 from ian at gcc dot gnu.org <ian at gcc dot gnu.org> ---
Author: ian
Date: Fri Apr 25 04:28:48 2014
New Revision: 209776

URL: http://gcc.gnu.org/viewcvs?rev=209776&root=gcc&view=rev
Log:
    PR go/60931

runtime: Fix garbage collector issue with non 4kB system page size

The go garbage collector tracks memory in terms of 4kB pages.
Most of the code checks getpagesize() at runtime and does the
right thing.

On a 64kB ppc64 box I see SEGVs in long running processes
which has been diagnosed as a bug in scavengelist.
scavengelist does a madvise(MADV_DONTNEED) without rounding
the arguments to the system page size.  A strace of one of the
failures shows the problem:

madvise(0xc211030000, 4096, MADV_DONTNEED) = 0

The kernel rounds the length up to 64kB and we mark 60kB of
valid data as no longer needed.

Round start up to a system page and end down before calling
madvise.

Modified:
    branches/gcc-4_9-branch/libgo/runtime/mheap.c

[Bug go/60931] libgo has issues when page size is not 4k

[ian at airs dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=60931

Ian Lance Taylor <ian at airs dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |RESOLVED
         Resolution|---                         |FIXED

--- Comment #10 from Ian Lance Taylor <ian at airs dot com> ---
Thanks for the patch.  I committed a version of it to mainline and 4.9 branch.