[Bug c/97329] New: POWER9 default cache and line sizes appear to be wrong

[Bug c/97329] New: POWER9 default cache and line sizes appear to be wrong

[kip at thevertigo dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=97329

            Bug ID: 97329
           Summary: POWER9 default cache and line sizes appear to be wrong
           Product: gcc
           Version: 10.2.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: c
          Assignee: unassigned at gcc dot gnu.org
          Reporter: kip at thevertigo dot com
  Target Milestone: ---

While investigating the memory hierarchy on my Romulus POWER9 (CPU revision
2.2) I discovered GCC's default L1 cache and line sizes on POWER9 are not
correct. 

I think whoever specified the default cache size of 64KB may not have realized
the L1 cache is banked, and not unified. On POWER9 that 64KB is split between
separate instructions and data spaces. Only 32KB is actually available for
data.

GCC's documentation specifies that the l1-cache-size parameter is suppose to
refer to data only, but not instructions.

Further, the default l1-cache-line-size is wrong. It's currently set at 32
bytes. The correct value is actually four times that at 128 bytes.

As things are right now, the resulting generated code may not be properly
optimized because the optimizer plans around the wrong parameters.

When this happens the generated program may have a higher than necessary cache
miss rate. This could be a big deal since the CPU retrieving data from the L1
may only take one or two cycles, but a cache miss could mean several hundred
while the block is transferred.

[Bug target/97329] POWER9 default cache and line sizes appear to be wrong

[kip at thevertigo dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=97329

Kip Warner <kip at thevertigo dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Version|10.2.0                      |11.0

--- Comment #1 from Kip Warner <kip at thevertigo dot com> ---
Just tested with Git head (11.0.0 20201008) and same issue:

$ gcc --version
gcc (GCC) 11.0.0 20201008 (experimental)
Copyright (C) 2020 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

$ gcc -Q -mcpu=power9 --help=param | grep -i cache
  --param=l1-cache-line-size=           128
  --param=l1-cache-size=                32
  --param=l2-cache-size=                512

$ getconf -a | grep CACHE
LEVEL1_ICACHE_SIZE                 32768
LEVEL1_ICACHE_ASSOC                32
LEVEL1_ICACHE_LINESIZE             128
LEVEL1_DCACHE_SIZE                 32768
LEVEL1_DCACHE_ASSOC                32
LEVEL1_DCACHE_LINESIZE             128
LEVEL2_CACHE_SIZE                  524288
LEVEL2_CACHE_ASSOC                 2048
LEVEL2_CACHE_LINESIZE              32
LEVEL3_CACHE_SIZE                  10485760
LEVEL3_CACHE_ASSOC                 40960
LEVEL3_CACHE_LINESIZE              32
LEVEL4_CACHE_SIZE                  0
LEVEL4_CACHE_ASSOC                 0
LEVEL4_CACHE_LINESIZE              0

$ cat /proc/cpuinfo
processor       : 0
cpu             : POWER9, altivec supported
clock           : 2933.000000MHz
revision        : 2.2 (pvr 004e 1202)

(...)

processor       : 143
cpu             : POWER9, altivec supported
clock           : 2166.000000MHz
revision        : 2.2 (pvr 004e 1202)

timebase        : 512000000
platform        : PowerNV
model           : 0000000000000000
machine         : PowerNV 0000000000000000
firmware        : OPAL
MMU             : Radix

[Bug target/97329] POWER9 default cache and line sizes appear to be wrong

[kip at thevertigo dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=97329

--- Comment #2 from Kip Warner <kip at thevertigo dot com> ---
Sorry, not same issue. It appears as though this was fixed in gcc-11.

[Bug target/97329] POWER9 default cache and line sizes appear to be wrong

[segher at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=97329

Segher Boessenkool <segher at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Last reconfirmed|                            |2020-10-08
     Ever confirmed|0                           |1
             Status|UNCONFIRMED                 |NEW
                 CC|                            |segher at gcc dot gnu.org

--- Comment #3 from Segher Boessenkool <segher at gcc dot gnu.org> ---
At least as far back as GCC 5 we report D-L1 size 64kB (for most CPUs,
not just p9).  Confirmed.

[Bug target/97329] POWER9 default cache and line sizes appear to be wrong

[kip at thevertigo dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=97329

--- Comment #4 from Kip Warner <kip at thevertigo dot com> ---
I'm going to do some more testing tonight and report back after.

[Bug target/97329] POWER9 default cache and line sizes appear to be wrong

[segher at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=97329

--- Comment #5 from Segher Boessenkool <segher at gcc dot gnu.org> ---
So both the cache line size and the cache size are wrong for GCC 10
and before, but okay on trunk, on all compiler I tested (I tested on
Linux only so far).

[Bug target/97329] POWER9 default cache and line sizes appear to be wrong

[kip at thevertigo dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=97329

--- Comment #6 from Kip Warner <kip at thevertigo dot com> ---
Created attachment 49333
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=49333&action=edit
Autoconf configuration log on POWER9.

[Bug target/97329] POWER9 default cache and line sizes appear to be wrong

[kip at thevertigo dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=97329

--- Comment #7 from Kip Warner <kip at thevertigo dot com> ---
So it looks like even with GCC 11 in trunk it's still sometimes wrong on
power9.

Wrong L2 cache size when no -mcpu specified:

$ gcc -Q --help=param | grep -i cache
  --param=l1-cache-line-size=           128
  --param=l1-cache-size=                32
  --param=l2-cache-size=                256

Correct when manually specifying native (power9) cpu:

$ gcc -Q -mcpu=native --help=param | grep -i cache
  --param=l1-cache-line-size=           128
  --param=l1-cache-size=                32
  --param=l2-cache-size=                512

Correct when manually specifying power9 cpu:

$ gcc -Q -mcpu=power9 --help=param | grep -i cache
  --param=l1-cache-line-size=           128
  --param=l1-cache-size=                32
  --param=l2-cache-size=                512

Wrong L2 cache size when powerpc64le is selected in place of power9:

$ gcc -Q -mcpu=powerpc64le --help=param | grep -i cach
  --param=l1-cache-line-size=           128
  --param=l1-cache-size=                32
  --param=l2-cache-size=                256

Looks like this might be a clue. GCC did not identify the host/build/target as
power9 automatically:

$ gcc -dumpmachine
powerpc64le-unknown-linux-gnu

I built it from trunk last night on a power9 machine. I've attached my
config.log.

$ gcc --version
gcc (GCC) 11.0.0 20201008 (experimental)
(...)

[Bug target/97329] POWER9 default cache and line sizes appear to be wrong

[segher at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=97329

--- Comment #8 from Segher Boessenkool <segher at gcc dot gnu.org> ---
The default -mcpu= for a compiler targeting powerpc64le-linux is
normally power8 (you can change this with the --with-cpu= configure
option though).  -mcpu=powerpc64le is also (currently) equal to
-mcpu=power8.  But the numbers for Power8 (in power8_cost) are
wrong it seems: it has a 64kB L1-D cache, and a 512kB L2 cache (it
looks like we have simply copied the Power7 numbers here; 32 and
256 is correct for Power7).

[Bug target/97329] POWER9 default cache and line sizes appear to be wrong

[luoxhu at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=97329

luoxhu at gcc dot gnu.org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |luoxhu at gcc dot gnu.org

--- Comment #9 from luoxhu at gcc dot gnu.org ---
Yes, it seems a copy paste error for Power8 from Power7.  Is this supposed to
be fix by gcc-12 stage1? And any performance evaluation required?


diff --git a/gcc/config/rs6000/rs6000.c b/gcc/config/rs6000/rs6000.c
index 616dae35bae..34c4edae20e 100644
--- a/gcc/config/rs6000/rs6000.c
+++ b/gcc/config/rs6000/rs6000.c
@@ -1055,7 +1055,7 @@ struct processor_costs power8_cost = {
   COSTS_N_INSNS (17),  /* ddiv */
   128,                 /* cache line size */
   32,                  /* l1 cache */
-  256,                 /* l2 cache */
+  512,                 /* l2 cache */
   12,                  /* prefetch streams */
   COSTS_N_INSNS (3),   /* SF->DF convert */
 };

[Bug target/97329] POWER9 default cache and line sizes appear to be wrong

[segher at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=97329

--- Comment #10 from Segher Boessenkool <segher at gcc dot gnu.org> ---
GCC 11 stage 4 will be fine.

I doubt you can ever measure a difference, but you can try :-)

[Bug target/97329] POWER9 default cache and line sizes appear to be wrong

[luoxhu at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=97329

luoxhu at gcc dot gnu.org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|---                         |FIXED

--- Comment #11 from luoxhu at gcc dot gnu.org ---
Fixed with r11-7821-g08103e4d6ada9b57366f2df2a2b745babfab914c.

[Bug target/97329] POWER9 default cache and line sizes appear to be wrong

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=97329

--- Comment #12 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The releases/gcc-10 branch has been updated by Xiong Hu Luo
<luoxhu@gcc.gnu.org>:

https://gcc.gnu.org/g:52eacca2455f7468d7ddb990259e8583028c5185

commit r10-9541-g52eacca2455f7468d7ddb990259e8583028c5185
Author: Xionghu Luo <luoxhu@linux.ibm.com>
Date:   Wed Mar 24 23:45:58 2021 -0500

    rs6000: Correct Power8 cost of l2 cache size [PR97329]

    This patch is a backport to gcc 10 from master.
    L2 cache size for Power8 is 512kB, it was copied from Power7 before
    public.  Tested no performance change for SPEC2017.

    gcc/
    2021-03-25  Xionghu Luo  <luoxhu@linux.ibm.com>

            PR target/97329
            * config/rs6000/rs6000.c (power8_costs): Change l2 cache
            from 256 to 512.

    (cherry picked from commit 08103e4d6ada9b57366f2df2a2b745babfab914c)

[Bug target/97329] POWER9 default cache and line sizes appear to be wrong

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=97329

--- Comment #13 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The releases/gcc-9 branch has been updated by Xiong Hu Luo
<luoxhu@gcc.gnu.org>:

https://gcc.gnu.org/g:48354138267e0682f61866003b67a9851d3be3a2

commit r9-9307-g48354138267e0682f61866003b67a9851d3be3a2
Author: Xionghu Luo <luoxhu@linux.ibm.com>
Date:   Wed Mar 24 23:45:58 2021 -0500

    rs6000: Correct Power8 cost of l2 cache size [PR97329]

    This patch is a backport to gcc 9 from master.
    L2 cache size for Power8 is 512kB, it was copied from Power7 before
    public.  Tested no performance change for SPEC2017.

    gcc/
    2021-03-25  Xionghu Luo  <luoxhu@linux.ibm.com>

            PR target/97329
            * config/rs6000/rs6000.c (power8_costs): Change l2 cache
            from 256 to 512.

    (cherry picked from commit 08103e4d6ada9b57366f2df2a2b745babfab914c)