[PATCH 0/3] Add some missing cache infomation

[PATCH 0/3] Add some missing cache infomation

[Richard Henderson]

I was looking to use _SC_LEVEL1_ICACHE_LINESIZE in QEMU,
and notice that some of our common hosts are missing this
information.

For my purposes, a guess at the linesize is good enough,
and I can't imagine that returning the linesize used for
flushing the cache could be incorrect.


r~


Richard Henderson (3):
  Guess L1 cache linesize for aarch64
  Add hidden_proto for getauxval
  Add cache info for powerpc64

 include/sys/auxv.h                                 |  2 +
 misc/getauxval.c                                   |  1 +
 sysdeps/unix/sysv/linux/aarch64/sysconf.c          | 55 +++++++++++++
 .../unix/sysv/linux/powerpc/powerpc64/sysconf.c    | 90 ++++++++++++++++++++++
 4 files changed, 148 insertions(+)
 create mode 100644 sysdeps/unix/sysv/linux/aarch64/sysconf.c
 create mode 100644 sysdeps/unix/sysv/linux/powerpc/powerpc64/sysconf.c

-- 
2.9.4

[PATCH 2/3] Add hidden_proto for getauxval

[Richard Henderson]

---
 include/sys/auxv.h | 2 ++
 misc/getauxval.c   | 1 +
 2 files changed, 3 insertions(+)

diff --git a/include/sys/auxv.h b/include/sys/auxv.h
index dede2c3..2e1f86b 100644
--- a/include/sys/auxv.h
+++ b/include/sys/auxv.h
@@ -1 +1,3 @@
 #include <misc/sys/auxv.h>
+
+libc_hidden_proto (getauxval)
diff --git a/misc/getauxval.c b/misc/getauxval.c
index c83fbce..458f58d 100644
--- a/misc/getauxval.c
+++ b/misc/getauxval.c
@@ -43,3 +43,4 @@ __getauxval (unsigned long int type)
 }
 
 weak_alias (__getauxval, getauxval)
+libc_hidden_weak (getauxval)
-- 
2.9.4

[PATCH 3/3] Add cache info for powerpc64

[Richard Henderson]

The actual cache info was added for 4.11, but have a guess at the
L1 linesizes using info provided by older kernels.

	* sysdeps/unix/sysv/linux/powerpc/powerpc64/sysconf.c: New file.

Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Steven Munroe <sjmunroe@us.ibm.com>
---
 .../unix/sysv/linux/powerpc/powerpc64/sysconf.c    | 90 ++++++++++++++++++++++
 1 file changed, 90 insertions(+)
 create mode 100644 sysdeps/unix/sysv/linux/powerpc/powerpc64/sysconf.c

diff --git a/sysdeps/unix/sysv/linux/powerpc/powerpc64/sysconf.c b/sysdeps/unix/sysv/linux/powerpc/powerpc64/sysconf.c
new file mode 100644
index 0000000..9cac9df
--- /dev/null
+++ b/sysdeps/unix/sysv/linux/powerpc/powerpc64/sysconf.c
@@ -0,0 +1,90 @@
+/* Copyright (C) 2017 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library.  If not, see
+   <http://www.gnu.org/licenses/>.  */
+
+#include <assert.h>
+#include <stdbool.h>
+#include <stdlib.h>
+#include <unistd.h>
+#include <sys/auxv.h>
+
+
+static long int linux_sysconf (int name);
+
+/* Get the value of the system variable NAME.  */
+long int
+__sysconf (int name)
+{
+  unsigned long tmp;
+
+  /* The auxv entries describing the proper cache geometry were
+     added for kernel 4.11.  Thankfully, getauxval returns 0 when
+     the entry isn't present and is also the return from sysconf
+     when the value is unknown.
+
+     For the L1 linesize names, when the proper auxv entries are
+     not present, we fall back on older auxv entries that provide
+     the "block" linesize used for cache flushing.  It is a fair
+     guess that this does in fact correspond to the L1 shape.  */
+  switch (name)
+    {
+    case _SC_LEVEL1_ICACHE_SIZE:
+      return getauxval(AT_L1I_CACHESIZE);
+    case _SC_LEVEL1_ICACHE_ASSOC:
+      return (getauxval(AT_L1I_CACHEGEOMETRY) >> 16) & 0xffff;
+    case _SC_LEVEL1_ICACHE_LINESIZE:
+      tmp = getauxval(AT_L1I_CACHEGEOMETRY);
+      if (tmp)
+        return tmp & 0xffff;
+      return getauxval(AT_ICACHEBSIZE);
+
+    case _SC_LEVEL1_DCACHE_SIZE:
+      return getauxval(AT_L1D_CACHESIZE);
+    case _SC_LEVEL1_DCACHE_ASSOC:
+      return (getauxval(AT_L1D_CACHEGEOMETRY) >> 16) & 0xffff;
+    case _SC_LEVEL1_DCACHE_LINESIZE:
+      tmp = getauxval(AT_L1D_CACHEGEOMETRY);
+      if (tmp)
+        return tmp & 0xffff;
+      return getauxval(AT_DCACHEBSIZE);
+
+    case _SC_LEVEL2_CACHE_SIZE:
+      return getauxval(AT_L2_CACHESIZE);
+    case _SC_LEVEL2_CACHE_ASSOC:
+      return (getauxval(AT_L2_CACHEGEOMETRY) >> 16) & 0xffff;
+    case _SC_LEVEL2_CACHE_LINESIZE:
+      return getauxval(AT_L2_CACHEGEOMETRY) & 0xffff;
+
+    case _SC_LEVEL3_CACHE_SIZE:
+      return getauxval(AT_L3_CACHESIZE);
+    case _SC_LEVEL3_CACHE_ASSOC:
+      return (getauxval(AT_L3_CACHEGEOMETRY) >> 16) & 0xffff;
+    case _SC_LEVEL3_CACHE_LINESIZE:
+      return getauxval(AT_L3_CACHEGEOMETRY) & 0xffff;
+
+    case _SC_LEVEL4_CACHE_SIZE:
+    case _SC_LEVEL4_CACHE_ASSOC:
+    case _SC_LEVEL4_CACHE_LINESIZE:
+      return 0;
+    }
+
+  return linux_sysconf (name);
+}
+
+/* Now the generic Linux version.  */
+#undef __sysconf
+#define __sysconf static linux_sysconf
+#include <sysdeps/unix/sysv/linux/sysconf.c>
-- 
2.9.4

[PATCH 1/3] Guess L1 cache linesize for aarch64

[Richard Henderson]

Using the cache hierarchy linesize minimum in CTR_EL0.
See the comment within the code for rationale.

	* sysdeps/unix/sysv/linux/aarch64/sysconf.c: New file.

Cc: Marcus Shawcroft <marcus.shawcroft@arm.com>
---
 sysdeps/unix/sysv/linux/aarch64/sysconf.c | 55 +++++++++++++++++++++++++++++++
 1 file changed, 55 insertions(+)
 create mode 100644 sysdeps/unix/sysv/linux/aarch64/sysconf.c

diff --git a/sysdeps/unix/sysv/linux/aarch64/sysconf.c b/sysdeps/unix/sysv/linux/aarch64/sysconf.c
new file mode 100644
index 0000000..30608dd
--- /dev/null
+++ b/sysdeps/unix/sysv/linux/aarch64/sysconf.c
@@ -0,0 +1,55 @@
+/* Copyright (C) 2017 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library.  If not, see
+   <http://www.gnu.org/licenses/>.  */
+
+#include <assert.h>
+#include <stdbool.h>
+#include <stdlib.h>
+#include <unistd.h>
+
+
+static long int linux_sysconf (int name);
+
+/* Get the value of the system variable NAME.  */
+long int
+__sysconf (int name)
+{
+  unsigned ctr;
+
+  /* Unfortunately, the registers that contain the actual cache info
+     (CCSIDR_EL1, CLIDR_EL1, and CSSELR_EL1) are protected by the Linux
+     kernel (though they need not have been).  However, CTR_EL0 contains
+     the *minimum* linesize in the entire cache hierarchy, and is
+     accessible to userland, for use in __aarch64_sync_cache_range,
+     and it is a reasonable assumption that the L1 cache will have that
+     minimum line size.  */
+  switch (name)
+    {
+    case _SC_LEVEL1_ICACHE_LINESIZE:
+      asm("mrs\t%0, ctr_el0" : "=r"(ctr));
+      return 4 << (ctr & 0xf);
+    case _SC_LEVEL1_DCACHE_LINESIZE:
+      asm("mrs\t%0, ctr_el0" : "=r"(ctr));
+      return 4 << ((ctr >> 16) & 0xf);
+    }
+
+  return linux_sysconf (name);
+}
+
+/* Now the generic Linux version.  */
+#undef __sysconf
+#define __sysconf static linux_sysconf
+#include <sysdeps/unix/sysv/linux/sysconf.c>
-- 
2.9.4

Re: [PATCH 1/3] Guess L1 cache linesize for aarch64

[Siddhesh Poyarekar]

On Friday 09 June 2017 04:27 AM, Richard Henderson wrote:
> Using the cache hierarchy linesize minimum in CTR_EL0.
> See the comment within the code for rationale.
> 
> 	* sysdeps/unix/sysv/linux/aarch64/sysconf.c: New file.
> 

Looks good to me.

Thanks,
Siddhesh

Re: [PATCH 1/3] Guess L1 cache linesize for aarch64

[Andrew Pinski]

On Thu, Jun 8, 2017 at 10:51 PM, Siddhesh Poyarekar <siddhesh@gotplt.org> wrote:
> On Friday 09 June 2017 04:27 AM, Richard Henderson wrote:
>> Using the cache hierarchy linesize minimum in CTR_EL0.
>> See the comment within the code for rationale.
>>
>>       * sysdeps/unix/sysv/linux/aarch64/sysconf.c: New file.
>>
>
> Looks good to me.

And me too.

Thanks,
ANdrew

>
> Thanks,
> Siddhesh

Re: [PATCH 2/3] Add hidden_proto for getauxval

[Siddhesh Poyarekar]

On Friday 09 June 2017 04:27 AM, Richard Henderson wrote:
> ---
>  include/sys/auxv.h | 2 ++
>  misc/getauxval.c   | 1 +
>  2 files changed, 3 insertions(+)
> 
> diff --git a/include/sys/auxv.h b/include/sys/auxv.h
> index dede2c3..2e1f86b 100644
> --- a/include/sys/auxv.h
> +++ b/include/sys/auxv.h
> @@ -1 +1,3 @@
>  #include <misc/sys/auxv.h>
> +
> +libc_hidden_proto (getauxval)
> diff --git a/misc/getauxval.c b/misc/getauxval.c
> index c83fbce..458f58d 100644
> --- a/misc/getauxval.c
> +++ b/misc/getauxval.c
> @@ -43,3 +43,4 @@ __getauxval (unsigned long int type)
>  }
>  
>  weak_alias (__getauxval, getauxval)
> +libc_hidden_weak (getauxval)

This is fine, but you could just merge this into 3/3 (where this gets
used first), which I won't review since I haven't looked at a powerpc
box in anger for years :)

Siddhesh

Re: [PATCH 2/3] Add hidden_proto for getauxval

[Florian Weimer]

On 06/09/2017 12:57 AM, Richard Henderson wrote:
> +libc_hidden_weak (getauxval)

This should be libc_hidden_proto (__getauxval).  I don't think you can
call getauxval from libc.so because it leads to a namespace violation.

Thanks,
Florian

Re: [PATCH 3/3] Add cache info for powerpc64

[Florian Weimer]

On 06/09/2017 12:57 AM, Richard Henderson wrote:
> +    case _SC_LEVEL1_ICACHE_SIZE:
> +      return getauxval(AT_L1I_CACHESIZE);

Style (missing space) and potential namespace violation (should be
__getauxval).

Thanks,
Florian

Re: [PATCH 3/3] Add cache info for powerpc64

[Tulio Magno Quites Machado Filho]

Richard Henderson <rth@twiddle.net> writes:

> The actual cache info was added for 4.11, but have a guess at the
> L1 linesizes using info provided by older kernels.
>
> 	* sysdeps/unix/sysv/linux/powerpc/powerpc64/sysconf.c: New file.

There is already a patch reviewed and approved here:
https://patchwork.sourceware.org/patch/20202/

But this patch is blocked because it depends on another patch that's waiting
for review:
https://patchwork.sourceware.org/patch/20582/

-- 
Tulio Magno

Re: [PATCH 2/3] Add hidden_proto for getauxval

[Richard Henderson]

On 06/08/2017 11:58 PM, Florian Weimer wrote:
> On 06/09/2017 12:57 AM, Richard Henderson wrote:
>> +libc_hidden_weak (getauxval)
> 
> This should be libc_hidden_proto (__getauxval).  I don't think you can
> call getauxval from libc.so because it leads to a namespace violation.

 From libc.so you certainly can, because we'll actually call __GI_getauxval.


r~

Re: [PATCH 2/3] Add hidden_proto for getauxval

[Florian Weimer]

On 06/09/2017 07:45 PM, Richard Henderson wrote:
> On 06/08/2017 11:58 PM, Florian Weimer wrote:
>> On 06/09/2017 12:57 AM, Richard Henderson wrote:
>>> +libc_hidden_weak (getauxval)
>>
>> This should be libc_hidden_proto (__getauxval).  I don't think you can
>> call getauxval from libc.so because it leads to a namespace violation.
> 
> From libc.so you certainly can, because we'll actually call __GI_getauxval.

Yes, but not from libc.a, which is better to have the hidden alias for
the __ variant.

Florian

Re: [PATCH 3/3] Add cache info for powerpc64

[Richard Henderson]

On 06/09/2017 06:12 AM, Tulio Magno Quites Machado Filho wrote:
> Richard Henderson <rth@twiddle.net> writes:
> 
>> The actual cache info was added for 4.11, but have a guess at the
>> L1 linesizes using info provided by older kernels.
>>
>> 	* sysdeps/unix/sysv/linux/powerpc/powerpc64/sysconf.c: New file.
> 
> There is already a patch reviewed and approved here:
> https://patchwork.sourceware.org/patch/20202/


> +static long
> +auxv2sysconf (unsigned long type)
> +{
> +  long rc;
> +  rc = getauxval (type);
> +  if (rc == 0)
> +  {
> +    __set_errno (EINVAL);
> +    rc = -1;
> +  }
> +  return rc;

Setting EINVAL is wrong.  That would imply that the _SC_* name supplied by the 
caller is invalid somehow.

The return value should be 0 when the cache parameter is unknown, and -1 only 
when it is known that the cache level does not exist.

You should be using __getauxval to match the hidden_proto...

> 
> But this patch is blocked because it depends on another patch that's waiting
> for review:
> https://patchwork.sourceware.org/patch/20582/
> 

... defined here.  But this second patch does look right.


r~

Re: [PATCH 2/3] Add hidden_proto for getauxval

[Richard Henderson]

On 06/09/2017 10:49 AM, Florian Weimer wrote:
> On 06/09/2017 07:45 PM, Richard Henderson wrote:
>> On 06/08/2017 11:58 PM, Florian Weimer wrote:
>>> On 06/09/2017 12:57 AM, Richard Henderson wrote:
>>>> +libc_hidden_weak (getauxval)
>>>
>>> This should be libc_hidden_proto (__getauxval).  I don't think you can
>>> call getauxval from libc.so because it leads to a namespace violation.
>>
>>  From libc.so you certainly can, because we'll actually call __GI_getauxval.
> 
> Yes, but not from libc.a, which is better to have the hidden alias for
> the __ variant.

Quite right.


r~

Re: [PATCH 1/3] Guess L1 cache linesize for aarch64

[Siddhesh Poyarekar]

On Friday 09 June 2017 04:27 AM, Richard Henderson wrote:
> Using the cache hierarchy linesize minimum in CTR_EL0.
> See the comment within the code for rationale.
> 
> 	* sysdeps/unix/sysv/linux/aarch64/sysconf.c: New file.

Hi Richard,

I noticed that you did not push this patch yet.  Are you waiting for
additional review?

Siddhesh

Re: [PATCH 1/3] Guess L1 cache linesize for aarch64

[Szabolcs Nagy]

On 08/06/17 23:57, Richard Henderson wrote:
> +  /* Unfortunately, the registers that contain the actual cache info
> +     (CCSIDR_EL1, CLIDR_EL1, and CSSELR_EL1) are protected by the Linux
> +     kernel (though they need not have been).  However, CTR_EL0 contains
> +     the *minimum* linesize in the entire cache hierarchy, and is
> +     accessible to userland, for use in __aarch64_sync_cache_range,
> +     and it is a reasonable assumption that the L1 cache will have that
> +     minimum line size.  */

maybe

> +    case _SC_LEVEL1_ICACHE_LINESIZE:
> +    case _SC_LEVEL1_DCACHE_LINESIZE:

i can't find documentation for these, what meaning do users expect?

Re: [PATCH 1/3] Guess L1 cache linesize for aarch64

[Siddhesh Poyarekar]

On Tuesday 10 October 2017 03:49 PM, Szabolcs Nagy wrote:
> On 08/06/17 23:57, Richard Henderson wrote:
>> +  /* Unfortunately, the registers that contain the actual cache info
>> +     (CCSIDR_EL1, CLIDR_EL1, and CSSELR_EL1) are protected by the Linux
>> +     kernel (though they need not have been).  However, CTR_EL0 contains
>> +     the *minimum* linesize in the entire cache hierarchy, and is
>> +     accessible to userland, for use in __aarch64_sync_cache_range,
>> +     and it is a reasonable assumption that the L1 cache will have that
>> +     minimum line size.  */
> 
> maybe

Right, but that's an architectural detail that may not be relevant for
sysconf.  That is, the assumption may be suitable for the way the
sysconf is typically used.

>> +    case _SC_LEVEL1_ICACHE_LINESIZE:
>> +    case _SC_LEVEL1_DCACHE_LINESIZE:
> 
> i can't find documentation for these, what meaning do users expect?

Applications may use these hints to try and align their code/data
suitably or read/write data in an optimal manner.  It needs to be
documented and I hope to have a patch ready for it soon, but I wanted to
be sure that this patch was in place since otherwise the documentation
does not make sense.

Siddhesh

Re: [PATCH 1/3] Guess L1 cache linesize for aarch64

[Szabolcs Nagy]

On 10/10/17 11:37, Siddhesh Poyarekar wrote:
> On Tuesday 10 October 2017 03:49 PM, Szabolcs Nagy wrote:
>> On 08/06/17 23:57, Richard Henderson wrote:
>>> +    case _SC_LEVEL1_ICACHE_LINESIZE:
>>> +    case _SC_LEVEL1_DCACHE_LINESIZE:
>>
>> i can't find documentation for these, what meaning do users expect?
> 
> Applications may use these hints to try and align their code/data
> suitably or read/write data in an optimal manner.  It needs to be

that's different from the given libgcc clear_cache example

> documented and I hope to have a patch ready for it soon, but I wanted to
> be sure that this patch was in place since otherwise the documentation
> does not make sense.

either there is existing meaning or it's a new api with
some proposed meaning, i wanted to look at that to tell
if the implementation is acceptable.

Re: [PATCH 1/3] Guess L1 cache linesize for aarch64

[Siddhesh Poyarekar]

On Tuesday 10 October 2017 04:31 PM, Szabolcs Nagy wrote:
>> Applications may use these hints to try and align their code/data
>> suitably or read/write data in an optimal manner.  It needs to be
> 
> that's different from the given libgcc clear_cache example

Line sizes reported by ctr_el0 must be usable for clearing/invalidating
cache lines in a loop so it should be compatible with that use case too.

> either there is existing meaning or it's a new api with
> some proposed meaning, i wanted to look at that to tell
> if the implementation is acceptable.

This is an old API and the existing meaning is literally what it says,
the size of the L1 cache line.  There is no specification defining what
it can or cannot be used for since it is a GNU extension.

To comply with the name 1:1 we would have to emulate reeading clidr_el1,
ccsidr_el1, etc. which is overkill given that the value returned is
valid for almost everything.  The only place it goes wrong is where an
application might use it to report system architecture and that's where
we need to add a documentation snippet stating that it isn't quite what
it says it is, but is close.

The other alternative is to never implement this information on aarch64,
which is potentially sub-optimal for all of the other use cases.

Siddhesh

Re: [PATCH 1/3] Guess L1 cache linesize for aarch64

[Richard Henderson]

On 10/10/2017 03:19 AM, Szabolcs Nagy wrote:
> On 08/06/17 23:57, Richard Henderson wrote:
>> +  /* Unfortunately, the registers that contain the actual cache info
>> +     (CCSIDR_EL1, CLIDR_EL1, and CSSELR_EL1) are protected by the Linux
>> +     kernel (though they need not have been).  However, CTR_EL0 contains
>> +     the *minimum* linesize in the entire cache hierarchy, and is
>> +     accessible to userland, for use in __aarch64_sync_cache_range,
>> +     and it is a reasonable assumption that the L1 cache will have that
>> +     minimum line size.  */
> 
> maybe

Have you ever seen a system for which this was not true?
I don't believe I have.

>> +    case _SC_LEVEL1_ICACHE_LINESIZE:
>> +    case _SC_LEVEL1_DCACHE_LINESIZE:
> 
> i can't find documentation for these, what meaning do users expect?

They expect them to be exactly what they say they are.
The question is, what do they expect to be able to do with that information.

Speaking for myself, I expect to be able to dynamically align objects on this
linesize and for that to have some predictable effect on performance.


r~

Re: [PATCH 1/3] Guess L1 cache linesize for aarch64

[Szabolcs Nagy]

On 08/06/17 23:57, Richard Henderson wrote:
> Using the cache hierarchy linesize minimum in CTR_EL0.
> See the comment within the code for rationale.
> 
> 	* sysdeps/unix/sysv/linux/aarch64/sysconf.c: New file.

OK.

Re: [PATCH 1/3] Guess L1 cache linesize for aarch64

[Siddhesh Poyarekar]

On Tuesday 10 October 2017 07:50 PM, Richard Henderson wrote:
>> maybe
> 
> Have you ever seen a system for which this was not true?
> I don't believe I have.

The Qualcomm Falkor chip shows the L2 dcache line size here instead of L1.

> They expect them to be exactly what they say they are.
> The question is, what do they expect to be able to do with that information.
> 
> Speaking for myself, I expect to be able to dynamically align objects on this
> linesize and for that to have some predictable effect on performance.

This should continue to work well even for falkor.  In fact, if a core
does this (i.e. deviate from L1 line size in ctr_el0) and also not
perform better (or at least not regress) then the designers should
understand that such a design would be sub-optimal.

Siddhesh