public inbox for gcc-patches@gcc.gnu.org
 help / color / mirror / Atom feed
* [PATCH] Fix detection of setrlimit in libstdc++ testsuite
@ 2015-11-11 16:56 Maxim Kuvyrkov
  2015-12-10 13:48 ` Maxim Kuvyrkov
  0 siblings, 1 reply; 8+ messages in thread
From: Maxim Kuvyrkov @ 2015-11-11 16:56 UTC (permalink / raw)
  To: GCC Patches

[-- Attachment #1: Type: text/plain, Size: 2337 bytes --]

Hi,

This patch fixes an obscure cross-testing problem that crashed (OOMed) our boards at Linaro.  Several tests in libstdc++ (e.g., [1]) limit themselves to some reasonable amount of RAM and then try to allocate 32 gigs.  Unfortunately, the configure test that checks presence of setrlimit is rather strange: if target is native, then try compile file with call to setrlimit -- if compilation succeeds, then use setrlimit, otherwise, ignore setrlimit.  The strange part is that the compilation check is done only for native targets, as if cross-toolchains can't generate working executables.  [This is rather odd, and I might be missing some underlaying caveat.]

Therefore, when testing a cross toolchain, the test [1] still tries to allocate 32GB of RAM with no setrlimit restrictions.  On most targets that people use for cross-testing this is not an issue because either
- the target is 32-bit, so there is no 32GB user-space to speak of, or
- the target board has small amount of RAM and no swap, so allocation immediately fails, or
- the target board has plenty of RAM, so allocating 32GB is not an issue.

However, if one is testing on a 64-bit board with 16GB or RAM and 16GB of swap, then one gets into an obscure near-OOM swapping condition.  This is exactly the case with cross-testing aarch64-linux-gnu toolchains on APM Mustang.

The attached patch removes "native" restriction from configure test for setrlimit.  This enables setrlimit restrictions on the testsuite, and the test [1] expectedly fails to allocate 32GB due to setrlimit restriction.

I have tested it on x86_64-linux-gnu and i686-linux-gnu native toolchains, and aarch64-linux-gnu and arm-linux-gnueabi[hf] cross-toolchains with no regressions [*].

OK to commit?

I didn't go as far as enabling setenv/locale tests when cross-testing libstdc++ because I remember of issues with generating locales in cross-built glibc.  In any case, locale tests are unlikely to OOM the test board the way that absence of setrlimit does.

[1] 27_io/ios_base/storage/2.cc

[*] Cross-testing using user-mode QEMU made 27_io/fpos/14775.cc execution test to FAIL.  This test uses setrlimit set max file size, and is misbehaving only under QEMU.  I believe this a QEMU issue with not handling setrlimit correctly.

--
Maxim Kuvyrkov
www.linaro.org



[-- Attachment #2: 0001-Use-setrlimit-for-testing-libstdc-in-cross-toolchain.patch --]
[-- Type: application/octet-stream, Size: 2039 bytes --]

From d4330fb411366ae39865c303c92e35cec73543c9 Mon Sep 17 00:00:00 2001
From: Maxim Kuvyrkov <maxim.kuvyrkov@linaro.org>
Date: Fri, 30 Oct 2015 14:05:40 +0100
Subject: [PATCH] Use setrlimit for testing libstdc++ in cross toolchains

	* acinclude.m4 (GLIBCXX_CONFIGURE_TESTSUITE): Check for presence of
	setrlimit on both native and cross targets.
	* configure: Regenerate.

Change-Id: I72745beedc7d8976e0c142806348450785face82
---
 libstdc++-v3/acinclude.m4 | 6 +++---
 libstdc++-v3/configure    | 4 ++--
 2 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/libstdc++-v3/acinclude.m4 b/libstdc++-v3/acinclude.m4
index abf2e93..aed5be8 100644
--- a/libstdc++-v3/acinclude.m4
+++ b/libstdc++-v3/acinclude.m4
@@ -632,10 +632,10 @@ dnl  baseline_dir
 dnl  baseline_subdir_switch
 dnl
 AC_DEFUN([GLIBCXX_CONFIGURE_TESTSUITE], [
-  if $GLIBCXX_IS_NATIVE ; then
-    # Do checks for resource limit functions.
-    GLIBCXX_CHECK_SETRLIMIT
+  # Do checks for resource limit functions.
+  GLIBCXX_CHECK_SETRLIMIT
 
+  if $GLIBCXX_IS_NATIVE ; then
     # Look for setenv, so that extended locale tests can be performed.
     GLIBCXX_CHECK_STDLIB_DECL_AND_LINKAGE_3(setenv)
   fi
diff --git a/libstdc++-v3/configure b/libstdc++-v3/configure
index 1eb6db4..614e703 100755
--- a/libstdc++-v3/configure
+++ b/libstdc++-v3/configure
@@ -78612,8 +78612,7 @@ $as_echo "$ac_cv_x86_rdrand" >&6; }
 
 # This depends on GLIBCXX_ENABLE_SYMVERS and GLIBCXX_IS_NATIVE.
 
-  if $GLIBCXX_IS_NATIVE ; then
-    # Do checks for resource limit functions.
+  # Do checks for resource limit functions.
 
   setrlimit_have_headers=yes
   for ac_header in unistd.h sys/time.h sys/resource.h
@@ -78842,6 +78841,7 @@ $as_echo "#define _GLIBCXX_RES_LIMITS 1" >>confdefs.h
 $as_echo "$ac_res_limits" >&6; }
 
 
+  if $GLIBCXX_IS_NATIVE ; then
     # Look for setenv, so that extended locale tests can be performed.
 
   { $as_echo "$as_me:${as_lineno-$LINENO}: checking for setenv declaration" >&5
-- 
1.9.1


^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH] Fix detection of setrlimit in libstdc++ testsuite
  2015-11-11 16:56 [PATCH] Fix detection of setrlimit in libstdc++ testsuite Maxim Kuvyrkov
@ 2015-12-10 13:48 ` Maxim Kuvyrkov
  2016-03-02 10:08   ` Maxim Kuvyrkov
  0 siblings, 1 reply; 8+ messages in thread
From: Maxim Kuvyrkov @ 2015-12-10 13:48 UTC (permalink / raw)
  To: GCC Patches; +Cc: Mike Stump

> On Nov 11, 2015, at 7:56 PM, Maxim Kuvyrkov <maxim.kuvyrkov@linaro.org> wrote:
> 
> Hi,
> 
> This patch fixes an obscure cross-testing problem that crashed (OOMed) our boards at Linaro.  Several tests in libstdc++ (e.g., [1]) limit themselves to some reasonable amount of RAM and then try to allocate 32 gigs.  Unfortunately, the configure test that checks presence of setrlimit is rather strange: if target is native, then try compile file with call to setrlimit -- if compilation succeeds, then use setrlimit, otherwise, ignore setrlimit.  The strange part is that the compilation check is done only for native targets, as if cross-toolchains can't generate working executables.  [This is rather odd, and I might be missing some underlaying caveat.]
> 
> Therefore, when testing a cross toolchain, the test [1] still tries to allocate 32GB of RAM with no setrlimit restrictions.  On most targets that people use for cross-testing this is not an issue because either
> - the target is 32-bit, so there is no 32GB user-space to speak of, or
> - the target board has small amount of RAM and no swap, so allocation immediately fails, or
> - the target board has plenty of RAM, so allocating 32GB is not an issue.
> 
> However, if one is testing on a 64-bit board with 16GB or RAM and 16GB of swap, then one gets into an obscure near-OOM swapping condition.  This is exactly the case with cross-testing aarch64-linux-gnu toolchains on APM Mustang.
> 
> The attached patch removes "native" restriction from configure test for setrlimit.  This enables setrlimit restrictions on the testsuite, and the test [1] expectedly fails to allocate 32GB due to setrlimit restriction.
> 
> I have tested it on x86_64-linux-gnu and i686-linux-gnu native toolchains, and aarch64-linux-gnu and arm-linux-gnueabi[hf] cross-toolchains with no regressions [*].
> 
> OK to commit?
> 
> I didn't go as far as enabling setenv/locale tests when cross-testing libstdc++ because I remember of issues with generating locales in cross-built glibc.  In any case, locale tests are unlikely to OOM the test board the way that absence of setrlimit does.
> 
> [1] 27_io/ios_base/storage/2.cc
> 
> [*] Cross-testing using user-mode QEMU made 27_io/fpos/14775.cc execution test to FAIL.  This test uses setrlimit set max file size, and is misbehaving only under QEMU.  I believe this a QEMU issue with not handling setrlimit correctly.
> 

Ping.

--
Maxim Kuvyrkov
www.linaro.org


^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH] Fix detection of setrlimit in libstdc++ testsuite
  2015-12-10 13:48 ` Maxim Kuvyrkov
@ 2016-03-02 10:08   ` Maxim Kuvyrkov
  2016-03-02 17:38     ` Mike Stump
  0 siblings, 1 reply; 8+ messages in thread
From: Maxim Kuvyrkov @ 2016-03-02 10:08 UTC (permalink / raw)
  To: GCC Patches; +Cc: Mike Stump

[-- Attachment #1: Type: text/plain, Size: 2708 bytes --]

PING ^ 2.  The patch is sitting without comments for 4+ months.

--
Maxim Kuvyrkov
www.linaro.org



> On Dec 10, 2015, at 4:47 PM, Maxim Kuvyrkov <maxim.kuvyrkov@linaro.org> wrote:
> 
>> On Nov 11, 2015, at 7:56 PM, Maxim Kuvyrkov <maxim.kuvyrkov@linaro.org> wrote:
>> 
>> Hi,
>> 
>> This patch fixes an obscure cross-testing problem that crashed (OOMed) our boards at Linaro.  Several tests in libstdc++ (e.g., [1]) limit themselves to some reasonable amount of RAM and then try to allocate 32 gigs.  Unfortunately, the configure test that checks presence of setrlimit is rather strange: if target is native, then try compile file with call to setrlimit -- if compilation succeeds, then use setrlimit, otherwise, ignore setrlimit.  The strange part is that the compilation check is done only for native targets, as if cross-toolchains can't generate working executables.  [This is rather odd, and I might be missing some underlaying caveat.]
>> 
>> Therefore, when testing a cross toolchain, the test [1] still tries to allocate 32GB of RAM with no setrlimit restrictions.  On most targets that people use for cross-testing this is not an issue because either
>> - the target is 32-bit, so there is no 32GB user-space to speak of, or
>> - the target board has small amount of RAM and no swap, so allocation immediately fails, or
>> - the target board has plenty of RAM, so allocating 32GB is not an issue.
>> 
>> However, if one is testing on a 64-bit board with 16GB or RAM and 16GB of swap, then one gets into an obscure near-OOM swapping condition.  This is exactly the case with cross-testing aarch64-linux-gnu toolchains on APM Mustang.
>> 
>> The attached patch removes "native" restriction from configure test for setrlimit.  This enables setrlimit restrictions on the testsuite, and the test [1] expectedly fails to allocate 32GB due to setrlimit restriction.
>> 
>> I have tested it on x86_64-linux-gnu and i686-linux-gnu native toolchains, and aarch64-linux-gnu and arm-linux-gnueabi[hf] cross-toolchains with no regressions [*].
>> 
>> OK to commit?
>> 
>> I didn't go as far as enabling setenv/locale tests when cross-testing libstdc++ because I remember of issues with generating locales in cross-built glibc.  In any case, locale tests are unlikely to OOM the test board the way that absence of setrlimit does.
>> 
>> [1] 27_io/ios_base/storage/2.cc
>> 
>> [*] Cross-testing using user-mode QEMU made 27_io/fpos/14775.cc execution test to FAIL.  This test uses setrlimit set max file size, and is misbehaving only under QEMU.  I believe this a QEMU issue with not handling setrlimit correctly.
>> 
> 
> Ping.
> 
> --
> Maxim Kuvyrkov
> www.linaro.org

[-- Attachment #2: 0001-Use-setrlimit-for-testing-libstdc-in-cross-toolchain.patch --]
[-- Type: application/octet-stream, Size: 2039 bytes --]

From d4330fb411366ae39865c303c92e35cec73543c9 Mon Sep 17 00:00:00 2001
From: Maxim Kuvyrkov <maxim.kuvyrkov@linaro.org>
Date: Fri, 30 Oct 2015 14:05:40 +0100
Subject: [PATCH] Use setrlimit for testing libstdc++ in cross toolchains

	* acinclude.m4 (GLIBCXX_CONFIGURE_TESTSUITE): Check for presence of
	setrlimit on both native and cross targets.
	* configure: Regenerate.

Change-Id: I72745beedc7d8976e0c142806348450785face82
---
 libstdc++-v3/acinclude.m4 | 6 +++---
 libstdc++-v3/configure    | 4 ++--
 2 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/libstdc++-v3/acinclude.m4 b/libstdc++-v3/acinclude.m4
index abf2e93..aed5be8 100644
--- a/libstdc++-v3/acinclude.m4
+++ b/libstdc++-v3/acinclude.m4
@@ -632,10 +632,10 @@ dnl  baseline_dir
 dnl  baseline_subdir_switch
 dnl
 AC_DEFUN([GLIBCXX_CONFIGURE_TESTSUITE], [
-  if $GLIBCXX_IS_NATIVE ; then
-    # Do checks for resource limit functions.
-    GLIBCXX_CHECK_SETRLIMIT
+  # Do checks for resource limit functions.
+  GLIBCXX_CHECK_SETRLIMIT
 
+  if $GLIBCXX_IS_NATIVE ; then
     # Look for setenv, so that extended locale tests can be performed.
     GLIBCXX_CHECK_STDLIB_DECL_AND_LINKAGE_3(setenv)
   fi
diff --git a/libstdc++-v3/configure b/libstdc++-v3/configure
index 1eb6db4..614e703 100755
--- a/libstdc++-v3/configure
+++ b/libstdc++-v3/configure
@@ -78612,8 +78612,7 @@ $as_echo "$ac_cv_x86_rdrand" >&6; }
 
 # This depends on GLIBCXX_ENABLE_SYMVERS and GLIBCXX_IS_NATIVE.
 
-  if $GLIBCXX_IS_NATIVE ; then
-    # Do checks for resource limit functions.
+  # Do checks for resource limit functions.
 
   setrlimit_have_headers=yes
   for ac_header in unistd.h sys/time.h sys/resource.h
@@ -78842,6 +78841,7 @@ $as_echo "#define _GLIBCXX_RES_LIMITS 1" >>confdefs.h
 $as_echo "$ac_res_limits" >&6; }
 
 
+  if $GLIBCXX_IS_NATIVE ; then
     # Look for setenv, so that extended locale tests can be performed.
 
   { $as_echo "$as_me:${as_lineno-$LINENO}: checking for setenv declaration" >&5
-- 
1.9.1


^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH] Fix detection of setrlimit in libstdc++ testsuite
  2016-03-02 10:08   ` Maxim Kuvyrkov
@ 2016-03-02 17:38     ` Mike Stump
  2016-03-04 15:27       ` Jonathan Wakely
  2016-04-05 11:20       ` Jonathan Wakely
  0 siblings, 2 replies; 8+ messages in thread
From: Mike Stump @ 2016-03-02 17:38 UTC (permalink / raw)
  To: Maxim Kuvyrkov; +Cc: GCC Patches, libstdc++

[-- Attachment #1: Type: text/plain, Size: 3029 bytes --]

On Mar 2, 2016, at 2:08 AM, Maxim Kuvyrkov <maxim.kuvyrkov@linaro.org> wrote:
> PING ^ 2.  The patch is sitting without comments for 4+ months.

So the libstdc++ people are usually pretty active and responsive, I usually let them review these sorts of patches as domain experts.  I only kick in if they are unreasonably absent.  Reviewing the case, I don’t see any hint that the libstdc++ list was ever sent an email.  You will want to include it in any patches and pings.

I did look at the patch, and it does seem reasonable.  Given that the libstdc+ folks never saw it, I’lll defer to them.  You can ping patches once a week, if you would like.

Below is the original email and patch:


This patch fixes an obscure cross-testing problem that crashed (OOMed) our boards at Linaro.  Several tests in libstdc++ (e.g., [1]) limit themselves to some reasonable amount of RAM and then try to allocate 32 gigs.  Unfortunately, the configure test that checks presence of setrlimit is rather strange: if target is native, then try compile file with call to setrlimit -- if compilation succeeds, then use setrlimit, otherwise, ignore setrlimit.  The strange part is that the compilation check is done only for native targets, as if cross-toolchains can't generate working executables.  [This is rather odd, and I might be missing some underlaying caveat.]

Therefore, when testing a cross toolchain, the test [1] still tries to allocate 32GB of RAM with no setrlimit restrictions.  On most targets that people use for cross-testing this is not an issue because either
- the target is 32-bit, so there is no 32GB user-space to speak of, or
- the target board has small amount of RAM and no swap, so allocation immediately fails, or
- the target board has plenty of RAM, so allocating 32GB is not an issue.

However, if one is testing on a 64-bit board with 16GB or RAM and 16GB of swap, then one gets into an obscure near-OOM swapping condition.  This is exactly the case with cross-testing aarch64-linux-gnu toolchains on APM Mustang.

The attached patch removes "native" restriction from configure test for setrlimit.  This enables setrlimit restrictions on the testsuite, and the test [1] expectedly fails to allocate 32GB due to setrlimit restriction.

I have tested it on x86_64-linux-gnu and i686-linux-gnu native toolchains, and aarch64-linux-gnu and arm-linux-gnueabi[hf] cross-toolchains with no regressions [*].

OK to commit?

I didn't go as far as enabling setenv/locale tests when cross-testing libstdc++ because I remember of issues with generating locales in cross-built glibc. In any case, locale tests are unlikely to OOM the test board the way that absence of setrlimit does.

[1] 27_io/ios_base/storage/2.cc

[*] Cross-testing using user-mode QEMU made 27_io/fpos/14775.cc execution test to FAIL.  This test uses setrlimit set max file size, and is misbehaving only under QEMU.  I believe this a QEMU issue with not handling setrlimit correctly.

--
Maxim Kuvyrkov
www.linaro.org



[-- Attachment #2: 0001-Use-setrlimit-for-testing-libstdc-in-cross-toolchain.patch --]
[-- Type: application/octet-stream, Size: 2039 bytes --]

From d4330fb411366ae39865c303c92e35cec73543c9 Mon Sep 17 00:00:00 2001
From: Maxim Kuvyrkov <maxim.kuvyrkov@linaro.org>
Date: Fri, 30 Oct 2015 14:05:40 +0100
Subject: [PATCH] Use setrlimit for testing libstdc++ in cross toolchains

	* acinclude.m4 (GLIBCXX_CONFIGURE_TESTSUITE): Check for presence of
	setrlimit on both native and cross targets.
	* configure: Regenerate.

Change-Id: I72745beedc7d8976e0c142806348450785face82
---
 libstdc++-v3/acinclude.m4 | 6 +++---
 libstdc++-v3/configure    | 4 ++--
 2 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/libstdc++-v3/acinclude.m4 b/libstdc++-v3/acinclude.m4
index abf2e93..aed5be8 100644
--- a/libstdc++-v3/acinclude.m4
+++ b/libstdc++-v3/acinclude.m4
@@ -632,10 +632,10 @@ dnl  baseline_dir
 dnl  baseline_subdir_switch
 dnl
 AC_DEFUN([GLIBCXX_CONFIGURE_TESTSUITE], [
-  if $GLIBCXX_IS_NATIVE ; then
-    # Do checks for resource limit functions.
-    GLIBCXX_CHECK_SETRLIMIT
+  # Do checks for resource limit functions.
+  GLIBCXX_CHECK_SETRLIMIT
 
+  if $GLIBCXX_IS_NATIVE ; then
     # Look for setenv, so that extended locale tests can be performed.
     GLIBCXX_CHECK_STDLIB_DECL_AND_LINKAGE_3(setenv)
   fi
diff --git a/libstdc++-v3/configure b/libstdc++-v3/configure
index 1eb6db4..614e703 100755
--- a/libstdc++-v3/configure
+++ b/libstdc++-v3/configure
@@ -78612,8 +78612,7 @@ $as_echo "$ac_cv_x86_rdrand" >&6; }
 
 # This depends on GLIBCXX_ENABLE_SYMVERS and GLIBCXX_IS_NATIVE.
 
-  if $GLIBCXX_IS_NATIVE ; then
-    # Do checks for resource limit functions.
+  # Do checks for resource limit functions.
 
   setrlimit_have_headers=yes
   for ac_header in unistd.h sys/time.h sys/resource.h
@@ -78842,6 +78841,7 @@ $as_echo "#define _GLIBCXX_RES_LIMITS 1" >>confdefs.h
 $as_echo "$ac_res_limits" >&6; }
 
 
+  if $GLIBCXX_IS_NATIVE ; then
     # Look for setenv, so that extended locale tests can be performed.
 
   { $as_echo "$as_me:${as_lineno-$LINENO}: checking for setenv declaration" >&5
-- 
1.9.1


^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH] Fix detection of setrlimit in libstdc++ testsuite
  2016-03-02 17:38     ` Mike Stump
@ 2016-03-04 15:27       ` Jonathan Wakely
  2016-04-05 11:20       ` Jonathan Wakely
  1 sibling, 0 replies; 8+ messages in thread
From: Jonathan Wakely @ 2016-03-04 15:27 UTC (permalink / raw)
  To: Mike Stump; +Cc: Maxim Kuvyrkov, GCC Patches, libstdc++

On 02/03/16 09:38 -0800, Mike Stump wrote:
>On Mar 2, 2016, at 2:08 AM, Maxim Kuvyrkov <maxim.kuvyrkov@linaro.org> wrote:
>> PING ^ 2.  The patch is sitting without comments for 4+ months.
>
>So the libstdc++ people are usually pretty active and responsive, I usually let them review these sorts of patches as domain experts.  I only kick in if they are unreasonably absent.  Reviewing the case, I don’t see any hint that the libstdc++ list was ever sent an email.  You will want to include it in any patches and pings.

Thanks for forwarding this, Mike.

Libstdc++ patches need to be sent to the libstdc++ list, see both
https://gcc.gnu.org/lists.html and
https://gcc.gnu.org/onlinedocs/libstdc++/manual/appendix_contributing.html#list.patches

I don't comment on patches I'm not aware of :-)


>I did look at the patch, and it does seem reasonable.  Given that the libstdc+ folks never saw it, I’lll defer to them.  You can ping patches once a week, if you would like.

Paolo and I are both attending the WG21 C++ meeting this week. I'm
busy and have not had much time to keep up with email, but I'll review
the patch ASAP.

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH] Fix detection of setrlimit in libstdc++ testsuite
  2016-03-02 17:38     ` Mike Stump
  2016-03-04 15:27       ` Jonathan Wakely
@ 2016-04-05 11:20       ` Jonathan Wakely
  2016-08-31 11:22         ` Maxim Kuvyrkov
  1 sibling, 1 reply; 8+ messages in thread
From: Jonathan Wakely @ 2016-04-05 11:20 UTC (permalink / raw)
  To: Maxim Kuvyrkov; +Cc: Mike Stump, GCC Patches, libstdc++

>This patch fixes an obscure cross-testing problem that crashed (OOMed) our boards at Linaro.  Several tests in libstdc++ (e.g., [1]) limit themselves to some reasonable amount of RAM and then try to allocate 32 gigs.  Unfortunately, the configure test that checks presence of setrlimit is rather strange: if target is native, then try compile file with call to setrlimit -- if compilation succeeds, then use setrlimit, otherwise, ignore setrlimit.  The strange part is that the compilation check is done only for native targets, as if cross-toolchains can't generate working executables.  [This is rather odd, and I might be missing some underlaying caveat.]

I went spelunking, and the IS_NATIVE check has been there since
r70167, which replaced:

  if test  x"$GLIBCXX_IS_CROSS_COMPILING" = xfalse; then
    # Do checks for memory limit functions.
    GLIBCXX_CHECK_SETRLIMIT

That arrived in r68067, but that seems to eb just a refactoring, and I
got lost tracking it further.

So there has been a similar check since at least 2003.

>Therefore, when testing a cross toolchain, the test [1] still tries to allocate 32GB of RAM with no setrlimit restrictions.  On most targets that people use for cross-testing this is not an issue because either
>- the target is 32-bit, so there is no 32GB user-space to speak of, or
>- the target board has small amount of RAM and no swap, so allocation immediately fails, or
>- the target board has plenty of RAM, so allocating 32GB is not an issue.
>
>However, if one is testing on a 64-bit board with 16GB or RAM and 16GB of swap, then one gets into an obscure near-OOM swapping condition.  This is exactly the case with cross-testing aarch64-linux-gnu toolchains on APM Mustang.
>
>The attached patch removes "native" restriction from configure test for setrlimit.  This enables setrlimit restrictions on the testsuite, and the test [1] expectedly fails to allocate 32GB due to setrlimit restriction.
>
>I have tested it on x86_64-linux-gnu and i686-linux-gnu native toolchains, and aarch64-linux-gnu and arm-linux-gnueabi[hf] cross-toolchains with no regressions [*].
>
>OK to commit?

This issue has been present for well over a decade so it doesn't seem
critical to fix in stage4, but as it only affects the testsuite I am
OK with the change if the RMs have no objections.

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH] Fix detection of setrlimit in libstdc++ testsuite
  2016-04-05 11:20       ` Jonathan Wakely
@ 2016-08-31 11:22         ` Maxim Kuvyrkov
  2016-08-31 13:23           ` Jonathan Wakely
  0 siblings, 1 reply; 8+ messages in thread
From: Maxim Kuvyrkov @ 2016-08-31 11:22 UTC (permalink / raw)
  To: Jonathan Wakely; +Cc: Mike Stump, GCC Patches, libstdc++

> On Apr 5, 2016, at 2:20 PM, Jonathan Wakely <jwakely@redhat.com> wrote:
> 
>> This patch fixes an obscure cross-testing problem that crashed (OOMed) our boards at Linaro.  Several tests in libstdc++ (e.g., [1]) limit themselves to some reasonable amount of RAM and then try to allocate 32 gigs.  Unfortunately, the configure test that checks presence of setrlimit is rather strange: if target is native, then try compile file with call to setrlimit -- if compilation succeeds, then use setrlimit, otherwise, ignore setrlimit.  The strange part is that the compilation check is done only for native targets, as if cross-toolchains can't generate working executables.  [This is rather odd, and I might be missing some underlaying caveat.]
> 
> I went spelunking, and the IS_NATIVE check has been there since
> r70167, which replaced:
> 
> if test  x"$GLIBCXX_IS_CROSS_COMPILING" = xfalse; then
>   # Do checks for memory limit functions.
>   GLIBCXX_CHECK_SETRLIMIT
> 
> That arrived in r68067, but that seems to eb just a refactoring, and I
> got lost tracking it further.
> 
> So there has been a similar check since at least 2003.
> 
>> Therefore, when testing a cross toolchain, the test [1] still tries to allocate 32GB of RAM with no setrlimit restrictions.  On most targets that people use for cross-testing this is not an issue because either
>> - the target is 32-bit, so there is no 32GB user-space to speak of, or
>> - the target board has small amount of RAM and no swap, so allocation immediately fails, or
>> - the target board has plenty of RAM, so allocating 32GB is not an issue.
>> 
>> However, if one is testing on a 64-bit board with 16GB or RAM and 16GB of swap, then one gets into an obscure near-OOM swapping condition.  This is exactly the case with cross-testing aarch64-linux-gnu toolchains on APM Mustang.
>> 
>> The attached patch removes "native" restriction from configure test for setrlimit.  This enables setrlimit restrictions on the testsuite, and the test [1] expectedly fails to allocate 32GB due to setrlimit restriction.
>> 
>> I have tested it on x86_64-linux-gnu and i686-linux-gnu native toolchains, and aarch64-linux-gnu and arm-linux-gnueabi[hf] cross-toolchains with no regressions [*].
>> 
>> OK to commit?
> 
> This issue has been present for well over a decade so it doesn't seem
> critical to fix in stage4, but as it only affects the testsuite I am
> OK with the change if the RMs have no objections.

Hi Jonathan,

This issue dropped off my patch queue.  The original patch still applies without conflicts, and I'm retesting it on fresh mainline -- both cross and native.

OK to commit if no regressions?

Thanks,

--
Maxim Kuvyrkov
www.linaro.org



^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH] Fix detection of setrlimit in libstdc++ testsuite
  2016-08-31 11:22         ` Maxim Kuvyrkov
@ 2016-08-31 13:23           ` Jonathan Wakely
  0 siblings, 0 replies; 8+ messages in thread
From: Jonathan Wakely @ 2016-08-31 13:23 UTC (permalink / raw)
  To: Maxim Kuvyrkov; +Cc: Mike Stump, GCC Patches, libstdc++

On 31/08/16 14:22 +0300, Maxim Kuvyrkov wrote:
>> On Apr 5, 2016, at 2:20 PM, Jonathan Wakely <jwakely@redhat.com> wrote:
>>
>>> This patch fixes an obscure cross-testing problem that crashed (OOMed) our boards at Linaro.  Several tests in libstdc++ (e.g., [1]) limit themselves to some reasonable amount of RAM and then try to allocate 32 gigs.  Unfortunately, the configure test that checks presence of setrlimit is rather strange: if target is native, then try compile file with call to setrlimit -- if compilation succeeds, then use setrlimit, otherwise, ignore setrlimit.  The strange part is that the compilation check is done only for native targets, as if cross-toolchains can't generate working executables.  [This is rather odd, and I might be missing some underlaying caveat.]
>>
>> I went spelunking, and the IS_NATIVE check has been there since
>> r70167, which replaced:
>>
>> if test  x"$GLIBCXX_IS_CROSS_COMPILING" = xfalse; then
>>   # Do checks for memory limit functions.
>>   GLIBCXX_CHECK_SETRLIMIT
>>
>> That arrived in r68067, but that seems to eb just a refactoring, and I
>> got lost tracking it further.
>>
>> So there has been a similar check since at least 2003.
>>
>>> Therefore, when testing a cross toolchain, the test [1] still tries to allocate 32GB of RAM with no setrlimit restrictions.  On most targets that people use for cross-testing this is not an issue because either
>>> - the target is 32-bit, so there is no 32GB user-space to speak of, or
>>> - the target board has small amount of RAM and no swap, so allocation immediately fails, or
>>> - the target board has plenty of RAM, so allocating 32GB is not an issue.
>>>
>>> However, if one is testing on a 64-bit board with 16GB or RAM and 16GB of swap, then one gets into an obscure near-OOM swapping condition.  This is exactly the case with cross-testing aarch64-linux-gnu toolchains on APM Mustang.
>>>
>>> The attached patch removes "native" restriction from configure test for setrlimit.  This enables setrlimit restrictions on the testsuite, and the test [1] expectedly fails to allocate 32GB due to setrlimit restriction.
>>>
>>> I have tested it on x86_64-linux-gnu and i686-linux-gnu native toolchains, and aarch64-linux-gnu and arm-linux-gnueabi[hf] cross-toolchains with no regressions [*].
>>>
>>> OK to commit?
>>
>> This issue has been present for well over a decade so it doesn't seem
>> critical to fix in stage4, but as it only affects the testsuite I am
>> OK with the change if the RMs have no objections.
>
>Hi Jonathan,
>
>This issue dropped off my patch queue.  The original patch still applies without conflicts, and I'm retesting it on fresh mainline -- both cross and native.
>
>OK to commit if no regressions?

Yes, OK. Thanks.


^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2016-08-31 13:23 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-11-11 16:56 [PATCH] Fix detection of setrlimit in libstdc++ testsuite Maxim Kuvyrkov
2015-12-10 13:48 ` Maxim Kuvyrkov
2016-03-02 10:08   ` Maxim Kuvyrkov
2016-03-02 17:38     ` Mike Stump
2016-03-04 15:27       ` Jonathan Wakely
2016-04-05 11:20       ` Jonathan Wakely
2016-08-31 11:22         ` Maxim Kuvyrkov
2016-08-31 13:23           ` Jonathan Wakely

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).