libstdc++ test suite still drives machine into swap

libstdc++ test suite still drives machine into swap

[Zack Weinberg]

The libstdc++ test suite still drives my machine, which has plenty of
RAM thank you, into swap.  I suspect this is because ulimit -d does
not actually work on Linux, because libc allocates huge blocks with
anonymous mmap, and anonymous mmap ignores that particular limit.

The last time I brought this up I tried to explain why these tests
shouldn't be done at all and no one got it.  Code like

  try {
    csz01 = str01.max_size();
    std::string str03(csz01 - 1, 'A');
    VERIFY( str03.size() == csz01 - 1 );
    VERIFY( str03.size() <= str03.capacity() );
  } catch (std::bad_alloc) {
    VERIFY (true);
  }

*does not test* that size() is equal to csz01 - 1 or <= capacity().
It doesn't even test that std::bad_alloc is thrown.

What actually happens is that execution never gets past the
std::string constructor.  Malloc lies.  It gives you a pointer to
memory that does not exist, on the theory that you won't *really* use
all of it.  Then the operating system kills the process - if you're
lucky - while it's trying to write 'A' over a gigabyte of virtual
memory that the computer does not actually have.[1]

The *only* way you can test std::bad_alloc reliably is to override
libc malloc with something that fails on huge requests.  Resource
limits are not sufficient.  Assuming that malloc will return NULL 
most certainly does not cut it.

The *only* way you can test the constraints on max_size, size,
capacity, etc. is if you can somehow create a basic_string<>
instantiation for which max_size is small enough that you *can*
allocate that much memory without bringing the computer to its knees.[2]

[Sample code above was taken from 21_strings/ctor_copy_dtor.cc.  That
particular test has been #if 0'ed out with a comment referencing one
of the previous threads on this subject.  However, not all the tests
with this problem have been disabled.  Right now I'm not sure which
test is the current problem.]

[I might add that surely it is only necessary to test this sort of
thing *once*, not the five or six times it appears to be done in the
current test suite.]

-- 
zw   The beginning of almost every story is actually a bone, something with
     which to court the dog, which may bring you closer to the lady.
     	-- Amos Oz, _The Story Begins_

[1] If you're unlucky, the computer crashes, or the operating system
kills half a dozen random innocent processes before it hits the one
that's eating all the memory.

[2] I tried to figure out how to do that and got lost in a maze of
twisty little template classes.  But I don't really speak C++.  I
would suggest that #define __STD_STRING_MAX_SIZE or some such should
override the default for std::string's max_size() method.

Re: libstdc++ test suite still drives machine into swap

[Wolfram Gloger]

Hello,

> The libstdc++ test suite still drives my machine, which has plenty of
> RAM thank you, into swap.  I suspect this is because ulimit -d does
> not actually work on Linux, because libc allocates huge blocks with
> anonymous mmap, and anonymous mmap ignores that particular limit.

Yes, so if you want to limit RAM usage, why not just use 'ulimit -v'?

> What actually happens is that execution never gets past the
> std::string constructor.  Malloc lies.  It gives you a pointer to
> memory that does not exist, on the theory that you won't *really* use
> all of it.

Actuallly, I think it's more accurate to state that the kernel lies.
But we all like over-commitment, don't we..

> The *only* way you can test std::bad_alloc reliably is to override
> libc malloc with something that fails on huge requests.  Resource
> limits are not sufficient.

I believe they are, 'ulimit -v' works for me.  Of course there may
also be other solutions.

Regards,
Wolfram.

Re: libstdc++ test suite still drives machine into swap

[Zack Weinberg]

On Wed, Jun 27, 2001 at 12:10:26PM +0200, Wolfram Gloger wrote:
> Hello,
> 
> > The libstdc++ test suite still drives my machine, which has plenty of
> > RAM thank you, into swap.  I suspect this is because ulimit -d does
> > not actually work on Linux, because libc allocates huge blocks with
> > anonymous mmap, and anonymous mmap ignores that particular limit.
> 
> Yes, so if you want to limit RAM usage, why not just use 'ulimit -v'?

Well, for one thing, with my /bin/sh:

$ ulimit -v 2048
ulimit: Illegal option -v

I suppose we could wire a setrlimit(RLIMIT_AS, ...) call into the test
case itself - frankly, that is also the only way I'd be confident that
the limit actually affected the relevant process.

But that's not the right call to make on other systems.  If we went
this way, it would make sense to export a helper function from
libstdc++ that did the Right Thing for you.

> > What actually happens is that execution never gets past the
> > std::string constructor.  Malloc lies.  It gives you a pointer to
> > memory that does not exist, on the theory that you won't *really* use
> > all of it.
> 
> Actuallly, I think it's more accurate to state that the kernel lies.
> But we all like over-commitment, don't we..

True enough.  I've seen people argue that overcommitment is a
violation of the C standard (not that I necessarily agree).  The
program certainly can't tell whether it's the kernel or the C library
doing the lying...

-- 
zw     I *will* wrestle it into shape eventually. I *will* clean it out,
       *without* resorting to the redirection of a major navigable river.
       	-- Ray Radlein on lumber rooms

Re: libstdc++ test suite still drives machine into swap

[Phil Edwards]

I'm revisting this topic because my machine is dying.

The results from mkcheck don't match the results from the dejagnu check.
Which probably needs looking into, but I don't have the dejagnu knowledge
to do so myself.  (Nor, to be brutally honest, the desire.  I could express
my professional optinon about dejagnu, but Mark has asked us to not use
those words on the public lists.)

Anyhow, I noticed that the machine swapped itself to pieces, which lead
me to this thread from 26 June.

There is a minor patch at the end.  The real point of this message is to
get the patch approved.  If no discussion is generated, that's okay.  :-)

A month ago, Zack Weinberg wrote:
> The libstdc++ test suite still drives my machine, which has plenty of
> RAM thank you, into swap.  I suspect this is because ulimit -d does
> not actually work on Linux, because libc allocates huge blocks with
> anonymous mmap, and anonymous mmap ignores that particular limit.

Somewhere is a thread from me, Gaby, and a couple other people explaining
that ulimit doesn't work with dejagnu anyhow.  The ulimit is executed
in a subshell, which instantly exits.  DG doesn't allow executation of
arbitrary commands before running a test executable in the same shell as
the executable.  The same technique works fine in the mkcheck script.

On Linux, you must use -v instead of -d.  Which we do, in mklimit.
Still doesn't help DG.


> The last time I brought this up I tried to explain why these tests
> shouldn't be done at all and no one got it.

I may be wrong, but I think you're mistaken.

> What actually happens is that execution never gets past the
> std::string constructor.  Malloc lies.  It gives you a pointer to
> memory that does not exist, on the theory that you won't *really* use
> all of it.  Then the operating system kills the process - if you're
> lucky - while it's trying to write 'A' over a gigabyte of virtual
> memory that the computer does not actually have.[1]
[...]
> [1] If you're unlucky, the computer crashes, or the operating system
> kills half a dozen random innocent processes before it hits the one
> that's eating all the memory.

Maybe on your OS.  This doesn't happen to me under Solaris, or Linux, or
OSF/1, or IRIX.  Even without resource limits being applied to the user's
shell and inherited by subprocesses.  Granted that many implementations
of malloc overcommit, I just don't see random processes dying.  More on
this below.


> The *only* way you can test the constraints on max_size, size,
> capacity, etc. is if you can somehow create a basic_string<>
> instantiation for which max_size is small enough that you *can*
> allocate that much memory without bringing the computer to its knees.[2]
[...]
> [2] I tried to figure out how to do that and got lost in a maze of
> twisty little template classes.  But I don't really speak C++.  I
> would suggest that #define __STD_STRING_MAX_SIZE or some such should
> override the default for std::string's max_size() method.

I like this idea on the face of it.  I haven't really thought it through,
but this sounds useful.


> [Sample code above was taken from 21_strings/ctor_copy_dtor.cc.  That
> particular test has been #if 0'ed out with a comment referencing one
> of the previous threads on this subject.  However, not all the tests
> with this problem have been disabled.  Right now I'm not sure which
> test is the current problem.]

We're doing the same thing in 21_strings/insert.cc; we create a maximum-sized
string and then copy it (fortunately it's refcounting and only goes into
terminal thrash once).  Included is a patch that does the same limiting
we did last October.  Comments anyone?

> [I might add that surely it is only necessary to test this sort of
> thing *once*, not the five or six times it appears to be done in the
> current test suite.]

And here's the real issue, or close to the real issue.  Do we really
need to test these corner cases, especially when we know that malloc
might overcommit, or that the kernel is useless when it comes to properly
killing processes that exceed their resource limits?  Do we need to test
"identical" cases using different member functions?

I say yes, for two reasons:  first, the cases might not actually be
identical, and the point of the testsuite is to let us mess with the
implementation without changing the semantics.  Let's say we test a
"too large" string using its constructor; do we need to test "too large"
semantics when inserting data into an already-close-to-maximal string?
I submit that we do.  (Zack's suggestion about artificially limiting
max_size()/npos would make this painless, I think, but I don't know how
to pass arbitrary -D options inside dejagnu.)

Secondly:  some user out there is going to try and allocate (size_t)(-1)
bytes for a string, and we have to behave correctly.  This argues /against/
limiting max_size()/npos artificially.  We need to know what actually
happens in these cases.  And if that causes the kernel to assassinate
random processes, then the solution IMO isn't to change the testcase,
the solution is to tell the user "on such-and-such OS, don't do that".


'Night,
Phil


2001-07-31  Phil Edwards  <pme@sources.redhat.com>

	* insert.cc:  Don't allocate large amounts of memory; see change
	to ctor_copy_dtor.cc on 2000-10-21.


Index: insert.cc
===================================================================
RCS file: /home/pme/Repositories/GCC/gcc/libstdc++-v3/testsuite/21_strings/insert.cc,v
retrieving revision 1.3
diff -u -3 -p -r1.3 insert.cc
--- insert.cc	2001/05/12 16:51:41	1.3
+++ insert.cc	2001/08/01 01:11:05
@@ -79,12 +79,14 @@ int test01(void)
 
   csz01 = str01.max_size();
   try {
-    std::string str04(csz01, 'b'); 
+//    std::string str04(csz01, 'b'); 
+std::string str04(1, 'b'); 
     str03 = str04; 
     csz02 = str02.size();
     try {
       str03.insert(0, str02, 0, 5);
-      VERIFY( false );
+//      VERIFY( false );
+VERIFY( str03.size() == 6 );
     }		 
     catch(std::length_error& fail) {
       VERIFY( true );

Re: libstdc++ test suite still drives machine into swap

[Gabriel Dos Reis]

Phil Edwards <pedwards@disaster.jaj.com> writes:

| I'm revisting this topic because my machine is dying.
| 
| The results from mkcheck don't match the results from the dejagnu check.
| Which probably needs looking into, but I don't have the dejagnu knowledge
| to do so myself.  (Nor, to be brutally honest, the desire.  I could express
| my professional optinon about dejagnu, but Mark has asked us to not use
| those words on the public lists.)

I guess, it is really time to stop doing all kind of activity and bug Rob
and myself until I get it work properly.

[...]

| 'Night,

It is now morning on the other side of the Atlantic here, :-))

-- Gaby

Re: libstdc++ test suite still drives machine into swap

[Benjamin Kosnik]

> | The results from mkcheck don't match the results from the dejagnu check.
> 
> I guess, it is really time to stop doing all kind of activity and bug Rob
> and myself until I get it work properly.

I was just about to start doing this.

Anyway. 

I think Zack and I will just have to disagree on what constitutes 
a testsuite. For me, it includes pathological edge cases as defined in 
the standard. He thinks they are tested elsewhere, I disagree. Since this 
topic seems to be activated by cron jobs every 3 months, I just thought 
I'd reply, yet again.

Phil please don't remove these tests.

-benjamin

Re: libstdc++ test suite still drives machine into swap

[Gabriel Dos Reis]

Benjamin Kosnik <bkoz@redhat.com> writes:

| > | The results from mkcheck don't match the results from the dejagnu check.
| > 
| > I guess, it is really time to stop doing all kind of activity and bug Rob
| > and myself until I get it work properly.
| 
| I was just about to start doing this.

Ah, so?

-- Gaby

Re: libstdc++ test suite still drives machine into swap

[Phil Edwards]

On Tue, Jul 31, 2001 at 10:59:52PM -0700, Benjamin Kosnik wrote:
> I think Zack and I will just have to disagree on what constitutes 
> a testsuite. For me, it includes pathological edge cases as defined in 
> the standard. He thinks they are tested elsewhere, I disagree.

I agree with you here; testing pathological cases is Good.


> Since this 
> topic seems to be activated by cron jobs every 3 months,

Yes, my "email some reminders" cron table entry fired Sunday night.

I'm not joking....


> Phil please don't remove these tests.

But I disagree with you here.  :-)  My patch just makes insert.cc do the
same thing that ctor_copy_dtor.cc is already doing.  All the tests are
still there.  I'll be the first one to suggest re-pathologicalizing (!) the
size function arguments once we can make DG force the limits properly.

Having the machine kill itself for 5-15 minutes on every test run is
just annoying.  It does however force us to take a typing break.

I could also work up a patch to artificially limit max_size()/npos.
That could conceivably be useful to users in any case.


Phil

-- 
Would I had phrases that are not known, utterances that are strange, in
new language that has not been used, free from repetition, not an utterance
which has grown stale, which men of old have spoken.
                                     - anonymous Egyptian scribe, c.1700 BC

Re: libstdc++ test suite still drives machine into swap

[Stephen M. Webb]

On Wed, 01 Aug 2001, Phil Edwards wrote:

> .....I'll be the first one to suggest re-pathologicalizing (!) the
> size function arguments once we can make DG force the limits properly.
> 
> Having the machine kill itself for 5-15 minutes on every test run is
> just annoying.  It does however force us to take a typing break.
> 
> I could also work up a patch to artificially limit max_size()/npos.
> That could conceivably be useful to users in any case.

Keep in mind that all of the max_size member functions are broken for
all the containers, it's just that there are no automated testsuite entires for
them (yet).  


Stephen M. Webb

Re: libstdc++ test suite still drives machine into swap

[Phil Edwards]

On Thu, Aug 02, 2001 at 04:24:46AM -0400, Stephen M. Webb wrote:
> Keep in mind that all of the max_size member functions are broken for
> all the containers, it's just that there are no automated testsuite entires for
> them (yet).  

Sorry, this is the first I can recall hearing of this.  Can you point me
to the right place in the archives?  (Has this been brought up before?)


Phil

-- 
Would I had phrases that are not known, utterances that are strange, in
new language that has not been used, free from repetition, not an utterance
which has grown stale, which men of old have spoken.
                                     - anonymous Egyptian scribe, c.1700 BC

Re: libstdc++ test suite still drives machine into swap

[Stephen M. Webb]

On Wed, 01 Aug 2001, Phil Edwards wrote:
> On Thu, Aug 02, 2001 at 04:24:46AM -0400, Stephen M. Webb wrote:
> > Keep in mind that all of the max_size member functions are broken for
> > all the containers, it's just that there are no automated testsuite entires for
> > them (yet).  
> 
> Sorry, this is the first I can recall hearing of this.  Can you point me
> to the right place in the archives?  (Has this been brought up before?)

I can't say if it's been brought up before, but since using the result of any
tests of max_size on any containers (or anything using the default allocator)
will have the same exhaust-the-memory-till-you-die effect as basic_string,
I thought it was relevant.  I guess it's on the recod now.

I noticed while I was trying to write some fairly comprehensive test
suites that all the containers simply return size_type(-1) for max_size. 
That's incorrect for a number of reasons, least of which is that's not the
behaviour mandated by the standard, but it also means valid tests using
max_size would kill my machine regardless of whether the underlying
malloc lies or not, so I'm not including a test for it.  Even with that ulimit
fix to dejagnu, I can't guarantee certain categories of tests are valid unless
max_size returns valid numbers, although I can still run them without fear
of killing my machine.

Unfortunately the fix for max_size is somewhat large and convoluted.  It
would require proper (standard-mandated) support for allocators in the
containers, which should be a desireable thing anyway.  It would also
require that malloc not lie or that the default allocator work around that
lie somehow.

I though the first order of business before proposing such a change was to
develop a comprehensive regression and conformance test suite, one that
runs without killing my machine.  That's where what you were talking about
came in.

_______
Stephen M. Webb

Re: libstdc++ test suite still drives machine into swap

[Phil Edwards]

On Thu, Aug 02, 2001 at 08:26:15PM -0400, Stephen M. Webb wrote:
> I noticed while I was trying to write some fairly comprehensive test
> suites that all the containers simply return size_type(-1) for max_size. 
> That's incorrect for a number of reasons, least of which is that's not the
> behaviour mandated by the standard,

Have you looked at LWG issue 197?  Looks like they're going to specify
max_size for an allocator as something different than max_size for
a container, which is probably smart.  But for the first case, where
it represents "the biggest size that could be meaningfully passed to
allocate()," #197 notes that the LWG believes the intended meaning to be
"as much memory as your platform could conceivably have."

And arguably size_type(-1) is a fair representation of that amount of memory.
Something in me wants to see a more user-controlled value, but whatever.


> I though the first order of business before proposing such a change was to
> develop a comprehensive regression and conformance test suite, one that
> runs without killing my machine.  That's where what you were talking about
> came in.

For now I'm no longer running the dejagnu testsuite.  I'm using the mkcheck
script instead, and charting the number of unexpected failures.  (Well,
number of failures minus the number of *.cc files containing the word
"xfail".)

Phil

-- 
Would I had phrases that are not known, utterances that are strange, in
new language that has not been used, free from repetition, not an utterance
which has grown stale, which men of old have spoken.
                                     - anonymous Egyptian scribe, c.1700 BC

Re: libstdc++ test suite still drives machine into swap

[Phil Edwards]

On Thu, Aug 02, 2001 at 07:40:10PM -0400, Zack Weinberg wrote:
> On Thu, Aug 02, 2001 at 07:21:30PM -0400, Phil Edwards wrote:
> > That /is/ the test.  It's not only "test big strings," it's also "test
> > the failure mode of string::allocator running out of memory."
> 
> I've misunderstood then.  This snippet (from ctor_copy_dtor.cc) very much
> reads as if it's the size() and capacity() invariants (inside the try block)
> that are meant to be tested.

Yeah... we're really testing one of two possible outcomes.  /If/ it works,
then we /also/ need to check the size() and capacity() invariants.  /If/
it fails, then we /also/ need to check the failure mode.

The fact that the test has caused this much confusion points up its lack
of a good comment, I guess.  Once we hammer out a good strategy, I'll
write something better above the tests.


> > I don't think we have any control over whether the kernel guns the process
> > rather than malloc setting ENOMEM.  We could perhaps replace malloc() with
> > our own version, but then we're not testing the same execution environment
> > that the users would have.
> 
> I'm not aware of any implementation under which malloc will succeed if you
> try to allocate a chunk of memory larger than the total memory rlimit all
> at once.  (Is that sentence clear?)

I think it's clear... the test is almost guaranteed to fail, unless the
user has chunkloads of virtual memory, in which case we still have to do
the right thing- arrg.  If we all agreed to go back to 16-bit size_t's
this problem would go away.  :-|


Phil

-- 
Would I had phrases that are not known, utterances that are strange, in
new language that has not been used, free from repetition, not an utterance
which has grown stale, which men of old have spoken.
                                     - anonymous Egyptian scribe, c.1700 BC

Re: libstdc++ test suite still drives machine into swap

[Stephen M. Webb]

On Thu, 02 Aug 2001, Hans-Peter Nilsson wrote:
> On Thu, 2 Aug 2001, Phil Edwards wrote:
> > I don't think we have any control over whether the kernel guns the process
> > rather than malloc setting ENOMEM.  We could perhaps replace malloc() with
> > our own version, but then we're not testing the same execution environment
> > that the users would have.
> 
> Wouldn't the same argument apply for using setrlimit during testing?
> 
> On the surface this problem looks very much like it would be
> solved by just linking those tests with a special-purpose Very
> Simple malloc library that fails for Large Chunks, avoiding
> side-effects of the host library overcommitting.  (Though
> perhaps those systems are then not usable as host systems. ;-)

Perhaps the solution is as simple (?) as modifying new and allocator::allocate
so that they are aware of getrlimit and getrusage values, and throw if a memory
allocation request would exceed those values.  Changing malloc is probably not
such a good idea, but new and allocator::allocate have better-defined
behaviour and are under C++ standard library control.

A build-time check would be necessary for such support, but on those platforms
without getrusage and getrlimit, you'd be no further behind.


_______
Stephen m. Webb

Re: libstdc++ test suite still drives machine into swap

[Hans-Peter Nilsson]

On Thu, 2 Aug 2001, Phil Edwards wrote:
> That /is/ the test.  It's not only "test big strings," it's also "test
> the failure mode of string::allocator running out of memory."
>
> If allocation fails, we need to make sure that bad_alloc is thrown (as
> opposed to throwing something else, or not throwing anything and just
> returning).  There's nothing "after" that code, really, at least nothing
> that we're specifically testing there and nowhere else.  If it succeeds,
> fine, if it fails and throws bad_alloc, fine, anything else (that we can
> control) is not fine.
>
> I don't think we have any control over whether the kernel guns the process
> rather than malloc setting ENOMEM.  We could perhaps replace malloc() with
> our own version, but then we're not testing the same execution environment
> that the users would have.

Nosy bystander chimes in: Wouldn't the same argument apply for
using setrlimit during testing?

On the surface this problem looks very much like it would be
solved by just linking those tests with a special-purpose Very
Simple malloc library that fails for Large Chunks, avoiding
side-effects of the host library overcommitting.  (Though
perhaps those systems are then not usable as host systems. ;-)

brgds, H-P

Re: libstdc++ test suite still drives machine into swap

[Zack Weinberg]

On Thu, Aug 02, 2001 at 07:21:30PM -0400, Phil Edwards wrote:
> On Thu, Aug 02, 2001 at 06:41:41PM -0400, Zachary Weinberg wrote:
> > Allocating and initializing a string of
> > max_size() can only succeed on a machine with more than a gigabyte
> > of virtual memory.  On a normal machine with less memory, either
> > the process dies, or bad_alloc is thrown.  Either way, the actual
> > test - the code intended to be executed after the allocation
> > completes - never gets executed.
> 
> That /is/ the test.  It's not only "test big strings," it's also "test
> the failure mode of string::allocator running out of memory."

I've misunderstood then.  This snippet (from ctor_copy_dtor.cc) very much
reads as if it's the size() and capacity() invariants (inside the try block)
that are meant to be tested.

  // Build a maxsize - 1 lengthed string consisting of all A's
  try {
    std::string str03(csz01 - 1, 'A');
    VERIFY( str03.size() == csz01 - 1 );
    VERIFY( str03.size() <= str03.capacity() );
  }              
  // NB: bad_alloc is regrettable but entirely kosher for
  // out-of-memory situations.
  catch(std::bad_alloc& fail) {
    VERIFY( true );
  }
  catch(...) {
    VERIFY( false );
  }

same deal with the problematic tests in insert.cc.

> I don't think we have any control over whether the kernel guns the process
> rather than malloc setting ENOMEM.  We could perhaps replace malloc() with
> our own version, but then we're not testing the same execution environment
> that the users would have.

I'm not aware of any implementation under which malloc will succeed if you
try to allocate a chunk of memory larger than the total memory rlimit all
at once.  (Is that sentence clear?)

> > The problem of the machine getting driven into swap by the test is
> > an independent problem with the same cause.  I would suggest that
> > that one be solved by adding a utility routine to libstdc++ which
> > makes the appropriate setrlimit() calls to constrain process memory,
> > and then calling it from test cases that are known to thrash the computer.
> 
> That's a very good idea.
> 
> I'm familiar with ulimit at the shell level but not the underlying calls.
> Do we need anything other than a simpleminded setrlimit() autoconf test,
> i.e., is the function signature fairly well-standardized across platforms
> which have it?

To the best of my knowledge it's always int setrlimit(int, struct rlimit *);
The tricky bit is that the appropriate RLIMIT_* constant varies with the
operating system - RLIMIT_DATA, RLIMIT_RSS, RLIMIT_AS, etc.  It may work
to set all of the limits that are #defined.

zw

Re: libstdc++ test suite still drives machine into swap

[Phil Edwards]

On Thu, Aug 02, 2001 at 06:41:41PM -0400, Zachary Weinberg wrote:
> Allocating and initializing a string of
> max_size() can only succeed on a machine with more than a gigabyte
> of virtual memory.  On a normal machine with less memory, either
> the process dies, or bad_alloc is thrown.  Either way, the actual
> test - the code intended to be executed after the allocation
> completes - never gets executed.

That /is/ the test.  It's not only "test big strings," it's also "test
the failure mode of string::allocator running out of memory."

If allocation fails, we need to make sure that bad_alloc is thrown (as
opposed to throwing something else, or not throwing anything and just
returning).  There's nothing "after" that code, really, at least nothing
that we're specifically testing there and nowhere else.  If it succeeds,
fine, if it fails and throws bad_alloc, fine, anything else (that we can
control) is not fine.

I don't think we have any control over whether the kernel guns the process
rather than malloc setting ENOMEM.  We could perhaps replace malloc() with
our own version, but then we're not testing the same execution environment
that the users would have.

(Am I making sense?  Feels like we're talking past each other.)


> The problem of the machine getting driven into swap by the test is
> an independent problem with the same cause.  I would suggest that
> that one be solved by adding a utility routine to libstdc++ which
> makes the appropriate setrlimit() calls to constrain process memory,
> and then calling it from test cases that are known to thrash the computer.

That's a very good idea.

I'm familiar with ulimit at the shell level but not the underlying calls.
Do we need anything other than a simpleminded setrlimit() autoconf test,
i.e., is the function signature fairly well-standardized across platforms
which have it?


Phil

-- 
Would I had phrases that are not known, utterances that are strange, in
new language that has not been used, free from repetition, not an utterance
which has grown stale, which men of old have spoken.
                                     - anonymous Egyptian scribe, c.1700 BC

Re: libstdc++ test suite still drives machine into swap

[Zachary Weinberg]

Benjamin Kosnik wrote:

> I think Zack and I will just have to disagree on what constitutes
> a testsuite. For me, it includes pathological edge cases as defined in
> the standard. He thinks they are tested elsewhere, I disagree. Since this
> topic seems to be activated by cron jobs every 3 months, I just thought
> I'd reply, yet again.

This is not exactly my position.  I agree that pathological edge
cases should be tested.  My argument is that the existing code
which claims to test them, doesn't.  This should be quite clear
just from inspection.  Allocating and initializing a string of
max_size() can only succeed on a machine with more than a gigabyte
of virtual memory.  On a normal machine with less memory, either
the process dies, or bad_alloc is thrown.  Either way, the actual
test - the code intended to be executed after the allocation
completes - never gets executed.

The problem of the machine getting driven into swap by the test is
an independent problem with the same cause.  I would suggest that
that one be solved by adding a utility routine to libstdc++ which
makes the appropriate setrlimit() calls to constrain process memory,
and then calling it from test cases that are known to thrash the computer.
This should not, however, be seen as a solution to the problem that
the tests are not testing what they are intended to.

zw
semi-temporary new address
not subscribed to gcc lists at the moment