* Many new test failures (execvp: Too many open files)
@ 2007-07-05 13:13 Mark Wielaard
2007-07-05 14:23 ` Andrew Cagney
0 siblings, 1 reply; 9+ messages in thread
From: Mark Wielaard @ 2007-07-05 13:13 UTC (permalink / raw)
To: frysk
Hi,
On my Fedora 7 x86 machine (2.6.21-1.3228.fc7 SMP) I am suddenly seeing
lots and lots of new test failures/errors. Lots mention "execvp: Too
many open files", I am not seeing this on my x86_64 Fedora Core 6 setup.
And they only started to massively fail today it seems (although on irc
some people say they saw them already earlier this week). Anyone know
what that is about?
I also saw that at the same time there was a renaming of "broken" to
"unresolved" for the TestCases. Is that a functional change or just a
textual change? Maybe some tests were mislabeled in the process, so they
suddenly started running on my system and before they didn't and so
triggering this "execvp: Too many open files" now?
Cheers,
Mark
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Many new test failures (execvp: Too many open files)
2007-07-05 13:13 Many new test failures (execvp: Too many open files) Mark Wielaard
@ 2007-07-05 14:23 ` Andrew Cagney
2007-07-05 15:14 ` Mark Wielaard
0 siblings, 1 reply; 9+ messages in thread
From: Andrew Cagney @ 2007-07-05 14:23 UTC (permalink / raw)
To: Mark Wielaard; +Cc: frysk
Mark Wielaard wrote:
> Hi,
>
> On my Fedora 7 x86 machine (2.6.21-1.3228.fc7 SMP) I am suddenly seeing
> lots and lots of new test failures/errors. Lots mention "execvp: Too
> many open files", I am not seeing this on my x86_64 Fedora Core 6 setup.
> And they only started to massively fail today it seems (although on irc
> some people say they saw them already earlier this week). Anyone know
> what that is about?
>
>
I'm guessing this wasn't a scratch build. Bug 4742.
> I also saw that at the same time there was a renaming of "broken" to
> "unresolved" for the TestCases. Is that a functional change or just a
> textual change? Maybe some tests were mislabeled in the process, so they
> suddenly started running on my system and before they didn't and so
> triggering this "execvp: Too many open files" now?
>
Two things to do in this situation are to review the changes you suspect
could be problematic, and to scratch build before/after trees of the
specific change showing it caused the breakage.
Andrew
>
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Many new test failures (execvp: Too many open files)
2007-07-05 14:23 ` Andrew Cagney
@ 2007-07-05 15:14 ` Mark Wielaard
2007-07-05 16:14 ` Andrew Cagney
0 siblings, 1 reply; 9+ messages in thread
From: Mark Wielaard @ 2007-07-05 15:14 UTC (permalink / raw)
To: Andrew Cagney; +Cc: frysk
On Thu, 2007-07-05 at 10:23 -0400, Andrew Cagney wrote:
> Mark Wielaard wrote:
> > On my Fedora 7 x86 machine (2.6.21-1.3228.fc7 SMP) I am suddenly seeing
> > lots and lots of new test failures/errors. Lots mention "execvp: Too
> > many open files", I am not seeing this on my x86_64 Fedora Core 6 setup.
> > And they only started to massively fail today it seems (although on irc
> > some people say they saw them already earlier this week). Anyone know
> > what that is about?
> >
> I'm guessing this wasn't a scratch build.
Of course it was a scratch build, otherwise I would not reported it :)
I always do a scratch build when doing any cvs update because our build
system doesn't seem very robust in the face of file deletion, addition
or renaming. I wasn't even aware incremental build should work unless
you only made changes to existing files. Is it supposed to?
I always start my day with a cvs update and then a scratch build and get
myself some breakfast :)
> Bug 4742.
Interesting, make sure you report that one upstream. I don't believe
many, if any, other project rely on the dependency generation feature of
gcj so they probably have not heard about that yet.
> > I also saw that at the same time there was a renaming of "broken" to
> > "unresolved" for the TestCases. Is that a functional change or just a
> > textual change? Maybe some tests were mislabeled in the process, so they
> > suddenly started running on my system and before they didn't and so
> > triggering this "execvp: Too many open files" now?
> >
> Two things to do in this situation are to review the changes you suspect
> could be problematic, and to scratch build before/after trees of the
> specific change showing it caused the breakage.
Could you explain why the changes were made and what they were supposed
to do? That would help in reviewing the issue.
Thanks,
Mark
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Many new test failures (execvp: Too many open files)
2007-07-05 15:14 ` Mark Wielaard
@ 2007-07-05 16:14 ` Andrew Cagney
2007-07-05 17:00 ` Mark Wielaard
2007-07-09 9:25 ` Mark Wielaard
0 siblings, 2 replies; 9+ messages in thread
From: Andrew Cagney @ 2007-07-05 16:14 UTC (permalink / raw)
To: Mark Wielaard; +Cc: frysk
Mark Wielaard wrote:
> On Thu, 2007-07-05 at 10:23 -0400, Andrew Cagney wrote:
>
>> Mark Wielaard wrote:
>>
>>> On my Fedora 7 x86 machine (2.6.21-1.3228.fc7 SMP) I am suddenly seeing
>>> lots and lots of new test failures/errors. Lots mention "execvp: Too
>>> many open files", I am not seeing this on my x86_64 Fedora Core 6 setup.
>>> And they only started to massively fail today it seems (although on irc
>>> some people say they saw them already earlier this week). Anyone know
>>> what that is about?
>>>
>>>
>> I'm guessing this wasn't a scratch build.
>>
>
> Of course it was a scratch build, otherwise I would not reported it :)
> I always do a scratch build when doing any cvs update because our build
> system doesn't seem very robust in the face of file deletion, addition
> or renaming. I wasn't even aware incremental build should work unless
> you only made changes to existing files. Is it supposed to?
> I always start my day with a cvs update and then a scratch build and get
> myself some breakfast :)
>
>
>> Bug 4742.
>>
>
> Interesting, make sure you report that one upstream. I don't believe
> many, if any, other project rely on the dependency generation feature of
> gcj so they probably have not heard about that yet.
>
>
>>> I also saw that at the same time there was a renaming of "broken" to
>>> "unresolved" for the TestCases. Is that a functional change or just a
>>> textual change? Maybe some tests were mislabeled in the process, so they
>>> suddenly started running on my system and before they didn't and so
>>> triggering this "execvp: Too many open files" now?
>>>
>>>
>> Two things to do in this situation are to review the changes you suspect
>> could be problematic, and to scratch build before/after trees of the
>> specific change showing it caused the breakage.
>>
>
> Could you explain why the changes were made and what they were supposed
> to do? That would help in reviewing the issue.
>
You described it; s/brokenXXX/unresolvedOnYYY/ ; this finishes the
introduction of UNRESOLVED I started some time ago. You'll also see I
killed some unused functions; and simplified unresolvedOnUtrace (as fc5
is gone). Perhaps it is that last change?
Andrew
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Many new test failures (execvp: Too many open files)
2007-07-05 16:14 ` Andrew Cagney
@ 2007-07-05 17:00 ` Mark Wielaard
2007-07-05 18:03 ` Andrew Cagney
2007-07-09 9:25 ` Mark Wielaard
1 sibling, 1 reply; 9+ messages in thread
From: Mark Wielaard @ 2007-07-05 17:00 UTC (permalink / raw)
To: Andrew Cagney; +Cc: frysk
[-- Attachment #1: Type: text/plain, Size: 1124 bytes --]
Hi Andrew,
On Thu, 2007-07-05 at 12:14 -0400, Andrew Cagney wrote:
> > Could you explain why the changes were made and what they were supposed
> > to do? That would help in reviewing the issue.
> >
> You described it; s/brokenXXX/unresolvedOnYYY/ ; this finishes the
> introduction of UNRESOLVED I started some time ago.
OK, I must have missed that sorry. So there is not supposed to be any
functional change just a global search/replace on the whole code base
because unresolved is a better term than broken?
> You'll also see I
> killed some unused functions; and simplified unresolvedOnUtrace (as fc5
> is gone). Perhaps it is that last change?
I'll try that out tomorrow morning. On irc we discussed the problem of
leaking file descriptors in things like Elf objects which currently seem
to rely on garbage collection to close out unused descriptors. So it
might also be some subtle change in memory use that make the garbage
collector trigger earlier or later. In that case it is probably that we
just fail to close()/destroy() the right object at the right time.
Cheers,
Mark
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Many new test failures (execvp: Too many open files)
2007-07-05 17:00 ` Mark Wielaard
@ 2007-07-05 18:03 ` Andrew Cagney
0 siblings, 0 replies; 9+ messages in thread
From: Andrew Cagney @ 2007-07-05 18:03 UTC (permalink / raw)
To: Mark Wielaard; +Cc: frysk
Mark Wielaard wrote:
> I'll try that out tomorrow morning. On irc we discussed the problem of
> leaking file descriptors in things like Elf objects which currently seem
> to rely on garbage collection to close out unused descriptors. So it
> might also be some subtle change in memory use that make the garbage
> collector trigger earlier or later. In that case it is probably that we
> just fail to close()/destroy() the right object at the right time.
>
Thanks.
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Many new test failures (execvp: Too many open files)
2007-07-05 16:14 ` Andrew Cagney
2007-07-05 17:00 ` Mark Wielaard
@ 2007-07-09 9:25 ` Mark Wielaard
2007-07-09 12:50 ` Andrew Cagney
1 sibling, 1 reply; 9+ messages in thread
From: Mark Wielaard @ 2007-07-09 9:25 UTC (permalink / raw)
To: Andrew Cagney; +Cc: frysk
Hi Andrew,
On Thu, 2007-07-05 at 12:14 -0400, Andrew Cagney wrote:
> Mark Wielaard wrote:
> > Could you explain why the changes were made and what they were supposed
> > to do? That would help in reviewing the issue.
> >
> You described it; s/brokenXXX/unresolvedOnYYY/ ; this finishes the
> introduction of UNRESOLVED I started some time ago. You'll also see I
> killed some unused functions; and simplified unresolvedOnUtrace (as fc5
> is gone). Perhaps it is that last change?
Indeed. The old version returned false on Fedora 7, the new version
returns true. Meaning in particular that several tests in
TestTaskObserver and TestTaskObserver now run while they didn't before
this patch. But this doesn't seem to be the root cause for the execvp
issues.
The thing that worked around it for me for now seems to be to disable
the frysk.rt.TestStepping.testASMSingleStep() test that seems to fail
with ERROR java.lang.ArrayIndexOutOfBoundsException: 0
Apparently this failure causes no more file descriptors to be available.
Reported as http://sourceware.org/bugzilla/show_bug.cgi?id=4751
And adjusted the TestStepping accordingly.
2007-07-09 Mark Wielaard <mwielaard@redhat.com>
* TestStepping.java (testASMSingleStep): Add unresolved for bug #4751.
Cheers,
Mark
diff -u -r1.39 TestStepping.java
--- frysk-core/frysk/rt/TestStepping.java 5 Jul 2007 03:01:50 -0000 1.39
+++ frysk-core/frysk/rt/TestStepping.java 9 Jul 2007 09:23:40 -0000
@@ -254,6 +254,9 @@
if (unresolvedOnPPC(3277))
return;
+ if (unresolved(4751))
+ return;
+
initial = true;
this.lineMap = new HashMap();
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Many new test failures (execvp: Too many open files)
2007-07-09 9:25 ` Mark Wielaard
@ 2007-07-09 12:50 ` Andrew Cagney
2007-07-10 11:14 ` unresolvedOnUtrace (Was: Many new test failures (execvp: Too many open files)) Mark Wielaard
0 siblings, 1 reply; 9+ messages in thread
From: Andrew Cagney @ 2007-07-09 12:50 UTC (permalink / raw)
To: Mark Wielaard; +Cc: frysk
Mike and I tracked down the likely cause - Dwfl's aren't been cleaned up
aggressively enough (something that is dependent on many things
including the garbage collector). We're making two changes:
- adding teardown code to frysk.dwfl that will close all open Dwfls
- enabling code that closes old dwfl's as new ones are created
One thing though:
> Indeed. The old version returned false on Fedora 7, the new version
> returns true. Meaning in particular that several tests in
> TestTaskObserver and TestTaskObserver now run while they didn't before
> this patch. But this doesn't seem to be the root cause for the execvp
> issues.
>
This is not correct. unresolvedOnUtrace returning TRUE causes the test
to be _skipped_; so this causes less tests to be run. Given that Fedora
7 has UTRACE, it appears I fixed a latent bug.
Andrew
^ permalink raw reply [flat|nested] 9+ messages in thread
* unresolvedOnUtrace (Was: Many new test failures (execvp: Too many open files))
2007-07-09 12:50 ` Andrew Cagney
@ 2007-07-10 11:14 ` Mark Wielaard
0 siblings, 0 replies; 9+ messages in thread
From: Mark Wielaard @ 2007-07-10 11:14 UTC (permalink / raw)
To: Andrew Cagney; +Cc: frysk
[-- Attachment #1: Type: text/plain, Size: 948 bytes --]
Hi Andrew,
On Mon, 2007-07-09 at 08:50 -0400, Andrew Cagney wrote:
> This is not correct. unresolvedOnUtrace returning TRUE causes the test
> to be _skipped_; so this causes less tests to be run. Given that Fedora
> 7 has UTRACE, it appears I fixed a latent bug.
Duh, you are right, I had my results crossed. Sorry about that.
But that is interesting because I now really double checked and it looks
like on Fedora 7 (2.6.21-1.3228.fc7 x86 SMP) bugs #3595 and #3737 are
fixed and unresolvedOnUtrace() is setting the following test results to
UNRESOLVED for these 4 tests that actually PASS on my machine:
Running testMultiThreadedStoppedAckDaemon(frysk.proc.TestProcStopped) ...PASS
Running testMultiThreadedStoppedDetached(frysk.proc.TestProcStopped) ...PASS
Running testDetachExitingMainTask(frysk.proc.TestTaskObserver) ...PASS
Running testDetachExitingOtherTask(frysk.proc.TestTaskObserver) ...PASS
Cheers,
Mark
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2007-07-10 11:14 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2007-07-05 13:13 Many new test failures (execvp: Too many open files) Mark Wielaard
2007-07-05 14:23 ` Andrew Cagney
2007-07-05 15:14 ` Mark Wielaard
2007-07-05 16:14 ` Andrew Cagney
2007-07-05 17:00 ` Mark Wielaard
2007-07-05 18:03 ` Andrew Cagney
2007-07-09 9:25 ` Mark Wielaard
2007-07-09 12:50 ` Andrew Cagney
2007-07-10 11:14 ` unresolvedOnUtrace (Was: Many new test failures (execvp: Too many open files)) Mark Wielaard
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).