From: William Cohen <wcohen@redhat.com>
To: Quentin Barnes <qbarnes@urbana.css.mot.com>
Cc: systemtap@sources.redhat.com
Subject: Re: testsuite and hardcoded timeouts
Date: Fri, 11 May 2007 21:08:00 -0000 [thread overview]
Message-ID: <4644DB46.1070705@redhat.com> (raw)
In-Reply-To: <20070511191420.GA12285@urbana.css.mot.com>
Quentin Barnes wrote:
> I mentioned this issue as an aside and asked about it a month ago,
> but I don't think it got a response at the time.
>
> In porting the Systemtap testsuite to an embedded ARM platform
> (~350Mhz CPU with 64MB and running using an NFS root and swap), I
> found many of the existing hardcoded timeout parameters are way too
> short. Several of them I had to at least triple if not increase by
> a larger factor, sometimes 6x-15x to get them to pass.
>
> How do we want to deal with this portability problem of hardcoded
> timeouts?
>
>
> There are a few ways I can think of to address this:
>
> 1) Let the hardcoded numbers stay, but up them large enough to
> handle the slowest platform we might ever run on. If that's
> still not slow enough someday, up them some more when the time
> comes.
>
> This has the advantage of simplicity, but can greatly slow down
> suite runs on faster processors when tests do get stuck.
>
> 2) Ban all standalone hardcoded timeouts replacing them with an
> expression involving a multiplier and/or a constant and a
> multiplier.
>
> This is not the cleanest because some tests are slow due to I/O
> bandwidth or paging where others are slow due to CPU limitations.
> But it does have the advantage that if someone is having timeout
> issues, they can up the multiplier value and rerun to see if the
> problem goes away without having to edit all sorts of wrapper
> scripts and tests.
>
> If we go with a multiplier, the multiplier could be set
> automatically by reading the cpuinfo and taking a stab at it based
> on the machine's BogoMIPS or MHz. We'd still need a way to have a user
> straightforwardly tweak it beyond that manually.
>
> Unfortunately, I don't understand the Systemtap testsuite framework
> yet well enough to make specific suggestions.
>
> Thoughts?
>
> Quentin
Hi Quentin,
I have some machines regularly downloading cvs snapshots of systemtap and
running the tests. I have encountered the same problem, particularly on the slow
pentium III machine. I have increased some of the timeouts as a result of this.
However, the problem is we don't know how long some of the tests take to run. In
addition to the processor speed the kernel/debuginfo could affect the time
required to build/install the tests.
I don't have good solutions to this problem. However, it might be good to start
listing the tests that are "too slow." People running probe might be okay with
a script taking a little time to get started, but they might not be so patient
when it takes minutes for the script to translate and start running. Running
them by hand with the "-v" to get information about which phases time is being
spent would be helpful.
-Will
next prev parent reply other threads:[~2007-05-11 21:08 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2007-05-11 19:14 Quentin Barnes
2007-05-11 21:08 ` William Cohen [this message]
2007-05-14 16:50 ` David Wilder
2007-05-14 20:43 ` William Cohen
2007-05-14 21:01 ` David Wilder
2007-05-15 22:35 ` Quentin Barnes
2007-05-15 22:47 ` Frank Ch. Eigler
2007-05-16 0:42 ` Quentin Barnes
2007-05-16 19:03 ` William Cohen
2007-05-18 0:36 ` Quentin Barnes
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=4644DB46.1070705@redhat.com \
--to=wcohen@redhat.com \
--cc=qbarnes@urbana.css.mot.com \
--cc=systemtap@sources.redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).