Build Pass from file name hyphen->underscore

Build Pass from file name hyphen->underscore

[naresh kamboju]

Hi,

I have been investigating SystemTap test suite, its sub modules and
expected pass and expected failures.

As per my investigation I have noticed there are test cases name with
– hyphen are not able to build. When I have changed it to underscore _
it is able to build.
As per stap, If you add ‘-‘ (hyphen) in the file name, then stap is
able to consider as invalid module name.

BUILD_FAIL				BUILD_PASS
aux_syscalls-embedded.stp -> aux_syscalls_embedded.stp


There were almost 36 test cases in this category.
Are considering these expected failures?

Best regards
Naresh Kamboju

Re: Build Pass from file name hyphen->underscore

[Frank Ch. Eigler]

Hi -

> As per my investigation I have noticed there are test cases name with
> hyphen are not able to build. When I have changed it to underscore _
> it is able to build.
> [...]
> BUILD_FAIL				BUILD_PASS
> aux_syscalls-embedded.stp -> aux_syscalls_embedded.stp

This would be a bug, and I haven't seen anything like it.  Dejazilla
shows lots of PASS results with that testcase, so something strange
must be going on.  Maybe a character set issue on your machine?

- FChE

Regression on statistics

[Wenji Huang]

Hi,

There's one regression on statistics operation. For example,

Running ./systemtap.maps/ix.exp ...
executing: stap ./systemtap.maps/ix.stp
FAIL: ./systemtap.maps/ix.stp
line 1: expected "foo[0]: count:3  sum:98  avg:32  min:-2  max:100"
Got "foo[0]: count:3  sum:98  avg:32  min:0  max:-2"

It seems this is introduced by the commit.

commit 98c783852039061db8c1611742660aaded0eab77
Author: Anton Vorontsov <avorontsov@ru.mvista.com>
Date:   Sat Nov 28 01:33:47 2009 +0300

    Use proper types for do_div

The test case works fine once I reverted the commit.

Regards,
Wenji

[PATCH] Fix regression in statistic operations

[Anton Vorontsov]

In commit 98c783852039061db8c1611742660aaded0eab77 ("Use proper types
for do_div") I imprudently changed some variables to an unsigned type
while in some places the code actually relies on a sign.

So, let's be a bit smarter now and use temporary variables.

Reported-by: Wenji Huang <wenji.huang@oracle.com>
Signed-off-by: Anton Vorontsov <avorontsov@ru.mvista.com>
---

On Thu, Dec 03, 2009 at 03:44:42PM +0800, Wenji Huang wrote:
> Hi,
> 
> There's one regression on statistics operation. For example,
> 
> Running ./systemtap.maps/ix.exp ...
> executing: stap ./systemtap.maps/ix.stp
> FAIL: ./systemtap.maps/ix.stp
> line 1: expected "foo[0]: count:3  sum:98  avg:32  min:-2  max:100"
> Got "foo[0]: count:3  sum:98  avg:32  min:0  max:-2"
> 
> It seems this is introduced by the commit.
> 
> commit 98c783852039061db8c1611742660aaded0eab77
> Author: Anton Vorontsov <avorontsov@ru.mvista.com>
> Date:   Sat Nov 28 01:33:47 2009 +0300
> 
>     Use proper types for do_div
> 
> The test case works fine once I reverted the commit.

Oops. :-/ This patch should fix the issue, thanks for noticing and
sorry for the inconvenience.

This time I checked (on x86_64) that no new regressions introduced.

Before all my patches:

                === systemtap Summary ===

# of expected passes            439
# of unexpected failures        275
# of expected failures          214
# of known failures             3
# of untested testcases         202
# of unsupported tests          4

After all my patches:

                === systemtap Summary ===

# of expected passes            439
# of unexpected failures        275
# of expected failures          214
# of known failures             3
# of untested testcases         202
# of unsupported tests          4

And that was before this particular fix:

                === systemtap Summary ===

# of expected passes            428
# of unexpected failures        286
# of expected failures          214
# of known failures             3
# of untested testcases         202
# of unsupported tests          4

 runtime/stat-common.c |   18 ++++++++++++------
 1 files changed, 12 insertions(+), 6 deletions(-)

diff --git a/runtime/stat-common.c b/runtime/stat-common.c
index f970304..fabe840 100644
--- a/runtime/stat-common.c
+++ b/runtime/stat-common.c
@@ -34,9 +34,10 @@ static int _stp_stat_calc_buckets(int stop, int start, int interval)
 	return buckets;
 }
 
-static int needed_space(uint64_t v)
+static int needed_space(int64_t v)
 {
 	int space = 0;
+	uint64_t tmp;
 
 	if (v == 0)
 		return 1;
@@ -45,9 +46,10 @@ static int needed_space(uint64_t v)
 		space++;
 		v = -v;
 	}
-	while (v) {
+	tmp = v;
+	while (tmp) {
 		/* v /= 10; */
-		do_div (v, 10);
+		do_div(tmp, 10);
 		space++;
 	}
 	return space;
@@ -134,7 +136,8 @@ static void _stp_stat_print_histogram_buf(char *buf, size_t size, Hist st, stat
 {
 	int scale, i, j, val_space, cnt_space;
 	int low_bucket = -1, high_bucket = 0, over = 0, under = 0;
-	uint64_t val, v, valmax = 0;
+	int64_t val, valmax = 0;
+	uint64_t v;
 	int eliding = 0;
 	char *cur_buf = buf, *fake = buf;
 	char **bufptr = (buf == NULL ? &fake : &cur_buf);
@@ -282,7 +285,7 @@ static void _stp_stat_print_histogram(Hist st, stat *sd)
 	_stp_print_flush();
 }
 
-static void __stp_stat_add (Hist st, stat *sd, uint64_t val)
+static void __stp_stat_add(Hist st, stat *sd, int64_t val)
 {
 	int n;
 	if (sd->count == 0) {
@@ -310,7 +313,10 @@ static void __stp_stat_add (Hist st, stat *sd, uint64_t val)
 		if (val < 0)
 			val = 0;
 		else {
-			do_div (val, st->interval);
+			uint64_t tmp = val;
+
+			do_div(tmp, st->interval);
+			val = tmp;
 			val++;
 		}
 
-- 
1.6.3.3

Re: [PATCH] Fix regression in statistic operations

[Wenji Huang]

Anton Vorontsov wrote:
[...]
> 
> Oops. :-/ This patch should fix the issue, thanks for noticing and
> sorry for the inconvenience.
> 
> This time I checked (on x86_64) that no new regressions introduced.
> 
Confirmed. The patch fixed the regression. Thanks.

Regards,
Wenji

Re: Build Pass from file name hyphen->underscore

[David Smith]

On 12/02/2009 07:11 AM, naresh kamboju wrote:
> Hi,
> 
> I have been investigating SystemTap test suite, its sub modules and
> expected pass and expected failures.
> 
> As per my investigation I have noticed there are test cases name with
> – hyphen are not able to build. When I have changed it to underscore _
> it is able to build.
> As per stap, If you add ‘-‘ (hyphen) in the file name, then stap is
> able to consider as invalid module name.
> 
> BUILD_FAIL				BUILD_PASS
> aux_syscalls-embedded.stp -> aux_syscalls_embedded.stp
> 
> 
> There were almost 36 test cases in this category.
> Are considering these expected failures?

This doesn't make much sense to me.  Yes, '-' is an illegal character in
a module name, but that doesn't matter here.  Besides the fact that
aux_syscalls-embedded.stp just gets compiled and not actually inserted,
that isn't the module name.  Systemtap generated a random string of the
form 'stap_XXXXX...' for the module name (unless someone uses the '-m
NAME' option).

So, unless someone is doing 'stap -m aux_syscalls-embedded
aux_syscalls-embedded.stp', the above problem shouldn't exist.

Can you give us more details here about your system and the failure
messages you are getting?

-- 
David Smith
dsmith@redhat.com
Red Hat
http://www.redhat.com
256.217.0141 (direct)
256.837.0057 (fax)

Re: Build Pass from file name hyphen->underscore

[naresh kamboju]

Hi,

On Thu, Dec 3, 2009 at 1:07 AM, David Smith <dsmith@redhat.com> wrote:
> On 12/02/2009 07:11 AM, naresh kamboju wrote:
>> Hi,
>>
>> I have been investigating SystemTap test suite, its sub modules and
>> expected pass and expected failures.
>>
>> As per my investigation I have noticed there are test cases name with
>> – hyphen are not able to build. When I have changed it to underscore _
>> it is able to build.
>> As per stap, If you add ‘-‘ (hyphen) in the file name, then stap is
>> able to consider as invalid module name.
>>
>> BUILD_FAIL                            BUILD_PASS
>> aux_syscalls-embedded.stp -> aux_syscalls_embedded.stp
>>
>>
>> There were almost 36 test cases in this category.
>> Are considering these expected failures?
>
> This doesn't make much sense to me.  Yes, '-' is an illegal character in
> a module name, but that doesn't matter here.  Besides the fact that
> aux_syscalls-embedded.stp just gets compiled and not actually inserted,
> that isn't the module name.  Systemtap generated a random string of the
> form 'stap_XXXXX...' for the module name (unless someone uses the '-m
> NAME' option).
I agree with this.
>
> So, unless someone is doing 'stap -m aux_syscalls-embedded
> aux_syscalls-embedded.stp', the above problem shouldn't exist.
>
In my case I have been investigating each module behavior and functionality.
Because of that have to use –m options for each build.
If it is like stap_XXX, I could not find from where it is generated.

So, shall I take up this issue and fix?

I hope this would be helpful for the most of the developers.
It would be 36 test cases.

Best regards
Naresh Kamboju
> Can you give us more details here about your system and the failure
> messages you are getting?
>
> --
> David Smith
> dsmith@redhat.com
> Red Hat
> http://www.redhat.com
> 256.217.0141 (direct)
> 256.837.0057 (fax)
>

Re: Build Pass from file name hyphen->underscore

[David Smith]

On 12/03/2009 09:18 AM, naresh kamboju wrote:
> Hi,
> 
> On Thu, Dec 3, 2009 at 1:07 AM, David Smith <dsmith@redhat.com> wrote:
>> On 12/02/2009 07:11 AM, naresh kamboju wrote:
>>> Hi,
>>>
>>> I have been investigating SystemTap test suite, its sub modules and
>>> expected pass and expected failures.
>>>
>>> As per my investigation I have noticed there are test cases name with
>>> – hyphen are not able to build. When I have changed it to underscore _
>>> it is able to build.
>>> As per stap, If you add ‘-‘ (hyphen) in the file name, then stap is
>>> able to consider as invalid module name.
>>>
>>> BUILD_FAIL                            BUILD_PASS
>>> aux_syscalls-embedded.stp -> aux_syscalls_embedded.stp
>>>
>>>
>>> There were almost 36 test cases in this category.
>>> Are considering these expected failures?
>>
>> This doesn't make much sense to me.  Yes, '-' is an illegal character in
>> a module name, but that doesn't matter here.  Besides the fact that
>> aux_syscalls-embedded.stp just gets compiled and not actually inserted,
>> that isn't the module name.  Systemtap generated a random string of the
>> form 'stap_XXXXX...' for the module name (unless someone uses the '-m
>> NAME' option).
> I agree with this.
>>
>> So, unless someone is doing 'stap -m aux_syscalls-embedded
>> aux_syscalls-embedded.stp', the above problem shouldn't exist.
>>
> In my case I have been investigating each module behavior and functionality.
> Because of that have to use –m options for each build.
> If it is like stap_XXX, I could not find from where it is generated.

I'm confused here. Are you running all the modules at once?  If not, how
are you getting confused about which script you are running?

> So, shall I take up this issue and fix?
> 
> I hope this would be helpful for the most of the developers.
> It would be 36 test cases.

I'm still not sure I see a problem here since the testsuite works correctly.

-- 
David Smith
dsmith@redhat.com
Red Hat
http://www.redhat.com
256.217.0141 (direct)
256.837.0057 (fax)

Re: Build Pass from file name hyphen->underscore

[Masami Hiramatsu]

Hi,

I'm not sure why don't we just add below code ;-)

for (s = modname; *s != '\0'; s++)
	if (*s == '-')
		*s = '_';

Thank you,

David Smith wrote:
> On 12/03/2009 09:18 AM, naresh kamboju wrote:
>> Hi,
>>
>> On Thu, Dec 3, 2009 at 1:07 AM, David Smith<dsmith@redhat.com>  wrote:
>>> On 12/02/2009 07:11 AM, naresh kamboju wrote:
>>>> Hi,
>>>>
>>>> I have been investigating SystemTap test suite, its sub modules and
>>>> expected pass and expected failures.
>>>>
>>>> As per my investigation I have noticed there are test cases name with
>>>> – hyphen are not able to build. When I have changed it to underscore _
>>>> it is able to build.
>>>> As per stap, If you add ‘-‘ (hyphen) in the file name, then stap is
>>>> able to consider as invalid module name.
>>>>
>>>> BUILD_FAIL                            BUILD_PASS
>>>> aux_syscalls-embedded.stp ->  aux_syscalls_embedded.stp
>>>>
>>>>
>>>> There were almost 36 test cases in this category.
>>>> Are considering these expected failures?
>>>
>>> This doesn't make much sense to me.  Yes, '-' is an illegal character in
>>> a module name, but that doesn't matter here.  Besides the fact that
>>> aux_syscalls-embedded.stp just gets compiled and not actually inserted,
>>> that isn't the module name.  Systemtap generated a random string of the
>>> form 'stap_XXXXX...' for the module name (unless someone uses the '-m
>>> NAME' option).
>> I agree with this.
>>>
>>> So, unless someone is doing 'stap -m aux_syscalls-embedded
>>> aux_syscalls-embedded.stp', the above problem shouldn't exist.
>>>
>> In my case I have been investigating each module behavior and functionality.
>> Because of that have to use –m options for each build.
>> If it is like stap_XXX, I could not find from where it is generated.
>
> I'm confused here. Are you running all the modules at once?  If not, how
> are you getting confused about which script you are running?
>
>> So, shall I take up this issue and fix?
>>
>> I hope this would be helpful for the most of the developers.
>> It would be 36 test cases.
>
> I'm still not sure I see a problem here since the testsuite works correctly.
>

-- 
Masami Hiramatsu

Software Engineer
Hitachi Computer Products (America), Inc.
Software Solutions Division

e-mail: mhiramat@redhat.com

Re: Build Pass from file name hyphen->underscore

[David Smith]

On 12/03/2009 10:21 AM, Masami Hiramatsu wrote:
> Hi,
> 
> I'm not sure why don't we just add below code ;-)
> 
> for (s = modname; *s != '\0'; s++)
>     if (*s == '-')
>         *s = '_';

We could, at the cost of surprising the user.  With '-m', the user has
said, "hey, don't use the standard stap_XXXX module name, I need the
exact module name FOO-BAR".  Then, behind the user's back, we change
'FOO-BAR' to 'FOO_BAR'.  The user then goes looking for 'FOO-BAR', and
he won't find it.

-- 
David Smith
dsmith@redhat.com
Red Hat
http://www.redhat.com
256.217.0141 (direct)
256.837.0057 (fax)

Re: Build Pass from file name hyphen->underscore

[Frank Ch. Eigler]

naresh kamboju <naresh.kernel@gmail.com> writes:

> [...]
> In my case I have been investigating each module behavior and functionality.
> Because of that have to use -m options for each build.
> If it is like stap_XXX, I could not find from where it is generated.
> [...]

We could canonicalize the -m name as mhiramat suggested.  We could
also add some script metadata into the generated .c/.ko files, to
allow one to look up their ancestry.

- FChE

Re: Build Pass from file name hyphen->underscore

[Frank Ch. Eigler]

Hi -

On Mon, Mar 29, 2010 at 11:27:59PM +0530, naresh kamboju wrote:
> [...]
> Reason to prepare this patch.
> 
> I am executing Systemtap on ARM arch, here i would not use 'make
> check'   so i have to build each test case module with -m options with
> cross compilation and it wont allow module name with hypen, because of
> this reason i have to change the test case name from hypend to
> underscore.
> BUILD_FAIL                                    BUILD_PASS
> aux_syscalls-embedded.stp    ->     aux_syscalls_embedded.stp
> [...]

Please let's think of another way to do this.  It seems as though the
chain of issues leading to this underscore problem could have been
snipped earlier.

For example #1, you could teach stap to tolerate "-m FOO-BAR" names
(and map them internally to FOO_BAR).

For example #2, in your hand-made testing script, instead of using -m
FOO for test case FOO.stp, you could use any old synthetic identifier,
such as a serial number, or a "-" to "_" converted form.

For example #3, we outlined two more general cross-compilation testing
approaches in a previous email.  Neither requires hand-built 'stap -m
FOO'.

- FChE