Hints for cross-compiling staprun?

Hints for cross-compiling staprun?

[Grant Edwards]

I've found plenty of docs for cross-compiling a script to create a
kernel module for a target arch different from the host where "stap"
is being run.  I've got that working with no problem.

I've also found a few vague hints about cross compiling a full-up
systemtap installation (requiring elfutils and a handfull of other
libraries that don't appear to be needed by staprun).

What I can't find is any documentation on cross-compiling staprun.

Most of the docs that talk about cross-debugging just toss off advice
like "install the systemtap-runtime RPM on the target".

How does one cross-compile staprun without dragging in all of the
baggage required for a full-up systemtap install?

-- 
Grant Edwards               grant.b.edwards        Yow! Not SENSUOUS ... only
                                  at               "FROLICSOME" ... and in
                              gmail.com            need of DENTAL WORK ... in
                                                   PAIN!!!

Re: Hints for cross-compiling staprun?

[Frank Ch. Eigler]

Grant Edwards <grant.b.edwards@gmail.com> writes:

> [...]  What I can't find is any documentation on cross-compiling
> staprun.  Most of the docs that talk about cross-debugging just toss
> off advice like "install the systemtap-runtime RPM on the
> target". [...]

I believe some have had success with using the 
  --host=FOO-BAR --disable-translator
configure option, but have not tried it myself yet.

- FChE

Re: Hints for cross-compiling staprun?

[Grant Edwards]

On 2010-08-27, Frank Ch. Eigler <fche@redhat.com> wrote:
> Grant Edwards <grant.b.edwards@gmail.com> writes:
>
>> [...]  What I can't find is any documentation on cross-compiling
>> staprun.  Most of the docs that talk about cross-debugging just toss
>> off advice like "install the systemtap-runtime RPM on the
>> target". [...]
>
> I believe some have had success with using the 
>   --host=FOO-BAR --disable-translator
> configure option, but have not tried it myself yet.

Thanks.  I was about to try something like that.  If I get it to work,
I'll post a recipe.

-- 
Grant Edwards               grant.b.edwards        Yow! I own seven-eighths of
                                  at               all the artists in downtown
                              gmail.com            Burbank!

Re: Hints for cross-compiling staprun?

[Grant Edwards]

On 2010-08-27, Grant Edwards <grant.b.edwards@gmail.com> wrote:

> How does one cross-compile staprun without dragging in all of the
> baggage required for a full-up systemtap install?

I've hit a few glitches, but I'm making progress.  One issue is that
the configure script refuses to check for the existence of files when
cross-compiling.  I'm not sure why that is, but you can short-circuit
that by setting some environment variables:

   ac_cv_file__usr_include_nspr=no
   ac_cv_file__usr_include_nspr4=no
   ac_cv_file__usr_include_nss=no
   ac_cv_file__usr_include_nss3=no
   ac_cv_file__usr_include_avahi_client=no
   ac_cv_file__usr_include_avahi_common=no

The other problem is that the sources use a number of non-standard
APIs without checking to see if they are, in fact, supported by the
build environment.  The ones that I tripped over are:

   index,rindex   obsolete and not present in POSIX.1-2008.
   
   __progname     doesn't seem to be part of any standard, and isn't
                  present in any headers on any Linux system I've got.

   wordexp        actually, I think this one is in POSIX.1-2008 -- it
                  just wasn't enabled in my build of libc.

It would result in a more graceful failure, if the configure script
checked for those.
                  
I've rebuilt my libc with support for those API's, and now the
systemtap runtime stuff compiles, but fails to install because the
systemtap's version of install-sh doesn't support the '-D' option, but
systemtap's Makefile uses that option.  ("make" output attached
below.)

If the systemtap Makefile.in is going to use "install-sh -D",
shouldn't it come with an install-sh that supports "-D"?

[...]
>>> systemtap 1.3 Installing to target
/usr/bin/make -j2 DESTDIR=/home/nextgen/buildroot/buildroot/output/target  install-strip -C /home/nextgen/buildroot/buildroot/output/build/systemtap-1.3/
make[1]: Entering directory `/home/nextgen/buildroot/buildroot/output/build/systemtap-1.3'
/usr/bin/make  INSTALL_PROGRAM="/bin/sh /home/nextgen/buildroot/buildroot/output/build/systemtap-1.3/install-sh -c -s" \
	  install_sh_PROGRAM="/bin/sh /home/nextgen/buildroot/buildroot/output/build/systemtap-1.3/install-sh -c -s" INSTALL_STRIP_FLAG=-s \
	  `test -z '/home/nextgen/toolchain/bin/arm-linux-strip' || \
	    echo "INSTALL_PROGRAM_ENV=STRIPPROG='/home/nextgen/toolchain/bin/arm-linux-strip'"` install
make[2]: Entering directory `/home/nextgen/buildroot/buildroot/output/build/systemtap-1.3'
/bin/sh ./git_version.sh -k -s . -o git_version.h
git_version.sh: Not a git repo, keeping existing git_version.h
/usr/bin/make  install-recursive
[...]
make[5]: Entering directory `/home/nextgen/buildroot/buildroot/output/build/systemtap-1.3'
[...]
/usr/bin/make  install-exec-hook
make[6]: Entering directory `/home/nextgen/buildroot/buildroot/output/build/systemtap-1.3'
if [ `id -u` -eq 0 ]; then chmod 04111 "/home/nextgen/buildroot/buildroot/output/target/usr/bin/staprun"; fi
make[6]: Leaving directory `/home/nextgen/buildroot/buildroot/output/build/systemtap-1.3'
(cd ./runtime/unwind; find . \( -name '*.[ch]' \) -print \
		| while read f; do /usr/bin/install -c -m 644 -D $f /home/nextgen/buildroot/buildroot/output/target/usr/share/systemtap/runtime/unwind/$f; done)
(cd ./runtime/transport; for f in *.[ch]; \
		do /usr/bin/install -c -m 644 -D $f /home/nextgen/buildroot/buildroot/output/target/usr/share/systemtap/runtime/transport/$f; done)
(cd ./runtime/uprobes; for f in Makefile *.[ch]; \
		do /usr/bin/install -c -m 644 -D $f /home/nextgen/buildroot/buildroot/output/target/usr/share/systemtap/runtime/uprobes/$f; done)
(cd ./runtime/uprobes2; for f in *.[ch]; \
		do /usr/bin/install -c -m 644 -D $f /home/nextgen/buildroot/buildroot/output/target/usr/share/systemtap/runtime/uprobes2/$f; done)
(cd ./tapset; find . \( -name '*.stp' -o -name README \) -print \
		| while read f; do /usr/bin/install -c -m 644 -D $f /home/nextgen/buildroot/buildroot/output/target/usr/share/systemtap/tapset/$f; done)
(cd ./testsuite/systemtap.examples; find . -type f -print \
		| egrep -v 'check.exp|ChangeLog|examples-index-gen.pl' \
		| while read f; do if test -x $f; then \
			i_cmd="/bin/sh /home/nextgen/buildroot/buildroot/output/build/systemtap-1.3/install-sh -c -s"; else \
			i_cmd="/usr/bin/install -c -m 644"; fi; \
			$i_cmd -D $f /home/nextgen/buildroot/buildroot/output/target/usr/share/doc/systemtap/examples/$f; done)
/home/nextgen/buildroot/buildroot/output/build/systemtap-1.3/install-sh: invalid option: -D
/home/nextgen/buildroot/buildroot/output/build/systemtap-1.3/install-sh: invalid option: -D
/home/nextgen/buildroot/buildroot/output/build/systemtap-1.3/install-sh: invalid option: -D
/home/nextgen/buildroot/buildroot/output/build/systemtap-1.3/install-sh: invalid option: -D
/home/nextgen/buildroot/buildroot/output/build/systemtap-1.3/install-sh: invalid option: -D
/home/nextgen/buildroot/buildroot/output/build/systemtap-1.3/install-sh: invalid option: -D
[ 70 identical lines deleted ]
/home/nextgen/buildroot/buildroot/output/build/systemtap-1.3/install-sh: invalid option: -D
/home/nextgen/buildroot/buildroot/output/build/systemtap-1.3/install-sh: invalid option: -D
/home/nextgen/buildroot/buildroot/output/build/systemtap-1.3/install-sh: invalid option: -D
/home/nextgen/buildroot/buildroot/output/build/systemtap-1.3/install-sh: invalid option: -D
/home/nextgen/buildroot/buildroot/output/build/systemtap-1.3/install-sh: invalid option: -D
make[5]: *** [install-data-local] Error 1
make[5]: Leaving directory `/home/nextgen/buildroot/buildroot/output/build/systemtap-1.3'
make[4]: *** [install-am] Error 2
make[4]: Leaving directory `/home/nextgen/buildroot/buildroot/output/build/systemtap-1.3'
make[3]: *** [install-recursive] Error 1
make[3]: Leaving directory `/home/nextgen/buildroot/buildroot/output/build/systemtap-1.3'
make[2]: *** [install] Error 2
make[2]: Leaving directory `/home/nextgen/buildroot/buildroot/output/build/systemtap-1.3'
make[1]: *** [install-strip] Error 2
make[1]: Leaving directory `/home/nextgen/buildroot/buildroot/output/build/systemtap-1.3'
make: *** [/home/nextgen/buildroot/buildroot/output/build/systemtap-1.3/.stamp_target_installed] Error 2
make: Leaving directory `/home/nextgen/buildroot/buildroot'

   
-- 
Grant Edwards               grant.b.edwards        Yow! I request a weekend in
                                  at               Havana with Phil Silvers!
                              gmail.com            

Re: Hints for cross-compiling staprun?

[David Smith]

On 08/30/2010 03:56 PM, Grant Edwards wrote:

... stuff deleted ...

> The other problem is that the sources use a number of non-standard
> APIs without checking to see if they are, in fact, supported by the
> build environment.  The ones that I tripped over are:
> 
>    index,rindex   obsolete and not present in POSIX.1-2008.

These should be updated - I'll fix them.

>    __progname     doesn't seem to be part of any standard, and isn't
>                   present in any headers on any Linux system I've got.

Hmm, I don't see this identifier anywhere in the current source.  Where
did you see it?

>    wordexp        actually, I think this one is in POSIX.1-2008 -- it
>                   just wasn't enabled in my build of libc.

According to the wordexp man page, this conforms to POSIX.1-2001.

I believe if you had specified '--disable-server' (which you'd probably
want to do anyway), we wouldn't have used 'wordexp'.

-- 
David Smith
dsmith@redhat.com
Red Hat
http://www.redhat.com
256.217.0141 (direct)
256.837.0057 (fax)

Re: Hints for cross-compiling staprun?

[Frank Ch. Eigler]

Hi -

grant.b.edwards wrote:

> [...]
> I've hit a few glitches, but I'm making progress.  One issue is that
> the configure script refuses to check for the existence of files when
> cross-compiling.  [...]

Perhaps we're using the wrong configure.ac directive to test for them.

> [...]
> The other problem is that the sources use a number of non-standard
> APIs without checking to see if they are, in fact, supported by the
> build environment.  The ones that I tripped over are:
>
>    index,rindex   obsolete and not present in POSIX.1-2008.

OK, will fix.

>    __progname     doesn't seem to be part of any standard, and isn't
>                   present in any headers on any Linux system I've got.

(Where do we refer to that?)

>    wordexp        actually, I think this one is in POSIX.1-2008 -- it
>                   just wasn't enabled in my build of libc.
>
> It would result in a more graceful failure, if the configure script
> checked for those.

Well, we should be able to presume plain POSIX, but a few more tests
wouldn't hurt, I guess.

> [...]
> If the systemtap Makefile.in is going to use "install-sh -D",
> shouldn't it come with an install-sh that supports "-D"?

I guess on native linux boxes it was using /usr/bin/install, which
does understand -D.  We should be able to adapt that to some
combination of -d / -t or mkdir -p.  Will fix.


- FChE

Re: Hints for cross-compiling staprun?

[Grant Edwards]

On 2010-08-30, Frank Ch. Eigler <fche@redhat.com> wrote:
> grant.b.edwards wrote:
>
>> [...]
>> I've hit a few glitches, but I'm making progress.  One issue is that
>> the configure script refuses to check for the existence of files when
>> cross-compiling.  [...]
>
> Perhaps we're using the wrong configure.ac directive to test for them.

I'm not sure, but I don't think so.

After googling the error message "cannot check for file existence when
cross compiling" it seems to be a common issue among people who
cross-compile things.  I didn't see any suggested solution other than
faking config cache entries by setting environment variables
indicating whether the file was to be considered present or not.

>> [...]
>> The other problem is that the sources use a number of non-standard
>> APIs without checking to see if they are, in fact, supported by the
>> build environment.  The ones that I tripped over are:
>>
>>    index,rindex   obsolete and not present in POSIX.1-2008.
>
> OK, will fix.

Thanks!

>>    __progname     doesn't seem to be part of any standard, and isn't
>>                   present in any headers on any Linux system I've got.
>
> (Where do we refer to that?)

Ah, my mistake.  That's an undefined symbol in libssp, and apparently
systemtap is the only thing on my target system that uses the
offending portion of that library.

>> It would result in a more graceful failure if the configure script
>> checked for those.
>
> Well, we should be able to presume plain POSIX, but a few more tests
> wouldn't hurt, I guess.

You're right.  You should be able to presume plain POSIX.  I wouldn't
bother adding tests for stuff that's required by POSIX.

>> [...]
>> If the systemtap Makefile.in is going to use "install-sh -D",
>> shouldn't it come with an install-sh that supports "-D"?
>
> I guess on native linux boxes it was using /usr/bin/install, which
> does understand -D.  We should be able to adapt that to some
> combination of -d / -t or mkdir -p.  Will fix.

It turns out that the only time that -D is used is when installing
examples.  Since there's no point in installing examples on a machine
that has only the runtime, I nuked that section of Makefile.in.  Now
it builds and installs without problems (haven't tried actually using
it yet).

Adding a --disable-examples option is preferable to patching
Makefile.in, but I haven't had time to figure out how to do that yet.

-- 
Grant Edwards               grant.b.edwards        Yow! I didn't order any
                                  at               WOO-WOO ... Maybe a YUBBA
                              gmail.com            ... But no WOO-WOO!

Re: Hints for cross-compiling staprun?

[Roland McGrath]

I always found the --disable-translator et al approach fairly insane.
I've added a standalone configure script and makefile for the runtime
programs.

You can use this by taking your git checkout (or unpacking a future dist
tar file), and then running in some build directory:

	.../runtime/staprun/configure --host=cross-target --prefix=/usr
	make
	make install DESTDIR=/cross/place

i.e., it's a simple standard build setup and it should support cross
compilation as well as such things usually do.  This configure.ac and
Makefile.am are quite small and simple, so if you have cross-compile issues
you can just get generic help on cross compilation for your environment
and it should apply to this case as well.

I made this configure rely on pkg-config for nss support, rather than the
massive manual hooey the main systemtap configure uses.  So that might
complicate things.  But you could just punt the certificate-checking
support and not worry about it.  Moreover, whatever complications you have
can probably be resolved just by asking your cross-compilation community
for help with a package that uses PKG_CHECK_MODULES.

If this works well for everyone, then we should probably remove all the
staprun-related cruft from the main configure.ac and Makefile.am, and
just use AC_CONFIG_SUBDIRS(runtime/staprun).  But for now, the new stuff
should not have any effect on using the existing main configure et al.


Thanks,
Roland

Re: Hints for cross-compiling staprun?

[Grant Edwards]

On 2010-08-30, Roland McGrath <roland@redhat.com> wrote:

> I always found the --disable-translator et al approach fairly insane.
> I've added a standalone configure script and makefile for the runtime
> programs.
>
> You can use this by taking your git checkout (or unpacking a future dist
> tar file), and then running in some build directory:
>
> 	.../runtime/staprun/configure --host=cross-target --prefix=/usr
> 	make
> 	make install DESTDIR=/cross/place
>
> i.e., it's a simple standard build setup and it should support cross
> compilation as well as such things usually do.  This configure.ac and
> Makefile.am are quite small and simple, so if you have cross-compile issues
> you can just get generic help on cross compilation for your environment
> and it should apply to this case as well.

Brilliant!

After I get my current build of 1.3 working, I'll give the new method
a try.

-- 
Grant Edwards               grant.b.edwards        Yow! HUGH BEAUMONT died
                                  at               in 1982!!
                              gmail.com