Re: What about MIPS support?

[Victor Kamensky <kamensky@cisco.com>]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 11212 bytes --]

Hi All,

It took us much much longer than originally expected. But it is better 
late than never. Here is our patches tarbal attached. Inside of tarbal 
(README file) and copied below I put short description of each patch. Some 
of patches are Cisco specific and may not be any interest for you. Some 
like MIPS support, cross compilation improvements, misc general fixes (i.e 
systemtap issue 6977), I think, could be quite useful to community. Since 
it is quilt patch series and patches may have patch apply order issues, we 
are publishing all of them, exactly as we use/apply them, regardless 
whether we consider them Cisco specific or not.

As it was discussed, for now, I am just dropping our patches as is. Latter 
as time permits, I can work on cleaning them up, moving to latest 
systemtap git and try to prepare them for commit into systemtap tree. It 
would be great if folks could take a look at patches and/or patches 
description and indicate priorities/interests for specific patches on 
which I could start working first.

Please advise if you would like me to post the same on different subject 
line on this mailing list (since it covers more than just mips support). I 
thought I just post it on original thread to get it proper closure.

==== systemtap-1.6-cisco-patches/README start =============================

This is patches series that Cisco did on top of systemtap-1.6. Some of them are 
Cisco specific and most likely are not interesting for community. We decided to
provide them here anyway in order to avoid breakage in applying patches, because
of patch application order issues.

Other like MIPS support, cross compilation improvements, misc general fixes may
be some interest for SystemTap community.

Patch description follows order of patches in series file:

systemtap-configure_cross_compile.patch - change in configure logic to make it
     more cross compilation environment friendly (in cross compile environment
     one cannot test presence of /usr/include files; those are from host). We
     don't use nss3 and avahi in our target so for now just introduce option
     that could disable them. Most likely issue is fixed in latest tree.

systemtap-unwind-table-size.patch - IOS code is too big, we are hitting limit
     on unwind symbol table size. Increase the limit for now. In future it would
     be good code to adjust those values dynamically.

systemtap-fde_byte_order_fix.patch - Fix reading fde entry in case of cross
     compilation where target and host have different byte order (i.e x86 host,
     be mips is target).

systemtap-pointer-byte_size.patch - Fix related to ICC (intel compiler). There
     is a difference how DW_AT_byte_size is stored between gcc and icc. Current
     stap code tailored to gcc way. Need to go back and dig out more details
     here.

systemtap-userland_caller.patch - Trying to introduce ucaller_addr and ucaller
     systemtap function. Most likely this code does not work. But currently
     patch is in our series file so keeping it for now.

systemtap-biendian.patch - Support ICC biendian feature. It is described at
     http://software.intel.com/en-us/articles/dwarf-extensions-for-bi-endian-support/. Most
     like this code has no interest to anyone except Cisco.

systemtap-old_compiler-task_finder.patch - Older gcc compiler (like gcc 3.4.x)
     produce compilation warning here, incorrectly assuming that dentry may be
     used uninitialized

systemtap-cross_compile_helper.patch - Cross compilation assist. Actually
     include a lot of different small things. Here is the list of them (may not
     be complete):
     x) target system build may use difference 'gcc' and different 'make'.
     Different values could be specified through SYSTEMTAP_TARGET_CC and
     SYSTEMTAP_MAKE environment variable
     x) -y option introduced. Once specified it will indicate that systemtap
     compiler operates in cross compilation mode and value of option serves
     sysroot path value. Stap compiler supports several such options (details
     below)
     x) if compiler in cross compilation mode it does not use PATH and
     LD_LIBRARY_PATH to search for executables and libraries. Instead it uses
     SYSTEMTAP_CROSS_PATH and SYSTEMTAP_CROSS_LD_LIBRARY_PATH environment
     variables. PATH and LD_LIBRARY_PATH are usually different between host and
     target. Note SYSTEMTAP_CROSS_PATH and SYSTEMTAP_CROSS_LD_LIBRARY_PATH could
     be specified as relative of sysroot values, as well as host absolute
     pathes
     x) if stap compiler is in cross compilation mode it does not try to rebuild
     uprobes. It assumes that uprobes.ko is already built and installed in
     proper location in sysroot
     x) find_executable function is replaced with special sysroot class that
     handles different ways how file path needed during stap compilation (some
     times it needs file path in host terms, sometimes it would need it as it
     would be seen on target). Sysroot supports multiple sysroot. For example
     like in openwrt case one may have target root file system and overlay file
     system. Stap compiler in cross compilation mode can search both. Normally
     look up happens as iteration over sysroots and for each sysroot we iterate
     over list of directories specified in SYSTEMTAP_CROSS_PATH and
     SYSTEMTAP_CROSS_LD_LIBRARY_PATH. Quite normal case when one sysroot is
     specified so executables will be searched by adding sysroot value in front
     of directories specified by SYSTEMTAP_CROSS_PATH and
     SYSTEMTAP_CROSS_LD_LIBRARY_PATH.

     It is understood that above description may not be really sufficient to get
     full understanding of this patch. If community is interested I can give
     details explanation and see whether we can rework it. Above I just tried to
     give a taste of what sort of issue we are dealing with when we work in
     cross compiled environment

systemtap-data-in_another_cu.patch - Fix for issue
     6977 http://sourceware.org/bugzilla/show_bug.cgi?id=6977. We found it is
     very annoying and limiting for script writers, so we fixed it. It deals
     with  situation of accessing global variables that are defined in *other*
     compilation units.

systemtap-composed_rootfs_finder.patch - in embedded universe final root file
     system may be composed as link farm to set of files system. Example is
     Openwrt root file system and overlay. Systemtap runtime searches
     executables by canonical name. It won't work for link farm case. For
     link farm case ELF file name compiled into script is used as primary file
     path to check for file in file system or as key in list of file systems
     that participate in composition of final link farm. Outside of Cisco
     patch may have interest to distros like OpenWrt.

systemtap-line_range_issue_fix.patch - memory leak and crash in
     iterate_over_srcfile_lines in case of line ranges.

systemtap-icc_line_number.patch - deal with the way how ICC generates line
     number information.

systemtap-data-in_library.patch - fix issues of accessing global data in
     libraries (opposed to executables) Not sure whether it is applicable to
     latest tree.

systemtap-mips-uprobes.patch - MIPS uprobes kernel module code.

systemtap-mips.patch - MIPS systemtap runtime and and tapset code.

systemtap-ppc32.patch - runtime fixes for powerpc 32bit case.

systemtap-smaller_buffer_size.patch - stap debugfs 1Mb buffer size is too big
     for systems like  WRT54GL, where whole box has 16Mb or 32Mb of memory.
     Change 'b' option handling, so smaller buffers could be specified (i.e in
     Kbytes). Patch tries to be backward compatible with previous option usage
     - if no size letter is specified, it is treated like Megabytes

systemtap-incorrect_kernel_buildid_hack.patch - there are idiosyncrasies in
     Cisco build system when kernel rebuilt twice and buildid in target image
     does not correspond to one we left to deal with on the host. This is hack
     to deal with it. Should be ignored by everyone else.

systemtap-ios_keyword.patch - Cisco specific patch. Should be ignored by
     everybody else. Introduces ios keyword (along with 'process' and 'kernel').
     It deals with the fact that IOS process  canonical name changes depending
     on IOS image feature key. 'ios' keyword allows us to have scripts reusable
     across different target images. Real ios process name is passed with -i
     option by higher level wrapper script.

systemtap-no_userland_prologue_search.patch - Add option '-U' that disables
     function prologue  searches in user-land processes. Current -P affects only
     kernel probes.

systemtap-kernel_source_tree.patch - Option -T allows to specify location of
     kernel sources. Normally needed only in cross compilation environment.

systemtap-uprobes_in_kernel_as_module.patch - Fix for case when in cross
     compilation environment we  may have uprobes built as module in kernel
     build (opposite to built by systemtap). In this case, in order for
     kernel_built_uprobes function to see unregister_uprobe symbol it has to be
     added into kernel_exports even-though it comes from module (not uprobes)

systemtap-remote_hack.patch - allows remote_uris mechanism to work in cross
     compilation environment, where target kernel release won't be the same as
     host kernel release

systemtap-cross_testsuite.patch - Attempt to change testsuite to operate in
     cross compilation environment where access to target happens either through
     dejagnu remote target facilities or through stap --remote mechanism.
     Massive patch, across bunch of test cases, not sure whether community has
     any interest in it. It was used by us to validate systemtap operation on our
     targets. In order to run, it requires stap board specific config (not
     provided here), which would specify things like stap_board_args, and others.

systemtap-enable_vma_tracker.patch - cover corner cases where
     _stp_umodule_relocate is used but vma_tracker is not enabled, while it
     should be, becuase it uses stap_find_vma_map_info_user function.

Thanks,
Victor

On Thu, 23 Aug 2012, Josh Stone wrote:

> On 08/22/2012 01:04 PM, Victor Kamensky wrote:
>> Hi Per, Josh,
>>
>> We (Cisco) have pretty much full systemtap MIPS port.
> [...]
>> We always meant to publish all of it, but never get our hands on it. I
>> hope this thread would be final push :). We will do it. Unfortunately, I
>> cannot publish our patch series right away. I need to get permission from
>> our company legal. It should not be a problem, but it may take two or
>> three weeks.
>
> This is good news, and better that you're spurred now to share it. :)
>
>> Our latest series of patches is based on systemtap-1.6. I'll see whether I
>> can find time and move them to latest git ... In worst case scenario we
>> just publish what we have.
>
> IMO, it would be good to go ahead and post the 1.6 patches directly
> after your legal clears it.  Then if you're willing and able to rebase
> that onto the latest git, that's great, but we don't need to hold up
> everything for that possibility.
>
> Thanks,
> Josh
>

[-- Attachment #2: Type: APPLICATION/OCTET-STREAM, Size: 60382 bytes --]