Ping: [PATCH V4] gprofng: a new GNU profiler

[Vladimir Mezentsev <vladimir.mezentsev@oracle.com>]

  Ping.


-------- Forwarded Message --------
Subject: 	Ping: [PATCH V4] gprofng: a new GNU profiler
Date: 	Mon, 15 Nov 2021 10:13:51 -0800
From: 	Vladimir Mezentsev <vladimir.mezentsev@oracle.com>
To: 	binutils@sourceware.org




Any suggestions, recomendations  to make the gprofng sources public ?
We need a code review.
Thank you,
-Vladimir



-------- Forwarded Message --------
Subject: 	[PATCH V4] gprofng: a new GNU profiler
Date: 	Wed, 3 Nov 2021 11:05:16 -0700
From: 	Vladimir Mezentsev <vladimir.mezentsev@oracle.com>
To: 	binutils@sourceware.org




Hi people!

Here is a slightly updated patch for the gprofng utility. We have fixed
some issues we have noticed while testing and added first support for
hardware event counter profiling on Arm.

We think gprofng will be a good addition to binutils and hope that
we can move forward. In particular we'd like to ask for the review
process to start.

This is a high level description of the various components in gprofng
with the purpose of helping reviewers:

gprofng tools:
     gprofng         - the driver program
     gp-collect-app  - create experiment with program performance data
     gp-display-text - print an ASCII report from one or more 
performance experiments
     gp-display-html - generate an HTML based directory structure from 
an experiment
     gp-archive      - archive the binaries and sources for experiment
     gp-display-src  - print a source or dissasembly listing for the 
load object

     lib/libgprofng.so - read experiments.
                         libgprofng.so is linked with each gprofng tools.

gprofng collector libraries:
     lib/libgp-collector.so - gprofng collector library
     lib/libgp-heap.so      - Heap tracing library
     lib/libgp-iotrace.so   - I/O tracing library
     lib/libgp-sync.so      - Synchronization tracing library


gprofng source tree:
     gprofng/doc/*          - gprofng documentation
     gprofng/libcollector/* - sources of libgp-collector.so, libgp-heap.so,
                              libgp-iotrace.so and libgp-sync.so
     gprofng/src/*          - sources of all gprofng tools and libgprofng.so
     gprofng/common/*       - these files are common components used
                              when building the gprofng libraries


Known problems
==============
- To convert cycles to time, we need the frequency the core is running at.
   On Arm we read the cntrfrq_e10 register, but this sometimes returns an
   incorrect value. For the time being, it may be safest to ignore the
   cycles derived timings on Arm.

- The support for hardware event counters on recent processors is limited.
   You can use the “gprofng collect app -h” command to check for support.
   In case a specific processor is not supported, a warning message will be
   printed.  Otherwise a list with the supported events is printed.


How to get gprofng
==================

Due to the size of the contribution, we thought it would be better to submit
it in the form of a git branch, instead of a regular emailed patch series.

Repository:https://www.github.com/oracle/binutils-gdb
Branch:     oracle/gprofng-v4


Build gprofng
=============

For x86_64/i686, gprofng should be built twice to support profiling for
both 32-bit and 64-bit applications:
   PREFIX=<YOUR_INSTALL_DIR>
   configure --enable-shared --prefix=$PREFIX --libdir=$PREFIX/lib64 && 
make && make install
   configure --enable-shared --prefix=$PREFIX CC='gcc -m32' CXX='g++ -m32' \
          --target=i686-pc-linux-gnu --disable-gprofng-tools && make && 
make install

For ARM:
   configure --enable-shared --prefix=$PREFIX && make install

To support java profiling, configure the build with -with-jdk=PATH_TO_JDK in
case javac is not in your PATH.


11/02/2021: Changes from V3 of the patch
============================================
- First support for hardware event counter profiling on Arm.
- Fixed a problem with dlopen() for libc on Arm.
- Fixed several bugs and implemented some small changes in the output


09/01/2021: Changes from V2 of the patch
========================================

  - Renamed experiment.h to gp-experiment.h to fixe a problem on Windows 
that Hannes
    Domani reported.
  - Fixed a regression in the build on Arm.
  - Improved the gprofng testsuite.


08/25/2021: Changes from V1 of the patch
========================================

Addressed all of Joseph Myers's remarks about gprofng Makefiles and 
configure scripts:
  - Report an error when bison is absent, or a version of bison is too old.
  - Added preload_libdirs in ${prefix}/etc/gprofng.rc to configure the 
location of the
    gprofng libraries (this was hard-coded previously).

We also took note of Jim Wilson's comment about the naming of files in 
src/machinemodels.
In particular, file m5.ermm and the suggestion to make it clear this is 
for a SPARC
processor. For example sparc-m5.ermm.

This is going to be part of a bigger overhaul of the naming convention 
for these files.
The most obvious choice is to include the product name, but we 
definitely appreciate any
suggestions this group may have.



A new GNU profiler
==============================

In this submission we are contributing a new profiler to the GNU binary
utilities, called gprofng (for GNU profiler, next generation).

Why a new profiler?
===================

The GNU profiler, gprof, works well enough in many cases. However, it
hasn't aged well and it is not that very well suited for profiling
modern-world applications. Examples of its limitations are lack of support
for profiling multithreaded programs, and shared objects. Both are
ubiquitous nowadays.

Main characteristics of gprofng
===============================

gprofng supports profiling C, C++ and Java programs. Unlike the old
gprof, it doesn't require to build annotated versions of the programs.
Profiling "production" binaries should work just fine.

Another distinguishing feature of gprofng is the support for various filters
that allow the user to easily drill deeper into an area of interest.

The profiler is commanded through a driver program called `gprofng'.
This driver supports the following sub-commands:

gprofng collect app EXECUTABLE

This runs EXECUTABLE and collects application performance data.

gprofng display text EXPERIMENT

This runs a client command-line interface that provides access to the
collected performance data stored in the experiment directory.

gprofng display html EXPERIMENT

This generates an HTML report from the collected performance data
stored in the experiment directory.

gprofng display src OBJECT-FILE

This displays source (if available) or disassembly interleaved
with the source code.

gprofng archive EXPERIMENT

Archive the associated application binaries, load objects and source
files in an existing experiment directory to make it self contained.

There is also an extensive graphical user interface (written in Java)
that displays and analyzes gprofng collected data in a very sophisticated
way. We plan to release this GUI as a separate project.

While WIP, we would like to share some screenshots of the current
development version. These show the following:

pic1.png - a flame graph:
https://jemarch.net/gprofng-pics/pic1.png

pic2.png - color coded call stacks as a function of time ("the timeline"):
https://jemarch.net/gprofng-pics/pic2.png

pic3.png - zoom in on the timeline and adapt colors to identify details:
https://jemarch.net/gprofng-pics/pic3.png

pic4.png - compare two mulithreaded profiles:
https://jemarch.net/gprofng-pics/pic4.png


Notes on the implementation
===========================

- The gp-display-html tool is written in Perl. All other components are
written in C/C++.

- gprofng sources are mostly contained in a new top-level directory
gprofng/.

- The libcollector.h, libfcollector.h, and collectorAPI.h header files
are header files are distributed in the top-level include/ directory.

- Currently gprofng supports profiling programs in GNU/Linux systems
running on x86_64 and aarch64 hardware. It is possible to add support
for additional architectures.

- The tools come with a set of man pages. They are generated upon
installation and can be found in the installation directory under
share/man/man1.

Platform support
================

The basic profiling features are supported on most processors from
Intel. Regarding AMD we did not yet test on their recent EPYC
processors, but do not expect serious issues for the callstack
sampling. We also support the Arm processors as used in systems
from Ampere.

Hardware event counters are very processor specific. While gprofng
supports using such counters, we currently lack support for recent
Intel and AMD processors, like the EPYC architecture in case of
the latter.
If a particular processor is not supported, but a hardware event
counter experiment is requested, a warning message will be issued
before gprofng terminates.

This code has been developed and tested on Oracle Linux 8 with the
latest GNU toolchain from the sourceware git repos.

Structure of the patch series
=============================

The first patch is preparatory and makes the x86 disassembler in opcodes
to be thread-safe. This is so it can be used by gprofng.

The second patch is the implementation of gprofng proper. This includes
source code for the libraries (libcollector, libgprofng) and the
utilities (gp-collect, etc).

In this patch there are also updates to the corresponding build machinery
(e.g. configure.ac, Makefile.def, plus binutils/MAINTAINERS to cover 
gprofng)

The third patch adds a testsuite in gprofng/testsuite.

The fourth patch adds a Texinfo manual for gprofng. The manual is still
WIP but already provides a tutorial-like introduction to the tools.


Limitations
===========

The gp-display-html tool is present, and can be executed, but it is not
functional yet. Full support for this tool is expected to be delivered
in a future patch.

Requirements
===========

In order to successfully build gprofng, the following versions of
external components are required:

- Bison 3.7.5, or higher
- Texinfo 6.7, or higher
- Java include files (--with-jdk=PATH) if java profiling should be enabled

Maintenance
===========

We are of course volunteering to maintain gprofng once it is
incorporated into the main binutils distribution.
We suggest having feedback and discussion in this mail thread.