Using stapdyn to probe child processes of the target?

Using stapdyn to probe child processes of the target?

[Serhei Makarov]

Hello all,

I've been trying to observe gcc's execution using Stapdyn. Most of the
actual work is done by a child process (cc1) forked from gcc. (Below
I've attached a script, proctree.stp, for listing processes forked by
a target command.) If I'm using kernel systemtap, I can just target
cc1 directly, for example:

stap -ve 'probe
process("/usr/libexec/gcc/x86_64-redhat-linux/6.3.1/cc1").insn {
counter<<<1 } global counter probe end { printf("%d calls\n",
@count(counter)) }' -c "gcc test/widget3.c"

Instrumenting using kernel uprobes introduces quite a large
performance overhead, so I would prefer Dyninst for this. But the same
script doesn't work with --dyninst option, because the target process
selected using -c is not cc1.

As far as I can tell, my only options for observing cc1 with Stapdyn are:

(a) use -c option : invoke cc1 directly. Feasible in this case, may
not be feasible in other software where the parent/child processes
coordinate using more complex IPC. This also doesn't give a profile of
the full GCC execution (unlike, say, "valgrind --trace-children=yes").
(b) use -x option : attach to cc1 by PID after it starts running. Not
feasible in this case since cc1 is a batch program, not a long-running
daemon.

Am I missing any possibilities? Is it currently possible to use
Stapdyn to attach to processes other than a single target?

All the best,
     Serhei Makarov

PS Here's the proctree.stp script:

global pids

probe begin {
  pids[target()] = 0
}

probe process.begin {
  child = pid(); parent = ppid()
  if (parent in pids) {
    printf("fork %s(%d) -> %s(%d)\n", pexecname(), parent, execname(), child)
    pids[child] = parent
  }
}

This outputs things like:

[serhei@beatrice e-tracer]$ stap proctree.stp -c "gcc test/widget3.c"
Missing separate debuginfos, use: debuginfo-install
kernel-core-4.10.10-200.fc25.x86_64
fork gcc(2725) -> gcc(2728)
fork gcc(2725) -> cc1(2728)
fork gcc(2725) -> gcc(2729)
fork gcc(2725) -> as(2729)
fork gcc(2725) -> gcc(2730)
fork gcc(2725) -> collect2(2730)
fork collect2(2730) -> collect2(2731)
fork collect2(2730) -> ld(2731)

Or, here's a Firefox browsing session:

[serhei@beatrice e-tracer]$ stap proctree.stp -c "firefox --new-instance"
fork firefox(3179) -> firefox(3182)
fork firefox(3179) -> basename(3182)
fork firefox(3179) -> firefox(3183)
fork firefox(3179) -> uname(3183)
fork firefox(3179) -> firefox(3184)
fork firefox(3179) -> pidof(3184)
fork firefox(3179) -> firefox(3185)
fork firefox(3179) -> mkdir(3185)
fork firefox(3179) -> firefox(3186)
fork firefox(3186) -> firefox(3187)
fork firefox(3186) -> firefox(3188)
fork firefox(3186) -> sed(3188)
fork firefox(3179) -> firefox(3189)
fork firefox(3189) -> firefox(3190)
fork firefox(3189) -> firefox(3191)
fork firefox(3189) -> sed(3191)
fork firefox(3179) -> firefox(3192)
fork firefox(3179) -> expr(3192)
fork run-mozilla.sh(3179) -> run-mozilla.sh(3193)
fork run-mozilla.sh(3179) -> basename(3193)
fork run-mozilla.sh(3179) -> run-mozilla.sh(3194)
fork run-mozilla.sh(3179) -> dirname(3194)
fork run-mozilla.sh(3179) -> run-mozilla.sh(3195)
fork run-mozilla.sh(3179) -> uname(3195)
fork firefox(3179) -> firefox(3196)
fork firefox(3179) -> firefox(3198)
fork Gecko_IOThread(3179) -> Gecko_IOThread(3266)
fork Gecko_IOThread(3179) -> firefox(3266)

Re: Using stapdyn to probe child processes of the target?

[Arkady]

I have never tried user space hooking. My immediate idea is to run gcc
under strace, filter all fork-exec pairs (a Python script here?) and
attach stap to all child PIDs

On Fri, Jun 16, 2017 at 12:47 AM, Serhei Makarov <serhei.etc@gmail.com> wrote:
> Hello all,
>
> I've been trying to observe gcc's execution using Stapdyn. Most of the
> actual work is done by a child process (cc1) forked from gcc. (Below
> I've attached a script, proctree.stp, for listing processes forked by
> a target command.) If I'm using kernel systemtap, I can just target
> cc1 directly, for example:
>
> stap -ve 'probe
> process("/usr/libexec/gcc/x86_64-redhat-linux/6.3.1/cc1").insn {
> counter<<<1 } global counter probe end { printf("%d calls\n",
> @count(counter)) }' -c "gcc test/widget3.c"
>
> Instrumenting using kernel uprobes introduces quite a large
> performance overhead, so I would prefer Dyninst for this. But the same
> script doesn't work with --dyninst option, because the target process
> selected using -c is not cc1.
>
> As far as I can tell, my only options for observing cc1 with Stapdyn are:
>
> (a) use -c option : invoke cc1 directly. Feasible in this case, may
> not be feasible in other software where the parent/child processes
> coordinate using more complex IPC. This also doesn't give a profile of
> the full GCC execution (unlike, say, "valgrind --trace-children=yes").
> (b) use -x option : attach to cc1 by PID after it starts running. Not
> feasible in this case since cc1 is a batch program, not a long-running
> daemon.
>
> Am I missing any possibilities? Is it currently possible to use
> Stapdyn to attach to processes other than a single target?
>
> All the best,
>      Serhei Makarov
>
> PS Here's the proctree.stp script:
>
> global pids
>
> probe begin {
>   pids[target()] = 0
> }
>
> probe process.begin {
>   child = pid(); parent = ppid()
>   if (parent in pids) {
>     printf("fork %s(%d) -> %s(%d)\n", pexecname(), parent, execname(), child)
>     pids[child] = parent
>   }
> }
>
> This outputs things like:
>
> [serhei@beatrice e-tracer]$ stap proctree.stp -c "gcc test/widget3.c"
> Missing separate debuginfos, use: debuginfo-install
> kernel-core-4.10.10-200.fc25.x86_64
> fork gcc(2725) -> gcc(2728)
> fork gcc(2725) -> cc1(2728)
> fork gcc(2725) -> gcc(2729)
> fork gcc(2725) -> as(2729)
> fork gcc(2725) -> gcc(2730)
> fork gcc(2725) -> collect2(2730)
> fork collect2(2730) -> collect2(2731)
> fork collect2(2730) -> ld(2731)
>
> Or, here's a Firefox browsing session:
>
> [serhei@beatrice e-tracer]$ stap proctree.stp -c "firefox --new-instance"
> fork firefox(3179) -> firefox(3182)
> fork firefox(3179) -> basename(3182)
> fork firefox(3179) -> firefox(3183)
> fork firefox(3179) -> uname(3183)
> fork firefox(3179) -> firefox(3184)
> fork firefox(3179) -> pidof(3184)
> fork firefox(3179) -> firefox(3185)
> fork firefox(3179) -> mkdir(3185)
> fork firefox(3179) -> firefox(3186)
> fork firefox(3186) -> firefox(3187)
> fork firefox(3186) -> firefox(3188)
> fork firefox(3186) -> sed(3188)
> fork firefox(3179) -> firefox(3189)
> fork firefox(3189) -> firefox(3190)
> fork firefox(3189) -> firefox(3191)
> fork firefox(3189) -> sed(3191)
> fork firefox(3179) -> firefox(3192)
> fork firefox(3179) -> expr(3192)
> fork run-mozilla.sh(3179) -> run-mozilla.sh(3193)
> fork run-mozilla.sh(3179) -> basename(3193)
> fork run-mozilla.sh(3179) -> run-mozilla.sh(3194)
> fork run-mozilla.sh(3179) -> dirname(3194)
> fork run-mozilla.sh(3179) -> run-mozilla.sh(3195)
> fork run-mozilla.sh(3179) -> uname(3195)
> fork firefox(3179) -> firefox(3196)
> fork firefox(3179) -> firefox(3198)
> fork Gecko_IOThread(3179) -> Gecko_IOThread(3266)
> fork Gecko_IOThread(3179) -> firefox(3266)

Re: Using stapdyn to probe child processes of the target?

[Arkady]

If the timing is important you can add some code to the script. For
example you can call to processAttach() in the script when a certain
function in the GCC starts a new process.

On Fri, Jun 16, 2017 at 7:05 AM, Arkady <arkady.miasnikov@gmail.com> wrote:
> I have never tried user space hooking. My immediate idea is to run gcc
> under strace, filter all fork-exec pairs (a Python script here?) and
> attach stap to all child PIDs
>
> On Fri, Jun 16, 2017 at 12:47 AM, Serhei Makarov <serhei.etc@gmail.com> wrote:
>> Hello all,
>>
>> I've been trying to observe gcc's execution using Stapdyn. Most of the
>> actual work is done by a child process (cc1) forked from gcc. (Below
>> I've attached a script, proctree.stp, for listing processes forked by
>> a target command.) If I'm using kernel systemtap, I can just target
>> cc1 directly, for example:
>>
>> stap -ve 'probe
>> process("/usr/libexec/gcc/x86_64-redhat-linux/6.3.1/cc1").insn {
>> counter<<<1 } global counter probe end { printf("%d calls\n",
>> @count(counter)) }' -c "gcc test/widget3.c"
>>
>> Instrumenting using kernel uprobes introduces quite a large
>> performance overhead, so I would prefer Dyninst for this. But the same
>> script doesn't work with --dyninst option, because the target process
>> selected using -c is not cc1.
>>
>> As far as I can tell, my only options for observing cc1 with Stapdyn are:
>>
>> (a) use -c option : invoke cc1 directly. Feasible in this case, may
>> not be feasible in other software where the parent/child processes
>> coordinate using more complex IPC. This also doesn't give a profile of
>> the full GCC execution (unlike, say, "valgrind --trace-children=yes").
>> (b) use -x option : attach to cc1 by PID after it starts running. Not
>> feasible in this case since cc1 is a batch program, not a long-running
>> daemon.
>>
>> Am I missing any possibilities? Is it currently possible to use
>> Stapdyn to attach to processes other than a single target?
>>
>> All the best,
>>      Serhei Makarov
>>
>> PS Here's the proctree.stp script:
>>
>> global pids
>>
>> probe begin {
>>   pids[target()] = 0
>> }
>>
>> probe process.begin {
>>   child = pid(); parent = ppid()
>>   if (parent in pids) {
>>     printf("fork %s(%d) -> %s(%d)\n", pexecname(), parent, execname(), child)
>>     pids[child] = parent
>>   }
>> }
>>
>> This outputs things like:
>>
>> [serhei@beatrice e-tracer]$ stap proctree.stp -c "gcc test/widget3.c"
>> Missing separate debuginfos, use: debuginfo-install
>> kernel-core-4.10.10-200.fc25.x86_64
>> fork gcc(2725) -> gcc(2728)
>> fork gcc(2725) -> cc1(2728)
>> fork gcc(2725) -> gcc(2729)
>> fork gcc(2725) -> as(2729)
>> fork gcc(2725) -> gcc(2730)
>> fork gcc(2725) -> collect2(2730)
>> fork collect2(2730) -> collect2(2731)
>> fork collect2(2730) -> ld(2731)
>>
>> Or, here's a Firefox browsing session:
>>
>> [serhei@beatrice e-tracer]$ stap proctree.stp -c "firefox --new-instance"
>> fork firefox(3179) -> firefox(3182)
>> fork firefox(3179) -> basename(3182)
>> fork firefox(3179) -> firefox(3183)
>> fork firefox(3179) -> uname(3183)
>> fork firefox(3179) -> firefox(3184)
>> fork firefox(3179) -> pidof(3184)
>> fork firefox(3179) -> firefox(3185)
>> fork firefox(3179) -> mkdir(3185)
>> fork firefox(3179) -> firefox(3186)
>> fork firefox(3186) -> firefox(3187)
>> fork firefox(3186) -> firefox(3188)
>> fork firefox(3186) -> sed(3188)
>> fork firefox(3179) -> firefox(3189)
>> fork firefox(3189) -> firefox(3190)
>> fork firefox(3189) -> firefox(3191)
>> fork firefox(3189) -> sed(3191)
>> fork firefox(3179) -> firefox(3192)
>> fork firefox(3179) -> expr(3192)
>> fork run-mozilla.sh(3179) -> run-mozilla.sh(3193)
>> fork run-mozilla.sh(3179) -> basename(3193)
>> fork run-mozilla.sh(3179) -> run-mozilla.sh(3194)
>> fork run-mozilla.sh(3179) -> dirname(3194)
>> fork run-mozilla.sh(3179) -> run-mozilla.sh(3195)
>> fork run-mozilla.sh(3179) -> uname(3195)
>> fork firefox(3179) -> firefox(3196)
>> fork firefox(3179) -> firefox(3198)
>> fork Gecko_IOThread(3179) -> Gecko_IOThread(3266)
>> fork Gecko_IOThread(3179) -> firefox(3266)

Re: Using stapdyn to probe child processes of the target?

[Arkady]

I see that there is pre-fork and post-fork hooks in the Dyninst code.

On Fri, Jun 16, 2017 at 7:17 AM, Arkady <arkady.miasnikov@gmail.com> wrote:
> If the timing is important you can add some code to the script. For
> example you can call to processAttach() in the script when a certain
> function in the GCC starts a new process.
>
> On Fri, Jun 16, 2017 at 7:05 AM, Arkady <arkady.miasnikov@gmail.com> wrote:
>> I have never tried user space hooking. My immediate idea is to run gcc
>> under strace, filter all fork-exec pairs (a Python script here?) and
>> attach stap to all child PIDs
>>
>> On Fri, Jun 16, 2017 at 12:47 AM, Serhei Makarov <serhei.etc@gmail.com> wrote:
>>> Hello all,
>>>
>>> I've been trying to observe gcc's execution using Stapdyn. Most of the
>>> actual work is done by a child process (cc1) forked from gcc. (Below
>>> I've attached a script, proctree.stp, for listing processes forked by
>>> a target command.) If I'm using kernel systemtap, I can just target
>>> cc1 directly, for example:
>>>
>>> stap -ve 'probe
>>> process("/usr/libexec/gcc/x86_64-redhat-linux/6.3.1/cc1").insn {
>>> counter<<<1 } global counter probe end { printf("%d calls\n",
>>> @count(counter)) }' -c "gcc test/widget3.c"
>>>
>>> Instrumenting using kernel uprobes introduces quite a large
>>> performance overhead, so I would prefer Dyninst for this. But the same
>>> script doesn't work with --dyninst option, because the target process
>>> selected using -c is not cc1.
>>>
>>> As far as I can tell, my only options for observing cc1 with Stapdyn are:
>>>
>>> (a) use -c option : invoke cc1 directly. Feasible in this case, may
>>> not be feasible in other software where the parent/child processes
>>> coordinate using more complex IPC. This also doesn't give a profile of
>>> the full GCC execution (unlike, say, "valgrind --trace-children=yes").
>>> (b) use -x option : attach to cc1 by PID after it starts running. Not
>>> feasible in this case since cc1 is a batch program, not a long-running
>>> daemon.
>>>
>>> Am I missing any possibilities? Is it currently possible to use
>>> Stapdyn to attach to processes other than a single target?
>>>
>>> All the best,
>>>      Serhei Makarov
>>>
>>> PS Here's the proctree.stp script:
>>>
>>> global pids
>>>
>>> probe begin {
>>>   pids[target()] = 0
>>> }
>>>
>>> probe process.begin {
>>>   child = pid(); parent = ppid()
>>>   if (parent in pids) {
>>>     printf("fork %s(%d) -> %s(%d)\n", pexecname(), parent, execname(), child)
>>>     pids[child] = parent
>>>   }
>>> }
>>>
>>> This outputs things like:
>>>
>>> [serhei@beatrice e-tracer]$ stap proctree.stp -c "gcc test/widget3.c"
>>> Missing separate debuginfos, use: debuginfo-install
>>> kernel-core-4.10.10-200.fc25.x86_64
>>> fork gcc(2725) -> gcc(2728)
>>> fork gcc(2725) -> cc1(2728)
>>> fork gcc(2725) -> gcc(2729)
>>> fork gcc(2725) -> as(2729)
>>> fork gcc(2725) -> gcc(2730)
>>> fork gcc(2725) -> collect2(2730)
>>> fork collect2(2730) -> collect2(2731)
>>> fork collect2(2730) -> ld(2731)
>>>
>>> Or, here's a Firefox browsing session:
>>>
>>> [serhei@beatrice e-tracer]$ stap proctree.stp -c "firefox --new-instance"
>>> fork firefox(3179) -> firefox(3182)
>>> fork firefox(3179) -> basename(3182)
>>> fork firefox(3179) -> firefox(3183)
>>> fork firefox(3179) -> uname(3183)
>>> fork firefox(3179) -> firefox(3184)
>>> fork firefox(3179) -> pidof(3184)
>>> fork firefox(3179) -> firefox(3185)
>>> fork firefox(3179) -> mkdir(3185)
>>> fork firefox(3179) -> firefox(3186)
>>> fork firefox(3186) -> firefox(3187)
>>> fork firefox(3186) -> firefox(3188)
>>> fork firefox(3186) -> sed(3188)
>>> fork firefox(3179) -> firefox(3189)
>>> fork firefox(3189) -> firefox(3190)
>>> fork firefox(3189) -> firefox(3191)
>>> fork firefox(3189) -> sed(3191)
>>> fork firefox(3179) -> firefox(3192)
>>> fork firefox(3179) -> expr(3192)
>>> fork run-mozilla.sh(3179) -> run-mozilla.sh(3193)
>>> fork run-mozilla.sh(3179) -> basename(3193)
>>> fork run-mozilla.sh(3179) -> run-mozilla.sh(3194)
>>> fork run-mozilla.sh(3179) -> dirname(3194)
>>> fork run-mozilla.sh(3179) -> run-mozilla.sh(3195)
>>> fork run-mozilla.sh(3179) -> uname(3195)
>>> fork firefox(3179) -> firefox(3196)
>>> fork firefox(3179) -> firefox(3198)
>>> fork Gecko_IOThread(3179) -> Gecko_IOThread(3266)
>>> fork Gecko_IOThread(3179) -> firefox(3266)

Re: Using stapdyn to probe child processes of the target?

[Serhei Makarov]

Ok, I've found the code (in mutator::post_fork_callback) which is
meant to instrument the child process, but that code is not being
triggered when I run any of my scripts (I enabled the logging just to
check). I'll have time to investigate further on Monday. Most likely
there is a PEBKAC of some sort going on [1].

[1] https://en.wiktionary.org/wiki/PEBCAK

On Fri, Jun 16, 2017 at 12:22 AM, Arkady <arkady.miasnikov@gmail.com> wrote:
> I see that there is pre-fork and post-fork hooks in the Dyninst code.
>
> On Fri, Jun 16, 2017 at 7:17 AM, Arkady <arkady.miasnikov@gmail.com> wrote:
>> If the timing is important you can add some code to the script. For
>> example you can call to processAttach() in the script when a certain
>> function in the GCC starts a new process.
>>
>> On Fri, Jun 16, 2017 at 7:05 AM, Arkady <arkady.miasnikov@gmail.com> wrote:
>>> I have never tried user space hooking. My immediate idea is to run gcc
>>> under strace, filter all fork-exec pairs (a Python script here?) and
>>> attach stap to all child PIDs
>>>
>>> On Fri, Jun 16, 2017 at 12:47 AM, Serhei Makarov <serhei.etc@gmail.com> wrote:
>>>> Hello all,
>>>>
>>>> I've been trying to observe gcc's execution using Stapdyn. Most of the
>>>> actual work is done by a child process (cc1) forked from gcc. (Below
>>>> I've attached a script, proctree.stp, for listing processes forked by
>>>> a target command.) If I'm using kernel systemtap, I can just target
>>>> cc1 directly, for example:
>>>>
>>>> stap -ve 'probe
>>>> process("/usr/libexec/gcc/x86_64-redhat-linux/6.3.1/cc1").insn {
>>>> counter<<<1 } global counter probe end { printf("%d calls\n",
>>>> @count(counter)) }' -c "gcc test/widget3.c"
>>>>
>>>> Instrumenting using kernel uprobes introduces quite a large
>>>> performance overhead, so I would prefer Dyninst for this. But the same
>>>> script doesn't work with --dyninst option, because the target process
>>>> selected using -c is not cc1.
>>>>
>>>> As far as I can tell, my only options for observing cc1 with Stapdyn are:
>>>>
>>>> (a) use -c option : invoke cc1 directly. Feasible in this case, may
>>>> not be feasible in other software where the parent/child processes
>>>> coordinate using more complex IPC. This also doesn't give a profile of
>>>> the full GCC execution (unlike, say, "valgrind --trace-children=yes").
>>>> (b) use -x option : attach to cc1 by PID after it starts running. Not
>>>> feasible in this case since cc1 is a batch program, not a long-running
>>>> daemon.
>>>>
>>>> Am I missing any possibilities? Is it currently possible to use
>>>> Stapdyn to attach to processes other than a single target?
>>>>
>>>> All the best,
>>>>      Serhei Makarov
>>>>
>>>> PS Here's the proctree.stp script:
>>>>
>>>> global pids
>>>>
>>>> probe begin {
>>>>   pids[target()] = 0
>>>> }
>>>>
>>>> probe process.begin {
>>>>   child = pid(); parent = ppid()
>>>>   if (parent in pids) {
>>>>     printf("fork %s(%d) -> %s(%d)\n", pexecname(), parent, execname(), child)
>>>>     pids[child] = parent
>>>>   }
>>>> }
>>>>
>>>> This outputs things like:
>>>>
>>>> [serhei@beatrice e-tracer]$ stap proctree.stp -c "gcc test/widget3.c"
>>>> Missing separate debuginfos, use: debuginfo-install
>>>> kernel-core-4.10.10-200.fc25.x86_64
>>>> fork gcc(2725) -> gcc(2728)
>>>> fork gcc(2725) -> cc1(2728)
>>>> fork gcc(2725) -> gcc(2729)
>>>> fork gcc(2725) -> as(2729)
>>>> fork gcc(2725) -> gcc(2730)
>>>> fork gcc(2725) -> collect2(2730)
>>>> fork collect2(2730) -> collect2(2731)
>>>> fork collect2(2730) -> ld(2731)
>>>>
>>>> Or, here's a Firefox browsing session:
>>>>
>>>> [serhei@beatrice e-tracer]$ stap proctree.stp -c "firefox --new-instance"
>>>> fork firefox(3179) -> firefox(3182)
>>>> fork firefox(3179) -> basename(3182)
>>>> fork firefox(3179) -> firefox(3183)
>>>> fork firefox(3179) -> uname(3183)
>>>> fork firefox(3179) -> firefox(3184)
>>>> fork firefox(3179) -> pidof(3184)
>>>> fork firefox(3179) -> firefox(3185)
>>>> fork firefox(3179) -> mkdir(3185)
>>>> fork firefox(3179) -> firefox(3186)
>>>> fork firefox(3186) -> firefox(3187)
>>>> fork firefox(3186) -> firefox(3188)
>>>> fork firefox(3186) -> sed(3188)
>>>> fork firefox(3179) -> firefox(3189)
>>>> fork firefox(3189) -> firefox(3190)
>>>> fork firefox(3189) -> firefox(3191)
>>>> fork firefox(3189) -> sed(3191)
>>>> fork firefox(3179) -> firefox(3192)
>>>> fork firefox(3179) -> expr(3192)
>>>> fork run-mozilla.sh(3179) -> run-mozilla.sh(3193)
>>>> fork run-mozilla.sh(3179) -> basename(3193)
>>>> fork run-mozilla.sh(3179) -> run-mozilla.sh(3194)
>>>> fork run-mozilla.sh(3179) -> dirname(3194)
>>>> fork run-mozilla.sh(3179) -> run-mozilla.sh(3195)
>>>> fork run-mozilla.sh(3179) -> uname(3195)
>>>> fork firefox(3179) -> firefox(3196)
>>>> fork firefox(3179) -> firefox(3198)
>>>> fork Gecko_IOThread(3179) -> Gecko_IOThread(3266)
>>>> fork Gecko_IOThread(3179) -> firefox(3266)

Re: Using stapdyn to probe child processes of the target?

[Frank Ch. Eigler]

serhei.etc wrote:

> [...] If I'm using kernel systemtap, I can just target
> cc1 directly, for example:
>
> stap -ve 'probe
> process("/usr/libexec/gcc/x86_64-redhat-linux/6.3.1/cc1").insn {
> counter<<<1 } global counter probe end { printf("%d calls\n",
> @count(counter)) }' -c "gcc test/widget3.c"
> [...]

As you noticed, this is a terribly inefficient way of counting
instructions.  You could try instead perfcounters to estimate.  It
samples an instruction perfcounter for a target process (thread) at
every function-return, and since this number only gets larger, it will
eventually get close to the overall count as the process exits.
(We should be able to sample that counter at a process.end type event
too, but our implementation is limited.)

% cat > countem.stp << 'END'
@define cc1 %( "/usr/libexec/gcc/x86_64-redhat-linux/6.3.1/cc1" %)
global insns
probe perf.hw.instructions.process(@cc1).counter("foo") {}
probe process(@cc1).function("*").return {
   insns[execname(),tid()] = @perf("foo") // implicit max()
}
END

% sudo stap countem.stp -c "gcc ...."
insns["cc1",2102]=0x367fb494
%

- FChE