Re: [PATCHv3] gdb: add Python events for program space addition and removal

[Simon Marchi <simark@simark.ca>]

On 9/29/23 09:24, Andrew Burgess via Gdb-patches wrote:
> In v3:
> 
>   - Rebased onto current master (there were some conflicts),
> 
>   - No other changes.
> 
> Code is already approved, just needs a doc review.
> 
> Thanks,
> Andrew
> 
> 
> ---
> 
> Initially I just wanted a Python event for when GDB removes a program
> space, I'm writing a Python extension that caches information for each
> program space, and need to know when I should discard entries for a
> particular program space.
> 
> But, it seemed easy enough to also add an event for when GDB adds a
> new program space, so I went ahead and added both new events.
> 
> Of course, we don't currently have an observable for program space
> addition or removal, so I first needed to add these.  After that it's
> pretty simple to add two new Python events and have these trigger.
> 
> The two new event registries are:
> 
>   events.new_progspace
>   events.free_progspace
> 
> These emit NewProgspaceEvent and FreeProgspaceEvent objects
> respectively, each of these new event types has a 'progspace'
> attribute that contains the relevant gdb.Progspace object.
> 
> There's a couple of things to be mindful of.
> 
> First, it is not possible to catch the NewProgspaceEvent for the very
> first program space, the one that is created when GDB first starts, as
> this program space is created before any Python scripts are sourced.
> 
> In order to allow this event to be caught we would need to defer
> creating the first program space, and as a consequence the first
> inferior, until some later time.  But, existing scripts could easily
> depend on there being an initial inferior, so I really don't think we
> should change that -- and so, we end up with the consequence that we
> can't catch the event for the first program space.
> 
> The second, I think minor, issue, is that GDB doesn't clean up its
> program spaces upon exit -- or at least, they are not cleaned up
> before Python is shut down.  As a result, any program spaces in use at
> the time GDB exits don't generate a FreeProgspaceEvent.  I'm not
> particularly worried about this for my use case, I'm using the event
> to ensure that a cache doesn't hold stale entries within a single GDB
> session.  It's also easy enough to add a Python at-exit callback which
> can do any final cleanup if needed.
> 
> Finally, when testing, I did hit a slightly weird issue with some of
> the remote boards (e.g. remote-stdio-gdbserver).  As a consequence of
> this issue I see some output like this in the gdb.log:
> 
>   (gdb) PASS: gdb.python/py-progspace-events.exp: inferior 1
>   step
>   FreeProgspaceEvent: <gdb.Progspace object at 0x7fb7e1d19c10>
>   warning: cannot close "target:/lib64/libm.so.6": Cannot execute this command while the target is running.
>   Use the "interrupt" command to stop the target
>   and then try again.
>   warning: cannot close "target:/lib64/libc.so.6": Cannot execute this command while the target is running.
>   Use the "interrupt" command to stop the target
>   and then try again.
>   warning: cannot close "target:/lib64/ld-linux-x86-64.so.2": Cannot execute this command while the target is running.
>   Use the "interrupt" command to stop the target
>   and then try again.
>   do_parent_stuff () at py-progspace-events.c:41
>   41        ++global_var;
>   (gdb) PASS: gdb.python/py-progspace-events.exp: step
> 
> The 'FreeProgspaceEvent ...' line is expected, that's my test Python
> extension logging the event.  What isn't expected are all the blocks
> like:
> 
>   warning: cannot close "target:/lib64/libm.so.6": Cannot execute this command while the target is running.
>   Use the "interrupt" command to stop the target
>   and then try again.
> 
> It turns out that this has nothing to do with my changes, this is just
> a consequence of reading files over the remote protocol.  The test
> forks a child process which GDB stays attached too.  When the child
> exits, GDB cleans up by calling prune_inferiors, which in turn can
> result in GDB trying to close some files that are open because of the
> inferior being deleted.
> 
> If the prune_inferiors call occurs when the remote target is
> running (and in non-async mode) then GDB will try to send a fileio
> packet while the remote target is waiting for a stop reply, and the
> remote target will throw an error, see remote_target::putpkt_binary in
> remote.c for details.
> 
> I'm going to look at fixing this, but, as I said, this is nothing to
> do with this change, I just mention it because I ended up needing to
> account for these warning messages in one of my tests, and it all
> looks a bit weird.
> 
> Approved-By: Tom Tromey <tom@tromey.com>

Hi Andrew,

The new test fails with native-gdbserver:


    Running /home/smarchi/src/binutils-gdb/gdb/testsuite/gdb.python/py-progspace-events.exp ...
    ERROR: GDB process no longer exists

    # of expected passes            9
    # of unresolved testcases       4

GDB aborts when calling error here, apparently nothing catches the
exception so std::terminate is called:

    #4  0x000055a4a0fae18d in error (fmt=0x55a49adcd020 "Cannot execute this command while the target is running.\nUse the \"interrupt\" command to stop the target\nand then try again.") at /home/smarchi/src/binutils-gdb/gdbsupport/errors.cc:46
    #5  0x000055a49ec2f495 in remote_target::putpkt_binary (this=0x61b00003f080, buf=0x62d00371e400 "qXfer:auxv:read::0,1000", cnt=23) at /home/smarchi/src/binutils-gdb/gdb/remote.c:9740
    #6  0x000055a49ec2efc4 in remote_target::putpkt (this=0x61b00003f080, buf=0x62d00371e400 "qXfer:auxv:read::0,1000") at /home/smarchi/src/binutils-gdb/gdb/remote.c:9698
    #7  0x000055a49ec8b287 in remote_target::putpkt (this=0x61b00003f080, buf=std::__debug::vector of length 36862, capacity 36862 = {...}) at /home/smarchi/src/binutils-gdb/gdb/remote.c:1211
    #8  0x000055a49ec45c59 in remote_target::remote_read_qxfer (this=0x61b00003f080, object_name=0x55a49adced00 "auxv", annex=0x0, readbuf=0x62100057f900 '\276' <repeats 200 times>..., offset=0, len=3904966623561266976, xfered_len=0x7f5f31c5f220, which_packet=18) at /home/smarchi/src/binutils-gdb/gdb/remote.c:11316
    #9  0x000055a49ec47573 in remote_target::xfer_partial (this=0x61b00003f080, object=TARGET_OBJECT_AUXV, annex=0x0, readbuf=0x62100057f900 '\276' <repeats 200 times>..., writebuf=0x0, offset=0, len=4096, xfered_len=0x7f5f31c5f220) at /home/smarchi/src/binutils-gdb/gdb/remote.c:11438
    #10 0x000055a49f4595b1 in target_xfer_partial (ops=0x61b00003f080, object=TARGET_OBJECT_AUXV, annex=0x0, readbuf=0x62100057f900 '\276' <repeats 200 times>..., writebuf=0x0, offset=0, len=4096, xfered_len=0x7f5f31c5f220) at /home/smarchi/src/binutils-gdb/gdb/target.c:1717
    #11 0x000055a49f45aed9 in target_read_partial (ops=0x61b00003f080, object=TARGET_OBJECT_AUXV, annex=0x0, buf=0x62100057f900 '\276' <repeats 200 times>..., offset=0, len=4096, xfered_len=0x7f5f31c5f220) at /home/smarchi/src/binutils-gdb/gdb/target.c:1951
    #12 0x000055a49f501472 in target_read_alloc_1<unsigned char> (ops=0x61b00003f080, object=TARGET_OBJECT_AUXV, annex=0x0) at /home/smarchi/src/binutils-gdb/gdb/target.c:2286
    #13 0x000055a49f45c9c2 in target_read_alloc (ops=0x61b00003f080, object=TARGET_OBJECT_AUXV, annex=0x0) at /home/smarchi/src/binutils-gdb/gdb/target.c:2315
    #14 0x000055a49c920733 in target_read_auxv_raw (ops=0x61b00003f080) at /home/smarchi/src/binutils-gdb/gdb/auxv.c:379
    #15 0x000055a49c920501 in target_read_auxv () at /home/smarchi/src/binutils-gdb/gdb/auxv.c:368
    #16 0x000055a49c920c06 in target_auxv_search (match=0, valp=0x7f5f31a2fc60) at /home/smarchi/src/binutils-gdb/gdb/auxv.c:415
    #17 0x000055a49e13e669 in linux_is_uclinux () at /home/smarchi/src/binutils-gdb/gdb/linux-tdep.c:433
    #18 0x000055a49e13e6fa in linux_has_shared_address_space (gdbarch=0x61c00001e080) at /home/smarchi/src/binutils-gdb/gdb/linux-tdep.c:440
    #19 0x000055a49c720d6f in gdbarch_has_shared_address_space (gdbarch=0x61c00001e080) at /home/smarchi/src/binutils-gdb/gdb/gdbarch.c:4889
    #20 0x000055a49e7ce076 in program_space::~program_space (this=0x61300004af00, __in_chrg=<optimized out>) at /home/smarchi/src/binutils-gdb/gdb/progspace.c:124
    #21 0x000055a49de5d91e in delete_inferior (inf=0x61800007f080) at /home/smarchi/src/binutils-gdb/gdb/inferior.c:290
    #22 0x000055a49de604b7 in prune_inferiors () at /home/smarchi/src/binutils-gdb/gdb/inferior.c:480
    #23 0x000055a49debd2d9 in fetch_inferior_event () at /home/smarchi/src/binutils-gdb/gdb/infrun.c:4558

Simon