Re: [PATCHv3] gdb: add Python events for program space addition and removal

[Andrew Burgess <aburgess@redhat.com>]

Simon Marchi <simark@simark.ca> writes:

> On 9/29/23 09:24, Andrew Burgess via Gdb-patches wrote:
>> In v3:
>> 
>>   - Rebased onto current master (there were some conflicts),
>> 
>>   - No other changes.
>> 
>> Code is already approved, just needs a doc review.
>> 
>> Thanks,
>> Andrew
>> 
>> 
>> ---
>> 
>> Initially I just wanted a Python event for when GDB removes a program
>> space, I'm writing a Python extension that caches information for each
>> program space, and need to know when I should discard entries for a
>> particular program space.
>> 
>> But, it seemed easy enough to also add an event for when GDB adds a
>> new program space, so I went ahead and added both new events.
>> 
>> Of course, we don't currently have an observable for program space
>> addition or removal, so I first needed to add these.  After that it's
>> pretty simple to add two new Python events and have these trigger.
>> 
>> The two new event registries are:
>> 
>>   events.new_progspace
>>   events.free_progspace
>> 
>> These emit NewProgspaceEvent and FreeProgspaceEvent objects
>> respectively, each of these new event types has a 'progspace'
>> attribute that contains the relevant gdb.Progspace object.
>> 
>> There's a couple of things to be mindful of.
>> 
>> First, it is not possible to catch the NewProgspaceEvent for the very
>> first program space, the one that is created when GDB first starts, as
>> this program space is created before any Python scripts are sourced.
>> 
>> In order to allow this event to be caught we would need to defer
>> creating the first program space, and as a consequence the first
>> inferior, until some later time.  But, existing scripts could easily
>> depend on there being an initial inferior, so I really don't think we
>> should change that -- and so, we end up with the consequence that we
>> can't catch the event for the first program space.
>> 
>> The second, I think minor, issue, is that GDB doesn't clean up its
>> program spaces upon exit -- or at least, they are not cleaned up
>> before Python is shut down.  As a result, any program spaces in use at
>> the time GDB exits don't generate a FreeProgspaceEvent.  I'm not
>> particularly worried about this for my use case, I'm using the event
>> to ensure that a cache doesn't hold stale entries within a single GDB
>> session.  It's also easy enough to add a Python at-exit callback which
>> can do any final cleanup if needed.
>> 
>> Finally, when testing, I did hit a slightly weird issue with some of
>> the remote boards (e.g. remote-stdio-gdbserver).  As a consequence of
>> this issue I see some output like this in the gdb.log:
>> 
>>   (gdb) PASS: gdb.python/py-progspace-events.exp: inferior 1
>>   step
>>   FreeProgspaceEvent: <gdb.Progspace object at 0x7fb7e1d19c10>
>>   warning: cannot close "target:/lib64/libm.so.6": Cannot execute this command while the target is running.
>>   Use the "interrupt" command to stop the target
>>   and then try again.
>>   warning: cannot close "target:/lib64/libc.so.6": Cannot execute this command while the target is running.
>>   Use the "interrupt" command to stop the target
>>   and then try again.
>>   warning: cannot close "target:/lib64/ld-linux-x86-64.so.2": Cannot execute this command while the target is running.
>>   Use the "interrupt" command to stop the target
>>   and then try again.
>>   do_parent_stuff () at py-progspace-events.c:41
>>   41        ++global_var;
>>   (gdb) PASS: gdb.python/py-progspace-events.exp: step
>> 
>> The 'FreeProgspaceEvent ...' line is expected, that's my test Python
>> extension logging the event.  What isn't expected are all the blocks
>> like:
>> 
>>   warning: cannot close "target:/lib64/libm.so.6": Cannot execute this command while the target is running.
>>   Use the "interrupt" command to stop the target
>>   and then try again.
>> 
>> It turns out that this has nothing to do with my changes, this is just
>> a consequence of reading files over the remote protocol.  The test
>> forks a child process which GDB stays attached too.  When the child
>> exits, GDB cleans up by calling prune_inferiors, which in turn can
>> result in GDB trying to close some files that are open because of the
>> inferior being deleted.
>> 
>> If the prune_inferiors call occurs when the remote target is
>> running (and in non-async mode) then GDB will try to send a fileio
>> packet while the remote target is waiting for a stop reply, and the
>> remote target will throw an error, see remote_target::putpkt_binary in
>> remote.c for details.
>> 
>> I'm going to look at fixing this, but, as I said, this is nothing to
>> do with this change, I just mention it because I ended up needing to
>> account for these warning messages in one of my tests, and it all
>> looks a bit weird.
>> 
>> Approved-By: Tom Tromey <tom@tromey.com>
>
> Hi Andrew,
>
> The new test fails with native-gdbserver:
>
>
>     Running /home/smarchi/src/binutils-gdb/gdb/testsuite/gdb.python/py-progspace-events.exp ...
>     ERROR: GDB process no longer exists
>
>     # of expected passes            9
>     # of unresolved testcases       4
>
> GDB aborts when calling error here, apparently nothing catches the
> exception so std::terminate is called:

Thanks for bringing this to my attention.  I was sure I'd tested this on
all boards (I've been trying to make use of make-check-all.sh lately),
but I guess I messed up here :-/

I will 100% look into this issue, however, it's worth nothing that this
is not something my patch broke, it's just my test exposed this issue,
which is a regression since GDB 13.

I say this only in case anyone else has any thoughts.

I will pick this up tomorrow to investigate.

Thanks,
Andrew


>
>     #4  0x000055a4a0fae18d in error (fmt=0x55a49adcd020 "Cannot execute this command while the target is running.\nUse the \"interrupt\" command to stop the target\nand then try again.") at /home/smarchi/src/binutils-gdb/gdbsupport/errors.cc:46
>     #5  0x000055a49ec2f495 in remote_target::putpkt_binary (this=0x61b00003f080, buf=0x62d00371e400 "qXfer:auxv:read::0,1000", cnt=23) at /home/smarchi/src/binutils-gdb/gdb/remote.c:9740
>     #6  0x000055a49ec2efc4 in remote_target::putpkt (this=0x61b00003f080, buf=0x62d00371e400 "qXfer:auxv:read::0,1000") at /home/smarchi/src/binutils-gdb/gdb/remote.c:9698
>     #7  0x000055a49ec8b287 in remote_target::putpkt (this=0x61b00003f080, buf=std::__debug::vector of length 36862, capacity 36862 = {...}) at /home/smarchi/src/binutils-gdb/gdb/remote.c:1211
>     #8  0x000055a49ec45c59 in remote_target::remote_read_qxfer (this=0x61b00003f080, object_name=0x55a49adced00 "auxv", annex=0x0, readbuf=0x62100057f900 '\276' <repeats 200 times>..., offset=0, len=3904966623561266976, xfered_len=0x7f5f31c5f220, which_packet=18) at /home/smarchi/src/binutils-gdb/gdb/remote.c:11316
>     #9  0x000055a49ec47573 in remote_target::xfer_partial (this=0x61b00003f080, object=TARGET_OBJECT_AUXV, annex=0x0, readbuf=0x62100057f900 '\276' <repeats 200 times>..., writebuf=0x0, offset=0, len=4096, xfered_len=0x7f5f31c5f220) at /home/smarchi/src/binutils-gdb/gdb/remote.c:11438
>     #10 0x000055a49f4595b1 in target_xfer_partial (ops=0x61b00003f080, object=TARGET_OBJECT_AUXV, annex=0x0, readbuf=0x62100057f900 '\276' <repeats 200 times>..., writebuf=0x0, offset=0, len=4096, xfered_len=0x7f5f31c5f220) at /home/smarchi/src/binutils-gdb/gdb/target.c:1717
>     #11 0x000055a49f45aed9 in target_read_partial (ops=0x61b00003f080, object=TARGET_OBJECT_AUXV, annex=0x0, buf=0x62100057f900 '\276' <repeats 200 times>..., offset=0, len=4096, xfered_len=0x7f5f31c5f220) at /home/smarchi/src/binutils-gdb/gdb/target.c:1951
>     #12 0x000055a49f501472 in target_read_alloc_1<unsigned char> (ops=0x61b00003f080, object=TARGET_OBJECT_AUXV, annex=0x0) at /home/smarchi/src/binutils-gdb/gdb/target.c:2286
>     #13 0x000055a49f45c9c2 in target_read_alloc (ops=0x61b00003f080, object=TARGET_OBJECT_AUXV, annex=0x0) at /home/smarchi/src/binutils-gdb/gdb/target.c:2315
>     #14 0x000055a49c920733 in target_read_auxv_raw (ops=0x61b00003f080) at /home/smarchi/src/binutils-gdb/gdb/auxv.c:379
>     #15 0x000055a49c920501 in target_read_auxv () at /home/smarchi/src/binutils-gdb/gdb/auxv.c:368
>     #16 0x000055a49c920c06 in target_auxv_search (match=0, valp=0x7f5f31a2fc60) at /home/smarchi/src/binutils-gdb/gdb/auxv.c:415
>     #17 0x000055a49e13e669 in linux_is_uclinux () at /home/smarchi/src/binutils-gdb/gdb/linux-tdep.c:433
>     #18 0x000055a49e13e6fa in linux_has_shared_address_space (gdbarch=0x61c00001e080) at /home/smarchi/src/binutils-gdb/gdb/linux-tdep.c:440
>     #19 0x000055a49c720d6f in gdbarch_has_shared_address_space (gdbarch=0x61c00001e080) at /home/smarchi/src/binutils-gdb/gdb/gdbarch.c:4889
>     #20 0x000055a49e7ce076 in program_space::~program_space (this=0x61300004af00, __in_chrg=<optimized out>) at /home/smarchi/src/binutils-gdb/gdb/progspace.c:124
>     #21 0x000055a49de5d91e in delete_inferior (inf=0x61800007f080) at /home/smarchi/src/binutils-gdb/gdb/inferior.c:290
>     #22 0x000055a49de604b7 in prune_inferiors () at /home/smarchi/src/binutils-gdb/gdb/inferior.c:480
>     #23 0x000055a49debd2d9 in fetch_inferior_event () at /home/smarchi/src/binutils-gdb/gdb/infrun.c:4558
>
> Simon