From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pl1-x633.google.com (mail-pl1-x633.google.com [IPv6:2607:f8b0:4864:20::633]) by sourceware.org (Postfix) with ESMTPS id 34F103858D39 for ; Sun, 12 Mar 2023 17:24:19 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 34F103858D39 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=gmail.com Received: by mail-pl1-x633.google.com with SMTP id i3so10585147plg.6 for ; Sun, 12 Mar 2023 10:24:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1678641858; h=content-transfer-encoding:in-reply-to:content-language:references :to:from:subject:user-agent:mime-version:date:message-id:from:to:cc :subject:date:message-id:reply-to; bh=9WE0Ro5atTykIY+PVMQeTVsEcl6EcOZahnf3NVIq4rk=; b=NKGuLimIZfloBcJ9b1EESuGLlMED3OJ+HzYQyjrhPs2MCTDSmRMjPZkV0EynLUscRH 76POY+vNuvCQrvdE1vhqEXGG3YNpq+5vd/gOWo5RevblJY8XOEFJb0SJM1AcKOMg5FK5 n2w6Ps/yX+WDNGAaDKOeBGJ52cHyF1Mj/6/qxSdk5ziiNS8H3Drk6/hIgt8w3gcHM4sy vK6YPh83g/0sSny8O4CYohOtpGkDWHX+Phy8sl96NVDdfz2D8QKyRcvwAaABVkOaHNxI mo0KW5D3MlXq7kr202RpfGbIqzrHpjzZU2f53Q1CZCA1cJ86SYULQmL/qloi2MwL4txT LUsw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1678641858; h=content-transfer-encoding:in-reply-to:content-language:references :to:from:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=9WE0Ro5atTykIY+PVMQeTVsEcl6EcOZahnf3NVIq4rk=; b=yR7wt1e2AzWOmh7NnsaqdjdOtNfckHvoGEWPze7p3q15XV6KoyL7UfL73oO70CgbP5 dUXz+Ls085eI6I241Fg4J4VAekzwTOhXqGIiMqR0vNpBklFHZqw7ZFDVISyKUVl7DrzE Q921NxSS4XD5BUyMn+WAS0h9I9OZz/fN6/WYGyp+sAB4sD1tpF+H0PSkTF6/DsBjXb/1 qs92SEvfjKFiLwxWm6QMckYWUdoPxcOOKTdOA+5Ssff0FTtk9CJVgOaMzwlforyYI2fT 3ptf6TP5wW3FDCZs+xJ1eqWgWlfWTbkN7DnC6klZXveAN5r9DxCtX/SxXXVieoZKKWnx JyjA== X-Gm-Message-State: AO0yUKWQUdfCFBN6xsGh6QwLNGXMGdhj8KE5P34glufKtVBwvAtzCW3p FK/2y4WnyI89nSquUZ9wfWc= X-Google-Smtp-Source: AK7set8U7ttOWoF1Xk04B26iV6m+1vjOBip6e7f2uw7N2K8DPBih1Z/pzXED8zr1cfOuM+nq03yxDA== X-Received: by 2002:a17:902:e84c:b0:19e:7628:9165 with SMTP id t12-20020a170902e84c00b0019e76289165mr35684801plg.4.1678641857880; Sun, 12 Mar 2023 10:24:17 -0700 (PDT) Received: from [192.168.2.101] (61-220-36-70.hinet-ip.hinet.net. [61.220.36.70]) by smtp.gmail.com with ESMTPSA id iy14-20020a170903130e00b001a0450da45csm654514plb.185.2023.03.12.10.24.16 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 12 Mar 2023 10:24:17 -0700 (PDT) Message-ID: <87cd3287-1267-2cc7-21d6-e2e183963d6f@gmail.com> Date: Mon, 13 Mar 2023 01:24:15 +0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.8.0 Subject: [PING] [RFC] [PATCH v3] [PR python/29603] Disable out-of-scope watchpoints From: Johnson Sun To: SimonMarchi , LancelotSIX , gdb-patches@sourceware.org References: <20221020175707.193041-1-j3.soon777@gmail.com> <20221020181101.193226-1-j3.soon777@gmail.com> <9e6aa582-efd6-b5bf-d4b9-aebeb2e9e845@simark.ca> <915dd1be-feb5-967d-2d32-af61c3fee9a7@gmail.com> <6f2d7693-7f29-c125-1b7a-44a8a4fbd76e@gmail.com> Content-Language: en-US In-Reply-To: <6f2d7693-7f29-c125-1b7a-44a8a4fbd76e@gmail.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-10.2 required=5.0 tests=BAYES_00,BODY_8BITS,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_ENVFROM_END_DIGIT,FREEMAIL_FROM,GIT_PATCH_0,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: Ping for comments: . Johnson On 2/26/2023 2:16 PM, Johnson Sun wrote: > Request for comments: > . > > Johnson > > On 2/19/2023 12:26 AM, JohnsonSun wrote: >> Ping for: >> . >> >> I would like to ask for some feedback before submitting the v4 patch. >> >> Johnson >> >> On 1/23/2023 6:15 PM, Johnson Sun wrote: >>> Hi, >>> >>> Thanks for reviewing this! >>> >>> > the supplied test does not pass for me >>> >>> The current test doesn't seem to produce consistent results across >>> different >>> machines, I'll try to fix it in the next version. >>> >>> > Is there a reason to do it here in particular, and not, let's say as >>> > soon as we change the disposition to disp_del_at_next_stop? >>> >>> I have implemented this in the v1 patch, I called >>> `disable_breakpoint' as >>> soon as we change the disposition to `disp_del_at_next_stop' >>> (in `watchpoint_del_at_next_stop'). However, >>> LancelotSIX mentioned that the fix is in a >>> non-extension-related code path, and suggested disabling it in >>> `bpstat_check_breakpoint_conditions' instead (the v3 patch). >>> See: >>> https://sourceware.org/pipermail/gdb-patches/2022-September/192120.html >>> >>> Both the v1 and v3 patch fixes the issue. I personally prefer the v1 >>> patch. >>> (i.e., modifying `watchpoint_del_at_next_stop') >>> >>> > shouldn't marking the watchpoint as >>> > disp_del_at_next_stop make it so it won't be inserted next time we >>> > resume?  After all should_be_inserted returns false for breakpoint >>> > locations that are disp_del_at_next_stop.  Perhaps it's because >>> since we >>> > don't do a "normal" stop, breakpoint locations stay inserted, and >>> > there's nothing that pulls this location out of the target? >>> Therefore, >>> > maybe a solution, to keep things coherent, would be to pull the >>> > breakpoint locations out when marking the breakpoint as >>> > disp_del_at_next_stop?  This way, we would respect the invariant >>> that a >>> > breakpoint disp_del_at_next_stop can't be inserted (I don't know >>> if this >>> > is really what is happening though, it's just speculation). >>> >>> For software watchpoints, they cannot be pulled out of the target, >>> since >>> they may not be inserted into the target in the first place: >>> >>>      /* Locations of type bp_loc_other and >>>     bp_loc_software_watchpoint are only maintained at GDB side, >>>     so there is no need to remove them.  */ >>> >>>     -- gdb/breakpoint.c:3840 >>> >>> Their expressions are re-evaluated by single-stepping through the >>> program: >>> >>>     Software watchpoints are very slow, since GDB needs to single-step >>>     the program being debugged and test the value of the watched >>>     expression(s) after each instruction. >>> >>>     -- https://sourceware.org/gdb/wiki/Internals%20Watchpoints >>> >>> Therefore, we must either disable or delete the software watchpoint. >>> We cannot pull it out of the target since it's only maintained on the >>> GDB side. >>> >>> > And, in a broader scope, why wait until the next normal stop to >>> delete >>> > the watchpoint?  A "normal" stop being a stop that we present to the >>> > user (the normal_stop function).  We currently call >>> > breakpoint_auto_delete in normal_stop, which is why we don't reach it >>> > when your Breakpoint.stop method returns False.  At the end of, let's >>> > say, handle_signal_stop, could we go over the bpstat chain and delete >>> > any breakpoint we've just hit that is marked for deletion? >>> >>> I believe this choice is due to performance issues. >>> >>> In an early attempt, I tried to call `breakpoint_auto_delete' on all >>> kinds >>> of stops. However, this implementation is very slow when we have a >>> large >>> number of breakpoints, since we need to go over the entire bpstat >>> chain on >>> all stops. I recall that this implementation fails on certain >>> testcases with >>> a large number of breakpoints with many breakpoint stops. >>> >>> Furthermore, the reason for using the `disp_del_at_next_stop' state >>> remains >>> unclear, as mentioned in the comments: >>> >>>     /* NOTE drow/2003-09-08: This state only exists for removing >>>        watchpoints.  It's not clear that it's necessary...  */ >>> >>>     -- gdb/breakpoint.c:2914 >>> >>> By tracing the git history, the `disp_del_at_next_stop' state is >>> introduced >>> in commit c906108c21474dfb4ed285bcc0ac6fe02cd400cc, which doesn't >>> provide >>> any proper explanation of this state. >>> >>> I think the best way to deal with this is to avoid going over the >>> entire >>> bpstat chain when deleting breakpoints. Potentially by keeping track of >>> a chain of breakpoints that should be deleted, and changing the >>> bpstat chain >>> to a doubly linked list for the ease of deletion. Such changes >>> deserve a >>> dedicated patch, though. >>> >>> To sum up, I prefer modifying `watchpoint_del_at_next_stop' (i.e., >>> the v1 patch). >>> If you also think it's appropriate, I'll fix the failing test and >>> prepare the >>> v4 patch accordingly. >>> >>> Johnson >>> >>> On 1/13/2023 11:40 PM, SimonMarchi wrote: >>>> >>>> On 10/20/22 14:11, Johnson Sun via Gdb-patches wrote: >>>>> Currently, when a local software watchpoint goes out of scope, GDB >>>>> sets the >>>>> watchpoint's disposition to `delete at next stop' and then normal >>>>> stops >>>>> (i.e., stop and wait for the next GDB command). When GDB normal >>>>> stops, it >>>>> automatically deletes the breakpoints with their disposition set to >>>>> `delete at next stop'. >>>>> >>>>> Suppose a Python script decides not to normal stop when a local >>>>> software >>>>> watchpoint goes out of scope, the watchpoint will not be >>>>> automatically >>>>> deleted even when its disposition is set to `delete at next stop'. >>>>> >>>>> Since GDB single-steps the program and tests the watched >>>>> expression after each >>>>> instruction, not deleting the watchpoint causes the watchpoint to >>>>> be hit many >>>>> more times than it should, as reported in PR python/29603. >>>>> >>>>> This was happening because the watchpoint is not deleted or >>>>> disabled when >>>>> going out of scope. >>>>> >>>>> This commit fixes this issue by disabling the watchpoint when >>>>> going out of >>>>> scope. It also adds a test to ensure this feature isn't regressed >>>>> in the >>>>> future. >>>>> >>>>> Two other solutions seem to solve this issue, but are in fact >>>>> inappropriate: >>>>> 1. Automatically delete breakpoints on all kinds of stops >>>>>     (in `fetch_inferior_event'). This solution is very slow since >>>>> the deletion >>>>>     requires O(N) time for N breakpoints. >>>>> 2. Disable the watchpoint after the watchpoint's disposition is >>>>> set to >>>>>     `delete at next stop' (in `watchpoint_del_at_next_stop'). This >>>>> solution >>>>>     modifies a non-extension-related code path, and isn't >>>>> preferred since this >>>>>     issue cannot occur without extension languages. (gdb scripts >>>>> always normal >>>>>     stop before continuing execution) >>>> I have a bit of trouble reviewing this, because I'm not too familiar >>>> with the lifecycle of watchpoints (I know the principle, but not the >>>> specifically where things happen in GDB).  So it's difficult to tell >>>> whether this is right or if there's a better way to handle it. >>>> >>>> However, the supplied test does not pass for me: >>>> >>>>      source >>>> /home/simark/build/binutils-gdb/gdb/testsuite/outputs/gdb.python/py-watchpoint/py-watchpoint.py >>>>      Watchpoint 2: i >>>>      Python script imported >>>>      (gdb) PASS: gdb.python/py-watchpoint.exp: import python scripts >>>>      python print(len(gdb.breakpoints())) >>>>      2 >>>>      (gdb) PASS: gdb.python/py-watchpoint.exp: check modified BP count >>>>      gdb_expect_list pattern: /Watchpoint Hit: ./ >>>>      continue >>>>      Continuing. >>>>      Watchpoint Hit: 1 >>>>      gdb_expect_list pattern: /[ >>>>      ]+Watchpoint . deleted because the program has left the block in/ >>>>      FAIL: gdb.python/py-watchpoint.exp: run until program stops >>>> (pattern 2) (timeout) >>>>      gdb_expect_list pattern: /[ >>>>      ]+which its expression is valid./ >>>>      gdb_expect_list pattern: /Watchpoint Hit: ./ >>>>      gdb_expect_list pattern: /[ >>>>      ]+012\[Inferior 1 \(process .*\) exited normally\]/ >>>>      gdb_expect_list pattern: // >>>>      python print(len(gdb.breakpoints())) >>>>      Watchpoint Hit: 2 >>>>      Watchpoint Hit: 3 >>>>      Watchpoint Hit: 4 >>>> >>>>      Watchpoint 2 deleted because the program has left the block in >>>>      which its expression is valid. >>>>      Watchpoint Hit: 5 >>>>      012[Inferior 1 (process 2381681) exited normally] >>>>      (gdb) FAIL: gdb.python/py-watchpoint.exp: check BP count (the >>>> program exited) >>>> >>>>> Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=29603 >>>>> --- >>>>>   gdb/breakpoint.c                           |  2 + >>>>>   gdb/testsuite/gdb.python/py-watchpoint.c   | 27 ++++++++++++ >>>>>   gdb/testsuite/gdb.python/py-watchpoint.exp | 48 >>>>> ++++++++++++++++++++++ >>>>>   gdb/testsuite/gdb.python/py-watchpoint.py  | 30 ++++++++++++++ >>>>>   4 files changed, 107 insertions(+) >>>>>   create mode 100644 gdb/testsuite/gdb.python/py-watchpoint.c >>>>>   create mode 100644 gdb/testsuite/gdb.python/py-watchpoint.exp >>>>>   create mode 100644 gdb/testsuite/gdb.python/py-watchpoint.py >>>>> >>>>> diff --git a/gdb/breakpoint.c b/gdb/breakpoint.c >>>>> index bff3bac7d1a..15f4ae2131c 100644 >>>>> --- a/gdb/breakpoint.c >>>>> +++ b/gdb/breakpoint.c >>>>> @@ -5340,6 +5340,8 @@ bpstat_check_breakpoint_conditions (bpstat >>>>> *bs, thread_info *thread) >>>>>     /* Evaluate extension language breakpoints that have a "stop" >>>>> method >>>>>        implemented.  */ >>>>>     bs->stop = breakpoint_ext_lang_cond_says_stop (b); >>>>> +  if (b->disposition == disp_del_at_next_stop) >>>>> +    disable_breakpoint(b); >>>> Is there a reason to do it here in particular, and not, let's say as >>>> soon as we change the disposition to disp_del_at_next_stop? >>>> >>>> Other question: shouldn't marking the watchpoint as >>>> disp_del_at_next_stop make it so it won't be inserted next time we >>>> resume?  After all should_be_inserted returns false for breakpoint >>>> locations that are disp_del_at_next_stop.  Perhaps it's because >>>> since we >>>> don't do a "normal" stop, breakpoint locations stay inserted, and >>>> there's nothing that pulls this location out of the target? Therefore, >>>> maybe a solution, to keep things coherent, would be to pull the >>>> breakpoint locations out when marking the breakpoint as >>>> disp_del_at_next_stop?  This way, we would respect the invariant >>>> that a >>>> breakpoint disp_del_at_next_stop can't be inserted (I don't know if >>>> this >>>> is really what is happening though, it's just speculation). >>>> >>>> And, in a broader scope, why wait until the next normal stop to delete >>>> the watchpoint?  A "normal" stop being a stop that we present to the >>>> user (the normal_stop function).  We currently call >>>> breakpoint_auto_delete in normal_stop, which is why we don't reach it >>>> when your Breakpoint.stop method returns False.  At the end of, let's >>>> say, handle_signal_stop, could we go over the bpstat chain and delete >>>> any breakpoint we've just hit that is marked for deletion? >>>> >>>> Simon >>>> >>