* Insight + ARM-9 + BDI2000: Hang on exec @ 2004-04-19 12:55 Toralf Lund 2004-04-19 16:52 ` Keith Seitz 0 siblings, 1 reply; 8+ messages in thread From: Toralf Lund @ 2004-04-19 12:55 UTC (permalink / raw) To: insight I'm using insight built for ARM along with the BDI2000 (BDIGDB) debug "POD" to debug a custom interface card based on the MC9328 (ARM-9 core) processor. Now, I can download the code, access memory, display registers etc. just fine. I was even able to start the application, stop at breakpoints, steptrace etc. for a while, but now when I hit "run", insight will hang completely. The window is still displayed, but none of the buttons or menus, or the sliders, respond to input, and the "run" icon doesn't even change to "stop". The only way out is to kill the app. gdb from the same build appears to work fine. As is often the case when those things happen, I'm unable to tell what has changed since everything worked. I've tried removing .gdbtkinit & reconfigured target settings to make sure the setup was clean, but this didn't help. Any ideas what may be wrong? Insight version is 6.0. Host OS is Red Hat Linux 9 - Toralf ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Insight + ARM-9 + BDI2000: Hang on exec 2004-04-19 12:55 Insight + ARM-9 + BDI2000: Hang on exec Toralf Lund @ 2004-04-19 16:52 ` Keith Seitz 2004-04-20 8:12 ` Toralf Lund 0 siblings, 1 reply; 8+ messages in thread From: Keith Seitz @ 2004-04-19 16:52 UTC (permalink / raw) To: Toralf Lund; +Cc: insight On Mon, 2004-04-19 at 05:55, Toralf Lund wrote: > I'm using insight built for ARM along with the BDI2000 (BDIGDB) debug > "POD" to debug a custom interface card based on the MC9328 (ARM-9 core) > processor. First things first: does it run under console gdb? ("insight -nw" or "gdb")? There are numerous reasons for failures of this kind. Typically, it is an error in the target code in gdb. Testing command-line gdb is the first step to ascertaining where the problem lies. Also, could you send a more complete, step-by-step description of what you did? From your post, I'm not entirely sure how you got it to hang. Keith ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Insight + ARM-9 + BDI2000: Hang on exec 2004-04-19 16:52 ` Keith Seitz @ 2004-04-20 8:12 ` Toralf Lund 2004-04-20 16:14 ` Keith Seitz 0 siblings, 1 reply; 8+ messages in thread From: Toralf Lund @ 2004-04-20 8:12 UTC (permalink / raw) To: insight Keith Seitz wrote: >On Mon, 2004-04-19 at 05:55, Toralf Lund wrote: > > >>I'm using insight built for ARM along with the BDI2000 (BDIGDB) debug >>"POD" to debug a custom interface card based on the MC9328 (ARM-9 core) >>processor. >> >> > >First things first: does it run under console gdb? ("insight -nw" or >"gdb")? > > Yes, it runs under gdb (I thought I said that). Well, actually, the code crashes after a while, but I can at least set some breakpoints, have it stop there, step the code etc. One problem, though: I sometimes get <host>:2001: Connection refused. for no apparent reason, when issuing a "target" command. If I try the same operation again, it usually works fine. Also, even insight was OK for a while, then something (related to the application - insight hasn't changed) happened that made it stop working - and I can't figure out what it is ;-/ >There are numerous reasons for failures of this kind. Typically, it is >an error in the target code in gdb. Testing command-line gdb is the >first step to ascertaining where the problem lies. > >Also, could you send a more complete, step-by-step description of what >you did? From your post, I'm not entirely sure how you got it to hang. > > I got it to hang simply by starting the code (which is why I didn't think it necessary to give any steps), i.e. 1. arm-coff-insight <application> 2. Run->Download 3. Run->Run After I wrote the original message, I discovered that there are other ways, like 1. arm-coff-insight <application> 2. Run->Download 3. File->Exit (But Exit does work if I do File->Disconnect first.) Furthermore, 1. arm-coff-insight <application> 2. Run->Download 3. Control->Next Asm Inst (since execution start with start() written in assembler. 4. Other "step" commands... Actually works, but if I try e.g. View->Registers, I get another hang, but of a slightly different type; this time I get a "stop" icon and "busy" cursor, but I'm not really allowed to stop the process. >Keith > > > ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Insight + ARM-9 + BDI2000: Hang on exec 2004-04-20 8:12 ` Toralf Lund @ 2004-04-20 16:14 ` Keith Seitz 2004-04-21 9:04 ` Toralf Lund 0 siblings, 1 reply; 8+ messages in thread From: Keith Seitz @ 2004-04-20 16:14 UTC (permalink / raw) To: Toralf Lund; +Cc: insight I think you're running into multiple problems... On Tue, 2004-04-20 at 01:12, Toralf Lund wrote: > Yes, it runs under gdb (I thought I said that). Well, actually, the code > crashes after a while, but I can at least set some breakpoints, have it > stop there, step the code etc. Ok, that's a first good step. > One problem, though: I sometimes get > <host>:2001: Connection refused. > for no apparent reason, when issuing a "target" command. If I try the > same operation again, it usually works fine. My guess would be that since you're reusing the same port on the target, you're hitting the 2MSL wait state. > Also, even insight was OK for a while, then something (related to the > application - insight hasn't changed) happened that made it stop working > - and I can't figure out what it is ;-/ :-( > I got it to hang simply by starting the code (which is why I didn't > think it necessary to give any steps), i.e. > > 1. arm-coff-insight <application> > 2. Run->Download > 3. Run->Run [Note: You can just use the Run button to start your application. The Target Settings dialog defines the default behavior of the run button for various targets. For Remote targets, it connects to the target, downloads and continues execution.] So, when you do this, the UI just "hangs"? Control buttons disabled (grayed-out)? If so, open a console window and type "tk gdb_target_has_execution". What's the return value? Also try "tk set ::gdb_running". > After I wrote the original message, I discovered that there are other > ways, like > > 1. arm-coff-insight <application> > 2. Run->Download > 3. File->Exit > > (But Exit does work if I do File->Disconnect first.) This is reminiscent of a long-standing gdb bug. But I could be wrong. The only way to know for sure is to step through the code and find out why it is hanging. Having the console window open might help, too. > Furthermore, > > 1. arm-coff-insight <application> > 2. Run->Download > 3. Control->Next Asm Inst (since execution start with start() written > in assembler. > 4. Other "step" commands... > > Actually works, but if I try e.g. View->Registers, I get another hang, > but of a slightly different type; this time I get a "stop" icon and > "busy" cursor, but I'm not really allowed to stop the process. I think that there is something wrong with register display in that version. Gdb has changed a whole lot since I last did any active work on Insight -- which means that no one else has active worked on it. As I recall, there have been numerous changes to register and memory handling, but let's deal with one issue at a time. Keith ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Insight + ARM-9 + BDI2000: Hang on exec 2004-04-20 16:14 ` Keith Seitz @ 2004-04-21 9:04 ` Toralf Lund 2004-04-21 9:45 ` Toralf Lund 2004-04-21 16:33 ` Keith Seitz 0 siblings, 2 replies; 8+ messages in thread From: Toralf Lund @ 2004-04-21 9:04 UTC (permalink / raw) To: Keith Seitz; +Cc: insight Keith Seitz wrote: >I think you're running into multiple problems... > >On Tue, 2004-04-20 at 01:12, Toralf Lund wrote: > > >>Yes, it runs under gdb (I thought I said that). Well, actually, the code >>crashes after a while, but I can at least set some breakpoints, have it >>stop there, step the code etc. >> >> > >Ok, that's a first good step. > > > >>One problem, though: I sometimes get >><host>:2001: Connection refused. >>for no apparent reason, when issuing a "target" command. If I try the >>same operation again, it usually works fine. >> >> > >My guess would be that since you're reusing the same port on the target, >you're hitting the 2MSL wait state. > > Probably something like that. According to Abatron (the company that makes the BDI2000) support, you can't expect re-connect to work due to timing issues. (They blame gdb for this, not their own unit...) > > >>Also, even insight was OK for a while, then something (related to the >>application - insight hasn't changed) happened that made it stop working >>- and I can't figure out what it is ;-/ >> >> > >:-( > > > >>I got it to hang simply by starting the code (which is why I didn't >>think it necessary to give any steps), i.e. >> >> 1. arm-coff-insight <application> >> 2. Run->Download >> 3. Run->Run >> >> > >[Note: You can just use the Run button to start your application. The >Target Settings dialog defines the default behavior of the run button >for various targets. For Remote targets, it connects to the target, >downloads and continues execution.] > > Yes. I figured that out eventually. If I use Remote and select "run" without download first, the hang does not occur directly. I'll get it a a later stage, though. >So, when you do this, the UI just "hangs"? Control buttons disabled >(grayed-out)? > No. Nothing changes; the UI just stops responding to input. I can't select icons, open menus, scroll the source view, or even close the window. > If so, open a console window and type "tk >gdb_target_has_execution". What's the return value? Also try "tk set >::gdb_running". > > No can do. I can't open the console after the hang occurs, and if I have one open before it does, that stops responding to input, too >>After I wrote the original message, I discovered that there are other >>ways, like >> >> 1. arm-coff-insight <application> >> 2. Run->Download >> 3. File->Exit >> >>(But Exit does work if I do File->Disconnect first.) >> >> > >This is reminiscent of a long-standing gdb bug. But I could be wrong. >The only way to know for sure is to step through the code and find out >why it is hanging. Having the console window open might help, too. > > The console window doesn't tell my anything, either... >>Furthermore, >> >> 1. arm-coff-insight <application> >> 2. Run->Download >> 3. Control->Next Asm Inst (since execution start with start() written >> in assembler. >> 4. Other "step" commands... >> >>Actually works, but if I try e.g. View->Registers, I get another hang, >>but of a slightly different type; this time I get a "stop" icon and >>"busy" cursor, but I'm not really allowed to stop the process. >> >> > >I think that there is something wrong with register display in that >version. Gdb has changed a whole lot since I last did any active work on >Insight -- which means that no one else has active worked on it. As I >recall, there have been numerous changes to register and memory >handling, but let's deal with one issue at a time. > > OK >Keith > > > > ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Insight + ARM-9 + BDI2000: Hang on exec 2004-04-21 9:04 ` Toralf Lund @ 2004-04-21 9:45 ` Toralf Lund 2004-04-21 16:33 ` Keith Seitz 1 sibling, 0 replies; 8+ messages in thread From: Toralf Lund @ 2004-04-21 9:45 UTC (permalink / raw) To: Keith Seitz; +Cc: insight Toralf Lund wrote: > Keith Seitz wrote: > >> I think you're running into multiple problems... >> >> On Tue, 2004-04-20 at 01:12, Toralf Lund wrote: > Hello again, I just found an even simpler way to get reproduce the hang. Simply 1. Start insight 2. Run->Connect to target is enough. I also get the hang if I disconnect first. ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Insight + ARM-9 + BDI2000: Hang on exec 2004-04-21 9:04 ` Toralf Lund 2004-04-21 9:45 ` Toralf Lund @ 2004-04-21 16:33 ` Keith Seitz 2004-04-22 10:07 ` Toralf Lund 1 sibling, 1 reply; 8+ messages in thread From: Keith Seitz @ 2004-04-21 16:33 UTC (permalink / raw) To: Toralf Lund; +Cc: insight On Wed, 2004-04-21 at 02:04, Toralf Lund wrote: > >So, when you do this, the UI just "hangs"? Control buttons disabled > >(grayed-out)? > > > No. Nothing changes; the UI just stops responding to input. I can't > select icons, open menus, scroll the source view, or even close the window. Ok, this screams of a malignant back-end or similar problem. If the UI doesn't even do screen updates, then I can pretty much guarantee that the back-end code for your target is locking the UI out (blocked in a system call). This happens rather more frequently than I wished. I get reports of this every few months. [However, you're using a standard "remote" target, and this should not happen, so something else must be amiss.] The problem is that gdb has several event loops. Insight further burdens the debugging model by adding yet another one. Let me quickly clarify. The two main event loops in gdb are the target event loop (where the target code is waiting for debug messages from the target) and the UI (or mi or cli) event loop, where the user interface is waiting for input from the user. Unfortunately, gdb and insight don't share the same event loops for these things, and they probably never will until all target event loops go async. In the meantime, we came up with a hack: have the target code periodically call out to the UI to allow it to run it's event loop. This is the evil "ui_loop_hook" that is in gdb. For example, one of the little nasty things that is done (for which I am regrettably responsible many years ago) involves breaking up serial and network read requests into small chunks. Instead of waiting N seconds for a reply from the target, gdb will wait 1 second, N times. After every second, it calls the ui_loop_hook to update the UI (in this case, only insight needs to register a hook callback). So, now for your problems, the question is: why isn't the ui_loop_hook being called? Better yet, why isn't the connect command completing and returning control to gdb-proper? To answer these questions may not be too difficult with a little diligence on your part. So, here are some questions/comments: 1) Let's see if we cannot get gdb/Insight to timeout after the connect and return control to the UI. Before connecting to the target, can you open a console and enter the command "set remote timeout 1". Now try connecting. Does it ever return control to you, i.e., does the UI update at all? 2) I believe you said you were using a remote tcp target, right? Hmm. Looking over src/gdb/ser-tcp.c, I see calls to ui_loop_hook. Well, to check this, we have no choice: you're going to have to load insight into gdb and set a breakpoint on "x_event" in gdbtk-hooks.c. When you attach to the target, does it ever stop at the breakpoint and run the hook? 3) The easy way: Run insight on gdb. Connect to your target (where the hang occurs), and interrupt insight and get a backtrace of where it stopped. Demonstrate anything? Blocked in a system call? 4) The hard way: If all else fails, you're going to have to set a break in remote_open in remote.c and follow it through to the protocol level (for tcp, that's in ser-tcp.c). Does it do the N 1-second timeouts, calling ui loop hook? Or is it blocked? 5) The really hard way: Use strace to find out what's going on. 6) Can you remind me what version of gdb/insight you're using? (Show complete output of "insight -v".) Keith ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Insight + ARM-9 + BDI2000: Hang on exec 2004-04-21 16:33 ` Keith Seitz @ 2004-04-22 10:07 ` Toralf Lund 0 siblings, 0 replies; 8+ messages in thread From: Toralf Lund @ 2004-04-22 10:07 UTC (permalink / raw) To: Keith Seitz; +Cc: insight Keith Seitz wrote: >On Wed, 2004-04-21 at 02:04, Toralf Lund wrote: > > >>>So, when you do this, the UI just "hangs"? Control buttons disabled >>>(grayed-out)? >>> >>> >>> >>No. Nothing changes; the UI just stops responding to input. I can't >>select icons, open menus, scroll the source view, or even close the window. >> >> I suddenly discovered what's been going on now - just as I was trying to debug the debugger ;-) It was something quite different from what we expected, and I should have noticed it earlier, but I just didn't consider the possibility, perhaps because it was too simple... Anyhow, the situation was simply that at the point of the "hang", insight would display a modal dialog window of some kind. On connect it would contain the message "Successfully connected", on "Run" it asked whether I wanted to restart the program, while View->Registers lead to an "internal error" popup (I'll investigate that later.) The problem was that I never could see these popups - not because I'm blind or something, but because they would always be opened in the "minimized" (iconic) state - so instead of the popup, I just got a tasklist entry that I never noticed. (Hmmm... Maybe I should run the "window list" applet after all, rather than just using the task pulldown of the menu panel. I otherwise find it quite useless when I have more than about 3 windows open, and I always do...) I still haven't figured out why this happened, and if it was insight itself or the window manager that caused it, but everything is OK now, after I played around a bit with minimize and unminize (or whatever you call it) of the popups and the main window. Sorry for the trouble... - Toralf > >Ok, this screams of a malignant back-end or similar problem. If the UI >doesn't even do screen updates, then I can pretty much guarantee that >the back-end code for your target is locking the UI out (blocked in a >system call). This happens rather more frequently than I wished. I get >reports of this every few months. [However, you're using a standard >"remote" target, and this should not happen, so something else must be >amiss.] > >The problem is that gdb has several event loops. Insight further burdens >the debugging model by adding yet another one. Let me quickly clarify. > >The two main event loops in gdb are the target event loop (where the >target code is waiting for debug messages from the target) and the UI >(or mi or cli) event loop, where the user interface is waiting for input >from the user. > >Unfortunately, gdb and insight don't share the same event loops for >these things, and they probably never will until all target event loops >go async. In the meantime, we came up with a hack: have the target code >periodically call out to the UI to allow it to run it's event loop. This >is the evil "ui_loop_hook" that is in gdb. > >For example, one of the little nasty things that is done (for which I am >regrettably responsible many years ago) involves breaking up serial and >network read requests into small chunks. Instead of waiting N seconds >for a reply from the target, gdb will wait 1 second, N times. After >every second, it calls the ui_loop_hook to update the UI (in this case, >only insight needs to register a hook callback). > >So, now for your problems, the question is: why isn't the ui_loop_hook >being called? Better yet, why isn't the connect command completing and >returning control to gdb-proper? > >To answer these questions may not be too difficult with a little >diligence on your part. > >So, here are some questions/comments: >1) Let's see if we cannot get gdb/Insight to timeout after the connect >and return control to the UI. Before connecting to the target, can you >open a console and enter the command "set remote timeout 1". Now try >connecting. Does it ever return control to you, i.e., does the UI update >at all? > >2) I believe you said you were using a remote tcp target, right? Hmm. >Looking over src/gdb/ser-tcp.c, I see calls to ui_loop_hook. Well, to >check this, we have no choice: you're going to have to load insight into >gdb and set a breakpoint on "x_event" in gdbtk-hooks.c. When you attach >to the target, does it ever stop at the breakpoint and run the hook? > >3) The easy way: Run insight on gdb. Connect to your target (where the >hang occurs), and interrupt insight and get a backtrace of where it >stopped. Demonstrate anything? Blocked in a system call? > >4) The hard way: If all else fails, you're going to have to set a break >in remote_open in remote.c and follow it through to the protocol level >(for tcp, that's in ser-tcp.c). Does it do the N 1-second timeouts, >calling ui loop hook? Or is it blocked? > >5) The really hard way: Use strace to find out what's going on. > >6) Can you remind me what version of gdb/insight you're using? (Show >complete output of "insight -v".) > >Keith > > > -- Toralf Lund <toralf@procaptura.com> +47 66 85 51 22 ProCaptura AS +47 66 85 51 00 (switchboard) http://www.procaptura.com/~toralf +47 66 85 51 01 (fax) ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2004-04-22 10:07 UTC | newest] Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2004-04-19 12:55 Insight + ARM-9 + BDI2000: Hang on exec Toralf Lund 2004-04-19 16:52 ` Keith Seitz 2004-04-20 8:12 ` Toralf Lund 2004-04-20 16:14 ` Keith Seitz 2004-04-21 9:04 ` Toralf Lund 2004-04-21 9:45 ` Toralf Lund 2004-04-21 16:33 ` Keith Seitz 2004-04-22 10:07 ` Toralf Lund
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).