Re: Is multiprocessor debugging multithreaded debugging?

[Steven Johnson <sjohnson@sakuraindustries.com>]

Ramana Radhakrishnan wrote:

>Hi ,
>
>As much as I agree with you there might be other use cases that are not
>possible with the setup you describe. One example is to be able to be
>able to single step 2 processors and let a 3rd continue merrily or
>single step in lockstep a group of processors. 
>  
>
I dont think "lock step" is achievable with Asynchronous 
multiprocessing.  Each processor has a different clock domain, they also 
do not have any "synchronizing" features in hardware.  The best you 
could approximate is "command 2 targets to Single step at once" which 
one actually single steps first would of course be unknown. And in fact 
if one processor is faster than another the processor that actually 
single stepped last, could be the first to complete its instruction.

In my case, if i needed to do this, i worked out which instruction 
should be executed first (for my particular test), went to that instance 
of GDB, single stepped.  Went to the next instance of GDB and then 
stepped it.  Given the asynchronous nature of what you describe i can 
imagine all sorts of race conditions which would mean you would get 
inconsistent debug results.  Things like, why did that happen, that bit 
shouldnt have been set yet type stuff.

>Providing relational breakpoints between multiple processors might be
>another nifty feature. 
>
>Lets say something like b <here> if val_in_pgm_in_proc2 is 0xcafebabe. 
>  
>
I also think to do this GDB would need to "simulate" the behavior.  It 
would have to put an unconditional break on <here>.  Every time the 
target breaks on <here> break the processor with the val_in_pgm_in_proc2 
variable, inspect it. restart the proc2 target (which might then 
actually change the value from 0xcafebabe, so you are not in the state 
you think you are) and then make a decision about whether to continue 
from the break at <here> or report it to the user.  Ive yet to see a 
JTAG of BDM type interface that allows "live" inspection of data, while 
the CPU is executing code.

I think all of these things are advanced User Interface concerns, and 
actually have little if anything to do with the fundamental GDB itself.  
Maybe what is really required, if anything, is a GDB front end, that can 
interface to multiple GDB back end instances (through MI), each one a 
potentially different processor architecture, and do these types of 
operations.  Tabbed pages for each CPU, simulated simultaneous step on n 
of m processors, cross processor break trigger conditioning.  Sounds 
like a bitching front end, but i dont think GDB itself should be made to 
do these things natively, as they are not actually native debug issues, 
but simulated "pseudo" operations.

Now if such a front end was GPL, could work on the majority of hosts 
that support GDB, and was assigned to the FSF it might (if the 
maintainers agree) even be able to live in the GDB CVS Tree (like the 
TUI front end), and be a standard example of an MI/GUI front end. (But 
now im dreaming).  Id shudder to think how unwieldy multi CPU debugging 
would be without multiple "screens" of information, you can switch 
between, so i really dont know how you would achieve anything really 
useful without a GUI in any event.

In fact what you describe is really no different to debugging two 
applications on the one PC, which interact, at the same time.  Again, 
something people do not infrequently, but they do it with multiple 
instances of the debugger running.

Maybe if Anupama can highlight what he thinks the deficiencies with the 
current arrangement are, i could see the problems with my view.

Now if you were talking about getting GDB to be able to read/set JTAG 
boundary scan information (on targets where that is applicable), and 
process BSDL files to show the states of all pins in a meaningful way I 
think that would be a useful project in this realm, but i think making 
GDB natively handle asynchronous multiprocessor debugging is a waste of 
time that could be better spent, as there is no problem to solve that i 
can see.

Steven

>My 2 bits on the topic. 
>
>cheers
>Ramana
>
>
>
>On Wed, 2005-09-28 at 21:16 +1100, Steven Johnson wrote:
>  
>
>>Anupama Chandwani wrote:
>>
>>    
>>
>>>In continuation with my prev mail..
>>>I want to extend gdb to debug homing ogenous multiprocessor system
>>> 
>>>
>>>(say multiple ARM or x86 processors on single chip) by remote
>>>debugging in a single session of gdb.
>>>
>>>What i want to know is are there enough applications being written on
>>>such multi processors? Also are there different executables being
>>>required to be debugged simultaneously? Coz this is what i want to
>>>extend in further.. Each processor running a different executable so
>>>the processors dont share memory & run with different images of code.
>>> 
>>>
>>>      
>>>
>>This is commonly called "Asynchronous" Multi Processing.
>>
>>    
>>
>>>An application of such debugger could be while building an OS but that
>>>wouldnt involve different executables.. So are there applications
>>>requiring to run different executables on each processor? Say for
>>>example a prog gives a certain bug on when there is certain other
>>>program running on the other processor or something similar to
>>>this....
>>> 
>>>
>>>      
>>>
>>Yes in the embedded world, there are many examples of Asynchronous Multi 
>>Processor designs.  They are by far the easiest multi processor design 
>>to implement.  I for example have worked on a board that had 3 MSP430's, 
>>each had a unique function, and they intercommunicated over a custom 
>>parallel bus to coordinate their activities.  Worked sweet, had high 
>>performance, and was really cheap.
>>
>>    
>>
>>>As far as i know this done by multiplexing the JTAG interface (for
>>>x86) &different sessions of gdb right now. Any other? And any flaws or
>>>inconvenience with present methods?
>>> 
>>>
>>>      
>>>
>>This is exactly how it is done, multiple sessions of GDB.  This, in my 
>>opinion is the right way to go.  Not all Asynchronous multi processor 
>>designs have homogeneous pprocessors (ie, you may have an MPC860 
>>handling comms, and a MIPS Chip doing some number crunching.  1 is a 
>>power PC, the other is a MIPS.  Both have different debug interfaces. 
>>
>>Now if you had a system say, where you had 3 MIPS Chips, hooked up on 
>>the same EJTAG interface, you would need to handle that with some nifty 
>>EJTAG code in your (pseudo) stub to ensure each device was uniquely 
>>addressed and they didnt interfere with one another, so that you could 
>>start up 3 GDB sessions to debug your 3 processors, but then it becomes 
>>a problem for the stub. 
>>
>>What im saying is I dont think a single instance of GDB needs to be 
>>complicated to try and debug multiple "tasks" simultaneously.  I dont 
>>have any problems with running GDB as many times as I want.  For example 
>>with the MSP430 example, I had (at various times) GDB running 5 times on 
>>the one PC.  One was debugging a local PC app that talked to my MSP430 
>>board.  3 were talking to the MSP430 board, the last was talking to yet 
>>another device (that had an MPC862 as its processor), I just ran each in 
>>a separate "Desktop" under KDE and then switched to the one i had to 
>>deal with at the time.  No problems, worked easily.
>>
>>Hope that gives you insight into one application of what you discussed.
>>Steven
>>    
>>