Decompilation

[Doug Evans <dje@transmeta.com>]

J. Grant writes:
 > I am currently working on some decompilation methods/ideas. I have been 
 > looking at the suitability of implementing using the GNU tools as a 
 > base. I realise this is a very complex process, so would like to ask 
 > peoples opinions before diving in and coding in all the wrong places.
 > 
 > I would like to achive something similar to the way that gcc is the 
 > front end for compiling. For each of the stages below I would welcome 
 > sugested  areas of binutils/GCC that I should focus my work on. I have 
 > been modifying objdump to produce the intermediate code.  Clearly a lot 
 > of new code needs to be written to complete this work. If anyone has 
 > sugestions for the direction I should take this is welcome.
 > 
 > Stage 1: Front end
 > Input machine code binary
 > Disassemble
 > Abstract intermediate code generation
 > Intermediate code output

What if you used cgen for stage 1?
I've always wanted to add the rtl to the opcodes files of cgen (*1),
but haven't had a reason or impetus to.
With that (and some suitable cover/utility fns) I believe you could easily
go from binary to intermediate code (*2).  Only for the targets that cgen
supports of course.

(*1): At some point I've been expecting binutils to want to boot cgen
out of libopcodes.  I dunno.  But I've always wanted to create libcgen too.
There's a lot more ISA utilities that can be provided with cgen and should
be made available in the form of a library, but whether they belong in
libopcodes and shipped with binutils is certainly debatable.

(*2): pedantic: insert all the usual caveats of determining what's code
and what's data.

 > Stage 2: Universal decompilation machine (UDM)
 > CFG generation
 > CFG analysis
 > Data Format analysis
 > 
 > Stage 3: Backend HLL target
 > HLL constructs identified
 > HLL output