Looking to contribute OMF support

Looking to contribute OMF support

[Bernd Jendrissek]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi all

It seems my employer should be willing to assign copyright for the OMF
support I've added using (in part) company time and equipment, to the
FSF.  I'll need one of those pre-paperwork-preparation forms that I've
seen around here regularly; would somebody please send me one of those?
Private email or here is fine.

Since this will be certainly my, and likely the company's, first such
submission, I'm a little daunted by the length (time) of the procedure.
I need to fill in a form for requesting the real form, right?  Is it
only the second set of forms which have to cross the ocean by
snail-mail?

As for the OMF support itself:

I wrote the object file format support against .OBJ files produced by
TopSpeed C 3.10, which AFAICT is supposed to be compatible with the MS
(and Borland too, probably) tools of the era, except that it defines a
few comment records for specifying dependencies, and defines multiple
sections with the same name (not sure how kosher that is to other
OMF-aware tools).

I didn't bother trying to write complete support, and I can't even
guarantee that I interpret records correctly either; this is one of
those "it works Well Enough For Me for what I want to do with it" ports.

Notably, what works WEFM is:
 - section dumping and disassembly and of course 'size' which was my
   scratch to my itch of wanting to know which modules needed the most
   memory
 - listing of symbols and relocations (OMF relocs are hell, expect bugs)
 - surprisingly to me, ld was able correctly to locate a tiny dummy
   module that contains "offset16" relocs.

NOT working or untested:
 - linking of more complex modules
 - objcopy
 - assembler output

I hope that's good enough for an otherwise nonexistent port?  (I'd
prefer to show-and-tell once the paperwork is done.)

- -- 
You're proposing to build a box with a light on top of it.  The light is
supposed to go off when you carry the box into a room that has a Unicorn
in it.  How do you show that it works?
         - Dr. Gene "spaf" Spafford, at Dr. Wenliang Du's qualifing exam
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2 (GNU/Linux)
Comment: Please fetch my new key 804177F8 from hkp://wwwkeys.eu.pgp.net/

iD8DBQFEEYZfwyMv24BBd/gRAnE7AJ48CJ7ceIk+aDn9kIY8FEx40wD7FACdG5hD
dWN1uFxt4Lx42iTcJagxtTc=
=Owyj
-----END PGP SIGNATURE-----

Re: Looking to contribute OMF support

[Nick Clifton]

[-- Attachment #1: Type: text/plain, Size: 1352 bytes --]

Hi Bernd,

> It seems my employer should be willing to assign copyright for the OMF
> support I've added using (in part) company time and equipment, to the
> FSF.  I'll need one of those pre-paperwork-preparation forms that I've
> seen around here regularly; would somebody please send me one of those?

Attached.

> I need to fill in a form for requesting the real form, right?

Right.

> Is it only the second set of forms which have to cross the ocean by
> snail-mail?

I think that the FSF might even use faxes....

> NOT working or untested:
>  - linking of more complex modules
>  - objcopy
>  - assembler output
> 
> I hope that's good enough for an otherwise nonexistent port? 

Yes.  Ideally however it would be nice if there was sufficient code to 
allow the port to be tested.  Ie supporting assembler output would 
really be a good idea.  If this is something that you are planning to 
add in the future then this is fine, but if not, then how can we, the 
binutils maintainers, tell if your code is still working ?  We test the 
code all the time and we try to make sure that it all works.  We also 
have a process whereby we retire unsupported and no-longer-working 
ports.  If we cannot test your port, how can we make sure that future 
changes to the generic parts of the binutils sources do not break the 
OMF port ?

Cheers
   Nick



[-- Attachment #2: future --]
[-- Type: text/plain, Size: 1005 bytes --]

---------------------------------------------------------------------------

request-assign.future:

Please email the following information to fsf-records@gnu.org, and we
will send you the assignment form for your past and future changes.
Please use your full name as the subject line of the message.


[What is the name of the program or package you're contributing to?]


[Did you copy any files or text written by someone else in these changes?
Even if that material is free software, we need to know about it.]


[Do you have an employer who might have a basis to claim to own
your changes?  Do you attend a school which might make such a claim?]


[For the copyright registration, what country are you a citizen of?]


[What year were you born?]


[Please write your email address here.]


[Please write your snail address here.]





[Which files have you changed so far, and which new files have you written
so far?]





---------------------------------------------------------------------------

Re: Looking to contribute OMF support

[Bernd Jendrissek]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Tue, Mar 14, 2006 at 11:58:20AM +0000, Nick Clifton wrote:
> Attached. / Right. / I think that the FSF might even use faxes....

Great, thanks for that!

> >NOT working or untested:
> > - linking of more complex modules
> > - objcopy
> > - assembler output
> >
> >I hope that's good enough for an otherwise nonexistent port? 
> 
> Yes.  Ideally however it would be nice if there was sufficient code to
> allow the port to be tested.  Ie supporting assembler output would
> really be a good idea.

Seems to me BFD_JUMP_TABLE_WRITE has only one substantive method
(_bfd_set_section_contents) but for anything interesting I'll have to
figure out how to write relocs to the output file.  Do I do that from
within _bfd_set_section_contents too?

> If this is something that you are planning to add in the future then
> this is fine, but if not, then how can we, the binutils maintainers,
> tell if your code is still working ?  We test the code all the time
> and we try to make sure that it all works.  We also have a process
> whereby we retire unsupported and no-longer-working ports.  If we
> cannot test your port, how can we make sure that future changes to the
> generic parts of the binutils sources do not break the OMF port ?

I won't make any promises about contributing zillions of patches but as
long as you don't uncover rocket science bugs I may have written I'm
sure you can gently bully me into fixing stuff when it breaks. :)

Are there any things specific to BFD that I can do to make my code more
or less resilient to bit rot?  One thing I'm worried about that I
thought was okay, is that I bfd_alloc gobs of memory everywhere without
freeing them, relying instead on bfd_close to release the memory when
objdump et al are done with the BFD.  But then I saw a recent message
that seemed to say that that constituted a memory leak...

Another general question I have is about "segment groups" in the DOS
world.  OMF objects already specify a mapping from input sections
("segments") to output sections ("groups") and I don't have the foggiest
where to begin teaching the linker to understand these GRPDEFs if I
wanted to.  Any ideas?  As a last resort one could probably generate a
linker script as a separate step between assembly and linking, where a
script would fish out the GRPDEFs from objdump -p and rewrite them as,
say, DGROUP : { foo.obj(BSS.1) foo.obj(DATA.1) } etc.

- -- 
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing on usenet?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2 (GNU/Linux)
Comment: Please fetch my new key 804177F8 from hkp://wwwkeys.eu.pgp.net/

iD8DBQFEFsTWwyMv24BBd/gRAkkmAJ9IHaPBzqxYSN0yj742DGiQDeX0LQCgjHJs
IhkGXFVm5vXysUKVSeRN++k=
=AOe/
-----END PGP SIGNATURE-----

Re: Looking to contribute OMF support

[Ian Lance Taylor]

Bernd Jendrissek <berndj@prism.co.za> writes:

> Seems to me BFD_JUMP_TABLE_WRITE has only one substantive method
> (_bfd_set_section_contents) but for anything interesting I'll have to
> figure out how to write relocs to the output file.  Do I do that from
> within _bfd_set_section_contents too?

The object file creator (e.g., gas) will call bfd_set_reloc.  The
write_contents code is responsible for getting those relocs into the
object file.

Ian

Re: Looking to contribute OMF support

[Nick Clifton]

Hi Bernd,

> Are there any things specific to BFD that I can do to make my code more
> or less resilient to bit rot?

Sorry - not really.

> One thing I'm worried about that I
> thought was okay, is that I bfd_alloc gobs of memory everywhere without
> freeing them, relying instead on bfd_close to release the memory when
> objdump et al are done with the BFD.  But then I saw a recent message
> that seemed to say that that constituted a memory leak...

Well it does, but it is not a serious memory leak.  As a general 
principle it is always best to explicitly free any memory that you 
allocate, as soon as you are sure that you no longer need it.  But 
sometimes knowing when the memory is no longer needed is difficult to 
decide.  The only real problem with memory leaks is if they prevent 
programs from running, and this is unlikely for most situations you will 
encounter whilst programming the binutils.

> Another general question I have is about "segment groups" in the DOS
> world.  OMF objects already specify a mapping from input sections
> ("segments") to output sections ("groups") and I don't have the foggiest
> where to begin teaching the linker to understand these GRPDEFs if I
> wanted to.  Any ideas?  As a last resort one could probably generate a
> linker script as a separate step between assembly and linking, where a
> script would fish out the GRPDEFs from objdump -p and rewrite them as,
> say, DGROUP : { foo.obj(BSS.1) foo.obj(DATA.1) } etc.

So these GRPDEFs are present in object files, but are only intended to 
be used when creating executables ?

It sounds to me like you might want to use some of the linker's 
emulation routines.  Have a look at ld/ldemul.h and the function 
create_output_section_statements().  You might be able to define this 
function for your OMF backend and have it scan for GRPDEFs in the input 
bfd's and then synthesise DGROUP definitions as you showed above.

Cheers
   Nick

Re: Looking to contribute OMF support

[Bernd Jendrissek]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Thu, Mar 16, 2006 at 08:53:32AM +0000, Nick Clifton wrote:
> >One thing I'm worried about that I thought was okay, is that I
> >bfd_alloc gobs of memory everywhere without freeing them, relying
> >instead on bfd_close to release the memory when objdump et al are
> >done with the BFD.  But then I saw a recent message that seemed to
> >say that that constituted a memory leak...
> 
> Well it does, but it is not a serious memory leak.  As a general
> principle it is always best to explicitly free any memory that you
> allocate, as soon as you are sure that you no longer need it.  But
> sometimes knowing when the memory is no longer needed is difficult to
> decide.  The only real problem with memory leaks is if they prevent
> programs from running, and this is unlikely for most situations you
> will encounter whilst programming the binutils.

In that case I won't feel too guilty about it.  I would "no longer need"
the string tables and section data when I "no longer need" everything
else, i.e. at program exit.

> >Another general question I have is about "segment groups" in the DOS
> >world.  OMF objects already specify a mapping from input sections
> >("segments") to output sections ("groups") and I don't have the
> >foggiest where to begin teaching the linker to understand these
> >GRPDEFs if I wanted to.  Any ideas?  As a last resort one could
> >probably generate a linker script as a separate step between assembly
> >and linking, where a script would fish out the GRPDEFs from objdump
> >-p and rewrite them as, say, DGROUP : { foo.obj(BSS.1)
> >foo.obj(DATA.1) } etc.
> 
> So these GRPDEFs are present in object files, but are only intended to
> be used when creating executables ?

Something like that.  In that crazy segment:offset world you may want to
define your stack and default data segments as separate sections, in
order to have separate name spaces for the symbols therein, but you want
16-bit pointers to be able to point to any object in either of them, so
they need to be grouped into the same "group".  I think what ELF calls
"sections" the DOS world would call "segment", and ELF "segments" are
"groups".

> It sounds to me like you might want to use some of the linker's
> emulation routines.  Have a look at ld/ldemul.h and the function
> create_output_section_statements().  You might be able to define this
> function for your OMF backend and have it scan for GRPDEFs in the
> input bfd's and then synthesise DGROUP definitions as you showed
> above.

Thanks for the pointer, that looks just about exactly like what I want!

- -- 
/"\  ASCII Ribbon Campaign - against html email
\ /                        - against microsoft office attachments
 X                         - against text above fullquote below
/ \                        - against lines longer than 79 characters
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2 (GNU/Linux)
Comment: Please fetch my new key 804177F8 from hkp://wwwkeys.eu.pgp.net/

iD8DBQFEGUhzwyMv24BBd/gRAn13AJ487YZieuA4M7fJtLuS2evdbCzr+gCfaQ6W
N9JFOBGPpwzMQrGsvxB/il0=
=RZQY
-----END PGP SIGNATURE-----

Re: Looking to contribute OMF support

[Ben Elliston]

> I won't make any promises about contributing zillions of patches but
> as long as you don't uncover rocket science bugs I may have written
> I'm sure you can gently bully me into fixing stuff when it
> breaks. :)

The problem is knowing when stuff has broken ..

> Are there any things specific to BFD that I can do to make my code
> more or less resilient to bit rot?  One thing I'm worried about that
> I thought was okay, is that I bfd_alloc gobs of memory everywhere
> without freeing them, relying instead on bfd_close to release the
> memory when objdump et al are done with the BFD.  But then I saw a
> recent message that seemed to say that that constituted a memory
> leak...

I think the best thing that you could do to ensure that your new code
doesn't break is to contribute test cases that can be used to catch
regressions.  The more tests, the merrier (within limits!)

Cheers, Ben

Re: Looking to contribute OMF support

[Greatwolf]

Bernd Jendrissek <berndj@prism.co.za> wrote in
news:20060310140046.GA11152@prism.co.za: 

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Hi all
> 
> It seems my employer should be willing to assign copyright for the OMF
> support I've added using (in part) company time and equipment, to the
> FSF.  I'll need one of those pre-paperwork-preparation forms that I've
> seen around here regularly; would somebody please send me one of
> those? Private email or here is fine.
> 
> Since this will be certainly my, and likely the company's, first such
> submission, I'm a little daunted by the length (time) of the
> procedure. I need to fill in a form for requesting the real form,
> right?  Is it only the second set of forms which have to cross the
> ocean by snail-mail?
> 
> As for the OMF support itself:
> 
> I wrote the object file format support against .OBJ files produced by
> TopSpeed C 3.10, which AFAICT is supposed to be compatible with the MS
> (and Borland too, probably) tools of the era, except that it defines a
> few comment records for specifying dependencies, and defines multiple
> sections with the same name (not sure how kosher that is to other
> OMF-aware tools).
> 
> I didn't bother trying to write complete support, and I can't even
> guarantee that I interpret records correctly either; this is one of
> those "it works Well Enough For Me for what I want to do with it"
> ports. 
> 
> Notably, what works WEFM is:
>  - section dumping and disassembly and of course 'size' which was my
>    scratch to my itch of wanting to know which modules needed the most
>    memory
>  - listing of symbols and relocations (OMF relocs are hell, expect
>  bugs) - surprisingly to me, ld was able correctly to locate a tiny
>  dummy 
>    module that contains "offset16" relocs.
> 
> NOT working or untested:
>  - linking of more complex modules
>  - objcopy
>  - assembler output
> 
> I hope that's good enough for an otherwise nonexistent port?  (I'd
> prefer to show-and-tell once the paperwork is done.)
> 
> - -- 
> You're proposing to build a box with a light on top of it.  The light
> is supposed to go off when you carry the box into a room that has a
> Unicorn in it.  How do you show that it works?
>          - Dr. Gene "spaf" Spafford, at Dr. Wenliang Du's qualifing
>          exam 
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.2 (GNU/Linux)
> Comment: Please fetch my new key 804177F8 from
> hkp://wwwkeys.eu.pgp.net/ 
> 
> iD8DBQFEEYZfwyMv24BBd/gRAnE7AJ48CJ7ceIk+aDn9kIY8FEx40wD7FACdG5hD
> dWN1uFxt4Lx42iTcJagxtTc=
> =Owyj
> -----END PGP SIGNATURE-----
> 

This thread is kind of old but I was wondering whatever happened to this
endeavor of hacking OMF support into libbfd. Did it ever come to
fruitation? 

I'm looking for borland/delphi source debugging support with gdb and
came across this thread. If OMF support is fully implemented into bfd,
would anything more have to be done to get source level debugging to
work in gdb for executables/dll's compiled with something other than
gcc? Specifically compiles that emit OMF symbolic debug info like
borland c++ build. I beleive Digital Mars Compiler emits OMF symbolic
info as well. 

Thanks

Re: Looking to contribute OMF support

[Nick Clifton]

Hi Greatwolf,

> This thread is kind of old but I was wondering whatever happened to this
> endeavor of hacking OMF support into libbfd. Did it ever come to
> fruitation? 

I am afraid not.  At least not as far as making it into the official 
binutils sources.

Cheers
   Nick

Re: Looking to contribute OMF support

[Bernd Jendrissek]

On Mon, Dec 21, 2009 at 11:38 AM, Greatwolf <gmane.greatwolf@mamber.net> wrote:
> This thread is kind of old but I was wondering whatever happened to this
> endeavor of hacking OMF support into libbfd. Did it ever come to
> fruitation?
>
> I'm looking for borland/delphi source debugging support with gdb and
> came across this thread. If OMF support is fully implemented into bfd,
> would anything more have to be done to get source level debugging to
> work in gdb for executables/dll's compiled with something other than
> gcc? Specifically compiles that emit OMF symbolic debug info like
> borland c++ build. I beleive Digital Mars Compiler emits OMF symbolic
> info as well.

I'm busy sorting out the paperwork for implementation #2
(reimplemented it for reasons I'll omit here).  I'm quite a bit behind
the mainline repository; I'm not sure how much rework I'll have to do
to get it up to date.

It's fairly minimal support too, and it's only for 16-bit OMF
(although 32-bit objects seem to be merely an extension of the 16-bit
format).  Specifically, I did not even try to add support for
debugging info.  My OMF support only provides for disassembling OMF
files, and possibly linking them; that seemed far more valuable than
also adding support for OMF as an output format, since there are
already free assemblers such as NASM that can do that job.

OTOH, if you could document the debug info (or point me at a resource
that does so), I could at least know how much effort it would be to
add.  Feel like sending me some sample object files to work with?
Send them directly to me, so I don't lose them in a pile of 3000-odd
unread binutils messages!