Re: gas 2.34 on MacOS

Re: gas 2.34 on MacOS

[Gunther Nikl]

Kevan Hashemi wrote:
> I would like to install the GNU Assembler version 2.34 on MacOS.
> I have version 1.38, but it's giving me the following error when I
> try to complie 64-bit code;
>
> "suffix or operands invalid for `movq'"

I am curious: Is the version "1.38" really correct? That would be a
really _ancient_ version from 1990! What does -version (or -V?) say?

Regards,
Gunther

Re: gas 2.34 on MacOS

[Kevan Hashemi]

Dear Nick,

> To start with I would suggest investigating
> the assembler.

I have been doing some investigating. The CLANG assembler on MacOS works fine for assembler produced by GCC or CLANG compilers, so long as we're not using GCC 3 or earlier. With GCC 3 or earlier, the -g option produces .stab directives that the CLANG assembler does not recognise. Assembler produced by GCC 4 and later causes no problems, because they don't use .stabs directives.

So, that's why nobody has ported gas or ld 2.34 to MacOSX: there appears to be no need. What we need on MacOSX is the other binutils, and they are all ported. I won't bore you with why I was messing around with GCC 3, but suffice to say I have worked around my need to use GCC 3, and my conclusion is that the existing Binutils port to MacOSX does everything it needs to do.

Best, Kevan

-- 
Kevan Hashemi, Electrical Engineer
Physics Department, Brandeis University
http://www.bndhep.net

Re: gas 2.34 on MacOS

[Kevan Hashemi]

Dear Alan,

> I doubt that is sufficient, sorry.  See
> https://www.gnu.org/licenses/why-assign.en.html

Ah, that is interesting. I think I understand, but in the case of the Binutils code, there is no author named in the sources that I can find.

> If you copy code, I think you need permission from all the authors of
> that code to assign copyright to the FSF.

There are no persons named in the source code. Every file begins with a declaration that copyright is held by the Free Software Foundation, and refers to the GPL. The COPYING file is an FSF copyright notice.

https://opensource.apple.com/source/cctools/cctools-900/as/

> The authors may have assigned copyright to the FSF
> already of course, I don't know.

They appear to have done so. But please correct me if I am wrong.

Best, Kevan

-- 
Kevan Hashemi, Electrical Engineer
Physics Department, Brandeis University
http://www.bndhep.net

Re: gas 2.34 on MacOS

[Alan Modra]

On Thu, May 07, 2020 at 10:16:10AM -0400, Kevan Hashemi wrote:
> Hi Alan,
> 
> > Do be careful regarding the license terms of any software you copy.
> 
> The source code of the ctools gas on MacOS has the FSF copyright at the top, so believe we can copy snippets directly. Please confirm.

I doubt that is sufficient, sorry.  See
https://www.gnu.org/licenses/why-assign.en.html

If you copy code, I think you need permission from all the authors of
that code to assign copyright to the FSF.  The situation here appears
to be that people took the gas-1.38 sources, modified them, and made
their modifications available under the GPL.  Which is all good, but
it doesn't mean they relinquished their own copyright over their
modifications.  The authors may have assigned copyright to the FSF
already of course, I don't know.

Dealing with legal stuff is a royal pain.

-- 
Alan Modra
Australia Development Lab, IBM

Re: gas 2.34 on MacOS

[Kevan Hashemi]

Hi Alan,

> Do be careful regarding the license terms of any software you copy.

The source code of the ctools gas on MacOS has the FSF copyright at the top, so believe we can copy snippets directly. Please confirm.

> People making significant contributions to GNU binutils need to assign
> copyright to the FSF.

If I rise to the level of making a contribution to GAS or LD, I will assign copyright to FSF. That's not new for me: all my work is GPL, including circuit diagrams, printed circuit board layouts, and logic firmware.

Best, Kevan

-- 
Kevan Hashemi, Electrical Engineer
Physics Department, Brandeis University
http://www.bndhep.net

Re: gas 2.34 on MacOS

[Kevan Hashemi]

> Thats to be expected since there never was a binutils version 1.38.1.

Thank you Gunther, that explains a few things.

> However, there is a GAS version 1.38.1 and apparently Apple based it
> cctools as on that GAS version. The as sources of the Apple version are
> quite different to the old FSF version.

There are 23 changes made to as.c of GAS 1.38.1 FSF in order to produce the as.c of ctools. The copyright at the top of the ctools as.c is the FSF copyright dated 1987. Am I to assume that Apple edited the code extensively, but did not change the version number announced by the assembler at the command line?

> BTW, old gas and binutils releases are available here:

Thank you. I have the original as.c from the 1991 distribution.


-- 
Kevan Hashemi, Electrical Engineer
Physics Department, Brandeis University
http://www.bndhep.net

Re: gas 2.34 on MacOS

[Alan Modra]

On Wed, May 06, 2020 at 01:31:40PM -0400, Kevan Hashemi wrote:
> Dear Matt,
> 
> > https://opensource.apple.com/source/cctools/
> 
> Thanks for that link. That site lets me browse the files, with nice color rendering.
> 
> > <string>http://ftp.gnu.org/gnu/binutils/binutils-1.38.1.tar.gz</string>
> 
> That link is broken: the ftp side does not have versions earlier than 2.6 archived.
> 
> > What i'm getting I guess, is it seems unlikely these sources are going
> > to help in any way with regards to getting this architecture support
> > upstreamed to FSF binutils.
> 
> When you are as ignorant of the process as I am, working 64-bit assembler code is always welcome, so thank you again.

Do be careful regarding the license terms of any software you copy.
People making significant contributions to GNU binutils need to assign
copyright to the FSF.  If you have copied software with license terms
incompatible with the GPL then you may not be able to make a legal
copyright assignment.

-- 
Alan Modra
Australia Development Lab, IBM

Re: gas 2.34 on MacOS

[Gunther Nikl]

Hello Kevan,

> > <string>http://ftp.gnu.org/gnu/binutils/binutils-1.38.1.tar.gz</string>
>
> That link is broken: the ftp side does not have versions earlier than
> 2.6 archived.

Thats to be expected since there never was a binutils version 1.38.1.
However, there is a GAS version 1.38.1 and apparently Apple based it
cctools as on that GAS version. The as sources of the Apple version are
quite different to the old FSF version.

BTW, old gas and binutils releases are available here:

	ftp://sourceware.org/pub/binutils/old-releases/

Regards,
Gunther

Re: gas 2.34 on MacOS

[Kevan Hashemi]

Dear Matt,

> https://opensource.apple.com/source/cctools/

Thanks for that link. That site lets me browse the files, with nice color rendering.

> <string>http://ftp.gnu.org/gnu/binutils/binutils-1.38.1.tar.gz</string>

That link is broken: the ftp side does not have versions earlier than 2.6 archived.

> What i'm getting I guess, is it seems unlikely these sources are going
> to help in any way with regards to getting this architecture support
> upstreamed to FSF binutils.

When you are as ignorant of the process as I am, working 64-bit assembler code is always welcome, so thank you again.

Best, Kevan

-- 
Kevan Hashemi, Electrical Engineer
Physics Department, Brandeis University
http://www.bndhep.net

Re: gas 2.34 on MacOS

[Matt Rice]

On Wed, May 6, 2020 at 3:24 PM Kevan Hashemi <hashemi@brandeis.edu> wrote:
>
> Dear Nick,
>
> > I would think that 100 hours is much more likely than 1000 hours.
>
> In that case, I will start reading. I have assembler and linker problems to surmount if I am to get a 64-bit version of my software working on MacOS. Understanding the assembler and linker, and being able to modify ther behavior, is my preferred path to success.
>
> > Since there already is a port for the x86_64 architecture, all that
> > you will need to do is tweak it so that its output is suitable for
> > the MacOS format.
>
> Indeed, that reduces the work by a factor of ten. Even better, there is a 64-bit GNU Assembler bundled with the MacOS Command Line Tools. Here's what it says about itself:
>
> % as -v
> Apple Inc version cctools-900, GNU assembler version 1.38
>
> This assembler acts like a GNU Assembler: recognises .stabs directives, for example, and talks like a GNU Assembler. So I have been assuming it actuall is a GNU Assembler. Its source code would be of great utility. But I can't figure out where it came from. On Linux and MinGW, the assembler version number matches the binutils version number. Given that binutils 2.7 was released in 1996, however, I find it hard to believe that it comes from binutils 1.38. So, I have written to Apple to see if they will hand over the source files.
>

While the source files are here:
https://opensource.apple.com/source/cctools/
https://opensource.apple.com/source/cctools/cctools-949.0.1/as/cctools.plist.auto.html
Does in fact have a link to
<string>http://ftp.gnu.org/gnu/binutils/binutils-1.38.1.tar.gz</string>

given that these were forked so long ago, apples own changes seem to
be released under the following rather than the GPL.
https://opensource.apple.com/source/cctools/cctools-949.0.1/APPLE_LICENSE.auto.html

afaik I don't really recall apple ever proposing to upstream any of
it/triggering any kind of copyright assignment,
What i'm getting I guess, is it seems unlikely these sources are going
to help in any way with regards to getting this architecture support
upstreamed to FSF binutils.


> > Plus the assembler already supports generating 32-bit MacOS binaries,
> > so there is that to follow as well.86)
>
> Also a good starting point.
>
> > There are a couple of files in the source repository that document
> > *some* of the internals (gas/doc/internals.texi,
> > bfd/doc/bfdint.texi)
>
> Those look good. It would have taken me ages to find them on my own.
>
> > https://sourceware.org/binutils/binutils-porting-guide.txt
>
> Also looks most helpful. You are modest about the documentation, but it's looking pretty good to me.
>
> Best, Kevan
>
> --
> Kevan Hashemi, Electrical Engineer
> Physics Department, Brandeis University
> http://www.bndhep.net

Re: gas 2.34 on MacOS

[Kevan Hashemi]

Dear Nick,

> I would think that 100 hours is much more likely than 1000 hours.

In that case, I will start reading. I have assembler and linker problems to surmount if I am to get a 64-bit version of my software working on MacOS. Understanding the assembler and linker, and being able to modify ther behavior, is my preferred path to success.

> Since there already is a port for the x86_64 architecture, all that
> you will need to do is tweak it so that its output is suitable for 
> the MacOS format.

Indeed, that reduces the work by a factor of ten. Even better, there is a 64-bit GNU Assembler bundled with the MacOS Command Line Tools. Here's what it says about itself:

% as -v
Apple Inc version cctools-900, GNU assembler version 1.38

This assembler acts like a GNU Assembler: recognises .stabs directives, for example, and talks like a GNU Assembler. So I have been assuming it actuall is a GNU Assembler. Its source code would be of great utility. But I can't figure out where it came from. On Linux and MinGW, the assembler version number matches the binutils version number. Given that binutils 2.7 was released in 1996, however, I find it hard to believe that it comes from binutils 1.38. So, I have written to Apple to see if they will hand over the source files.

> Plus the assembler already supports generating 32-bit MacOS binaries,
> so there is that to follow as well.86)

Also a good starting point.

> There are a couple of files in the source repository that document
> *some* of the internals (gas/doc/internals.texi, 
> bfd/doc/bfdint.texi)

Those look good. It would have taken me ages to find them on my own.

> https://sourceware.org/binutils/binutils-porting-guide.txt

Also looks most helpful. You are modest about the documentation, but it's looking pretty good to me.

Best, Kevan

-- 
Kevan Hashemi, Electrical Engineer
Physics Department, Brandeis University
http://www.bndhep.net

Re: gas 2.34 on MacOS

[Nick Clifton]

Hi Kevan,

>> The support is absent primarily because nobody has done the work to
>> create a port of the assembler or linker to this particular OS.
>> Porting these tools is not a trivial piece of work, but if someone is
>> interested in doing it then we would be happy to help them.
> 
> In principle, yes, I would like to do this port. If it were to take me only 100 hours, for sure I would do it. Even if it were to take me 1000 hours, I would consider doing it. If you already know that an experienced binutils contributor would take 1000 hours to do the work, please say so, because in that case, I would not be able to get it done in the time I have available.

I cannot give you an exact figure, but I would think that 100 hours is much
more likely than 1000 hours.  To start with I would suggest investigating
the assembler.  Since there already is a port for the x86_64 architecture,
all that you will need to do is tweak it so that its output is suitable for
the MacOS format.  Plus the assembler already supports generating 32-bit 
MacOS binaries, so there is that to follow as well.

Note that saying "all that you will need to do" above is a bit of a lie.
It will not be simple.  You will need to dig into the BFD library to see
how that works.  Plus you will need to get familiar with the 64-bit MacOS
file format.  I have no idea how well/poorly this is documented, so it may
not be an easy task.


> In any case, I would like educate myself on the internal workings of "ld" and "gas". Can you recommend a text book, or an on-line treatise, that describes how the source code is organised, and how it works?

Ah - yes - and here is the core of the problem.  There really isn't any
documentation like this.  There are a couple of files in the source 
repository that document *some* of the internals (gas/doc/internals.texi,
bfd/doc/bfdint.texi) but they are by no means authoritative.  

The real documentation is the source code itself, and the most common way
to create a new port, or extend an old one, is to start with the existing
code and copy/modify it to your needs.

On the plus side we are always willing to help out potential contributors.
So feel free to post questions, ask for advice, or tell us how bad the
documentation is.  (Actually we know about that last one already).  Of
course if you are feeling really keen then you could write some documentation
youself and then contribute that as well.

Having said all of that, there are some web pages that you might find useful:

https://sourceware.org/binutils/binutils-porting-guide.txt
https://opensourceforu.com/2010/01/binutils-porting-guide-to-a-new-target-architecture/

Cheers
  Nick

Re: gas 2.34 on MacOS

[Kevan Hashemi]

Dear Nick,

Thank you for your answer.

> Sadly the binutils does not support creating either a linker or an
> assembler for MacOS.

Now everything makes sense.

> So you will have to use the system tools for these functions.

I'll try and get by with GNU Assembler 1.38.

> The support is absent primarily because nobody has done the work to
> create a port of the assembler or linker to this particular OS.
> Porting these tools is not a trivial piece of work, but if someone is
> interested in doing it then we would be happy to help them.

In principle, yes, I would like to do this port. If it were to take me only 100 hours, for sure I would do it. Even if it were to take me 1000 hours, I would consider doing it. If you already know that an experienced binutils contributor would take 1000 hours to do the work, please say so, because in that case, I would not be able to get it done in the time I have available.

In any case, I would like educate myself on the internal workings of "ld" and "gas". Can you recommend a text book, or an on-line treatise, that describes how the source code is organised, and how it works?


Best, Kevan

-- 
Kevan Hashemi, Electrical Engineer
Physics Department, Brandeis University
http://www.bndhep.net

Re: gas 2.34 on MacOS

[Nick Clifton]

Hi Kevan,

> So I would like to try 2.34. On MacOS 10.12 and 10.15, I download binutils-2.34.tar.gz, configure, compile, and install. I get no errors, and a bunch of utilities get installed, but neither "ld" nor "as" are installed. When I do the same on Linux, I get both "ld" and "as" installed, as I expect.

> Am I missing something fundamental here?

Sadly the binutils does not support creating either a linker or an assembler for MacOS.
(Well it does support creating an assembler for 32-bit i686 MacOS, but not for 64-bit
x86_64 MacOS).  So you will have to use the system tools for these functions.

The support is absent primarily because nobody has done the work to create a port of
the assembler or linker to this particular OS.  Porting these tools is not a trivial
piece of work, but if someone is interested in doing it then we would be happy to help
them.

Cheers
  Nick

gas 2.34 on MacOS

[Kevan Hashemi]

Dear Binutils Community,

I would like to install the GNU Assembler version 2.34 on MacOS. I have version 1.38, but it's giving me the following error when I try to complie 64-bit code;

"suffix or operands invalid for `movq'"

So I would like to try 2.34. On MacOS 10.12 and 10.15, I download binutils-2.34.tar.gz, configure, compile, and install. I get no errors, and a bunch of utilities get installed, but neither "ld" nor "as" are installed. When I do the same on Linux, I get both "ld" and "as" installed, as I expect.

On MacOS 10.12 and 10.15, I try installing with Homebrew, but still don't get "as" or "ld". Indeed: the Homebrew installation appears to be identical, although it's location is different.

Am I missing something fundamental here?

Best, Kevan

-- 
Kevan Hashemi, Electrical Engineer
Physics Department, Brandeis University
http://www.bndhep.net