i370 port

i370 port

[Paul Edwards]

Hello GCC maintainers.

There used to be an i370 target for GCC.  It was written in 1989,
and put into GCC 2.5 in 1993.

It has always been semi-working, but never good enough to
actually use.

It was dropped from GCC 4 when there was supposedly no
maintainer available.  Actually, Dave Pitts and myself were
both maintaining it at that time, but we were both still working
on an old version of it (3.2). So gcc 3.4.6, circa 2004, was the
last time it was included in the normal GCC distribution.

We were both maintaining it, and continue to maintain it,
because MVS doesn't have any alternate free C compiler
available.

As of yesterday, after years of work, an i370 version was 
released that is now fully working.  The code generator has 
no known bugs that would stop it from being C90-compliant,
and GCC can fully regenerate itself, with full optimization.  

You can see it here:

http://gccmvs.sourceforge.net

It's based on GCC 3.2.3.

There is also a free i370 emulator (Hercules) and a free i370-based 
operating system (MVS/380) that enables the compiler to be fully 
tested and fully regenerate itself on its native environment.  Not
only that, but there is an effort underway to allow this free
environment to be made available on the internet so that Unix
users can do an MVS build (takes around 4 hours if done properly
- ie 3 stage, full optimization, on an emulated machine), from the 
safety of their Unix box.

Both of those products are also under active development by a
community of mainframe enthusiasts.

In addition, that code has been ported to GCC 3.4.6, which is now
working as a cross-compiler at least.  It's still some months away
from working natively though.  It takes a lot of effort to convert
the Posix-expecting GCC compiler into C90 compliance.  This has
been done though, in a way that has minimal code changes to the
GCC mainline.

There is a lot of other activity (e.g. availability of REXX, PL/1, Cobol)
that rests on the C compiler being available.

So, my question is - what is required to get the i370 port reinstated
into the GCC mainline?

Yes, I'm aware that there is an S/390 port, but it isn't EBCDIC, isn't
HLASM, isn't 370, isn't C90, isn't MVS.  It may well be possible to
change all those things, and I suspect that in a few years from now
I may be sending another message asking what I need to do to get 
all my changes to the s390 target into the s390 target.  At that time,
I suspect there will be a lot of objection to "polluting" the s390 target
with all those "unnecessary" things.

So, here's hoping that the i370 port can end up where it was originally
intended to end up, and that all the effort that was spent in the GCC
mainline to get rid of the ASCII assumptions can now be put to good
use.

BFN.  Paul.

Re: i370 port

[Joseph S. Myers]

On Fri, 5 Jun 2009, Paul Edwards wrote:

> It was dropped from GCC 4 when there was supposedly no
> maintainer available.  Actually, Dave Pitts and myself were
> both maintaining it at that time, but we were both still working
> on an old version of it (3.2). So gcc 3.4.6, circa 2004, was the
> last time it was included in the normal GCC distribution.

(For reference, the port was removed in SVN revision 77216; before then it 
had had various largely mechanical changes as part of changes to multiple 
back ends or target-independent code, with r69086 as the last vaguely 
i370-only change but no changes appearing to come from someone 
specifically working and testing on i370 for some years before then.  "svn 
log svn://gcc.gnu.org/svn/gcc/trunk/gcc/config/i370@77215" shows the 
history.)

> We were both maintaining it, and continue to maintain it,
> because MVS doesn't have any alternate free C compiler
> available.

To merge back into FSF GCC, the people who have made changes that would be 
merged back will need to have copyright assignments on file at the FSF 
(and disclaimers from any relevant employers).  I don't have a current 
copy of the assignments list (my very old copy does show assignments from 
David G. Pitts with an employer disclaimer dating from 1993).

> So, my question is - what is required to get the i370 port reinstated
> into the GCC mainline?

The basic requirements for a resurrected port are the same as for a new 
port; it needs to be assigned to the FSF, to pass the normal technical 
review, and the SC needs to approve someone as a maintainer of the port 
(there may be a bottleneck with the last stage, since there are currently 
at least three new ports pending approval).  It is a very good idea if you 
can run the testsuite for the port and will be posting results to 
gcc-testresults regularly.

I would encourage going through all the changes made to the i370 port on 
GCC mainline, after 3.1/3.2 branched and before the port was removed, to 
see what should be merged into your version for mainline; ultimately it 
would be up to you how you get it updated for all the mechanical changes 
on mainline since 3.2, but those changes (see command above to get logs) 
may be a useful guide to how to do some of the updates.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: i370 port

[Paul Edwards]

>> We were both maintaining it, and continue to maintain it,
>> because MVS doesn't have any alternate free C compiler
>> available.
>
> To merge back into FSF GCC, the people who have made changes that would be
> merged back will need to have copyright assignments on file at the FSF
> (and disclaimers from any relevant employers).  I don't have a current
> copy of the assignments list (my very old copy does show assignments from
> David G. Pitts with an employer disclaimer dating from 1993).

There's only 3 people who have made changes.  Dave Pitts, Linas
Vepstas and myself.  Dave you already have apparently.  Linas's
code is largely already merged - just his last set of changes didn't
get put in.  That leaves me.  I work as a contractor and I'd probably
be sacked if I tried to either bring in or attempt to maintain gcc.
All my work was done at home.

>> So, my question is - what is required to get the i370 port reinstated
>> into the GCC mainline?
>
> The basic requirements for a resurrected port are the same as for a new
> port; it needs to be assigned to the FSF, to pass the normal technical
> review, and the SC needs to approve someone as a maintainer of the port
> (there may be a bottleneck with the last stage, since there are currently
> at least three new ports pending approval).  It is a very good idea if you
> can run the testsuite for the port and will be posting results to
> gcc-testresults regularly.

The port is to a pure C90 environment (ie not posix, not unix).  It was a
major effort to achieve that, and it has only just been completed to the
point where the compiler recompiles itself with full optimization.  The
environment where it runs is not set up to run shell scripts or makes
or test suites.  It's set up to run JCL, and there's a stack of JCL card
decks to allow GCC to compile, which would be good to have included
in the i370 directory.

It's difficult enough just to get GCC to know to open dd:include(xxx)
for an include of "xxx.h" and dd:sysincl(xxx) for an include of <xxx.h>.
That logic was revamped in gcc 3.4.6 so I haven't put it in yet.  It'll
probably be months before I do that, because I can't test it until it
gets up onto the mainframe.  And once again, in preparation for that,
I need to make it a pure C90 application.  So that is where I spend
my effort.

Note that the i370 port used to be in GCC even though it was riddled
with bugs.  It took literally years to get rid of them.  I would have
thought that GCC recompiling itself was a damn good start for
inclusion, irrespective of any test suites (assuming those test
suites are even C90 code - as they would need to be to work).

> I would encourage going through all the changes made to the i370 port on
> GCC mainline, after 3.1/3.2 branched and before the port was removed, to
> see what should be merged into your version for mainline; ultimately it
> would be up to you how you get it updated for all the mechanical changes
> on mainline since 3.2, but those changes (see command above to get logs)
> may be a useful guide to how to do some of the updates.

I have already merged the changes made from 3.2.3 to 3.4.6 into my
code, and have a diff against 3.4.6 available already.  ie, that covers
all known code changes.  But it only works as a cross-compiler at the
moment.  It's probably a few months away from being native.

BFN.  Paul.

Re: i370 port

[Joseph S. Myers]

On Sat, 6 Jun 2009, Paul Edwards wrote:

> The port is to a pure C90 environment (ie not posix, not unix).  It was a
> major effort to achieve that, and it has only just been completed to the
> point where the compiler recompiles itself with full optimization.  The
> environment where it runs is not set up to run shell scripts or makes
> or test suites.  It's set up to run JCL, and there's a stack of JCL card
> decks to allow GCC to compile, which would be good to have included
> in the i370 directory.

You can test a cross compiler if you have some way of copying a test 
executable to the i370 system, running it and getting its output and exit 
status back (actually you don't need to be able to get the exit status 
since DejaGnu has wrappers to include it in the output if needed).  There 
is no need for the target to be able to run shell scripts or makes.  You 
would need to write your own DejaGnu board file that deals with copying 
to/from the i370 system and running programs there.  The testsuite 
routinely runs for much more limited embedded systems (using appropriate 
board files).

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: i370 port

[Paul Edwards]

>> The port is to a pure C90 environment (ie not posix, not unix).  It was a
>> major effort to achieve that, and it has only just been completed to the
>> point where the compiler recompiles itself with full optimization.  The
>> environment where it runs is not set up to run shell scripts or makes
>> or test suites.  It's set up to run JCL, and there's a stack of JCL card
>> decks to allow GCC to compile, which would be good to have included
>> in the i370 directory.
> 
> You can test a cross compiler if you have some way of copying a test 
> executable to the i370 system

It doesn't build executables either.

Only the "-S" option is used.

With that restriction, GCC merely reads a bunch of text files and
writes a text file, and thus is amenable to being a pure C90
application.  That's how it manages to work at all.

> running it and getting its output and exit 
> status back (actually you don't need to be able to get the exit status 
> since DejaGnu has wrappers to include it in the output if needed).  

It so happens that MVS/380 has the ability to be run in batch, and
extracting the exit code won't be a problem either.

Note however that I normally do all my GCC work in Windows,
and the batch running etc is done with batch files.

> There 
> is no need for the target to be able to run shell scripts or makes.  You 
> would need to write your own DejaGnu board file that deals with copying 
> to/from the i370 system and running programs there.  The testsuite 
> routinely runs for much more limited embedded systems (using appropriate 
> board files).

I have a large backlog of work to do with the i370 port already, starting
with getting gcc 3.4.6 running natively.  Isn't that a more productive
thing to do?  Even after 3.4.6 is done, so that every scrap of code is
available, then there's version 4 to do!

BFN.  Paul.

Re: i370 port

[Joseph S. Myers]

On Sat, 6 Jun 2009, Paul Edwards wrote:

> > There is no need for the target to be able to run shell scripts or makes.
> > You would need to write your own DejaGnu board file that deals with copying
> > to/from the i370 system and running programs there.  The testsuite routinely
> > runs for much more limited embedded systems (using appropriate board files).
> 
> I have a large backlog of work to do with the i370 port already, starting
> with getting gcc 3.4.6 running natively.  Isn't that a more productive
> thing to do?  Even after 3.4.6 is done, so that every scrap of code is
> available, then there's version 4 to do!

It's up to you what priorities you assign to different things involved in 
getting this on mainline, but we use test results postings as evidence of 
what sort of state each port is in, where a particular test failure is 
appearing and whether each port is being used, so there may be reluctance 
to accept a port that will not have test results posted regularly for 
mainline.  (This is much less of a problem for OS ports than for CPU 
ports; if one OS for a given CPU has results routinely posted, it doesn't 
matter so much if other OSes don't, though having results for different 
OSes is still useful.)

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: i370 port - in search of hooks

[Paul Edwards]

I'm making quite good progress with cleaning up the 3.4.6 i370
port.  I've even got optimization working to some degree.

Meanwhile, on a different machine (a Linux machine I program
on on the way to/from work), I have managed to build 4.4.0,
which means I have an environment to work on a more modern
version of GCC.

That modern version is what I would like to ask about today.
Leaving aside the issue of the actual target, I'm more interested
in the intrusive hooks I expect I will need (I won't know until
I start doing the work, and I can't do that until I find out whether
4.4 is good enough - chicken and egg).  Here is what I needed
for 3.4.6:


1. Ability to build a standalone executable.  Simply put, I need
gcc to do a function call like this:

#ifdef SINGLE_EXECUTABLE
     {
         int cnt = 0;

         while (commands[i].argv[cnt] != NULL)
         {
             cnt++;
         }
         if (strcmp(string, "cc1") == 0)
         {
             ret_code = toplev_main(cnt, commands[i].argv);
             if (ret_code != 0) break;
         }
      }
#else

Doesn't need to be exactly like that, but some sort of hook to be
made available so that I can bypass system() or any variation of
that.  C90 doesn't guarantee that system() will do anything in
particular.  And my C environment indeed doesn't work too well
if you try that.  Can't have two programs opening and closing
the same DDNAME.  Can only have 100 characters worth of
parameters too.


2. A completely different way of handling include files.  After going
through the normal remap process, I then want the following
transformations:

#include "fred.h"

gets translated into an fopen("dd:include(fred)", "r")

#include <fred.h>

gets translated into an fopen("dd:sysincl(fred)", "r")

None of this checking to see if something is a directory etc.  There's
no such call available in C90, nor my C library on MVS.  The code
above looks trivial enough, but when it's time to actually find where
to put that, it's non-trivial.


3. There is some complicated parameter lookup facility - it does a binary
search on the parameters.  This requires a whole lot of infrastructure to
be present to generate the code.  Infrastructure which I don't have.  I'd
like a simple sequential search to be available as an option I can
activate.


4. There are a whole lot of includes that don't exist, like sys/types.h, and
I'd like them to be masked out like all other includes are done (even
things that are part of C90 have masks!!!).


5. It'd be cool if some of the names could be unique in the first 8 
characters
(C90 guarantees 6 for externals, but I have the luxury of 8).  I have a
mostly non-intrusive way of remapping everything, but there are a few
that I need to do intrusively, because I can't #define away something
that is already #defined.  Problem is compounded by the fact that
I link together code that normally isn't linked together.  Note that I don't
need things like an assembler and linker linked in together - I just need
the stuff required for the "-S" option to work.  ie text file (ie C code) 
in,
and text file (ie assembler code) out - a text processing program that
should be (and in fact, has already been made that way) possible to
do in pure C90.


6. It would be nice if all the non-C90 Posix functions were masked out,
but so far I have been able to kludge around that without requiring a
lot of intrusive changes.  It would be good to get the worst of them
out though.


My questions are:

1. Are changes like the above likely to be accepted into the head version
going forward?

2. If they are, what version should I work on to get that to happen?


Ideally I'd like to work on a stable version, perhaps 4.4, and later have
those changes merged onto the head.  But I fear if I do that I will end
up in the same position I am in now with 3.4.6, ie too many changes
such that my patches are never actually relevant.

But it's quite daunting to get this working at all.

So I thought that what I might be able to do instead is get the hooks
in place first.  Not necessarily all at once.  Possibly over the course
of a year.

Eventually all the hooks will exist, there will be a stable release cut
containing all the hooks I need, and then I may be in with a chance of
getting i370 working on that environment.  That would also be done
over the course of a year or whatever, as the GMP and MPFR need to
be set up on MVS too (just having a S/390 port is not sufficient - I
need S/370 HLASM).

Hopefully at the end of that process, I'll have an i370 port that is done
in a "standard" way so that updating to the latest GCC is fairly trivial
*regardless* of whether the i370 port is included in GCC proper due
to the yet more technical challenges that requires, another job for
another year.  :-)

This is a parallel effort to my 3.4.6 work which is done on a different
PC at a different time.  3.4.6 is mostly about getting it to run on
real MVS.  4.4/x is simply to get a cross-compile to work for now.

Thanks.  Paul.

Re: i370 port

[Paul Edwards]

Hi Joseph. (reviving a thread from 2009)

I have been streamlining the execution
of MVS (i370) via my "runmvs" script
recently, and now it only takes a few
seconds to do a test on MVS because
I have taken out all the unnecessary
pauses and made lots of other
improvements, and bypassing bugs
in Hercules.

Numerous other i370 bugs have been
fixed in GCC 3.2.3 too.

So I decided it was time to take a look
at the test suite to see what state that
was in.

With the aid of a script (below), I got
the following results:

C:\devel\gcc\gcc\config\i370\test>showres

C:\devel\gcc\gcc\config\i370\test>grep passed results.txt   | wc
    516    1032   10278

C:\devel\gcc\gcc\config\i370\test>grep failed results.txt   | wc
    117     234    2373

C:\devel\gcc\gcc\config\i370\test>


I looked at the first few errors. One
was deficiencies with "long long"
which is currently in the "too hard"
basket.

Some were ASCII assumptions (the
i370 port is EBCDIC).

And there was a genuine bug which I
have passed on to Dave Pitts for
analysis.

Regarding the ASCII errors, here’s an
example:

C:\devel\gcc\gcc\testsuite\gcc.c-torture\execute>type 20000412-3.c
typedef struct {
  char y;
  char x[32];
} X;

int z (void)
{
  X xxx;
  xxx.x[0] =
  xxx.x[31] = '0';
  xxx.y = 0xf;
  return f (xxx, xxx);
}

int main (void)
{
  int val;

  val = z ();
  if (val != 0x60)
    abort ();
  exit (0);
}

int f(X x, X y)
{
  if (x.y != y.y)
    return 'F';

  return x.x[0] + y.x[0];
}


They have assumed that '0' is 0x30,
but on EBCDIC it is 0xf0.

The documentation says:

C:\devel\gcc\gcc\testsuite>grep PORTABLE readme.gcc
DO NOT PUT NON-PORTABLE TESTCASES IN gcc.c-torture.


So what do you suggest I do with this class
of error?

The choices seem to be:

1. Remove/ignore it as it is non-portable so
shouldn't be there in the first place.

2. Create a new internal define like __EBCDIC__
and if that is set, do something different.

3. Create a user-define like -DEBCDIC to be
tested.

4. Do something like:

#if '0' == 0x30

(untested)

5. Use an existing define, like __MVS__
and if that is set, do something-or-other.


Any suggestions?

I know 3.2.3 is very old, but this is a very
long road, and this is where the most
stable i370.md environment is.

BTW, I saw someone mention something
about a "compile farm". MVS/380 is a
freely available i370 platform which
allows you to run tests on MVS pretty
easily, if a time comes when people
want to run tests on MVS to see if
their code works on EBCDIC as C90
allows.

Here is what it looks like on my system:

Just call this script:

call mvsgccr cprog.c output.txt

and it produces an output file that
includes:

07.28.30 JOB    1  $HASP373 MVSGCCR  STARTED - INIT  3 - CLASS C - SYS BSP1
07.28.30 JOB    1  IEF403I MVSGCCR - STARTED - TIME=07.28.30
07.28.30 JOB    1  IEFACTRT - Stepname  Procstep  Program   Retcode
07.28.31 JOB    1  MVSGCCR    S1        CREATE    IEFBR14   RC= 0000
07.28.31 JOB    1 *IEF233A M 401,PCTOMF,,MVSGCCR,COMP
07.28.32 JOB    1  MVSGCCR    S1        COMP      GCCNOMM   RC= 0000
07.28.33 JOB    1  MVSGCCR    S1        ASM       ASMA90    RC= 0000
07.28.33 JOB    1  MVSGCCR    S1        LKED      IEWL      RC= 0000
07.28.33 JOB    1  MVSGCCR    S1        EXECC     CPROG     RC= 0000
07.28.33 JOB    1  IEF404I MVSGCCR - ENDED - TIME=07.28.33
07.28.33 JOB    1  $HASP395 MVSGCCR  ENDED

On a failure of a test case, this line:

07.28.33 JOB    1  MVSGCCR    S1        EXECC     CPROG     RC= 0000

changes to:

07.28.33 JOB    1  MVSGCCR    S1        EXECC     CPROG     RC= 0012


MVS is a very interesting environment
in my opinion, and it's cool to have
code working there too.

I was thinking that maybe for people
who don't wish to install MVS/380
or TK4- on their system, there could
be some sort of hacked ftp so that
people could go:

ftp some.site
put cprog.c
put go.txt (this hangs while the C
program is compiled)
get output.txt
bye

Or for the more generic case:
ftp some.different.site
put in.jcl
put in.zip
put go.txt (hangs possibly 10 minutes)
get output.txt
get out.zip
bye

(for when you wish to build an entire
application, like "sed", on MVS).

go.txt could have the required runmvs
command, such as:

runmvs in.jcl output.txt in.zip out.zip


Usage here:

C:\mvs380>runmvs
Usage runmvs jcl.file print.file [aux_input.file|none] 
[aux_output.file|none] [ascii|binary|rdwund|rdwvar|vart] [ascii|binary] 
[awstap.file|none]
e.g. runmvs compile.jcl output.txt world.c world.s ascii ascii
or runmvs buildgcc.jcl output.txt source.zip xmit.dat
default is for files (if provided) to be binary


BTW, is that the main testsuite I should
be running? There's 633 C files there. Is
that enough, given that I'm only building
and testing C, or should I also compile
the stuff in the "compile" directory at the
same level as "execute"?

Thanks. Paul.





C:\devel\gcc\gcc\config\i370\test>type onecomp.bat
rem this should be run in a directory under config/i370
copy ..\..\..\testsuite\gcc.c-torture\execute\%1 cprog.c
call mvsgccr cprog.c output.txt
grep "EXECC     CPROG     RC= 0000" output.txt
if errorlevel 1 goto bad
echo %1 passed >>results.txt
goto exit
:bad
echo %1 failed >>results.txt
:exit

C:\devel\gcc\gcc\config\i370\test>



C:\devel\gcc\gcc\config\i370\test>head dotests.bat
del results.txt

call onecomp 20000112-1.c
call onecomp 20000113-1.c
call onecomp 20000121-1.c
call onecomp 20000205-1.c
call onecomp 20000217-1.c
call onecomp 20000223-1.c
call onecomp 20000224-1.c
call onecomp 20000225-1.c

C:\devel\gcc\gcc\config\i370\test>





-----Original Message----- 
From: Joseph S. Myers
Sent: Saturday, June 6, 2009 1:02 AM
To: Paul Edwards
Cc: gcc@gcc.gnu.org
Subject: Re: i370 port

On Sat, 6 Jun 2009, Paul Edwards wrote:

> The port is to a pure C90 environment (ie not posix, not unix).  It was a
> major effort to achieve that, and it has only just been completed to the
> point where the compiler recompiles itself with full optimization.  The
> environment where it runs is not set up to run shell scripts or makes
> or test suites.  It's set up to run JCL, and there's a stack of JCL card
> decks to allow GCC to compile, which would be good to have included
> in the i370 directory.

You can test a cross compiler if you have some way of copying a test
executable to the i370 system, running it and getting its output and exit
status back (actually you don't need to be able to get the exit status
since DejaGnu has wrappers to include it in the output if needed).  There
is no need for the target to be able to run shell scripts or makes.  You
would need to write your own DejaGnu board file that deals with copying
to/from the i370 system and running programs there.  The testsuite
routinely runs for much more limited embedded systems (using appropriate
board files).

-- 
Joseph S. Myers
joseph@codesourcery.com 

Re: i370 port

[Paul Edwards]

>> It was dropped from GCC 4 when there was supposedly no
>> maintainer available.  Actually, Dave Pitts and myself were
>> both maintaining it at that time, but we were both still working
>> on an old version of it (3.2). So gcc 3.4.6, circa 2004, was the
>> last time it was included in the normal GCC distribution.
>
> (For reference, the port was removed in SVN revision 77216; before then it
> had had various largely mechanical changes as part of changes to multiple
> back ends or target-independent code, with r69086 as the last vaguely
> i370-only change but no changes appearing to come from someone
> specifically working and testing on i370 for some years before then.  "svn
> log svn://gcc.gnu.org/svn/gcc/trunk/gcc/config/i370@77215" shows the
> history.)

I just took a look to see if there was anything changed on the head
that didn't make it into 3.4.6.  Short answer: no.

Long answer:

C:\devel\gccoff>diff gccnew gcchead
Only in gcchead: .svn
diff gccnew/i370.h gcchead/i370.h
530,531c530
< #define INIT_CUMULATIVE_ARGS(CUM, FNTYPE, LIBNAME, INDIRECT, N_NAMED_ARGS) 
\
<   ((CUM) = 0)
---
> #define INIT_CUMULATIVE_ARGS(CUM, FNTYPE, LIBNAME, INDIRECT)  ((CUM) = 0)

I just need to delete that N_NAMED_ARGS when upgrading.

I'm sure that will be the least of my worries though.  I'm more
worried about things like the below.  :-)

Incidentally, 15 years of effort to get a hosted GCC compiler on
real MVS (not USS) came to realization on Jan 11, 2004:

http://osdir.com/ml/emulators.hercules390.mvs/2004-01/msg00031.html

(although it wasn't released until March 2004 when it was good
enough to self-compile.)

The revision that deleted i370 was made on Feb 4th, 2004.  How's
that for timing?

Incidentally, while it may look from this angle like the i370 port
wasn't being worked on, it's actually had a huge amount of
effort put into it.  The activity has just been in different forums
(like the one above).  It wasn't just GCC though.  A lot of
infrastructure was required.  E.g. even the assembler didn't
support such a huge number of externals that was being
generated by GCC, so first I hacked around that in GCC
itself, mapping a whole lot of "unused" flags to be the same,
and then reversed that out when someone came up with a
modification to the assembler.

Then we needed to switch from 24-bit mode to 31-bit mode to
get the required memory for GCC to self-compile.  Another
huge enterprise.

> I would encourage going through all the changes made to the i370 port on
> GCC mainline, after 3.1/3.2 branched and before the port was removed, to
> see what should be merged into your version for mainline; ultimately it
> would be up to you how you get it updated for all the mechanical changes
> on mainline since 3.2, but those changes (see command above to get logs)
> may be a useful guide to how to do some of the updates.

All the merging has already been done in the 3.4.6 effort.  The only
thing that I know of that is still "at large" is someone else who was
working "offline" and made some changes.  Specifically this:

http://osdir.com/ml/emulators.hercules390.mvs/2004-01/msg00008.html

as I just discovered the same problem with both of these strict moves
now that I too am using 3.4.6.

The i370.md looks correct to me (this is the movstricthi one):

> ;(define_insn ""
> ;  [(set (strict_low_part (match_operand:HI 0 "register_operand" "+d"))
> ;     (match_operand:HI 1 "general_operand" "dSi"))]
> ;  ""
> ;  "*
> ;{
> ;  check_label_emit ();
> ;  if (REG_P (operands[1]))
> ;    {
> ;      mvs_check_page (0, 8, 0);
> ;      return \"STH   %1,\" CONVLO \"(,13)\;ICM       %0,3,\" CONVLO 
> \"(13)\";

> ;    }
> ;  else if (GET_CODE (operands[1]) == CONST_INT)
> ;    {
> ;      mvs_check_page (0, 4, 2);
> ;      return \"ICM   %0,3,%H1\";
> ;    }
> ;  mvs_check_page (0, 4, 0);
> ;  return \"ICM       %0,3,%1\";
> ;}"
> ;   [(set_attr "length" "8")]
> ;)

but I have had to comment it out, because otherwise I get code like
this generated:

L691     EQU   *
         SLR   2,2
         IC    2,0(8)
         LA    5,92(0,0)
         CLR   2,5
         BE    L699
         BH    L702
         ICM   5,3,=H'64'
         BE    L696
         ICM   5,3,=H'78'
         BE    L694
         B     L701

ie the LA and CLR combination are what I would expect, but gcc 3.4.6
has decided to use an ICM to move a constant in, which seems an
awful waste to me instead of using LA, but the real problem is that
it hasn't generated a CLR afterwards (it needs to compare against
register 2), so isn't taking a branch it should be.

I didn't have this problem in 3.2.3, which has a virtually identical
machine definition.  But I'd be really surprised to find a serious
compiler bug outside of the i370 code?!  I assume I'm just looking
in the wrong spot.

But at least I'm making progress.  :-)

BFN.  Paul.

Re: i370 port

[Ulrich Weigand]

Paul Edwards wrote:

> In addition, that code has been ported to GCC 3.4.6, which is now
> working as a cross-compiler at least.  It's still some months away
> from working natively though.  It takes a lot of effort to convert
> the Posix-expecting GCC compiler into C90 compliance.  This has
> been done though, in a way that has minimal code changes to the
> GCC mainline.

You're referring to building GCC for a non-Posix *host*, right?
I assume those changes are not (primarily) in the back-end, but
throughout GCC common code?

> Yes, I'm aware that there is an S/390 port, but it isn't EBCDIC, isn't
> HLASM, isn't 370, isn't C90, isn't MVS.  It may well be possible to
> change all those things, and I suspect that in a few years from now
> I may be sending another message asking what I need to do to get 
> all my changes to the s390 target into the s390 target.  At that time,
> I suspect there will be a lot of objection to "polluting" the s390 target
> with all those "unnecessary" things.

Actually, I would really like to see the s390 target optionally support
the MVS ABI and HLASM assembler format, so I wouldn't have any objection
to patches that add these features ...

I understand current GCC supports various source and target character
sets a lot better out of the box, so it may be EBCDIC isn't even an
issue any more.   If there are other problems related to MVS host
support, I suppose those would need to be fixed in common code anyway,
no matter whether the s390 or i370 back-ends are used.

The only point in your list I'm sceptical about is 370 architecture
support -- I don't quite see why this is still useful today (the s390
port does require at a minimum a S/390 G2 with the branch relative
instructions ... but those have been around for nearly 15 years).

Bye,
Ulrich

-- 
  Dr. Ulrich Weigand
  GNU Toolchain for Linux on System z and Cell BE
  Ulrich.Weigand@de.ibm.com

Re: i370 port

[Paul Edwards]

Wow, what a lot of responses.  Last time I tried making
contact, I didn't get a single response!

>> In addition, that code has been ported to GCC 3.4.6, which is now
>> working as a cross-compiler at least.  It's still some months away
>> from working natively though.  It takes a lot of effort to convert
>> the Posix-expecting GCC compiler into C90 compliance.  This has
>> been done though, in a way that has minimal code changes to the
>> GCC mainline.
>
> You're referring to building GCC for a non-Posix *host*, right?

Yep.

> I assume those changes are not (primarily) in the back-end, but
> throughout GCC common code?

Yes.  Or rather, they would be, if it weren't for sleight-of-hand to
minimize that.  I dummied up all the Posix calls to point back to
C90 functions.

Please take a look at the actual changes to GCC.  There's not
a lot:

Here's the exact file:

https://sourceforge.net/project/downloading.php?group_id=195127&filename=gccmvs-3_2_3-7_0.zip&a=50206324

Most of the size is generated code from the md, or other new files,
and not changes to GCC proper.

However, in fact, GCC is turned on its head.  It's a single executable.
C90 doesn't guarantee, and the host doesn't support, the ability to do
a fork() and exec().

>> Yes, I'm aware that there is an S/390 port, but it isn't EBCDIC, isn't
>> HLASM, isn't 370, isn't C90, isn't MVS.  It may well be possible to
>> change all those things, and I suspect that in a few years from now
>> I may be sending another message asking what I need to do to get
>> all my changes to the s390 target into the s390 target.  At that time,
>> I suspect there will be a lot of objection to "polluting" the s390 target
>> with all those "unnecessary" things.
>
> Actually, I would really like to see the s390 target optionally support
> the MVS ABI and HLASM assembler format, so I wouldn't have any objection
> to patches that add these features ...

Great.

> I understand current GCC supports various source and target character
> sets a lot better out of the box, so it may be EBCDIC isn't even an
> issue any more.

It looks that way from what I've seen of 3.4.6 so far.  However, I
won't know for sure until it's on the host and self-generating.

> If there are other problems related to MVS host
> support, I suppose those would need to be fixed in common code anyway,
> no matter whether the s390 or i370 back-ends are used.

Well, I would like to see that - I don't know why there are all
those calls to open() instead of fopen() etc in the first place,
but I don't see that happening.

> The only point in your list I'm sceptical about is 370 architecture
> support -- I don't quite see why this is still useful today (the s390
> port does require at a minimum a S/390 G2 with the branch relative
> instructions ... but those have been around for nearly 15 years).

The last free MVS was MVS 3.8j which runs on the S/370 architecture.
If you want to write MVS software, for free, your only option is to
pick that up.  It doesn't run on S/390 hardware.

However.  :-)

That's where MVS/380 comes in.

http://mvs380.sourceforge.net

So yes, we can sort of cope with S/390 instructions.  It's just not as
widely supported as the proper S/370 (emulated) machine.  Or rather,
I think we can.  It's into unchartered territory what restrictions
S/380 actually has in its current form.  It's known that it's enough to
run 31-bit GCC using 20 MB of memory though.  Which again, is a
damn good start.

BFN.  Paul.

Re: i370 port

[Daniel Jacobowitz]

On Sat, Jun 06, 2009 at 01:39:07AM +1000, Paul Edwards wrote:
>> I understand current GCC supports various source and target character
>> sets a lot better out of the box, so it may be EBCDIC isn't even an
>> issue any more.
>
> It looks that way from what I've seen of 3.4.6 so far.  However, I
> won't know for sure until it's on the host and self-generating.

Why are you migrating to 3.4.6 now, instead of to a current version?
If you want to include this in mainline some day, then eventually it
has to be caught up - and 3.4.6 is older than it may appear from the
release date, since it branched off of mainline five years ago.  A lot
has changed since then.

-- 
Daniel Jacobowitz
CodeSourcery

Re: i370 port

[Paul Edwards]

>>> I understand current GCC supports various source and target character
>>> sets a lot better out of the box, so it may be EBCDIC isn't even an
>>> issue any more.
>>
>> It looks that way from what I've seen of 3.4.6 so far.  However, I
>> won't know for sure until it's on the host and self-generating.
> 
> Why are you migrating to 3.4.6 now, instead of to a current version?
> If you want to include this in mainline some day, then eventually it
> has to be caught up - and 3.4.6 is older than it may appear from the
> release date, since it branched off of mainline five years ago.  A lot
> has changed since then.

3.4.6 made some revamps to the i370 port (compared to 3.2.3), and I
need to make sure those changes have been digested, and no code
has been lost, so that I can pick up the final i370 port and move it.

It's less daunting to get 3.4.6 working first.  At least there the target
actually exists and does obstensibly appear to work!

Migrating from 3.4.6 to 4.x is not going to be made any bigger a deal.
It's not like someone else is making incompatible changes to the
i370 port at the same time as me!

BFN.  Paul.

Re: i370 port

[Joseph S. Myers]

On Sat, 6 Jun 2009, Paul Edwards wrote:

> 3.4.6 made some revamps to the i370 port (compared to 3.2.3), and I
> need to make sure those changes have been digested, and no code
> has been lost, so that I can pick up the final i370 port and move it.

But there were probably also (mechanical) changes to the port between when 
3.4 branched (long before 3.4.6 was released) and when the port was 
removed from trunk - that's why the last revision before it was removed 
from trunk may be a better starting point.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: i370 port

[Paul Edwards]

>> 3.4.6 made some revamps to the i370 port (compared to 3.2.3), and I
>> need to make sure those changes have been digested, and no code
>> has been lost, so that I can pick up the final i370 port and move it.
>
> But there were probably also (mechanical) changes to the port between when
> 3.4 branched (long before 3.4.6 was released) and when the port was
> removed from trunk - that's why the last revision before it was removed
> from trunk may be a better starting point.

Ok, I understand now.  So there were some changes, that would
nominally have made it to GCC 4, but were in fact never officially
released, because it was dropped before release.

So, prior to starting a GCC 4 port (ie the changes may not be
desirable for the GCC 3.4.6 port I am currently working on), I
need to get GCC 3.4 as a base, GCC 3.4.6 as one derivative,
and SVN 77215, then do a 3-way diff/merge to obtain the
"nominal GCC 4" changes.

Or perhaps not.

I don't want the 3.4.6 changes at that point, since anything of
value will be covered by SVN 77215.  So I need to use GCC 3.4.6
as the base, my personal version as one derivative, SVN 77215
as the second derivative, and feed that into the 3-way diff.

Ok, I'll do that when I'm in a position to do the GCC 4 port
attempt.  I'm still months away from completing the GCC 3.4.6
port, and there are other MVS-related projects that are more
important than what is basically a transparent 3.2.3 to 4
upgrade that will only start being useful when it enables something
else to happen (such as the PL/1 front-end).

So I'll be working on that stable 3.4.6 before taking my chances
with what is basically an axed beta (SVN 77215), with still no
indication of whether even a perfectly working self-compiling
i370 target will be accepted unless the testsuite is working first
(and even if it was working, that still may not be enough - as
the next hoop may be an s390 merge - and a requirement to
switch from 370 to 390).

BFN.  Paul.

Re: i370 port

[Paul Edwards]

> Why are you migrating to 3.4.6 now, instead of to a current version?
> If you want to include this in mainline some day, then eventually it
> has to be caught up - and 3.4.6 is older than it may appear from the
> release date, since it branched off of mainline five years ago.  A lot
> has changed since then.

A question about those changes.

One of the things that I experienced when porting 3.2.3 to MVS
was that GCC was sensitive to the exact floating point representation.

Register selection was different depending on those slight differences.

Below is what documentation I have for it.  Dave Edwards, who
wrote another S/370 emulator, was the one who discovered that.

Does anyone know if that was changed somewhere along the line?

BFN.  Paul.



17. The assembler code generated by gccmvs when run on the
PC is slightly different (even when the same parameters
are used for code generation) from that when run on the
mainframe, if -O2 is used instead of -Os. But functionally
equivalent. This non-deterministic nature of the compiler
is disconcerting. It seems to not always allocate registers
consistently. This has been traced to floating point code
in predict.c and local-alloc.c which is sensitive to the
very small changes in floating point representation. This
should be changed to include deltas when comparing floating
point values. Here's an example of what's happening:

*** c-lex.s Mon Jan 14 20:48:35 2008
--- temp.dat Mon Jan 14 21:14:04 2008
***************
*** 1328,1335 ****
           SLR   15,15
           STC   15,0(3,4)
           SLR   6,6
-          LR    9,6
           LR    8,6
           L     2,192(13)
           CLR   2,5
           BNL   L303
--- 1328,1335 ----
           SLR   15,15
           STC   15,0(3,4)
           SLR   6,6
           LR    8,6
+          LR    9,6
           L     2,192(13)
           CLR   2,5
           BNL   L303

Re: i370 port

[Ulrich Weigand]

Paul Edwards wrote:

> One of the things that I experienced when porting 3.2.3 to MVS
> was that GCC was sensitive to the exact floating point representation.
> 
> Register selection was different depending on those slight differences.
> 
> Below is what documentation I have for it.  Dave Edwards, who
> wrote another S/370 emulator, was the one who discovered that.
> 
> Does anyone know if that was changed somewhere along the line?
> 
> BFN.  Paul.
> 
> 
> 
> 17. The assembler code generated by gccmvs when run on the
> PC is slightly different (even when the same parameters
> are used for code generation) from that when run on the
> mainframe, if -O2 is used instead of -Os. But functionally
> equivalent. This non-deterministic nature of the compiler
> is disconcerting. It seems to not always allocate registers
> consistently. This has been traced to floating point code
> in predict.c and local-alloc.c which is sensitive to the
> very small changes in floating point representation. This
> should be changed to include deltas when comparing floating
> point values. Here's an example of what's happening:

I agree that GCC output should not depend on details of the host
floating point representation.  (Ideally, the output of GCC built
as a cross-compiler should not depend on the host architecture
at all.)

However, it is hard to say whether such observations made on a
GCC 3.2 code base have any relevance to the current code -- for
example, local-alloc.c does not even exist any more, we now have
a completely new register allocator.

I'd recommend you go ahead with a port to current mainline and
verify whether you still see problems along those lines; if so,
it would be appropriate to open a bug report against GCC.

Bye,
Ulrich

-- 
  Dr. Ulrich Weigand
  GNU Toolchain for Linux on System z and Cell BE
  Ulrich.Weigand@de.ibm.com

Re: i370 port

[Paul Edwards]

Hi guys.

The last class of warning I have from the machine definition is this:

./config/i370/i370.md:784: warning: destination operand 0 allows non-lvalue

which is because I have used r_or_s_operand like this:

;
; movdi instruction pattern(s).
;

(define_insn ""
  [(set (match_operand:DI 0 "r_or_s_operand" "=dm,dS")
        (match_operand:DI 1 "r_or_s_operand" "dm*fF,i"))]
  "TARGET_CHAR_INSTRUCTIONS"
  "*
{
  check_label_emit ();
  if (REG_P (operands[0]))

and I believe what is happening is that r_or_s_operand allows
a constant.

But sometimes r_or_s_operand is being used as a source, in
which case, the constant is fine.  But when it is used as a
destination, it is not fine.

What is the *simplest* way of changing the setup so that the
code generation remains the same, but the warning is eliminated?

Note that this is 3.2.3 and I'm not wanting to restructure the
whole thing to be the same as s390.md.  I am thinking I can
define something like r_or_s_lval_operand which would be
the same as r_or_s_operand except not allow constants.

But wondering if there's something simpler and neater.

Thanks.  Paul.

Re: i370 port

[Ulrich Weigand]

Paul Edwards wrote:

> But sometimes r_or_s_operand is being used as a source, in
> which case, the constant is fine.  But when it is used as a
> destination, it is not fine.
> 
> What is the *simplest* way of changing the setup so that the
> code generation remains the same, but the warning is eliminated?

Well, I guess you need to do two things:

- Define the PREDICATE_CODES macro -- if this is undefined,
  genrecog will consider *any* target-defined predicate as
  potentially allowing non-lvalues, because it cannot know
  better.

- Actually define two distinct predicates, one that allows
  non-lvalue and one that doesn't, and use them as appropriate.

> But wondering if there's something simpler and neater.

Well, you could also simply ignore the warning -- nothing
is going to go wrong because of it.

Bye,
Ulrich

-- 
  Dr. Ulrich Weigand
  GNU Toolchain for Linux on System z and Cell BE
  Ulrich.Weigand@de.ibm.com

Re: i370 port

[Paul Edwards]

>> But sometimes r_or_s_operand is being used as a source, in
>> which case, the constant is fine.  But when it is used as a
>> destination, it is not fine.
>> 
>> What is the *simplest* way of changing the setup so that the
>> code generation remains the same, but the warning is eliminated?
> 
> Well, I guess you need to do two things:
> 
> - Define the PREDICATE_CODES macro -- if this is undefined,
>  genrecog will consider *any* target-defined predicate as
>  potentially allowing non-lvalues, because it cannot know
>  better.
> 
> - Actually define two distinct predicates, one that allows
>  non-lvalue and one that doesn't, and use them as appropriate.
> 
>> But wondering if there's something simpler and neater.
> 
> Well, you could also simply ignore the warning -- nothing
> is going to go wrong because of it.

Thanks for pointing me in the right direction Ulrich.  Unfortunately
just short of what would have been best for this situation.

I defined the PREDICATE_CODES macro and put in the things
I thought were appropriate:

C:\devel\gcc\gcc\config\i370>cvs diff -r 1.33 -r 1.34 i370.h
Index: i370.h
===================================================================
RCS file: c:\cvsroot/gcc/gcc/config/i370/i370.h,v
retrieving revision 1.33
retrieving revision 1.34
diff -r1.33 -r1.34
200a201,204
> #define PREDICATE_CODES \
>   {"r_or_s_operand", { REG, SUBREG, MEM, CONST_INT }}, \
>   {"s_operand", { MEM, CONST_INT }},
>

It had no effect on anything that I could see, but that was to be expected.

I then had a closer look at the machine definitions and realised that some
of them that were defined as r_or_s_operand could instead be
nonimmediate_operand, which started eliminating warnings.

So I proceeded down this track, eliminating things here and there, or
in some cases, opening things up to be more general, with the hope
that I would eventually have things so that the only r_or_s_operand
I needed were ones that didn't require literals, so that I could (at the
end), make this change:

C:\devel\gcc\gcc\config\i370>cvs diff -r 1.34 -r 1.35 i370.h
Index: i370.h
===================================================================
RCS file: c:\cvsroot/gcc/gcc/config/i370/i370.h,v
retrieving revision 1.34
retrieving revision 1.35
diff -r1.34 -r1.35
202,203c202,203
<   {"r_or_s_operand", { REG, SUBREG, MEM, CONST_INT }}, \
<   {"s_operand", { MEM, CONST_INT }},
---
>   {"r_or_s_operand", { REG, SUBREG, MEM }}, \
>   {"s_operand", { MEM }},

and something similar in i370.c, to make constants invalid, so that I
could eliminate the warnings.

It took a month to do that, because I kept on being hit by presumed
bugs, which started generating bad or unexpected code when I made
a seemingly innocuous change.  To make matters worse, sometimes
I could only find out whether the code was good or not by running it on
MVS, via the emulator, which means a 2 hour wait for a result.

However, I did get it down to just a handful of warnings, which would
be eliminated now that I could drop the CONST_INT.  And I would
check the generated code to see what I had missed when I took
off the CONST_INT.

Anyway, I took off the CONST_INT, and the warnings all disappeared,
and the code didn't change!

I then found out that even with old versions of the machine definition,
I can have the warning removed by simply not defining CONST_INT
in the PREDICATE_CODES, even though it is allowed when the
function is called.  ie it seems to have no effect on the code
generation, but succeeds in eliminating the warning, and without
needing to define an extra constraint for non-constant situations.

So I've revamped the machine definition unnecessarily.  Well, I
did make things defined more consistently, and did make code
generation improvements, but they're not major, and I wouldn't
have done any of it if I knew I could have just defined a predicate
that wasn't really going to be used.

Oh well.  At the end of the day, the warning has gone, the code
is better and the machine definition is more correct.  :-)

I've also got 3.4.6 working to some extent.  I have managed to
build an single GCC executable, and if I call it with no parameters,
it prints its usage (as designed).

However, if I try to pass parameters it doesn't recognize them.
It could be something to do with not having run the appropriate
stuff through bison (or flex) on an EBCDIC platform.

Normally I use this to get around that problem:

C:\devel\gccnew\gcc>cvs diff -r release-3_4_6 c-parse.c
Index: c-parse.c
===================================================================
RCS file: c:\cvsroot/gccnew/gcc/c-parse.c,v
retrieving revision 1.1.1.1
retrieving revision 1.4
diff -r1.1.1.1 -r1.4
497a498,503
> #if defined(__MVS__) || defined(__CMS__)
> #define YYTRANSLATE(YYX)                      \
>   ((unsigned int) (YYX) <= YYMAXUTOK ? \
>   ((unsigned int) (YYX) < 256 ? yytranslate[_sch_ebcasc[YYX]] \
>   : yytranslate[YYX]) : YYUNDEFTOK)
> #else
499a506
> #endif

But perhaps the new gengtyp lex and yacc, although not used in the
actual GCC executable, are causing a problem by being run on the
ASCII machine.

Next step is to see if I can run the generated files all on MVS so
that that is eliminated from the equation.  Since I actually do have
an EBCDIC flex and bison available now (thanks to 3.2.3).

I also had problems with 3.4.6 when I switched on optimization.
A couple of internal errors popped up, which will presumably
trace back to the i370 code, but be outside my knowledge to
fix.

BFN.  Paul.

Re: i370 port

[Ulrich Weigand]

Paul Edwards wrote:

> I then found out that even with old versions of the machine definition,
> I can have the warning removed by simply not defining CONST_INT
> in the PREDICATE_CODES, even though it is allowed when the
> function is called.  ie it seems to have no effect on the code
> generation, but succeeds in eliminating the warning, and without
> needing to define an extra constraint for non-constant situations.

Actually PREDICATE_CODES does have to match the predicate definitions.
If it does not, you can run into subtle bugs, as the code in genrecog.c
that generates the finite state automaton matching an RTX pattern
against the .md insn definitions *does* make use of PREDICATE_CODES;
for example, it will assume that two predicates with disjoint sets
of PREDICATE_CODES can never both match the same RTX.

This may or may not lead to visible differences when compiling simple
test cases, but I'd expect effects to be visible in more complex
scenarios.

Bye,
Ulrich

-- 
  Dr. Ulrich Weigand
  GNU Toolchain for Linux on System z and Cell BE
  Ulrich.Weigand@de.ibm.com

Re: i370 port

[Paul Edwards]

>> I then found out that even with old versions of the machine definition,
>> I can have the warning removed by simply not defining CONST_INT
>> in the PREDICATE_CODES, even though it is allowed when the
>> function is called.  ie it seems to have no effect on the code
>> generation, but succeeds in eliminating the warning, and without
>> needing to define an extra constraint for non-constant situations.
>
> Actually PREDICATE_CODES does have to match the predicate definitions.
> If it does not, you can run into subtle bugs, as the code in genrecog.c
> that generates the finite state automaton matching an RTX pattern
> against the .md insn definitions *does* make use of PREDICATE_CODES;
> for example, it will assume that two predicates with disjoint sets
> of PREDICATE_CODES can never both match the same RTX.
>
> This may or may not lead to visible differences when compiling simple
> test cases, but I'd expect effects to be visible in more complex
> scenarios.

Thanks Ulrich.  When I went back to this I found that I had
misdiagnosed - actually the r_or_s didn't allow constants after
all.  It was only const_rtx that it allowed.  So the machine definition
no longer has any warnings and all looks good.

That's with 3.2.3.

With 3.4.6 I have now managed to get an MVS load module,
unoptimized (I already know that optimized doesn't work), to
compile a hello world program successfully, although it abends
at the end of that and I haven't investigated that yet.

So the theoretical EBCDIC support is less theoretical now.
I "needed" to change the parameter search algorithm to stop
being binary search though.

GCC 4 complained (on my Linux system) that I didn't have
various things (gimp etc) installed, which means I would need
that other software to be ported too, which is not a project
I want to work on at the moment.  However, at least it means
that i have successfully merged all the GCCMVS changes
to 3.2.3 in with the (last available) 3.4.6 development, which
was a precursor to any GCC 4 port anyway.  I'll see over time
how GCC 3.4.6 pans out.

Until then, back at GCC 3.2.3, I have a "need" to make the entry
point 0 in my MVS modules.

Currently I generate this:

         COPY  PDPTOP
         CSECT
* X-var bytes
         ENTRY BYTES
* Program data area
         DS    0F
BYTES    EQU   *
         DC    F'258'
* Program text area
LC0      EQU   *
         DC    C'bytes is %d'
         DC    X'15'
         DC    X'0'
         DS    0F
         DC    C'GCCMVS!!'
         EXTRN @@CRT0
         ENTRY @@MAIN
@@MAIN   DS    0H
         USING *,15
         L     15,=V(@@CRT0)
         BR    15
         DROP  15
         LTORG
* X-func main prologue
MAIN     PDPPRLG CINDEX=0,FRAME=96,BASER=12,ENTRY=YES
...
* Function main page table
         DS    0F
PGT0     EQU   *
         DC    A(PG0)
         END   @@MAIN

for a main program.  In order to get the entry point to 0, I need to move 
that
@@MAIN function to the top of the file.

But I only really want that function if this is a main program.  I have 
found
somewhere where I can add some code to see if the function "main" is
being compiled, and set a global variable.

Unfortunately, the place that happens (c-decl) isn't called until after all
the data has been written out!

I looked at the documentation:

http://gcc.gnu.org/onlinedocs/gcc-3.2.3/gccint/Assembler-Format.html#Assembler%20Format

but maybe I'm looking in the wrong place.  Any ideas?

And I have another question - where is the code for __asm__?

Currently that is generating garbage for me:

unsigned int bytes = 258;

__asm__("? :");

int main(void)

BYTES    EQU   *
         DC    F'258'
         o@z
* Program text area

when done in cross-compiler mode, and need to find out where
the literal is being translated (or more likely - failing to be
translated in the first instance).  Any idea where that is?

And final thing - the interest in the __asm__ comes from the hope
that in the generated 370 assembler, it would be possible to have
the C code interspersed to whatever extent possible.  The plan was
to basically somehow get every line of C code, e.g. "int main(void)"
above, and automatically generate an:
__asm__("int main void)");

although I'm not sure what to do about this:

int main(void)
__asm__("? :");
{

which gives a syntax error.  Any idea how to get the mixed
C/assembler when I'm running with the "-S" option and don't
have the luxury of calling the normal "as" which would
normally handle that?

Thanks.  Paul.

Re: i370 port

[Ulrich Weigand]

Paul Edwards wrote:

> GCC 4 complained (on my Linux system) that I didn't have
> various things (gimp etc) installed, which means I would need
> that other software to be ported too, which is not a project
> I want to work on at the moment.  However, at least it means
> that i have successfully merged all the GCCMVS changes
> to 3.2.3 in with the (last available) 3.4.6 development, which
> was a precursor to any GCC 4 port anyway.  I'll see over time
> how GCC 3.4.6 pans out.

This probably is not gimp (the graphics editor) but gmp (the
multi-precision integer operation library) and mpfr (same for
floating-point).  To build any recent GCC you'll indeed need
these two libraries.  Fortunately, they're already available
on s390(x) on Linux, and shouldn't really contain anything
that is OS-specific, so porting those to MVS should hopefully
be straightforward ...

> Until then, back at GCC 3.2.3, I have a "need" to make the entry
> point 0 in my MVS modules.

> Currently I generate this:
[snip]
> for a main program.  In order to get the entry point to 0, I need to move 
> that
> @@MAIN function to the top of the file.

I don't think there is a reliable way to change the sequence of
functions / objects in the output file.

However, it seems to me it shouldn't really be necessary to treat "main"
special.  Usually, the "entry point" isn't really "main", but rather some
function in crt0.o, which then in turn *calls* main after all startup
processing has been done.  As crt0.o can be (and usually is) completely
written in assembler, you can arrange for everything to be in the sequence
that is required.  (On the linker command line, crt0.o would be placed
first, so if the entry point is the first thing in crt0.o it will then
also be the first thing in the executable.)

> And I have another question - where is the code for __asm__?
> 
> Currently that is generating garbage for me:
> 
> unsigned int bytes = 258;
> 
> __asm__("? :");
> 
> int main(void)
> 
> BYTES    EQU   *
>          DC    F'258'
>          o@z
> * Program text area
> 
> when done in cross-compiler mode, and need to find out where
> the literal is being translated (or more likely - failing to be
> translated in the first instance).  Any idea where that is?

Hmm, this seems to be the bug fixed by Eric Christopher in 2004:
http://gcc.gnu.org/ml/gcc-cvs/2004-02/msg01425.html


> And final thing - the interest in the __asm__ comes from the hope
> that in the generated 370 assembler, it would be possible to have
> the C code interspersed to whatever extent possible.  The plan was
> to basically somehow get every line of C code, e.g. "int main(void)"
> above, and automatically generate an:
> __asm__("int main void)");

As you note, this doesn't seem workable as the result wouldn't
be syntactically valid ...

> which gives a syntax error.  Any idea how to get the mixed
> C/assembler when I'm running with the "-S" option and don't
> have the luxury of calling the normal "as" which would
> normally handle that?

There doesn't seem to be an easy way to do this from the
compiler.  At the time compiler output is generated, the
original source code text isn't even kept any more; you'd
have to go back and re-read the original source files using
the line-number debug data, just as the assembler would ...

Bye,
Ulrich

-- 
  Dr. Ulrich Weigand
  GNU Toolchain for Linux on System z and Cell BE
  Ulrich.Weigand@de.ibm.com

Re: i370 port

[Paul Edwards]

> This probably is not gimp (the graphics editor) but gmp (the
> multi-precision integer operation library) and mpfr (same for
> floating-point).  To build any recent GCC you'll indeed need
> these two libraries.  Fortunately, they're already available
> on s390(x) on Linux, and shouldn't really contain anything
> that is OS-specific, so porting those to MVS should hopefully
> be straightforward ...

Ok, thanks Ulrich.

>> Until then, back at GCC 3.2.3, I have a "need" to make the entry
>> point 0 in my MVS modules.
>
>> Currently I generate this:
> [snip]
>> for a main program.  In order to get the entry point to 0, I need to move
>> that
>> @@MAIN function to the top of the file.
>
> I don't think there is a reliable way to change the sequence of
> functions / objects in the output file.

Sorry, my question may not have been clear.

When I see main(), I generate the normal MAIN code, and I don't
care where this goes.

However, given that I now know that a main() exists somewhere
in this source file, I need to change the ASM_FILE_START
output.

I expected that the assembler generation wouldn't happen until
after the optimization was completed, and thus, by then, I
would know whether this was a main.

> However, it seems to me it shouldn't really be necessary to treat "main"
> special.  Usually, the "entry point" isn't really "main", but rather some
> function in crt0.o, which then in turn *calls* main after all startup
> processing has been done.

That is roughly the case, yes.

> As crt0.o can be (and usually is) completely
> written in assembler, you can arrange for everything to be in the sequence
> that is required.  (On the linker command line, crt0.o would be placed
> first, so if the entry point is the first thing in crt0.o it will then
> also be the first thing in the executable.)

Yes, I can do that, but that means I need to have a linker command
line!  The way the MVS linker works, I can link my main program,
(which obviously doesn't have crt0 in it), and, thanks to the "END"
statement, I can specify an entry point.  This means no complaint
from the linker about a default (and wrong) entry point, and no
need for any linker statements.  It all automatically resolves.

It all works really great!

Except - I would ideally like to have an entry point of 0, instead
of the typical "58" or whatever my static variables happen to be.
(my main is normally at the top).

>> And I have another question - where is the code for __asm__?
>>
>> Currently that is generating garbage for me:
>>
>> unsigned int bytes = 258;
>>
>> __asm__("? :");
>>
>> int main(void)
>>
>> BYTES    EQU   *
>>          DC    F'258'
>>          o@z
>> * Program text area
>>
>> when done in cross-compiler mode, and need to find out where
>> the literal is being translated (or more likely - failing to be
>> translated in the first instance).  Any idea where that is?
>
> Hmm, this seems to be the bug fixed by Eric Christopher in 2004:
> http://gcc.gnu.org/ml/gcc-cvs/2004-02/msg01425.html

Thanks again!  That didn't seem to make it into 3.4.6.  I was able
to apply most of his stuff to 3.4.6, and I will see how far that gets
me, before trying to find it in 3.2.3.

Or maybe I'll skip it, since the problem doesn't occur on a pure
EBCDIC system.  :-)

>> And final thing - the interest in the __asm__ comes from the hope
>> that in the generated 370 assembler, it would be possible to have
>> the C code interspersed to whatever extent possible.  The plan was
>> to basically somehow get every line of C code, e.g. "int main(void)"
>> above, and automatically generate an:
>> __asm__("int main void)");
>
> As you note, this doesn't seem workable as the result wouldn't
> be syntactically valid ...
>
>> which gives a syntax error.  Any idea how to get the mixed
>> C/assembler when I'm running with the "-S" option and don't
>> have the luxury of calling the normal "as" which would
>> normally handle that?
>
> There doesn't seem to be an easy way to do this from the
> compiler.  At the time compiler output is generated, the
> original source code text isn't even kept any more; you'd
> have to go back and re-read the original source files using
> the line-number debug data, just as the assembler would ...

Ok, well - I would be content with just the source line number
appearing in the output assembler.  Is this information
available as each assembler line is output?

Thanks.  Paul.

Re: i370 port

[Ulrich Weigand]

Paul Edwards wrote:

> I expected that the assembler generation wouldn't happen until
> after the optimization was completed, and thus, by then, I
> would know whether this was a main.

That depends a bit on the compiler version and optimization level,
but (in particular in the 3.x time frame) GCC may output assembler
code on a function-by-function basis, without necessarily reading
in the whole source file first.

As I said, it seems the best way would be to not have care at all
whether or not any particular source file contains a main routine,
but instead do all entry-point processing in the crt0 startup file.

> > As crt0.o can be (and usually is) completely
> > written in assembler, you can arrange for everything to be in the sequence
> > that is required.  (On the linker command line, crt0.o would be placed
> > first, so if the entry point is the first thing in crt0.o it will then
> > also be the first thing in the executable.)
> 
> Yes, I can do that, but that means I need to have a linker command
> line!  The way the MVS linker works, I can link my main program,
> (which obviously doesn't have crt0 in it), and, thanks to the "END"
> statement, I can specify an entry point.  This means no complaint
> from the linker about a default (and wrong) entry point, and no
> need for any linker statements.  It all automatically resolves.

I don't know exactly how your port handles this on MVS, but usually
GCC itself will invoke the linker, and will itself prepare an
appropriate command linker for the linker.  As part of this command
line, things like crt files will be specified.  You should set the
STARTFILE_SPEC macro in your back-end to do that ...

> > There doesn't seem to be an easy way to do this from the
> > compiler.  At the time compiler output is generated, the
> > original source code text isn't even kept any more; you'd
> > have to go back and re-read the original source files using
> > the line-number debug data, just as the assembler would ...
> 
> Ok, well - I would be content with just the source line number
> appearing in the output assembler.  Is this information
> available as each assembler line is output?

In current GCC, every insn contains "location" information pointing
back to source code line (and column) numbers.  However, in GCC 3.x
I think this wasn't yet present, but you had to rely on line number
notes interspersed with the insn stream.

In any case, if you build with -g and use in addition the "debug
assembler output" flag -dA the assembler output should contain
human-readable comments containing line number information.


Bye,
Ulrich

-- 
  Dr. Ulrich Weigand
  GNU Toolchain for Linux on System z and Cell BE
  Ulrich.Weigand@de.ibm.com

Re: i370 port

[Paul Edwards]

> That depends a bit on the compiler version and optimization level,
> but (in particular in the 3.x time frame) GCC may output assembler
> code on a function-by-function basis, without necessarily reading
> in the whole source file first.

Ok, actually it doesn't matter if it doesn't work all the time.  I'll
always be compiling with -Os anyway, so it sounds like I'm in
with a chance of the whole file being read first?

If so, where is my first opportunity, in 3.2.3, to see if there's a
"main" function in this file?

>> > As crt0.o can be (and usually is) completely
>> > written in assembler, you can arrange for everything to be in the 
>> > sequence
>> > that is required.  (On the linker command line, crt0.o would be placed
>> > first, so if the entry point is the first thing in crt0.o it will then
>> > also be the first thing in the executable.)
>>
>> Yes, I can do that, but that means I need to have a linker command
>> line!  The way the MVS linker works, I can link my main program,
>> (which obviously doesn't have crt0 in it), and, thanks to the "END"
>> statement, I can specify an entry point.  This means no complaint
>> from the linker about a default (and wrong) entry point, and no
>> need for any linker statements.  It all automatically resolves.
>
> I don't know exactly how your port handles this on MVS, but usually
> GCC itself will invoke the linker, and will itself prepare an
> appropriate command linker for the linker.

GCC on MVS *only* outputs assembler.  ie you always have to
use the "-S" option.

By doing so, it turns GCC into a pure text-processing application,
which will thus work in any C90 environment.

> In current GCC, every insn contains "location" information pointing
> back to source code line (and column) numbers.  However, in GCC 3.x
> I think this wasn't yet present, but you had to rely on line number
> notes interspersed with the insn stream.
>
> In any case, if you build with -g and use in addition the "debug
> assembler output" flag -dA the assembler output should contain
> human-readable comments containing line number information.

The GCC assembler is never invoked.  After getting the assembler
output from the -S option, this is fed into IFOX00/IEV90/ASMA90.

Regardless, if line number notes are interspersed in the stream,
maybe whenever I write out an assembler instruction I will have
access to those notes and can print out a comment.

Thanks.  Paul.

Re: i370 port

[Paolo Bonzini]

>> In any case, if you build with -g and use in addition the "debug
>> assembler output" flag -dA the assembler output should contain
>> human-readable comments containing line number information.
>
> Regardless, if line number notes are interspersed in the stream,
> maybe whenever I write out an assembler instruction I will have
> access to those notes and can print out a comment.

No, that might break when you upgrade later.  GCC already has support 
for annotating the assembly output with human-readable line number info. 
  That is option -dA as Ulrich pointed out.

Only DWARF-2 output supports it currently, but if you want to use it say 
together with STABS, it is just a matter of modifying dbxout_source_line 
and add a single statement like this:

       if (flag_debug_asm)
         fprintf (asm_out_file, "\t%s %s:%d\n", ASM_COMMENT_START,
                  filename, line);

Paolo

Re: i370 port

[Ulrich Weigand]

Paul Edwards wrote:
> > That depends a bit on the compiler version and optimization level,
> > but (in particular in the 3.x time frame) GCC may output assembler
> > code on a function-by-function basis, without necessarily reading
> > in the whole source file first.
> 
> Ok, actually it doesn't matter if it doesn't work all the time.  I'll
> always be compiling with -Os anyway, so it sounds like I'm in
> with a chance of the whole file being read first?
> 
> If so, where is my first opportunity, in 3.2.3, to see if there's a
> "main" function in this file?

Hmm, it seems 3.2.x would *always* operate on a function-by-function
basis.  The unit-at-a-time mode was only introduced with 3.4 (I don't
recall if it was already present in 3.3).  I don't think there is any
way in 3.2.3 to check whether there is a "main" function in the file
before it is processed ...

> > I don't know exactly how your port handles this on MVS, but usually
> > GCC itself will invoke the linker, and will itself prepare an
> > appropriate command linker for the linker.
> 
> GCC on MVS *only* outputs assembler.  ie you always have to
> use the "-S" option.
> 
> By doing so, it turns GCC into a pure text-processing application,
> which will thus work in any C90 environment.

Huh.  So the user will always have to invoke the linker manually, and
pass all the correct options (libraries etc.)?

What is the problem with having GCC itself invoke the linker, just like
it does on other platforms?

> > In current GCC, every insn contains "location" information pointing
> > back to source code line (and column) numbers.  However, in GCC 3.x
> > I think this wasn't yet present, but you had to rely on line number
> > notes interspersed with the insn stream.
> >
> > In any case, if you build with -g and use in addition the "debug
> > assembler output" flag -dA the assembler output should contain
> > human-readable comments containing line number information.
> 
> The GCC assembler is never invoked.  After getting the assembler
> output from the -S option, this is fed into IFOX00/IEV90/ASMA90.

As Paolo mentioned, the -dA flag is an option to the *compiler* that
causes it to place additional information into its output stream
(the assembler source code).


Bye,
Ulrich

-- 
  Dr. Ulrich Weigand
  GNU Toolchain for Linux on System z and Cell BE
  Ulrich.Weigand@de.ibm.com

Re: i370 port

[Paul Edwards]

Thanks guys for your reply.

> As Paolo mentioned, the -dA flag is an option to the *compiler* that
> causes it to place additional information into its output stream
> (the assembler source code).

[from Paolo]

> Only DWARF-2 output supports it currently, but if you want to use it say 
> together with STABS, it is just a matter of modifying dbxout_source_line 
> and add a single statement like this:
> 
>       if (flag_debug_asm)
>         fprintf (asm_out_file, "\t%s %s:%d\n", ASM_COMMENT_START,
>                  filename, line);

Ok, after a few false leads, I found something that produced a
pleasing result.  Fantastic I should say!

With this mod:

C:\devel\gcc\gcc>cvs diff debug.c
Index: debug.c
===================================================================
RCS file: c:\cvsroot/gcc/gcc/debug.c,v
retrieving revision 1.1.1.1
diff -r1.1.1.1 debug.c
20a21,31
> #include "output.h"
> #include "flags.h"
>
> void
> fff (line, text)
>      unsigned int line ATTRIBUTE_UNUSED;
>      const char *text ATTRIBUTE_UNUSED;
> {
>    if (flag_debug_asm)
>    fprintf(asm_out_file, "fff %d %s\n", line, text);
> }
34c45
<   debug_nothing_int_charstar, /* source_line */
---
>   fff /*debug_nothing_int_charstar*/, /* source_line */

and the -g -dA options

I was able to get exactly what I needed:

* Function main code
fff 32 world.c
         * basic block 0
         L     2,=V(X)
         L     2,0(2)
         LTR   2,2
         BE    L2
fff 34 world.c
         * basic block 1
         MVC   88(4,13),=A(LC0)
         B     L4
L2       EQU   *
         * basic block 2
fff 38 world.c
         MVC   88(4,13),=A(LC1)
L4       EQU   *
         * basic block 3
         LA    1,88(,13)
         L     15,=V(PRINTF)
         BALR  14,15
fff 41 world.c
         SLR   15,15
* Function main epilogue

But let me guess - I'm not allowed to modify debug.c and have
to decide whether MVS is an sdb, xcoff, dbx, dwarf 1/2, or vms?

>> > That depends a bit on the compiler version and optimization level,
>> > but (in particular in the 3.x time frame) GCC may output assembler
>> > code on a function-by-function basis, without necessarily reading
>> > in the whole source file first.
>> 
>> Ok, actually it doesn't matter if it doesn't work all the time.  I'll
>> always be compiling with -Os anyway, so it sounds like I'm in
>> with a chance of the whole file being read first?
>> 
>> If so, where is my first opportunity, in 3.2.3, to see if there's a
>> "main" function in this file?
> 
> Hmm, it seems 3.2.x would *always* operate on a function-by-function
> basis.  The unit-at-a-time mode was only introduced with 3.4 (I don't
> recall if it was already present in 3.3).  I don't think there is any
> way in 3.2.3 to check whether there is a "main" function in the file
> before it is processed ...

Ok, I'll wait until 3.4.6 is working before attempting to get
the entry point to 0 then.  :-)

>> > I don't know exactly how your port handles this on MVS, but usually
>> > GCC itself will invoke the linker, and will itself prepare an
>> > appropriate command linker for the linker.
>> 
>> GCC on MVS *only* outputs assembler.  ie you always have to
>> use the "-S" option.
>> 
>> By doing so, it turns GCC into a pure text-processing application,
>> which will thus work in any C90 environment.
> 
> Huh.  So the user will always have to invoke the linker manually, and
> pass all the correct options (libraries etc.)?

Correct.  Very normal MVS.  Where would we be without IEWL?

> What is the problem with having GCC itself invoke the linker, just like
> it does on other platforms?

1. That requires extensions to the C90 standard.  The behaviour of
system() is undefined, much less even the existence of fork() etc.

2. The attempt to add functionality to system() in MVS has made
leaps and bounds, but has not reached the stage of being able
to detect if the SYSPRINT DCB is already open so it knows not
to reopen it, and close it, stuffing up the caller.

3. MVS compilers don't normally invoke the linker.  That's always
a separate step.  GCCMVS is an MVS compiler also.  It would
be normal to generate object code though.  But the compiler
normally generates the object code, rather than invoking the
assembler.  In any case, the facility to allocate temporary
datasets for HLASM and deciding what space parameters
should be used etc etc has not yet been determined, and is
unwieldy regardless, and the functionality doesn't exist yet
anyway, and may be years away still, if it even makes sense.

BFN.  Paul.

Re: i370 port

[Paolo Bonzini]

> Ok, after a few false leads, I found something that produced a
> pleasing result. Fantastic I should say!
>
> With this mod:
>
> C:\devel\gcc\gcc>cvs diff debug.c
> Index: debug.c
> ===================================================================
> RCS file: c:\cvsroot/gcc/gcc/debug.c,v
> retrieving revision 1.1.1.1
> diff -r1.1.1.1 debug.c
> 20a21,31
>> #include "output.h"
>> #include "flags.h"
>>
>> void
>> fff (line, text)
>> unsigned int line ATTRIBUTE_UNUSED;
>> const char *text ATTRIBUTE_UNUSED;
>> {
>> if (flag_debug_asm)
>> fprintf(asm_out_file, "fff %d %s\n", line, text);
>> }

with s/fff/ASM_COMMENT_START/ and with a proper function name, this is 
actually an almost acceptable patch.  Only, you also have to do 
app_enable/app_disable (search for the other occurrences of 
flag_debug_asm in dwarf2out.c, they also explain why this is needed). 
These probably can be moved to common code in general.

> But let me guess - I'm not allowed to modify debug.c and have
> to decide whether MVS is an sdb, xcoff, dbx, dwarf 1/2, or vms?

No, that's fine.  -dA can actually be handled by common code.

Paolo

Re: i370 port

[Ulrich Weigand]

Paul Edwards wrote:

> > What is the problem with having GCC itself invoke the linker, just like
> > it does on other platforms?
> 
> 1. That requires extensions to the C90 standard.  The behaviour of
> system() is undefined, much less even the existence of fork() etc.
> 
> 2. The attempt to add functionality to system() in MVS has made
> leaps and bounds, but has not reached the stage of being able
> to detect if the SYSPRINT DCB is already open so it knows not
> to reopen it, and close it, stuffing up the caller.
> 
> 3. MVS compilers don't normally invoke the linker.  That's always
> a separate step.  GCCMVS is an MVS compiler also.  It would
> be normal to generate object code though.  But the compiler
> normally generates the object code, rather than invoking the
> assembler.  In any case, the facility to allocate temporary
> datasets for HLASM and deciding what space parameters
> should be used etc etc has not yet been determined, and is
> unwieldy regardless, and the functionality doesn't exist yet
> anyway, and may be years away still, if it even makes sense.

I cannot really comment on what would be desirable behaviour for
MVS ...   Just as an implementation note, while it is true that
the GCC compiler driver needs to be able to execute other processes
(preprocessor, compiler, assembler, linker) as sub-tasks, it does
not require the full POSIX system/fork/exec functionality to do so.
GCC relies on the libiberty "pex" family of routines, which are
much more narrow in scope, and have in fact been ported to several
non-UNIX systems, including even MS-DOS.  Providing a port of "pex"
to MVS should be much easier that porting a full Unix "system" or
"fork" feature.

B.t.w. if you cannot execute sub-tasks at all, how does the
MVS GCC invoke the preprocessor (I think this was still a
separate process in 3.2) and the core compiler (cc1) from
the compiler driver?   Do you even have a separate compiler
driver?

Bye,
Ulrich

-- 
  Dr. Ulrich Weigand
  GNU Toolchain for Linux on System z and Cell BE
  Ulrich.Weigand@de.ibm.com

Re: i370 port

[Paul Edwards]

> GCC relies on the libiberty "pex" family of routines, which are
> much more narrow in scope, and have in fact been ported to several
> non-UNIX systems, including even MS-DOS.  Providing a port of "pex"
> to MVS should be much easier that porting a full Unix "system" or
> "fork" feature.

Ok.  But I don't even have much of system() yet.

> B.t.w. if you cannot execute sub-tasks at all, how does the
> MVS GCC invoke the preprocessor (I think this was still a
> separate process in 3.2) and the core compiler (cc1) from
> the compiler driver?   Do you even have a separate compiler
> driver?

Sleight of hand.  :-)

> #ifdef SINGLE_EXECUTABLE
>   int ret_code = 0;
> #endif
2836a2853,2872
> #ifdef SINGLE_EXECUTABLE
>      {
>          int cnt = 0;
>
>          while (commands[i].argv[cnt] != NULL)
>          {
>              cnt++;
>          }
>          if (strcmp(string, "cpp0") == 0)
>          {
>              ret_code = cpp(cnt, commands[i].argv);
>              if (ret_code != 0) break;
>          }
>          else if (strcmp(string, "cc1") == 0)
>          {
>              ret_code = toplev_main(cnt, commands[i].argv);
>              if (ret_code != 0) break;
>          }
>       }
> #else
2850c2886
<
---
> #endif
2853a2890,2892
> #ifdef SINGLE_EXECUTABLE
>     return (ret_code);


BTW, here's what I am going with:

C:\devel\gcc\gcc>cvs diff -r release-3_2_3 debug.c
Index: debug.c
===================================================================
RCS file: c:\cvsroot/gcc/gcc/debug.c,v
retrieving revision 1.1.1.1
diff -r1.1.1.1 debug.c
20a21,35
> #include "output.h"
> #include "flags.h"
>
> static void
> debug_source_line (line, text)
>      unsigned int line ATTRIBUTE_UNUSED;
>      const char *text ATTRIBUTE_UNUSED;
> {
>    if (flag_debug_asm)
>    {
>        app_enable();
>        fprintf(asm_out_file, "%s %d %s\n", ASM_COMMENT_START, line, text);
>        app_disable();
>    }
> }
34c49
<   debug_nothing_int_charstar, /* source_line */
---
>   debug_source_line,  /* source_line */


Thanks again for everyone's assistance.

Just need to find out who's emitting this warning:

C:\devel\gcc\gcc>gccmvs -DUSE_MEMMGR -Os -S -ansi -pedantic-errors -DHAVE_CONFIG
_H -DIN_GCC -DPUREISO -I ../../pdos/pdpclib -I . -I config/i370 -I 
../include -d
A world.c       -g
world.c:0: warning: `-g': unknown or unsupported -g option

and it'll be a wrap.  :-)

There will be some very happy people who are currently manually
putting in __asm__ lines to get insight.  :-)

BFN.  Paul.

Re: i370 port

[Paul Edwards]

> Hmm, it seems 3.2.x would *always* operate on a function-by-function
> basis.  The unit-at-a-time mode was only introduced with 3.4 (I don't
> recall if it was already present in 3.3).  I don't think there is any
> way in 3.2.3 to check whether there is a "main" function in the file
> before it is processed ...

Ulrich, this comment got me thinking.  If global optimization is not
being done, then less of the source file would need to be in memory
at the same time.

How much memory do you think is required to process the largest
GCC 3.2.3 source file (ie when self-compiling)?

My experience is that fold-const.c requires 20 MB of memory (not
including the size of the executable) to compile with -Os.  That's
the biggest.

Is that typical/expected?

Because it just occurred to me that maybe the lack of a "normal"
implementation of alloca() is causing memory to not be released,
and it's taking more space than it needs to.

Thanks.  Paul.

Re: i370 port

[Paolo Bonzini]

> My experience is that fold-const.c requires 20 MB of memory (not
> including the size of the executable) to compile with -Os. That's
> the biggest.
>
> Is that typical/expected?

It doesn't seem too big.

> Because it just occurred to me that maybe the lack of a "normal"
> implementation of alloca() is causing memory to not be released,
> and it's taking more space than it needs to.

It may use a little more memory, but not much.

Paolo

Re: i370 port

[Paul Edwards]

>> > That depends a bit on the compiler version and optimization level,
>> > but (in particular in the 3.x time frame) GCC may output assembler
>> > code on a function-by-function basis, without necessarily reading
>> > in the whole source file first.
>>
>> Ok, actually it doesn't matter if it doesn't work all the time.  I'll
>> always be compiling with -Os anyway, so it sounds like I'm in
>> with a chance of the whole file being read first?
>>
>> If so, where is my first opportunity, in 3.2.3, to see if there's a
>> "main" function in this file?
>
> Hmm, it seems 3.2.x would *always* operate on a function-by-function
> basis.  The unit-at-a-time mode was only introduced with 3.4 (I don't
> recall if it was already present in 3.3).  I don't think there is any
> way in 3.2.3 to check whether there is a "main" function in the file
> before it is processed ...

Does that mean I could take advantage of this behaviour?  Currently
I have this change:

/* Store in OUTPUT a string (made with alloca) containing an
   assembler-name for a local static variable named NAME.
   LABELNO is an integer which is different for each call.  */

#ifdef TARGET_PDPMAC
#define ASM_FORMAT_PRIVATE_NAME(OUTPUT, NAME, LABELNO)  ^I^I\
{^I^I^I^I^I^I^I^I^I\
  (OUTPUT) = (char *) alloca (strlen ((NAME)) + 10);^I^I^I\
  sprintf ((OUTPUT), "__%d", (LABELNO));^I^I^I^I\
}

to give the static functions unique names within the file.

However, it has the unfortunate side-effect that this code:

#if defined(TARGET_DIGNUS) || defined(TARGET_PDPMAC)
#define ASM_DECLARE_FUNCTION_NAME(FILE, NAME, DECL)^I^I^I\
{^I^I^I^I^I^I^I^I^I\
  if (strlen (NAME) + 1 > mvs_function_name_length)^I^I^I\
    {^I^I^I^I^I^I^I^I^I\
      if (mvs_function_name)^I^I^I^I^I^I\
^Ifree (mvs_function_name);^I^I^I^I^I\
      mvs_function_name = 0;^I^I^I^I^I^I\
    }^I^I^I^I^I^I^I^I^I\
  if (!mvs_function_name)^I^I^I^I^I^I\
    {^I^I^I^I^I^I^I^I^I\
      mvs_function_name_length = strlen (NAME) * 2 + 1;^I^I^I\
      mvs_function_name = (char *) xmalloc (mvs_function_name_length);^I\
    }^I^I^I^I^I^I^I^I^I\
  strcpy (mvs_function_name, NAME);^I^I^I^I^I\
  mvs_need_to_globalize = 1;^I^I^I^I^I^I\
}
#endif

static void
i370_output_function_prologue (f, l)
     FILE *f;
     HOST_WIDE_INT l;
{
/* Don't print stack and args in PDPMAC as it makes the
   comment too long */
#ifdef TARGET_PDPMAC
  fprintf (f, "* %c-func %s prologue\n",
           mvs_need_entry ? 'X' : 'S',
           mvs_function_name);
#else

is producing this output:

* S-func __0 prologue
@@0      PDPPRLG CINDEX=1,FRAME=88,BASER=12,ENTRY=NO
         B     FEN1
         LTORG
FEN1     EQU   *
         DROP  12
         BALR  12,0
         USING *,12
PG1      EQU   *
         LR    11,1
         L     10,=A(PGT1)
* Function __0 code
* 46 world.c
         SLR   15,15
* Function __0 epilogue
         PDPEPIL
* Function __0 literal pool
         DS    0F
         LTORG
* Function __0 page table
         DS    0F
PGT1     EQU   *
         DC    A(PG1)

for this function:

static int aaa(void)
{
    return (0);
}

ie the "aaa" is being completely lost, being replaced by __0 everywhere.
Ideally the __0 would be aaa everywhere, with just the @@0 remaining
generated.

Indeed, I have just made this change:

C:\devel\gcc\gcc>cd config\i370

C:\devel\gcc\gcc\config\i370>cvs diff
cvs diff: Diffing .
Index: i370.c
===================================================================
RCS file: c:\cvsroot/gcc/gcc/config/i370/i370.c,v
retrieving revision 1.34
diff -r1.34 i370.c
84a85,87
> /* original name of a static variable */
> char origmvsstatic[150];
>
1457c1460
<            mvs_function_name);
---
>            mvs_need_entry ? mvs_function_name : origmvsstatic);
1599c1602,1603
<   fprintf (f, "* Function %s code\n", mvs_function_name);
---
>   fprintf (f, "* Function %s code\n",
>            mvs_need_entry ? mvs_function_name : origmvsstatic);
1787c1791,1792
<   fprintf (file, "* Function %s epilogue\n", mvs_function_name);
---
>   fprintf (file, "* Function %s epilogue\n",
>            mvs_need_entry ? mvs_function_name : origmvsstatic);
1818c1823,1824
<   fprintf (file, "* Function %s literal pool\n", mvs_function_name);
---
>   fprintf (file, "* Function %s literal pool\n",
>            mvs_need_entry ? mvs_function_name : origmvsstatic);
1821c1827,1828
<   fprintf (file, "* Function %s page table\n", mvs_function_name);
---
>   fprintf (file, "* Function %s page table\n",
>            mvs_need_entry ? mvs_function_name : origmvsstatic);

C:\devel\gcc\gcc\config\i370>cvs diff -c i370.h
Index: i370.h
===================================================================
RCS file: c:\cvsroot/gcc/gcc/config/i370/i370.h,v
retrieving revision 1.35
diff -c -r1.35 i370.h
*** i370.h      18 Jul 2009 08:56:33 -0000      1.35
--- i370.h      20 Aug 2009 10:40:35 -0000
***************
*** 57,62 ****
--- 57,63 ----
  /* The source file module.  */

  extern char *mvs_module;
+ extern char origmvsstatic[];

  /* Compile using char instructions (mvc, nc, oc, xc).  On 4341 use this 
since
     these are more than twice as fast as load-op-store.
***************
*** 1832,1837 ****
--- 1833,1839 ----
  #define ASM_FORMAT_PRIVATE_NAME(OUTPUT, NAME, LABELNO)                \
  {                                                                     \
    (OUTPUT) = (char *) alloca (strlen ((NAME)) + 10);                  \
+   strcpy(origmvsstatic, NAME); \
    sprintf ((OUTPUT), "__%d", (LABELNO));                              \
  }
  #else

and it is working:

* S-func aaa prologue
@@0      PDPPRLG CINDEX=1,FRAME=88,BASER=12,ENTRY=NO
         B     FEN1
         LTORG
FEN1     EQU   *
         DROP  12
         BALR  12,0
         USING *,12
PG1      EQU   *
         LR    11,1
         L     10,=A(PGT1)
* Function aaa code
* 46 world.c
         SLR   15,15
* Function aaa epilogue
         PDPEPIL
* Function aaa literal pool
         DS    0F
         LTORG
* Function aaa page table
         DS    0F
PGT1     EQU   *
         DC    A(PG1)


But is that guaranteed?

Thanks.  Paul.

Re: i370 port

[Ulrich Weigand]

Paul Edwards wrote:

> > Hmm, it seems 3.2.x would *always* operate on a function-by-function
> > basis.  The unit-at-a-time mode was only introduced with 3.4 (I don't
> > recall if it was already present in 3.3).  I don't think there is any
> > way in 3.2.3 to check whether there is a "main" function in the file
> > before it is processed ...
> 
> Does that mean I could take advantage of this behaviour?

I don't think this would be a good idea.

> /* Store in OUTPUT a string (made with alloca) containing an
>    assembler-name for a local static variable named NAME.
>    LABELNO is an integer which is different for each call.  */
> 
> #ifdef TARGET_PDPMAC
> #define ASM_FORMAT_PRIVATE_NAME(OUTPUT, NAME, LABELNO)  \
> { \
>   (OUTPUT) = (char *) alloca (strlen ((NAME)) + 10); \
>   sprintf ((OUTPUT), "__%d", (LABELNO)); \
> }

How does this work?  ASM_FORMAT_PRIVATE_NAME is not supposed
to completely ignore the NAME argument, the function may well
be called with the same LABELNO but different NAME strings,
and this must not result in conflicting symbols ...

> static void
> i370_output_function_prologue (f, l)
>      FILE *f;
>      HOST_WIDE_INT l;
> {
> /* Don't print stack and args in PDPMAC as it makes the
>    comment too long */
> #ifdef TARGET_PDPMAC
>   fprintf (f, "* %c-func %s prologue\n",
>            mvs_need_entry ? 'X' : 'S',
>            mvs_function_name);
> #else

At this point, you may refer to "current_function_decl" to
retrieve information about the function currently being output.
In particular, you can retrieve the original source-level name
associated with the routine via DECL_NAME (current_function_decl).

Bye,
Ulrich

-- 
  Dr. Ulrich Weigand
  GNU Toolchain for Linux on System z and Cell BE
  Ulrich.Weigand@de.ibm.com

Re: i370 port

[Paul Edwards]

>> > Hmm, it seems 3.2.x would *always* operate on a function-by-function
>> > basis.  The unit-at-a-time mode was only introduced with 3.4 (I don't
>> > recall if it was already present in 3.3).  I don't think there is any
>> > way in 3.2.3 to check whether there is a "main" function in the file
>> > before it is processed ...
>>
>> Does that mean I could take advantage of this behaviour?
>
> I don't think this would be a good idea.

You're right.  With your new change, I was able to see the difference,
and I can see that the static functions sometimes get numbers
assigned in a different order (I think first one called), and in that case,
with my method the name ends up wrong, but yours works.

* S-func dump_new_line prologue
@@3      PDPPRLG CINDEX=4,FRAME=104,BASER=12,ENTRY=NO
* Function dump_new_line code
* Function dump_new_line epilogue
         PDPEPIL
* Function dump_new_line literal pool
         DS    0F
         LTORG
* Function dump_new_line page table
         DS    0F
* S-func dump_maybe_newline prologue (!!!! your new technique)
@@2      PDPPRLG CINDEX=5,FRAME=104,BASER=12,ENTRY=NO
         B     FEN5
         L     10,=A(PGT5)
* Function dump_new_line code (!!!!!! wrong name)

>> /* Store in OUTPUT a string (made with alloca) containing an
>>    assembler-name for a local static variable named NAME.
>>    LABELNO is an integer which is different for each call.  */
>>
>> #ifdef TARGET_PDPMAC
>> #define ASM_FORMAT_PRIVATE_NAME(OUTPUT, NAME, LABELNO)  \
>> { \
>>   (OUTPUT) = (char *) alloca (strlen ((NAME)) + 10); \
>>   sprintf ((OUTPUT), "__%d", (LABELNO)); \
>> }
>
> How does this work?  ASM_FORMAT_PRIVATE_NAME is not supposed
> to completely ignore the NAME argument, the function may well
> be called with the same LABELNO but different NAME strings,
> and this must not result in conflicting symbols ...

I have compiled the entire GCC and not come up with any duplicate
static function names, so I think the number is always unique.

>> static void
>> i370_output_function_prologue (f, l)
>>      FILE *f;
>>      HOST_WIDE_INT l;
>> {
>> /* Don't print stack and args in PDPMAC as it makes the
>>    comment too long */
>> #ifdef TARGET_PDPMAC
>>   fprintf (f, "* %c-func %s prologue\n",
>>            mvs_need_entry ? 'X' : 'S',
>>            mvs_function_name);
>> #else
>
> At this point, you may refer to "current_function_decl" to
> retrieve information about the function currently being output.
> In particular, you can retrieve the original source-level name
> associated with the routine via DECL_NAME (current_function_decl).

Thanks a lot!  I couldn't use that directly, but this:

c:\devel\gcc\gcc\config\i370>cvs diff -r 1.37 i370.c
Index: i370.c
===================================================================
RCS file: c:\cvsroot/gcc/gcc/config/i370/i370.c,v
retrieving revision 1.37
retrieving revision 1.38
diff -r1.37 -r1.38
1457c1457
<            mvs_function_name);
---
>            fname_as_string(0));
1599c1599
<   fprintf (f, "* Function %s code\n", mvs_function_name);
---
>   fprintf (f, "* Function %s code\n", fname_as_string(0));
1786c1786
<   fprintf (file, "* Function %s epilogue\n", mvs_function_name);
---
>   fprintf (file, "* Function %s epilogue\n", fname_as_string(0));
1817c1817
<   fprintf (file, "* Function %s literal pool\n", mvs_function_name);
---
>   fprintf (file, "* Function %s literal pool\n", fname_as_string(0));
1820c1820
<   fprintf (file, "* Function %s page table\n", mvs_function_name);
---
>   fprintf (file, "* Function %s page table\n", fname_as_string(0));

seems to do the trick!

BFN.  Paul.

Re: i370 port

[Ulrich Weigand]

Paul Edwards wrote:

> >> /* Store in OUTPUT a string (made with alloca) containing an
> >>    assembler-name for a local static variable named NAME.
> >>    LABELNO is an integer which is different for each call.  */
> >>
> >> #ifdef TARGET_PDPMAC
> >> #define ASM_FORMAT_PRIVATE_NAME(OUTPUT, NAME, LABELNO)  \
> >> { \
> >>   (OUTPUT) = (char *) alloca (strlen ((NAME)) + 10); \
> >>   sprintf ((OUTPUT), "__%d", (LABELNO)); \
> >> }
> >
> > How does this work?  ASM_FORMAT_PRIVATE_NAME is not supposed
> > to completely ignore the NAME argument, the function may well
> > be called with the same LABELNO but different NAME strings,
> > and this must not result in conflicting symbols ...
> 
> I have compiled the entire GCC and not come up with any duplicate
> static function names, so I think the number is always unique.

Hmm, I see that in the 3.2.x code base this is indeed true.
However, in later compilers ASM_FORMAT_PRIVATE_NAME is used
for other purposes by the middle-end, not just static function
or variable names.  You definitely can get number collisions
in later compilers ...

> > At this point, you may refer to "current_function_decl" to
> > retrieve information about the function currently being output.
> > In particular, you can retrieve the original source-level name
> > associated with the routine via DECL_NAME (current_function_decl).
> 
> Thanks a lot!  I couldn't use that directly, but this:

Why not?  I'd have thought something like

  printf ("%s", IDENTIFIER_POINTER (DECL_NAME (current_function_decl)));

should work fine ...


> c:\devel\gcc\gcc\config\i370>cvs diff -r 1.37 i370.c

B.t.w. if you use the -u or -c option to cvs diff, the diffs are
a lot more readable ...

> <            mvs_function_name);
> ---
> >            fname_as_string(0));

This is a bit problematic as fname_as_string is a function defined in
the C front-end.  If you were e.g. to build the Fortran compiler, your
back-end gets linked against the Fortran front-end instead of the C
front-end, and that function simply will not be there.  Generally,
the rule is that the back-end must not directly call front-end routines.

In any case, for C source code fname_as_string does pretty much
nothing else than what I suggested above ...

Bye,
Ulrich

-- 
  Dr. Ulrich Weigand
  GNU Toolchain for Linux on System z and Cell BE
  Ulrich.Weigand@de.ibm.com

Re: i370 port

[Joseph S. Myers]

On Fri, 5 Jun 2009, Ulrich Weigand wrote:

> I understand current GCC supports various source and target character
> sets a lot better out of the box, so it may be EBCDIC isn't even an
> issue any more.   If there are other problems related to MVS host

I think the EBCDIC support is largely theoretical and not tested on any 
actual EBCDIC host (or target).  cpplib knows the character set name 
UTF-EBCDIC, but whenever it does anything internally that involves the 
encoding of its internal character set it uses UTF-8 rules (which is not 
something valid to do with UTF-EBCDIC).

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: i370 port

[Paul Edwards]

>> I understand current GCC supports various source and target character
>> sets a lot better out of the box, so it may be EBCDIC isn't even an
>> issue any more.   If there are other problems related to MVS host
>
> I think the EBCDIC support is largely theoretical and not tested on any
> actual EBCDIC host (or target).  cpplib knows the character set name
> UTF-EBCDIC, but whenever it does anything internally that involves the
> encoding of its internal character set it uses UTF-8 rules (which is not
> something valid to do with UTF-EBCDIC).

From the hercules-os380 files section, here's the relevant change
to 3.4.6 to stop it being theoretical:

Index: gccnew/gcc/cppcharset.c
diff -c gccnew/gcc/cppcharset.c:1.1.1.1 gccnew/gcc/cppcharset.c:1.6
*** gccnew/gcc/cppcharset.c:1.1.1.1 Wed Apr 15 16:26:16 2009
--- gccnew/gcc/cppcharset.c Wed May 13 11:07:08 2009
***************
*** 23,28 ****
--- 23,30 ----
  #include "cpplib.h"
  #include "cpphash.h"
  #include "cppucnid.h"
+ #include "coretypes.h"
+ #include "tm.h"

  /* Character set handling for C-family languages.

***************
*** 529,534 ****
--- 531,561 ----
    return conversion_loop (one_utf32_to_utf8, cd, from, flen, to);
  }

+ #ifdef MAP_OUTCHAR
+ /* convert ASCII to EBCDIC */
+ static bool
+ convert_asc_ebc (iconv_t cd ATTRIBUTE_UNUSED,
+          const uchar *from, size_t flen, struct _cpp_strbuf *to)
+ {
+   size_t x;
+   int c;
+
+   if (to->len + flen > to->asize)
+     {
+       to->asize = to->len + flen;
+       to->text = xrealloc (to->text, to->asize);
+     }
+   for (x = 0; x < flen; x++)
+     {
+       c = from[x];
+       c = MAP_OUTCHAR(c);
+       to->text[to->len + x] = c;
+     }
+   to->len += flen;
+   return true;
+ }
+ #endif
+
  /* Identity conversion, used when we have no alternative.  */
  static bool
  convert_no_conversion (iconv_t cd ATTRIBUTE_UNUSED,
***************
*** 606,611 ****
--- 633,641 ----
    { "UTF-32BE/UTF-8", convert_utf32_utf8, (iconv_t)1 },
    { "UTF-16LE/UTF-8", convert_utf16_utf8, (iconv_t)0 },
    { "UTF-16BE/UTF-8", convert_utf16_utf8, (iconv_t)1 },
+ #if defined(TARGET_EBCDIC)
+   { "UTF-8/UTF-EBCDIC", convert_asc_ebc, (iconv_t)0 },
+ #endif
  };

  /* Subroutine of cpp_init_iconv: initialize and return a
***************
*** 683,688 ****
--- 713,722 ----

    bool be = CPP_OPTION (pfile, bytes_big_endian);

+ #if defined(TARGET_EBCDIC)
+   ncset = "UTF-EBCDIC";
+   wcset = "UTF-EBCDIC";
+ #else
    if (CPP_OPTION (pfile, wchar_precision) >= 32)
      default_wcset = be ? "UTF-32BE" : "UTF-32LE";
    else if (CPP_OPTION (pfile, wchar_precision) >= 16)
***************
*** 696,701 ****
--- 730,736 ----
      ncset = SOURCE_CHARSET;
    if (!wcset)
      wcset = default_wcset;
+ #endif

    pfile->narrow_cset_desc = init_iconv_desc (pfile, ncset, 
SOURCE_CHARSET);
    pfile->wide_cset_desc = init_iconv_desc (pfile, wcset, SOURCE_CHARSET);


The generated code appears to be fine from visual inspection.

BFN.  Paul.

Re: i370 port

[Paul Edwards]

>> I understand current GCC supports various source and target character
>> sets a lot better out of the box, so it may be EBCDIC isn't even an
>> issue any more.   If there are other problems related to MVS host
>
> I think the EBCDIC support is largely theoretical and not tested on any
> actual EBCDIC host (or target).  cpplib knows the character set name
> UTF-EBCDIC, but whenever it does anything internally that involves the
> encoding of its internal character set it uses UTF-8 rules (which is not
> something valid to do with UTF-EBCDIC).

Results are finally in.

EBCDIC (or arbitrary character set) support was introduced in 3.4.6,
and continues to be the same today, correct?

I've just succeeded today in getting gcc 3.4.6 to self-compile on an
EBCDIC host.  :-)  That's after a gcc 3.4.6 ascii to ebcdic cross-compile.

It's fascinating to look back at what it took.  Note that there are still
some relatively minor cleanups I have yet to do, but it won't change
much.

Caveats:

1. The generated (from machine definition) files are still generated on
the PC.

2. I am unable to do an optimized compile even as a cross-compile,
I get an internal error in this function:

gcse.c:

static void
compute_hash_table_work (struct hash_table *table)
{
...
  if (!current_bb) /* +++ why are we getting NULL here? */
  {
      printf("internal error in gcse\n");
      exit(EXIT_FAILURE);
  }
  FOR_EACH_BB (current_bb)

and indeed, I can't see anything that would initialize that current_bb,
so it's not that surprising that it's NULL!

3. As with gcc 3.2.3, the compiler is still producing slightly different
results on the PC vs mainframe, probably still because of floating
point comparisons being done to select the next register to use
or something like that.

4. There is one thing that doesn't have proper ASCII to EBCDIC
translation being done - the __FUNCTION__ builtin.

So here is the code generated on the PC:

         COPY  PDPTOP
         CSECT
* Program text area
@@V1     EQU   *
         DC    X'78'
         DC    X'31'
         DC    X'32'
         DC    X'33'
         DC    X'0'
LC0      EQU   *
         DC    C'%s %d %s'
         DC    X'15'
         DC    X'0'
LC1      EQU   *
         DC    C'zatest.c'
         DC    X'0'
         DS    0F
* X-func x123 prologue
X123     PDPPRLG CINDEX=0,FRAME=104,BASER=12,ENTRY=YES
         B     FEN0
         LTORG
FEN0     EQU   *
         DROP  12
         BALR  12,0
         USING *,12
PG0      EQU   *
         LR    11,1
         L     10,=A(PGT0)
* Function x123 code
         MVC   88(4,13),=A(LC0)
         MVC   92(4,13),=A(LC1)
         MVC   96(4,13),=F'5'
         MVC   100(4,13),=A(@@V1)
         LA    1,88(,13)
         L     15,=V(PRINTF)
         BALR  14,15
* Function x123 epilogue
         PDPEPIL
* Function x123 literal pool
         DS    0F
         LTORG
* Function x123 page table
         DS    0F
PGT0     EQU   *
         DC    A(PG0)
         END

for this source:

#include <stdlib.h>

void x123(void)
{
    printf("%s %d %s\n", __FILE__, __LINE__, __FUNCTION__);
}

However, that anomaly is not integral to getting the compiler on the
mainframe, and once on the mainframe, the problem goes away
with the next pass.  :-)

I think the problem is this function:

c-decl.c:
c_make_fname_decl (tree id, int type_dep)

needs to call cpp_interpret_string or something like that to get
converted into EBCDIC.


There's not that much mainline code that needs to be changed.
You can see for youself here:

http://rapidshare.com/files/277287822/gccnew-beta51.zip

The file is 250k compressed containing diffs, but most of them are
simply the generated files (which appear as new).  Here are the
real file changes:

diff -c gccnew/gcc/builtins.c:1.1.1.1 gccnew/gcc/builtins.c:1.3
diff -c gccnew/gcc/c-common.c:1.1.1.1 gccnew/gcc/c-common.c:1.2
diff -c gccnew/gcc/c-incpath.c:1.1.1.1 gccnew/gcc/c-incpath.c:1.3
diff -c gccnew/gcc/c-opts.c:1.1.1.1 gccnew/gcc/c-opts.c:1.2
diff -c gccnew/gcc/c-parse.c:1.1.1.1 gccnew/gcc/c-parse.c:1.5
diff -c gccnew/gcc/c-pch.c:1.1.1.1 gccnew/gcc/c-pch.c:1.3
diff -c gccnew/gcc/cppcharset.c:1.1.1.1 gccnew/gcc/cppcharset.c:1.6
diff -c gccnew/gcc/cpperror.c:1.1.1.1 gccnew/gcc/cpperror.c:1.2
diff -c gccnew/gcc/cppfiles.c:1.1.1.1 gccnew/gcc/cppfiles.c:1.7
diff -c gccnew/gcc/cpplib.h:1.1.1.1 gccnew/gcc/cpplib.h:1.2
diff -c gccnew/gcc/cppmacro.c:1.1.1.1 gccnew/gcc/cppmacro.c:1.2
diff -c gccnew/gcc/cppspec.c:1.1.1.1 gccnew/gcc/cppspec.c:1.3
diff -c gccnew/gcc/gcc.c:1.1.1.1 gccnew/gcc/gcc.c:1.6
diff -c gccnew/gcc/gcc.h:1.1.1.1 gccnew/gcc/gcc.h:1.2
diff -c gccnew/gcc/gcov-io.c:1.1.1.1 gccnew/gcc/gcov-io.c:1.2
diff -c gccnew/gcc/gcov-io.h:1.1.1.1 gccnew/gcc/gcov-io.h:1.2
diff -c gccnew/gcc/gcse.c:1.1.1.1 gccnew/gcc/gcse.c:1.3
diff -c gccnew/gcc/hwint.h:1.1.1.1 gccnew/gcc/hwint.h:1.2
diff -c gccnew/gcc/longlong.h:1.1.1.1 gccnew/gcc/longlong.h:1.2
diff -c gccnew/gcc/opts.c:1.1.1.1 gccnew/gcc/opts.c:1.2
diff -c gccnew/gcc/opts.h:1.1.1.1 gccnew/gcc/opts.h:1.3
diff -c gccnew/gcc/opts.sh:1.1.1.1 gccnew/gcc/opts.sh:1.1.1.2
diff -c gccnew/gcc/pretty-print.c:1.1.1.1 gccnew/gcc/pretty-print.c:1.2
diff -c gccnew/gcc/read-rtl.c:1.1.1.1 gccnew/gcc/read-rtl.c:1.2
diff -c gccnew/gcc/real.c:1.1.1.1 gccnew/gcc/real.c:1.3
diff -c gccnew/gcc/system.h:1.1.1.1 gccnew/gcc/system.h:1.2
diff -c gccnew/gcc/toplev.c:1.1.1.1 gccnew/gcc/toplev.c:1.3
diff -c gccnew/gcc/varasm.c:1.1.1.1 gccnew/gcc/varasm.c:1.2
diff -c gccnew/gcc/version.c:1.1.1.1 gccnew/gcc/version.c:1.2
diff -c gccnew/gcc/config/i370/i370-c.c:1.1.1.1 
gccnew/gcc/config/i370/i370-c.c:1.2
diff -c gccnew/gcc/config/i370/i370-protos.h:1.1.1.1 
gccnew/gcc/config/i370/i370-protos.h:1.2
diff -c gccnew/gcc/config/i370/i370.c:1.1.1.1 
gccnew/gcc/config/i370/i370.c:1.23
diff -c gccnew/gcc/config/i370/i370.h:1.1.1.1 
gccnew/gcc/config/i370/i370.h:1.15
diff -c gccnew/gcc/config/i370/i370.md:1.1.1.1 
gccnew/gcc/config/i370/i370.md:1.3
diff -c gccnew/gcc/config/i370/linux.h:1.1.1.1 
gccnew/gcc/config/i370/linux.h:1.1.1.2
diff -c gccnew/gcc/config/i370/oe.h:1.1.1.1 
gccnew/gcc/config/i370/oe.h:1.1.1.2
diff -c gccnew/include/fnmatch.h:1.1.1.1 gccnew/include/fnmatch.h:1.2
diff -c gccnew/include/safe-ctype.h:1.1.1.1 
gccnew/include/safe-ctype.h:1.1.1.2
diff -c gccnew/include/sort.h:1.1.1.1 gccnew/include/sort.h:1.2
diff -c gccnew/include/xregex2.h:1.1.1.1 gccnew/include/xregex2.h:1.2
diff -c gccnew/libiberty/argv.c:1.1.1.1 gccnew/libiberty/argv.c:1.4
diff -c gccnew/libiberty/concat.c:1.1.1.1 gccnew/libiberty/concat.c:1.2
diff -c gccnew/libiberty/cplus-dem.c:1.1.1.1 
gccnew/libiberty/cplus-dem.c:1.2
diff -c gccnew/libiberty/fdmatch.c:1.1.1.1 gccnew/libiberty/fdmatch.c:1.2
diff -c gccnew/libiberty/getpagesize.c:1.1.1.1 
gccnew/libiberty/getpagesize.c:1.2
diff -c gccnew/libiberty/getpwd.c:1.1.1.1 gccnew/libiberty/getpwd.c:1.2
diff -c gccnew/libiberty/getruntime.c:1.1.1.1 
gccnew/libiberty/getruntime.c:1.2
diff -c gccnew/libiberty/hashtab.c:1.1.1.1 gccnew/libiberty/hashtab.c:1.2
diff -c gccnew/libiberty/hex.c:1.1.1.1 gccnew/libiberty/hex.c:1.1.1.2
diff -c gccnew/libiberty/lrealpath.c:1.1.1.1 
gccnew/libiberty/lrealpath.c:1.2
diff -c gccnew/libiberty/make-temp-file.c:1.1.1.1 
gccnew/libiberty/make-temp-file.c:1.2
diff -c gccnew/libiberty/md5.c:1.1.1.1 gccnew/libiberty/md5.c:1.2
diff -c gccnew/libiberty/mkstemps.c:1.1.1.1 gccnew/libiberty/mkstemps.c:1.2
diff -c gccnew/libiberty/obstack.c:1.1.1.1 gccnew/libiberty/obstack.c:1.2
diff -c gccnew/libiberty/physmem.c:1.1.1.1 gccnew/libiberty/physmem.c:1.2
diff -c gccnew/libiberty/regex.c:1.1.1.1 gccnew/libiberty/regex.c:1.2
diff -c gccnew/libiberty/safe-ctype.c:1.1.1.1 
gccnew/libiberty/safe-ctype.c:1.4
diff -c gccnew/libiberty/vasprintf.c:1.1.1.1 
gccnew/libiberty/vasprintf.c:1.1.1.2
diff -c gccnew/libiberty/xmalloc.c:1.1.1.1 gccnew/libiberty/xmalloc.c:1.3
diff -c gccnew/libiberty/xmemdup.c:1.1.1.1 gccnew/libiberty/xmemdup.c:1.2
diff -c gccnew/libiberty/xstrdup.c:1.1.1.1 gccnew/libiberty/xstrdup.c:1.2

(most files are just a few lines of change though).

The most intrusive change was the #include processing which is
done differently on MVS.

Anyway, I'm planning on spending a bit more time consolidating this
build.  Not sure if the optimization problem can be solved, but even if
it can't, at least all the i370 code is now consolidated ready for the
push forward.  Hopefully these issues are more readily solved in the
latest codebase.

Hopefully also some of those other minor issues (like including
non-standard header files like sys/types) can also be remedied.  :-)

This has certainly taken a while.  The number of people who have
asked me why I don't use 3.4.6 as if it is trivial to do these things,
just change a couple of numbers surely.  :-)

Interesting how well the EBCDIC conversion held up too.  GCC 3.2.3
was a lot more intrusive to get working in that regard.

Let me know if you find anything that should have been done
differently too.  :-)

BFN.  Paul.

Re: i370 port

[Ulrich Weigand]

Paul Edwards wrote:

Just one comment on this particular point:

> 4. There is one thing that doesn't have proper ASCII to EBCDIC
> translation being done - the __FUNCTION__ builtin.

It looks this was indeed a bug that was fixed for GCC 4.0.0:
http://gcc.gnu.org/ml/gcc-patches/2004-05/msg01773.html
(note that there were some minor revisions to the patch
before it was committed).

Bye,
Ulrich

-- 
  Dr. Ulrich Weigand
  GNU Toolchain for Linux on System z and Cell BE
  Ulrich.Weigand@de.ibm.com

s390 port

[Paul Edwards]

Hi Ulrich.

Sorry for the necro - things happen slowly Down Under. :-)

Anyway, I am helping someone with their public domain
project, UDOS - https://github.com/udos-project/udos

(just a hobby, won't be big and professional like Linux)

We got the IPL process in place on ESA/390, and then
I decided that the next thing to do would be to switch
to z/Arch so that we could get rid of the AMODE 31
architectural limit on 32-bit programs.

It all worked fine, and we were able to use GCC 11 to
target S/390 and use the -m31 to generate 32-bit code,
run it under z/Arch as AM64, sort of making it the
equivalent of AM32. Really it is the equivalent of
AM-infinity, and there's the rub - GCC 11 is generating
negative indexes, which cause memory above 4 GiB
to be accessed (instead of wrapping at 2/4 GiB), which
of course fails.

Do you have any idea how to stop the S/390 target
from generating negative indexes? I thought the
solution might be to change the Pmode to DImode
even for non-TARGET64, but as you can see here:

http://www.pdos.org/gccfail.png

we got an internal compile error - maximum number
of generated reload insns per insn achieved (90).

I then tried changing the other SImode reference
(CASE_VECTOR_MODE) to DImode too, but that gave
the same internal error.

Here is what the failure looks like (see the large R4):

01:28:27 PSW=00042001 80000000 0000000000005870 INST=A73A0001     AHI   3,1 
add_halfword_immediate
01:28:27 R0=00000000000001FD R1=00000000000000E2 R2=000000000009E579 
R3=00000000000080B2
01:28:27 R4=00000000FFFFF000 R5=000000000001E5C8 R6=0000000000007FFF 
R7=0000000000002000
01:28:27 R8=000000000000201F R9=0000000000000000 RA=00000000000080B0 
RB=00000000000080B2
01:28:27 RC=000000000009E580 RD=0000000000008138 RE=0000000000007B4C 
RF=000000000001E4E4
01:28:27 PSW=00042001 80000000 0000000000005874 INST=42142FFF     STC 
1,4095(4,2)            store_character
01:28:27 R:000000010009E578: Translation exception 0005
01:28:27 R0=00000000000001FD R1=00000000000000E2 R2=000000000009E579 
R3=00000000000080B3
01:28:27 R4=00000000FFFFF000 R5=000000000001E5C8 R6=0000000000007FFF 
R7=0000000000002000
01:28:27 R8=000000000000201F R9=0000000000000000 RA=00000000000080B0 
RB=00000000000080B2
01:28:27 RC=000000000009E580 RD=0000000000008138 RE=0000000000007B4C 
RF=000000000001E4E4
01:28:27 HHCCP014I CPU0000: Addressing exception CODE=0005 ILC=4
01:28:27 PSW=00042001 80000000 0000000000005878 INST=42142FFF     STC 
1,4095(4,2)            store_character
01:28:27 R:000000010009E578: Translation exception 0005
01:28:27 R0=00000000000001FD R1=00000000000000E2 R2=000000000009E579 
R3=00000000000080B3
01:28:27 R4=00000000FFFFF000 R5=000000000001E5C8 R6=0000000000007FFF 
R7=0000000000002000
01:28:27 R8=000000000000201F R9=0000000000000000 RA=00000000000080B0 
RB=00000000000080B2
01:28:27 RC=000000000009E580 RD=0000000000008138 RE=0000000000007B4C 
RF=000000000001E4E4
01:28:27 HHCCP043I Wait state PSW loaded: PSW=00060001 80000000 
0000000000000444
01:28:40 quit
01:28:40 HHCIN900I Begin Hercules shutdown

Any idea what we can do?

Thanks. Paul.




-----Original Message----- 
From: Ulrich Weigand
Sent: Saturday, June 6, 2009 1:20 AM
To: Paul Edwards
Cc: gcc@gcc.gnu.org
Subject: Re: i370 port

Paul Edwards wrote:

> In addition, that code has been ported to GCC 3.4.6, which is now
> working as a cross-compiler at least.  It's still some months away
> from working natively though.  It takes a lot of effort to convert
> the Posix-expecting GCC compiler into C90 compliance.  This has
> been done though, in a way that has minimal code changes to the
> GCC mainline.

You're referring to building GCC for a non-Posix *host*, right?
I assume those changes are not (primarily) in the back-end, but
throughout GCC common code?

> Yes, I'm aware that there is an S/390 port, but it isn't EBCDIC, isn't
> HLASM, isn't 370, isn't C90, isn't MVS.  It may well be possible to
> change all those things, and I suspect that in a few years from now
> I may be sending another message asking what I need to do to get
> all my changes to the s390 target into the s390 target.  At that time,
> I suspect there will be a lot of objection to "polluting" the s390 target
> with all those "unnecessary" things.

Actually, I would really like to see the s390 target optionally support
the MVS ABI and HLASM assembler format, so I wouldn't have any objection
to patches that add these features ...

I understand current GCC supports various source and target character
sets a lot better out of the box, so it may be EBCDIC isn't even an
issue any more.   If there are other problems related to MVS host
support, I suppose those would need to be fixed in common code anyway,
no matter whether the s390 or i370 back-ends are used.

The only point in your list I'm sceptical about is 370 architecture
support -- I don't quite see why this is still useful today (the s390
port does require at a minimum a S/390 G2 with the branch relative
instructions ... but those have been around for nearly 15 years).

Bye,
Ulrich

-- 
  Dr. Ulrich Weigand
  GNU Toolchain for Linux on System z and Cell BE
  Ulrich.Weigand@de.ibm.com 

Re: s390 port

[Ulrich Weigand]

Hi Paul,

"Paul Edwards" <mutazilah@gmail.com> wrote on 02.09.2021 10:15:44:

> We got the IPL process in place on ESA/390, and then
> I decided that the next thing to do would be to switch
> to z/Arch so that we could get rid of the AMODE 31
> architectural limit on 32-bit programs.
>
> It all worked fine, and we were able to use GCC 11 to
> target S/390 and use the -m31 to generate 32-bit code,
> run it under z/Arch as AM64, sort of making it the
> equivalent of AM32. Really it is the equivalent of
> AM-infinity, and there's the rub - GCC 11 is generating
> negative indexes, which cause memory above 4 GiB
> to be accessed (instead of wrapping at 2/4 GiB), which
> of course fails.

Can you elaborate what exactly your goals are?  The point
of the -m31 vs. -m64 option is exactly to match the
AMODE 31 vs. AMODE 64 hardware distinction, so trying to
run -m31 code in AMODE 64 is not supposed to work.

Bye,
Ulrich

Re: s390 port

[Paul Edwards]

Hi Ulrich.

Thanks a lot for your reply.

Could you give me an example of an instruction
generated by –m31 that is not expected to work
on an AM64 system?

E.g. the 32-bit

LR R2,R3

will definitely work on AM64.

So what specifically won’t work? How many different
things won’t work?

Thanks. Paul.




From: Ulrich Weigand 
Sent: Friday, September 3, 2021 12:34 AM
To: Paul Edwards 
Cc: gcc@gcc.gnu.org ; Ulrich Weigand 
Subject: Re: s390 port

Hi Paul,

"Paul Edwards" <mutazilah@gmail.com> wrote on 02.09.2021 10:15:44:

> We got the IPL process in place on ESA/390, and then
> I decided that the next thing to do would be to switch
> to z/Arch so that we could get rid of the AMODE 31
> architectural limit on 32-bit programs.
> 
> It all worked fine, and we were able to use GCC 11 to
> target S/390 and use the -m31 to generate 32-bit code,
> run it under z/Arch as AM64, sort of making it the
> equivalent of AM32. Really it is the equivalent of
> AM-infinity, and there's the rub - GCC 11 is generating
> negative indexes, which cause memory above 4 GiB
> to be accessed (instead of wrapping at 2/4 GiB), which
> of course fails.

Can you elaborate what exactly your goals are?  The point
of the -m31 vs. -m64 option is exactly to match the
AMODE 31 vs. AMODE 64 hardware distinction, so trying to
run -m31 code in AMODE 64 is not supposed to work.

Bye,
Ulrich

Re: s390 port

[Ulrich Weigand]

"Paul Edwards" <mutazilah@gmail.com> wrote on 02.09.2021 16:50:35:

> Could you give me an example of an instruction
> generated by –m31 that is not expected to work
> on an AM64 system?

Well, everything related to address computation, of course.

For example, GCC may use LA on -m31 to implement a
31-bit addition, while it may use LA on -m64 to
implement a 64-bit addition.

Bye,
Ulrich

Re: s390 port

[Paul Edwards]

Hi Ulrich.

I just checked my copy of s390.md and I don’t see
LA being used for arithmetic.

If your copy of s390.md is using LA for arithmetic
then would it be possible to have an option to
use a normal mathematics instruction instead of
LA?

Do you have any more examples besides LA being
used for maths instead of a proper maths instruction?

Also, I just realized – if GCC is using LA for maths
for 32-bit registers, then values will be limited to
2 GiB instead of 4 GiB for unsigned, but that is not
the case.

BFN. Paul.




From: Ulrich Weigand 
Sent: Friday, September 3, 2021 12:53 AM
To: Paul Edwards 
Cc: gcc@gcc.gnu.org ; Ulrich Weigand 
Subject: Re: s390 port

"Paul Edwards" <mutazilah@gmail.com> wrote on 02.09.2021 16:50:35:

> Could you give me an example of an instruction
> generated by –m31 that is not expected to work
> on an AM64 system?

Well, everything related to address computation, of course.

For example, GCC may use LA on -m31 to implement a
31-bit addition, while it may use LA on -m64 to
implement a 64-bit addition.

Bye,
Ulrich

Re: s390 port

[Ulrich Weigand]

Hi Paul,

> I just checked my copy of s390.md and I don’t see
> LA being used for arithmetic.

This would be the "*la_31" and "*la_31_and" patterns.
(Note that the addition is implicit in the use of
the "address_operand" constraint.)

> If your copy of s390.md is using LA for arithmetic
> then would it be possible to have an option to
> use a normal mathematics instruction instead of
> LA?

LA was just an example.  It doesn't usually make sense
to reason on an "use instruction X" basis, that's not
how compiler optimizations work.  You rather start with
a set of semantic invariants and then make sure those
are preserved through all transformations.

Therefore again my question, what is the actual goal
you want to achieve?   I'm still not sure I understand
that ...

> Also, I just realized – if GCC is using LA for maths
> for 32-bit registers, then values will be limited to
> 2 GiB instead of 4 GiB for unsigned, but that is not
> the case.

That's why GCC makes sure to only use the instruction
when a 31-bit addition is wanted.  This can be the
case either when GCC can prove that the involved
operands are pointer values (which are by definition
restricted to 31-bit values in -m31 mode), or when
there is an explict 31-bit addition (using e.g. an
& 0x7fffffff) in the source code.

Bye,
Ulrich

Re: s390 port

[Paul Edwards]

>> I just checked my copy of s390.md and I don’t see
>> LA being used for arithmetic.

> This would be the "*la_31" and "*la_31_and" patterns.
Sorry, I did a grep for “LA”, forgetting that
s390.md doesn’t use uppercase instructions.

> (Note that the addition is implicit in the use of
> the "address_operand" constraint.)

If it is an address we are talking about, then that LA
instruction is going to work perfectly fine in AM24,
AM31 and AM64, and in the AM64 case it is going
to be the equivalent of AM32, so maybe the s390
port could have a “-m32” option for use when
running 32-bit applications as AM64?

>> If your copy of s390.md is using LA for arithmetic
>> then would it be possible to have an option to
>> use a normal mathematics instruction instead of
>> LA?

> LA was just an example.  It doesn't usually make sense
> to reason on an "use instruction X" basis, that's not
> how compiler optimizations work.  You rather start with
> a set of semantic invariants and then make sure those
> are preserved through all transformations.

Ok, that’s above my head.

> Therefore again my question, what is the actual goal
> you want to achieve?   I'm still not sure I understand
> that ...

I would like to know what is required to implement
“-m32” in the S/390 target. I realize that z/Arch
doesn’t have a specific AM32, but I don’t need a
specific AM32. What would actually happen if you
coded a “-m32” and then ran it in an AM64
environment?

My experiments show “with one single problem
discovered so far, actually –m31 and –m32 are
identical and work fine under AM64”.

>> Also, I just realized – if GCC is using LA for maths
>> for 32-bit registers, then values will be limited to
>> 2 GiB instead of 4 GiB for unsigned, but that is not
>> the case.

> That's why GCC makes sure to only use the instruction
> when a 31-bit addition is wanted.  This can be the
> case either when GCC can prove that the involved
> operands are pointer values (which are by definition
> restricted to 31-bit values in -m31 mode)

The compiler doesn’t create a restriction there.
It just generates a simple LA and it works
differently depending on whether it is AM24/31/64.

> or when
> there is an explict 31-bit addition (using e.g. an
> & 0x7fffffff) in the source code.

Ok, thankyou, this is what I needed to know.
I believe I would like to have a –m32 that
drops this test. I don’t want GCC to assume
that such an AND instruction can be implemented
with the use of the “LA” instruction. I want
to see an explicit “N” instruction used. Can
I have this as part of “-m32”?

Thanks. Paul.

Re: s390 port

[Ulrich Weigand]

"Paul Edwards" <mutazilah@gmail.com> wrote on 02.09.2021 17:26:25:

> > Therefore again my question, what is the actual goal
> > you want to achieve?   I'm still not sure I understand
> > that ...
> I would like to know what is required to implement
> “-m32” in the S/390 target. I realize that z/Arch
> doesn’t have a specific AM32, but I don’t need a
> specific AM32. What would actually happen if you
> coded a “-m32” and then ran it in an AM64
> environment?

That depends on what that would actually do.  I'm still not
quite sure what the actual requirements are.

Is this about supporting a 4GB address space instead
of a 2GB space?  (I'm not aware of that being used
anywhere currently.)

Is it about supporting a 32-bit pointer type in an
otherwise AM64 environment?  (This is already used
by the TPF target, but the 32-bit pointer will still
refer to a 2GB address space.)

Is it something else?

In either case, what is the actual benefit of that mode?
(I.e. what benefit would justify the effort to implement it?)


> >> Also, I just realized – if GCC is using LA for maths
> >> for 32-bit registers, then values will be limited to
> >> 2 GiB instead of 4 GiB for unsigned, but that is not
> >> the case.
>
> > That's why GCC makes sure to only use the instruction
> > when a 31-bit addition is wanted.  This can be the
> > case either when GCC can prove that the involved
> > operands are pointer values (which are by definition
> > restricted to 31-bit values in -m31 mode)
>
> The compiler doesn’t create a restriction there.
> It just generates a simple LA and it works
> differently depending on whether it is AM24/31/64.

It is the other way around.  The compiler knows
exactly how the LA instruction behaves in hardware,
and will use the instruction whenever that behavior
matches the semantics of (a part of) the program.
Since the behavior of the instruction differs based
on the addressing mode, the compiler will have to
know which mode the executable will be running in.


Currently, the -m31/-m64 switch basically changes several
things (at the same time):
- the assumption on which AM the executable will run in
- the (used) size of a general-purpose register
- the (default) size of a pointer type
- ABI (function calling convention) details

In theory, it would be possible to split this apart
into distinct features, so that it would be possible
to implement a mode where you can have code that uses
32-bit pointers but is running in AM64 (which would
then support a 4 GB address space).

Is this what you mean by an "-m32" mode?


Basically, this would involve looking at all uses of
the TARGET_64BIT macro in the back-end and determine
which of them actually depend on which of the above
features, and disentangle it accordingly.

I guess that would be possible, but it requires a
nontrivial effort.


Bye,
Ulrich

Re: s390 port

[Paul Edwards]

Hi Ulrich. Thanks for your detailed reply.
>> > Therefore again my question, what is the actual goal
>> > you want to achieve?   I'm still not sure I understand
>> > that ...

>> I would like to know what is required to implement
>> “-m32” in the S/390 target. I realize that z/Arch
>> doesn’t have a specific AM32, but I don’t need a
>> specific AM32. What would actually happen if you
>> coded a “-m32” and then ran it in an AM64
>> environment?

> That depends on what that would actually do.  I'm still not
> quite sure what the actual requirements are.

> Is this about supporting a 4GB address space instead
> of a 2GB space?
Yes, correct.
> (I'm not aware of that being used anywhere currently.)
I’m about to use it. I just need to get past
the problem with negative indexes being used,
and I need your help.

> Is it about supporting a 32-bit pointer type in an
> otherwise AM64 environment?  (This is already used
> by the TPF target, but the 32-bit pointer will still
> refer to a 2GB address space.)

Yes, all pointers will be 32-bit – a normal 32-bit system.

> Is it something else?

Nope, you got it.

> In either case, what is the actual benefit of that mode?
> (I.e. what benefit would justify the effort to implement it?)

The “legacy” environment of z/Linux etc would be 32-bit
instead of 31-bit. IBM’s reputation will be restored. IBM
will have the best architecture on the planet. Better than
x64 because no mode switch is required shifting between
32-bit and 64-bit applications. All run as AM64 = AM-infinity.

>> >> Also, I just realized – if GCC is using LA for maths
>> >> for 32-bit registers, then values will be limited to
>> >> 2 GiB instead of 4 GiB for unsigned, but that is not
>> >> the case.
> 
>> > That's why GCC makes sure to only use the instruction
>> > when a 31-bit addition is wanted.  This can be the
>> > case either when GCC can prove that the involved
>> > operands are pointer values (which are by definition
>> > restricted to 31-bit values in -m31 mode)
>  
>> The compiler doesn’t create a restriction there.
>> It just generates a simple LA and it works
>> differently depending on whether it is AM24/31/64.

> It is the other way around.  The compiler knows
> exactly how the LA instruction behaves in hardware,
> and will use the instruction whenever that behavior
> matches the semantics of (a part of) the program.
> Since the behavior of the instruction differs based
> on the addressing mode, the compiler will have to
> know which mode the executable will be running in.

The i370 port produces code that works in AM24, AM31,
AM32 and AM64 (except for negative indexes). I’m surprised
the s390 port doesn’t too. As far as I can remember from
using IBM C, it supports execution in any AMODE too.

> Currently, the -m31/-m64 switch basically changes several
> things (at the same time)
> - the assumption on which AM the executable will run in 
> - the (used) size of a general-purpose register
> - the (default) size of a pointer type
> - ABI (function calling convention) details

> In theory, it would be possible to split this apart
> into distinct features, so that it would be possible
> to implement a mode where you can have code that uses
> 32-bit pointers but is running in AM64 (which would
> then support a 4 GB address space).

> Is this what you mean by an "-m32" mode?

Yes, correct.

> Basically, this would involve looking at all uses of
> the TARGET_64BIT macro in the back-end and determine
> which of them actually depend on which of the above
> features, and disentangle it accordingly.

> I guess that would be possible, but it requires a
> nontrivial effort.

I’d like to approach the problem from the other
direction – what modifications are required to
be made to “-m31” so that it does “-m32” instead?
I’m happy to simply retire “-m31”, but I don’t care
if both exist.

If “-m31” is retired, and made an alias for “-m32”,
my guess is that 20 lines of code need to be changed.

The most important thing is to stop generating
negative indexes.

ie if you have “char *p” and you go p[-1] I don’t
want 0xFFFFFFFF generated as an index. I instead
want a subtraction done.

I was under the impression that this was governed
by the Pmode – whether it was set to DImode or
SImode. But I tried forcing Pmode to DImode, even
for “–m31”, but it gave an internal error, which I
showed you already.

What am I missing?

Thanks. Paul.

Re: s390 port

[Andreas Schwab]

On Sep 03 2021, Paul Edwards via Gcc wrote:

> The “legacy” environment of z/Linux etc would be 32-bit
> instead of 31-bit. IBM’s reputation will be restored. IBM
> will have the best architecture on the planet. Better than
> x64 because no mode switch is required shifting between
> 32-bit and 64-bit applications. All run as AM64 = AM-infinity.

That looks like -mabi=ilp32 on aarch64, or -mx32 on x86_64.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

Re: s390 port

[Ulrich Weigand]

"Paul Edwards" <mutazilah@gmail.com> wrote on 02.09.2021 22:05:39:
> > Is this about supporting a 4GB address space instead
> > of a 2GB space?
>
> Yes, correct.

OK, that makes things clearer.  This implies in particular:

- 4GB address space means you need to run in AMODE64

- AMODE64 means the native address size is 64 bits.  This
  implies that Pmode has to be DImode, since Pmode tells
  the compiler what the native address size is.

  Specifically, if you try to run AMODE64 with Pmode equals
  SImode, the compiler will not be aware that the hardware
  uses the high 32 bits of base and index registers, and
  will not necessarily keep them zero.  Also, the compiler
  will assume the base + index (+ displacement) arithmetic
  will operate in 32 bits -- I'm pretty sure this is
  actually the root cause of your "negative index" problem.

> > Is it about supporting a 32-bit pointer type in an
> > otherwise AM64 environment?  (This is already used
> > by the TPF target, but the 32-bit pointer will still
> > refer to a 2GB address space.)
> Yes, all pointers will be 32-bit – a normal 32-bit system.

Note that even if Pmode == DImode, you can still use 32-bit
*pointer* sizes.  This is exactly what e.g. the Intel x32
mode does (as was mentioned by Andreas).

> I’d like to approach the problem from the other
> direction – what modifications are required to
> be made to “-m31” so that it does “-m32” instead?
> I’m happy to simply retire “-m31”, but I don’t care
> if both exist.

If you want to go for an "x32" like mode, I think this
is wrong approach.  The right approach would be to
start from "-m64", and simply modify the pointer size
to be 32 bits.

This would work by setting POINTER_SIZE to 32, while
leaving everything else like for -m64.  I'm sure there
will be a few other places that need adaptation, but
it should be pretty straightforward.  You can also
check the Intel back-end where they're using the
TARGET_X32 macro.


We've thought about implementing this mode for Linux,
but decided not to do it, since it would only provide
marginal performance improvements, and has the drawback
of being another new ABI that would be incompatible to
the whole existing software ecosystem.

(The latter point may not be an issue for you if you're
looking into a completely new OS anyway.)

Bye,
Ulrich

Re: s390 port

[Paul Edwards]

> - AMODE64 means the native address size is 64 bits.  This
>  implies that Pmode has to be DImode, since Pmode tells
>  the compiler what the native address size is.

>  Specifically, if you try to run AMODE64 with Pmode equals
>  SImode, the compiler will not be aware that the hardware
>  uses the high 32 bits of base and index registers, and
>  will not necessarily keep them zero.

The compiler naturally keeps them zero. The

instructions that are used to load registers

do not pollute the high-order 32 bits.



>  Also, the compiler
>  will assume the base + index (+ displacement) arithmetic
>  will operate in 32 bits -- I'm pretty sure this is
>  actually the root cause of your "negative index" problem.


Where is this logic please? Can I do a #if 0 or similar

to disable it?


> Note that even if Pmode == DImode, you can still use 32-bit
> *pointer* sizes.  This is exactly what e.g. the Intel x32
> mode does (as was mentioned by Andreas).


I’m happy to try the approach from BOTH directions

and see which one hits “-m32” first.


>> I’d like to approach the problem from the other
>> direction – what modifications are required to
>> be made to “-m31” so that it does “-m32” instead?
>> I’m happy to simply retire “-m31”, but I don’t care
>> if both exist.

> If you want to go for an "x32" like mode, I think this
> is wrong approach.  The right approach would be to
> start from "-m64", and simply modify the pointer size
> to be 32 bits.


> This would work by setting POINTER_SIZE to 32, while
> leaving everything else like for -m64.



That will generate 64-bit z/Arch instructions.

I wish to generate ESA/390 instructions.



> I'm sure there
> will be a few other places that need adaptation, but
> it should be pretty straightforward.

No, modifying GCC is beyond my ability. I

need 20 lines of code from someone who is

familiar with the system.



>  You can also
> check the Intel back-end where they're using the
> TARGET_X32 macro.


See above about beyond my ability.

> We've thought about implementing this mode for Linux,
> but decided not to do it, since it would only provide
> marginal performance improvements, and has the drawback
> of being another new ABI that would be incompatible to
> the whole existing software ecosystem.


Shouldn’t the end user be able to decide this

for themselves? No-one at all is interested in

32-bit mainframes?


> (The latter point may not be an issue for you if you're
> looking into a completely new OS anyway.)


Correct.

Thanks. Paul.

Re: s390 port

[Ulrich Weigand]

"Paul Edwards" <mutazilah@gmail.com> wrote on 03.09.2021 13:35:10:
> >  Specifically, if you try to run AMODE64 with Pmode equals
> >  SImode, the compiler will not be aware that the hardware
> >  uses the high 32 bits of base and index registers, and
> >  will not necessarily keep them zero.
> The compiler naturally keeps them zero. The
> instructions that are used to load registers
> do not pollute the high-order 32 bits.

While this is true for most instructions, the compiler will not
restrict itself to using only those.  (As just one obvious
example, the compiler may use "lay" with a negative displacement,
which will set the high bits of a GPR in AMODE64.)

It is of course possible to change the back-end to ensure that
SImode operations always leave the high part unmodified; for
example LLVM does that, because it wants to allocate the high
parts seperately for use with the high-word facility instructions.
But GCC currently does not do so.

> >  Also, the compiler
> >  will assume the base + index (+ displacement) arithmetic
> >  will operate in 32 bits -- I'm pretty sure this is
> >  actually the root cause of your "negative index" problem.
> Where is this logic please? Can I do a #if 0 or similar
> to disable it?

This is not in one single place, but spread throughout the
compiler, both common code and back-end.  I do not think it will
be possible to get the compiler to generate correct code if
you do not specify the address size correctly.  AMODE64 will
require Pmode == DImode.

(And, b.t.w. not the -m31 DImode, which is a pair of 32-bit
GPRs, but rather the -m64 DImode, which is a single 64-bit GPR.)

> > If you want to go for an "x32" like mode, I think this
> > is wrong approach.  The right approach would be to
> > start from "-m64", and simply modify the pointer size
> > to be 32 bits.
> > This would work by setting POINTER_SIZE to 32, while
> > leaving everything else like for -m64.
>
> That will generate 64-bit z/Arch instructions.
> I wish to generate ESA/390 instructions.

Why? AMODE64 exists only in z/Arch, so of course there
will be z/Arch instructions available ...

> > We've thought about implementing this mode for Linux,
> > but decided not to do it, since it would only provide
> > marginal performance improvements, and has the drawback
> > of being another new ABI that would be incompatible to
> > the whole existing software ecosystem.
> Shouldn’t the end user be able to decide this
> for themselves?

It's open source, of course everybode can decide what they
want to work on themselves.  But we decide what we spend
our own time on based on we think is useful ...

> No-one at all is interested in 32-bit mainframes?

Not any more, at least not in Linux.  Linux is pretty much
64-bit only at this point.


Bye,
Ulrich

Re: s390 port

[Paul Edwards]

>> >  Also, the compiler
>> >  will assume the base + index (+ displacement) arithmetic
>> >  will operate in 32 bits -- I'm pretty sure this is
>> >  actually the root cause of your "negative index" problem.

>> Where is this logic please? Can I do a #if 0 or similar
>> to disable it?

> This is not in one single place, but spread throughout the
> compiler, both common code and back-end.  I do not think it will
> be possible to get the compiler to generate correct code if
> you do not specify the address size correctly.
1. Is there any way to put a constraint on index
registers, to say that a particular machine can
only index in the range of –512 to +512 or some
other arbitrary set? If so, I can do 0 to 2 GiB.
2. Is there a way of saying a machine doesn’t
support indexing at all?
>> > If you want to go for an "x32" like mode, I think this
>> > is wrong approach.  The right approach would be to
>> > start from "-m64", and simply modify the pointer size
>> > to be 32 bits.
>> > This would work by setting POINTER_SIZE to 32, while
>> > leaving everything else like for -m64.
>  
>> That will generate 64-bit z/Arch instructions.
>> I wish to generate ESA/390 instructions.

> Why? AMODE64 exists only in z/Arch, so of course there
> will be z/Arch instructions available ...

For the same reason people constructed Babbage’s
invention, I wish to demonstrate the minor changes
that would have been required to the S/360 so that
we would never have arrived at a 31-bit black hole,
and we could have in fact had the perfect 32-bit
machine. Almost identical to the 31-bit machine.
A S/360+, a S/370+ and a S/390+. 

>> > We've thought about implementing this mode for Linux,
>> > but decided not to do it, since it would only provide
>> > marginal performance improvements, and has the drawback
>> > of being another new ABI that would be incompatible to
>> > the whole existing software ecosystem.
>> Shouldn’t the end user be able to decide this
>> for themselves?

> It's open source, of course everybode can decide what they
> want to work on themselves.  But we decide what we spend
> our own time on based on we think is useful ...

Sure.

>> No-one at all is interested in 32-bit mainframes?

> Not any more, at least not in Linux.  Linux is pretty much
> 64-bit only at this point.

I think z/OS is pretty much still 31-bit only,
as far as apps are concerned, right? I’d like to
bump that up to 32-bit.

BFN. Paul.

Re: s390 port

[Jakub Jelinek]

On Fri, Sep 03, 2021 at 10:38:36PM +1000, Paul Edwards via Gcc wrote:
> > This is not in one single place, but spread throughout the
> > compiler, both common code and back-end.  I do not think it will
> > be possible to get the compiler to generate correct code if
> > you do not specify the address size correctly.
> 1. Is there any way to put a constraint on index
> registers, to say that a particular machine can
> only index in the range of –512 to +512 or some
> other arbitrary set? If so, I can do 0 to 2 GiB.
> 2. Is there a way of saying a machine doesn’t
> support indexing at all?

There is a way to do that, but it isn't about changing a single or a couple
of spots, one needs to change a lot of *.md patterns, a lot of macros,
target hooks and as Ulrich said, most important is to use the right Pmode
which can differ from ptr_mode provided one e.g. defines ptr_extend pattern
etc.
Just look at the amount of work needed for the x32 or aarch64 ilp32 support,
and not just work spent one time on adding that support, but the continuous
amount of work on maintaining it.  The initial work is certainly a few
weeks if not months of work, then there needs to be somebody who regularly
tests gcc trunk and branches in such configuration so that it doesn't
bitrot, and not just that but somebody who actually fixes bugs in it.

If something doesn't fit into 2GB of address space, isn't it likely it won't
fit into 4GB of address space in a year or two?

	Jakub

Re: s390 port

[Paul Edwards]

>> > This is not in one single place, but spread throughout the
>> > compiler, both common code and back-end.  I do not think it will
>> > be possible to get the compiler to generate correct code if
>> > you do not specify the address size correctly.

>> 1. Is there any way to put a constraint on index
>> registers, to say that a particular machine can
>> only index in the range of –512 to +512 or some
>> other arbitrary set? If so, I can do 0 to 2 GiB.

>> 2. Is there a way of saying a machine doesn’t
>> support indexing at all?

> There is a way to do that, but it isn't about changing a single or a 
> couple
> of spots, one needs to change a lot of *.md patterns, a lot of macros,
> target hooks and as Ulrich said, most important is to use the right Pmode
> which can differ from ptr_mode provided one e.g. defines ptr_extend 
> pattern
> etc.

Pardon? All that is required just to put a constraint
on an index register? If a range of a machine is
limited to -512 to +512, it shouldn't be necessary
to change md patterns etc etc.

> Just look at the amount of work needed for the x32 or aarch64 ilp32 
> support,

That's different. That's because Intel stuffed up.
IBM didn't. IBM came within an ace of a perfect
architecture. It's as if Intel had created an x32
instead of an 80386 in 1986.

IBM got it almost right in the 1960s.

> and not just work spent one time on adding that support, but the 
> continuous
> amount of work on maintaining it.  The initial work is certainly a few
> weeks if not months of work,

I've been trying to figure out how to lift the 31-bit
restriction on mainframes since around 1987.

If I have to pay someone for 2 month of work, at
this stage, I'm willing to do that, but:

1. I would like it done on GCC 3.2.3 plus maybe
GCC 3.4.6.

2. How much will it cost in US$?

> then there needs to be somebody who regularly
> tests gcc trunk and branches in such configuration so that it doesn't
> bitrot, and not just that but somebody who actually fixes bugs in it.

I'll take responsibility for giving the GCC 3.X.X
releases the TLC they deserve. And I'll encourage
my daughter to maintain them after I've kicked
the bucket.

> If something doesn't fit into 2GB of address space,
> isn't it likely it won't fit into 4GB of address space
> in a year or two?

Nope. 2 GiB is already a shitload of memory. It only
takes something like 23 MB for GCC 3.2.3 to recompile
itself, and I think 60 MB for GCC 3.4.6 to recompile
itself. That's the heaviest real workload I do. A 4 GiB
limitation instead of 2 GiB makes it just that much
less likely I'll ever hit a real limit.

Someone told me that the only non-scientific application
they knew of that came close to hitting the 2 GiB limit
was IBM's C compiler. I doubt that IBM's C compiler
technology is evolving at such a rate that it only takes
1-2 years for them to subsequently hit 4 GiB. Quite
apart from the fact that I don't really trust that even
IBM C is hitting a 2 GiB limit for what GCC can do in
23 MiB. But it could be true - I'm not familiar with
compiler internals.

BFN. Paul.

Re: s390 port

[Paul Edwards]

[-- Attachment #1: Type: text/plain, Size: 1931 bytes --]

On Fri, 3 Sept 2021 at 20:12, Ulrich Weigand <Ulrich.Weigand@de.ibm.com>
wrote:

> "Paul Edwards" <mutazilah@gmail.com> wrote on 03.09.2021 13:35:10:
> > >  Specifically, if you try to run AMODE64 with Pmode equals
> > >  SImode, the compiler will not be aware that the hardware
> > >  uses the high 32 bits of base and index registers, and
> > >  will not necessarily keep them zero.
> > The compiler naturally keeps them zero. The
> > instructions that are used to load registers
> > do not pollute the high-order 32 bits.
>
> While this is true for most instructions, the compiler will not
> restrict itself to using only those.  (As just one obvious
> example, the compiler may use "lay" with a negative displacement,
> which will set the high bits of a GPR in AMODE64.)
>
> (And, b.t.w. not the -m31 DImode, which is a pair of 32-bit
> GPRs, but rather the -m64 DImode, which is a single 64-bit GPR.)
>

Hi all.

Turns out I have been asking the wrong question for several years.

I was going to generate a peephole (an idea from the author of
UDOS, now KinnowOS) to detect when a negative index was
being used, and force an addition instead of an index, when I
realized that it wasn't just literals that could use a negative
value.

That is when I realized that negative numbers were perfectly
valid/normal for indexing, and that it is the OS/hardware that
needs to adapt to this reality when transitioning from 32-bit
hardware to 64-bit hardware.

As such, I have updated z/PDOS-32 to use DAT to map the
4 GiB to 8 GiB region to 0 to 4 GiB, so that negative indexing
works fine.

You can download this from http://pdos.org (down the bottom).

So would it be possible now to update gcc to make -m32 and
-m31 and -m24 all work, as they all generate the exact same
code, regrardless of whether you are running as AM24 on
S/370, AM31 on S/390 or AM32 on Hercules/380 or AM64
with DAT set appropriately on z/Arch.

Thanks. Paul.