access to include path in front end

access to include path in front end

[James K. Lowden]

I don't understand how to access in a front end the arguments to the -I
option on the command line.  

Cobol has a feature similar to the C preprecessor, known as the
Compiler Directing Facility (CDF).  The CDF has a COPY statement that
resembles an #include directive in C, and shares the property that COPY
names a file that is normally found in a "copybook" which, for our
purposes, is a directory of such files.  The name of that directory is
defined outside the Cobol program.  

I would like to use the -I option to pass the names of copybook
directories to the cobol front end.  A bit of exploration yesterday left
me with the sense that the -I argument, in C at least, is not passed to
the compiler, but to the preprocessor. Access to -fmax-errors I think
I've figured out, but -I is a mystery. 

I'm a little puzzled by the status quo as I understand it.  Unless I
missed it, it's not discussed in gccint.  ISTM ideally there would be
some kind of getopt(3) processing, and the whole set of command-line
options captured in an array of structures accessible to any front
end.  Is that not the case and, if not, why not?  

Many thanks.  

--jkl

Re: access to include path in front end

[Richard Biener]

On Tue, Nov 29, 2022 at 4:41 PM James K. Lowden
<jklowden@schemamania.org> wrote:
>
> I don't understand how to access in a front end the arguments to the -I
> option on the command line.
>
> Cobol has a feature similar to the C preprecessor, known as the
> Compiler Directing Facility (CDF).  The CDF has a COPY statement that
> resembles an #include directive in C, and shares the property that COPY
> names a file that is normally found in a "copybook" which, for our
> purposes, is a directory of such files.  The name of that directory is
> defined outside the Cobol program.
>
> I would like to use the -I option to pass the names of copybook
> directories to the cobol front end.  A bit of exploration yesterday left
> me with the sense that the -I argument, in C at least, is not passed to
> the compiler, but to the preprocessor. Access to -fmax-errors I think
> I've figured out, but -I is a mystery.
>
> I'm a little puzzled by the status quo as I understand it.  Unless I
> missed it, it's not discussed in gccint.  ISTM ideally there would be
> some kind of getopt(3) processing, and the whole set of command-line
> options captured in an array of structures accessible to any front
> end.  Is that not the case and, if not, why not?

The frontends have access to the set of passed options via the
option processing langhooks (and there's also global_options
and global_options_set), -I would be the OPT_I option (currently
only enabled for some frontends, you can add that in your
language specific .opt file).  The set of include directories is
not in any way special here.

Richard.

> Many thanks.
>
> --jkl

Re: access to include path in front end

[Michael Matz]

Hello,

On Tue, 29 Nov 2022, James K. Lowden wrote:

> I don't understand how to access in a front end the arguments to the -I
> option on the command line.  
> 
> Cobol has a feature similar to the C preprecessor, known as the
> Compiler Directing Facility (CDF).  The CDF has a COPY statement that
> resembles an #include directive in C, and shares the property that COPY
> names a file that is normally found in a "copybook" which, for our
> purposes, is a directory of such files.  The name of that directory is
> defined outside the Cobol program.  
> 
> I would like to use the -I option to pass the names of copybook
> directories to the cobol front end.  A bit of exploration yesterday left
> me with the sense that the -I argument, in C at least, is not passed to
> the compiler, but to the preprocessor. Access to -fmax-errors I think
> I've figured out, but -I is a mystery. 
> 
> I'm a little puzzled by the status quo as I understand it.  Unless I
> missed it, it's not discussed in gccint.  ISTM ideally there would be
> some kind of getopt(3) processing, and the whole set of command-line
> options captured in an array of structures accessible to any front
> end.

There is, it's just much more complicated than getopt :)

If you're looking at the C frontends for inspiration, then:

c-family/c.opt defines which options are recognized and several specifics 
about them, e.g. for -I it has:

----
I
C ObjC C++ ObjC++ Joined Separate MissingArgError(missing path after %qs)
-I <dir>        Add <dir> to the end of the main include path.
----

(look at some other examples therein, also in common.opt to get a feel).

Then code in c-family/c-opts.c:c_common_handle_option actually handles the 
option:

    case OPT_I:
      if (strcmp (arg, "-"))
        add_path (xstrdup (arg), INC_BRACKET, 0, true);
      else
      .,.

That function is made a langhook for option processing so that it's 
actually called via c/c-objc-common.h:

  #define LANG_HOOKS_HANDLE_OPTION c_common_handle_option

If you're also using the model of a compiler driver (i.e. the gcc program, 
source in gcc.cc) that actually calls compiler (cc1), assembler and 
linker, then you also need to arrange for that program to pass all -I 
options to the compiler proper.  That is done with the spec language, by 
somewhere having '{I*}' in the specs for invoking the cobol compiler.  
E.g. look in gcc.cc for '@c' (matching the file extension) how that entry 
uses '%(cpp_unique_options)', and how cpp_unique_options is defined for 
the specs language:

  INIT_STATIC_SPEC ("cpp_unique_options",       &cpp_unique_options),

and

static const char *cpp_unique_options =
  "%{!Q:-quiet} %{nostdinc*} %{C} %{CC} %{v} %@{I*&F*} %{P} %I\  

(the specs language used here is documented in a lengthy comment early in 
gcc.cc, "The Specs Language")

The "%@{I*F*}" is the one that makes gcc pass -Iwhatever to cc1 (and 
ensures relative order with -F options is retained and puts all these into 
an @file if one is given on the cmdline, otherwise leaves it on cmdline).  
If you use the compiler driver then using '-v' when invoking it will 
quickly tell you if that options passing worked, as it will show the 
concrete command it exec's for the compiler proper.

Hope this helps.


Ciao,
Michael.

Re: access to include path in front end

[James K. Lowden]

On Wed, 30 Nov 2022 08:49:35 +0100
Richard Biener <richard.guenther@gmail.com> wrote:

> > I would like to use the -I option to pass the names of copybook
> > directories to the cobol front end.  A bit of exploration yesterday
> > left me with the sense that the -I argument, in C at least, is not
> > passed to the compiler, but to the preprocessor. Access to
> > -fmax-errors I think I've figured out, but -I is a mystery.
> 
> The frontends have access to the set of passed options via the
> option processing langhooks (and there's also global_options
> and global_options_set), -I would be the OPT_I option (currently
> only enabled for some frontends, you can add that in your
> language specific .opt file).  The set of include directories is
> not in any way special here.

Thanks for clearing that up for me, Richard.  I somehow got the
impression that the langhooks were needed only for adding options
specific to the language.  

--jkl

Re: access to include path in front end

[Jonathan Wakely]

On Wed, 30 Nov 2022 at 15:59, Michael Matz wrote:
> If you're looking at the C frontends for inspiration, then:
>
> c-family/c.opt defines which options are recognized and several specifics
> about them, e.g. for -I it has:
>
> ----
> I
> C ObjC C++ ObjC++ Joined Separate MissingArgError(missing path after %qs)
> -I <dir>        Add <dir> to the end of the main include path.
> ----
>
> (look at some other examples therein, also in common.opt to get a feel).

There is also https://gcc.gnu.org/onlinedocs/gccint/Options.html
describing these files.


> If you're also using the model of a compiler driver (i.e. the gcc program,
> source in gcc.cc) that actually calls compiler (cc1), assembler and
> linker, then you also need to arrange for that program to pass all -I
> options to the compiler proper.  That is done with the spec language, by
> somewhere having '{I*}' in the specs for invoking the cobol compiler.
> E.g. look in gcc.cc for '@c' (matching the file extension) how that entry
> uses '%(cpp_unique_options)', and how cpp_unique_options is defined for
> the specs language:
>
>   INIT_STATIC_SPEC ("cpp_unique_options",       &cpp_unique_options),
>
> and
>
> static const char *cpp_unique_options =
>   "%{!Q:-quiet} %{nostdinc*} %{C} %{CC} %{v} %@{I*&F*} %{P} %I\
>
> (the specs language used here is documented in a lengthy comment early in
> gcc.cc, "The Specs Language")


Also documented at https://gcc.gnu.org/onlinedocs/gcc/Spec-Files.html

Re: access to include path in front end

[James K. Lowden]

On Wed, 30 Nov 2022 15:58:40 +0000 (UTC)
Michael Matz <matz@suse.de> wrote:

Hi Michael, 

First, thanks for a great answer, and to Jonathan for reminding me of
what documentation we do have for this. I'm now using -I in cobol1, but
I'm not getting it from gcobol. I guess I need to extend the spec
options, but I'm puzzled because so far I've been able to dodge that
bullet.  

> E.g. look in gcc.cc for '@c' (matching the file extension) how that
> entry uses '%(cpp_unique_options)', and how cpp_unique_options is
> defined for the specs language:
> 
>   INIT_STATIC_SPEC ("cpp_unique_options",       &cpp_unique_options),
> 
> and
> 
> static const char *cpp_unique_options =
>   "%{!Q:-quiet} %{nostdinc*} %{C} %{CC} %{v} %@{I*&F*} %{P} %I\  

Please tell me if this looks right and complete to you:

1.  Add an element to the static_specs array: 

	  INIT_STATIC_SPEC ("cobol_options", &cobol_options),

2.  Define the referenced structure: 

	static const char *cobol_options =  "%{v} %@{I*&F*}"
  or just
	static const char *cobol_options =  "%{v} %@{I*}"

  because I don't know what -F does, or if I need it.  

I'm using "cobol_options" instead of "cobol_unique_options" because the
options aren't unique to Cobol, and because only cpp seems to have
unique options.  

I'm including %{v} against the future, when the cobol1 compiler
supports a -v option. 

3.  Correct the entries in the default_compilers array.  Currently I
have in cobol/lang-specs.h:

    {".cob", "@cobol", 0, 0, 0},
    {".COB", "@cobol", 0, 0, 0},
    {".cbl", "@cobol", 0, 0, 0},
    {".CBL", "@cobol", 0, 0, 0},
    {"@cobol", 
	"cobol1 %i %(cc1_options) %{!fsyntax-only:%(invoke_as)}", 
	0, 0, 0}, 

That last one is a doozy.  Is it even slightly right? IIUC, I should at
least remove 

	%{!fsyntax-only:%(invoke_as)}

because I don't need the options from the invoke_as string in gcc.cc. 

(Actually, we do have a syntax-only feature.  IIUC, the we could define
our own command-line options to invoke it, and use our own static
variable cobol_foo (instead of invoke_as) to pass those options to
cobol1 when -fsyntax-only is used.  Or, alternatively, we could just
honor -fsyntax-only literally, like any other option.) 

That would still leave me with too much, because cobol1 ignores most of
the options cc1 accepts.  What would you do?  

I don't understand the relationship between default_compliers and
static_specs.  

I have made no special provision for "compiler can deal with
multiple source files", except that cobol1 accepts multiple source
files on the command line, and iterates over them.  If that's enough,
then I'll set compiler::combinable to 1.  

I'm trying those 3 today.  I'd like to get it right, as long as I'm in
the neighborhood.  :-)  

+++

As I mentioned, for a year I've been able to avoid the Specs Language,
apparently because some things happen by default.  The options defined
in cobol/lang.opt are passed from gcobol to cobol1.  The value of the
-findicator-column option is available (but only if the option starts
with "f"; -indicator-column doesn't work).  cobol1 sees the value of
-fmax-errors. 

Is that because I'm using gcobol, and not "gcc -x cobol" ?  If not, how
do I know what's passed by default, and what needs specification?  

Thanks again for your guidance.  I doubt I'll ever really understand
what's going on at this level, but I'm glad to plug in the right magic
values.  

Regards, 

--jkl

Re: access to include path in front end

[Michael Matz]

Hey,

On Thu, 1 Dec 2022, James K. Lowden wrote:

> > E.g. look in gcc.cc for '@c' (matching the file extension) how that
> > entry uses '%(cpp_unique_options)', and how cpp_unique_options is
> > defined for the specs language:
> > 
> >   INIT_STATIC_SPEC ("cpp_unique_options",       &cpp_unique_options),
> > 
> > and
> > 
> > static const char *cpp_unique_options =
> >   "%{!Q:-quiet} %{nostdinc*} %{C} %{CC} %{v} %@{I*&F*} %{P} %I\  
> 
> Please tell me if this looks right and complete to you:
> 
> 1.  Add an element to the static_specs array: 
> 
> 	  INIT_STATIC_SPEC ("cobol_options", &cobol_options),

That, or expand its contents where you'd use '%(cobol_options)' in the 
strings.

> 
> 2.  Define the referenced structure: 
> 
> 	static const char *cobol_options =  "%{v} %@{I*&F*}"
>   or just
> 	static const char *cobol_options =  "%{v} %@{I*}"
> 
>   because I don't know what -F does, or if I need it.

I.e. as long as it's that short expanding inline would work nicely.

> I'm using "cobol_options" instead of "cobol_unique_options" because the
> options aren't unique to Cobol, and because only cpp seems to have
> unique options.  
> 
> I'm including %{v} against the future, when the cobol1 compiler
> supports a -v option. 

Makes sense.

> 3.  Correct the entries in the default_compilers array.  Currently I
> have in cobol/lang-specs.h:
> 
>     {".cob", "@cobol", 0, 0, 0},
>     {".COB", "@cobol", 0, 0, 0},
>     {".cbl", "@cobol", 0, 0, 0},
>     {".CBL", "@cobol", 0, 0, 0},
>     {"@cobol", 
> 	"cobol1 %i %(cc1_options) %{!fsyntax-only:%(invoke_as)}", 
> 	0, 0, 0}, 
> 
> That last one is a doozy.  Is it even slightly right?

It misses %(cpp_unique_options) which was the reason why your -I arguments 
weren't passed to cobol1.  You would just your new %(cobol_options), or 
simply '%{v} %{I*}' directly in addition to cc1_options.

> IIUC, I should at
> least remove 
> 
> 	%{!fsyntax-only:%(invoke_as)}
> 
> because I don't need the options from the invoke_as string in gcc.cc. 

I think you do, as cobol1 will write out assembler code (it does, right?), 
so to get an object file the driver needs to invoke 'as' as well.  
Basically invoke_as tacks another command to run at the end of the already 
build command line (the one that above would start with 'cobol1 
inputfile.cob ... all the options ...'.  It will basically tack the 
equivalent of '| as tempfile.s -o realoutput.o' to the end (which 
eventually will make the driver executate that command as well).

> That would still leave me with too much, because cobol1 ignores most of
> the options cc1 accepts.  What would you do?

I would go through all cc1_options and see if they _really_ shouldn't be 
interpreted by cobol1.  I guess for instance '-Ddefine' really doesn't 
make sense, but e.g. '-m...' and '-f...' do, and maybe -quiet as well, and 
suchlike.  In that case I'd just use cc1_options (in addition to your new 
%{I*} snippet).

If you then really determine, that no, most options do not make sense you 
need to extract a subset of cc1_options that do, and write them into the 
@cobol entry.  Look e.g. what the fortran frontend does (in 
fortran/lang-specs.h) it simply attaches more things to cc1_options.

> I don't understand the relationship between default_compliers and
> static_specs.  

static_specs lists the names of 'variables' you can use within the specs 
strings, and to what they should expand.  E.g. when I would want to use 
'%(foobar)' in any of my specs strings that needs to be registered in 
static_spec[]:

  INIT_STATIC_SPEC ("foobar", &a_variable_containing_a_string)

The specs parse would then replace '%(foobar)' in specs strings with 
whatever that variable contains.

Using such variables mostly makes sense if you want to enable users (who 
can provide their own specs file) to refer to well-known snippets 
maintained by GCC itself.  For most such strings it's not necessary, and 
you'd be fine with the approach of fortran:

 #define MY_FOOBAR_STRING "%{v} ... this and that ..."

...

  {@mylang, ... "lang1 %i " MY_FOOBAR_STRING "" ... }

> I have made no special provision for "compiler can deal with
> multiple source files", except that cobol1 accepts multiple source
> files on the command line, and iterates over them.  If that's enough,
> then I'll set compiler::combinable to 1.

No good advise here for combinable.  Try it :)

> As I mentioned, for a year I've been able to avoid the Specs Language,
> apparently because some things happen by default.  The options defined
> in cobol/lang.opt are passed from gcobol to cobol1.  The value of the
> -findicator-column option is available (but only if the option starts
> with "f"; -indicator-column doesn't work).  cobol1 sees the value of
> -fmax-errors. 

That's because "%{f*}" is contained in %(cc1_options): 'pass on all 
options starting with "f"', and because you listed cc1_options in your 
cobol1 command line.


Ciao,
Michael.

Re: access to include path in front end

[James K. Lowden]

On Thu, 1 Dec 2022 17:14:31 +0000 (UTC)
Michael Matz <matz@suse.de> wrote:

> > 3.  Correct the entries in the default_compilers array.  Currently I
> > have in cobol/lang-specs.h:
> > 
> >     {".cob", "@cobol", 0, 0, 0},
> >     {".COB", "@cobol", 0, 0, 0},
> >     {".cbl", "@cobol", 0, 0, 0},
> >     {".CBL", "@cobol", 0, 0, 0},
> >     {"@cobol", 
> > 	"cobol1 %i %(cc1_options) %{!fsyntax-only:%(invoke_as)}", 
> > 	0, 0, 0}, 
> 
> It misses %(cpp_unique_options) which was the reason why your -I
> arguments weren't passed to cobol1.  

If I understood you correctly, I don't need to modify gcc.cc.  I only
need to modify cobol/lang-specs.h, which I've done.  But that's
evidently not all I need to do, because it doesn't seem to work.  

The last element in the fragment in cobol/lang-specs.h is now: 

    {"@cobol",
	"cobol1 %i %(cc1_options) %{!fsyntax-only:%(invoke_as)} "
	"%(cpp_unique_options) ",
	0, 0, 0},

(using string constant concatenation).  That's the only change I made,
so far, because everything we said (AIUI) boiled down to, "just add
cpp_unique_options to your spec string".  

> You would just your new %(cobol_options), or simply '%{v} %{I*}'
> directly in addition to cc1_options.

I didn't do that.  IIUC, the 2nd element in the struct can be a string
constant or the address of a char* variable.  So, I chose a string
constant. 

Getting wiser with age, I used -### to dump the cobol1 command
line.  It appears to be consistent with observed behavior: when I run
under gdb and stop at the cobol1::main function, argc is 14, and argv
does not include the -I option. 

The -### output reports the cobol1 command line (as 14 strings,
exactly) on line 10.  In additon to what was supplied (by me, invoking
gcobol), it shows:

	-quiet
	-dumpdir o- -dumpbase csytst10.cbl 
	-dumpbase-ext .cbl
	"-main=gcc/cobol/prim/samples/CUBES/cobol/csytst10.cbl"
	"-mtune=generic" 
	"-march=x86-64" 
	-o /tmp/ccLYrc6D.s

The -main I can explain later if it matters.  The others are magic I
don't understand and don't think matter, but I show them so you know.  

Note that I'm invoking gcobol from the build tree, using -B (among
others) to make it DTRT.  

I see the -B and -I options, and others, with their arguments, contained
in COLLECT_GCC_OPTIONS on lines 9 and 11.  I guess that represents an
environment string?  Or, anyway, a string that holds the options that
will be passed to collect2?  

The contents of COLLECT_GCC_OPTIONS appear to be a superset of the
options supplied on the cobol1 command line. 

It would seem the single change I made is less than the minimum
required, but I confess I've lost track of why anything more is
needed.  

--jkl

Re: access to include path in front end

[Michael Matz]

Hey,

On Fri, 2 Dec 2022, James K. Lowden wrote:

> > > 3.  Correct the entries in the default_compilers array.  Currently I
> > > have in cobol/lang-specs.h:
> > > 
> > >     {".cob", "@cobol", 0, 0, 0},
> > >     {".COB", "@cobol", 0, 0, 0},
> > >     {".cbl", "@cobol", 0, 0, 0},
> > >     {".CBL", "@cobol", 0, 0, 0},
> > >     {"@cobol", 
> > > 	"cobol1 %i %(cc1_options) %{!fsyntax-only:%(invoke_as)}", 
> > > 	0, 0, 0}, 
> > 
> > It misses %(cpp_unique_options) which was the reason why your -I
> > arguments weren't passed to cobol1.  
> 
> If I understood you correctly, I don't need to modify gcc.cc.  I only
> need to modify cobol/lang-specs.h, which I've done.  But that's
> evidently not all I need to do, because it doesn't seem to work.  
> 
> The last element in the fragment in cobol/lang-specs.h is now: 
> 
>     {"@cobol",
> 	"cobol1 %i %(cc1_options) %{!fsyntax-only:%(invoke_as)} "
> 	"%(cpp_unique_options) ",

%(invoke_as) needs to be last.  What it does is effectively add this to 
the command line (under certain conditions): "-somemoreoptions | as".  
Note the pipe symbol.  Like in normal shell commands also the gcc driver 
interprets this as "and now start the following command as well connection 
stdout of the first to stdin of the second".  So all in all the generated 
cmdline will be somewhat like:

  cobol1 input.cbl -stuff-from-cc1-options | as - -stuff-from-cpp-options

Your cpp_unique_options addition will effectively be options to that 'as' 
command, but you wanted it to be options for cobol1.  So, just switch 
order of elements.

> I see the -B and -I options, and others, with their arguments, contained
> in COLLECT_GCC_OPTIONS on lines 9 and 11.  I guess that represents an
> environment string?

Yes.  It's our round-about-way of passing the gcc options as the user gave 
them downwards in case collect2 (a wrapper for the linking step for, gah, 
don't ask) needs to call gcc itself recursively.  But in the -### (or -v) 
output, if the assembler is invoked in your example (i.e. cobol1 doesn't 
fail for some reason) you should see your -I options being passed to that 
one (wrongly so, of course :) ).


Ciao,
Michael.