Re: [RFC] database with API information

Re: [RFC] database with API information

[Richard Sandiford]

Ulrich Drepper via Gcc <gcc@gcc.gnu.org> writes:
> I talked to Jonathan the other day about adding all the C++ library APIs to
> the name hint file now that the size of the table is not really a concern
> anymore.
>
> Jonathan mentioned that he has to create and maintain a similar file for
> the module support.  It needs to list all the exported interfaces and this
> is mostly a superset of the entries in the hint table.
>
> Instead of duplicating the information it should be kept in one place.
> Neither file itself is a natural fit because the additional information
> needed  (e.g., the standard version information for the name hint table) is
> not needed in the other location.
>
> Hence, let's use a simple database, a CSV file for simplicity, and generate
> both files from this.  Easily done, I have an appropriate script and a CSV
> file with the information of both Jonathan's current export file and the
> current state of the name hint table.
>
> The only detail that keeps me from submitting this right now is the way the
> script is implemented.  This is just a natural fit for a Python script.
> The default installation comes with a csv module and there are nice ways to
> adjust and output boilerplate headers like those needed in those files.
>
> It would be possible to create separate awk scripts (there is only one
> Python script) but it'll be rather ugly and harder to maintain than the
> Python version.
>
> Of course the problem is: I don't think that there is yet any maintainer
> tool written in Python (except some release engineering tools).  The
> question is therefore: is it time to lift this restriction?  I cannot today
> imagine any machine capable of serving a gcc developer which doesn't also
> have a Python implementation.  As long as there is no dependency on exotic
> modules I doubt that anything will break.

FWIW, I agree it's past time to lift the no-Python restriction,
and that Python is a natural fit for stuff like this.

Thanks,
Richard

Re: [RFC] database with API information

[Martin Liška]

On 9/7/22 12:56, Richard Sandiford via Gcc wrote:
> Ulrich Drepper via Gcc <gcc@gcc.gnu.org> writes:
>> I talked to Jonathan the other day about adding all the C++ library APIs to
>> the name hint file now that the size of the table is not really a concern
>> anymore.
>>
>> Jonathan mentioned that he has to create and maintain a similar file for
>> the module support.  It needs to list all the exported interfaces and this
>> is mostly a superset of the entries in the hint table.
>>
>> Instead of duplicating the information it should be kept in one place.
>> Neither file itself is a natural fit because the additional information
>> needed  (e.g., the standard version information for the name hint table) is
>> not needed in the other location.
>>
>> Hence, let's use a simple database, a CSV file for simplicity, and generate
>> both files from this.  Easily done, I have an appropriate script and a CSV
>> file with the information of both Jonathan's current export file and the
>> current state of the name hint table.
>>
>> The only detail that keeps me from submitting this right now is the way the
>> script is implemented.  This is just a natural fit for a Python script.
>> The default installation comes with a csv module and there are nice ways to
>> adjust and output boilerplate headers like those needed in those files.
>>
>> It would be possible to create separate awk scripts (there is only one
>> Python script) but it'll be rather ugly and harder to maintain than the
>> Python version.
>>
>> Of course the problem is: I don't think that there is yet any maintainer
>> tool written in Python (except some release engineering tools).  The
>> question is therefore: is it time to lift this restriction?  I cannot today
>> imagine any machine capable of serving a gcc developer which doesn't also
>> have a Python implementation.  As long as there is no dependency on exotic
>> modules I doubt that anything will break.
> 
> FWIW, I agree it's past time to lift the no-Python restriction,
> and that Python is a natural fit for stuff like this.

+1 for me!

Martin

> 
> Thanks,
> Richard

Re: [RFC] database with API information

[SAIFI]

> 
> let's use a simple database, a CSV file for simplicity, and generate both files from this.

Curious to learn, why wouldn't you want to do it in C++ itself ?

As an example, please see
https://github.com/cplusplus/lib-issues-software

warm regards
Saifi.

Re: [RFC] database with API information

[Jonathan Wakely]

On Fri, 9 Sept 2022 at 12:17, SAIFI wrote:
>
> >
> > let's use a simple database, a CSV file for simplicity, and generate both files from this.
>
> Curious to learn, why wouldn't you want to do it in C++ itself ?
>
> As an example, please see
> https://github.com/cplusplus/lib-issues-software

That repo has been dead for years, maybe you want the still maintained code at:
https://github.com/cplusplus/LWG/tree/master/src

I'm not persuaded that 4000 lines of C++ code for processing XML into
HTML is really a good alternative to 150 lines of Python.

Re: [RFC] database with API information

[SAIFI]

On 9 September 2022 11:47:30 am UTC, Jonathan Wakely <jwakely.gcc@gmail.com> wrote:
>On Fri, 9 Sept 2022 at 12:17, SAIFI wrote:
>>
>> >
>> > let's use a simple database, a CSV file for simplicity, and generate both files from this.
>>
>> Curious to learn, why wouldn't you want to do it in C++ itself ?
>>
>> As an example, please see
>> https://github.com/cplusplus/lib-issues-software
>
>That repo has been dead for years, maybe you want the still maintained code at:
>https://github.com/cplusplus/LWG/tree/master/src
>
>I'm not persuaded that 4000 lines of C++ code for processing XML into
>HTML is really a good alternative to 150 lines of Python.

To quote again, the OP words for emphasis, "let's use a simple database, a CSV file for simplicity, and generate both files from this."

Do you think the effort estimate to implement this requirement is "4000 lines of C++" ? 

Am i missing something here ?

warm regards
Saifi.

Re: [RFC] database with API information

[Jonathan Wakely]

On Fri, 9 Sept 2022 at 13:06, SAIFI <msk@strikr.io> wrote:
>
> On 9 September 2022 11:47:30 am UTC, Jonathan Wakely <jwakely.gcc@gmail.com> wrote:
> >On Fri, 9 Sept 2022 at 12:17, SAIFI wrote:
> >>
> >> >
> >> > let's use a simple database, a CSV file for simplicity, and generate both files from this.
> >>
> >> Curious to learn, why wouldn't you want to do it in C++ itself ?
> >>
> >> As an example, please see
> >> https://github.com/cplusplus/lib-issues-software
> >
> >That repo has been dead for years, maybe you want the still maintained code at:
> >https://github.com/cplusplus/LWG/tree/master/src
> >
> >I'm not persuaded that 4000 lines of C++ code for processing XML into
> >HTML is really a good alternative to 150 lines of Python.
>
> To quote again, the OP words for emphasis, "let's use a simple database, a CSV file for simplicity, and generate both files from this."
>
> Do you think the effort estimate to implement this requirement is "4000 lines of C++" ?

No, but it was you who linked to the LWG repo with 4000 lines of C++,
I don't know why.

> Am i missing something here ?

OK ...

This script would be needed to bootstrap GCC. Using C++ would mean
that we need to restrict it to C++11 (since that's all that is needed
for bootstrapping GCC), and deal with all the peculiarities of the
platform's C++ compiler and linker. We already do that to build GCC
itself, but now we'd also need to do it for this tool that has to be
built first, and would need to use all the right CXXFLAGS and LDFLAGS
etc. to compile on the build machine (which might have unconventional
linker properties, for example). We'd also need to implement a CSV
parser from scratch.

Or we could just use 150 lines of Python, which already has a csv
module to do the parsing correctly.

C++ is a great tool for some things. For parsing a CSV file and
printing out some text there are simpler tools.

Re: [RFC] database with API information

[Iain Sandoe]

> On 7 Sep 2022, at 13:33, Martin Liška <mliska@suse.cz> wrote:
> 
> On 9/7/22 12:56, Richard Sandiford via Gcc wrote:
>> Ulrich Drepper via Gcc <gcc@gcc.gnu.org> writes:
>>> I talked to Jonathan the other day about adding all the C++ library APIs to
>>> the name hint file now that the size of the table is not really a concern
>>> anymore.
>>> 
>>> Jonathan mentioned that he has to create and maintain a similar file for
>>> the module support.  It needs to list all the exported interfaces and this
>>> is mostly a superset of the entries in the hint table.
>>> 
>>> Instead of duplicating the information it should be kept in one place.
>>> Neither file itself is a natural fit because the additional information
>>> needed  (e.g., the standard version information for the name hint table) is
>>> not needed in the other location.
>>> 
>>> Hence, let's use a simple database, a CSV file for simplicity, and generate
>>> both files from this.  Easily done, I have an appropriate script and a CSV
>>> file with the information of both Jonathan's current export file and the
>>> current state of the name hint table.
>>> 
>>> The only detail that keeps me from submitting this right now is the way the
>>> script is implemented.  This is just a natural fit for a Python script.
>>> The default installation comes with a csv module and there are nice ways to
>>> adjust and output boilerplate headers like those needed in those files.
>>> 
>>> It would be possible to create separate awk scripts (there is only one
>>> Python script) but it'll be rather ugly and harder to maintain than the
>>> Python version.
>>> 
>>> Of course the problem is: I don't think that there is yet any maintainer
>>> tool written in Python (except some release engineering tools).  The
>>> question is therefore: is it time to lift this restriction?  I cannot today
>>> imagine any machine capable of serving a gcc developer which doesn't also
>>> have a Python implementation.  As long as there is no dependency on exotic
>>> modules I doubt that anything will break.
>> 
>> FWIW, I agree it's past time to lift the no-Python restriction,
>> and that Python is a natural fit for stuff like this.

( no objection to using Python here )

One small request, I realise that Python 2 is dead, but I regularly bootstrap GCC
on older machines that only have Python 2 installations.  If possible (and it sounds
plausible if the job is really quite simple) - it would be good to support those older
machines without having to take a detour to find a way to build Python 3 on them first.

Iain

Re: [RFC] database with API information

[SAIFI]

>  -------Original Message-------
>  From: Jonathan Wakely <jwakely.gcc@gmail.com>
>  
>  > Am i missing something here ?
>  
>  OK ...
>  
>  This script would be needed to bootstrap GCC. Using C++ would mean
>  that we need to restrict it to C++11 (since that's all that is needed
>  for bootstrapping GCC), and deal with all the peculiarities of the
>  platform's C++ compiler and linker. We already do that to build GCC
>  itself, but now we'd also need to do it for this tool that has to be
>  built first, and would need to use all the right CXXFLAGS and LDFLAGS
>  etc. to compile on the build machine (which might have unconventional
>  linker properties, for example). 

Perhaps this is something the current build scripts take care of anyway. Is it possible to reuse ?


> We'd also need to implement a CSV parser from scratch.
>  

Would any of the C++11 compliant CSV parsing libraries make the cut in your analysis ?

Vince CSV parser (RFC4180 + flavours)
https://github.com/vincentlaucsb/csv-parser

C++11 header only (MT can be turned on/off).
https://github.com/ben-strasser/fast-cpp-csv-parser

If not, Is there is something else that is needed for the CSV parsers to qualify ?

Thanks for sharing your perspective. Much appreciated !


warm regards
Saifi.

Re: [RFC] database with API information

[Iain Sandoe]

> On 9 Sep 2022, at 18:07, Ulrich Drepper via Gcc <gcc@gcc.gnu.org> wrote:
> 
> On Fri, Sep 9, 2022 at 5:26 PM Iain Sandoe <idsandoe@googlemail.com> wrote:
> 
>> One small request, I realise that Python 2 is dead, but I regularly
>> bootstrap GCC
>> on older machines that only have Python 2 installations.  If possible (and
>> it sounds
>> plausible if the job is really quite simple) - it would be good to support
>> those older
>> machines without having to take a detour to find a way to build Python 3
>> on them first.
>> 
> 
> Would this really be an issue?  Just as is the case for the gperf-generated
> files, the repository would contain the generated files and gperf/python
> would only be needed if someone changes those files or explicitly removes
> them.

If that’s the case then no, I handle most update/commit stuff on newer machines
 - it is probably a misinterpretation of “this would be needed for bootstrap” in
another part of the thread.

Iain

Re: [RFC] database with API information

[SAIFI]

>  -------Original Message-------
>  From: Jonathan Wakely <jwakely.gcc@gmail.com>
> 
>  > Would any of the C++11 compliant CSV parsing libraries make the cut
>  > in your analysis ?
>  
>  I'm not going to waste my time looking.
>  
>  > Vince CSV parser (RFC4180 + flavours)
>  > https://github.com/vincentlaucsb/csv-parser
>  >
>  > C++11 header only (MT can be turned on/off).
>  > https://github.com/ben-strasser/fast-cpp-csv-parser
>  >
>  > If not, Is there is something else that is needed for the CSV
>  > parsers to qualify ?
>  
>  Why add an extra dependency that would have to be included in the GCC
>  sources, when Python already has what's needed to do this.
>  
>  Somebody has written a new tool, and you seem to be asking them to
>  rewrite it for your preferences. If you want to rewrite the script in
>  C++ feel free to proposed a patch, but this digression for your
>  curiousity doesn't seem productive to me.

Well, it seems then the 'innocent CSV' proposed by Ulrich is the strawman that leads to meson !

Please feel free to correct me if that is not the 'unstated' intent.


warm regards
Saifi.

Re: [RFC] database with API information

[Jonathan Wakely]

On Fri, 9 Sept 2022 at 19:41, SAIFI <msk@strikr.io> wrote:
>
> >  -------Original Message-------
> >  From: Jonathan Wakely <jwakely.gcc@gmail.com>
> >
> >  > Would any of the C++11 compliant CSV parsing libraries make the cut
> >  > in your analysis ?
> >
> >  I'm not going to waste my time looking.
> >
> >  > Vince CSV parser (RFC4180 + flavours)
> >  > https://github.com/vincentlaucsb/csv-parser
> >  >
> >  > C++11 header only (MT can be turned on/off).
> >  > https://github.com/ben-strasser/fast-cpp-csv-parser
> >  >
> >  > If not, Is there is something else that is needed for the CSV
> >  > parsers to qualify ?
> >
> >  Why add an extra dependency that would have to be included in the GCC
> >  sources, when Python already has what's needed to do this.
> >
> >  Somebody has written a new tool, and you seem to be asking them to
> >  rewrite it for your preferences. If you want to rewrite the script in
> >  C++ feel free to proposed a patch, but this digression for your
> >  curiousity doesn't seem productive to me.
>
> Well, it seems then the 'innocent CSV' proposed by Ulrich is the strawman that leads to meson !
>
> Please feel free to correct me if that is not the 'unstated' intent.

I have no idea what you're even talking about.