Porting the Docs to Sphinx - project status

Porting the Docs to Sphinx - project status

[Martin Liška]

Hello.

It's about 5 months since the last project status update:
https://gcc.gnu.org/pipermail/gcc-patches/2021-August/577108.html
Now it's pretty clear that it won't be merged before GCC 12.1 gets released.

So where we are? I contacted documentation maintainers (Gerald, Sandra and Joseph) at the
end of the year in a private email, where I pinged the patches. My take away is that both
Gerald and Joseph are fine with the porting, while Sandra has some concerns. Based on her
feedback, I was able to improve the PDF generated output significantly and I'm pleased by the
provided feedback. That led to the following 2 Sphinx pulls requests that need to be merged
before we can migrate the documentation: [1], [2].

Since the last time I also made one more round of proofreading and the layout was improved
(mainly for PDF part). Current version of the documentation can be seen here:
https://splichal.eu/scripts/sphinx/

I would like to finish the transition once GCC 12.1 gets released in May/June this year.
There are still some minor regressions, but overall the Sphinx-based documentation should
be a significant improvement over what we've got right now.

Please take this email as urgent call for a feedback!

Thank you,
Martin

[1] https://github.com/sphinx-doc/sphinx/pull/10087
[2] https://github.com/sphinx-doc/sphinx/pull/10001

Re: Porting the Docs to Sphinx - project status

[Matthias Klose]

On 1/31/22 15:06, Martin Liška wrote:
> Hello.
> 
> It's about 5 months since the last project status update:
> https://gcc.gnu.org/pipermail/gcc-patches/2021-August/577108.html
> Now it's pretty clear that it won't be merged before GCC 12.1 gets released.
> 
> So where we are? I contacted documentation maintainers (Gerald, Sandra and
> Joseph) at the
> end of the year in a private email, where I pinged the patches. My take away is
> that both
> Gerald and Joseph are fine with the porting, while Sandra has some concerns.
> Based on her
> feedback, I was able to improve the PDF generated output significantly and I'm
> pleased by the
> provided feedback. That led to the following 2 Sphinx pulls requests that need
> to be merged
> before we can migrate the documentation: [1], [2].
> 
> Since the last time I also made one more round of proofreading and the layout
> was improved
> (mainly for PDF part). Current version of the documentation can be seen here:
> https://splichal.eu/scripts/sphinx/
> 
> I would like to finish the transition once GCC 12.1 gets released in May/June
> this year.
> There are still some minor regressions, but overall the Sphinx-based
> documentation should
> be a significant improvement over what we've got right now.
> 
> Please take this email as urgent call for a feedback!

Please take care about the copyrights.  I only checked the D frontend manual,
and this one suddenly has a copyright with invariant sections, compared to the
current gdc.texi which has a copyright *without* the invariant sections.  Debian
doesn't allow me to ship documentation with invariant sections ...

I didn't look how much you reorganized the sources, but it would nice to split
the files into those documenting command line options (used to generate the man
pages) and other documentation.  This is already done for gcc/doc, but not for
other frontends.  It would allow having manual pages with a copyright requiring
front and back cover texts in the manual pages.

It would also be nice to require the latest sphinx version (and probably some
plugins), so that distros can build the docs with older sphinx versions as well.

Matthias

Re: Porting the Docs to Sphinx - project status

[Martin Liška]

On 2/4/22 14:40, Matthias Klose wrote:
> On 1/31/22 15:06, Martin Liška wrote:
>> Hello.
>>
>> It's about 5 months since the last project status update:
>> https://gcc.gnu.org/pipermail/gcc-patches/2021-August/577108.html
>> Now it's pretty clear that it won't be merged before GCC 12.1 gets released.
>>
>> So where we are? I contacted documentation maintainers (Gerald, Sandra and
>> Joseph) at the
>> end of the year in a private email, where I pinged the patches. My take away is
>> that both
>> Gerald and Joseph are fine with the porting, while Sandra has some concerns.
>> Based on her
>> feedback, I was able to improve the PDF generated output significantly and I'm
>> pleased by the
>> provided feedback. That led to the following 2 Sphinx pulls requests that need
>> to be merged
>> before we can migrate the documentation: [1], [2].
>>
>> Since the last time I also made one more round of proofreading and the layout
>> was improved
>> (mainly for PDF part). Current version of the documentation can be seen here:
>> https://splichal.eu/scripts/sphinx/
>>
>> I would like to finish the transition once GCC 12.1 gets released in May/June
>> this year.
>> There are still some minor regressions, but overall the Sphinx-based
>> documentation should
>> be a significant improvement over what we've got right now.
>>
>> Please take this email as urgent call for a feedback!
> 
> Please take care about the copyrights.  I only checked the D frontend manual,
> and this one suddenly has a copyright with invariant sections, compared to the
> current gdc.texi which has a copyright *without* the invariant sections.  Debian
> doesn't allow me to ship documentation with invariant sections ...

Oh, thank you very much for the pointer. I didn't notice the Copyright sections
differ quite a lot. It should be fixed now.

> 
> I didn't look how much you reorganized the sources, but it would nice to split
> the files into those documenting command line options (used to generate the man
> pages) and other documentation.

Well, the current splitting is done into multiple .rst files and a bunch of them
actually constructs command line options. Please check View page source button
on each HTML page.

> This is already done for gcc/doc, but not for
> other frontends.  It would allow having manual pages with a copyright requiring
> front and back cover texts in the manual pages.

How exactly does it work? Does it mean you don't use official GCC tarballs?
I would expect you just package built man/info pages and don't distribute PDF/HTML
version of a documenation, or?

> 
> It would also be nice to require the latest sphinx version (and probably some
> plugins), so that distros can build the docs with older sphinx versions as well.

I'm sorry but this would be very difficult. It's mainly caused by fact I've reported
quite some changes to upstream, where having them leads to a reasonable HTML/PDF output.

Note you can quite easily utilize pip&virtualenv for Sphinx installation.

Cheers,
Martin

> 
> Matthias

Re: Porting the Docs to Sphinx - project status

[Martin Liška]

On 1/31/22 15:06, Martin Liška wrote:
> Hello.
> 
> It's about 5 months since the last project status update:
> https://gcc.gnu.org/pipermail/gcc-patches/2021-August/577108.html
> Now it's pretty clear that it won't be merged before GCC 12.1 gets released.
> 
> So where we are? I contacted documentation maintainers (Gerald, Sandra and Joseph) at the
> end of the year in a private email, where I pinged the patches. My take away is that both
> Gerald and Joseph are fine with the porting, while Sandra has some concerns. Based on her
> feedback, I was able to improve the PDF generated output significantly and I'm pleased by the
> provided feedback. That led to the following 2 Sphinx pulls requests that need to be merged
> before we can migrate the documentation: [1], [2].
> 
> Since the last time I also made one more round of proofreading and the layout was improved
> (mainly for PDF part). Current version of the documentation can be seen here:
> https://splichal.eu/scripts/sphinx/
> 
> I would like to finish the transition once GCC 12.1 gets released in May/June this year.
> There are still some minor regressions, but overall the Sphinx-based documentation should
> be a significant improvement over what we've got right now.
> 
> Please take this email as urgent call for a feedback!
> 
> Thank you,
> Martin
> 
> [1] https://github.com/sphinx-doc/sphinx/pull/10087
> [2] https://github.com/sphinx-doc/sphinx/pull/10001

Hello.

It's been another 5 months since the last project status update. In order to address some Sandra comments
I had to work with upstream in order to merge a few pull requests. That has been achieved and Sphinx 5.1.0
can built the converted documentation.

What has changed since the last time:
1) I made a couple of proof-reading rounds and the documentation should not contain any significant defects.
2) I used a more responsive HTML template Furo that works nicely on mobile devices as well.
3) Number of Sphinx warnings has been reduced.
4) configure script supports not --with-sphinx-build ([1]) that can be used when Sphinx is installed into pip env.
5) Building Documentation chapter was added to gccint manual ([2]).
6) I made a couple of screenshot that show up how is the docs better ([3], Chapter 1).

I'm not planning sending a patchset as most of the patches would not fit in the email limit, so please fetch the following
branch (the last 2 fixme commits will not be installed):

$ git fetch origin refs/users/marxin/heads/sphinx-v7
$ git checkout FETCH_HEAD

The generated documentation (built from GCC source based on the git branch) can be seen here:
https://splichal.eu/gccsphinx-final/

I would like to merge the documentation during the summer if possible before a bigger changes will land in fall.

Thank you,
Martin

[1] https://splichal.eu/gccsphinx-final/html/install/configuration.html#cmdoption-with-sphinx-build
[2] https://splichal.eu/gccsphinx-final/html/gccint/source-tree-structure-and-build-system/the-gcc-subdirectory/building-documentation.html
[3] https://splichal.eu/scripts/sphinx/demo/_build/latex/demo.pdf

Announcement: Porting the Docs to Sphinx - 9. November 2022

[Martin Liška]

Hello.

Based on the very positive feedback I was given at the Cauldron Sphinx Documentation BoF,
I'm planning migrating the documentation on 9th November. There are still some minor comments
from Sandra when it comes to the PDF output, but we can address that once the conversion is done.

The reason I'm sending the email now is that I was waiting for latest Sphinx release (5.3.0) that
simplifies reference format for options and results in much simpler Option summary section ([1])

The current GCC master (using Sphinx 5.3.0) converted docs can be seen here:
https://splichal.eu/scripts/sphinx/

If you see any issues with the converted documentation, or have a feedback about it,
please reply to this email.

Cheers,
Martin

[1] https://github.com/sphinx-doc/sphinx/pull/10840
[1] https://splichal.eu/scripts/sphinx/gcc/_build/html/gcc-command-options/option-summary.html

Re: Announcement: Porting the Docs to Sphinx - 9. November 2022

[Paul Iannetta]

Hi Martin,

Thank you very much for porting the documentation to Sphinx, it is
very convenient to use, especially the menu on the left and the
search bar.

However, I also regularly browse and search the documentation through
info, especially when I want to use regexps to search or need to
include a special character (eg.,+,-,_,(; this can happen when I
search for '(define' ) for example) in the search string.

Does the port to Sphinx means the end of texinfo? Or, will both be
available as it is the case now with the official texinfo and your
unofficial splichal.eu pages.

Paul

On Mon, Oct 17, 2022 at 03:28:34PM +0200, Martin Liška wrote:
> Hello.
> 
> Based on the very positive feedback I was given at the Cauldron Sphinx Documentation BoF,
> I'm planning migrating the documentation on 9th November. There are still some minor comments
> from Sandra when it comes to the PDF output, but we can address that once the conversion is done.
> 
> The reason I'm sending the email now is that I was waiting for latest Sphinx release (5.3.0) that
> simplifies reference format for options and results in much simpler Option summary section ([1])
> 
> The current GCC master (using Sphinx 5.3.0) converted docs can be seen here:
> https://splichal.eu/scripts/sphinx/
> 
> If you see any issues with the converted documentation, or have a feedback about it,
> please reply to this email.
> 
> Cheers,
> Martin
> 
> [1] https://github.com/sphinx-doc/sphinx/pull/10840
> [1] https://splichal.eu/scripts/sphinx/gcc/_build/html/gcc-command-options/option-summary.html
> 
> 
> 
> 

Re: Announcement: Porting the Docs to Sphinx - 9. November 2022

[Martin Liška]

On 10/17/22 16:16, Paul Iannetta wrote:
> Hi Martin,
> 
> Thank you very much for porting the documentation to Sphinx, it is
> very convenient to use, especially the menu on the left and the
> search bar.

Thanks, I also like it!

> 
> However, I also regularly browse and search the documentation through
> info, especially when I want to use regexps to search or need to
> include a special character (eg.,+,-,_,(; this can happen when I
> search for '(define' ) for example) in the search string.
> 
> Does the port to Sphinx means the end of texinfo? Or, will both be
> available as it is the case now with the official texinfo and your
> unofficial splichal.eu pages.

It will be still available same as now where manual pages and info pages
are built if you compile GCC from sources. We haven't been publishing manual
pages and info pages on our web pages, people typically get these from
their distribution packages.

Does it help? Or do you expect any change regarding what we publish at:
https://gcc.gnu.org/onlinedocs/
?

Cheers,
Martin

> 
> Paul
> 
> On Mon, Oct 17, 2022 at 03:28:34PM +0200, Martin Liška wrote:
>> Hello.
>>
>> Based on the very positive feedback I was given at the Cauldron Sphinx Documentation BoF,
>> I'm planning migrating the documentation on 9th November. There are still some minor comments
>> from Sandra when it comes to the PDF output, but we can address that once the conversion is done.
>>
>> The reason I'm sending the email now is that I was waiting for latest Sphinx release (5.3.0) that
>> simplifies reference format for options and results in much simpler Option summary section ([1])
>>
>> The current GCC master (using Sphinx 5.3.0) converted docs can be seen here:
>> https://splichal.eu/scripts/sphinx/
>>
>> If you see any issues with the converted documentation, or have a feedback about it,
>> please reply to this email.
>>
>> Cheers,
>> Martin
>>
>> [1] https://github.com/sphinx-doc/sphinx/pull/10840
>> [1] https://splichal.eu/scripts/sphinx/gcc/_build/html/gcc-command-options/option-summary.html
>>
>>
>>
>>
> 
> 
> 
> 

Re: Announcement: Porting the Docs to Sphinx - 9. November 2022

[Paul Iannetta]

On Wed, Oct 19, 2022 at 09:24:06AM +0200, Martin Liška wrote:
> On 10/17/22 16:16, Paul Iannetta wrote:
> > Hi Martin,
> > 
> > Thank you very much for porting the documentation to Sphinx, it is
> > very convenient to use, especially the menu on the left and the
> > search bar.
> 
> Thanks, I also like it!
> 
> > 
> > However, I also regularly browse and search the documentation through
> > info, especially when I want to use regexps to search or need to
> > include a special character (eg.,+,-,_,(; this can happen when I
> > search for '(define' ) for example) in the search string.
> > 
> > Does the port to Sphinx means the end of texinfo? Or, will both be
> > available as it is the case now with the official texinfo and your
> > unofficial splichal.eu pages.
> 
> It will be still available same as now where manual pages and info pages
> are built if you compile GCC from sources. We haven't been publishing manual
> pages and info pages on our web pages, people typically get these from
> their distribution packages.

As long as it is possible to build the info manual with "make info", even through
something like rst2texinfo,  I would be happy.  Would it be possible
to see the rst source of the port so as to try rst2texinfo on it?

> 
> Does it help? Or do you expect any change regarding what we publish at:
> https://gcc.gnu.org/onlinedocs/
> ?
Currently, there is a tarball with texinfo sources for all the manuals
for each version.

Thanks,
Paul

> 
> Cheers,
> Martin
> 
> > 
> > Paul
> > 
> > On Mon, Oct 17, 2022 at 03:28:34PM +0200, Martin Liška wrote:
> >> Hello.
> >>
> >> Based on the very positive feedback I was given at the Cauldron Sphinx Documentation BoF,
> >> I'm planning migrating the documentation on 9th November. There are still some minor comments
> >> from Sandra when it comes to the PDF output, but we can address that once the conversion is done.
> >>
> >> The reason I'm sending the email now is that I was waiting for latest Sphinx release (5.3.0) that
> >> simplifies reference format for options and results in much simpler Option summary section ([1])
> >>
> >> The current GCC master (using Sphinx 5.3.0) converted docs can be seen here:
> >> https://splichal.eu/scripts/sphinx/
> >>
> >> If you see any issues with the converted documentation, or have a feedback about it,
> >> please reply to this email.
> >>
> >> Cheers,
> >> Martin
> >>
> >> [1] https://github.com/sphinx-doc/sphinx/pull/10840
> >> [1] https://splichal.eu/scripts/sphinx/gcc/_build/html/gcc-command-options/option-summary.html
> >>
> >>
> >>
> >>
> > 
> > 
> > 
> > 
> 
> 
> 
> 
> 

Re: Announcement: Porting the Docs to Sphinx - 9. November 2022

[Martin Liška]

On 10/19/22 10:13, Paul Iannetta wrote:
> On Wed, Oct 19, 2022 at 09:24:06AM +0200, Martin Liška wrote:
>> On 10/17/22 16:16, Paul Iannetta wrote:
>>> Hi Martin,
>>>
>>> Thank you very much for porting the documentation to Sphinx, it is
>>> very convenient to use, especially the menu on the left and the
>>> search bar.
>>
>> Thanks, I also like it!
>>
>>>
>>> However, I also regularly browse and search the documentation through
>>> info, especially when I want to use regexps to search or need to
>>> include a special character (eg.,+,-,_,(; this can happen when I
>>> search for '(define' ) for example) in the search string.
>>>
>>> Does the port to Sphinx means the end of texinfo? Or, will both be
>>> available as it is the case now with the official texinfo and your
>>> unofficial splichal.eu pages.
>>
>> It will be still available same as now where manual pages and info pages
>> are built if you compile GCC from sources. We haven't been publishing manual
>> pages and info pages on our web pages, people typically get these from
>> their distribution packages.
> 
> As long as it is possible to build the info manual with "make info", even through
> something like rst2texinfo,  I would be happy.  Would it be possible
> to see the rst source of the port so as to try rst2texinfo on it?

Well, .rst source files can be seen right now here:
https://github.com/marxin/texi2rst-generated

And 'texinfo' is created with the standard Sphinx builder:
https://www.sphinx-doc.org/en/master/man/sphinx-build.html#cmdoption-sphinx-build-b

> 
>>
>> Does it help? Or do you expect any change regarding what we publish at:
>> https://gcc.gnu.org/onlinedocs/
>> ?
> Currently, there is a tarball with texinfo sources for all the manuals
> for each version.

Well, then equivalent would be packaging all .rst files together with the corresponding
conf.py, logo.* and other files. But I don't see it much useful.

Thanks,
Martin

> 
> Thanks,
> Paul
> 
>>
>> Cheers,
>> Martin
>>
>>>
>>> Paul
>>>
>>> On Mon, Oct 17, 2022 at 03:28:34PM +0200, Martin Liška wrote:
>>>> Hello.
>>>>
>>>> Based on the very positive feedback I was given at the Cauldron Sphinx Documentation BoF,
>>>> I'm planning migrating the documentation on 9th November. There are still some minor comments
>>>> from Sandra when it comes to the PDF output, but we can address that once the conversion is done.
>>>>
>>>> The reason I'm sending the email now is that I was waiting for latest Sphinx release (5.3.0) that
>>>> simplifies reference format for options and results in much simpler Option summary section ([1])
>>>>
>>>> The current GCC master (using Sphinx 5.3.0) converted docs can be seen here:
>>>> https://splichal.eu/scripts/sphinx/
>>>>
>>>> If you see any issues with the converted documentation, or have a feedback about it,
>>>> please reply to this email.
>>>>
>>>> Cheers,
>>>> Martin
>>>>
>>>> [1] https://github.com/sphinx-doc/sphinx/pull/10840
>>>> [1] https://splichal.eu/scripts/sphinx/gcc/_build/html/gcc-command-options/option-summary.html
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>>
>>
>>
>>
>>
>>
> 
> 
> 
> 

Re: Announcement: Porting the Docs to Sphinx - 9. November 2022

[Joseph Myers]

[-- Attachment #1: Type: text/plain, Size: 683 bytes --]

On Wed, 19 Oct 2022, Martin Liška wrote:

> > Currently, there is a tarball with texinfo sources for all the manuals
> > for each version.
> 
> Well, then equivalent would be packaging all .rst files together with the corresponding
> conf.py, logo.* and other files. But I don't see it much useful.

I think we should have such a source tarball when the sources are .rst, as 
the successor to the Texinfo source tarball.

(Unfortunately when I added that source tarball - 
https://gcc.gnu.org/legacy-ml/gcc-patches/2004-01/msg00140.html - I didn't 
give specific references to any of the individual requests that resulted 
in adding it.)

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: Announcement: Porting the Docs to Sphinx - 9. November 2022

[Martin Liška]

On 10/19/22 18:42, Joseph Myers wrote:
> On Wed, 19 Oct 2022, Martin Liška wrote:
> 
>>> Currently, there is a tarball with texinfo sources for all the manuals
>>> for each version.
>>
>> Well, then equivalent would be packaging all .rst files together with the corresponding
>> conf.py, logo.* and other files. But I don't see it much useful.
> 
> I think we should have such a source tarball when the sources are .rst, as 
> the successor to the Texinfo source tarball.

All right, putting that on my TODO list after the conversion happens. Won't be difficult
to achieve.

Cheers,
Martin

> 
> (Unfortunately when I added that source tarball - 
> https://gcc.gnu.org/legacy-ml/gcc-patches/2004-01/msg00140.html - I didn't 
> give specific references to any of the individual requests that resulted 
> in adding it.)

Re: Announcement: Porting the Docs to Sphinx - 9. November 2022

[Sandra Loosemore]

On 10/17/22 07:28, Martin Liška wrote:
> Hello.
> 
> Based on the very positive feedback I was given at the Cauldron Sphinx Documentation BoF,
> I'm planning migrating the documentation on 9th November. There are still some minor comments
> from Sandra when it comes to the PDF output, but we can address that once the conversion is done.

My main complaint about the PDF is that the blue color used for link 
text is so light it interferes with readability.  Few people are going 
to print the document on paper any more, but I did try printing a sample 
page on a grayscale printer and the blue link text came out so faint 
that it was barely visible at all.  An E-ink reader device would 
probably have similar problems.

I'm generally not a fan of the other colors being used for formatting, 
either.  To me it seems like they all interfere with readability, plus 
in code samples it seems like random things get highlighted in random 
colors, instead of focusing on the thing the example is trying to 
demonstrate.

I've been preferring to use the PDF form of the GNU manuals because it 
is easier to search the whole document that way.  The search feature in 
the new web version doesn't quite cut it....  it gives you a list of web 
pages and then you have to do a second browser search within each page 
to find the reference.  So I hope we can continue to support the PDF as 
a canonical format and better tune it for easy readability, instead of 
assuming that most people will only care about the online web version.

-Sandra

Re: Announcement: Porting the Docs to Sphinx - 9. November 2022

[Martin Liška]

On 10/18/22 00:26, Sandra Loosemore wrote:
> On 10/17/22 07:28, Martin Liška wrote:
>> Hello.
>>
>> Based on the very positive feedback I was given at the Cauldron Sphinx Documentation BoF,
>> I'm planning migrating the documentation on 9th November. There are still some minor comments
>> from Sandra when it comes to the PDF output, but we can address that once the conversion is done.
> 
> My main complaint about the PDF is that the blue color used for link text is so light it interferes with readability.  Few people are going to print the document on paper any more, but I did try printing a sample page on a grayscale printer and the blue link text came out so faint that it was barely visible at all.

Sure, I've just added support for monochromatic PDF output where one needs to use
MONOCHROMATIC=1 make latexpdf ...

and I linked the file here:
https://splichal.eu/scripts/sphinx/gcc/_build/latexmonochromatic/gcc.pdf

right now I build only one PDF in this mode and it's mentioned here:
https://splichal.eu/scripts/sphinx/

What do you think about it now?

>  An E-ink reader device would probably have similar problems.

There ePUB would be likely better output format. What do you think?

Martin

> 
> I'm generally not a fan of the other colors being used for formatting, either.  To me it seems like they all interfere with readability, plus in code samples it seems like random things get highlighted in random colors, instead of focusing on the thing the example is trying to demonstrate.
> 
> I've been preferring to use the PDF form of the GNU manuals because it is easier to search the whole document that way.  The search feature in the new web version doesn't quite cut it....  it gives you a list of web pages and then you have to do a second browser search within each page to find the reference.  So I hope we can continue to support the PDF as a canonical format and better tune it for easy readability, instead of assuming that most people will only care about the online web version.
> 
> -Sandra
> 
> 
> 
> 

Re: Announcement: Porting the Docs to Sphinx - 9. November 2022

[Martin Liška]

On 10/19/22 13:09, Martin Liška wrote:
> There ePUB would be likely better output format. What do you think?

I've just included ePUB books:
https://splichal.eu/scripts/sphinx/#epub

Martin

Re: Announcement: Porting the Docs to Sphinx - 9. November 2022

[Sandra Loosemore]

On 10/19/22 05:09, Martin Liška wrote:
> On 10/18/22 00:26, Sandra Loosemore wrote:
>> On 10/17/22 07:28, Martin Liška wrote:
>>> Hello.
>>>
>>> Based on the very positive feedback I was given at the Cauldron Sphinx Documentation BoF,
>>> I'm planning migrating the documentation on 9th November. There are still some minor comments
>>> from Sandra when it comes to the PDF output, but we can address that once the conversion is done.
>>
>> My main complaint about the PDF is that the blue color used for link text is so light it interferes with readability.  Few people are going to print the document on paper any more, but I did try printing a sample page on a grayscale printer and the blue link text came out so faint that it was barely visible at all.
> 
> Sure, I've just added support for monochromatic PDF output where one needs to use
> MONOCHROMATIC=1 make latexpdf ...
> 
> and I linked the file here:
> https://splichal.eu/scripts/sphinx/gcc/_build/latexmonochromatic/gcc.pdf
> 
> right now I build only one PDF in this mode and it's mentioned here:
> https://splichal.eu/scripts/sphinx/
> 
> What do you think about it now?

Hmmm, removing *all* visual cues that something is a link does not seem 
so great either, especially since the new format has changed the link 
text for @xref to remove the page and section information.  E.g. we used 
to get "See Section 3.4 [Options Controlling C Dialect], page 44." and 
now it just reads "See Options Controlling C Dialect."

I realize there is a can of worms here involving philosophical issues 
about whether the PDF manual is intended to be formatted for reading as 
a book or is just a handy way to repackage the hyperlinked web 
presentation for offline reference.  Also there is another can of worms 
involving making the documentation accessible to people who have visual 
disabilities, specifically color blindness issues.  Just speaking for 
myself, I'd be happy if the PDF just used a darker blue color for links 
that is both distinguishing and higher contrast with the background than 
the current light blue, but I think it is one of the principles of 
accessible design that color really shouldn't be the *only* indication 
of something that initiates an action.  Maybe underlining, or a little 
link glyph, or restoring the section/page info to the link text?

> 
>>    An E-ink reader device would probably have similar problems.
> 
> There ePUB would be likely better output format. What do you think?

Ooof, a lot of problems there.  I looked at your new generated .epub in 
both the "ebook-viewer" utility on my laptop and on my Kobo Forma.  The 
Kobo uses the default proportionally-spaced font for everything; even 
the code examples fail to come out in a fixed-width font.  ebook-viewer 
shows fixed-width fonts for code examples and inline references to e.g. 
command line options, but the names of options in the option tables 
sections are in the proportional body font.  Also in both viewers I see 
hyperlinks to https://splicha.eu/...  in place of internal links in some 
references to command-line options and the like, and the formatting of 
the option summary tables really sucks, with lines breaking at hyphens 
in the middle of option names.

I suggest we try to focus our efforts on the currently-supported formats 
before adding EPUB as a new format.

-Sandra

Re: Announcement: Porting the Docs to Sphinx - 9. November 2022

[Martin Liška]

On 10/19/22 18:30, Sandra Loosemore wrote:
> On 10/19/22 05:09, Martin Liška wrote:
>> On 10/18/22 00:26, Sandra Loosemore wrote:
>>> On 10/17/22 07:28, Martin Liška wrote:
>>>> Hello.
>>>>
>>>> Based on the very positive feedback I was given at the Cauldron Sphinx Documentation BoF,
>>>> I'm planning migrating the documentation on 9th November. There are still some minor comments
>>>> from Sandra when it comes to the PDF output, but we can address that once the conversion is done.
>>>
>>> My main complaint about the PDF is that the blue color used for link text is so light it interferes with readability.  Few people are going to print the document on paper any more, but I did try printing a sample page on a grayscale printer and the blue link text came out so faint that it was barely visible at all.
>>
>> Sure, I've just added support for monochromatic PDF output where one needs to use
>> MONOCHROMATIC=1 make latexpdf ...
>>
>> and I linked the file here:
>> https://splichal.eu/scripts/sphinx/gcc/_build/latexmonochromatic/gcc.pdf
>>
>> right now I build only one PDF in this mode and it's mentioned here:
>> https://splichal.eu/scripts/sphinx/
>>
>> What do you think about it now?
> 
> Hmmm, removing *all* visual cues that something is a link does not seem so great either, especially since the new format has changed the link text for @xref to remove the page and section information.  E.g. we used to get "See Section 3.4 [Options Controlling C Dialect], page 44." and now it just reads "See Options Controlling C Dialect."
> 

Hey.

> I realize there is a can of worms here involving philosophical issues about whether the PDF manual is intended to be formatted for reading as a book or is just a handy way to repackage the hyperlinked web presentation for offline reference.  Also there is another can of worms involving making the documentation accessible to people who have visual disabilities, specifically color blindness issues.  Just speaking for myself, I'd be happy if the PDF just used a darker blue color for links that is both distinguishing and higher contrast with the background than the current light blue, but I think it is one of the principles of accessible design that color really shouldn't be the *only* indication of something that initiates an action.  Maybe underlining, or a little link glyph, or restoring the section/page info to the link text?

I've just tweaked the monochrom. PDF where dark blue color is used for links. About the links, there are multiple PDF viewers (like Evince) which can do a preview if you hover
over a link. Plus a page number is showed in a toolbar.

What it comes to the philosophical issues of the monochrom. PDF, well, I would recommend discussing that with Sphinx upstream project. I bet
they must have other projects who's readers might request similar needs. Intention of my monochrom. PDF was to show that Sphinx PDF output
can be quite easily adjusted.

> 
>>
>>>    An E-ink reader device would probably have similar problems.
>>
>> There ePUB would be likely better output format. What do you think?
> 
> Ooof, a lot of problems there.  I looked at your new generated .epub in both the "ebook-viewer" utility on my laptop and on my Kobo Forma.  The Kobo uses the default proportionally-spaced font for everything; even the code examples fail to come out in a fixed-width font.  ebook-viewer shows fixed-width fonts for code examples and inline references to e.g. command line options, but the names of options in the option tables sections are in the proportional body font.  Also in both viewers I see hyperlinks to https://splicha.eu/...  in place of internal links in some references to command-line options and the like, and the formatting of the option summary tables really sucks, with lines breaking at hyphens in the middle of option names.

Sure, let's leave it for now and keep it as a might-have thing for the future!

Appreciate the feedback,
Cheers,
Martin

> 
> I suggest we try to focus our efforts on the currently-supported formats before adding EPUB as a new format.
> 
> -Sandra

Re: Announcement: Porting the Docs to Sphinx - 9. November 2022

[Xi Ruoyao]

On Mon, 2022-10-17 at 15:28 +0200, Martin Liška wrote:
> Hello.
> 
> Based on the very positive feedback I was given at the Cauldron Sphinx Documentation BoF,
> I'm planning migrating the documentation on 9th November. There are still some minor comments
> from Sandra when it comes to the PDF output, but we can address that once the conversion is done.
> 
> The reason I'm sending the email now is that I was waiting for latest Sphinx release (5.3.0) that
> simplifies reference format for options and results in much simpler Option summary section ([1])
> 
> The current GCC master (using Sphinx 5.3.0) converted docs can be seen here:
> https://splichal.eu/scripts/sphinx/
> 
> If you see any issues with the converted documentation, or have a feedback about it,
> please reply to this email.

Ouch.  This will be very painful for Linux From Scratch.  We'll need to
add 23 Python modules to build the documentation, while we only have 88
packages in total currently...  And we don't want to omit GCC
documentation in our system.

Could generated man and info pages be provided as a tarball on
gcc.gnu.org or ftp.gnu.org?

Re: Announcement: Porting the Docs to Sphinx - 9. November 2022

[Martin Liška]

On 10/20/22 04:26, Xi Ruoyao wrote:
> On Mon, 2022-10-17 at 15:28 +0200, Martin Liška wrote:
>> Hello.
>>
>> Based on the very positive feedback I was given at the Cauldron Sphinx Documentation BoF,
>> I'm planning migrating the documentation on 9th November. There are still some minor comments
>> from Sandra when it comes to the PDF output, but we can address that once the conversion is done.
>>
>> The reason I'm sending the email now is that I was waiting for latest Sphinx release (5.3.0) that
>> simplifies reference format for options and results in much simpler Option summary section ([1])
>>
>> The current GCC master (using Sphinx 5.3.0) converted docs can be seen here:
>> https://splichal.eu/scripts/sphinx/
>>
>> If you see any issues with the converted documentation, or have a feedback about it,
>> please reply to this email.
> 
> Ouch.  This will be very painful for Linux From Scratch.  We'll need to
> add 23 Python modules to build the documentation, while we only have 88
> packages in total currently...  And we don't want to omit GCC
> documentation in our system.

Various other distros will have to face it too. The proper solution is a multi-build
package (gcc:doc) which can be built later in the dependency chain. Btw. do you also
provide PDF documentation in your system?

> 
> Could generated man and info pages be provided as a tarball on
> gcc.gnu.org or ftp.gnu.org?

Not planning doing that.

Cheers,
Martin

Re: Announcement: Porting the Docs to Sphinx - 9. November 2022

[Xi Ruoyao]

(CC our team members.)

On Thu, 2022-10-20 at 13:27 +0200, Martin Liška wrote:
> > Ouch.  This will be very painful for Linux From Scratch.  We'll need to
> > add 23 Python modules to build the documentation, while we only have 88
> > packages in total currently...  And we don't want to omit GCC
> > documentation in our system.
> 
> Various other distros will have to face it too. The proper solution is a multi-build
> package (gcc:doc) which can be built later in the dependency chain. Btw. do you also
> provide PDF documentation in your system?

No (texlive is much heavier than Sphinx).  But generally we expect man
pages and info pages.

We can separate man and info into the second-time build in BLFS (we're
already doing this now for Go, Objective C, etc.), but I don't really
like to omit the man and info pages...

> > Could generated man and info pages be provided as a tarball on
> > gcc.gnu.org or ftp.gnu.org?
> 
> Not planning doing that.

Re: Announcement: Porting the Docs to Sphinx - 9. November 2022

[Martin Liška]

On 10/20/22 13:49, Xi Ruoyao wrote:
> (CC our team members.)
> 
> On Thu, 2022-10-20 at 13:27 +0200, Martin Liška wrote:
>>> Ouch.  This will be very painful for Linux From Scratch.  We'll need to
>>> add 23 Python modules to build the documentation, while we only have 88
>>> packages in total currently...  And we don't want to omit GCC
>>> documentation in our system.
>>
>> Various other distros will have to face it too. The proper solution is a multi-build
>> package (gcc:doc) which can be built later in the dependency chain. Btw. do you also
>> provide PDF documentation in your system?
> 
> No (texlive is much heavier than Sphinx).  But generally we expect man
> pages and info pages.
> 
> We can separate man and info into the second-time build in BLFS (we're
> already doing this now for Go, Objective C, etc.),

Do the same for GCC.

> but I don't really
> like to omit the man and info pages..

What should I do about it? We want to switch to a more modern documentation tool
called Sphinx and yes, it will make packaging of the GCC more complicated.

Martin

> 
>>> Could generated man and info pages be provided as a tarball on
>>> gcc.gnu.org or ftp.gnu.org?
>>
>> Not planning doing that.

Re: Announcement: Porting the Docs to Sphinx - 9. November 2022

[Xi Ruoyao]

On Thu, 2022-10-20 at 13:53 +0200, Martin Liška wrote:
> On 10/20/22 13:49, Xi Ruoyao wrote:
> > (CC our team members.)
> > 
> > On Thu, 2022-10-20 at 13:27 +0200, Martin Liška wrote:
> > > > Ouch.  This will be very painful for Linux From Scratch.  We'll need to
> > > > add 23 Python modules to build the documentation, while we only have 88
> > > > packages in total currently...  And we don't want to omit GCC
> > > > documentation in our system.
> > > 
> > > Various other distros will have to face it too. The proper solution is a multi-build
> > > package (gcc:doc) which can be built later in the dependency chain. Btw. do you also
> > > provide PDF documentation in your system?
> > 
> > No (texlive is much heavier than Sphinx).  But generally we expect man
> > pages and info pages.
> > 
> > We can separate man and info into the second-time build in BLFS (we're
> > already doing this now for Go, Objective C, etc.),
> 
> Do the same for GCC.
> 
> > but I don't really
> > like to omit the man and info pages..
> 
> What should I do about it? We want to switch to a more modern documentation tool
> called Sphinx and yes, it will make packaging of the GCC more complicated.

Nothing, I guess.  We'll handle it on our side (if we finally decide to
ship the man/info tarballs we can generate them by ourselves).

I was just trying to find a simpler solution before beginning all the
work :).

Thanks!

Re: Announcement: Porting the Docs to Sphinx - 9. November 2022

[Martin Liška]

On 10/20/22 13:55, Xi Ruoyao wrote:
> On Thu, 2022-10-20 at 13:53 +0200, Martin Liška wrote:
>> On 10/20/22 13:49, Xi Ruoyao wrote:
>>> (CC our team members.)
>>>
>>> On Thu, 2022-10-20 at 13:27 +0200, Martin Liška wrote:
>>>>> Ouch.  This will be very painful for Linux From Scratch.  We'll need to
>>>>> add 23 Python modules to build the documentation, while we only have 88
>>>>> packages in total currently...  And we don't want to omit GCC
>>>>> documentation in our system.
>>>>
>>>> Various other distros will have to face it too. The proper solution is a multi-build
>>>> package (gcc:doc) which can be built later in the dependency chain. Btw. do you also
>>>> provide PDF documentation in your system?
>>>
>>> No (texlive is much heavier than Sphinx).  But generally we expect man
>>> pages and info pages.
>>>
>>> We can separate man and info into the second-time build in BLFS (we're
>>> already doing this now for Go, Objective C, etc.),
>>
>> Do the same for GCC.
>>
>>> but I don't really
>>> like to omit the man and info pages..
>>
>> What should I do about it? We want to switch to a more modern documentation tool
>> called Sphinx and yes, it will make packaging of the GCC more complicated.
> 
> Nothing, I guess.  We'll handle it on our side (if we finally decide to
> ship the man/info tarballs we can generate them by ourselves).

Good!

> 
> I was just trying to find a simpler solution before beginning all the
> work :).

Sure, makes sense.

Martin

> 
> Thanks!
> 

Re: Announcement: Porting the Docs to Sphinx - 9. November 2022

[Joseph Myers]

[-- Attachment #1: Type: text/plain, Size: 1428 bytes --]

On Thu, 20 Oct 2022, Martin Liška wrote:

> > Could generated man and info pages be provided as a tarball on
> > gcc.gnu.org or ftp.gnu.org?
> 
> Not planning doing that.

Release tarballs (but not snapshots) currently include the info files and 
man pages, via gcc_release running a build with 
--enable-generated-files-in-srcdir before building the tarball.

I think they should continue to do so.  This means:

(a) --enable-generated-files-in-srcdir needs to cause those files to be 
generated in the source directory, as it does at present.

(b) gcc_release, for building a release but not a snapshot, needs to give 
an error if Sphinx is missing or too old and so those files weren't built 
properly (and thus people running gcc_release to build a release tarball 
will need new-enough Sphinx).

(c) It needs to be verified that building and installing from such a 
release tarball works even if Sphinx is missing or too old - that is, that 
it installs the prebuilt info / man files rather than giving an error or 
failing to install them.

Also, but not strictly part of the release issue:

(d) Builds with missing or old Sphinx should work regardless of whether 
such files are in the source directory - but if they aren't in the source 
directory, the effect of missing or old Sphinx (detected at configure 
time) should be to disable building and installing documentation.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: Announcement: Porting the Docs to Sphinx - 9. November 2022

[Martin Liška]

On 10/20/22 17:35, Joseph Myers wrote:
> On Thu, 20 Oct 2022, Martin Liška wrote:
> 
>>> Could generated man and info pages be provided as a tarball on
>>> gcc.gnu.org or ftp.gnu.org?
>>
>> Not planning doing that.
> 
> Release tarballs (but not snapshots) currently include the info files and 
> man pages, via gcc_release running a build with 
> --enable-generated-files-in-srcdir before building the tarball.
> 
> I think they should continue to do so.  This means:
> 
> (a) --enable-generated-files-in-srcdir needs to cause those files to be 
> generated in the source directory, as it does at present.
> 
> (b) gcc_release, for building a release but not a snapshot, needs to give 
> an error if Sphinx is missing or too old and so those files weren't built 
> properly (and thus people running gcc_release to build a release tarball 
> will need new-enough Sphinx).
> 
> (c) It needs to be verified that building and installing from such a 
> release tarball works even if Sphinx is missing or too old - that is, that 
> it installs the prebuilt info / man files rather than giving an error or 
> failing to install them.
> 
> Also, but not strictly part of the release issue:
> 
> (d) Builds with missing or old Sphinx should work regardless of whether 
> such files are in the source directory - but if they aren't in the source 
> directory, the effect of missing or old Sphinx (detected at configure 
> time) should be to disable building and installing documentation.

All right Joseph, is it something you're willing to help me once we start
using Sphinx? Apparently, there will be many consequent steps after we switch.

Cheers,
Martin

Re: Announcement: Porting the Docs to Sphinx - 9. November 2022

[Joseph Myers]

[-- Attachment #1: Type: text/plain, Size: 1722 bytes --]

On Thu, 20 Oct 2022, Martin Liška wrote:

> > Also, but not strictly part of the release issue:
> > 
> > (d) Builds with missing or old Sphinx should work regardless of whether 
> > such files are in the source directory - but if they aren't in the source 
> > directory, the effect of missing or old Sphinx (detected at configure 
> > time) should be to disable building and installing documentation.
> 
> All right Joseph, is it something you're willing to help me once we start
> using Sphinx? Apparently, there will be many consequent steps after we switch.

Sure, but most of the conditionals are *already* present, just need 
updating as part of the Sphinx transition.  E.g. gcc/Makefile.in has 
BUILD_INFO and GENERATED_MANPAGES conditionals based on configure tests 
for whether relevant tools are present and new enough; the rules for 
$(DESTDIR)$(infodir)/%.info quietly allow the info files not to be 
present, so installing also works without the info files or tools to build 
them, and the rules for installing man pages similarly ignore errors; and 
there are srcinfo and srcman rules, enabled based on @GENINSRC@, to copy 
those built files to the source directory, which are what's used when 
--enable-generated-files-in-srcdir is used as part of building a release 
tarball.

The main thing I've suggested that I think may actually be new is an error 
for trying to build a release tarball without new-enough Sphinx (I think 
the current rules would quietly not copy info / man pages to the source 
directory if build tools were missing - but having those tools missing 
when building a release tarball is much less likely than not having 
new-enough Sphinx).

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: Announcement: Porting the Docs to Sphinx - 9. November 2022

[Jakub Jelinek]

On Thu, Oct 20, 2022 at 04:43:06PM +0000, Joseph Myers wrote:
> On Thu, 20 Oct 2022, Martin Liška wrote:
> 
> > > Also, but not strictly part of the release issue:
> > > 
> > > (d) Builds with missing or old Sphinx should work regardless of whether 
> > > such files are in the source directory - but if they aren't in the source 
> > > directory, the effect of missing or old Sphinx (detected at configure 
> > > time) should be to disable building and installing documentation.
> > 
> > All right Joseph, is it something you're willing to help me once we start
> > using Sphinx? Apparently, there will be many consequent steps after we switch.
> 
> Sure, but most of the conditionals are *already* present, just need 
> updating as part of the Sphinx transition.  E.g. gcc/Makefile.in has 
> BUILD_INFO and GENERATED_MANPAGES conditionals based on configure tests 
> for whether relevant tools are present and new enough; the rules for 
> $(DESTDIR)$(infodir)/%.info quietly allow the info files not to be 
> present, so installing also works without the info files or tools to build 
> them, and the rules for installing man pages similarly ignore errors; and 
> there are srcinfo and srcman rules, enabled based on @GENINSRC@, to copy 
> those built files to the source directory, which are what's used when 
> --enable-generated-files-in-srcdir is used as part of building a release 
> tarball.
> 
> The main thing I've suggested that I think may actually be new is an error 
> for trying to build a release tarball without new-enough Sphinx (I think 
> the current rules would quietly not copy info / man pages to the source 
> directory if build tools were missing - but having those tools missing 
> when building a release tarball is much less likely than not having 
> new-enough Sphinx).

But perhaps that test should go to maintainer-scripts/gcc_release.
Can be either of the form of checking if Sphinx is new enough, or checking
of make actually built the documentation before creating the tarballs.

	Jakub

Announcement: Porting the Docs to Sphinx - tomorrow

[Martin Liška]

Hi.

Tomorrow in the morning (UTC time), I'm going to migrate the documentation
to Sphinx. The final version of the branch can be seen here:

$ git fetch origin refs/users/marxin/heads/sphinx-final
$ git co FETCH_HEAD 

URL: https://splichal.eu/gccsphinx-final/

TL;DR;

After the migration, people should be able to build (and install) GCC even
if they miss Sphinx (similar happens now if you miss makeinfo). However, please
install Sphinx >= 5.3.0 (for manual and info pages - only *core* package is necessary) [1]

Steps following the migration:

1) update of web HTML (and PDF documentation) pages:
   I prepared a script and tested our server has all what we need.
2) gcc_release --enable-generated-files-in-srcdir: here I would like
   to ask Joseph for cooperation
3) URL for diagnostics (used for warning) - will utilize [3]
4) package source tarballs - https://gcc.gnu.org/onlinedocs/ (listed here)
5) updating links from gcc.gnu.org that point to documentation
6) removal of the further Texinfo leftovers
...

Cheers,
Martin

[1] https://splichal.eu/scripts/sphinx/gccint/_build/html/source-tree-structure-and-build-system/the-gcc-subdirectory/building-documentation.html#sphinx-install
[2] ./maintainer-scripts/update_web_docs_git.py
[3] https://pypi.org/project/sphinx-redirect-by-id/

Re: Announcement: Porting the Docs to Sphinx - tomorrow

[Sam James]

[-- Attachment #1: Type: text/plain, Size: 1367 bytes --]



> On 8 Nov 2022, at 13:55, Martin Liška <mliska@suse.cz> wrote:
> 
> Hi.
> 
> Tomorrow in the morning (UTC time), I'm going to migrate the documentation
> to Sphinx. The final version of the branch can be seen here:
> 
> $ git fetch origin refs/users/marxin/heads/sphinx-final
> $ git co FETCH_HEAD
> 
> URL: https://splichal.eu/gccsphinx-final/
> 
> TL;DR;
> 
> After the migration, people should be able to build (and install) GCC even
> if they miss Sphinx (similar happens now if you miss makeinfo). However, please
> install Sphinx >= 5.3.0 (for manual and info pages - only *core* package is necessary) [1]
> 
> Steps following the migration:
> 
> 1) update of web HTML (and PDF documentation) pages:
>   I prepared a script and tested our server has all what we need.
> 2) gcc_release --enable-generated-files-in-srcdir: here I would like
>   to ask Joseph for cooperation

Yes, please (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106899)
even for snapshots? Pretty please? :)

> 3) URL for diagnostics (used for warning) - will utilize [3]
> 4) package source tarballs - https://gcc.gnu.org/onlinedocs/ (listed here)
> 5) updating links from gcc.gnu.org that point to documentation
> 6) removal of the further Texinfo leftovers
> ...
> 

Thanks for working on this. Very excited.

> Cheers,
> Martin

Best,
sam

[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 358 bytes --]

Re: Announcement: Porting the Docs to Sphinx - tomorrow

[Joseph Myers]

On Tue, 8 Nov 2022, Sam James via Gcc wrote:

> Yes, please (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106899)
> even for snapshots? Pretty please? :)

I think we want snapshots to come out weekly even if the compiler or 
documentation build fails, which makes anything involving a build as part 
of the snapshot process problematic.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: Announcement: Porting the Docs to Sphinx - tomorrow

[Sam James]

[-- Attachment #1: Type: text/plain, Size: 731 bytes --]



> On 9 Nov 2022, at 00:00, Joseph Myers <joseph@codesourcery.com> wrote:
> 
> On Tue, 8 Nov 2022, Sam James via Gcc wrote:
> 
>> Yes, please (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106899)
>> even for snapshots? Pretty please? :)
> 
> I think we want snapshots to come out weekly even if the compiler or
> documentation build fails, which makes anything involving a build as part
> of the snapshot process problematic.

If that's your expectation, that's fine, but I'd regard it as pretty
serious if one didn't build, and I don't remember a time when it didn't.

It's not like it's that much use if it fails to build on a bog-standard
amd64 platform anyway, as if nothing else, you'd get a deluge
of duplicate bug reports.

[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 358 bytes --]

Re: Announcement: Porting the Docs to Sphinx - tomorrow

[Richard Biener]

On Wed, Nov 9, 2022 at 1:09 AM Sam James via Gcc-patches
<gcc-patches@gcc.gnu.org> wrote:
>
>
>
> > On 9 Nov 2022, at 00:00, Joseph Myers <joseph@codesourcery.com> wrote:
> >
> > On Tue, 8 Nov 2022, Sam James via Gcc wrote:
> >
> >> Yes, please (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106899)
> >> even for snapshots? Pretty please? :)
> >
> > I think we want snapshots to come out weekly even if the compiler or
> > documentation build fails, which makes anything involving a build as part
> > of the snapshot process problematic.
>
> If that's your expectation, that's fine, but I'd regard it as pretty
> serious if one didn't build, and I don't remember a time when it didn't.
>
> It's not like it's that much use if it fails to build on a bog-standard
> amd64 platform anyway, as if nothing else, you'd get a deluge
> of duplicate bug reports.

I'd say that doing a trunk snapshot build every day as CI would be nice, we
can then publish one once a week, skipping days where the build failed.

For release branches having generated files in the snapshots would be
even more useful and I very much hope the build there always succeeds.

Richard.

Re: Announcement: Porting the Docs to Sphinx - tomorrow

[Mark Wielaard]

Hi,

On Wed, Nov 09, 2022 at 08:49:22AM +0100, Richard Biener via Gcc wrote:
> On Wed, Nov 9, 2022 at 1:09 AM Sam James via Gcc-patches
> <gcc-patches@gcc.gnu.org> wrote:
> > > On 9 Nov 2022, at 00:00, Joseph Myers <joseph@codesourcery.com> wrote:
> > > On Tue, 8 Nov 2022, Sam James via Gcc wrote:
> >
> > If that's your expectation, that's fine, but I'd regard it as pretty
> > serious if one didn't build, and I don't remember a time when it didn't.
> >
> > It's not like it's that much use if it fails to build on a bog-standard
> > amd64 platform anyway, as if nothing else, you'd get a deluge
> > of duplicate bug reports.
> 
> I'd say that doing a trunk snapshot build every day as CI would be nice, we
> can then publish one once a week, skipping days where the build failed.
> 
> For release branches having generated files in the snapshots would be
> even more useful and I very much hope the build there always succeeds.

The current buildbot CI jobs already run various builds which just do
a simple non-bootstrap build (plus some that do a full build and make
check) to see if things work:
https://builder.sourceware.org/buildbot/#/builders?tags=gcc&tags=gcc-full

If we make sure the relevant machine/containers contain the necessary
prerequirements then can add the configure flags to some builders to
make sure the docs are generated:
https://sourceware.org/git/?p=builder.git;a=blob;f=README

And make it sent warning emails (to gcc-testresults) when things break.

Cheers,

Mark

Re: Announcement: Porting the Docs to Sphinx - tomorrow

[Joseph Myers]

On Wed, 9 Nov 2022, Richard Biener via Gcc wrote:

> I'd say that doing a trunk snapshot build every day as CI would be nice, we
> can then publish one once a week, skipping days where the build failed.

Note that each snapshot should have diffs relative to the previous 
published snapshot.  Not relative to some other one that was not 
published, for example.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: Announcement: Porting the Docs to Sphinx - tomorrow

[Richard Biener]

On Wed, Nov 9, 2022 at 6:11 PM Joseph Myers <joseph@codesourcery.com> wrote:
>
> On Wed, 9 Nov 2022, Richard Biener via Gcc wrote:
>
> > I'd say that doing a trunk snapshot build every day as CI would be nice, we
> > can then publish one once a week, skipping days where the build failed.
>
> Note that each snapshot should have diffs relative to the previous
> published snapshot.  Not relative to some other one that was not
> published, for example.

Sure, but diffs can be created from the two snapshot tarballs at
publishing time.

Richard.

>
> --
> Joseph S. Myers
> joseph@codesourcery.com

Re: Announcement: Porting the Docs to Sphinx - tomorrow

[Gerald Pfeifer]

[-- Attachment #1: Type: text/plain, Size: 1370 bytes --]

On Tue, 8 Nov 2022, Martin Liška wrote:
> After the migration, people should be able to build (and install) GCC 
> even if they miss Sphinx (similar happens now if you miss makeinfo). 

My nightly *install* (not build) on amd64-unknown-freebsd12.2 broke 
(from what I can tell due to this - it's been working fine most of 
the last several 1000 days):

  if [ -f doc/g++.1 ]; then rm -f /home/gerald/gcc-ref12-amd64/share/man/man1/g++.1; /usr/bin/install -c -m 644 doc/g++.1 /home/gerald/gcc-ref12-amd64/share/man/man1/g++.1; chmod a-x /home/gerald/gcc-ref12-amd64/share/man/man1/g++.1; fimake -C /scratch/tmp/gerald/GCC-HEAD/gcc/../doc man SOURCEDIR=/scratch/tmp/gerald/GCC-HEAD/gcc/fortran/doc/gfortran BUILDDIR=/scratch/tmp/gerald/OBJ-1111-0954/gcc/doc/gfortran/man SPHINXBUILD=
  make[3]: make[3]: don't know how to make w. Stop
  make[3]: stopped in /scratch/tmp/gerald/GCC-HEAD/doc
  gmake[2]: *** [/scratch/tmp/gerald/GCC-HEAD/gcc/fortran/Make-lang.in:164: doc/gfortran/man/man/gfortran.1] Error 2
  gmake[2]: Leaving directory '/scratch/tmp/gerald/OBJ-1111-0954/gcc'
  gmake[1]: *** [Makefile:5310: install-strip-gcc] Error 2
  gmake[1]: Leaving directory '/scratch/tmp/gerald/OBJ-1111-0954'
  gmake: *** [Makefile:2734: install-strip] Error 2

(This appears to be the case with "make -j1 install-strip". Not sure where 
that "w" target is coming from?)

Gerald

Re: Announcement: Porting the Docs to Sphinx - tomorrow

[Sandra Loosemore]

On 11/11/22 13:52, Gerald Pfeifer wrote:
> On Tue, 8 Nov 2022, Martin Liška wrote:
>> After the migration, people should be able to build (and install) GCC
>> even if they miss Sphinx (similar happens now if you miss makeinfo).
> 
> My nightly *install* (not build) on amd64-unknown-freebsd12.2 broke
> (from what I can tell due to this - it's been working fine most of
> the last several 1000 days):
> 
>    if [ -f doc/g++.1 ]; then rm -f /home/gerald/gcc-ref12-amd64/share/man/man1/g++.1; /usr/bin/install -c -m 644 doc/g++.1 /home/gerald/gcc-ref12-amd64/share/man/man1/g++.1; chmod a-x /home/gerald/gcc-ref12-amd64/share/man/man1/g++.1; fimake -C /scratch/tmp/gerald/GCC-HEAD/gcc/../doc man SOURCEDIR=/scratch/tmp/gerald/GCC-HEAD/gcc/fortran/doc/gfortran BUILDDIR=/scratch/tmp/gerald/OBJ-1111-0954/gcc/doc/gfortran/man SPHINXBUILD=
>    make[3]: make[3]: don't know how to make w. Stop
>    make[3]: stopped in /scratch/tmp/gerald/GCC-HEAD/doc
>    gmake[2]: *** [/scratch/tmp/gerald/GCC-HEAD/gcc/fortran/Make-lang.in:164: doc/gfortran/man/man/gfortran.1] Error 2
>    gmake[2]: Leaving directory '/scratch/tmp/gerald/OBJ-1111-0954/gcc'
>    gmake[1]: *** [Makefile:5310: install-strip-gcc] Error 2
>    gmake[1]: Leaving directory '/scratch/tmp/gerald/OBJ-1111-0954'
>    gmake: *** [Makefile:2734: install-strip] Error 2
> 
> (This appears to be the case with "make -j1 install-strip". Not sure where
> that "w" target is coming from?)

I've seen something similar:  "make install" seems to be passing an 
empty SPHINXBUILD= option to the docs Makefile which is not equipped to 
handle that.  I know the fix is to get a recent-enough version of Sphinx 
installed (and I'm going to work on that over the weekend), but it ought 
to fail more gracefully, or not try to install docs that cannot be built 
without Sphinx.

-Sandra

Re: Announcement: Porting the Docs to Sphinx - tomorrow

[Martin Liška]

On 11/11/22 22:10, Sandra Loosemore wrote:
> On 11/11/22 13:52, Gerald Pfeifer wrote:
>> On Tue, 8 Nov 2022, Martin Liška wrote:
>>> After the migration, people should be able to build (and install) GCC
>>> even if they miss Sphinx (similar happens now if you miss makeinfo).
>>
>> My nightly *install* (not build) on amd64-unknown-freebsd12.2 broke
>> (from what I can tell due to this - it's been working fine most of
>> the last several 1000 days):
>>
>>    if [ -f doc/g++.1 ]; then rm -f /home/gerald/gcc-ref12-amd64/share/man/man1/g++.1; /usr/bin/install -c -m 644 doc/g++.1 /home/gerald/gcc-ref12-amd64/share/man/man1/g++.1; chmod a-x /home/gerald/gcc-ref12-amd64/share/man/man1/g++.1; fimake -C /scratch/tmp/gerald/GCC-HEAD/gcc/../doc man SOURCEDIR=/scratch/tmp/gerald/GCC-HEAD/gcc/fortran/doc/gfortran BUILDDIR=/scratch/tmp/gerald/OBJ-1111-0954/gcc/doc/gfortran/man SPHINXBUILD=
>>    make[3]: make[3]: don't know how to make w. Stop
>>    make[3]: stopped in /scratch/tmp/gerald/GCC-HEAD/doc
>>    gmake[2]: *** [/scratch/tmp/gerald/GCC-HEAD/gcc/fortran/Make-lang.in:164: doc/gfortran/man/man/gfortran.1] Error 2
>>    gmake[2]: Leaving directory '/scratch/tmp/gerald/OBJ-1111-0954/gcc'
>>    gmake[1]: *** [Makefile:5310: install-strip-gcc] Error 2
>>    gmake[1]: Leaving directory '/scratch/tmp/gerald/OBJ-1111-0954'
>>    gmake: *** [Makefile:2734: install-strip] Error 2
>>
>> (This appears to be the case with "make -j1 install-strip". Not sure where
>> that "w" target is coming from?)
> 
> I've seen something similar:  "make install" seems to be passing an empty SPHINXBUILD= option to the docs Makefile which is not equipped to handle that.  I know the fix is to get a recent-enough version of Sphinx installed (and I'm going to work on that over the weekend), but it ought to fail more gracefully, or not try to install docs that cannot be built without Sphinx.
> 
> -Sandra
> 

Can you please update the current master, you should get a proper error message.

I'm going to take a look at make install-strip target.

Martin

Re: Announcement: Porting the Docs to Sphinx - 9. November 2022

[Martin Liška]

On 10/20/22 18:43, Joseph Myers wrote:
> On Thu, 20 Oct 2022, Martin Liška wrote:
> 
>>> Also, but not strictly part of the release issue:
>>>
>>> (d) Builds with missing or old Sphinx should work regardless of whether 
>>> such files are in the source directory - but if they aren't in the source 
>>> directory, the effect of missing or old Sphinx (detected at configure 
>>> time) should be to disable building and installing documentation.
>>
>> All right Joseph, is it something you're willing to help me once we start
>> using Sphinx? Apparently, there will be many consequent steps after we switch.
> 
> Sure, but most of the conditionals are *already* present, just need 
> updating as part of the Sphinx transition.  E.g. gcc/Makefile.in has 
> BUILD_INFO and GENERATED_MANPAGES conditionals based on configure tests 
> for whether relevant tools are present and new enough; the rules for 
> $(DESTDIR)$(infodir)/%.info quietly allow the info files not to be 
> present, so installing also works without the info files or tools to build 
> them, and the rules for installing man pages similarly ignore errors; and 
> there are srcinfo and srcman rules, enabled based on @GENINSRC@, to copy 
> those built files to the source directory, which are what's used when 
> --enable-generated-files-in-srcdir is used as part of building a release 
> tarball.
> 
> The main thing I've suggested that I think may actually be new is an error 
> for trying to build a release tarball without new-enough Sphinx (I think 
> the current rules would quietly not copy info / man pages to the source 
> directory if build tools were missing - but having those tools missing 
> when building a release tarball is much less likely than not having 
> new-enough Sphinx).
> 

Hello Joseph.

So the transition is done and I would to ask you Joseph for help with the
following 2 items:

1) not synchronized content among lib*/Makefile.in and lib*/Makefile.am.
Apparently, I modified the generated Makefile.in file with the rules like:

doc/info/texinfo/libitm.info: $(SPHINX_FILES)
	+ if [ x$(HAS_SPHINX_BUILD) = xhas-sphinx-build ]; then \
	  make -C $(srcdir)/../doc info SOURCEDIR=$(abs_srcdir)/doc BUILDDIR=$(abs_doc_builddir)/info SPHINXBUILD=$(SPHINX_BUILD); \
	else true; fi

Can you please modify Makefile.am in the corresponding manner and re-generate Makefile.in?

2) Adding proper support --enable-generated-files-in-srcdir in gcc_release:

As mentioned above in the quoted email.

Thanks in advance,
Martin

Re: Announcement: Porting the Docs to Sphinx - 9. November 2022

[Joseph Myers]

[-- Attachment #1: Type: text/plain, Size: 1098 bytes --]

On Wed, 9 Nov 2022, Martin Liška wrote:

> 1) not synchronized content among lib*/Makefile.in and lib*/Makefile.am.
> Apparently, I modified the generated Makefile.in file with the rules like:
> 
> doc/info/texinfo/libitm.info: $(SPHINX_FILES)
> 	+ if [ x$(HAS_SPHINX_BUILD) = xhas-sphinx-build ]; then \
> 	  make -C $(srcdir)/../doc info SOURCEDIR=$(abs_srcdir)/doc BUILDDIR=$(abs_doc_builddir)/info SPHINXBUILD=$(SPHINX_BUILD); \
> 	else true; fi
> 
> Can you please modify Makefile.am in the corresponding manner and re-generate Makefile.in?

I think someone else had best look at this.

> 2) Adding proper support --enable-generated-files-in-srcdir in gcc_release:

It looks like all the GENINSRC rules / conditionals are still present.  
So maybe there are details where the paths are wrong, or where fixes are 
needed to ensure the files get installed from the source directory when 
available in the source directory but not available in the build directory 
because Sphinx isn't available, but much of the code for the feature is 
still there.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: Announcement: Porting the Docs to Sphinx - 9. November 2022

[Martin Liška]

On 11/9/22 18:14, Joseph Myers wrote:
> On Wed, 9 Nov 2022, Martin Liška wrote:
> 
>> 1) not synchronized content among lib*/Makefile.in and lib*/Makefile.am.
>> Apparently, I modified the generated Makefile.in file with the rules like:
>>
>> doc/info/texinfo/libitm.info: $(SPHINX_FILES)
>> 	+ if [ x$(HAS_SPHINX_BUILD) = xhas-sphinx-build ]; then \
>> 	  make -C $(srcdir)/../doc info SOURCEDIR=$(abs_srcdir)/doc BUILDDIR=$(abs_doc_builddir)/info SPHINXBUILD=$(SPHINX_BUILD); \
>> 	else true; fi
>>
>> Can you please modify Makefile.am in the corresponding manner and re-generate Makefile.in?
> 
> I think someone else had best look at this.

All right, I've got a patch candidate for it, so I'll be hopefully able to manage.

> 
>> 2) Adding proper support --enable-generated-files-in-srcdir in gcc_release:
> 
> It looks like all the GENINSRC rules / conditionals are still present.
> So maybe there are details where the paths are wrong, or where fixes are
> needed to ensure the files get installed from the source directory when
> available in the source directory but not available in the build directory
> because Sphinx isn't available, but much of the code for the feature is
> still there.

I can investigate then. Is the option --enable-generated-files-in-srcdir suppose
to be used when building from a release tarball (that includes man/info in src),
or to create such a source tarball?

Cheers,
Martin

Re: Announcement: Porting the Docs to Sphinx - 9. November 2022

[Richard Biener]

On Thu, Nov 10, 2022 at 2:05 PM Martin Liška <mliska@suse.cz> wrote:
>
> On 11/9/22 18:14, Joseph Myers wrote:
> > On Wed, 9 Nov 2022, Martin Liška wrote:
> >
> >> 1) not synchronized content among lib*/Makefile.in and lib*/Makefile.am.
> >> Apparently, I modified the generated Makefile.in file with the rules like:
> >>
> >> doc/info/texinfo/libitm.info: $(SPHINX_FILES)
> >>      + if [ x$(HAS_SPHINX_BUILD) = xhas-sphinx-build ]; then \
> >>        make -C $(srcdir)/../doc info SOURCEDIR=$(abs_srcdir)/doc BUILDDIR=$(abs_doc_builddir)/info SPHINXBUILD=$(SPHINX_BUILD); \
> >>      else true; fi
> >>
> >> Can you please modify Makefile.am in the corresponding manner and re-generate Makefile.in?
> >
> > I think someone else had best look at this.
>
> All right, I've got a patch candidate for it, so I'll be hopefully able to manage.
>
> >
> >> 2) Adding proper support --enable-generated-files-in-srcdir in gcc_release:
> >
> > It looks like all the GENINSRC rules / conditionals are still present.
> > So maybe there are details where the paths are wrong, or where fixes are
> > needed to ensure the files get installed from the source directory when
> > available in the source directory but not available in the build directory
> > because Sphinx isn't available, but much of the code for the feature is
> > still there.
>
> I can investigate then. Is the option --enable-generated-files-in-srcdir suppose
> to be used when building from a release tarball (that includes man/info in src),
> or to create such a source tarball?

It's to create such a tarball which then can be built with a reduced toolset.

> Cheers,
> Martin
>

Re: Announcement: Porting the Docs to Sphinx - 9. November 2022

[David Malcolm]

On Mon, 2022-10-17 at 15:28 +0200, Martin Liška wrote:
> Hello.
> 
> Based on the very positive feedback I was given at the Cauldron
> Sphinx Documentation BoF,
> I'm planning migrating the documentation on 9th November. There are
> still some minor comments
> from Sandra when it comes to the PDF output, but we can address that
> once the conversion is done.
> 
> The reason I'm sending the email now is that I was waiting for latest
> Sphinx release (5.3.0) that
> simplifies reference format for options and results in much simpler
> Option summary section ([1])
> 
> The current GCC master (using Sphinx 5.3.0) converted docs can be
> seen here:
> https://splichal.eu/scripts/sphinx/
> 
> If you see any issues with the converted documentation, or have a
> feedback about it,
> please reply to this email.
> 
> Cheers,
> Martin
> 
> [1] https://github.com/sphinx-doc/sphinx/pull/10840
> [1]
> https://splichal.eu/scripts/sphinx/gcc/_build/html/gcc-command-options/option-summary.html
> 

FWIW, to help organize the various bugs people have reported due to the
sphinx migration, I've gone ahead and created a tracker bug in GCC
bugzilla.  See:
  https://gcc.gnu.org/bugzilla/showdependencytree.cgi?id=sphinx-migration
aka:
  https://gcc.gnu.org/bugzilla/showdependencytree.cgi?id=107655

Not all of them have "sphinx" in the title, so hope I got them all. 
Please add any I missed, and add any new ones you file as blocking this
bug.

Hope this is helpful
Dave