RFC: Experimental use of Sphinx for GCC documentation

RFC: Experimental use of Sphinx for GCC documentation

[David Malcolm]

I've been experimenting with using Sphinx [1] for GCC's documentation.

You can see an HTML sample of GCC docs built with Sphinx here:
https://dmalcolm.fedorapeople.org/gcc/2015-08-31/rst-experiment/gcc.html
(it's a work-in-progress; i.e. there are bugs).

Compare with:
 https://gcc.gnu.org/onlinedocs/gcc/index.html


In particular, note how options get stable, clickable URLs:
https://dmalcolm.fedorapeople.org/gcc/2015-08-31/rst-experiment/option-summary.html

as compared to:
https://gcc.gnu.org/onlinedocs/gcc/Option-Summary.html#Option-Summary


Example of an option URL, for "-ftree-loop-if-convert-stores" (also
showing syntax-highlighted example code):
https://dmalcolm.fedorapeople.org/gcc/2015-08-31/rst-experiment/options-that-control-optimization.html#cmdoption-ftree-loop-if-convert-stores

as compared to:
https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html#index-ftree-loop-if-convert-1054 (where I had to use via "View Source" to find that URL, and what's up with that "-1054" wart? note also that the number can change, making the URL unstable)


Example of a stable URL for "What does -O2 do?":
https://dmalcolm.fedorapeople.org/gcc/2015-08-31/rst-experiment/options-that-control-optimization.html#cmdoption-O2

...etc


Every HTML page also gets a "Show Source" link, showing the input
markup.


Sphinx is a modern, well-maintained documentation toolchain, implemented
in Python.  The input format is an easy minimal-markup "semantic"
format; the output is high-quality HTML, with PDF and texinfo supported
amongst other output formats.   It was created for documenting the
Python programming language, and is in use by many FLOSS projects for
their docs, including e.g. LLVM (http://llvm.org/docs/).  See also:
http://sphinx-doc.org/examples.html (though this list is far from
complete).   It's BSD-licensed.

We are currently using Sphinx for libgccjit:
  https://gcc.gnu.org/onlinedocs/jit/index.html
and, I believe, for Ada:
https://docs.adacore.com/gnat_ugn-docs/html/gnat_ugn/gnat_ugn.html [2]

I've also used it for generating slides for Cauldron presentations:
https://dmalcolm.fedorapeople.org/presentations/cauldron-2014/rtl/
https://dmalcolm.fedorapeople.org/presentations/cauldron-2014/jit/

and for gcc-python-plugin:
 https://gcc-python-plugin.readthedocs.org/en/latest/



I've written a tool called texi2rst which attempts to convert a .texi
based document to .rst ("restructured text", the input format for
Sphinx):
 https://github.com/davidmalcolm/texi2rst

This is what generated the examples above.

It doesn't *quite* do a direct .texi to .rst conversion yet: it can take
the XML output from texinfo's "makeinfo --xml", and generate either one
big .rst file, or a group of smaller .rst files.

My hope was that for every gcc/docs/foo.texi file we have, my tool would
be able to generate a gcc/docs/foo.rst (maybe retaining the name, to
allow for sane diff and hence sane patch review).

Unfortunately, "makeinfo --xml" resolves includes and conditional
processing, so the underlying input structure of .texinfo files is lost
at that point.

To fix that, I've been working on a frontend from texi2rst that
re-implements the .texi to xml processing, retaining information on
includes, and directives, so that I can translate them to
corresponding .rst directives.  Unfortunately it's clear that I'm not
going to finish that before stage 1 closes - but I think it's feasible
in the stage3 timeframe.

Hence in the example posted above, the doc is split into pages based on
nodes, named after the nodes, and thus get rather long names e.g.
options-that-control-optimization.html, generated from
options-that-control-optimization.rst.  In a more polished version,
these names would be saner.

The primary advantages of .rst/sphinx over .texi/texinfo I see are in
the generated HTML:

* sane, stable URLs (so e.g. there is a reliable URL for the docs for,
say, "-Wall").

* a page-splitting structure that make sense, to me, at least [3]

* much more use of markup, with restrained and well-chosen CSS
(texinfo's HTML seems to ignore much of the inline markup in
the .texinfo file)

* autogenerated internal links, so that almost everything is clickable,
and will take you somewhere sane, by default

* syntax-highlighting of code examples, with support for multiple
programming languages (note the mixture of C, C++, Fortran, etc in the
docs for the gcc options).

* looks modern and fresh (IMHO), letting casual observers see that the
project is alive and kicking.


Thoughts?
Dave

[1] http://sphinx-doc.org/
[2] I couldn't find Sphinx-built HTML for Ada on the gcc website, just
the Texinfo output here:
https://gcc.gnu.org/onlinedocs/gnat_ugn/index.html
[3] I have never fathomed the way texinfo's navigation works, for HTML,
at least, and I believe I'm not the only one; I generally pick the
all-in-one-HTML-page option when viewing texinfo-html docs and do
textual searches, since otherwise I usually can't find the thing I'm
looking for (or have to resort to a brute-force depth-first search of
clicking through the links).

Re: RFC: Experimental use of Sphinx for GCC documentation

[Arnaud Charlet]

We've switched the Ada doc to sphinx indeed, so can only be
in favor of this change for the rest of GCC.

We do have also a texi2rst script which handles 90% of the work, the
rest requiring manual adaptations. I can send the script we've used if
this can help.

Arno

Re: RFC: Experimental use of Sphinx for GCC documentation

[Kyrill Tkachov]

Hi David,

On 08/11/15 13:55, David Malcolm wrote:
> I've been experimenting with using Sphinx [1] for GCC's documentation.
>
> You can see an HTML sample of GCC docs built with Sphinx here:
> https://dmalcolm.fedorapeople.org/gcc/2015-08-31/rst-experiment/gcc.html
> (it's a work-in-progress; i.e. there are bugs).
>
> Compare with:
>   https://gcc.gnu.org/onlinedocs/gcc/index.html
>
>
> In particular, note how options get stable, clickable URLs:
> https://dmalcolm.fedorapeople.org/gcc/2015-08-31/rst-experiment/option-summary.html

FWIW, I think this all looks much better than the existing formatting.
One weird artifact I noticed while looking at the ARM options:
https://dmalcolm.fedorapeople.org/gcc/2015-08-31/rst-experiment/hardware-models-and-configurations.html#arm-options
In particular for -mcpu where it gives an explanation of what -mcpu=generic-<arch> is equivalent to, there seems
to be something weird going on.
The .texi source is:
@option{-mcpu=generic-@var{arch}} is also permissible, and is equivalent to @option{-march=@var{arch} -mtune=generic-@var{arch}}.

Whereas the output looks something like:
-mcpu=generic-``arch`` is also permissible, and is equivalent to -march=``arch` -mtune=generic-arch`

The backticks look somewhat inconsistent. But that may be due to invalid use of the @var and @option
constructs in the source. I'm not very familiar with the details.

Thanks,
Kyrill

> as compared to:
> https://gcc.gnu.org/onlinedocs/gcc/Option-Summary.html#Option-Summary
>
>
> Example of an option URL, for "-ftree-loop-if-convert-stores" (also
> showing syntax-highlighted example code):
> https://dmalcolm.fedorapeople.org/gcc/2015-08-31/rst-experiment/options-that-control-optimization.html#cmdoption-ftree-loop-if-convert-stores
>
> as compared to:
> https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html#index-ftree-loop-if-convert-1054 (where I had to use via "View Source" to find that URL, and what's up with that "-1054" wart? note also that the number can change, making the URL unstable)
>
>
> Example of a stable URL for "What does -O2 do?":
> https://dmalcolm.fedorapeople.org/gcc/2015-08-31/rst-experiment/options-that-control-optimization.html#cmdoption-O2
>
> ...etc
>
>
> Every HTML page also gets a "Show Source" link, showing the input
> markup.
>
>
> Sphinx is a modern, well-maintained documentation toolchain, implemented
> in Python.  The input format is an easy minimal-markup "semantic"
> format; the output is high-quality HTML, with PDF and texinfo supported
> amongst other output formats.   It was created for documenting the
> Python programming language, and is in use by many FLOSS projects for
> their docs, including e.g. LLVM (http://llvm.org/docs/).  See also:
> http://sphinx-doc.org/examples.html (though this list is far from
> complete).   It's BSD-licensed.
>
> We are currently using Sphinx for libgccjit:
>    https://gcc.gnu.org/onlinedocs/jit/index.html
> and, I believe, for Ada:
> https://docs.adacore.com/gnat_ugn-docs/html/gnat_ugn/gnat_ugn.html [2]
>
> I've also used it for generating slides for Cauldron presentations:
> https://dmalcolm.fedorapeople.org/presentations/cauldron-2014/rtl/
> https://dmalcolm.fedorapeople.org/presentations/cauldron-2014/jit/
>
> and for gcc-python-plugin:
>   https://gcc-python-plugin.readthedocs.org/en/latest/
>
>
>
> I've written a tool called texi2rst which attempts to convert a .texi
> based document to .rst ("restructured text", the input format for
> Sphinx):
>   https://github.com/davidmalcolm/texi2rst
>
> This is what generated the examples above.
>
> It doesn't *quite* do a direct .texi to .rst conversion yet: it can take
> the XML output from texinfo's "makeinfo --xml", and generate either one
> big .rst file, or a group of smaller .rst files.
>
> My hope was that for every gcc/docs/foo.texi file we have, my tool would
> be able to generate a gcc/docs/foo.rst (maybe retaining the name, to
> allow for sane diff and hence sane patch review).
>
> Unfortunately, "makeinfo --xml" resolves includes and conditional
> processing, so the underlying input structure of .texinfo files is lost
> at that point.
>
> To fix that, I've been working on a frontend from texi2rst that
> re-implements the .texi to xml processing, retaining information on
> includes, and directives, so that I can translate them to
> corresponding .rst directives.  Unfortunately it's clear that I'm not
> going to finish that before stage 1 closes - but I think it's feasible
> in the stage3 timeframe.
>
> Hence in the example posted above, the doc is split into pages based on
> nodes, named after the nodes, and thus get rather long names e.g.
> options-that-control-optimization.html, generated from
> options-that-control-optimization.rst.  In a more polished version,
> these names would be saner.
>
> The primary advantages of .rst/sphinx over .texi/texinfo I see are in
> the generated HTML:
>
> * sane, stable URLs (so e.g. there is a reliable URL for the docs for,
> say, "-Wall").
>
> * a page-splitting structure that make sense, to me, at least [3]
>
> * much more use of markup, with restrained and well-chosen CSS
> (texinfo's HTML seems to ignore much of the inline markup in
> the .texinfo file)
>
> * autogenerated internal links, so that almost everything is clickable,
> and will take you somewhere sane, by default
>
> * syntax-highlighting of code examples, with support for multiple
> programming languages (note the mixture of C, C++, Fortran, etc in the
> docs for the gcc options).
>
> * looks modern and fresh (IMHO), letting casual observers see that the
> project is alive and kicking.
>
>
> Thoughts?
> Dave
>
> [1] http://sphinx-doc.org/
> [2] I couldn't find Sphinx-built HTML for Ada on the gcc website, just
> the Texinfo output here:
> https://gcc.gnu.org/onlinedocs/gnat_ugn/index.html
> [3] I have never fathomed the way texinfo's navigation works, for HTML,
> at least, and I believe I'm not the only one; I generally pick the
> all-in-one-HTML-page option when viewing texinfo-html docs and do
> textual searches, since otherwise I usually can't find the thing I'm
> looking for (or have to resort to a brute-force depth-first search of
> clicking through the links).
>

Re: RFC: Experimental use of Sphinx for GCC documentation

[David Malcolm]

On Mon, 2015-11-09 at 16:54 +0000, Kyrill Tkachov wrote:
> Hi David,
> 
> On 08/11/15 13:55, David Malcolm wrote:
> > I've been experimenting with using Sphinx [1] for GCC's documentation.
> >
> > You can see an HTML sample of GCC docs built with Sphinx here:
> > https://dmalcolm.fedorapeople.org/gcc/2015-08-31/rst-experiment/gcc.html
> > (it's a work-in-progress; i.e. there are bugs).
> >
> > Compare with:
> >   https://gcc.gnu.org/onlinedocs/gcc/index.html
> >
> >
> > In particular, note how options get stable, clickable URLs:
> > https://dmalcolm.fedorapeople.org/gcc/2015-08-31/rst-experiment/option-summary.html
> 
> FWIW, I think this all looks much better than the existing formatting.
> One weird artifact I noticed while looking at the ARM options:
> https://dmalcolm.fedorapeople.org/gcc/2015-08-31/rst-experiment/hardware-models-and-configurations.html#arm-options
> In particular for -mcpu where it gives an explanation of what -mcpu=generic-<arch> is equivalent to, there seems
> to be something weird going on.
> The .texi source is:
> @option{-mcpu=generic-@var{arch}} is also permissible, and is equivalent to @option{-march=@var{arch} -mtune=generic-@var{arch}}.
> 
> Whereas the output looks something like:
> -mcpu=generic-``arch`` is also permissible, and is equivalent to -march=``arch` -mtune=generic-arch`
> 
> The backticks look somewhat inconsistent. But that may be due to invalid use of the @var and @option
> constructs in the source. I'm not very familiar with the details.

Thanks; I've filed this for myself as:
  https://github.com/davidmalcolm/texi2rst/issues/10

[...snip...]

Re: RFC: Experimental use of Sphinx for GCC documentation

[David Malcolm]

On Sun, 2015-11-08 at 16:16 +0100, Arnaud Charlet wrote:
> We've switched the Ada doc to sphinx indeed, so can only be
> in favor of this change for the rest of GCC.
> 
> We do have also a texi2rst script which handles 90% of the work, the
> rest requiring manual adaptations. I can send the script we've used if
> this can help.

I'm interested in seeing your script.  Can you post/upload it somewhere?

Thanks
Dave

Re: RFC: Experimental use of Sphinx for GCC documentation

[Arnaud Charlet]

> > We do have also a texi2rst script which handles 90% of the work, the
> > rest requiring manual adaptations. I can send the script we've used if
> > this can help.
> 
> I'm interested in seeing your script.  Can you post/upload it somewhere?

Yes I will. Let me get the latest version we've used and get back to you.

Arno

Re: RFC: Experimental use of Sphinx for GCC documentation

[Arnaud Charlet]

[-- Attachment #1: Type: text/plain, Size: 504 bytes --]

> > > We do have also a texi2rst script which handles 90% of the work, the
> > > rest requiring manual adaptations. I can send the script we've used if
> > > this can help.
> > 
> > I'm interested in seeing your script.  Can you post/upload it somewhere?
> 
> Yes I will. Let me get the latest version we've used and get back to you.

Here it is. We've used it to convert many docs at AdaCore (including
gnat_ugn and gnat_rm.texi). It does require manual postprocessing but gives
a good headstart.

Arno

[-- Attachment #2: texi2rst.py --]
[-- Type: text/x-python, Size: 13955 bytes --]

#!/usr/bin/python
# -*- coding: utf-8 -*-

"""Splits an existing .texinfo file into components suitable for
   makeinfo.py
   If "-node <name>" is specified, only that node and its children are
   kept
"""

import re
import sys
import optparse
import os.path


def finish_section(out, section, section_node, marker, with_label):
    if section_node == '' or section_node == 'Top':
        return

    # Create a label
    if with_label:
        out.write('.. _%s:\n\n' % section_node.replace(' ', '_'))

    # Create header

    if len(marker) == 2:
        out.write(marker[0] * len(section_node) + '\n')

    out.write(section_node + '\n')

    out.write(marker[0] * len(section_node) + '\n\n')

    list_level = 0
    prev_was_blank = False
    in_example = False
    in_table = 0
    in_menu = False
    example_end = ''
    table_marker = '*'

    def word(line, index=1):
        s = line.lstrip().split()
        if len(s) >= index:
            return s[index - 1]
        else:
            return ""

    for line in section.strip().splitlines():
        if word(line, 1) in ('@itemize', '@enumerate'):
            list_level = list_level + 1
            if not prev_was_blank:
                out.write('\n')
            prev_was_blank = False

        elif line.lstrip().startswith('@end itemize') \
                or line.lstrip().startswith('@end enumerate'):
            list_level = list_level - 1
            prev_was_blank = False

        elif word(line, 1) == '@table':
            out.write("\n")
            table_marker = '*'
            in_table += 1
            prev_was_blank = True

        elif in_table > 0 and line.lstrip().startswith('@end table'):
            in_table -= 1
            prev_was_blank = False

        elif line.lstrip().startswith('@menu'):
            out.write('.. toctree::\n')
            out.write('   :numbered:\n')
            out.write('   :maxdepth: 3\n')
            out.write('\n')
            in_menu = True

        elif in_menu:
            if line.startswith('@end menu'):
                in_menu = False
            else:
                entry = re.sub('::.*', '', line)
                entry = re.sub('^\* ', '', entry.strip())
                entry = entry.replace(' ', '_').replace('/', '_')
                out.write('   ' + entry + '\n')

        elif word(line, 1) in (
            "@deffn", "@defmethod", "@deftp", "@deftypemethod",
            "@deffnx", "@defmethodx", "@deftypefn", "@defun"):

            out.write(".. index:: %s\n\n" % line.lstrip().split(' ', 1)[1])
            out.write(line.split(' ', 1)[1].strip() + '\n')

            in_table += 1
            table_marker = '`'

        elif in_table > 0 \
            and word(line, 1) in ("@end") \
            and word(line, 2) in (
               "deffn", "defmethod", "deftp", "deftypemethod",
               "deffnx", "defmethodx", "deftypefn", "defun"):
            in_table -= 1

        elif word(line, 1) in ('@item', '@itemx'):
            line = line.lstrip().replace('@itemx', '')
            line = line.replace('@item', '')

            if in_table > 0:
                if line.strip().startswith(table_marker):
                    # Avoid lines like  "**Bold* text*" which of course
                    # sphinx doesn't like
                    table_marker = ""

                out.write('\n%s%s%s\n' % (
                    table_marker, line.strip(), table_marker))
                prev_was_blank = True
            else:
                out.write('  ' * (list_level - 1) + '* ' + line.strip() + '\n')
                prev_was_blank = False

        elif line.strip() == '':
            if not prev_was_blank:
                out.write('\n')
            prev_was_blank = True

        else:
            if '@example' in line:
                in_example = True
                example_end = '@end example'
                out.write('  ' * list_level + '::\n\n')
                continue
            elif '@smallexample' in line:
                in_example = True
                example_end = '@end smallexample'
                out.write('\n' + '  ' * list_level + '::\n\n')
                continue
            elif '@CODESAMPLE{' in line:
                line = line.replace("@CODESAMPLE{", "")
                example_end = '}'
                in_example = True
                out.write('\n')
                out.write('  ' * list_level + ".. highlight:: ada\n\n")
                out.write('  ' * list_level + '::\n\n')

            elif '@NOTE{' in line:
                line = line.replace("@NOTE{", "")
                in_example = True
                example_end = '}'
                out.write('  ' * list_level + '|Note|::\n\n')

            elif '@TIP{' in line:
                line = line.replace("@TIP{", "")
                in_example = True
                example_end = '}'
                out.write('  ' * list_level + '|Tip|::\n\n')

            elif '@IMPORTANT{' in line:
                line = line.replace("@IMPORTANT{", "")
                in_example = True
                example_end = '}'
                out.write('  ' * list_level + '|Important|::\n\n')

            level = '  ' * (list_level + in_table)

            if in_example and example_end in line:
                in_example = False
                line = line.replace(example_end, '')
                example_end = ''
                out.write(level + '  ' + line.lstrip() + '\n')

            else:
                if in_example:
                    out.write(level + '  ' + line + '\n')
                else:
                    out.write(level + line.lstrip() + '\n')

            prev_was_blank = False

    out.write('\n')


def get_section_marker(line):
    """Return the marker to use in Sphinx for the given section.
       The marker has two characters if overlines are needed.
    """

    if line.startswith('@chapter'):
        return '**'
    elif line.startswith('@section'):
        return '='
    elif line.startswith('@subsection'):
        return '-'
    elif line.startswith('@subsubsection'):
        return '^'
    elif line.startswith('@subsubsubsection'):
        return '"'
    elif line.startswith('@unnumbered'):
        return '~'
    else:
        raise Exception("Unknown section: %s" % line)


def image_name(basename):
    """Return the fullname for the given image"""

    for ext in ("png", "jpg"):
        if os.path.exists('%s.%s' % (basename, ext)):
            return '%s.%s' % (basename, ext)
    raise Exception("Image not found: %s" % basename)

def split(filename, options):
    skip_until = ''
    section = ''
    section_node = ''
    section_need_node = False

    levels = dict(
        chapter=1,
        section=2,
        subsection=3,
        subsubsection=4,
        sec=2,
        subsect=3,
        subsubsec=4,
        unnumbered=1,
        unnumberedsubsec=3) ### NEW LINE

    # Stores the @set definitions. They need to be applied in
    # order, since a later definition can reference an earlier
    # one. Hence we use a list

    macros = []

    # These two variables control whether the current text is output or not
    # (it might be filtered by a --node argument). preserve_level indicates
    # when the --node switch matched, and all sections below will be
    # displayed.

    preserve = options.node is None
    preserve_level = 1000  # Level under which we wan

    output = file("index.rst", "w") # Current output file (one per chapter)
    section_marker = '='
    section_node = ''
    prev_line_is_node = False

    content = file(filename).read()
    content = content.replace('\\', '\\\\')
    content = content.replace('@bye', '')
    content = content.replace('@printindex cp', '')
    content = content.replace('@node Index', '')
    content = content.replace('@unnumbered Index', '')
    content = content.replace('@NL{}', '')
    content = content.replace('@include gfdl.texi', '')
    content = content.replace('@copyright{}', 'C')
    content = content.replace('@noindent', '')
    content = content.replace('@dots{}', '...')
    content = content.replace('@result{}', '=>')
    content = content.replace('@error{}', '=> Error: ')
    content = content.replace('@@', '@')
    content = content.replace('@ifhtml', '')
    content = content.replace('@end ifhtml', '')

    content = re.sub("``(.*?)''", "'\\1'", content)

    content = re.sub('@iftex.*?@end iftex', '', content, flags=re.DOTALL)

    # Put all hyperlinks on a single line, so that we can do easier
    # substitution later on.

    for regexp in ("@(px)?ref\{.+?\}",
                   "@code\{.+?\}",
                   "@xref\{.+?\}",
                   "@[bi]\{.+?\}"):
        for link in re.finditer(regexp, content, re.DOTALL):
            start = link.start(0)
            end   = link.end(0)
            content = content[:start] \
                    + content[start:end].replace("\n", " ") \
                    + content[end:]

    lines = content.splitlines()
    line_number = 0

    # Use an explicit loop, since in some cases we need to parse several
    # lines, for instance for @smallexample

    while line_number < len(lines):
     try: ### ADDED THE try: ... except: BRACKETING
        line = lines[line_number]
        line_number += 1

        if line == "@c":
            # An empty comment line
            continue

        if skip_until != '':
            if line.find(skip_until) != -1:
                skip_until = ''
            continue

        # The regexps end with [^@] so that "@}" is properly taken as part
        # of the argument, and doesn't end on the "}".

        line = re.sub('@c\s.*', '',       line)   # comments

        line = re.sub('@i\{(.*?)\}',     '*\\1*', line)
        line = re.sub('@emph\{(.*?)\}',  '*\\1*', line)
        line = re.sub('@command\{(.*?)\}',  '*\\1*', line)
        line = re.sub('@option\{(.*?)\}',  '*\\1*', line)
        line = re.sub('@b\{(.*?)\}',     '**\\1**', line)
        line = re.sub('@code\{(.*?[^@])\}',
                      lambda(m): '`%s`' % m.group(1).strip(),
                      line)
        line = re.sub('@var\{(.*?[^@])\}',   '`\\1`', line)
        line = re.sub('@file\{(.*?[^@])\}',  ':file:`\\1`', line)
        line = re.sub('@key\{(.*?[^@])\}',   ':kbd:`\\1`', line)
        line = re.sub('@url\{(.*?[^@])\}',   '`\\1 <\\1>`_', line)
        line = re.sub('@uref\{(.*?[^@])\}',   '`\\1 <\\1>`_', line)
        line = re.sub('@email\{(.*?[^@])\}', '`\\1 <mail:\\1>`_', line)
        line = re.sub('@p?x?ref\{(.*?[^@])\}',
                      lambda(m): ':ref:`%s`' % m.group(1).replace(' ', '_'),
                      line)
        line = re.sub('@cindex (.*)',    '.. index:: \\1\n', line)
        line = re.sub('@anchor\{(.*?[^@])\}',
                      lambda(m): '.. _%s:\n\n' % m.group(1).replace(' ', '_'),
                      line)
        line = re.sub('@image\{([^,}]+)(,[^}]*)?\}',
                      lambda(m): '.. image:: %s\n' % image_name(m.group(1)),
                      line)
        line = line.replace('@{', '{')
        line = line.replace('@}', '}')

        for key, value in macros:
            line = line.replace(key, value)

        if line.startswith('@set '):
            _, name, value = line.rstrip().split(' ', 2)
            macros.append(('@value{%s}' % name, value))

        elif line.startswith('@node'):
            node = line.replace('@node ', '').strip()
            if node.startswith('Top'):
                section = ''
                section_node = node
            prev_line_is_node = True

        elif (line.startswith('@chapter')
             or line.startswith('@section')
             or line.startswith('@unnumbered ')  ### ADDED THR TRAILING ' '
             or line.startswith('@subsection')
             or line.startswith('@unnumberedsubsection') ### NEW LINE
             or line.startswith('@subsubsection')):

            # Finish current section
            finish_section(
                output, section, section_node, marker=section_marker,
                with_label=section_need_node)
            section = ''
            section_node = line.strip().split(' ', 1)[1].strip()
            section_need_node = prev_line_is_node

            # Start new section

            if line.startswith('@chapter'):
                output_name = section_node.replace(" ", "_").replace("/", "_")
                output = file('%s.rst' % output_name, "w")

            section_marker = get_section_marker(line)

            level = line[1:].split()[0]
            if level != 'unnumbered':
                level = level.replace('unnumbered', '')
            title = ' '.join(line.split()[1:])  # Rest of the line

            level = levels[level]
            if options.node == title:
                preserve_level = level
                preserve = True
            else:
                preserve = options.node is None or level > preserve_level
                if not preserve:
                    preserve_level = 1000

        elif line.startswith('@menu'):
            if section_node.startswith("Top"):
                section += line + "\n"
            else:
                skip_until = '@end menu'

        elif line.startswith('@detailmenu'):
            skip_until = '@end menu'

        else:
            section += line + "\n"
            prev_line_is_node = False
     except: ### ADDED THE EXCEPTION HANDLER
        print "Exception raised in line", str(line_number)+': ', '"'+line+'"'
        return
    if preserve:
        finish_section(
            output, section, section_node, marker=section_marker,
            with_label=prev_line_is_node)


if __name__ == '__main__':
    parse = optparse.OptionParser(
        description='Split a .texi document into components')
    parse.add_option(
        '--node', default=None,
        help='Only preserve that node and its children')

    (options, args) = parse.parse_args()

    for a in args:
        split(a, options)

Re: RFC: Experimental use of Sphinx for GCC documentation

[Sandra Loosemore]

On 11/08/2015 06:55 AM, David Malcolm wrote:
> I've been experimenting with using Sphinx [1] for GCC's documentation.
>
> [snip]
>
> The primary advantages of .rst/sphinx over .texi/texinfo I see are in
> the generated HTML:
>
> * sane, stable URLs (so e.g. there is a reliable URL for the docs for,
> say, "-Wall").
>
> * a page-splitting structure that make sense, to me, at least [3]
>
> * much more use of markup, with restrained and well-chosen CSS
> (texinfo's HTML seems to ignore much of the inline markup in
> the .texinfo file)
>
> * autogenerated internal links, so that almost everything is clickable,
> and will take you somewhere sane, by default
>
> * syntax-highlighting of code examples, with support for multiple
> programming languages (note the mixture of C, C++, Fortran, etc in the
> docs for the gcc options).
>
> * looks modern and fresh (IMHO), letting casual observers see that the
> project is alive and kicking.
>
>
> Thoughts?

If we're going to switch documentation formats, I'd rather we used 
DocBook.  I've had to use "restructured text" before and found it really 
awkward.

But, personal preferences aside, I also think it's more important that 
we commit documentation-person resources to making the content more 
correct, readable, and better organized, than to making the HTML output 
look "modern and fresh", or worse yet, translating the docs to another 
format and having to proofread them for conversion goofs.

BTW, Mentor Graphics' toolchains ship with a custom HTML stylesheet for 
the generated manuals, to make them a little "prettier".  Maybe 
something like that would go a long way towards solving the perceived 
problems here?  Or improvements to texinfo's HTML generation.

-Sandra

Re: RFC: Experimental use of Sphinx for GCC documentation

[Joseph Myers]

On Sun, 8 Nov 2015, David Malcolm wrote:

> I've been experimenting with using Sphinx [1] for GCC's documentation.
> 
> You can see an HTML sample of GCC docs built with Sphinx here:
> https://dmalcolm.fedorapeople.org/gcc/2015-08-31/rst-experiment/gcc.html
> (it's a work-in-progress; i.e. there are bugs).

Observations:

* Could you provide the PDF version there as well?

* The option summary 
<https://dmalcolm.fedorapeople.org/gcc/2015-08-31/rst-experiment/option-summary.html> 
seems a complete mess.

* The indexes are missing.

> It doesn't *quite* do a direct .texi to .rst conversion yet: it can take
> the XML output from texinfo's "makeinfo --xml", and generate either one
> big .rst file, or a group of smaller .rst files.
> 
> My hope was that for every gcc/docs/foo.texi file we have, my tool would
> be able to generate a gcc/docs/foo.rst (maybe retaining the name, to
> allow for sane diff and hence sane patch review).
> 
> Unfortunately, "makeinfo --xml" resolves includes and conditional
> processing, so the underlying input structure of .texinfo files is lost
> at that point.
> 
> To fix that, I've been working on a frontend from texi2rst that
> re-implements the .texi to xml processing, retaining information on
> includes, and directives, so that I can translate them to
> corresponding .rst directives.  Unfortunately it's clear that I'm not
> going to finish that before stage 1 closes - but I think it's feasible
> in the stage3 timeframe.

You do of course need to convert documentation fragments in target.def 
in-place and adapt genhooks (preserving the arrangements of both tm.rst.in 
and tm.rst being checked in, or some other such arrangement that ensures 
there are always both GPL and GFDL copies of the hook documentation 
checked in, with genhooks dealing with keeping them in sync).  Other 
things to consider: preserving comments (where applicable); preserving 
@ignore contents (where the function is to comment out text); keeping 
manpage generation (which currently uses @c man comments together with 
@ignore) working; keeping --with-pkgversion and --with-bugurl working; 
keeping the principle that BASE-VER is the only checked-in file with the 
version number and everything else gets the version number at build time; 
dealing with the INTERNALS conditionals in files used in multiple manuals 
(and some others such as cppmanual conditionals).

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: RFC: Experimental use of Sphinx for GCC documentation

[David Malcolm]

On Mon, 2015-11-09 at 16:37 -0700, Sandra Loosemore wrote:
> On 11/08/2015 06:55 AM, David Malcolm wrote:
> > I've been experimenting with using Sphinx [1] for GCC's documentation.
> >
> > [snip]
> >
> > The primary advantages of .rst/sphinx over .texi/texinfo I see are in
> > the generated HTML:
> >
> > * sane, stable URLs (so e.g. there is a reliable URL for the docs for,
> > say, "-Wall").
> >
> > * a page-splitting structure that make sense, to me, at least [3]
> >
> > * much more use of markup, with restrained and well-chosen CSS
> > (texinfo's HTML seems to ignore much of the inline markup in
> > the .texinfo file)
> >
> > * autogenerated internal links, so that almost everything is clickable,
> > and will take you somewhere sane, by default
> >
> > * syntax-highlighting of code examples, with support for multiple
> > programming languages (note the mixture of C, C++, Fortran, etc in the
> > docs for the gcc options).
> >
> > * looks modern and fresh (IMHO), letting casual observers see that the
> > project is alive and kicking.
> >
> >
> > Thoughts?
> 
> If we're going to switch documentation formats, I'd rather we used 
> DocBook.  I've had to use "restructured text" before and found it really 
> awkward.

My own preference is the opposite; I've used DocBook and rst, and I find
DocBook to be the awkward one [1].  I think DocBook may be OK as an
interchange format, but I find it overly verbose to author and to read
in plain-text form.

(I'm not so fond of some parts of .rst's inline markup syntax, but I
find its structural aspects to be extremely expressive and concise;
overall I find it a joy to work with).

> But, personal preferences aside, I also think it's more important that 
> we commit documentation-person resources to making the content more 
> correct, readable, and better organized, than to making the HTML output 
> look "modern and fresh", or worse yet, translating the docs to another 
> format and having to proofread them for conversion goofs.

Correct, readable and well-organized documentation are laudable goals...
but I think that, to a first approximation, we're already there: I just
feel that the content is hidden behind a poor tool chain.

I believe that no matter how good we make the .texi files, the issues
with URLs, HTML page-splitting, etc with how texinfo's HTML generation
works will hold gcc back.

> BTW, Mentor Graphics' toolchains ship with a custom HTML stylesheet for 
> the generated manuals, to make them a little "prettier".  Maybe 
> something like that would go a long way towards solving the perceived 
> problems here? 

I'm interested in seeing that, though presumably the URL and
page-splitting issues would remain (is this at the CSS level, or do you
make deeper changes to the HTML generation?)


> Or improvements to texinfo's HTML generation.

texinfo is implemented in Perl, and FWIW, for me to help, that's a
showstopper (sprry; I've tried several times to get my head around Perl,
but my brain seems incompatible with it).


One other approach might be to retain .texi as the canonical format, but
have a optional custom HTML generator (perhaps using texi2rst to
generate .rst for sphinx, this time as an intermediate step during "make
html", rather than as a one-time conversion).  The main thing I think
it's missing is a way to express the language of embedded source
examples, so that they can be syntax-highlighted.


Thanks; I hope this is constructive.
Dave

[1] fwiw, my opinion on this has changed; a decade or so ago I worked on
a DocBook editor,  http://conglomerate.org/ 

Re: RFC: Experimental use of Sphinx for GCC documentation

[Joseph Myers]

On Mon, 9 Nov 2015, Sandra Loosemore wrote:

> If we're going to switch documentation formats, I'd rather we used DocBook.
> I've had to use "restructured text" before and found it really awkward.

I should perhaps note that the Sphinx extensions to reST are a lot better 
documented than the ZWiki ones!

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: RFC: Experimental use of Sphinx for GCC documentation

[Sandra Loosemore]

[resending, gcc-patches won't allow a screenshot attachment]

On 11/10/2015 03:27 PM, David Malcolm wrote:
>
> Correct, readable and well-organized documentation are laudable goals...
> but I think that, to a first approximation, we're already there: I just
> feel that the content is hidden behind a poor tool chain.

Hmmm, I just searched bugzilla for "documentation", and got 539 bugs 
back.  :-S  (Bugzilla doesn't seem to have a component keyword for 
"docs" or "manual" or anything like that, so the doc issues aren't 
really tagged as such in any way.)

> I believe that no matter how good we make the .texi files, the issues
> with URLs, HTML page-splitting, etc with how texinfo's HTML generation
> works will hold gcc back.

I don't see these problems as being so severe that it's worth losing 
edit history by reformatting the whole document, and the time/effort it 
would take to do and proofread the conversion.

>> BTW, Mentor Graphics' toolchains ship with a custom HTML stylesheet for
>> the generated manuals, to make them a little "prettier".  Maybe
>> something like that would go a long way towards solving the perceived
>> problems here?
>
> I'm interested in seeing that, though presumably the URL and
> page-splitting issues would remain (is this at the CSS level, or do you
> make deeper changes to the HTML generation?)

Yes, CSS.  It's nothing fancy; we specify some fonts and colors, and put 
a shaded box around code examples to set them off better.  I'm not sure 
we have any online examples of HTML manuals accessible outside Mentor. 
Probably we could contribute the .css file if anybody wants to hack it 
up further to improve GCC's default manual formatting, but I'd have to 
check that we don't consider that shade of blue proprietary or anything 
like that.  ;-)

-Sandra

RFC: Sphinx for GCC documentation

[Martin Liška]

Hey.

I've returned to the David's project and I'm willing to finish his transition effort.
I believe using Sphinx documentation can rapidly improve readability, both HTML and PDF version,
of various GCC manuals ([1]). I've spent some time working on the David's texi2rsf conversion tool ([2])
and I'm presenting a result that is not perfect, but can be acceptable after a reasonable
amount of manual modifications.

So far I focused on the 2 biggest manuals we have:

(1) User documentation
https://splichal.eu/sphinx/
https://splichal.eu/sphinx/gcc.pdf

(2) GCC Internals Manual
https://splichal.eu/sphinx-internal
https://splichal.eu/sphinx-internal/gccint.pdf

I'm aware of missing pieces (thanks Joseph) and I'm willing to finish it.

That said, I'm asking the GCC community for a green light before I invest
more time on it?

Cheers,
Martin

[1] https://gcc.gnu.org/onlinedocs/
[2] https://github.com/davidmalcolm/texi2rst

Re: RFC: Sphinx for GCC documentation

[Koning, Paul]

Can  it provide EPUB or MOBI output?  Some of the documentation systems used in various open source products have that capability, and that is very nice to have.  I have seen this from the one used by the Python project, for example.  Converting other formats to EPUB sometimes works tolerably well, but often not -- in particular, PDF to anything else is generally utterly unuseable, which is actually a design goal of PDF.

	paul

> On Apr 1, 2021, at 9:30 AM, Martin Liška <mliska@suse.cz> wrote:
> 
> Hey.
> 
> I've returned to the David's project and I'm willing to finish his transition effort.
> I believe using Sphinx documentation can rapidly improve readability, both HTML and PDF version,
> of various GCC manuals ([1]). I've spent some time working on the David's texi2rsf conversion tool ([2])
> and I'm presenting a result that is not perfect, but can be acceptable after a reasonable
> amount of manual modifications.
> 
> So far I focused on the 2 biggest manuals we have:
> 
> (1) User documentation
> https://splichal.eu/sphinx/
> https://splichal.eu/sphinx/gcc.pdf
> 
> (2) GCC Internals Manual
> https://splichal.eu/sphinx-internal
> https://splichal.eu/sphinx-internal/gccint.pdf
> 
> I'm aware of missing pieces (thanks Joseph) and I'm willing to finish it.
> 
> That said, I'm asking the GCC community for a green light before I invest
> more time on it?
> 
> Cheers,
> Martin
> 
> [1] https://gcc.gnu.org/onlinedocs/
> [2] https://github.com/davidmalcolm/texi2rst

Re: RFC: Sphinx for GCC documentation

[Martin Liška]

On 4/1/21 3:42 PM, Koning, Paul wrote:
> Can  it provide EPUB or MOBI output?

Yes, [1] lists 'epub' as one of the possible "buildername" options.
Btw. what Python project do you speak about?

Cheers,
Martin

[1] https://www.sphinx-doc.org/en/master/man/sphinx-build.html#cmdoption-sphinx-build-b

Re: RFC: Sphinx for GCC documentation

[Koning, Paul]

On Apr 1, 2021, at 9:51 AM, Martin Liška <mliska@suse.cz<mailto:mliska@suse.cz>> wrote:


[EXTERNAL EMAIL]

On 4/1/21 3:42 PM, Koning, Paul wrote:
Can  it provide EPUB or MOBI output?

Yes, [1] lists 'epub' as one of the possible "buildername" options.
Btw. what Python project do you speak about?

Cheers,
Martin

[1] https://urldefense.com/v3/__https://www.sphinx-doc.org/en/master/man/sphinx-build.html*cmdoption-sphinx-build-b__;Iw!!LpKI!0_fWcPNFwNBId3ubX-HgUu7gNFW_aV4Esdt-et67FMeHmhLWOiOVEbZZktJiLH8w$ [sphinx-doc[.]org]

Good to know.  I mean the project that delivers Python (python.org<http://python.org>).  It delivers documentation in several forms, EPUB among them: https://docs.python.org/3/download.html

paul

Re: RFC: Sphinx for GCC documentation

[Marek Polacek]

On Thu, Apr 01, 2021 at 03:30:10PM +0200, Martin Liška wrote:
> Hey.
> 
> I've returned to the David's project and I'm willing to finish his transition effort.
> I believe using Sphinx documentation can rapidly improve readability, both HTML and PDF version,
> of various GCC manuals ([1]). I've spent some time working on the David's texi2rsf conversion tool ([2])
> and I'm presenting a result that is not perfect, but can be acceptable after a reasonable
> amount of manual modifications.
> 
> So far I focused on the 2 biggest manuals we have:
> 
> (1) User documentation
> https://splichal.eu/sphinx/
> https://splichal.eu/sphinx/gcc.pdf
> 
> (2) GCC Internals Manual
> https://splichal.eu/sphinx-internal
> https://splichal.eu/sphinx-internal/gccint.pdf
> 
> I'm aware of missing pieces (thanks Joseph) and I'm willing to finish it.
> 
> That said, I'm asking the GCC community for a green light before I invest
> more time on it?

I really like it, so +1 from my side.  (The web page more than the pdf.)

Marek

Re: RFC: Sphinx for GCC documentation

[David Malcolm]

On Thu, 2021-04-01 at 15:30 +0200, Martin Liška wrote:
> Hey.
> 
> I've returned to the David's project and I'm willing to finish his
> transition effort.

For reference, the original thread about this is in the archives here:
  https://gcc.gnu.org/pipermail/gcc-patches/2015-November/434055.html

> I believe using Sphinx documentation can rapidly improve readability,
> both HTML and PDF version,
> of various GCC manuals ([1]).

Specifically, some of the benefits over the status quo are:
- internal hyperlinks in both HTML and PDF output for things like
options (see how every reference to e.g. -Wall and -O2 becomes a link
to those options, and they themselves have links to the things they
encompass)
- sane, stable URLs for anchors
- syntax highlighting of code snippets (and the format lets us express
what language the code snippets are in)
- a page splitting structure that makes sense (IMHO)
- appropriate use of well-designed CSS

The sphinx toolchain is used successfully by numerous projects (e.g.
Python, the Linux kernel, and LLVM).

>  I've spent some time working on the David's texi2rsf conversion tool
> ([2])
> and I'm presenting a result that is not perfect, but can be
> acceptable after a reasonable
> amount of manual modifications.

I think some further tweaking of the conversion script is needed first;
I'd rather iron out the script rather than play whack-a-mole manually.
I enjoy proofreading conversions, but I'm trying to focus right now on
analyzer bugfixing for GCC 11.

> So far I focused on the 2 biggest manuals we have:
> 
> (1) User documentation
> https://splichal.eu/sphinx/
> https://splichal.eu/sphinx/gcc.pdf
> 
> (2) GCC Internals Manual
> https://splichal.eu/sphinx-internal
> https://splichal.eu/sphinx-internal/gccint.pdf
> 
> I'm aware of missing pieces (thanks Joseph) and I'm willing to finish
> it.

One important requirement is manpage output.  Sphinx can do that - what
does the output look like based on the texi2rst output?

> That said, I'm asking the GCC community for a green light before I
> invest
> more time on it?

+1 from me, but you probably guessed that :)

Dave

> Cheers,
> Martin
> 
> [1] https://gcc.gnu.org/onlinedocs/
> [2] https://github.com/davidmalcolm/texi2rst
> 

Re: RFC: Sphinx for GCC documentation

[Koning, Paul]

> On Apr 1, 2021, at 9:30 AM, Martin Liška <mliska@suse.cz> wrote:
> 
> Hey.
> 
> I've returned to the David's project and I'm willing to finish his transition effort.
> I believe using Sphinx documentation can rapidly improve readability, both HTML and PDF version,
> of various GCC manuals ([1]). I've spent some time working on the David's texi2rsf conversion tool ([2])
> and I'm presenting a result that is not perfect, but can be acceptable after a reasonable
> amount of manual modifications.
> ...
> That said, I'm asking the GCC community for a green light before I invest
> more time on it?

Looks VERY good to me.  Given what I've seen about Sphinx (now that I've refreshed my memory) this would be a major improvement.

	paul

Re: RFC: Sphinx for GCC documentation

[Martin Sebor]

On 4/1/21 7:30 AM, Martin Liška wrote:
> Hey.
> 
> I've returned to the David's project and I'm willing to finish his transition effort.
> I believe using Sphinx documentation can rapidly improve readability, both HTML and PDF version,
> of various GCC manuals ([1]). I've spent some time working on the David's texi2rsf conversion tool ([2])
> and I'm presenting a result that is not perfect, but can be acceptable after a reasonable
> amount of manual modifications.
> 
> So far I focused on the 2 biggest manuals we have:
> 
> (1) User documentation
> https://splichal.eu/sphinx/
> https://splichal.eu/sphinx/gcc.pdf
> 
> (2) GCC Internals Manual
> https://splichal.eu/sphinx-internal
> https://splichal.eu/sphinx-internal/gccint.pdf
> 
> I'm aware of missing pieces (thanks Joseph) and I'm willing to finish it.
> 
> That said, I'm asking the GCC community for a green light before I invest
> more time on it?

I like the output quite a bit, especially that things like option
references are automagically linked to their documentation.  I spent
just a bit of time looking through the HTML and PDF above and found
a few minor issues.  I suspect they're due to the conversion script
not being finished yet but just to confirm please see below.

Is the source we'd work with the same as what shows up when I click
the 'View page source' link?  (Or is there some preprocessing involved?)

I'm not excited about changing tools.  I like that Texinfo is a GNU
project; AFACT, Sphinx is not.  In the original thread David linked
to there was a suggestion to use Texinfo and Sphinx side by side.
I.e., keep .texi as the master source format and generate the .rst
files using the conversion script on demand.  Would making that
the first step of the transition be a viable option?  (With the idea
of giving it a try and deciding whether to convert after some time?)

Martin

[*] The linking doesn't always seem to work (e.g.,
:option:`-fabi-version=4` under -Wabi).  I'm guessing it's not yet
done in the conversion script (or there's no -fabi-version=4 in
the index) and not a Sphinx limitation.

It doesn't add links for attributes but presumably that could also
be made to happen in the conversion, right?

It doesn't seem to preserve newlines in long lists of options (like
under -Wall), so the result is just one long running series of options
that's hard (for me) to follow. Presumably there's a way to do that
too (add a newline after each :option?)

It mentions both the positive and the negative forms of options as
headings (like -Wmain, -Wno-main).  That seems superflous and, with
very long option names (I noticed this for example with
-Wno-analyzer-use-of-pointer-in-stale-stack-frame) the heading in
the PDF can run off the right edge of the page.  But based on
the source it also looks like it should be easy to adjust in
the conversion script if we wanted to keep just one form, right?


> 
> Cheers,
> Martin
> 
> [1] https://gcc.gnu.org/onlinedocs/
> [2] https://github.com/davidmalcolm/texi2rst
> 

Re: RFC: Sphinx for GCC documentation

[Koning, Paul]

> On Apr 2, 2021, at 11:40 AM, Martin Sebor via Gcc-patches <gcc-patches@gcc.gnu.org> wrote:
> 
> ...
> I'm not excited about changing tools.  I like that Texinfo is a GNU
> project; AFACT, Sphinx is not. 

Why is that important?  It's an open source tool, and if it better in interesting ways I don't see why its affiliation should matter.

I view its support for ebook output as a major benefit, which as far as I know TexInfo does not offer.  

	paul

Re: RFC: Sphinx for GCC documentation

[Martin Liška]

On 4/1/21 4:31 PM, David Malcolm wrote:
> I think some further tweaking of the conversion script is needed first;

Sure, I've got a collection of patches and I'm planning to make pull requests.

> I'd rather iron out the script rather than play whack-a-mole manually.
> I enjoy proofreading conversions, but I'm trying to focus right now on
> analyzer bugfixing for GCC 11.
> 
>> So far I focused on the 2 biggest manuals we have:
>>
>> (1) User documentation
>> https://splichal.eu/sphinx/
>> https://splichal.eu/sphinx/gcc.pdf
>>
>> (2) GCC Internals Manual
>> https://splichal.eu/sphinx-internal
>> https://splichal.eu/sphinx-internal/gccint.pdf
>>
>> I'm aware of missing pieces (thanks Joseph) and I'm willing to finish
>> it.
> One important requirement is manpage output.  Sphinx can do that - what
> does the output look like based on the texi2rst output?
> 

I was able to create some man pages and they are listed here:
https://splichal.eu/gccsphinx/

The main gcc.1 man page contains complete content of GCC manual, that needs to be resolved.
Another minor issue is SYNOPSIS, which cannot be generated automatically and we should
replace the man directives (@c, ...).

Martin

Re: RFC: Sphinx for GCC documentation

[Martin Liška]

On 4/2/21 5:40 PM, Martin Sebor wrote:
> On 4/1/21 7:30 AM, Martin Liška wrote:
>> Hey.
>>
>> I've returned to the David's project and I'm willing to finish his transition effort.
>> I believe using Sphinx documentation can rapidly improve readability, both HTML and PDF version,
>> of various GCC manuals ([1]). I've spent some time working on the David's texi2rsf conversion tool ([2])
>> and I'm presenting a result that is not perfect, but can be acceptable after a reasonable
>> amount of manual modifications.
>>
>> So far I focused on the 2 biggest manuals we have:
>>
>> (1) User documentation
>> https://splichal.eu/sphinx/
>> https://splichal.eu/sphinx/gcc.pdf
>>
>> (2) GCC Internals Manual
>> https://splichal.eu/sphinx-internal
>> https://splichal.eu/sphinx-internal/gccint.pdf
>>
>> I'm aware of missing pieces (thanks Joseph) and I'm willing to finish it.
>>
>> That said, I'm asking the GCC community for a green light before I invest
>> more time on it?

Hello.

> 
> I like the output quite a bit, especially that things like option
> references are automagically linked to their documentation.  I spent
> just a bit of time looking through the HTML and PDF above and found
> a few minor issues.  I suspect they're due to the conversion script
> not being finished yet but just to confirm please see below.

Exactly, the script is supposed to do 99% of the transition automatically,
the rest needs a manual intervention.

> 
> Is the source we'd work with the same as what shows up when I click
> the 'View page source' link?  (Or is there some preprocessing involved?)

Yes, it's the source as is.

> 
> I'm not excited about changing tools.  I like that Texinfo is a GNU
> project; AFACT, Sphinx is not.

I don't any particular benefit. As already mentioned, Sphinx is a open-source
project with rich history and broad collection of projects using it:
https://www.sphinx-doc.org/en/master/examples.html

> In the original thread David linked
> to there was a suggestion to use Texinfo and Sphinx side by side.
> I.e., keep .texi as the master source format and generate the .rst
> files using the conversion script on demand.  Would making that
> the first step of the transition be a viable option?  (With the idea
> of giving it a try and deciding whether to convert after some time?)

I've just set up a pipeline that can generate all the manuals from texi file:
https://splichal.eu/gccsphinx/

However, the generated .rst files will need a manual changes before we can
use it as a replacement. So once we make a decision, .texi files will be left
after the transition. Note it's very difficult to write a 1:1 conversion tool
from TEXI to RST.

> 
> Martin
> 
> [*] The linking doesn't always seem to work (e.g.,
> :option:`-fabi-version=4` under -Wabi).  I'm guessing it's not yet
> done in the conversion script (or there's no -fabi-version=4 in
> the index) and not a Sphinx limitation.

Should be fixed now, one needs to split the '=4' part out of :option: directibe.

> 
> It doesn't add links for attributes but presumably that could also
> be made to happen in the conversion, right?

I partially added that.

> 
> It doesn't seem to preserve newlines in long lists of options (like
> under -Wall), so the result is just one long running series of options
> that's hard (for me) to follow. Presumably there's a way to do that
> too (add a newline after each :option?)

That's not desired and it's up to builder (PDF, HTML, ...) how the options
should be formatted.

> 
> It mentions both the positive and the negative forms of options as
> headings (like -Wmain, -Wno-main).  That seems superflous and, with
> very long option names (I noticed this for example with
> -Wno-analyzer-use-of-pointer-in-stale-stack-frame) the heading in
> the PDF can run off the right edge of the page.  But based on
> the source it also looks like it should be easy to adjust in
> the conversion script if we wanted to keep just one form, right?

Yes. I updated the script to split the lines when we reach a long list of options.
Note that we likely want to preserve both position and negative options because
of references and in order to express what is default.

I've created a version 2 that contains various manuals (and man pages):
https://splichal.eu/gccsphinx/

Martin

> 
> 
>>
>> Cheers,
>> Martin
>>
>> [1] https://gcc.gnu.org/onlinedocs/
>> [2] https://github.com/davidmalcolm/texi2rst
>>
> 

Re: RFC: Sphinx for GCC documentation

[Michael Matz]

Hello,

On Wed, 7 Apr 2021, Martin Liška wrote:

> > I like the output quite a bit, especially that things like option
> > references are automagically linked to their documentation.  I spent
> > just a bit of time looking through the HTML and PDF above and found
> > a few minor issues.  I suspect they're due to the conversion script
> > not being finished yet but just to confirm please see below.
> 
> Exactly, the script is supposed to do 99% of the transition automatically,
> the rest needs a manual intervention.
> 
> > 
> > Is the source we'd work with the same as what shows up when I click
> > the 'View page source' link?  (Or is there some preprocessing involved?)
> 
> Yes, it's the source as is.

I think doing that might be a mistake.  I do like the end result quite 
much (ignoring things missing, like @itemx), no doubt, but if the .rst 
files are the sources then we loose something: they contain markup to 
describe presentation ("make this bold/italic/a list").  The .texi files 
(want to) contain markup to describe meaning ("this is an 
chapter/section/option/attribute/code").  The former can always be 
generated from the latter, the other direction is not possible.  For 
maintaining and writing documentation it's usually better (because more 
future proof) to describe meaning.  (I'm aware that our .texi files don't 
completely adhere to that ideal and also contain quite some presentation 
markup)

Random snippet for what I mean: the .texi contains:

"The @code{access} attribute specifies that a function to whose 
by-reference arguments the attribute applies accesses the referenced 
object according to @var{access-mode}.  The @var{access-mode} argument is 
required and must be"

the .rst has:

"The ``access`` attribute specifies that a function to whose by-reference 
arguments the attribute applies accesses the referenced object according 
to :samp:`{access-mode}`.  The :samp:`{access-mode}` argument is required 
and must be"

So, @code{}/@var{} vs. `` `` / :samp:``.  Keeping in mind that 
documentation also needs to be written it's inconceivable why there should 
be such difference in expressing code vs var markup in .rst.  
Now, you could say, just use `` in the .rst file, it's rendered the same 
anyway; but then the distinction gets lost.

(Again, I'm aware that the .texi files aren't perfect here, and @code/@var 
also seems a bit forced :) )

(I'm not proposing we go the full extreme of semantic markup that docbook 
tries, but the other end of the spectrum, i.e. markdown/rst with only 
presentation markup, also doesn't seem very well fitting)

One other practical concern: it seems there's a one-to-one correspondence 
of .rst files and (web)page.  Do we really want to maintain hundreds (or 
how many?) .rst files, instead of 60 .texi files?


Ciao,
Michael.

Re: RFC: Sphinx for GCC documentation

[Joseph Myers]

On Wed, 7 Apr 2021, Michael Matz wrote:

> Random snippet for what I mean: the .texi contains:
> 
> "The @code{access} attribute specifies that a function to whose 
> by-reference arguments the attribute applies accesses the referenced 
> object according to @var{access-mode}.  The @var{access-mode} argument is 
> required and must be"
> 
> the .rst has:
> 
> "The ``access`` attribute specifies that a function to whose by-reference 
> arguments the attribute applies accesses the referenced object according 
> to :samp:`{access-mode}`.  The :samp:`{access-mode}` argument is required 
> and must be"
> 
> So, @code{}/@var{} vs. `` `` / :samp:``.  Keeping in mind that 

@var in Texinfo is orthogonal to whether something is literal code.  It 
looks like Sphinx's equivalent is {} inside :samp:`` (so not supporting 
the use case of @var outside literal code)?

Although there *is* a difference in output as well as semantics between 
@code and @samp in Texinfo (@samp quotes the output in some cases where 
@code does not, I think), I'm not convinced the GCC manuals actually make 
much distinction between the two.

> (Again, I'm aware that the .texi files aren't perfect here, and @code/@var 
> also seems a bit forced :) )

@var is a metasyntactic variable (the text inside @var is converted to 
uppercase for info output), completely different in usage from @code.  
It's @code/@samp where there isn't much consistency (and other cases such 
as e.g. whether @option or @samp is used for a sequence of more than one 
option).

> (I'm not proposing we go the full extreme of semantic markup that docbook 
> tries, but the other end of the spectrum, i.e. markdown/rst with only 
> presentation markup, also doesn't seem very well fitting)

Presumably we could add our own roles (:gcc:our_own_role:`something`) and 
directives with custom Python code to implement them, whenever we want to 
keep semantics that aren't covered by the default Sphinx-flavoured reST?

However, I don't know how easy it is to keep such code (and manuals) 
compatible with a wide range of Sphinx versions.  Documentation requiring 
the latest version of Sphinx to build cleanly is a pain, I'd prefer to aim 
for e.g. working with any Sphinx release from the past five years.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: RFC: Sphinx for GCC documentation

[Martin Liška]

On 4/7/21 7:40 PM, Joseph Myers wrote:
> On Wed, 7 Apr 2021, Michael Matz wrote:
> 
>> Random snippet for what I mean: the .texi contains:
>>
>> "The @code{access} attribute specifies that a function to whose
>> by-reference arguments the attribute applies accesses the referenced
>> object according to @var{access-mode}.  The @var{access-mode} argument is
>> required and must be"
>>
>> the .rst has:
>>
>> "The ``access`` attribute specifies that a function to whose by-reference
>> arguments the attribute applies accesses the referenced object according
>> to :samp:`{access-mode}`.  The :samp:`{access-mode}` argument is required
>> and must be"
>>
>> So, @code{}/@var{} vs. `` `` / :samp:``.  Keeping in mind that
> 
> @var in Texinfo is orthogonal to whether something is literal code.  It
> looks like Sphinx's equivalent is {} inside :samp:`` (so not supporting
> the use case of @var outside literal code)?

Hello.

Yes, as Joseph says, it's equivalent to {var} within a :samp: directive as documented
here: https://www.sphinx-doc.org/en/master/usage/restructuredtext/roles.html#role-samp

To be honest, if we really want, we can easily come up with even more roles.
But I don't think we would benefit from it.

...

> One other practical concern: it seems there's a one-to-one correspondence 

> of .rst files and (web)page.  Do we really want to maintain hundreds (or 

> how many?) .rst files, instead of 60 .texi files?

Well, based what I know about RST and Sphinx, it's pretty natural that one HTML page
corresponds to a single RST file.

Looking at famous users for Sphinx, I can see the following stats:

linux/Documentation> find . -name '*.rst' | wc -l

2807


godot-docs> find . -name '*.rst' | wc -l

1030


Martin

Re: RFC: Sphinx for GCC documentation

[Martin Liška]

On 4/1/21 3:30 PM, Martin Liška wrote:
> That said, I'm asking the GCC community for a green light before I invest
> more time on it?

Hello.

So far, I've received just a small feedback about the transition. In most cases positive.

May I understand it as green light for the transition?

Thanks,
Martin

[1] https://splichal.eu/scripts/sphinx/

GCC documentation: porting to Sphinx

[Martin Liška]

Hello.

I've made quite some progress with the porting of the documentation and
I would like to present it to the community now:
https://splichal.eu/scripts/sphinx/

Note the documentation is automatically ([1]) generated from texinfo with a GitHub workflow ([2]).
It's built on the devel/sphinx GCC branch which I periodically with the master branch. One can
see the current source .rst files here: [3].

Changes made since the last time:
- a shared content is factored out ([4])
- conditional build is fully supported (even for shared parts)
- manual pages look reasonable well
- folders are created for files which have >= 5 TOC tree entries
- various formatting issues were resolved
- baseconf.py reads BASE-VER, DEV-PHASE, .. files

I've got couple of questions:

1) Do we have to you the following cover text?
        Copyright (c) 1988-2020 Free Software Foundation, Inc.

        Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation; with the Invariant Sections being "GNU General Public
        License" and "Funding Free Software", the Front-Cover texts being (a) (see below), and with the Back-Cover Texts being (b) (see below).  A copy of the license is included in the gfdl(7) man page.

        (a) The FSF's Front-Cover Text is:

             A GNU Manual

        (b) The FSF's Back-Cover Text is:

             You have freedom to copy and modify this GNU Manual, like GNU
             software.  Copies published by the Free Software Foundation raise
             funds for GNU development.

2) Do we want to generate fsf-funding, gpl and gfdl manual pages?
3) Do we want to preserve the current strange copy mechanism for ./gcc/doc/tm.texi.in ?
4) Do we want a copyright header for the created .rst files?

Thoughts?
Thanks,
Martin

[1] https://github.com/davidmalcolm/texi2rst
[2] https://github.com/davidmalcolm/texi2rst/actions
[3] https://github.com/marxin/texi2rst-generated/tree/master/sphinx
[4] https://github.com/marxin/texi2rst-generated/tree/master/sphinx/share

Re: GCC documentation: porting to Sphinx

[Michael Matz]

Hello Martin,

On Mon, 31 May 2021, Martin Liška wrote:

> I've made quite some progress with the porting of the documentation and
> I would like to present it to the community now:
> https://splichal.eu/scripts/sphinx/
>  
> Note the documentation is automatically ([1]) generated from texinfo with a
> GitHub workflow ([2]).

One other thing I was recently thinking about, in the Spinx vs. texinfo 
discussion: locally available documentation browsable/searchable in 
terminal with info(1) (or equivalents).  I think the above (i.e. 
generating .rst from the texinfo file) would immediately nullify all my 
worries.  So, just to be extra sure: your proposal now is to generate the 
.rst files, and that .texinfo remains the maintained sources, right?


Ciao,
Michael.

Re: GCC documentation: porting to Sphinx

[Martin Liška]

On 5/31/21 5:49 PM, Michael Matz wrote:
> Hello Martin,
> 
> On Mon, 31 May 2021, Martin Liška wrote:
> 
>> I've made quite some progress with the porting of the documentation and
>> I would like to present it to the community now:
>> https://splichal.eu/scripts/sphinx/
>>   
>> Note the documentation is automatically ([1]) generated from texinfo with a
>> GitHub workflow ([2]).
> 
> One other thing I was recently thinking about, in the Spinx vs. texinfo
> discussion: locally available documentation browsable/searchable in
> terminal with info(1) (or equivalents).

Yes, that's handy.

> I think the above (i.e.
> generating .rst from the texinfo file) would immediately nullify all my
> worries.  So, just to be extra sure: your proposal now is to generate the
> .rst files, and that .texinfo remains the maintained sources, right?

No, .texinfo files will be gone. However, Sphinx can output to info format:
https://www.sphinx-doc.org/en/master/man/sphinx-build.html#cmdoption-sphinx-build-M

And I've just added the generated Info pages here:
https://splichal.eu/scripts/sphinx/

Hope it helps?
Martin

> 
> 
> Ciao,
> Michael.
> 

Re: GCC documentation: porting to Sphinx

[Michael Matz]

Hello,

On Tue, 1 Jun 2021, Martin Liška wrote:

> On 5/31/21 5:49 PM, Michael Matz wrote:
> > Hello Martin,
> > 
> > On Mon, 31 May 2021, Martin Liška wrote:
> > 
> >> I've made quite some progress with the porting of the documentation and
> >> I would like to present it to the community now:
> >> https://splichal.eu/scripts/sphinx/
> >>   Note the documentation is automatically ([1]) generated from texinfo with
> >> a
> >> GitHub workflow ([2]).
> > 
> > One other thing I was recently thinking about, in the Spinx vs. texinfo
> > discussion: locally available documentation browsable/searchable in
> > terminal with info(1) (or equivalents).
> 
> Yes, that's handy.
> 
> > I think the above (i.e. generating .rst from the texinfo file) would 
> > immediately nullify all my worries.  So, just to be extra sure: your 
> > proposal now is to generate the .rst files, and that .texinfo remains 
> > the maintained sources, right?
> 
> No, .texinfo files will be gone. However, Sphinx can output to info 
> format: 
> https://www.sphinx-doc.org/en/master/man/sphinx-build.html#cmdoption-sphinx-build-M

I see, that's good to hear.

> And I've just added the generated Info pages here:
> https://splichal.eu/scripts/sphinx/

Okay, but there's something amiss, just compare a local gcc.info with 
that.  The sphinx generated one seems to only contain command line 
options, but none of the other topics, in particular it seems to contain 
the "Invoking GCC" chapter (and only that) as top-level, and all other 
ones are missing (like "C implementation", "C++ implementation", "C 
extension", and so on).

Looking at gccint.info I also seem quite some confusion, it's unclear to 
me if content is missing or not.  But e.g. the top-level structure has a 
different order (a less logical one, this one is btw. shared with the 
order of the HTML generated docu, so it's probably specific to sphinx 
setup or such).

Ignoring that missing content what is there right now does seem somewhat 
acceptable for local use, though.


Ciao,
Michael.

Re: GCC documentation: porting to Sphinx

[Martin Liška]

On 6/1/21 3:31 PM, Michael Matz wrote:
> Hello,
> 
> On Tue, 1 Jun 2021, Martin Liška wrote:
> 
>> On 5/31/21 5:49 PM, Michael Matz wrote:
>>> Hello Martin,
>>>
>>> On Mon, 31 May 2021, Martin Liška wrote:
>>>
>>>> I've made quite some progress with the porting of the documentation and
>>>> I would like to present it to the community now:
>>>> https://splichal.eu/scripts/sphinx/
>>>>    Note the documentation is automatically ([1]) generated from texinfo with
>>>> a
>>>> GitHub workflow ([2]).
>>>
>>> One other thing I was recently thinking about, in the Spinx vs. texinfo
>>> discussion: locally available documentation browsable/searchable in
>>> terminal with info(1) (or equivalents).
>>
>> Yes, that's handy.
>>
>>> I think the above (i.e. generating .rst from the texinfo file) would
>>> immediately nullify all my worries.  So, just to be extra sure: your
>>> proposal now is to generate the .rst files, and that .texinfo remains
>>> the maintained sources, right?
>>
>> No, .texinfo files will be gone. However, Sphinx can output to info
>> format:
>> https://www.sphinx-doc.org/en/master/man/sphinx-build.html#cmdoption-sphinx-build-M
> 
> I see, that's good to hear.
> 
>> And I've just added the generated Info pages here:
>> https://splichal.eu/scripts/sphinx/
> 
> Okay, but there's something amiss, just compare a local gcc.info with
> that.  The sphinx generated one seems to only contain command line
> options, but none of the other topics, in particular it seems to contain
> the "Invoking GCC" chapter (and only that) as top-level, and all other
> ones are missing (like "C implementation", "C++ implementation", "C
> extension", and so on).

You are right, I reduced that to 'Invoking GCC', which is simply what 'man gcc'
presents. However, I moved that back to the entire GCC manual what you can
see now in the info page.

> 
> Looking at gccint.info I also seem quite some confusion, it's unclear to
> me if content is missing or not.  But e.g. the top-level structure has a
> different order (a less logical one, this one is btw. shared with the
> order of the HTML generated docu, so it's probably specific to sphinx
> setup or such).

Yes, the organization was bad and I fixed that. Now it's much better.

Martin

> 
> Ignoring that missing content what is there right now does seem somewhat
> acceptable for local use, though.
> 
> 
> Ciao,
> Michael.
> 

Re: GCC documentation: porting to Sphinx

[Joel Sherrill]

For RTEMS, we switched from texinfo to Sphinx and the dependency
on Python3 for Sphinx has caused a bit of hassle. Is this going to be
an issue for GCC?

Also we rely on TexLive for PDF output and that's a bit of a pain to
install. Tex was incorrectly packaged on some RHEL/CentOS versions.

This ignores a couple of plugins we use that I don't expect GCC to use.

It works great but the host dependencies are sometimes a pain. We've
ended up writing host OS specific advice/howto's to address this. Any
expectations on host pain versus the pretty painless texinfo?

Thanks.

--joel
RTEMS

On Wed, Jun 2, 2021 at 2:37 AM Martin Liška <mliska@suse.cz> wrote:

> On 6/1/21 3:31 PM, Michael Matz wrote:
> > Hello,
> >
> > On Tue, 1 Jun 2021, Martin Liška wrote:
> >
> >> On 5/31/21 5:49 PM, Michael Matz wrote:
> >>> Hello Martin,
> >>>
> >>> On Mon, 31 May 2021, Martin Liška wrote:
> >>>
> >>>> I've made quite some progress with the porting of the documentation
> and
> >>>> I would like to present it to the community now:
> >>>> https://splichal.eu/scripts/sphinx/
> >>>>    Note the documentation is automatically ([1]) generated from
> texinfo with
> >>>> a
> >>>> GitHub workflow ([2]).
> >>>
> >>> One other thing I was recently thinking about, in the Spinx vs. texinfo
> >>> discussion: locally available documentation browsable/searchable in
> >>> terminal with info(1) (or equivalents).
> >>
> >> Yes, that's handy.
> >>
> >>> I think the above (i.e. generating .rst from the texinfo file) would
> >>> immediately nullify all my worries.  So, just to be extra sure: your
> >>> proposal now is to generate the .rst files, and that .texinfo remains
> >>> the maintained sources, right?
> >>
> >> No, .texinfo files will be gone. However, Sphinx can output to info
> >> format:
> >>
> https://www.sphinx-doc.org/en/master/man/sphinx-build.html#cmdoption-sphinx-build-M
> >
> > I see, that's good to hear.
> >
> >> And I've just added the generated Info pages here:
> >> https://splichal.eu/scripts/sphinx/
> >
> > Okay, but there's something amiss, just compare a local gcc.info with
> > that.  The sphinx generated one seems to only contain command line
> > options, but none of the other topics, in particular it seems to contain
> > the "Invoking GCC" chapter (and only that) as top-level, and all other
> > ones are missing (like "C implementation", "C++ implementation", "C
> > extension", and so on).
>
> You are right, I reduced that to 'Invoking GCC', which is simply what 'man
> gcc'
> presents. However, I moved that back to the entire GCC manual what you can
> see now in the info page.
>
> >
> > Looking at gccint.info I also seem quite some confusion, it's unclear to
> > me if content is missing or not.  But e.g. the top-level structure has a
> > different order (a less logical one, this one is btw. shared with the
> > order of the HTML generated docu, so it's probably specific to sphinx
> > setup or such).
>
> Yes, the organization was bad and I fixed that. Now it's much better.
>
> Martin
>
> >
> > Ignoring that missing content what is there right now does seem somewhat
> > acceptable for local use, though.
> >
> >
> > Ciao,
> > Michael.
> >
>
>

Re: GCC documentation: porting to Sphinx

[Joseph Myers]

On Wed, 2 Jun 2021, Joel Sherrill wrote:

> For RTEMS, we switched from texinfo to Sphinx and the dependency
> on Python3 for Sphinx has caused a bit of hassle. Is this going to be
> an issue for GCC?

What Sphinx (and, thus, Python) versions does the GCC manual build work 
with?  Can it work with e.g. any Sphinx versions from the past five years, 
or are there newer Sphinx features that are critical for the GCC manuals?  
I've seen a need for frequent Sphinx updates being a pain when building 
other software using Sphinx for its manual.

> Also we rely on TexLive for PDF output and that's a bit of a pain to
> install. Tex was incorrectly packaged on some RHEL/CentOS versions.

This is nothing new, since building PDF manuals from Texinfo sources also 
needs TeX.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: GCC documentation: porting to Sphinx

[Joseph Myers]

On Mon, 31 May 2021, Martin Liška wrote:

> https://splichal.eu/scripts/sphinx/

Looking at some examples there:

https://splichal.eu/scripts/sphinx/gcc/_build/html/c-implementation-defined-behavior/preprocessing-directives.html 
has some conversion problems:

* "See Implementation-defined behavior, for details of these aspects of 
implementation-defined behavior." is missing the link to the relevant 
section of the cpp manual that's present in the Texinfo source.

* "` character before the :samp:`" is a misconversion (whether from 
Texinfo to RST or from RST to HTML) of the Texinfo source

  @samp{\} character before the @samp{\}

which will need to be fixed.

* The corresponding PDF has the same issues as above (so probably they are 
issues with the conversion to RST, not with Sphinx itself).  In addition, 
the PDF manual ought to be using fixed-width fonts for literal code, 
command-line options, etc., just like the HTML manual, and the 
Texinfo-generated PDF manual, are.

https://splichal.eu/scripts/sphinx/gcc/_build/html/gcc-command-options/passing-options-to-the-assembler.html 
shows headings such as "-Wa,option, -Wa".  The ", -Wa" doesn't make sense, 
this option is just "-Wa,option".

https://splichal.eu/scripts/sphinx/gcc/_build/html/gcov-a-test-coverage-program.html 
has a hyphen between "gcov" and "a Test Coverage Program" in the heading.  
It should be an em dash, as in Texinfo.

https://splichal.eu/scripts/sphinx/gcc/_build/html/language-standards-supported-by-gcc/c%2B%2B-language.html 
has doubled slashes in various URLs where the Texinfo source has /@/ 
(Texinfo @/ means "allow line break", it should not be translated to /).

https://splichal.eu/scripts/sphinx/gcc/_build/html/gcc-command-options/machine-dependent-options/aarch64-options.html 
shows different formatting for the headings for "-mlow-precision-div, 
-mno-low-precision-div" and "-mtrack-speculation -mno-track-speculation".  
The formatting should be identical.  The only difference in the Texinfo 
source seems to be that the latter is missing @opindex directives.  And 
while it's a bug in the Texinfo source that those directives are missing, 
the presence or absence of index entries should not affect the formatting 
of the documentation for those options.

On that same page, the output for -march=name is broken, containing a 
literal :samp:{feature} (in general, checking for any places where RST 
directives such as :samp: appear in the HTML output might be a good idea 
to look for broken conversions).  The Texinfo source here has

@option{-march=@var{arch}@r{@{}+@r{[}no@r{]}@var{feature}@r{@}*}}

(where the use of @r{...} is to put the {}[]* characters in a 
variable-width font, since they are not literally part of the option, 
while the other characters that are literally part of the option should be 
in a variable-width font).

https://splichal.eu/scripts/sphinx/gcc/_build/html/language-standards-supported-by-gcc/references-for-other-languages.html 
has literal unconverted "@c man" and "@include" and other Texinfo 
directives.  Searching for such things in the HTML output (or the RST 
sources) is a good idea, just like searching for literal RST directives in 
the HTML output, to find other such conversion bugs.

https://splichal.eu/scripts/sphinx/gcc/_build/html/gcc-command-options.html 
says "See option-index", another case with a link that didn't get 
converted properly.  It also has raw :samp: uses indicating a 
misconversion.

I'm not sure how you're determining languages for code-block, but 
https://splichal.eu/scripts/sphinx/gcc/_build/html/gcc-command-options/options-to-control-diagnostic-messages-formatting.html 
certainly shows some cases where they have been misidentified (e.g. random 
C++ keywords highlighted in the default GCC_COLORS, some JSON being 
highlighted as such but other JSON not).

> - a shared content is factored out ([4])
> - conditional build is fully supported (even for shared parts)
> - manual pages look reasonable well
> - folders are created for files which have >= 5 TOC tree entries
> - various formatting issues were resolved
> - baseconf.py reads BASE-VER, DEV-PHASE, .. files

Could you give more detailed descriptions of how each of the various 
issues I listed in 2015 are addressed here?

https://gcc.gnu.org/legacy-ml/gcc-patches/2015-11/msg01139.html

> I've got couple of questions:
> 
> 1) Do we have to you the following cover text?
>        Copyright (c) 1988-2020 Free Software Foundation, Inc.
> 
>        Permission is granted to copy, distribute and/or modify this document
> under the terms of the GNU Free Documentation License, Version 1.3 or any
> later version published by the Free Software Foundation; with the Invariant
> Sections being "GNU General Public
>        License" and "Funding Free Software", the Front-Cover texts being (a)
> (see below), and with the Back-Cover Texts being (b) (see below).  A copy of
> the license is included in the gfdl(7) man page.
> 
>        (a) The FSF's Front-Cover Text is:
> 
>             A GNU Manual
> 
>        (b) The FSF's Back-Cover Text is:
> 
>             You have freedom to copy and modify this GNU Manual, like GNU
>             software.  Copies published by the Free Software Foundation raise
>             funds for GNU development.

We need to keep the Cover Texts and Invariant Sections, in the absence of 
FSF approval to remove them.

> 2) Do we want to generate fsf-funding, gpl and gfdl manual pages?

Yes, this is how the set of man pages as a whole keeps the invariant 
sections.

> 3) Do we want to preserve the current strange copy mechanism for
> ./gcc/doc/tm.texi.in ?

Yes, this is how we ensure we have both GPL and GFDL copies of the target 
hook documentation checked in and that someone copying from one place to 
another makes sure they have any relevant permissions.

> 4) Do we want a copyright header for the created .rst files?

Yes, all source files should have a copyright header.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: GCC documentation: porting to Sphinx

[Martin Sebor]

On 5/31/21 7:25 AM, Martin Liška wrote:
> Hello.
> 
> I've made quite some progress with the porting of the documentation and
> I would like to present it to the community now:
> https://splichal.eu/scripts/sphinx/

Just a few issues I noticed in the warnings section:

The headings of some warnings mention the same option twice (e.g.,
-Wabi, -Wabi, -Wno-abi;  -Wdouble-promotion, -Wdouble-promotion,
-Wno-double-promotion;  -Winit-self, -Winit-self, -Wno-init-self).
This looks like a pretty pervasive problem.

Mentioning the -Wno-xxx option is redundant in a heading for -Wxxx.

The headings of some other warnings also mention options that are
only remotely related to them.  E.g., -Wformat has all these:

   -Wformat, -Wno-format, -ffreestanding, -fno-builtin, -Wformat=

(I see the same problem in the attributes section where the headings
for some attributes include option names).

That seems quite puzzling.  I assume it's a consequence of having
index entries for the related options, but I don't think making
them visible in the headings is helfpful.

Headings that in the manual today include a level like

   -Wformat-overflow
   -Wformat-overflow=level

don't mention the level in the Spinx manual:

   -Wformat-overflow, -Wno-format-overflow

When the /level/ is then discussed in the rest of the text it's
not clear what it refers to.

Martin

> 
> Note the documentation is automatically ([1]) generated from texinfo 
> with a GitHub workflow ([2]).
> It's built on the devel/sphinx GCC branch which I periodically with the 
> master branch. One can
> see the current source .rst files here: [3].
> 
> Changes made since the last time:
> - a shared content is factored out ([4])
> - conditional build is fully supported (even for shared parts)
> - manual pages look reasonable well
> - folders are created for files which have >= 5 TOC tree entries
> - various formatting issues were resolved
> - baseconf.py reads BASE-VER, DEV-PHASE, .. files
> 
> I've got couple of questions:
> 
> 1) Do we have to you the following cover text?
>         Copyright (c) 1988-2020 Free Software Foundation, Inc.
> 
>         Permission is granted to copy, distribute and/or modify this 
> document under the terms of the GNU Free Documentation License, Version 
> 1.3 or any later version published by the Free Software Foundation; with 
> the Invariant Sections being "GNU General Public
>         License" and "Funding Free Software", the Front-Cover texts 
> being (a) (see below), and with the Back-Cover Texts being (b) (see 
> below).  A copy of the license is included in the gfdl(7) man page.
> 
>         (a) The FSF's Front-Cover Text is:
> 
>              A GNU Manual
> 
>         (b) The FSF's Back-Cover Text is:
> 
>              You have freedom to copy and modify this GNU Manual, like GNU
>              software.  Copies published by the Free Software Foundation 
> raise
>              funds for GNU development.
> 
> 2) Do we want to generate fsf-funding, gpl and gfdl manual pages?
> 3) Do we want to preserve the current strange copy mechanism for 
> ./gcc/doc/tm.texi.in ?
> 4) Do we want a copyright header for the created .rst files?
> 
> Thoughts?
> Thanks,
> Martin
> 
> [1] https://github.com/davidmalcolm/texi2rst
> [2] https://github.com/davidmalcolm/texi2rst/actions
> [3] https://github.com/marxin/texi2rst-generated/tree/master/sphinx
> [4] https://github.com/marxin/texi2rst-generated/tree/master/sphinx/share

Re: GCC documentation: porting to Sphinx

[Martin Liška]

On 6/2/21 10:41 PM, Martin Sebor wrote:
> On 5/31/21 7:25 AM, Martin Liška wrote:
>> Hello.
>>
>> I've made quite some progress with the porting of the documentation and
>> I would like to present it to the community now:
>> https://splichal.eu/scripts/sphinx/
> 

Hello.

Thank you for the review.

> Just a few issues I noticed in the warnings section:
> 
> The headings of some warnings mention the same option twice (e.g.,
> -Wabi, -Wabi, -Wno-abi;  -Wdouble-promotion, -Wdouble-promotion,
> -Wno-double-promotion;  -Winit-self, -Winit-self, -Wno-init-self).
> This looks like a pretty pervasive problem.

You are right, I fixed that.

> 
> Mentioning the -Wno-xxx option is redundant in a heading for -Wxxx.

Yes. Good reason for that is that Sphinx can then generated properly links
to the current non-documented version of the option. Hope it's improvement
over the current situation?

> 
> The headings of some other warnings also mention options that are
> only remotely related to them.  E.g., -Wformat has all these:
> 
>    -Wformat, -Wno-format, -ffreestanding, -fno-builtin, -Wformat=
> 
> (I see the same problem in the attributes section where the headings
> for some attributes include option names).
> 
> That seems quite puzzling.  I assume it's a consequence of having
> index entries for the related options, but I don't think making
> them visible in the headings is helfpful.

Oh, you are right. It was consequence of wrong parsing of index entries.
It should be fixed now.

> 
> Headings that in the manual today include a level like
> 
>    -Wformat-overflow
>    -Wformat-overflow=level
> 
> don't mention the level in the Spinx manual:
> 
>    -Wformat-overflow, -Wno-format-overflow
> 
> When the /level/ is then discussed in the rest of the text it's
> not clear what it refers to.

Should be also fixed now.

Can you please take a look at the current output and give me a feedback?
Thanks,
Martin

> 
> Martin
> 
>>
>> Note the documentation is automatically ([1]) generated from texinfo with a GitHub workflow ([2]).
>> It's built on the devel/sphinx GCC branch which I periodically with the master branch. One can
>> see the current source .rst files here: [3].
>>
>> Changes made since the last time:
>> - a shared content is factored out ([4])
>> - conditional build is fully supported (even for shared parts)
>> - manual pages look reasonable well
>> - folders are created for files which have >= 5 TOC tree entries
>> - various formatting issues were resolved
>> - baseconf.py reads BASE-VER, DEV-PHASE, .. files
>>
>> I've got couple of questions:
>>
>> 1) Do we have to you the following cover text?
>>         Copyright (c) 1988-2020 Free Software Foundation, Inc.
>>
>>         Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation; with the Invariant Sections being "GNU General Public
>>         License" and "Funding Free Software", the Front-Cover texts being (a) (see below), and with the Back-Cover Texts being (b) (see below).  A copy of the license is included in the gfdl(7) man page.
>>
>>         (a) The FSF's Front-Cover Text is:
>>
>>              A GNU Manual
>>
>>         (b) The FSF's Back-Cover Text is:
>>
>>              You have freedom to copy and modify this GNU Manual, like GNU
>>              software.  Copies published by the Free Software Foundation raise
>>              funds for GNU development.
>>
>> 2) Do we want to generate fsf-funding, gpl and gfdl manual pages?
>> 3) Do we want to preserve the current strange copy mechanism for ./gcc/doc/tm.texi.in ?
>> 4) Do we want a copyright header for the created .rst files?
>>
>> Thoughts?
>> Thanks,
>> Martin
>>
>> [1] https://github.com/davidmalcolm/texi2rst
>> [2] https://github.com/davidmalcolm/texi2rst/actions
>> [3] https://github.com/marxin/texi2rst-generated/tree/master/sphinx
>> [4] https://github.com/marxin/texi2rst-generated/tree/master/sphinx/share
> 

Re: GCC documentation: porting to Sphinx

[Martin Liška]

On 6/2/21 6:44 PM, Joseph Myers wrote:
> On Wed, 2 Jun 2021, Joel Sherrill wrote:
> 
>> For RTEMS, we switched from texinfo to Sphinx and the dependency
>> on Python3 for Sphinx has caused a bit of hassle. Is this going to be
>> an issue for GCC?
> 
> What Sphinx (and, thus, Python) versions does the GCC manual build work
> with?

I've just tried version 1.7.6 which we use for libgccjit and it's fine:
https://gcc.gnu.org/onlinedocs/jit/

About Python version: I'm not planning supporting Python2, it's dead 10 years already.

> Can it work with e.g. any Sphinx versions from the past five years,
> or are there newer Sphinx features that are critical for the GCC manuals?

The mentioned version is 3 years and I expect even older releases would work.
No, I don't rely on any new feature.

I would recommend testing the build. You can simply clone:
https://github.com/marxin/texi2rst-generated

and simply run 'make html' or 'make latexpdf'. Basic dependencies are mentioned here:
https://github.com/marxin/texi2rst-generated#requirements

Martin

> I've seen a need for frequent Sphinx updates being a pain when building
> other software using Sphinx for its manual.
> 
>> Also we rely on TexLive for PDF output and that's a bit of a pain to
>> install. Tex was incorrectly packaged on some RHEL/CentOS versions.
> 
> This is nothing new, since building PDF manuals from Texinfo sources also
> needs TeX.
> 

Re: GCC documentation: porting to Sphinx

[Joseph Myers]

On Thu, 3 Jun 2021, Martin Liška wrote:

> On 6/2/21 6:44 PM, Joseph Myers wrote:
> > On Wed, 2 Jun 2021, Joel Sherrill wrote:
> > 
> > > For RTEMS, we switched from texinfo to Sphinx and the dependency
> > > on Python3 for Sphinx has caused a bit of hassle. Is this going to be
> > > an issue for GCC?
> > 
> > What Sphinx (and, thus, Python) versions does the GCC manual build work
> > with?
> 
> I've just tried version 1.7.6 which we use for libgccjit and it's fine:
> https://gcc.gnu.org/onlinedocs/jit/
> 
> About Python version: I'm not planning supporting Python2, it's dead 10 years
> already.

There should be appropriate configure checks to avoid building manuals 
with too-old versions (i.e. disable the info/man manual build/install when 
Sphinx, or the Python version it's using, is too old or missing, not fail 
configure).

Actually this code is depending on Python 3.6 or later because of the use 
of an f-string in baseconf.py (without that f-string, it works with older 
versions, even 2.7).  Formally 3.5 and older are no longer supported 
upstream, but certainly still present in some maintained long-term-support 
distribution versions.

> I would recommend testing the build. You can simply clone:
> https://github.com/marxin/texi2rst-generated
> 
> and simply run 'make html' or 'make latexpdf'. Basic dependencies are
> mentioned here:
> https://github.com/marxin/texi2rst-generated#requirements

It appears "make html" works (with lots of WARNINGs) with Sphinx 1.6.1 but 
fails with 1.4 ("Theme error: unsupported theme option 
'prev_next_buttons_location' given").

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: GCC documentation: porting to Sphinx

[Martin Liška]

On 6/3/21 7:16 PM, Joseph Myers wrote:
> On Thu, 3 Jun 2021, Martin Liška wrote:
> 
>> On 6/2/21 6:44 PM, Joseph Myers wrote:
>>> On Wed, 2 Jun 2021, Joel Sherrill wrote:
>>>
>>>> For RTEMS, we switched from texinfo to Sphinx and the dependency
>>>> on Python3 for Sphinx has caused a bit of hassle. Is this going to be
>>>> an issue for GCC?
>>>
>>> What Sphinx (and, thus, Python) versions does the GCC manual build work
>>> with?
>>
>> I've just tried version 1.7.6 which we use for libgccjit and it's fine:
>> https://gcc.gnu.org/onlinedocs/jit/
>>
>> About Python version: I'm not planning supporting Python2, it's dead 10 years
>> already.
> 
> There should be appropriate configure checks to avoid building manuals
> with too-old versions (i.e. disable the info/man manual build/install when
> Sphinx, or the Python version it's using, is too old or missing, not fail
> configure).

Sure, that makes sense.

> 
> Actually this code is depending on Python 3.6 or later because of the use
> of an f-string in baseconf.py (without that f-string, it works with older
> versions, even 2.7).

Yeah, I used the f-string syntax only at one place and it does not pay off.

> Formally 3.5 and older are no longer supported
> upstream, but certainly still present in some maintained long-term-support
> distribution versions.

Makes sense.

> 
>> I would recommend testing the build. You can simply clone:
>> https://github.com/marxin/texi2rst-generated
>>
>> and simply run 'make html' or 'make latexpdf'. Basic dependencies are
>> mentioned here:
>> https://github.com/marxin/texi2rst-generated#requirements
> 
> It appears "make html" works (with lots of WARNINGs) with Sphinx 1.6.1 but
> fails with 1.4 ("Theme error: unsupported theme option
> 'prev_next_buttons_location' given").
> 

I checked that and the template needs at least version 1.6:
https://sphinx-rtd-theme.readthedocs.io/en/latest/installing.html#compatibility

so I added needs_sphinx to baseconf.py:
https://www.sphinx-doc.org/en/master/usage/configuration.html?highlight=conf.py#confval-needs_sphinx

The following message is displayed when one builds a manual:

$ make html

sphinx-build -b "html" -d _build/doctrees    . "_build/html"

Running Sphinx v4.0.2



Sphinx version error:

This project needs at least Sphinx v66.6 and therefore cannot be built with this version.

make: *** [Makefile:96: html] Error 2


Martin

Re: RFC: Sphinx for GCC documentation

[Tobias Burnus]

Hello,

On 13.05.21 13:45, Martin Liška wrote:
> On 4/1/21 3:30 PM, Martin Liška wrote:
>> That said, I'm asking the GCC community for a green light before I
>> invest
>> more time on it?
> So far, I've received just a small feedback about the transition. In
> most cases positive.
>
> [1] https://splichal.eu/scripts/sphinx/

The HTML output looks quite nice.

What I observed:

* Looking at
   https://splichal.eu/scripts/sphinx/gfortran/_build/html/intrinsic-procedures/access-checks-file-access-modes.html
why is the first argument description in bold?
It is also not very readable to have a scollbar there – linebreaks would be better.
→ I think that's because the assumption is that the first line contains a header
   and the rest the data

* https://splichal.eu/scripts/sphinx/gfortran/_build/latex/gfortran.pdf
   If I look at page 92 (alias 96), 8.2.13 _gfortran_caf_sendget, the first column
   is too small to fit the argument names. – Admittedly, the current gfortran.pdf
   is not much better – it is very tight but just fits. I don't know how to fix this.

* I note that we write before the argument index, that those are without -/-- prefix
   but that's not true. Something to fix after the conversation.

* The syntax highlighting for gfortran is odd. Looking at @smallexample:
- intrinsic.texi: All Fortran examples (F90/free-form)
- gfc-internals.texi: 4x Fortran, 4x C, 3x plain text
- gfortran.texi: Shell, Fortran, C, plain text.
- invoke.texi: 4x Shell, 2x C, 4x Fortran
Does not seem to be that simple, but it would be nice if at least all in
intrinsic.texi would be marked as Fortran.

Actually, I do not quite understand when the output is formatted a C (wrongly
or rightly) as Fortran (rarely but correctly) as plain or in some odd formatting
which randomly highlights some examples.
Possibly also an item for after the conversion.

Tobias

-----------------
Mentor Graphics (Deutschland) GmbH, Arnulfstrasse 201, 80634 München Registergericht München HRB 106955, Geschäftsführer: Thomas Heurung, Frank Thürauf

Re: RFC: Sphinx for GCC documentation

[Koning, Paul]

> On Jun 4, 2021, at 3:55 AM, Tobias Burnus <tobias@codesourcery.com> wrote:
> 
> Hello,
> 
> On 13.05.21 13:45, Martin Liška wrote:
>> On 4/1/21 3:30 PM, Martin Liška wrote:
>>> That said, I'm asking the GCC community for a green light before I
>>> invest
>>> more time on it?
>> So far, I've received just a small feedback about the transition. In
>> most cases positive.
>> 
>> [1] https://splichal.eu/scripts/sphinx/
> 
> The HTML output looks quite nice.
> 
> What I observed:
> 
> * Looking at
>  https://splichal.eu/scripts/sphinx/gfortran/_build/html/intrinsic-procedures/access-checks-file-access-modes.html
> why is the first argument description in bold?
> It is also not very readable to have a scollbar there – linebreaks would be better.
> → I think that's because the assumption is that the first line contains a header
>  and the rest the data

Explicit line breaks are likely to be wrong depending on the reader's window size.  I would suggest setting the table to have cells with line-wrapped contents.  That would typically be the default in HTML, I'm curious why that is not happening here.

	paul

Re: GCC documentation: porting to Sphinx

[Martin Sebor]

On 6/3/21 4:56 AM, Martin Liška wrote:
> On 6/2/21 10:41 PM, Martin Sebor wrote:
>> On 5/31/21 7:25 AM, Martin Liška wrote:
>>> Hello.
>>>
>>> I've made quite some progress with the porting of the documentation and
>>> I would like to present it to the community now:
>>> https://splichal.eu/scripts/sphinx/
>>
> 
> Hello.
> 
> Thank you for the review.
> 
>> Just a few issues I noticed in the warnings section:
>>
>> The headings of some warnings mention the same option twice (e.g.,
>> -Wabi, -Wabi, -Wno-abi;  -Wdouble-promotion, -Wdouble-promotion,
>> -Wno-double-promotion;  -Winit-self, -Winit-self, -Wno-init-self).
>> This looks like a pretty pervasive problem.
> 
> You are right, I fixed that.

Looks good.

> 
>>
>> Mentioning the -Wno-xxx option is redundant in a heading for -Wxxx.
> 
> Yes. Good reason for that is that Sphinx can then generated properly links
> to the current non-documented version of the option. Hope it's improvement
> over the current situation?

I think the linking is helpful.  But for warnings, the documented
convention is to only mention the one that's not the default:

   This manual lists only one of the two forms, whichever is not
   the default.

so including both blurs this (IMO rather subtle) distinction.
In addition, in options whose description says something like
"This warning is enabled by -Wall." it's now less clear which
one is the one the "this" refers to (see for example
-Wchar-subscripts).

If the heading can't be changed at a minimum we'll need to update
the convention above, e.g., by saying that the first option mentions
is the default. But again, I think this is too subtle for the casual
reader to pick up on.  The fact that the sentence quoted above appears
under -Wfatal-errors doesn't help.  We should also work on updating
the "This option is in -Wall." either to name the specific option
it refers to, or consider moving that into a Note box like the one
listing the languages the option applies to.)

> 
>>
>> The headings of some other warnings also mention options that are
>> only remotely related to them.  E.g., -Wformat has all these:
>>
>>    -Wformat, -Wno-format, -ffreestanding, -fno-builtin, -Wformat=
>>
>> (I see the same problem in the attributes section where the headings
>> for some attributes include option names).
>>
>> That seems quite puzzling.  I assume it's a consequence of having
>> index entries for the related options, but I don't think making
>> them visible in the headings is helfpful.
> 
> Oh, you are right. It was consequence of wrong parsing of index entries.
> It should be fixed now.

Looks good.

> 
>>
>> Headings that in the manual today include a level like
>>
>>    -Wformat-overflow
>>    -Wformat-overflow=level
>>
>> don't mention the level in the Spinx manual:
>>
>>    -Wformat-overflow, -Wno-format-overflow
>>
>> When the /level/ is then discussed in the rest of the text it's
>> not clear what it refers to.
> 
> Should be also fixed now.

Also looks good.

> 
> Can you please take a look at the current output and give me a feedback?

I noticed another minor issue that may already have been pointed
out by someone else.  Under -Wall (and -Wextra), some option names
are prefixed by :option: (e.g., (only with :option:-O2``).  Looks
like some sort of a transcription bug?

And a couple of questions:

References to options with an argument like -Warray-bounds=1 are
rendered in a way that makes it look like there's a space before
the equals: -Warray-bounds =1, with  the =1 being in a different
color and not part of the hyperlink. Is there a way to make it look
like there is no space?

I like how options are automatically linked, and I'd like to see
the same for other references like to attributes.  Can that be
automated as part of the migration or should we/I try to tackle
it in a followup?

In any event, thanks for working so hard on making this turn out
great!

Martin

> Thanks,
> Martin
> 
>>
>> Martin
>>
>>>
>>> Note the documentation is automatically ([1]) generated from texinfo 
>>> with a GitHub workflow ([2]).
>>> It's built on the devel/sphinx GCC branch which I periodically with 
>>> the master branch. One can
>>> see the current source .rst files here: [3].
>>>
>>> Changes made since the last time:
>>> - a shared content is factored out ([4])
>>> - conditional build is fully supported (even for shared parts)
>>> - manual pages look reasonable well
>>> - folders are created for files which have >= 5 TOC tree entries
>>> - various formatting issues were resolved
>>> - baseconf.py reads BASE-VER, DEV-PHASE, .. files
>>>
>>> I've got couple of questions:
>>>
>>> 1) Do we have to you the following cover text?
>>>         Copyright (c) 1988-2020 Free Software Foundation, Inc.
>>>
>>>         Permission is granted to copy, distribute and/or modify this 
>>> document under the terms of the GNU Free Documentation License, 
>>> Version 1.3 or any later version published by the Free Software 
>>> Foundation; with the Invariant Sections being "GNU General Public
>>>         License" and "Funding Free Software", the Front-Cover texts 
>>> being (a) (see below), and with the Back-Cover Texts being (b) (see 
>>> below).  A copy of the license is included in the gfdl(7) man page.
>>>
>>>         (a) The FSF's Front-Cover Text is:
>>>
>>>              A GNU Manual
>>>
>>>         (b) The FSF's Back-Cover Text is:
>>>
>>>              You have freedom to copy and modify this GNU Manual, 
>>> like GNU
>>>              software.  Copies published by the Free Software 
>>> Foundation raise
>>>              funds for GNU development.
>>>
>>> 2) Do we want to generate fsf-funding, gpl and gfdl manual pages?
>>> 3) Do we want to preserve the current strange copy mechanism for 
>>> ./gcc/doc/tm.texi.in ?
>>> 4) Do we want a copyright header for the created .rst files?
>>>
>>> Thoughts?
>>> Thanks,
>>> Martin
>>>
>>> [1] https://github.com/davidmalcolm/texi2rst
>>> [2] https://github.com/davidmalcolm/texi2rst/actions
>>> [3] https://github.com/marxin/texi2rst-generated/tree/master/sphinx
>>> [4] 
>>> https://github.com/marxin/texi2rst-generated/tree/master/sphinx/share
>>
> 

Re: RFC: Sphinx for GCC documentation

[Martin Liška]

On 6/4/21 9:55 AM, Tobias Burnus wrote:
> Hello,
> 
> On 13.05.21 13:45, Martin Liška wrote:
>> On 4/1/21 3:30 PM, Martin Liška wrote:
>>> That said, I'm asking the GCC community for a green light before I
>>> invest
>>> more time on it?
>> So far, I've received just a small feedback about the transition. In
>> most cases positive.
>>
>> [1] https://splichal.eu/scripts/sphinx/
> 

Hi.

> The HTML output looks quite nice.

Thanks.

> 
> What I observed:
> 
> * Looking at
>    https://splichal.eu/scripts/sphinx/gfortran/_build/html/intrinsic-procedures/access-checks-file-access-modes.html
> why is the first argument description in bold?
> It is also not very readable to have a scollbar there – linebreaks would be better.
> → I think that's because the assumption is that the first line contains a header
>    and the rest the data

I've converted the problematic table to '.. function::' directive that has :returns: and :param: arguments.
Hope the output is fine now?

> 
> * https://splichal.eu/scripts/sphinx/gfortran/_build/latex/gfortran.pdf
>    If I look at page 92 (alias 96), 8.2.13 _gfortran_caf_sendget, the first column
>    is too small to fit the argument names. – Admittedly, the current gfortran.pdf
>    is not much better – it is very tight but just fits. I don't know how to fix this.

This is also converted and should look much better.

> 
> * I note that we write before the argument index, that those are without -/-- prefix
>    but that's not true. Something to fix after the conversation.

Can you please show me a few examples of it?

> 
> * The syntax highlighting for gfortran is odd. Looking at @smallexample:
> - intrinsic.texi: All Fortran examples (F90/free-form)
> - gfc-internals.texi: 4x Fortran, 4x C, 3x plain text
> - gfortran.texi: Shell, Fortran, C, plain text.
> - invoke.texi: 4x Shell, 2x C, 4x Fortran

Should be fixed now as I set 'fortran' code-block in the fortran manual.
Right now, there are few warnings that a code block is C/C++, but that's
quite a small fallout.

> Does not seem to be that simple, but it would be nice if at least all in
> intrinsic.texi would be marked as Fortran.

Should be better now?

> 
> Actually, I do not quite understand when the output is formatted a C (wrongly
> or rightly) as Fortran (rarely but correctly) as plain or in some odd formatting
> which randomly highlights some examples.

We make quests based on keyworks in a code-block and we also consider texinfo filename.

> Possibly also an item for after the conversion.

Sure, there are still some warnings that can be seen with 'make html' (or other target).

Thanks for review,
Martin

> 
> Tobias
> 
> -----------------
> Mentor Graphics (Deutschland) GmbH, Arnulfstrasse 201, 80634 München Registergericht München HRB 106955, Geschäftsführer: Thomas Heurung, Frank Thürauf

Re: RFC: Sphinx for GCC documentation

[Martin Liška]

On 6/4/21 4:24 PM, Koning, Paul wrote:
> 
> 
>> On Jun 4, 2021, at 3:55 AM, Tobias Burnus <tobias@codesourcery.com> wrote:
>>
>> Hello,
>>
>> On 13.05.21 13:45, Martin Liška wrote:
>>> On 4/1/21 3:30 PM, Martin Liška wrote:
>>>> That said, I'm asking the GCC community for a green light before I
>>>> invest
>>>> more time on it?
>>> So far, I've received just a small feedback about the transition. In
>>> most cases positive.
>>>
>>> [1] https://splichal.eu/scripts/sphinx/
>>
>> The HTML output looks quite nice.
>>
>> What I observed:
>>
>> * Looking at
>>   https://splichal.eu/scripts/sphinx/gfortran/_build/html/intrinsic-procedures/access-checks-file-access-modes.html
>> why is the first argument description in bold?
>> It is also not very readable to have a scollbar there – linebreaks would be better.
>> → I think that's because the assumption is that the first line contains a header
>>   and the rest the data
> 
> Explicit line breaks are likely to be wrong depending on the reader's window size.  I would suggest setting the table to have cells with line-wrapped contents.  That would typically be the default in HTML, I'm curious why that is not happening here.

Note that Sphinx supports 2 types of tables: Grid Table and Simple table. We prefer the later and
one can do proper line breaking in the Grid type.

https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html#tables

Anyway, this is resolved as I use more appropriate directive:
https://splichal.eu/scripts/sphinx/gfortran/_build/html/intrinsic-procedures/access-checks-file-access-modes.html

Martin

> 
> 	paul
> 
> 

Re: RFC: Sphinx for GCC documentation

[Tobias Burnus]

On 07.06.21 15:28, Martin Liška wrote:
>> * I note that we write before the argument index, that those are
>> without -/-- prefix
>>    but that's not true. Something to fix after the conversation.
>
> Can you please show me a few examples of it?

* https://splichal.eu/scripts/sphinx/gfortran/_build/html/option-index.html

* https://splichal.eu/scripts/sphinx/gcc/_build/html/option-index.html

* https://splichal.eu/scripts/sphinx/cpp/_build/html/option-index.html

Read the text on those page – and then look at the converted index,
which has all the '-' and '--' before the options.

That's not a conversion bug – a change which needs to be done as part of
or after the conversion.

Tobias

-----------------
Mentor Graphics (Deutschland) GmbH, Arnulfstrasse 201, 80634 München Registergericht München HRB 106955, Geschäftsführer: Thomas Heurung, Frank Thürauf

Re: RFC: Sphinx for GCC documentation

[Bernhard Reutner-Fischer]

On Mon, 7 Jun 2021 15:30:22 +0200
Martin Liška <mliska@suse.cz> wrote:

> Anyway, this is resolved as I use more appropriate directive:
> https://splichal.eu/scripts/sphinx/gfortran/_build/html/intrinsic-procedures/access-checks-file-access-modes.html

ISTM there's a typo s/Tailing/Trailing/ in gcc/fortran/intrinsic.texi

git grep -wi Tailing
seems to highlight a couple more.
Maybe you have time to fix these?

PS: The occurrence in gcc/testsuite/gcc.dg/format/strfmon-1.c sounds
odd.
TIA,

Re: RFC: Sphinx for GCC documentation

[Martin Liška]

On 6/7/21 11:26 PM, Bernhard Reutner-Fischer wrote:
> On Mon, 7 Jun 2021 15:30:22 +0200
> Martin Liška <mliska@suse.cz> wrote:
> 
>> Anyway, this is resolved as I use more appropriate directive:
>> https://splichal.eu/scripts/sphinx/gfortran/_build/html/intrinsic-procedures/access-checks-file-access-modes.html
> 
> ISTM there's a typo s/Tailing/Trailing/ in gcc/fortran/intrinsic.texi

Yes, it is :)

> 
> git grep -wi Tailing
> seems to highlight a couple more.
> Maybe you have time to fix these?
> 
> PS: The occurrence in gcc/testsuite/gcc.dg/format/strfmon-1.c sounds
> odd.
> TIA,
> 

Fixed that with a commit I've just pushed.

Martin

Re: GCC documentation: porting to Sphinx

[Martin Liška]

On 6/4/21 5:10 PM, Martin Sebor wrote:
> On 6/3/21 4:56 AM, Martin Liška wrote:
>> On 6/2/21 10:41 PM, Martin Sebor wrote:
>>> On 5/31/21 7:25 AM, Martin Liška wrote:
>>>> Hello.
>>>>
>>>> I've made quite some progress with the porting of the documentation and
>>>> I would like to present it to the community now:
>>>> https://splichal.eu/scripts/sphinx/
>>>
>>
>> Hello.
>>
>> Thank you for the review.
>>
>>> Just a few issues I noticed in the warnings section:
>>>
>>> The headings of some warnings mention the same option twice (e.g.,
>>> -Wabi, -Wabi, -Wno-abi;  -Wdouble-promotion, -Wdouble-promotion,
>>> -Wno-double-promotion;  -Winit-self, -Winit-self, -Wno-init-self).
>>> This looks like a pretty pervasive problem.
>>
>> You are right, I fixed that.
> 
> Looks good.
> 
>>
>>>
>>> Mentioning the -Wno-xxx option is redundant in a heading for -Wxxx.
>>
>> Yes. Good reason for that is that Sphinx can then generated properly links
>> to the current non-documented version of the option. Hope it's improvement
>> over the current situation?

Hello.

Back to this after some thinking.

> 
> I think the linking is helpful.  But for warnings, the documented
> convention is to only mention the one that's not the default:
> 
>    This manual lists only one of the two forms, whichever is not
>    the default.
> 
> so including both blurs this (IMO rather subtle) distinction.
> In addition, in options whose description says something like
> "This warning is enabled by -Wall." it's now less clear which
> one is the one the "this" refers to (see for example
> -Wchar-subscripts).

Yes, that's really confusing and we should likely explicitly document
both options as shown here (-Wno-shift-overflow3):

https://splichal.eu/scripts/sphinx/demo/_build/html/#cmdoption-Wno-shift-overflow3

Doing that, one has 2 unique links, that would be needed for get_option_url function.
Plus, both :option:`-Wfoo` and :option:`-Wno-foo` references are going to work.

> 
> If the heading can't be changed at a minimum we'll need to update
> the convention above, e.g., by saying that the first option mentions
> is the default. But again, I think this is too subtle for the casual
> reader to pick up on.  The fact that the sentence quoted above appears
> under -Wfatal-errors doesn't help.  We should also work on updating
> the "This option is in -Wall." either to name the specific option
> it refers to, or consider moving that into a Note box like the one
> listing the languages the option applies to.)

Yes, we should explicitly mark one of them as default value,
something like "Default option value for -Wshift-overflow3."?
And the corresponding counter-part should have "Enabled by -Wall.". I'm not fully
convinced about usage of note as it's quite big visual component.

> 
>>
>>>
>>> The headings of some other warnings also mention options that are
>>> only remotely related to them.  E.g., -Wformat has all these:
>>>
>>>    -Wformat, -Wno-format, -ffreestanding, -fno-builtin, -Wformat=
>>>
>>> (I see the same problem in the attributes section where the headings
>>> for some attributes include option names).
>>>
>>> That seems quite puzzling.  I assume it's a consequence of having
>>> index entries for the related options, but I don't think making
>>> them visible in the headings is helfpful.
>>
>> Oh, you are right. It was consequence of wrong parsing of index entries.
>> It should be fixed now.
> 
> Looks good.
> 
>>
>>>
>>> Headings that in the manual today include a level like
>>>
>>>    -Wformat-overflow
>>>    -Wformat-overflow=level
>>>
>>> don't mention the level in the Spinx manual:
>>>
>>>    -Wformat-overflow, -Wno-format-overflow
>>>
>>> When the /level/ is then discussed in the rest of the text it's
>>> not clear what it refers to.
>>
>> Should be also fixed now.
> 
> Also looks good.
> 
>>
>> Can you please take a look at the current output and give me a feedback?
> 
> I noticed another minor issue that may already have been pointed
> out by someone else.  Under -Wall (and -Wextra), some option names
> are prefixed by :option: (e.g., (only with :option:-O2``).  Looks
> like some sort of a transcription bug?

Yes, that should be fixed now. Most comment root cause is that some inline
roles are wrapped (that's not supported).

> 
> And a couple of questions:
> 
> References to options with an argument like -Warray-bounds=1 are
> rendered in a way that makes it look like there's a space before
> the equals: -Warray-bounds =1, with  the =1 being in a different
> color and not part of the hyperlink. Is there a way to make it look
> like there is no space?

Likely not. That's the best I was able to come up with. Reason why the '=1'
argument is in :samp: is that :option:`-Wfoo=123` does not properly generate link
to the option directive. And there's actually no space in HTML, it's only slightly
visually separated and PDF version is fine for me.

> 
> I like how options are automatically linked, and I'd like to see
> the same for other references like to attributes.  Can that be
> automated as part of the migration or should we/I try to tackle
> it in a followup?

Right now, I abuse a bit .. option:: my_attribute directive for them.
For the future, yes, one can make a link to them with :option:`my_attribute`.

Thanks for useful feedback!
Martin

> 
> In any event, thanks for working so hard on making this turn out
> great!
> 
> Martin
> 
>> Thanks,
>> Martin
>>
>>>
>>> Martin
>>>
>>>>
>>>> Note the documentation is automatically ([1]) generated from texinfo with a GitHub workflow ([2]).
>>>> It's built on the devel/sphinx GCC branch which I periodically with the master branch. One can
>>>> see the current source .rst files here: [3].
>>>>
>>>> Changes made since the last time:
>>>> - a shared content is factored out ([4])
>>>> - conditional build is fully supported (even for shared parts)
>>>> - manual pages look reasonable well
>>>> - folders are created for files which have >= 5 TOC tree entries
>>>> - various formatting issues were resolved
>>>> - baseconf.py reads BASE-VER, DEV-PHASE, .. files
>>>>
>>>> I've got couple of questions:
>>>>
>>>> 1) Do we have to you the following cover text?
>>>>         Copyright (c) 1988-2020 Free Software Foundation, Inc.
>>>>
>>>>         Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation; with the Invariant Sections being "GNU General Public
>>>>         License" and "Funding Free Software", the Front-Cover texts being (a) (see below), and with the Back-Cover Texts being (b) (see below).  A copy of the license is included in the gfdl(7) man page.
>>>>
>>>>         (a) The FSF's Front-Cover Text is:
>>>>
>>>>              A GNU Manual
>>>>
>>>>         (b) The FSF's Back-Cover Text is:
>>>>
>>>>              You have freedom to copy and modify this GNU Manual, like GNU
>>>>              software.  Copies published by the Free Software Foundation raise
>>>>              funds for GNU development.
>>>>
>>>> 2) Do we want to generate fsf-funding, gpl and gfdl manual pages?
>>>> 3) Do we want to preserve the current strange copy mechanism for ./gcc/doc/tm.texi.in ?
>>>> 4) Do we want a copyright header for the created .rst files?
>>>>
>>>> Thoughts?
>>>> Thanks,
>>>> Martin
>>>>
>>>> [1] https://github.com/davidmalcolm/texi2rst
>>>> [2] https://github.com/davidmalcolm/texi2rst/actions
>>>> [3] https://github.com/marxin/texi2rst-generated/tree/master/sphinx
>>>> [4] https://github.com/marxin/texi2rst-generated/tree/master/sphinx/share
>>>
>>
> 

Re: RFC: Sphinx for GCC documentation

[Martin Liška]

On 6/7/21 4:19 PM, Tobias Burnus wrote:
> 
> On 07.06.21 15:28, Martin Liška wrote:
>>> * I note that we write before the argument index, that those are
>>> without -/-- prefix
>>>    but that's not true. Something to fix after the conversation.
>>
>> Can you please show me a few examples of it?
> 
> * https://splichal.eu/scripts/sphinx/gfortran/_build/html/option-index.html
> 
> * https://splichal.eu/scripts/sphinx/gcc/_build/html/option-index.html
> 
> * https://splichal.eu/scripts/sphinx/cpp/_build/html/option-index.html
> 
> Read the text on those page – and then look at the converted index,
> which has all the '-' and '--' before the options.

You are right, I updated the Option Index pages.

Martin

> 
> That's not a conversion bug – a change which needs to be done as part of
> or after the conversion.
> 
> Tobias
> 
> -----------------
> Mentor Graphics (Deutschland) GmbH, Arnulfstrasse 201, 80634 München Registergericht München HRB 106955, Geschäftsführer: Thomas Heurung, Frank Thürauf

Re: GCC documentation: porting to Sphinx

[Martin Liška]

On 6/10/21 11:07 AM, Martin Liška wrote:
> Doing that, one has 2 unique links, that would be needed for get_option_url function.
> Plus, both :option:`-Wfoo` and :option:`-Wno-foo` references are going to work.

And I've actually did the transformation and one can see it e.g. here:
https://splichal.eu/scripts/sphinx/gcc/_build/html/gcc-command-options/options-to-request-or-suppress-warnings.html#cmdoption-Wprio-ctor-dtor

Martin

Re: GCC documentation: porting to Sphinx

[Martin Liška]

On 6/2/21 7:27 PM, Joseph Myers wrote:
> On Mon, 31 May 2021, Martin Liška wrote:
> 
>> https://splichal.eu/scripts/sphinx/
> 
> Looking at some examples there:
> 
> https://splichal.eu/scripts/sphinx/gcc/_build/html/c-implementation-defined-behavior/preprocessing-directives.html
> has some conversion problems:
> 
> * "See Implementation-defined behavior, for details of these aspects of
> implementation-defined behavior." is missing the link to the relevant
> section of the cpp manual that's present in the Texinfo source.

Yes, I'm aware of various cross-manual links that are currently not working.
It will likely require an extension called Intersphinx:
https://www.sphinx-doc.org/en/master/usage/extensions/intersphinx.html

> 
> * "` character before the :samp:`" is a misconversion (whether from
> Texinfo to RST or from RST to HTML) of the Texinfo source
> 
>    @samp{\} character before the @samp{\}
> 
> which will need to be fixed.

Yes, I fixed various :samp:, :option: leftovers all over the documentation.

> 
> * The corresponding PDF has the same issues as above (so probably they are
> issues with the conversion to RST, not with Sphinx itself).  In addition,
> the PDF manual ought to be using fixed-width fonts for literal code,
> command-line options, etc., just like the HTML manual, and the
> Texinfo-generated PDF manual, are.

Hm, I think the generated PDF properly uses a fixed-width font for option names,
commands and so one. Moreover, option directives are bold, while links to them
use normal font. I see the default font selection made by Sphinx readable.

> 
> https://splichal.eu/scripts/sphinx/gcc/_build/html/gcc-command-options/passing-options-to-the-assembler.html
> shows headings such as "-Wa,option, -Wa".  The ", -Wa" doesn't make sense,
> this option is just "-Wa,option".

I fixed various these issues.

> 
> https://splichal.eu/scripts/sphinx/gcc/_build/html/gcov-a-test-coverage-program.html
> has a hyphen between "gcov" and "a Test Coverage Program" in the heading.
> It should be an em dash, as in Texinfo.

Oh yeah. Apparently, we can use "smart quotes" (-- and ---) for dashes:
https://www.sphinx-doc.org/en/master/usage/configuration.html#confval-smartquotes

Fixed that in the current version.

> 
> https://splichal.eu/scripts/sphinx/gcc/_build/html/language-standards-supported-by-gcc/c%2B%2B-language.html
> has doubled slashes in various URLs where the Texinfo source has /@/
> (Texinfo @/ means "allow line break", it should not be translated to /).

Good point, also fixed.

> 
> https://splichal.eu/scripts/sphinx/gcc/_build/html/gcc-command-options/machine-dependent-options/aarch64-options.html
> shows different formatting for the headings for "-mlow-precision-div,
> -mno-low-precision-div" and "-mtrack-speculation -mno-track-speculation".
> The formatting should be identical.  The only difference in the Texinfo
> source seems to be that the latter is missing @opindex directives.  And
> while it's a bug in the Texinfo source that those directives are missing,
> the presence or absence of index entries should not affect the formatting
> of the documentation for those options.

As discussed with Martin Sebor, I emitted non-default option directive.

> 
> On that same page, the output for -march=name is broken, containing a
> literal :samp:{feature} (in general, checking for any places where RST
> directives such as :samp: appear in the HTML output might be a good idea
> to look for broken conversions).  The Texinfo source here has
> 
> @option{-march=@var{arch}@r{@{}+@r{[}no@r{]}@var{feature}@r{@}*}}
> 
> (where the use of @r{...} is to put the {}[]* characters in a
> variable-width font, since they are not literally part of the option,
> while the other characters that are literally part of the option should be
> in a variable-width font).

Also fixed.

> 
> https://splichal.eu/scripts/sphinx/gcc/_build/html/language-standards-supported-by-gcc/references-for-other-languages.html
> has literal unconverted "@c man" and "@include" and other Texinfo
> directives.  Searching for such things in the HTML output (or the RST
> sources) is a good idea, just like searching for literal RST directives in
> the HTML output, to find other such conversion bugs.

Clean up these.

> 
> https://splichal.eu/scripts/sphinx/gcc/_build/html/gcc-command-options.html
> says "See option-index", another case with a link that didn't get
> converted properly.  It also has raw :samp: uses indicating a
> misconversion.
> 
> I'm not sure how you're determining languages for code-block, but
> https://splichal.eu/scripts/sphinx/gcc/_build/html/gcc-command-options/options-to-control-diagnostic-messages-formatting.html
> certainly shows some cases where they have been misidentified (e.g. random
> C++ keywords highlighted in the default GCC_COLORS, some JSON being
> highlighted as such but other JSON not).

I fixed all code-block warnings. Some of JSON syntax highlighting was not working because
the JSON syntax was invalid. Should be fine:
https://splichal.eu/scripts/sphinx/gcc/_build/html/gcc-command-options/options-to-control-diagnostic-messages-formatting.html#cmdoption-fdiagnostics-format

> 
>> - a shared content is factored out ([4])
>> - conditional build is fully supported (even for shared parts)
>> - manual pages look reasonable well
>> - folders are created for files which have >= 5 TOC tree entries
>> - various formatting issues were resolved
>> - baseconf.py reads BASE-VER, DEV-PHASE, .. files
> 
> Could you give more detailed descriptions of how each of the various
> issues I listed in 2015 are addressed here?

Sure:

1) documentation fragments in target.def
I've got a script that rewrites them to RST format and we'll use the slightly modified
gcc/genhooks.c for replacement.

2) comments
Are preserved by the conversion tool.

3) man pages support and @ignore
That's done, we have shared content and conditional build (..only directive).

4) support for BASE-VER, DATESTAMP, DEV-PHASE

It's read right now in baseconf.py, we need to add support for bugurl and package_version,
it won't be difficult.

> 
> https://gcc.gnu.org/legacy-ml/gcc-patches/2015-11/msg01139.html
> 
>> I've got couple of questions:
>>
>> 1) Do we have to you the following cover text?
>>         Copyright (c) 1988-2020 Free Software Foundation, Inc.
>>
>>         Permission is granted to copy, distribute and/or modify this document
>> under the terms of the GNU Free Documentation License, Version 1.3 or any
>> later version published by the Free Software Foundation; with the Invariant
>> Sections being "GNU General Public
>>         License" and "Funding Free Software", the Front-Cover texts being (a)
>> (see below), and with the Back-Cover Texts being (b) (see below).  A copy of
>> the license is included in the gfdl(7) man page.
>>
>>         (a) The FSF's Front-Cover Text is:
>>
>>              A GNU Manual
>>
>>         (b) The FSF's Back-Cover Text is:
>>
>>              You have freedom to copy and modify this GNU Manual, like GNU
>>              software.  Copies published by the Free Software Foundation raise
>>              funds for GNU development.
> 
> We need to keep the Cover Texts and Invariant Sections, in the absence of
> FSF approval to remove them.

Added that to copyright.rst file.

> 
>> 2) Do we want to generate fsf-funding, gpl and gfdl manual pages?
> 
> Yes, this is how the set of man pages as a whole keeps the invariant
> sections.

Done.

> 
>> 3) Do we want to preserve the current strange copy mechanism for
>> ./gcc/doc/tm.texi.in ?
> 
> Yes, this is how we ensure we have both GPL and GFDL copies of the target
> hook documentation checked in and that someone copying from one place to
> another makes sure they have any relevant permissions.

As mentioned, it will be supported.

> 
>> 4) Do we want a copyright header for the created .rst files?
> 
> Yes, all source files should have a copyright header.
> 

Done.

@Joseph: May I ask you for another round of review? The generated pages are quite fine,
I addressed various smaller issues. Hopefully we are quite close to something that can
be send to gcc-patches.

Another set of questions I have:

1) Can we organize the new documentation in $gccroot/doc folder similarly to what
I have in texi2rst-generated repo? Would be beneficial as we can have a single Makefile
and shared content will be in a same depth to the individual manuals.

2) About libiberty - who's in charge of the library? Is it GCC community? I'm asking
if we want to migrate to Sphinx as well?

Thanks,
Martin

Re: GCC documentation: porting to Sphinx

[Joseph Myers]

On Thu, 10 Jun 2021, Martin Liška wrote:

> 1) Can we organize the new documentation in $gccroot/doc folder 
> similarly to what I have in texi2rst-generated repo? Would be beneficial 
> as we can have a single Makefile and shared content will be in a same 
> depth to the individual manuals.

Where languages have their own manuals, I think it's more appropriate for 
those to go under the language-specific directories.

That doesn't stop the use of shared makefile code.  Make-lang.in is a 
fragment included from gcc/Makefile.in ("-include $(LANG_MAKEFRAGS)").  I 
certainly expect it should be possible to write GNU make code in 
gcc/Makefile.in for building and installing manuals, such that 
subdirectories only need to define a few variables describing what manuals 
they have and everything else is handled by common code.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: GCC documentation: porting to Sphinx

[Martin Sebor]

On 6/10/21 7:18 AM, Martin Liška wrote:
> On 6/10/21 11:07 AM, Martin Liška wrote:
>> Doing that, one has 2 unique links, that would be needed for get_option_url function. 
>>
>> Plus, both :option:`-Wfoo` and :option:`-Wno-foo` references are going to work. 
>>
> 
> And I've actually did the transformation and one can see it e.g. here:
> https://splichal.eu/scripts/sphinx/gcc/_build/html/gcc-command-options/options-to-request-or-suppress-warnings.html#cmdoption-Wprio-ctor-dtor 

I find the style you have below right now clearer than keeping both
options in the same heading and adding a Note explaining the default
etc.  I.e., this

   _________________
   -Wchar-subscripts

     Warn if an array subscript has type char. This is a common cause
     of error, as programmers often forget that this type is signed on
     some machines. This warning is enabled by -Wall.

   ____________________
   -Wno-char-subscripts

    Default option value for -Wchar-subscripts.

on this page right now:
https://splichal.eu/scripts/sphinx/gcc/_build/html/gcc-command-options/options-to-request-or-suppress-warnings.html#cmdoption-Wchar-subscripts 


seems better than this:

   _________________________________________________________
   -Wno-shift-overflow, -Wshift-overflow=n, -Wshift-overflow

     +--------------------------------------------------------------+
     | (!) Note                                                     |
     +---------------------------------------------------------------
     | Default value is -Wno-shift-overflow, -Wshift-overflow is    |
     | enabled by -Wall.                                            |
     +--------------------------------------------------------------+

     These options control warnings about left shift overflows.

and also better than the alternative with (non-default) after the option
in the heading.

https://splichal.eu/scripts/sphinx/demo/_build/html/#cmdoption-Wno-shift-overflow3

Martin

> 
> 
> Martin

Re: GCC documentation: porting to Sphinx

[Martin Liška]

On 6/10/21 6:49 PM, Joseph Myers wrote:
> On Thu, 10 Jun 2021, Martin Liška wrote:
> 
>> 1) Can we organize the new documentation in $gccroot/doc folder
>> similarly to what I have in texi2rst-generated repo? Would be beneficial
>> as we can have a single Makefile and shared content will be in a same
>> depth to the individual manuals.
> 
> Where languages have their own manuals, I think it's more appropriate for
> those to go under the language-specific directories.

So it will require the following folder structure:

$gccroot/gcc/doc/gcc - for GCC documentation
$gccroot/gcc/doc/gccint - for GCC internal documentation
$gccroot/gcc/doc/gfortran - for Fortran documentation
$gccroot/gcc/doc/gccgo - for GO documentation
...
$gccroot/doc/share - shared components
$gccroot/libgomp/doc - for libgomp documentation
...

Are you fine with that?

> 
> That doesn't stop the use of shared makefile code.  Make-lang.in is a
> fragment included from gcc/Makefile.in ("-include $(LANG_MAKEFRAGS)").  I
> certainly expect it should be possible to write GNU make code in
> gcc/Makefile.in for building and installing manuals, such that
> subdirectories only need to define a few variables describing what manuals
> they have and everything else is handled by common code.
> 

The Sphinx Makefile will be capable of e.g.
make html -C $gccroot/gcc/doc/gcc BUILDDIR=`pwd/put_it_somewhere`

and the only configure dependency will VERSION_PACKAGE and BUGURL which will
be provided in env:
https://github.com/marxin/texi2rst-generated/blob/6cfcb7b8ae6497d49ea23a38262dfa26854bdb40/sphinx/baseconf.py#L38-L39

Martin

Re: GCC documentation: porting to Sphinx

[Martin Liška]

On 6/11/21 1:48 AM, Martin Sebor wrote:
> On 6/10/21 7:18 AM, Martin Liška wrote:
>> On 6/10/21 11:07 AM, Martin Liška wrote:
>>> Doing that, one has 2 unique links, that would be needed for get_option_url function.
>>> Plus, both :option:`-Wfoo` and :option:`-Wno-foo` references are going to work.
>>
>> And I've actually did the transformation and one can see it e.g. here:
>> https://splichal.eu/scripts/sphinx/gcc/_build/html/gcc-command-options/options-to-request-or-suppress-warnings.html#cmdoption-Wprio-ctor-dtor 
> 
> I find the style you have below right now clearer than keeping both
> options in the same heading and adding a Note explaining the default
> etc.  I.e., this
> 
>    _________________
>    -Wchar-subscripts
> 
>      Warn if an array subscript has type char. This is a common cause
>      of error, as programmers often forget that this type is signed on
>      some machines. This warning is enabled by -Wall.
> 
>    ____________________
>    -Wno-char-subscripts
> 
>     Default option value for -Wchar-subscripts.

Yes, fully agree with you!

Thanks,
Martin

> 
> on this page right now:
> https://splichal.eu/scripts/sphinx/gcc/_build/html/gcc-command-options/options-to-request-or-suppress-warnings.html#cmdoption-Wchar-subscripts
> 
> seems better than this:
> 
>    _________________________________________________________
>    -Wno-shift-overflow, -Wshift-overflow=n, -Wshift-overflow
> 
>      +--------------------------------------------------------------+
>      | (!) Note                                                     |
>      +---------------------------------------------------------------
>      | Default value is -Wno-shift-overflow, -Wshift-overflow is    |
>      | enabled by -Wall.                                            |
>      +--------------------------------------------------------------+
> 
>      These options control warnings about left shift overflows.
> 
> and also better than the alternative with (non-default) after the option
> in the heading.
> 
> https://splichal.eu/scripts/sphinx/demo/_build/html/#cmdoption-Wno-shift-overflow3
> 
> Martin
> 
>>
>>
>> Martin
> 

Re: GCC documentation: porting to Sphinx

[Joseph Myers]

On Fri, 11 Jun 2021, Martin Liška wrote:

> > Where languages have their own manuals, I think it's more appropriate for
> > those to go under the language-specific directories.
> 
> So it will require the following folder structure:
> 
> $gccroot/gcc/doc/gcc - for GCC documentation
> $gccroot/gcc/doc/gccint - for GCC internal documentation
> $gccroot/gcc/doc/gfortran - for Fortran documentation
> $gccroot/gcc/doc/gccgo - for GO documentation

I'm thinking of

$gccroot/gcc/fortran/doc
$gccroot/gcc/go/doc

(or subdirectories thereof if desired) for the Fortran and Go manuals, so 
they go alongside the front end sources.

> The Sphinx Makefile will be capable of e.g.

My concern with makefiles is what the main GCC build system does, with 
"make" run at the top level of the build tree and with the targets defined 
by the GNU Coding Standards, not with what happens if someone manually 
does make in a subdirectory of the source or build tree.

"make" at top level should build all the info manuals and man pages, as at 
present (if a suitable Sphinx version is installed), and "make install" 
should install them, in the same directories as at present.

"make html" at top level should build all the HTML manuals, and "make 
install-html" should install them.

"make pdf" and "make install-pdf" at top level should work likewise.

"make install-html" and "make install-pdf" should put things under 
$(DESTDIR)$(htmldir) and $(DESTDIR)$(pdfdir) as at present.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: GCC documentation: porting to Sphinx

[Koning, Paul]

> On Jun 11, 2021, at 11:50 AM, Joseph Myers <joseph@codesourcery.com> wrote:
> 
> ...
> 
> "make" at top level should build all the info manuals and man pages, as at 
> present (if a suitable Sphinx version is installed), and "make install" 
> should install them, in the same directories as at present.
> 
> "make html" at top level should build all the HTML manuals, and "make 
> install-html" should install them.
> 
> "make pdf" and "make install-pdf" at top level should work likewise.
> 
> "make install-html" and "make install-pdf" should put things under 
> $(DESTDIR)$(htmldir) and $(DESTDIR)$(pdfdir) as at present.

And in addition, it would be nice to have additional make <foo> and make install-<foo> targets for other output formats that Sphinx can generate for us, at least some of them.  "epub" comes to mind as an example I would like to have.

	paul

Re: GCC documentation: porting to Sphinx

[Martin Liška]

Hello.

I've just made a first version of the patchset that sits in GCC source tree:
https://gcc.gnu.org/git/gitweb.cgi?p=gcc.git;a=log;h=refs/users/marxin/heads/sphinx-v2

Changes since the last submission:
1) I made a brief proofreading and fixed most of the formatting and other issues
2) target hook documentation was ported to RST format and gcc/genhooks generated a tm.rst.in
    that is then used via .. include:: directives
3) sphinx-build detection is implemented in gcc/configure.ac
3) make integration is done (currently covering info and man pages) for gcc and gcc/fortran targets

As before, one can see the result of generated output here:
https://splichal.eu/scripts/sphinx/

Known limitations:
1) I found a bug in man page generation that is currently fixed:
https://github.com/sphinx-doc/sphinx/issues/1860#issuecomment-861793094
Note the fix will appear in the upcoming 4.1.0 release. Without the patch, one can see wrong
font selection in a generated manual page

2) Right now, I rely on caching capability of sphinx-build. That means when no source change is detected,
sphinx-build exits immediately. However, it's not working for all targets (info, man) and I've suggested
a patch for it:
https://github.com/sphinx-doc/sphinx/issues/9359

3) I haven't prepared patch for .texi removal (should be quite easily doable).

4) the currently existing Sphinx manuals (Ada and libgccjit) are not integrated yet.

@Joseph: Can you share your thoughts about the used Makefile integration? What do you suggest for 2)
(note that explicit listing of all .rst file would be crazy)?

Thoughts?
Thanks,
Martin

Re: GCC documentation: porting to Sphinx

[Joseph Myers]

On Wed, 23 Jun 2021, Martin Liška wrote:

> @Joseph: Can you share your thoughts about the used Makefile integration? What
> do you suggest for 2)
> (note that explicit listing of all .rst file would be crazy)?

You can write dependencies on e.g. doc/gcc/*.rst (which might be more 
files than actually are relevant in some cases, if the directory includes 
some common files shared by some but not all manuals, but should be 
conservatively safe if you list appropriate directories there), rather 
than needing to name all the individual files.  Doing things with makefile 
dependencies seems better than relying on what sphinx-build does when 
rerun unnecessarily (if sphinx-build avoids rebuilding in some cases where 
the makefiles think a rebuild is needed, that's fine as an optimization).

It looks like this makefile integration loses some of the srcinfo / srcman 
support.  That support should stay (be updated for the use of Sphinx) so 
that release tarballs (as generated by maintainer-scripts/gcc_release, 
which uses --enable-generated-files-in-srcdir) continue to include man 
pages / info files (and make sure that, if those files are present in the 
source directory, then building and installing GCC does install them even 
when sphinx-build is absent at build/install time).

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: GCC documentation: porting to Sphinx

[Martin Liška]

On 6/23/21 6:00 PM, Joseph Myers wrote:
> On Wed, 23 Jun 2021, Martin Liška wrote:
> 
>> @Joseph: Can you share your thoughts about the used Makefile integration? What
>> do you suggest for 2)
>> (note that explicit listing of all .rst file would be crazy)?
> 
> You can write dependencies on e.g. doc/gcc/*.rst (which might be more
> files than actually are relevant in some cases, if the directory includes
> some common files shared by some but not all manuals, but should be
> conservatively safe if you list appropriate directories there), rather
> than needing to name all the individual files.  Doing things with makefile
> dependencies seems better than relying on what sphinx-build does when
> rerun unnecessarily (if sphinx-build avoids rebuilding in some cases where
> the makefiles think a rebuild is needed, that's fine as an optimization).

All right. I've just done that and it was easier than I expected. Now the dependencies
are properly followed.

> 
> It looks like this makefile integration loses some of the srcinfo / srcman
> support.  That support should stay (be updated for the use of Sphinx) so
> that release tarballs (as generated by maintainer-scripts/gcc_release,
> which uses --enable-generated-files-in-srcdir) continue to include man
> pages / info files (and make sure that, if those files are present in the
> source directory, then building and installing GCC does install them even
> when sphinx-build is absent at build/install time).
> 

Oh, and I've just recovered this one as well. Pushed changes to the me/sphinx-v2
branch and I'm waiting for more feedback.

In the meantime, I'm going to prepare further integration of other manuals and
targets (PDF, HTML).

Martin

Re: GCC documentation: porting to Sphinx

[Martin Liška]

Hello.

I've got something that is very close to be a patch candidate that can be
eventually merged. Right now, the patches are available here:
https://gcc.gnu.org/git/gitweb.cgi?p=gcc.git;a=log;h=refs/users/marxin/heads/sphinx-v3

Changes since last version:

- gdc manual was ported
- 'make doc' works fine both with and w/o installed sphinx-build
- 'make pdf' and 'make html' works fine
- libgccjit was ported to the shared Makefile and configuration
- likewise for 3 existing Ada manuals
- .texi files are removed

List of known issues (planned to be fixed after merging):
- cross manual references are not working
- update_web_docs_git needs to be changed - will simplify rapidly
- Sphinx warnings should be addressed
- remove texinfo references in Manuals
- list package requirements for Sphinx manual generation

I'm looking forward to a feedback.
Thanks,
Martin

Re: GCC documentation: porting to Sphinx

[Martin Liška]

On 6/25/21 3:11 PM, Martin Liška wrote:
> List of known issues (planned to be fixed after merging):

I forgot about:

- diagnostics URL (for e.g. warnings) needs to be adjusted

Martin

Re: GCC documentation: porting to Sphinx

[Arnaud Charlet]

> I've got something that is very close to be a patch candidate that can be
> eventually merged. Right now, the patches are available here:
> https://gcc.gnu.org/git/gitweb.cgi?p=gcc.git;a=log;h=refs/users/marxin/heads/sphinx-v3

FWIW I would prefer to review the changes posted here directly with all
the details.

In particular can you explain the motivation behind all the changes in the
gcc/ada/doc directory? That's lots of moving files around, so I'd like
to understand why and make sure these are not gratuituous changes, since
move files around is always tricky and I'd rather not have to undo it
later in case this causes troubles or have unexpected consequences.

Otherwise, glad to see the switch to sphinx finally moving in gcc!

Arno

Re: GCC documentation: porting to Sphinx

[Martin Liška]

On 6/28/21 12:23 PM, Arnaud Charlet wrote:
>> I've got something that is very close to be a patch candidate that can be
>> eventually merged. Right now, the patches are available here:
>> https://gcc.gnu.org/git/gitweb.cgi?p=gcc.git;a=log;h=refs/users/marxin/heads/sphinx-v3
> 
> FWIW I would prefer to review the changes posted here directly with all
> the details.

Sure, I'm going to send a proper patch set in an hour or so. As mentioned, I won't be able to attach
some of the patches as they will exceed 1MB email limit.

> 
> In particular can you explain the motivation behind all the changes in the
> gcc/ada/doc directory?

Sure:
1) All Sphinx manuals live in a directory where index page is called index.rst. That's why
I moved e.g. this: gcc/ada/doc/{gnat_rm.rst => gnat_rm/index.rst}
2) I moved latex_elements.py to ada_latex_elements.py as it clashes with Sphinx global variable
you modify in Sphinx config files
3) I created a shared Ada config (adabaseconf.py) that extends doc/baseconf.py and sets what
is shared in between 3 Ada manuals.
4) gnu_free_documentation_license.rst is taken from $root/doc/

> That's lots of moving files around, so I'd like
> to understand why and make sure these are not gratuituous changes, since
> move files around is always tricky and I'd rather not have to undo it
> later in case this causes troubles or have unexpected consequences.

Hope I explained all the reasonable changes?

> 
> Otherwise, glad to see the switch to sphinx finally moving in gcc!

You're welcome. I would be interested in testing your PRO configuration (based on Gnat_Build_Type,
see get_gnat_build_type) and I'm curious if you're fine with Sphinx template change?
It will be the same as for other manuals.

Cheers,
Martin

> 
> Arno
> 

[PATCH] Port GCC documentation to Sphinx

[Martin Liška]

Hello.

I'm sending the complete patch set that includes ChangeLog entries. Unfortunately,
majority of the patches are huge, that's why I sent like to a tarball:
https://splichal.eu/tmp/port-to-sphinx-v1.tar

The tarball contains the following patches:

19e06194746 Ada: port to Sphinx.
9a744ca431d Remove unused TEX files.
e624967b5e8 Port jit to new Sphinx layout.
8c4717b262a Build system: support Sphinx
d102880437e Add include directives for target macros.
08c3d3f0d8d Add RST files with config files.

Thanks,
Martin

Re: [PATCH] Port GCC documentation to Sphinx

[Joseph Myers]

Are formatted manuals (HTML, PDF, man, info) corresponding to this patch 
version also available for review?

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: [PATCH] Port GCC documentation to Sphinx

[Martin Liška]

On 6/28/21 5:33 PM, Joseph Myers wrote:
> Are formatted manuals (HTML, PDF, man, info) corresponding to this patch
> version also available for review?

I've just uploaded them here:
https://splichal.eu/gccsphinx-final/

Martin

Re: [PATCH] Port GCC documentation to Sphinx

[Richard Earnshaw]

On 29/06/2021 11:09, Martin Liška wrote:
> On 6/28/21 5:33 PM, Joseph Myers wrote:
>> Are formatted manuals (HTML, PDF, man, info) corresponding to this patch
>> version also available for review?
> 
> I've just uploaded them here:
> https://splichal.eu/gccsphinx-final/
> 
> Martin
> 

In the HTML version of the gcc manual the sidebar has an "Option index" 
link but no link to the general index.  When you follow that link the 
page contents is just a link to the "index" where everything is all 
lumped together.

If we can't have separate indexes for options and general entries, I 
think it would make more sense for the Option index link to be removed 
entirely.

R.

Re: GCC documentation: porting to Sphinx

[Arnaud Charlet]

> >In particular can you explain the motivation behind all the changes in the
> >gcc/ada/doc directory?
> 
> Sure:
> 1) All Sphinx manuals live in a directory where index page is called index.rst. That's why
> I moved e.g. this: gcc/ada/doc/{gnat_rm.rst => gnat_rm/index.rst}
> 2) I moved latex_elements.py to ada_latex_elements.py as it clashes with Sphinx global variable
> you modify in Sphinx config files
> 3) I created a shared Ada config (adabaseconf.py) that extends doc/baseconf.py and sets what
> is shared in between 3 Ada manuals.
> 4) gnu_free_documentation_license.rst is taken from $root/doc/

OK, this is really lots of changes, if we could minimize these changes
that would be best (and sorry but posting a link to a tarball also doesn't
help reviews, it was actually better with a link to a git repo previously...
At least the Ada part itself shouldn't be too big in particular once
simplified so could be posted standalone).

Arno

Re: [PATCH] Port GCC documentation to Sphinx

[Eli Zaretskii]

> From: Martin Liška <mliska@suse.cz>
> Date: Tue, 29 Jun 2021 12:09:23 +0200
> Cc: GCC Development <gcc@gcc.gnu.org>, gcc-patches@gcc.gnu.org
> 
> On 6/28/21 5:33 PM, Joseph Myers wrote:
> > Are formatted manuals (HTML, PDF, man, info) corresponding to this patch
> > version also available for review?
> 
> I've just uploaded them here:
> https://splichal.eu/gccsphinx-final/

Thanks.

I'm an Info junkie, so I grabbed gcc.info from there and skimmed
through it.  Please allow me a few unsolicited comments:

1. It sounds like Sphinx is heavily biased towards HTML format, and as
   result uglifies the Info format?

For example, many cross-references (AFAIU introduced as part of
migration to Sphinx) make the text illegible in Emacs.  Example:

  This standard, in both its forms, is commonly known as `C89', or
  occasionally as `C90', from the dates of ratification.  To select this
  standard in GCC, use one of the options *note -ansi *note -std
  .‘=c90’ or *note -std.‘=iso9899:1990’; to obtain all the diagnostics
  required by the standard, you should also specify *note -pedantic.
  (or *note -pedantic-errors. if you want them to be errors rather
  than warnings).  See *note Options Controlling C Dialect.
  [...]
  An amendment to the 1990 standard was published in 1995.  This amendment
  added digraphs and ‘__STDC_VERSION__’ to the language, but otherwise
  concerned the library.  This amendment is commonly known as `AMD1'; the
  amended standard is sometimes known as `C94' or `C95'.  To select this
  standard in GCC, use the option *note -std.‘=iso9899:199409’ (with,
  as for other standard versions, *note -pedantic. to receive all
  required diagnostics).

Or how about this:

  `Overall Options'

       See Options Controlling the Kind of Output.

       *note -c. *note -S. *note -E. *note -o. ‘`file'’
       *note -dumpbase. ‘`dumpbase'’ *note -dumpbase-ext.
       ‘`auxdropsuf'’ *note -dumpdir. ‘`dumppfx'’ ‘-x’ ‘`language'’
       *note -v. *note -###. *note –help.‘[=`class'[,...]]’
       *note –target-help. *note –version. *note -pass-exit-codes
       . *note -pipe. *note -specs.‘=`file'’ *note -wrapper
       .‘@`file'’ *note -ffile-prefix-map.‘=`old'=`new'’ *note
       -fplugin.‘=`file'’ ‘-fplugin-arg-’‘`name'=`arg'’
       ‘-fdump-ada-spec’‘[-`slim']’ *note -fada-spec-parent.‘=`unit'’
       *note -fdump-go-spec.‘=`file'’

In the first line, the emphasis became quotes, which sounds sub-optimal.
In the second line, the hyperlink was lost.
And the rest is not really readable.

Compare this with the original:

  _Overall Options_
       *Note Options Controlling the Kind of Output.
	    -c  -S  -E  -o FILE  -x LANGUAGE
	    -v  -###  --help[=CLASS[,...]]  --target-help  --version
	    -pass-exit-codes  -pipe  -specs=FILE  -wrapper
	    @FILE  -ffile-prefix-map=OLD=NEW
	    -fplugin=FILE  -fplugin-arg-NAME=ARG
	    -fdump-ada-spec[-slim]  -fada-spec-parent=UNIT  -fdump-go-spec=FILE

(Admittedly, Emacs by default hides some of the text of a
cross-reference, but not hiding them in this case produces an even
less legible text.)

In general, it is a well-known rule that Texinfo documentation should
NOT use @ref{foo} as if @ref will disappear without a trace, leaving
just the hyperlink to 'foo'.  Looks like the rewritten manual uses
that a lot.

This "see" consistently gets in the way throughout the entire manual.
A few more examples:

   -- Option: -flocal-ivars

       Default option value for *note -fno-local-ivars.
   ...
       For example *note -std.‘=gnu90 -Wpedantic’ warns about C++ style
       ‘//’ comments, while *note -std.‘=gnu99 -Wpedantic’ does not.
   ...
       If this option is not provided but *note -Wabi.‘=`n'’ is, that
       version is used for compatibility aliases.
   ...
       Below *note -std.‘=c++20’, *note -fconcepts. enables
       support for the C++ Extensions for Concepts Technical
       Specification, ISO 19217 (2015).
   ...
  gcov [ *note -v. | *note –version. ] [ ‘-h’ | *note –help. ]


2. The translation of @var produces double-quoting in Info, here's an
   example:

  The usual way to run GCC is to run the executable called ‘gcc’, or
  ‘`machine'-gcc’ when cross-compiling, or ‘`machine'-gcc-`version'’ to
  run a specific version of GCC.

vs, the old

   The usual way to run GCC is to run the executable called 'gcc', or
  'MACHINE-gcc' when cross-compiling, or 'MACHINE-gcc-VERSION' to run a
  specific version of GCC.

I think the new variant is less readable and more confusing, because
it isn't clear whether the quotes are part of the text.  Here's an
extreme example:

  ‘@`file'’

       Read command-line options from ‘`file'’.  The options read are
       inserted in place of the original ‘@`file'’ option.  If ‘`file'’
       does not exist, or cannot be read, then the option will be treated
       literally, and not removed.


3. Some cross-references lost the hyperlinks:

  See option-index, for an index to GCC’s options.

  ("option-index" was a hyperlink to the corresponding index section).

4. Menus lost the short descriptions of the sub-sections.  Example:

  * Designated Initializers
  * Case Ranges
  * Cast to a Union Type
  * Mixed Declarations, Labels and Code
  * Declaring Attributes of Functions

vs

  * Designated Inits::    Labeling elements of initializers.
  * Case Ranges::         'case 1 ... 9' and such.
  * Cast to Union::       Casting to union type from any member of the union.
  * Mixed Declarations::  Mixing declarations and code.
  * Function Attributes:: Declaring that functions have no side effects,
			  or that they can never return.

Looks like some bug to me.

Note also that nodes are now called by the same name as the section,
which means node names generally got much longer.  Is that really a
good idea?

5. There's some strange bug with symbols inside parentheses.  For
   example:

  In GNU C and C++, you can use function attributes to specify certain
  function properties that may help the compiler optimize calls or check
  code more carefully for correctness.  For example, you can use
  attributes to specify that a function never returns ( ‘noreturn’ ),
  returns a value depending only on the values of its arguments ( ‘const’
  ), or has ‘printf’ -style arguments ( ‘format’ ).

See the extra blanks inside parens?  The old format was nicer:

  In GNU C and C++, you can use function attributes to specify certain
  function properties that may help the compiler optimize calls or check
  code more carefully for correctness.  For example, you can use
  attributes to specify that a function never returns ('noreturn'),
  returns a value depending only on the values of its arguments ('const'),
  or has 'printf'-style arguments ('format').

6. Something's wrong with the second footnote below:

     ---------- Footnotes ----------

     (1) 
  https://sourceware.org/glibc/wiki/libmvec?action=AttachFile&do=view&target=VectorABI.txt

     (2) (1) A ‘call-used’ register is a register whose contents can be
  changed by a function call; therefore, a caller cannot assume that the
  register has the same contents on return from the function as it had
  before calling the function.  Such registers are also called
  ‘call-clobbered’, ‘caller-saved’, or ‘volatile’.

Why does the 2nd footnote have 2 note numbers?

7. Lines that shouldn't be broken, are:

  ‘`type' __sync_fetch_and_add (`type' *ptr, `type' value, ...)’  ‘`type'
  __sync_fetch_and_sub (`type' *ptr, `type' value, ...)’  ‘`type'
  __sync_fetch_and_or (`type' *ptr, `type' value, ...)’  ‘`type'
  __sync_fetch_and_and (`type' *ptr, `type' value, ...)’  ‘`type'
  __sync_fetch_and_xor (`type' *ptr, `type' value, ...)’  ‘`type'
  __sync_fetch_and_nand (`type' *ptr, `type' value, ...)’

vs

  'TYPE __sync_fetch_and_add (TYPE *ptr, TYPE value, ...)'
  'TYPE __sync_fetch_and_sub (TYPE *ptr, TYPE value, ...)'
  'TYPE __sync_fetch_and_or (TYPE *ptr, TYPE value, ...)'
  'TYPE __sync_fetch_and_and (TYPE *ptr, TYPE value, ...)'
  'TYPE __sync_fetch_and_xor (TYPE *ptr, TYPE value, ...)'
  'TYPE __sync_fetch_and_nand (TYPE *ptr, TYPE value, ...)'

HTH

Re: [PATCH] Port GCC documentation to Sphinx

[Eli Zaretskii]

> Date: Tue, 29 Jun 2021 19:57:11 +0300
> From: Eli Zaretskii via Gcc <gcc@gcc.gnu.org>
> Cc: gcc@gcc.gnu.org, gcc-patches@gcc.gnu.org, joseph@codesourcery.com
> 
> Or how about this:
> 
>   `Overall Options'
> 
>        See Options Controlling the Kind of Output.
> 
>        *note -c. *note -S. *note -E. *note -o. ‘`file'’
>        *note -dumpbase. ‘`dumpbase'’ *note -dumpbase-ext.
>        ‘`auxdropsuf'’ *note -dumpdir. ‘`dumppfx'’ ‘-x’ ‘`language'’
>        *note -v. *note -###. *note –help.‘[=`class'[,...]]’
>        *note –target-help. *note –version. *note -pass-exit-codes
>        . *note -pipe. *note -specs.‘=`file'’ *note -wrapper
>        .‘@`file'’ *note -ffile-prefix-map.‘=`old'=`new'’ *note
>        -fplugin.‘=`file'’ ‘-fplugin-arg-’‘`name'=`arg'’
>        ‘-fdump-ada-spec’‘[-`slim']’ *note -fada-spec-parent.‘=`unit'’
>        *note -fdump-go-spec.‘=`file'’

I see that when I copied text into the mail, the "see" that Emacs
displays got replaced by "*note" (which is what actually appears in
the Info file).  So if you want to understand my references to the
ubiquitous "see", imagine that each "*note" is displayed as "see".

Apologies for any confusion.

Re: [PATCH] Port GCC documentation to Sphinx

[Martin Liška]

On 6/29/21 12:50 PM, Richard Earnshaw wrote:
> 
> 
> On 29/06/2021 11:09, Martin Liška wrote:
>> On 6/28/21 5:33 PM, Joseph Myers wrote:
>>> Are formatted manuals (HTML, PDF, man, info) corresponding to this patch
>>> version also available for review?
>>
>> I've just uploaded them here:
>> https://splichal.eu/gccsphinx-final/
>>
>> Martin
>>
> 
> In the HTML version of the gcc manual the sidebar has an "Option index" link but no link to the general index.  When you follow that link the page contents is just a link to the "index" where everything is all lumped together.
> 
> If we can't have separate indexes for options and general entries, I think it would make more sense for the Option index link to be removed entirely.

Fully agree with you. Thanks for the feedback and I've changed that to the standard Sphinx section,
see e.g. https://splichal.eu/gccsphinx-final/html/gcc/indices-and-tables.html

Martin

> 
> R.

Re: GCC documentation: porting to Sphinx

[Martin Liška]

On 6/29/21 5:54 PM, Arnaud Charlet wrote:
>>> In particular can you explain the motivation behind all the changes in the
>>> gcc/ada/doc directory?
>>
>> Sure:
>> 1) All Sphinx manuals live in a directory where index page is called index.rst. That's why
>> I moved e.g. this: gcc/ada/doc/{gnat_rm.rst => gnat_rm/index.rst}
>> 2) I moved latex_elements.py to ada_latex_elements.py as it clashes with Sphinx global variable
>> you modify in Sphinx config files
>> 3) I created a shared Ada config (adabaseconf.py) that extends doc/baseconf.py and sets what
>> is shared in between 3 Ada manuals.
>> 4) gnu_free_documentation_license.rst is taken from $root/doc/
> 
> OK, this is really lots of changes, if we could minimize these changes
> that would be best (and sorry but posting a link to a tarball also doesn't
> help reviews, it was actually better with a link to a git repo previously...

All right, ideally please pull my branch:
https://gcc.gnu.org/git/gitweb.cgi?p=gcc.git;a=log;h=refs/users/marxin/heads/sphinx-v4

which I force push once I rebase it. One can fetch the branch with:
$ git fetch origin refs/users/marxin/heads/sphinx-v4

> At least the Ada part itself shouldn't be too big in particular once
> simplified so could be posted standalone).

Sorry, but the patch is still 400 kB when using zstd -22. Actually, the change is very small
if you ignore renames of the 3 files:

  gcc/ada/gnat-style.texi                                        |   954
  gcc/ada/gnat_rm.texi                                           | 29822
  gcc/ada/gnat_ugn.texi                                          | 29232

The only significant change is refactoring of the conf.py config.

Martin

> 
> Arno
> 

Re: [PATCH] Port GCC documentation to Sphinx

[Martin Liška]

On 6/29/21 6:57 PM, Eli Zaretskii wrote:
>> From: Martin Liška <mliska@suse.cz>
>> Date: Tue, 29 Jun 2021 12:09:23 +0200
>> Cc: GCC Development <gcc@gcc.gnu.org>, gcc-patches@gcc.gnu.org
>>
>> On 6/28/21 5:33 PM, Joseph Myers wrote:
>>> Are formatted manuals (HTML, PDF, man, info) corresponding to this patch
>>> version also available for review?
>>
>> I've just uploaded them here:
>> https://splichal.eu/gccsphinx-final/
> 
> Thanks.

Hey!

> 
> I'm an Info junkie, so I grabbed gcc.info from there and skimmed
> through it.  Please allow me a few unsolicited comments:

I really welcome them!

> 
> 1. It sounds like Sphinx is heavily biased towards HTML format, and as
>     result uglifies the Info format?

Hopefully not :)

> 
> For example, many cross-references (AFAIU introduced as part of
> migration to Sphinx) make the text illegible in Emacs.  Example:
> 
>    This standard, in both its forms, is commonly known as `C89', or
>    occasionally as `C90', from the dates of ratification.  To select this
>    standard in GCC, use one of the options *note -ansi *note -std
>    .‘=c90’ or *note -std.‘=iso9899:1990’; to obtain all the diagnostics
>    required by the standard, you should also specify *note -pedantic.
>    (or *note -pedantic-errors. if you want them to be errors rather
>    than warnings).  See *note Options Controlling C Dialect.
>    [...]
>    An amendment to the 1990 standard was published in 1995.  This amendment
>    added digraphs and ‘__STDC_VERSION__’ to the language, but otherwise
>    concerned the library.  This amendment is commonly known as `AMD1'; the
>    amended standard is sometimes known as `C94' or `C95'.  To select this
>    standard in GCC, use the option *note -std.‘=iso9899:199409’ (with,
>    as for other standard versions, *note -pedantic. to receive all
>    required diagnostics).
> 
> Or how about this:
> 
>    `Overall Options'
> 
>         See Options Controlling the Kind of Output.
> 
>         *note -c. *note -S. *note -E. *note -o. ‘`file'’
>         *note -dumpbase. ‘`dumpbase'’ *note -dumpbase-ext.
>         ‘`auxdropsuf'’ *note -dumpdir. ‘`dumppfx'’ ‘-x’ ‘`language'’
>         *note -v. *note -###. *note –help.‘[=`class'[,...]]’
>         *note –target-help. *note –version. *note -pass-exit-codes
>         . *note -pipe. *note -specs.‘=`file'’ *note -wrapper
>         .‘@`file'’ *note -ffile-prefix-map.‘=`old'=`new'’ *note
>         -fplugin.‘=`file'’ ‘-fplugin-arg-’‘`name'=`arg'’
>         ‘-fdump-ada-spec’‘[-`slim']’ *note -fada-spec-parent.‘=`unit'’
>         *note -fdump-go-spec.‘=`file'’
> 
> In the first line, the emphasis became quotes, which sounds sub-optimal.
> In the second line, the hyperlink was lost.
> And the rest is not really readable.
> 
> Compare this with the original:
> 
>    _Overall Options_
>         *Note Options Controlling the Kind of Output.
> 	    -c  -S  -E  -o FILE  -x LANGUAGE
> 	    -v  -###  --help[=CLASS[,...]]  --target-help  --version
> 	    -pass-exit-codes  -pipe  -specs=FILE  -wrapper
> 	    @FILE  -ffile-prefix-map=OLD=NEW
> 	    -fplugin=FILE  -fplugin-arg-NAME=ARG
> 	    -fdump-ada-spec[-slim]  -fada-spec-parent=UNIT  -fdump-go-spec=FILE
> 
> (Admittedly, Emacs by default hides some of the text of a
> cross-reference, but not hiding them in this case produces an even
> less legible text.)

If I'm correct, it's exactly what's documented in Sphinx FAQ here:
https://www.sphinx-doc.org/en/master/faq.html#displaying-links

and there's a suggested Emacs code snippet that should help with links.
Does it help?

> 
> In general, it is a well-known rule that Texinfo documentation should
> NOT use @ref{foo} as if @ref will disappear without a trace, leaving
> just the hyperlink to 'foo'.  Looks like the rewritten manual uses
> that a lot.
> 
> This "see" consistently gets in the way throughout the entire manual.
> A few more examples:
> 
>     -- Option: -flocal-ivars
> 
>         Default option value for *note -fno-local-ivars.
>     ...
>         For example *note -std.‘=gnu90 -Wpedantic’ warns about C++ style
>         ‘//’ comments, while *note -std.‘=gnu99 -Wpedantic’ does not.
>     ...
>         If this option is not provided but *note -Wabi.‘=`n'’ is, that
>         version is used for compatibility aliases.
>     ...
>         Below *note -std.‘=c++20’, *note -fconcepts. enables
>         support for the C++ Extensions for Concepts Technical
>         Specification, ISO 19217 (2015).
>     ...
>    gcov [ *note -v. | *note –version. ] [ ‘-h’ | *note –help. ]
> 
> 
> 2. The translation of @var produces double-quoting in Info, here's an
>     example:
> 
>    The usual way to run GCC is to run the executable called ‘gcc’, or
>    ‘`machine'-gcc’ when cross-compiling, or ‘`machine'-gcc-`version'’ to
>    run a specific version of GCC.
> 
> vs, the old
> 
>     The usual way to run GCC is to run the executable called 'gcc', or
>    'MACHINE-gcc' when cross-compiling, or 'MACHINE-gcc-VERSION' to run a
>    specific version of GCC.
> 
> I think the new variant is less readable and more confusing, because
> it isn't clear whether the quotes are part of the text.  Here's an
> extreme example:
> 
>    ‘@`file'’
> 
>         Read command-line options from ‘`file'’.  The options read are
>         inserted in place of the original ‘@`file'’ option.  If ‘`file'’
>         does not exist, or cannot be read, then the option will be treated
>         literally, and not removed.

I can confirm that, so e.g.
Show :samp:`Samp with a {variable}.`

is transformed into:
Show @code{Samp with a @emph{variable}.}

Default info formatting is selected as:

@definfoenclose strong,`,'
@definfoenclose emph,`,'

We can adjust 'emph' formatting to nil, what do you think?

> 
> 3. Some cross-references lost the hyperlinks:
> 
>    See option-index, for an index to GCC’s options.
> 
>    ("option-index" was a hyperlink to the corresponding index section).

That's removed in latest version.

> 
> 4. Menus lost the short descriptions of the sub-sections.  Example:
> 
>    * Designated Initializers
>    * Case Ranges
>    * Cast to a Union Type
>    * Mixed Declarations, Labels and Code
>    * Declaring Attributes of Functions
> 
> vs
> 
>    * Designated Inits::    Labeling elements of initializers.
>    * Case Ranges::         'case 1 ... 9' and such.
>    * Cast to Union::       Casting to union type from any member of the union.
>    * Mixed Declarations::  Mixing declarations and code.
>    * Function Attributes:: Declaring that functions have no side effects,
> 			  or that they can never return.
> 
> Looks like some bug to me.
> 
> Note also that nodes are now called by the same name as the section,
> which means node names generally got much longer.  Is that really a
> good idea?

Well, I intentionally removed these and used simple TOC tree links
which take display text for a section title.

> 
> 5. There's some strange bug with symbols inside parentheses.  For
>     example:
> 
>    In GNU C and C++, you can use function attributes to specify certain
>    function properties that may help the compiler optimize calls or check
>    code more carefully for correctness.  For example, you can use
>    attributes to specify that a function never returns ( ‘noreturn’ ),
>    returns a value depending only on the values of its arguments ( ‘const’
>    ), or has ‘printf’ -style arguments ( ‘format’ ).
> 
> See the extra blanks inside parens?  The old format was nicer:
> 
>    In GNU C and C++, you can use function attributes to specify certain
>    function properties that may help the compiler optimize calls or check
>    code more carefully for correctness.  For example, you can use
>    attributes to specify that a function never returns ('noreturn'),
>    returns a value depending only on the values of its arguments ('const'),
>    or has 'printf'-style arguments ('format').
> 
> 6. Something's wrong with the second footnote below:
> 
>       ---------- Footnotes ----------
> 
>       (1)
>    https://sourceware.org/glibc/wiki/libmvec?action=AttachFile&do=view&target=VectorABI.txt
> 
>       (2) (1) A ‘call-used’ register is a register whose contents can be
>    changed by a function call; therefore, a caller cannot assume that the
>    register has the same contents on return from the function as it had
>    before calling the function.  Such registers are also called
>    ‘call-clobbered’, ‘caller-saved’, or ‘volatile’.
> 
> Why does the 2nd footnote have 2 note numbers?

I can confirm the following code snippet:

Note1: ([#]_)
Note2: ([#]_)

.. [#] Future versions of GCC may zero-extend, or use a target-defined ``ptr_extend`` pattern.  Do not rely on sign extension.
.. [#] I am note 2.

emits the following texinfo:

Note1: (@footnote{@w{(1)}
Future versions of GCC may zero-extend, or use a target-defined @code{ptr_extend} pattern.  Do not rely on sign extension.
})
Note2: (@footnote{@w{(2)}
I am note 2.
})

Seems correct to be, but it's likely not. Let me investigate that.

> 
> 7. Lines that shouldn't be broken, are:
> 
>    ‘`type' __sync_fetch_and_add (`type' *ptr, `type' value, ...)’  ‘`type'
>    __sync_fetch_and_sub (`type' *ptr, `type' value, ...)’  ‘`type'
>    __sync_fetch_and_or (`type' *ptr, `type' value, ...)’  ‘`type'
>    __sync_fetch_and_and (`type' *ptr, `type' value, ...)’  ‘`type'
>    __sync_fetch_and_xor (`type' *ptr, `type' value, ...)’  ‘`type'
>    __sync_fetch_and_nand (`type' *ptr, `type' value, ...)’
> 
> vs
> 
>    'TYPE __sync_fetch_and_add (TYPE *ptr, TYPE value, ...)'
>    'TYPE __sync_fetch_and_sub (TYPE *ptr, TYPE value, ...)'
>    'TYPE __sync_fetch_and_or (TYPE *ptr, TYPE value, ...)'
>    'TYPE __sync_fetch_and_and (TYPE *ptr, TYPE value, ...)'
>    'TYPE __sync_fetch_and_xor (TYPE *ptr, TYPE value, ...)'
>    'TYPE __sync_fetch_and_nand (TYPE *ptr, TYPE value, ...)'

Yes, I can confirm that, it's on my TODO list right now.

Thanks for useful comments.
Martin

> 
> HTH
> 

Re: [PATCH] Port GCC documentation to Sphinx

[Richard Earnshaw]

On 30/06/2021 05:47, Martin Liška wrote:
> On 6/29/21 12:50 PM, Richard Earnshaw wrote:
>>
>>
>> On 29/06/2021 11:09, Martin Liška wrote:
>>> On 6/28/21 5:33 PM, Joseph Myers wrote:
>>>> Are formatted manuals (HTML, PDF, man, info) corresponding to this 
>>>> patch
>>>> version also available for review?
>>>
>>> I've just uploaded them here:
>>> https://splichal.eu/gccsphinx-final/
>>>
>>> Martin
>>>
>>
>> In the HTML version of the gcc manual the sidebar has an "Option 
>> index" link but no link to the general index.  When you follow that 
>> link the page contents is just a link to the "index" where everything 
>> is all lumped together.
>>
>> If we can't have separate indexes for options and general entries, I 
>> think it would make more sense for the Option index link to be removed 
>> entirely.
> 
> Fully agree with you. Thanks for the feedback and I've changed that to 
> the standard Sphinx section,
> see e.g. 
> https://splichal.eu/gccsphinx-final/html/gcc/indices-and-tables.html
> 
> Martin
> 
>>
>> R.
> 


Thanks.  Given that the manual is nominally in American English, it 
might be better to use the term "indexes" rather than "indices".

https://grammarist.com/usage/indexes-indices/

R.

Re: [PATCH] Port GCC documentation to Sphinx

[Martin Liška]

On 6/30/21 12:11 PM, Martin Liška wrote:
> Seems correct to be, but it's likely not. Let me investigate that.

It's a real issue in Sphinx. I've just addressed that with:
https://github.com/sphinx-doc/sphinx/pull/9390

Cheers,
Martin

Re: [PATCH] Port GCC documentation to Sphinx

[Andrey Belevantsev]

Hi Martin,

On 29.06.2021 13:09, Martin Liška wrote:
> On 6/28/21 5:33 PM, Joseph Myers wrote:
>> Are formatted manuals (HTML, PDF, man, info) corresponding to this patch
>> version also available for review?
>
> I've just uploaded them here:
> https://splichal.eu/gccsphinx-final/
>
> Martin

I've randomly looked at the PDF version of the GCC internals manual and the
table of contents there only has an introduction and an index (looks like
all other chapters went under introduction).  Other PDFs have the first
level chapters in the contents.  Maybe it will be a good idea to include
the lower level chapters as well, or at least to fix the gccint one :)

Best,
Andrey

Re: [PATCH] Port GCC documentation to Sphinx

[Martin Liška]

On 6/30/21 12:14 PM, Richard Earnshaw wrote:
> 
> 
> On 30/06/2021 05:47, Martin Liška wrote:
>> On 6/29/21 12:50 PM, Richard Earnshaw wrote:
>>>
>>>
>>> On 29/06/2021 11:09, Martin Liška wrote:
>>>> On 6/28/21 5:33 PM, Joseph Myers wrote:
>>>>> Are formatted manuals (HTML, PDF, man, info) corresponding to this patch
>>>>> version also available for review?
>>>>
>>>> I've just uploaded them here:
>>>> https://splichal.eu/gccsphinx-final/
>>>>
>>>> Martin
>>>>
>>>
>>> In the HTML version of the gcc manual the sidebar has an "Option index" link but no link to the general index.  When you follow that link the page contents is just a link to the "index" where everything is all lumped together.
>>>
>>> If we can't have separate indexes for options and general entries, I think it would make more sense for the Option index link to be removed entirely.
>>
>> Fully agree with you. Thanks for the feedback and I've changed that to the standard Sphinx section,
>> see e.g. https://splichal.eu/gccsphinx-final/html/gcc/indices-and-tables.html
>>
>> Martin
>>
>>>
>>> R.
>>
> 
> 
> Thanks.  Given that the manual is nominally in American English, it might be better to use the term "indexes" rather than "indices".

Sure, fixed (while preserving the filename, we have multi copies of it).

Martin

> 
> https://grammarist.com/usage/indexes-indices/
> 
> R.

Re: [PATCH] Port GCC documentation to Sphinx

[Martin Liška]

On 6/29/21 6:57 PM, Eli Zaretskii wrote:
> 5. There's some strange bug with symbols inside parentheses.  For
>     example:
> 
>    In GNU C and C++, you can use function attributes to specify certain
>    function properties that may help the compiler optimize calls or check
>    code more carefully for correctness.  For example, you can use
>    attributes to specify that a function never returns ( ‘noreturn’ ),
>    returns a value depending only on the values of its arguments ( ‘const’
>    ), or has ‘printf’ -style arguments ( ‘format’ ).
> 
> See the extra blanks inside parens?  The old format was nicer:
> 
>    In GNU C and C++, you can use function attributes to specify certain
>    function properties that may help the compiler optimize calls or check
>    code more carefully for correctness.  For example, you can use
>    attributes to specify that a function never returns ('noreturn'),
>    returns a value depending only on the values of its arguments ('const'),
>    or has 'printf'-style arguments ('format').

This issues is resolved now. Good point!

Martin

Re: [PATCH] Port GCC documentation to Sphinx

[Martin Liška]

On 6/30/21 12:55 PM, Andrey Belevantsev wrote:
> Hi Martin,
> 
> On 29.06.2021 13:09, Martin Liška wrote:
>> On 6/28/21 5:33 PM, Joseph Myers wrote:
>>> Are formatted manuals (HTML, PDF, man, info) corresponding to this patch
>>> version also available for review?
>>
>> I've just uploaded them here:
>> https://splichal.eu/gccsphinx-final/
>>
>> Martin
> 

Hello.

> I've randomly looked at the PDF version of the GCC internals manual and the
> table of contents there only has an introduction and an index (looks like
> all other chapters went under introduction).  Other PDFs have the first
> level chapters in the contents.  Maybe it will be a good idea to include
> the lower level chapters as well, or at least to fix the gccint one :)

Very good point and I fixed both your comments. It would be much better having
listed all chapters in TOC.

Martin

> 
> Best,
> Andrey
> 

Re: [PATCH] Port GCC documentation to Sphinx

[Eli Zaretskii]

> Cc: joseph@codesourcery.com, gcc@gcc.gnu.org, gcc-patches@gcc.gnu.org
> From: Martin Liška <mliska@suse.cz>
> Date: Wed, 30 Jun 2021 12:11:03 +0200
> 
> > (Admittedly, Emacs by default hides some of the text of a
> > cross-reference, but not hiding them in this case produces an even
> > less legible text.)
> 
> If I'm correct, it's exactly what's documented in Sphinx FAQ here:
> https://www.sphinx-doc.org/en/master/faq.html#displaying-links
> 
> and there's a suggested Emacs code snippet that should help with links.
> Does it help?

It helps some, but not all of the issues disappear.  For example,
stuff like this is still hard to read:

  To select this standard in GCC, use one of the options -ansi
                                                         -----
  -std.‘=c90’ or -std.‘=iso9899:1990’
  ----           ----

The quotes around the option values don't help.

Also, using the method proposed by Sphinx FAQ would need a change in
Emacs, which will take time to propagate.  So my suggestion is to
minimize the use of such "inline" hyperlinks.

> >    ‘@`file'’
> > 
> >         Read command-line options from ‘`file'’.  The options read are
> >         inserted in place of the original ‘@`file'’ option.  If ‘`file'’
> >         does not exist, or cannot be read, then the option will be treated
> >         literally, and not removed.
> 
> I can confirm that, so e.g.
> Show :samp:`Samp with a {variable}.`
> 
> is transformed into:
> Show @code{Samp with a @emph{variable}.}
> 
> Default info formatting is selected as:
> 
> @definfoenclose strong,`,'
> @definfoenclose emph,`,'
> 
> We can adjust 'emph' formatting to nil, what do you think?

Something like that, yes.  But the problem is: how will you format it
instead?  The known alternatives, _foo_ and *foo* both use punctuation
characters, which will get in the way similarly to the quotes.  Can
you format those in caps, like makeinfo does?

> > 4. Menus lost the short descriptions of the sub-sections.  Example:
> > 
> >    * Designated Initializers
> >    * Case Ranges
> >    * Cast to a Union Type
> >    * Mixed Declarations, Labels and Code
> >    * Declaring Attributes of Functions
> > 
> > vs
> > 
> >    * Designated Inits::    Labeling elements of initializers.
> >    * Case Ranges::         'case 1 ... 9' and such.
> >    * Cast to Union::       Casting to union type from any member of the union.
> >    * Mixed Declarations::  Mixing declarations and code.
> >    * Function Attributes:: Declaring that functions have no side effects,
> > 			  or that they can never return.
> > 
> > Looks like some bug to me.
> > 
> > Note also that nodes are now called by the same name as the section,
> > which means node names generally got much longer.  Is that really a
> > good idea?
> 
> Well, I intentionally removed these and used simple TOC tree links
> which take display text for a section title.

I would suggest to discuss these decisions first, perhaps on the
Texinfo mailing list?  I'm accustomed to these short descriptions, but
I'm not sure how important they are for others.

Re: [PATCH] Port GCC documentation to Sphinx

[Martin Liška]

On 6/29/21 6:57 PM, Eli Zaretskii wrote:
> 2. The translation of @var produces double-quoting in Info, here's an
>     example:
> 
>    The usual way to run GCC is to run the executable called ‘gcc’, or
>    ‘`machine'-gcc’ when cross-compiling, or ‘`machine'-gcc-`version'’ to
>    run a specific version of GCC.
> 
> vs, the old
> 
>     The usual way to run GCC is to run the executable called 'gcc', or
>    'MACHINE-gcc' when cross-compiling, or 'MACHINE-gcc-VERSION' to run a
>    specific version of GCC.
> 
> I think the new variant is less readable and more confusing, because
> it isn't clear whether the quotes are part of the text.  Here's an
> extreme example:
> 
>    ‘@`file'’
> 
>         Read command-line options from ‘`file'’.  The options read are
>         inserted in place of the original ‘@`file'’ option.  If ‘`file'’
>         does not exist, or cannot be read, then the option will be treated
>         literally, and not removed.

For this one, I've just created the following pull request:
https://github.com/sphinx-doc/sphinx/pull/9391

Cheers,
Martin

Re: [PATCH] Port GCC documentation to Sphinx

[Eli Zaretskii]

> Cc: joseph@codesourcery.com, gcc@gcc.gnu.org, gcc-patches@gcc.gnu.org
> From: Martin Liška <mliska@suse.cz>
> Date: Wed, 30 Jun 2021 15:28:40 +0200
> 
> >    ‘@`file'’
> > 
> >         Read command-line options from ‘`file'’.  The options read are
> >         inserted in place of the original ‘@`file'’ option.  If ‘`file'’
> >         does not exist, or cannot be read, then the option will be treated
> >         literally, and not removed.
> 
> For this one, I've just created the following pull request:
> https://github.com/sphinx-doc/sphinx/pull/9391

Thanks, but does that mean @var will no longer stand out in the
produced Info format?  That'd be sub-optimal, I think, because a clear
reference to a meta-syntactic variable will be lost.

Re: [PATCH] Port GCC documentation to Sphinx

[Martin Liška]

On 6/30/21 3:38 PM, Eli Zaretskii wrote:
>> Cc: joseph@codesourcery.com, gcc@gcc.gnu.org, gcc-patches@gcc.gnu.org
>> From: Martin Liška <mliska@suse.cz>
>> Date: Wed, 30 Jun 2021 15:28:40 +0200
>>
>>>     ‘@`file'’
>>>
>>>          Read command-line options from ‘`file'’.  The options read are
>>>          inserted in place of the original ‘@`file'’ option.  If ‘`file'’
>>>          does not exist, or cannot be read, then the option will be treated
>>>          literally, and not removed.
>>
>> For this one, I've just created the following pull request:
>> https://github.com/sphinx-doc/sphinx/pull/9391
> 
> Thanks, but does that mean @var will no longer stand out in the
> produced Info format?  That'd be sub-optimal, I think, because a clear
> reference to a meta-syntactic variable will be lost.

Yes. An alternative approach for:
Show :samp:`Samp with a {variable}.`

can be using @var{variable}, resulting with the following info output:
Show ‘Samp with a VARIABLE.’

Does it seem reasonable?
Thanks,
Martin

Re: [PATCH] Port GCC documentation to Sphinx

[Eli Zaretskii]

> Cc: joseph@codesourcery.com, gcc@gcc.gnu.org, gcc-patches@gcc.gnu.org
> From: Martin Liška <mliska@suse.cz>
> Date: Wed, 30 Jun 2021 16:04:32 +0200
> 
> > Thanks, but does that mean @var will no longer stand out in the
> > produced Info format?  That'd be sub-optimal, I think, because a clear
> > reference to a meta-syntactic variable will be lost.
> 
> Yes. An alternative approach for:
> Show :samp:`Samp with a {variable}.`
> 
> can be using @var{variable}, resulting with the following info output:
> Show ‘Samp with a VARIABLE.’
> 
> Does it seem reasonable?

Yes, the latter sounds reasonable.

Thanks.

Re: [PATCH] Port GCC documentation to Sphinx

[Martin Liška]

On 6/30/21 5:43 PM, Eli Zaretskii wrote:
>> Cc: joseph@codesourcery.com, gcc@gcc.gnu.org, gcc-patches@gcc.gnu.org
>> From: Martin Liška <mliska@suse.cz>
>> Date: Wed, 30 Jun 2021 16:04:32 +0200
>>
>>> Thanks, but does that mean @var will no longer stand out in the
>>> produced Info format?  That'd be sub-optimal, I think, because a clear
>>> reference to a meta-syntactic variable will be lost.
>>
>> Yes. An alternative approach for:
>> Show :samp:`Samp with a {variable}.`
>>
>> can be using @var{variable}, resulting with the following info output:
>> Show ‘Samp with a VARIABLE.’
>>
>> Does it seem reasonable?
> 
> Yes, the latter sounds reasonable.
> 
> Thanks.
> 

Good, I modified the existing pull request for it:
https://github.com/sphinx-doc/sphinx/pull/9391

Martin

Re: [PATCH] Port GCC documentation to Sphinx

[Martin Liška]

On 6/30/21 3:09 PM, Eli Zaretskii wrote:
>> Cc: joseph@codesourcery.com, gcc@gcc.gnu.org, gcc-patches@gcc.gnu.org
>> From: Martin Liška <mliska@suse.cz>
>> Date: Wed, 30 Jun 2021 12:11:03 +0200
>>
>>> (Admittedly, Emacs by default hides some of the text of a
>>> cross-reference, but not hiding them in this case produces an even
>>> less legible text.)
>>
>> If I'm correct, it's exactly what's documented in Sphinx FAQ here:
>> https://www.sphinx-doc.org/en/master/faq.html#displaying-links
>>
>> and there's a suggested Emacs code snippet that should help with links.
>> Does it help?
> 
> It helps some, but not all of the issues disappear.  For example,
> stuff like this is still hard to read:
> 
>    To select this standard in GCC, use one of the options -ansi
>                                                           -----
>    -std.‘=c90’ or -std.‘=iso9899:1990’
>    ----           ----

If I understand the notes correct, the '.' should be also hidden by e.g. Emacs.
About ‘=iso9899:1990’, yes, it's a :samp: and how it's wrapper by Sphinx by default.

> 
> The quotes around the option values don't help.
> 
> Also, using the method proposed by Sphinx FAQ would need a change in
> Emacs, which will take time to propagate.  So my suggestion is to
> minimize the use of such "inline" hyperlinks.
> 
>>>     ‘@`file'’
>>>
>>>          Read command-line options from ‘`file'’.  The options read are
>>>          inserted in place of the original ‘@`file'’ option.  If ‘`file'’
>>>          does not exist, or cannot be read, then the option will be treated
>>>          literally, and not removed.
>>
>> I can confirm that, so e.g.
>> Show :samp:`Samp with a {variable}.`
>>
>> is transformed into:
>> Show @code{Samp with a @emph{variable}.}
>>
>> Default info formatting is selected as:
>>
>> @definfoenclose strong,`,'
>> @definfoenclose emph,`,'
>>
>> We can adjust 'emph' formatting to nil, what do you think?
> 
> Something like that, yes.  But the problem is: how will you format it
> instead?  The known alternatives, _foo_ and *foo* both use punctuation
> characters, which will get in the way similarly to the quotes.  Can
> you format those in caps, like makeinfo does?

You are fully right, info is very simple format and it uses wrapping for the formatting
purpose (by default * and _). So, I don't have any elegant solution.

> 
>>> 4. Menus lost the short descriptions of the sub-sections.  Example:
>>>
>>>     * Designated Initializers
>>>     * Case Ranges
>>>     * Cast to a Union Type
>>>     * Mixed Declarations, Labels and Code
>>>     * Declaring Attributes of Functions
>>>
>>> vs
>>>
>>>     * Designated Inits::    Labeling elements of initializers.
>>>     * Case Ranges::         'case 1 ... 9' and such.
>>>     * Cast to Union::       Casting to union type from any member of the union.
>>>     * Mixed Declarations::  Mixing declarations and code.
>>>     * Function Attributes:: Declaring that functions have no side effects,
>>> 			  or that they can never return.
>>>
>>> Looks like some bug to me.
>>>
>>> Note also that nodes are now called by the same name as the section,
>>> which means node names generally got much longer.  Is that really a
>>> good idea?
>>
>> Well, I intentionally removed these and used simple TOC tree links
>> which take display text for a section title.
> 
> I would suggest to discuss these decisions first, perhaps on the
> Texinfo mailing list?  I'm accustomed to these short descriptions, but
> I'm not sure how important they are for others.

Well, it was decision done during the transition of texinfo files into Sphinx.
I don't see why it should be discussed in Texinfo community.

Cheers,
Martin

Re: [PATCH] Port GCC documentation to Sphinx

[Eli Zaretskii]

> Cc: joseph@codesourcery.com, gcc@gcc.gnu.org, gcc-patches@gcc.gnu.org
> From: Martin Liška <mliska@suse.cz>
> Date: Thu, 1 Jul 2021 14:44:10 +0200
> 
> > It helps some, but not all of the issues disappear.  For example,
> > stuff like this is still hard to read:
> > 
> >    To select this standard in GCC, use one of the options -ansi
> >                                                           -----
> >    -std.‘=c90’ or -std.‘=iso9899:1990’
> >    ----           ----
> 
> If I understand the notes correct, the '.' should be also hidden by e.g. Emacs.

No, it doesn't.  The actual text in the Info file is:

   *note -std: f.‘=iso9899:1990’

and the period after " f" isn't hidden.  Where does that "f" come from
and what is its purpose here? can it be removed (together with the
period)?

> About ‘=iso9899:1990’, yes, it's a :samp: and how it's wrapper by Sphinx by default.

Why is it a separate :samp:?  IMO, the correct markup is to make the
entire string -std=iso9899:1990 have the same typeface.  In Texinfo,
I'd use

   @option{-std=iso9899:1990}

> >> We can adjust 'emph' formatting to nil, what do you think?
> > 
> > Something like that, yes.  But the problem is: how will you format it
> > instead?  The known alternatives, _foo_ and *foo* both use punctuation
> > characters, which will get in the way similarly to the quotes.  Can
> > you format those in caps, like makeinfo does?
> 
> You are fully right, info is very simple format and it uses wrapping for the formatting
> purpose (by default * and _). So, I don't have any elegant solution.

Well, it sounds from another mail that you did find a solution: to
up-case the string in @var.

> >>> Note also that nodes are now called by the same name as the section,
> >>> which means node names generally got much longer.  Is that really a
> >>> good idea?
> >>
> >> Well, I intentionally removed these and used simple TOC tree links
> >> which take display text for a section title.
> > 
> > I would suggest to discuss these decisions first, perhaps on the
> > Texinfo mailing list?  I'm accustomed to these short descriptions, but
> > I'm not sure how important they are for others.
> 
> Well, it was decision done during the transition of texinfo files into Sphinx.
> I don't see why it should be discussed in Texinfo community.

This actually raises a more general issue with this Sphinx porting
initiative: what will be the canonical style guide for maintaining the
GCC manual in Sphinx, or more generally for writing GNU manuals in
Sphinx?  For Texinfo, we have the Texinfo manual, which both documents
the language and provides style guidelines for how to use Texinfo for
producing good manuals.  Contributors to GNU manuals are using those
guidelines for many years.  Is there, or will there be, an equivalent
style guide for Sphinx?  If not, how will the future contributors to
the GCC manuals know what are the writing and style conventions?

That is why I recommended to discuss this on the Texinfo list: that's
the place where such guidelines are discussed, and where we have
experts who understand the effects and consequences of using this or
that style.  The current style in GNU manuals is to have the menus as
we see them in the existing GCC manuals: with a short description.
Maybe there are good reasons to deviate from that style, but
shouldn't this be at least presented and discussed, before the
decision is made?  GCC developers are not the only ones who will be
reading the future GCC manuals.

Re: [PATCH] Port GCC documentation to Sphinx

[Martin Liška]

On 7/1/21 3:33 PM, Eli Zaretskii wrote:
>> Cc: joseph@codesourcery.com, gcc@gcc.gnu.org, gcc-patches@gcc.gnu.org
>> From: Martin Liška <mliska@suse.cz>
>> Date: Thu, 1 Jul 2021 14:44:10 +0200
>>
>>> It helps some, but not all of the issues disappear.  For example,
>>> stuff like this is still hard to read:
>>>
>>>     To select this standard in GCC, use one of the options -ansi
>>>                                                            -----
>>>     -std.‘=c90’ or -std.‘=iso9899:1990’
>>>     ----           ----
>>
>> If I understand the notes correct, the '.' should be also hidden by e.g. Emacs.
> 
> No, it doesn't.  The actual text in the Info file is:
> 
>     *note -std: f.‘=iso9899:1990’
> 
> and the period after " f" isn't hidden.  Where does that "f" come from
> and what is its purpose here? can it be removed (together with the
> period)?

It's name of the anchor used for the @ref. The names are automatically generated
by makeinfo. So there's an example:

This is the warning level of @ref{e,,-Wshift-overflow3} and …

becomes in info:
This is the warning level of *note -Wshift-overflow3: e. and …

I can ask the question at Sphinx, the Emacs script should hide that.

> 
>> About ‘=iso9899:1990’, yes, it's a :samp: and how it's wrapper by Sphinx by default.
> 
> Why is it a separate :samp:?  IMO, the correct markup is to make the
> entire string -std=iso9899:1990 have the same typeface.  In Texinfo,
> I'd use
> 
>     @option{-std=iso9899:1990}
> 
>>>> We can adjust 'emph' formatting to nil, what do you think?
>>>
>>> Something like that, yes.  But the problem is: how will you format it
>>> instead?  The known alternatives, _foo_ and *foo* both use punctuation
>>> characters, which will get in the way similarly to the quotes.  Can
>>> you format those in caps, like makeinfo does?
>>
>> You are fully right, info is very simple format and it uses wrapping for the formatting
>> purpose (by default * and _). So, I don't have any elegant solution.
> 
> Well, it sounds from another mail that you did find a solution: to
> up-case the string in @var.

I don't know. Some of them can be e.g. keywords and using upper-case does not seem
to me feasible.

> 
>>>>> Note also that nodes are now called by the same name as the section,
>>>>> which means node names generally got much longer.  Is that really a
>>>>> good idea?
>>>>
>>>> Well, I intentionally removed these and used simple TOC tree links
>>>> which take display text for a section title.
>>>
>>> I would suggest to discuss these decisions first, perhaps on the
>>> Texinfo mailing list?  I'm accustomed to these short descriptions, but
>>> I'm not sure how important they are for others.
>>
>> Well, it was decision done during the transition of texinfo files into Sphinx.
>> I don't see why it should be discussed in Texinfo community.
> 
> This actually raises a more general issue with this Sphinx porting
> initiative: what will be the canonical style guide for maintaining the
> GCC manual in Sphinx, or more generally for writing GNU manuals in
> Sphinx?  For Texinfo, we have the Texinfo manual, which both documents
> the language and provides style guidelines for how to use Texinfo for
> producing good manuals.  Contributors to GNU manuals are using those
> guidelines for many years.  Is there, or will there be, an equivalent
> style guide for Sphinx?  If not, how will the future contributors to
> the GCC manuals know what are the writing and style conventions?

No, I'm not planning any extra style guide. We will you standard Sphinx RST
manual and one can find many tutorials about how to do it.

> 
> That is why I recommended to discuss this on the Texinfo list: that's
> the place where such guidelines are discussed, and where we have
> experts who understand the effects and consequences of using this or
> that style.  The current style in GNU manuals is to have the menus as
> we see them in the existing GCC manuals: with a short description.
> Maybe there are good reasons to deviate from that style, but
> shouldn't this be at least presented and discussed, before the
> decision is made?  GCC developers are not the only ones who will be
> reading the future GCC manuals.
> 

That seems to me a subtle adjustment and it's standard way how people generate
TOC in Sphinx. See e.g. the Linux kernel documentation:
https://www.kernel.org/doc/html/latest/

Martin

Re: [PATCH] Port GCC documentation to Sphinx

[Michael Matz]

Hello,

On Thu, 1 Jul 2021, Martin Liška wrote:

> On 7/1/21 3:33 PM, Eli Zaretskii wrote:
> > > Cc: joseph@codesourcery.com, gcc@gcc.gnu.org, gcc-patches@gcc.gnu.org
> > > From: Martin Liška <mliska@suse.cz>
> > > Date: Thu, 1 Jul 2021 14:44:10 +0200
> > > 
> > > > It helps some, but not all of the issues disappear.  For example,
> > > > stuff like this is still hard to read:
> > > > 
> > > >     To select this standard in GCC, use one of the options -ansi
> > > >                                                            -----
> > > >     -std.‘=c90’ or -std.‘=iso9899:1990’
> > > >     ----           ----
> > > 
> > > If I understand the notes correct, the '.' should be also hidden by e.g.
> > > Emacs.
> > 
> > No, it doesn't.  The actual text in the Info file is:
> > 
> >     *note -std: f.‘=iso9899:1990’
> > 
> > and the period after " f" isn't hidden.  Where does that "f" come from
> > and what is its purpose here? can it be removed (together with the
> > period)?
> 
> It's name of the anchor used for the @ref. The names are automatically
> generated
> by makeinfo. So there's an example:
> 
> This is the warning level of @ref{e,,-Wshift-overflow3} and …
> 
> becomes in info:
> This is the warning level of *note -Wshift-overflow3: e. and …
> 
> I can ask the question at Sphinx, the Emacs script should hide that.

Not everyone reads info within Emacs; even if there's an emacs solution to 
postprocess info pages to make them nicer we can't rely on that.  It must 
look sensible without that.  In this case it seems that already the 
generated .texinfo input to makeinfo is bad, where does the 'e' (or 'f') 
come from?  The original texinfo file simply contains:

  @option{-std=iso9899:1990}

so that's what perhaps should be generated, or maybe the anchor in @ref is 
optional and could be left out if it doesn't provide any info (a single 
character as anchor doesn't seem very useful)?

> > > > > We can adjust 'emph' formatting to nil, what do you think?
> > > > 
> > > > Something like that, yes.  But the problem is: how will you format it
> > > > instead?  The known alternatives, _foo_ and *foo* both use punctuation
> > > > characters, which will get in the way similarly to the quotes.  Can
> > > > you format those in caps, like makeinfo does?
> > > 
> > > You are fully right, info is very simple format and it uses wrapping for
> > > the formatting
> > > purpose (by default * and _). So, I don't have any elegant solution.
> > 
> > Well, it sounds from another mail that you did find a solution: to
> > up-case the string in @var.
> 
> I don't know. Some of them can be e.g. keywords and using upper-case 
> does not seem to me feasible.

Then that needs to be different already in the input, so that the 
directive that (in info) capitalizes is only used in contexts where that 
makes sense.  People reading info pages will know that an all-caps word 
often means a syntactic variable/placeholder, so that should be preserved.


Ciao,
Michael.

Re: [PATCH] Port GCC documentation to Sphinx

[Eli Zaretskii]

> Cc: joseph@codesourcery.com, gcc@gcc.gnu.org, gcc-patches@gcc.gnu.org
> From: Martin Liška <mliska@suse.cz>
> Date: Thu, 1 Jul 2021 16:14:30 +0200
> 
> >> If I understand the notes correct, the '.' should be also hidden by e.g. Emacs.
> > 
> > No, it doesn't.  The actual text in the Info file is:
> > 
> >     *note -std: f.‘=iso9899:1990’
> > 
> > and the period after " f" isn't hidden.  Where does that "f" come from
> > and what is its purpose here? can it be removed (together with the
> > period)?
> 
> It's name of the anchor used for the @ref. The names are automatically generated
> by makeinfo. So there's an example:
> 
> This is the warning level of @ref{e,,-Wshift-overflow3} and …
> 
> becomes in info:
> This is the warning level of *note -Wshift-overflow3: e. and …
> 
> I can ask the question at Sphinx, the Emacs script should hide that.

Emacs doesn't hide the period.  But there shouldn't be a period to
begin with, since it's the middle of a sentence.  The correct way of
writing this in Texinfo is to have some punctuation: a comma or a
semi-colon, after the closing brace, like this:

  This is the warning level of @ref{e,,-Wshift-overflow3}, and …

Does Sphinx somehow generate the period if there's no comma, or does
it do it unconditionally, i.e. even if there is a punctuation after
the closing brace?

> > This actually raises a more general issue with this Sphinx porting
> > initiative: what will be the canonical style guide for maintaining the
> > GCC manual in Sphinx, or more generally for writing GNU manuals in
> > Sphinx?  For Texinfo, we have the Texinfo manual, which both documents
> > the language and provides style guidelines for how to use Texinfo for
> > producing good manuals.  Contributors to GNU manuals are using those
> > guidelines for many years.  Is there, or will there be, an equivalent
> > style guide for Sphinx?  If not, how will the future contributors to
> > the GCC manuals know what are the writing and style conventions?
> 
> No, I'm not planning any extra style guide. We will you standard Sphinx RST
> manual and one can find many tutorials about how to do it.

Are you sure everything there is good for our manuals?  Did you
compare the style conventions there with what we have in the Texinfo
manual?

Moreover, this means people who contribute to other manuals will now
have to learn two different styles, no?  And that's in addition to
learning one more language.

> > That is why I recommended to discuss this on the Texinfo list: that's
> > the place where such guidelines are discussed, and where we have
> > experts who understand the effects and consequences of using this or
> > that style.  The current style in GNU manuals is to have the menus as
> > we see them in the existing GCC manuals: with a short description.
> > Maybe there are good reasons to deviate from that style, but
> > shouldn't this be at least presented and discussed, before the
> > decision is made?  GCC developers are not the only ones who will be
> > reading the future GCC manuals.
> > 
> 
> That seems to me a subtle adjustment and it's standard way how people generate
> TOC in Sphinx. See e.g. the Linux kernel documentation:
> https://www.kernel.org/doc/html/latest/

I don't think the GCC manuals should necessarily be bound by the
Sphinx standards.  Where those standards are sub-optimal, it is
perfectly okay for GCC (and other projects) to deviate.  GCC and other
GNU manuals used a certain style and convention for decades, so
there's more than enough experience and tradition to build on.

I will no longer pursue this point, but let me just say that I
consider it a mistake to throw away all the experience collected using
Texinfo just because Sphinx folks have other traditions and
conventions.  It might be throwing the baby with the bathwater.

Re: [PATCH] Port GCC documentation to Sphinx

[Martin Liška]

On 7/1/21 5:44 PM, Eli Zaretskii wrote:
>> Cc: joseph@codesourcery.com, gcc@gcc.gnu.org, gcc-patches@gcc.gnu.org
>> From: Martin Liška <mliska@suse.cz>
>> Date: Thu, 1 Jul 2021 16:14:30 +0200
>>
>>>> If I understand the notes correct, the '.' should be also hidden by e.g. Emacs.
>>>
>>> No, it doesn't.  The actual text in the Info file is:
>>>
>>>      *note -std: f.‘=iso9899:1990’
>>>
>>> and the period after " f" isn't hidden.  Where does that "f" come from
>>> and what is its purpose here? can it be removed (together with the
>>> period)?
>>
>> It's name of the anchor used for the @ref. The names are automatically generated
>> by makeinfo. So there's an example:
>>
>> This is the warning level of @ref{e,,-Wshift-overflow3} and …
>>
>> becomes in info:
>> This is the warning level of *note -Wshift-overflow3: e. and …
>>
>> I can ask the question at Sphinx, the Emacs script should hide that.
> 
> Emacs doesn't hide the period.  But there shouldn't be a period to
> begin with, since it's the middle of a sentence.  The correct way of
> writing this in Texinfo is to have some punctuation: a comma or a
> semi-colon, after the closing brace, like this:
> 
>    This is the warning level of @ref{e,,-Wshift-overflow3}, and …

I don't see why we should put a comma after an option reference.

> 
> Does Sphinx somehow generate the period if there's no comma, or does
> it do it unconditionally, i.e. even if there is a punctuation after
> the closing brace?

It's all related to Texinfo. Sphinx generates e.g.
Enabled by @ref{7,,-Wall} and something else.

as documented here:
https://www.gnu.org/software/texinfo/manual/texinfo/html_node/_0040ref.html

Then it ends with the following info output:

      Enabled by *note -Wall: 7. and something else.

So the period is added by Texinfo. If I put comma after a reference, then
the period is not added there.

> 
>>> This actually raises a more general issue with this Sphinx porting
>>> initiative: what will be the canonical style guide for maintaining the
>>> GCC manual in Sphinx, or more generally for writing GNU manuals in
>>> Sphinx?  For Texinfo, we have the Texinfo manual, which both documents
>>> the language and provides style guidelines for how to use Texinfo for
>>> producing good manuals.  Contributors to GNU manuals are using those
>>> guidelines for many years.  Is there, or will there be, an equivalent
>>> style guide for Sphinx?  If not, how will the future contributors to
>>> the GCC manuals know what are the writing and style conventions?
>>
>> No, I'm not planning any extra style guide. We will you standard Sphinx RST
>> manual and one can find many tutorials about how to do it.
> 
> Are you sure everything there is good for our manuals?  Did you
> compare the style conventions there with what we have in the Texinfo
> manual?

I'm not sure, but I made the conversion from Texinfo as close as possible to RST files.
I see the documentation markup natural and it matches with we write documentation
in Texinfo.

> 
> Moreover, this means people who contribute to other manuals will now
> have to learn two different styles, no?  And that's in addition to
> learning one more language.

Yes, people will have to learn RST. Similarly to git conversion, people also
had to learn another version control system (we used svn).

> 
>>> That is why I recommended to discuss this on the Texinfo list: that's
>>> the place where such guidelines are discussed, and where we have
>>> experts who understand the effects and consequences of using this or
>>> that style.  The current style in GNU manuals is to have the menus as
>>> we see them in the existing GCC manuals: with a short description.
>>> Maybe there are good reasons to deviate from that style, but
>>> shouldn't this be at least presented and discussed, before the
>>> decision is made?  GCC developers are not the only ones who will be
>>> reading the future GCC manuals.
>>>
>>
>> That seems to me a subtle adjustment and it's standard way how people generate
>> TOC in Sphinx. See e.g. the Linux kernel documentation:
>> https://www.kernel.org/doc/html/latest/
> 
> I don't think the GCC manuals should necessarily be bound by the
> Sphinx standards.  Where those standards are sub-optimal, it is
> perfectly okay for GCC (and other projects) to deviate.  GCC and other
> GNU manuals used a certain style and convention for decades, so
> there's more than enough experience and tradition to build on.

What type of conversions and style are going to change with conversion to Sphinx?
Do you see any of them worse than what we have now?

> 
> I will no longer pursue this point, but let me just say that I
> consider it a mistake to throw away all the experience collected using
> Texinfo just because Sphinx folks have other traditions and
> conventions.  It might be throwing the baby with the bathwater.
> 

Again, please show up concrete examples. What you describe is very theoretical.

Thanks,
Martin

Re: [PATCH] Port GCC documentation to Sphinx

[Eli Zaretskii]

> Cc: joseph@codesourcery.com, gcc@gcc.gnu.org, gcc-patches@gcc.gnu.org
> From: Martin Liška <mliska@suse.cz>
> Date: Thu, 1 Jul 2021 18:04:24 +0200
> 
> > Emacs doesn't hide the period.  But there shouldn't be a period to
> > begin with, since it's the middle of a sentence.  The correct way of
> > writing this in Texinfo is to have some punctuation: a comma or a
> > semi-colon, after the closing brace, like this:
> > 
> >    This is the warning level of @ref{e,,-Wshift-overflow3}, and …
> 
> I don't see why we should put a comma after an option reference.

You explained it yourself later on:

> It's all related to Texinfo. Sphinx generates e.g.
> Enabled by @ref{7,,-Wall} and something else.
> 
> as documented here:
> https://www.gnu.org/software/texinfo/manual/texinfo/html_node/_0040ref.html
> 
> Then it ends with the following info output:
> 
>       Enabled by *note -Wall: 7. and something else.
> 
> So the period is added by Texinfo. If I put comma after a reference, then
> the period is not added there.
  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

So the purpose of having the comma there is to avoid having a period
in the middle of a sentence, which is added by makeinfo (because the
Info readers need that).  Having a comma there may seem a bit
redundant, but having a period will definitely look like a typo, if
not confuse the heck out of the reader, especially if you want to use
these inline cross-references so massively.

> > I don't think the GCC manuals should necessarily be bound by the
> > Sphinx standards.  Where those standards are sub-optimal, it is
> > perfectly okay for GCC (and other projects) to deviate.  GCC and other
> > GNU manuals used a certain style and convention for decades, so
> > there's more than enough experience and tradition to build on.
> 
> What type of conversions and style are going to change with conversion to Sphinx?

Anything that is different from the style conventions described in the
Texinfo manual.  We have many such conventions.

> Do you see any of them worse than what we have now?

I didn't bother reading the Sphinx guidelines yet, and don't know when
(and if) I will have time for that.  I do think the comparison should
be part of the job or moving to Sphinx.

> > I will no longer pursue this point, but let me just say that I
> > consider it a mistake to throw away all the experience collected using
> > Texinfo just because Sphinx folks have other traditions and
> > conventions.  It might be throwing the baby with the bathwater.
> > 
> 
> Again, please show up concrete examples. What you describe is very theoretical.

We've already seen one: the style of writing inline cross-references
with the equivalent of @ref.  We also saw another: the way you
converted the menus.  It is quite clear to me that there will be
others.  So I'm not sure why you need more evidence that this could be
a real issue.

But maybe all of this is intentional: maybe the GCC project
consciously and deliberately decided to move away of the GNU
documentation style and conventions, and replace them with whatever
the Sphinx and RST conventions are?  In that case, there's no reason
for me to even mention these aspects.

Re: [PATCH] Port GCC documentation to Sphinx

[Martin Liška]

On 7/1/21 6:58 PM, Eli Zaretskii wrote:
>> Cc: joseph@codesourcery.com, gcc@gcc.gnu.org, gcc-patches@gcc.gnu.org
>> From: Martin Liška <mliska@suse.cz>
>> Date: Thu, 1 Jul 2021 18:04:24 +0200
>>
>>> Emacs doesn't hide the period.  But there shouldn't be a period to
>>> begin with, since it's the middle of a sentence.  The correct way of
>>> writing this in Texinfo is to have some punctuation: a comma or a
>>> semi-colon, after the closing brace, like this:
>>>
>>>     This is the warning level of @ref{e,,-Wshift-overflow3}, and …
>>
>> I don't see why we should put a comma after an option reference.
> 
> You explained it yourself later on:
> 
>> It's all related to Texinfo. Sphinx generates e.g.
>> Enabled by @ref{7,,-Wall} and something else.
>>
>> as documented here:
>> https://www.gnu.org/software/texinfo/manual/texinfo/html_node/_0040ref.html
>>
>> Then it ends with the following info output:
>>
>>        Enabled by *note -Wall: 7. and something else.
>>
>> So the period is added by Texinfo. If I put comma after a reference, then
>> the period is not added there.
>    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> 
> So the purpose of having the comma there is to avoid having a period
> in the middle of a sentence, which is added by makeinfo (because the
> Info readers need that).  Having a comma there may seem a bit
> redundant, but having a period will definitely look like a typo, if
> not confuse the heck out of the reader, especially if you want to use
> these inline cross-references so massively.

Well, then it's a bug in Makeinfo. If I want a comma (or a dot), I would
have put it these. So emitting 'Enabled by *note -Wall: 7. and something else.'
for 'Enabled by @ref{7,,-Wall} and something else.' seems bogus to me.

> 
>>> I don't think the GCC manuals should necessarily be bound by the
>>> Sphinx standards.  Where those standards are sub-optimal, it is
>>> perfectly okay for GCC (and other projects) to deviate.  GCC and other
>>> GNU manuals used a certain style and convention for decades, so
>>> there's more than enough experience and tradition to build on.
>>
>> What type of conversions and style are going to change with conversion to Sphinx?
> 
> Anything that is different from the style conventions described in the
> Texinfo manual.  We have many such conventions.

Which is supposed to be here:
https://www.gnu.org/prep/standards/html_node/Documentation.html#Documentation
right?

I've just the text. About the shortening of section names in a TOC. I couldn't find
it in the GNU Documentation manual.

> 
>> Do you see any of them worse than what we have now?
> 
> I didn't bother reading the Sphinx guidelines yet, and don't know when
> (and if) I will have time for that.  I do think the comparison should
> be part of the job or moving to Sphinx.
> 
>>> I will no longer pursue this point, but let me just say that I
>>> consider it a mistake to throw away all the experience collected using
>>> Texinfo just because Sphinx folks have other traditions and
>>> conventions.  It might be throwing the baby with the bathwater.
>>>
>>
>> Again, please show up concrete examples. What you describe is very theoretical.
> 
> We've already seen one: the style of writing inline cross-references
> with the equivalent of @ref.  We also saw another: the way you
> converted the menus.  It is quite clear to me that there will be
> others.  So I'm not sure why you need more evidence that this could be
> a real issue.

As explained, @ref are generated by Makeinfo in a strange way.
About the menus, I was unable to find it..

> 
> But maybe all of this is intentional: maybe the GCC project
> consciously and deliberately decided to move away of the GNU
> documentation style and conventions, and replace them with whatever
> the Sphinx and RST conventions are?  In that case, there's no reason
> for me to even mention these aspects.

My intention is preserving status quo as much as possible. Some constructs are different
in Sphinx and some Sphinx improvements (like option references) apparently bring up
next challenges to info manuals.

On the other hand, Sphinx provides quite some nice features why I wanted to use it.

Cheers,
Martin

Re: [PATCH] Port GCC documentation to Sphinx

[Martin Liška]

On 7/1/21 5:06 PM, Michael Matz wrote:
> Hello,
> 
> On Thu, 1 Jul 2021, Martin Liška wrote:
> 
>> On 7/1/21 3:33 PM, Eli Zaretskii wrote:
>>>> Cc: joseph@codesourcery.com, gcc@gcc.gnu.org, gcc-patches@gcc.gnu.org
>>>> From: Martin Liška <mliska@suse.cz>
>>>> Date: Thu, 1 Jul 2021 14:44:10 +0200
>>>>
>>>>> It helps some, but not all of the issues disappear.  For example,
>>>>> stuff like this is still hard to read:
>>>>>
>>>>>      To select this standard in GCC, use one of the options -ansi
>>>>>                                                             -----
>>>>>      -std.‘=c90’ or -std.‘=iso9899:1990’
>>>>>      ----           ----
>>>>
>>>> If I understand the notes correct, the '.' should be also hidden by e.g.
>>>> Emacs.
>>>
>>> No, it doesn't.  The actual text in the Info file is:
>>>
>>>      *note -std: f.‘=iso9899:1990’
>>>
>>> and the period after " f" isn't hidden.  Where does that "f" come from
>>> and what is its purpose here? can it be removed (together with the
>>> period)?
>>
>> It's name of the anchor used for the @ref. The names are automatically
>> generated
>> by makeinfo. So there's an example:
>>
>> This is the warning level of @ref{e,,-Wshift-overflow3} and …
>>
>> becomes in info:
>> This is the warning level of *note -Wshift-overflow3: e. and …
>>
>> I can ask the question at Sphinx, the Emacs script should hide that.
> 
> Not everyone reads info within Emacs; even if there's an emacs solution to
> postprocess info pages to make them nicer we can't rely on that.

Sure. What's new is that Sphinx can generate much more cross references, like
option references from option listing.

> It must
> look sensible without that.  In this case it seems that already the
> generated .texinfo input to makeinfo is bad, where does the 'e' (or 'f')
> come from?  The original texinfo file simply contains:

These are auto-numbered. Theoretically one can use the verbose anchor names:

@anchor{demo cmdoption-Wshift-overflow3}@anchor{e}@anchor{demo cmdoption-wshift-overflow3}@anchor{f}
@deffn {Option} @w{-}Wshift@w{-}overflow3=n, @w{-}Wshift@w{-}overflow3

Default option value for @ref{e,,-Wshift-overflow3}.

But these would lead to even longer '*note -Wshift-overflow3: demo cmdoption-wshift-overflow3' output.

> 
>    @option{-std=iso9899:1990}
> 
> so that's what perhaps should be generated, or maybe the anchor in @ref is
> optional and could be left out if it doesn't provide any info (a single
> character as anchor doesn't seem very useful)?

Yes, we can theoretically block emission of @refs for options.

Martin

> 
>>>>>> We can adjust 'emph' formatting to nil, what do you think?
>>>>>
>>>>> Something like that, yes.  But the problem is: how will you format it
>>>>> instead?  The known alternatives, _foo_ and *foo* both use punctuation
>>>>> characters, which will get in the way similarly to the quotes.  Can
>>>>> you format those in caps, like makeinfo does?
>>>>
>>>> You are fully right, info is very simple format and it uses wrapping for
>>>> the formatting
>>>> purpose (by default * and _). So, I don't have any elegant solution.
>>>
>>> Well, it sounds from another mail that you did find a solution: to
>>> up-case the string in @var.
>>
>> I don't know. Some of them can be e.g. keywords and using upper-case
>> does not seem to me feasible.
> 
> Then that needs to be different already in the input, so that the
> directive that (in info) capitalizes is only used in contexts where that
> makes sense.  People reading info pages will know that an all-caps word
> often means a syntactic variable/placeholder, so that should be preserved.
> 
> 
> Ciao,
> Michael.
> 

Re: [PATCH] Port GCC documentation to Sphinx

[Eli Zaretskii]

> Cc: joseph@codesourcery.com, gcc@gcc.gnu.org, gcc-patches@gcc.gnu.org
> From: Martin Liška <mliska@suse.cz>
> Date: Fri, 2 Jul 2021 11:30:02 +0200
> 
> > So the purpose of having the comma there is to avoid having a period
> > in the middle of a sentence, which is added by makeinfo (because the
> > Info readers need that).  Having a comma there may seem a bit
> > redundant, but having a period will definitely look like a typo, if
> > not confuse the heck out of the reader, especially if you want to use
> > these inline cross-references so massively.
> 
> Well, then it's a bug in Makeinfo.

No, it isn't a bug in makeinfo.  It's a (mis)feature of the Info
format: a cross-reference needs to have a punctuation character after
it, so that Info readers would know where's the end of the node/anchor
name to which the cross-reference points.  Info files are largely
plain-ASCII files, so the Info readers need help in this case, and
makeinfo produces what they need.

> >> What type of conversions and style are going to change with conversion to Sphinx?
> > 
> > Anything that is different from the style conventions described in the
> > Texinfo manual.  We have many such conventions.
> 
> Which is supposed to be here:
> https://www.gnu.org/prep/standards/html_node/Documentation.html#Documentation
> right?
> 
> I've just the text. About the shortening of section names in a TOC. I couldn't find
> it in the GNU Documentation manual.

No, there's also a lot of style guidelines in the Texinfo manual
itself.  Basically, the documentation of almost every Texinfo
directive includes some style guidelines, and there are also sections
which are pure guidelines, like the nodes "Conventions", "Node Names",
"Structuring Command Types", and some others.

> >> Again, please show up concrete examples. What you describe is very theoretical.
> > 
> > We've already seen one: the style of writing inline cross-references
> > with the equivalent of @ref.  We also saw another: the way you
> > converted the menus.  It is quite clear to me that there will be
> > others.  So I'm not sure why you need more evidence that this could be
> > a real issue.
> 
> As explained, @ref are generated by Makeinfo in a strange way.
> About the menus, I was unable to find it..

See the node "Menu Parts" in the Texinfo manual.  If you look at other
GNU manuals, you will see that it is a de-facto standard to provide
most menu items with short descriptions.

> > But maybe all of this is intentional: maybe the GCC project
> > consciously and deliberately decided to move away of the GNU
> > documentation style and conventions, and replace them with whatever
> > the Sphinx and RST conventions are?  In that case, there's no reason
> > for me to even mention these aspects.
> 
> My intention is preserving status quo as much as possible.

Well, but you definitely deviated from the status quo, and it sounds
like you did that deliberately, without any discussion.

> On the other hand, Sphinx provides quite some nice features why I wanted to use it.

Which features are those?

Re: [PATCH] Port GCC documentation to Sphinx

[Eli Zaretskii]

> Cc: Eli Zaretskii <eliz@gnu.org>, gcc@gcc.gnu.org, gcc-patches@gcc.gnu.org,
>  joseph@codesourcery.com
> From: Martin Liška <mliska@suse.cz>
> Date: Fri, 2 Jul 2021 11:40:06 +0200
> 
> > It must
> > look sensible without that.  In this case it seems that already the
> > generated .texinfo input to makeinfo is bad, where does the 'e' (or 'f')
> > come from?  The original texinfo file simply contains:
> 
> These are auto-numbered. Theoretically one can use the verbose anchor names:
> 
> @anchor{demo cmdoption-Wshift-overflow3}@anchor{e}@anchor{demo cmdoption-wshift-overflow3}@anchor{f}
> @deffn {Option} @w{-}Wshift@w{-}overflow3=n, @w{-}Wshift@w{-}overflow3
> 
> Default option value for @ref{e,,-Wshift-overflow3}.
> 
> But these would lead to even longer '*note -Wshift-overflow3: demo cmdoption-wshift-overflow3' output.

While auto-numbering is a nice feature, the human-readable anchors
have an advantage of hinting on the topic to which the cross-reference
points.

Re: [PATCH] Port GCC documentation to Sphinx

[Martin Liška]

On 7/2/21 12:31 PM, Eli Zaretskii wrote:
>> Cc: joseph@codesourcery.com, gcc@gcc.gnu.org, gcc-patches@gcc.gnu.org
>> From: Martin Liška <mliska@suse.cz>
>> Date: Fri, 2 Jul 2021 11:30:02 +0200
>>
>>> So the purpose of having the comma there is to avoid having a period
>>> in the middle of a sentence, which is added by makeinfo (because the
>>> Info readers need that).  Having a comma there may seem a bit
>>> redundant, but having a period will definitely look like a typo, if
>>> not confuse the heck out of the reader, especially if you want to use
>>> these inline cross-references so massively.
>>
>> Well, then it's a bug in Makeinfo.
> 
> No, it isn't a bug in makeinfo.  It's a (mis)feature of the Info
> format: a cross-reference needs to have a punctuation character after
> it, so that Info readers would know where's the end of the node/anchor
> name to which the cross-reference points.  Info files are largely
> plain-ASCII files, so the Info readers need help in this case, and
> makeinfo produces what they need.

Indeed. Agree that Info format is legacy format with very limited capability
of a formatting.

> 
>>>> What type of conversions and style are going to change with conversion to Sphinx?
>>>
>>> Anything that is different from the style conventions described in the
>>> Texinfo manual.  We have many such conventions.
>>
>> Which is supposed to be here:
>> https://www.gnu.org/prep/standards/html_node/Documentation.html#Documentation
>> right?
>>
>> I've just the text. About the shortening of section names in a TOC. I couldn't find
>> it in the GNU Documentation manual.
> 
> No, there's also a lot of style guidelines in the Texinfo manual
> itself.  Basically, the documentation of almost every Texinfo
> directive includes some style guidelines, and there are also sections
> which are pure guidelines, like the nodes "Conventions", "Node Names",
> "Structuring Command Types", and some others.

You are right, it's mentioned in "Node Names". Anyway, I declare it as a minor
regression to the current format.

> 
>>>> Again, please show up concrete examples. What you describe is very theoretical.
>>>
>>> We've already seen one: the style of writing inline cross-references
>>> with the equivalent of @ref.  We also saw another: the way you
>>> converted the menus.  It is quite clear to me that there will be
>>> others.  So I'm not sure why you need more evidence that this could be
>>> a real issue.
>>
>> As explained, @ref are generated by Makeinfo in a strange way.
>> About the menus, I was unable to find it..
> 
> See the node "Menu Parts" in the Texinfo manual.  If you look at other
> GNU manuals, you will see that it is a de-facto standard to provide
> most menu items with short descriptions.
> 
>>> But maybe all of this is intentional: maybe the GCC project
>>> consciously and deliberately decided to move away of the GNU
>>> documentation style and conventions, and replace them with whatever
>>> the Sphinx and RST conventions are?  In that case, there's no reason
>>> for me to even mention these aspects.
>>
>> My intention is preserving status quo as much as possible.
> 
> Well, but you definitely deviated from the status quo, and it sounds
> like you did that deliberately, without any discussion.
> 
>> On the other hand, Sphinx provides quite some nice features why I wanted to use it.
> 
> Which features are those?
> 

Feel free to read this email discussion, it's mentioned there multiple times what are
main benefits of the transition.

Cheers,
Martin

Re: [PATCH] Port GCC documentation to Sphinx

[Hans-Peter Nilsson]

On Wed, 30 Jun 2021, Eli Zaretskii via Gcc-patches wrote:
> > Cc: joseph@codesourcery.com, gcc@gcc.gnu.org, gcc-patches@gcc.gnu.org
> > From: Martin Li?ka <mliska@suse.cz>
> > Date: Wed, 30 Jun 2021 12:11:03 +0200
> > > 4. Menus lost the short descriptions of the sub-sections.  Example:
> > >
> > >    * Designated Initializers
> > >    * Case Ranges
> > >    * Cast to a Union Type
> > >    * Mixed Declarations, Labels and Code
> > >    * Declaring Attributes of Functions
> > >
> > > vs
> > >
> > >    * Designated Inits::    Labeling elements of initializers.
> > >    * Case Ranges::         'case 1 ... 9' and such.
> > >    * Cast to Union::       Casting to union type from any member of the union.
> > >    * Mixed Declarations::  Mixing declarations and code.
> > >    * Function Attributes:: Declaring that functions have no side effects,
> > > 			  or that they can never return.
> > >
> > > Looks like some bug to me.
> > >
> > > Note also that nodes are now called by the same name as the section,
> > > which means node names generally got much longer.  Is that really a
> > > good idea?
> >
> > Well, I intentionally removed these and used simple TOC tree links
> > which take display text for a section title.
>
> I would suggest to discuss these decisions first, perhaps on the
> Texinfo mailing list?  I'm accustomed to these short descriptions, but
> I'm not sure how important they are for others.

I'd miss them, and they're helpful when the title or term is not
familiar, when looking for a concept you know is documented in
there.

I've read the discussion downthread, but I seem to miss (a recap
of) the benefits of moving to Sphinx.  Maybe other have too and
it'd be a good idea to repeat them?  Otherwise, the impression
is not so good, as all I see is bits here and there getting lost
in translation.

brgds, H-P

Re: [PATCH] Port GCC documentation to Sphinx

[Richard Sandiford]

Hans-Peter Nilsson <hp@bitrange.com> writes:
> I've read the discussion downthread, but I seem to miss (a recap
> of) the benefits of moving to Sphinx.  Maybe other have too and
> it'd be a good idea to repeat them?  Otherwise, the impression
> is not so good, as all I see is bits here and there getting lost
> in translation.

Better cross-referencing is one big feature.  IMO this subthread has
demonstrated why the limitations of info formatting have held back
the amount of cross-referencing in the online html.  (And based on
emperical evidence, I get the impression that far more people use
the online html docs than the info docs.)

E.g. quoting from Richard's recent patch:

  @item -fmove-loop-stores
  @opindex fmove-loop-stores
  Enables the loop store motion pass in the GIMPLE loop optimizer.  This
  moves invariant stores to after the end of the loop in exchange for
  carrying the stored value in a register across the iteration.
  Note for this option to have an effect @option{-ftree-loop-im} has to 
  be enabled as well.  Enabled at level @option{-O1} and higher, except 
  for @option{-Og}.

In the online docs, this will just be plain text.  Anyone who doesn't
know what -ftree-loop-im is will have to search for it manually.

Adding the extra references to the html (and pdf) output but dropping
them from the info sounds like a good compromise.

Thanks,
Richard

Re: [PATCH] Port GCC documentation to Sphinx

[Eli Zaretskii]

> From: Richard Sandiford <richard.sandiford@arm.com>
> Cc: Eli Zaretskii <eliz@gnu.org>,  gcc@gcc.gnu.org,  gcc-patches@gcc.gnu.org,  joseph@codesourcery.com
> Date: Mon, 05 Jul 2021 10:17:38 +0100
> 
> Hans-Peter Nilsson <hp@bitrange.com> writes:
> > I've read the discussion downthread, but I seem to miss (a recap
> > of) the benefits of moving to Sphinx.  Maybe other have too and
> > it'd be a good idea to repeat them?  Otherwise, the impression
> > is not so good, as all I see is bits here and there getting lost
> > in translation.
> 
> Better cross-referencing is one big feature.

See below: the Info format has some features in addition to
cross-references that can make this a much smaller issue.  HTML has
just the cross-references, so "when you are a hammer, every problem
looks like a nail".

> IMO this subthread has demonstrated why the limitations of info
> formatting have held back the amount of cross-referencing in the
> online html.

I disagree with this conclusion, see below.

> (And based on emperical evidence, I get the impression that far more
> people use the online html docs than the info docs.)

HTML browsers currently lack some features that make Info the format
of choice for me when I need to use the documentation efficiently.
The most important feature I miss in HTML browsers is the index
search.  A good manual usually has extensive index (or indices) which
make it very easy to find a specific topic one is looking for,
i.e. use the manual as a reference (as opposed as a first-time
reading, when you read large portions of the manual in sequence).

Another important feature is regexp search across multiple sections
(with HTML you'd be forced to download the manual as a single large
file for that, and then you'll probably miss regexps).

Yet another feature which, when needed, is something to kill for, is
the "info apropos" command, which can search all the manuals on your
system and build a menu from the matching sections found in different
manuals.  And there are a few more.

(Texinfo folks are working on JavaScript code to add some missing
capabilities to Web browsers, but that effort is not yet complete.)

> E.g. quoting from Richard's recent patch:
> 
>   @item -fmove-loop-stores
>   @opindex fmove-loop-stores
>   Enables the loop store motion pass in the GIMPLE loop optimizer.  This
>   moves invariant stores to after the end of the loop in exchange for
>   carrying the stored value in a register across the iteration.
>   Note for this option to have an effect @option{-ftree-loop-im} has to 
>   be enabled as well.  Enabled at level @option{-O1} and higher, except 
>   for @option{-Og}.
> 
> In the online docs, this will just be plain text.  Anyone who doesn't
> know what -ftree-loop-im is will have to search for it manually.

First, even if there are no cross-references, manual search is not the
best way.  It is much easier to use index-search:

  i ftree TAB

will display a list of options that you could be after, and you can
simply choose from the list, or type a bit more until you have a
single match.

Moreover, adding cross-references is easy:

  @item -fmove-loop-stores
  @opindex fmove-loop-stores
  Enables the loop store motion pass in the GIMPLE loop optimizer.  This
  moves invariant stores to after the end of the loop in exchange for
  carrying the stored value in a register across the iteration.
  Note for this option to have an effect @option{-ftree-loop-im}
  (@pxref{Optimize Options, -ftree-loop-im}) 
  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  has be enabled as well.  Enabled at level @option{-O1} and higher,
  except for @option{-Og}.

If this looks like too much work, a simple Texinfo macro (two, if you
want an anchor where you point) will do.

> Adding the extra references to the html (and pdf) output but dropping
> them from the info sounds like a good compromise.

But that's not what happens.  And besides, how would you decide which
cross-references to drop and which to retain in Info?

Re: [PATCH] Port GCC documentation to Sphinx

[Richard Sandiford]

Eli Zaretskii <eliz@gnu.org> writes:
>> Hans-Peter Nilsson <hp@bitrange.com> writes:
>> > I've read the discussion downthread, but I seem to miss (a recap
>> > of) the benefits of moving to Sphinx.  Maybe other have too and
>> > it'd be a good idea to repeat them?  Otherwise, the impression
>> > is not so good, as all I see is bits here and there getting lost
>> > in translation.
>> 
>> Better cross-referencing is one big feature.
>
> See below: the Info format has some features in addition to
> cross-references that can make this a much smaller issue.  HTML has
> just the cross-references, so "when you are a hammer, every problem
> looks like a nail".
>
>> IMO this subthread has demonstrated why the limitations of info
>> formatting have held back the amount of cross-referencing in the
>> online html.
>
> I disagree with this conclusion, see below.
>
>> (And based on emperical evidence, I get the impression that far more
>> people use the online html docs than the info docs.)
>
> HTML browsers currently lack some features that make Info the format
> of choice for me when I need to use the documentation efficiently.
> The most important feature I miss in HTML browsers is the index
> search.  A good manual usually has extensive index (or indices) which
> make it very easy to find a specific topic one is looking for,
> i.e. use the manual as a reference (as opposed as a first-time
> reading, when you read large portions of the manual in sequence).
>
> Another important feature is regexp search across multiple sections
> (with HTML you'd be forced to download the manual as a single large
> file for that, and then you'll probably miss regexps).
>
> Yet another feature which, when needed, is something to kill for, is
> the "info apropos" command, which can search all the manuals on your
> system and build a menu from the matching sections found in different
> manuals.  And there are a few more.
>
> (Texinfo folks are working on JavaScript code to add some missing
> capabilities to Web browsers, but that effort is not yet complete.)

Whether info or HTML is the better format isn't the issue though.  The
point is that we do have HTML output that is (emperically) widely used.
And at the moment it isn't as good as it could be.

The question that I was replying to was: what is the benefit of moving
to Sphinx?  And one of the answers is that it improves the HTML output.

>> E.g. quoting from Richard's recent patch:
>> 
>>   @item -fmove-loop-stores
>>   @opindex fmove-loop-stores
>>   Enables the loop store motion pass in the GIMPLE loop optimizer.  This
>>   moves invariant stores to after the end of the loop in exchange for
>>   carrying the stored value in a register across the iteration.
>>   Note for this option to have an effect @option{-ftree-loop-im} has to 
>>   be enabled as well.  Enabled at level @option{-O1} and higher, except 
>>   for @option{-Og}.
>> 
>> In the online docs, this will just be plain text.  Anyone who doesn't
>> know what -ftree-loop-im is will have to search for it manually.
>
> First, even if there are no cross-references, manual search is not the
> best way.  It is much easier to use index-search:
>
>   i ftree TAB
>
> will display a list of options that you could be after, and you can
> simply choose from the list, or type a bit more until you have a
> single match.

Here too I was talking about this being plain text in the online docs,
i.e. in the HTML.

In HTML the user-friendly way of letting users answer the question
“what on earth is -ftree-loop-im” is to have “-ftree-loop-im” be a
hyperlink that goes straight to the documentation of the option.
Same for PDF when viewed digitally.

One of the things that the move to Sphinx does is give us those
hyperlinks.

> Moreover, adding cross-references is easy:
>
>   @item -fmove-loop-stores
>   @opindex fmove-loop-stores
>   Enables the loop store motion pass in the GIMPLE loop optimizer.  This
>   moves invariant stores to after the end of the loop in exchange for
>   carrying the stored value in a register across the iteration.
>   Note for this option to have an effect @option{-ftree-loop-im}
>   (@pxref{Optimize Options, -ftree-loop-im}) 
>   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>   has be enabled as well.  Enabled at level @option{-O1} and higher,
>   except for @option{-Og}.
>
> If this looks like too much work, a simple Texinfo macro (two, if you
> want an anchor where you point) will do.

But this would be redundant in HTML: “foo (see foo)”.

Also, the benefit of hyperlinks in HTML (not info) is that they can be
used outside of prose, such as in lists, without interrupting the flow.

>> Adding the extra references to the html (and pdf) output but dropping
>> them from the info sounds like a good compromise.
>
> But that's not what happens.

Not in the original patch, sure, but it's what I think Martin was
suggesting as a compromise (maybe I misunderstood).  The comment above
was supposed to be in support of doing that.

It sounds like those who use the info format (which includes me btw,
at least occasionally) are happy with the status quo in terms of which
links are present.  So the new links could be a non-info-only feature.

Thanks,
Richard

Benefits of using Sphinx documentation format

[Martin Liška]

Hello.

Let's make it a separate sub-thread where we can discuss motivation why
do I want moving to Sphinx format.

Benefits:
1) modern looking HTML output (before: [1], after: [2]):
    a) syntax highlighting for examples (code, shell commands, etc.)
    b) precise anchors, the current Texinfo anchors are not displayed (start with first line of an option)
    c) one can easily copy a link to an anchor (displayed as ¶)
    d) internal links are working, e.g. one can easily jump from listing of options
    e) left menu navigation provides better orientation in the manual
    f) Sphinx provides internal search capability: [3]
2) internal links are also provided in PDF version of the manual
3) some existing GCC manuals are already written in Sphinx (GNAT manuals and libgccjit)
4) support for various output formats, some people are interested in ePUB format
5) Sphinx is using RST which is quite minimal semantic markup language
6) TOC is automatically generated - no need for manual navigation like seen here: [5]

Disadvantages:

1) info pages are currently missing Page description in TOC
2) rich formatting is leading to extra wrapping in info output - beings partially addresses in [4]
3) one needs e.g. Emacs support for inline links (rendered as notes)

I'm willing to address issue 1) in next weeks and I tend to skip emission of links as mentioned in 3).
Generally speaking, I'm aware that some people still use Info, but I think we should more focus
on more modern documentation formats. That's HTML (and partially PDF).

Martin

[1] https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html#index-fstrict-aliasing
[2] https://splichal.eu/gccsphinx-final/html/gcc/gcc-command-options/options-that-control-optimization.html#cmdoption-fstrict-aliasing
[3] https://splichal.eu/gccsphinx-final/html/gcc/search.html?q=-fipa-icf&check_keywords=yes&area=default#
[4] https://github.com/sphinx-doc/sphinx/pull/9391
[5] @comment node-name,     next,          previous, up
     @node    Installing GCC, Binaries, , Top

Re: Benefits of using Sphinx documentation format

[Eli Zaretskii]

> Cc: gcc@gcc.gnu.org, gcc-patches@gcc.gnu.org, joseph@codesourcery.com
> From: Martin Liška <mliska@suse.cz>
> Date: Mon, 12 Jul 2021 15:25:47 +0200
> 
> Let's make it a separate sub-thread where we can discuss motivation why
> do I want moving to Sphinx format.

Thanks for starting this discussion.

> Benefits:
> 1) modern looking HTML output (before: [1], after: [2]):
>     a) syntax highlighting for examples (code, shell commands, etc.)
>     b) precise anchors, the current Texinfo anchors are not displayed (start with first line of an option)
>     c) one can easily copy a link to an anchor (displayed as ¶)
>     d) internal links are working, e.g. one can easily jump from listing of options
>     e) left menu navigation provides better orientation in the manual
>     f) Sphinx provides internal search capability: [3]
> 2) internal links are also provided in PDF version of the manual

How is this different from Texinfo?

> 3) some existing GCC manuals are already written in Sphinx (GNAT manuals and libgccjit)
> 4) support for various output formats, some people are interested in ePUB format

Texinfo likewise supports many output formats.  Someone presented a
very simple package to produce epub format from it.

> 5) Sphinx is using RST which is quite minimal semantic markup language

Is it more minimal than Texinfo?

> 6) TOC is automatically generated - no need for manual navigation like seen here: [5]

That is not needed in Texinfo as well, since long ago.  Nowadays, you
just say

  @node Whatever

and the rest is done automatically, as long as the manual's structure
is a proper tree (which it normally is, I know of only one manual that
is an exception).

> Disadvantages:
> 
> 1) info pages are currently missing Page description in TOC
> 2) rich formatting is leading to extra wrapping in info output - beings partially addresses in [4]
> 3) one needs e.g. Emacs support for inline links (rendered as notes)

 4) The need to learn yet another markup language.
    While this is not a problem for simple text, it does require a
    serious study of RST and Sphinx to use the more advanced features.

 5) Lack of macros.
    AFAIK, only simple textual substitution is available, no macros
    with arguments.

Re: Benefits of using Sphinx documentation format

[Jonathan Wakely]

On Mon, 12 Jul 2021 at 14:41, Eli Zaretskii via Gcc <gcc@gcc.gnu.org> wrote:
>
> > Cc: gcc@gcc.gnu.org, gcc-patches@gcc.gnu.org, joseph@codesourcery.com
> > From: Martin Liška <mliska@suse.cz>
> > Date: Mon, 12 Jul 2021 15:25:47 +0200
> >
> > Let's make it a separate sub-thread where we can discuss motivation why
> > do I want moving to Sphinx format.
>
> Thanks for starting this discussion.
>
> > Benefits:
> > 1) modern looking HTML output (before: [1], after: [2]):
> >     a) syntax highlighting for examples (code, shell commands, etc.)
> >     b) precise anchors, the current Texinfo anchors are not displayed (start with first line of an option)
> >     c) one can easily copy a link to an anchor (displayed as ¶)
> >     d) internal links are working, e.g. one can easily jump from listing of options

For me, these items are enough justification to switch away from
texinfo, which produces crap HTML pages with crap anchors. You can't
find out the anchors without inspecting (and searching) the HTML
source. That's utterly stupid. And even after you do that, the anchor
is at the wrong place:
https://gcc.gnu.org/onlinedocs/gcc/Overall-Options.html#index-c
As somebody who spends a lot of time helping users on the mailing
list, IRC, stackoverflow, and elsewhere, this "feature" of the texinfo
HTML has angered me for many years.

Yes, some people like texinfo, but some people also dislike it and
there are serious usability problems with the output. I support
replacing texinfo with anything that isn't texinfo.


> >     e) left menu navigation provides better orientation in the manual
> >     f) Sphinx provides internal search capability: [3]
> > 2) internal links are also provided in PDF version of the manual
>
> How is this different from Texinfo?
>
> > 3) some existing GCC manuals are already written in Sphinx (GNAT manuals and libgccjit)
> > 4) support for various output formats, some people are interested in ePUB format
>
> Texinfo likewise supports many output formats.  Someone presented a
> very simple package to produce epub format from it.
>
> > 5) Sphinx is using RST which is quite minimal semantic markup language
>
> Is it more minimal than Texinfo?
>
> > 6) TOC is automatically generated - no need for manual navigation like seen here: [5]
>
> That is not needed in Texinfo as well, since long ago.  Nowadays, you
> just say
>
>   @node Whatever
>
> and the rest is done automatically, as long as the manual's structure
> is a proper tree (which it normally is, I know of only one manual that
> is an exception).
>
> > Disadvantages:
> >
> > 1) info pages are currently missing Page description in TOC
> > 2) rich formatting is leading to extra wrapping in info output - beings partially addresses in [4]
> > 3) one needs e.g. Emacs support for inline links (rendered as notes)
>
>  4) The need to learn yet another markup language.
>     While this is not a problem for simple text, it does require a
>     serious study of RST and Sphinx to use the more advanced features.

This is a problem with texinfo too.

>
>  5) Lack of macros.
>     AFAIK, only simple textual substitution is available, no macros
>     with arguments.

Is this a problem for GCC docs though?

Re: Benefits of using Sphinx documentation format

[Jonathan Wakely]

On Mon, 12 Jul 2021 at 14:53, Jonathan Wakely wrote:
> For me, these items are enough justification to switch away from
> texinfo, which produces crap HTML pages with crap anchors. You can't
> find out the anchors without inspecting (and searching) the HTML
> source. That's utterly stupid. And even after you do that, the anchor
> is at the wrong place:
> https://gcc.gnu.org/onlinedocs/gcc/Overall-Options.html#index-c
> As somebody who spends a lot of time helping users on the mailing
> list, IRC, stackoverflow, and elsewhere, this "feature" of the texinfo
> HTML has angered me for many years.

To be clear, I give links to users frequently (several times a week,
every week, for decades) and prefer to give them a link to specific
options. Obviously I link to the online HTML docs rather than telling
them an 'info' command to run, because most people don't use info
pages or know how to navigate them. That means I can't provide decent
links, because the actual option name I'm trying to link to is always
off the top of the page. This is simply unacceptable IMHO. Texinfo
must go.

Re: Benefits of using Sphinx documentation format

[Eli Zaretskii]

> From: Jonathan Wakely <jwakely.gcc@gmail.com>
> Date: Mon, 12 Jul 2021 14:53:44 +0100
> Cc: Martin Liška <mliska@suse.cz>, 
> 	"gcc@gcc.gnu.org" <gcc@gcc.gnu.org>, gcc-patches <gcc-patches@gcc.gnu.org>, 
> 	"Joseph S. Myers" <joseph@codesourcery.com>
> 
> For me, these items are enough justification to switch away from
> texinfo, which produces crap HTML pages with crap anchors.

If we want to have a serious discussion with useful conclusions, I
suggest to avoid "loaded" terminology.

I get it that you dislike the HTML produced by Texinfo, but without
some examples of such bad HTML it is impossible to know what exactly
do you dislike and why.

> You can't find out the anchors without inspecting (and searching)
> the HTML source. That's utterly stupid.

I don't think I follow: find out the anchors with which means and for
what purposes?

> And even after you do that, the anchor
> is at the wrong place:
> https://gcc.gnu.org/onlinedocs/gcc/Overall-Options.html#index-c

IME, the anchor is where you put it.  If you show me the source of
that HTMl, maybe we can have a more useful discussion of the issue.

> As somebody who spends a lot of time helping users on the mailing
> list, IRC, stackoverflow, and elsewhere, this "feature" of the texinfo
> HTML has angered me for many years.

As somebody who spends a lot of time helping users on every possible
forum, and as someone who has wrote a lot of Texinfo, I don't
understand what angers you.  Please elaborate.

> Yes, some people like texinfo, but some people also dislike it and
> there are serious usability problems with the output. I support
> replacing texinfo with anything that isn't texinfo.

"Anything"?  Even plain text?  I hope not.

See, such "arguments" don't help to have a useful discussion.

> >  4) The need to learn yet another markup language.
> >     While this is not a problem for simple text, it does require a
> >     serious study of RST and Sphinx to use the more advanced features.
> 
> This is a problem with texinfo too.

Not for someone who already knows Texinfo.  We are talking about
switching away of it, so I'm thinking about people who contributed
patches for the manual in the past.  They already know Texinfo, at
least to some extent, and some of them know it very well.

> >  5) Lack of macros.
> >     AFAIK, only simple textual substitution is available, no macros
> >     with arguments.
> 
> Is this a problem for GCC docs though?

I don't know.  It could be, even if it isn't now.

Re: Benefits of using Sphinx documentation format

[Eli Zaretskii]

> From: Jonathan Wakely <jwakely.gcc@gmail.com>
> Date: Mon, 12 Jul 2021 15:05:11 +0100
> Cc: Martin Liška <mliska@suse.cz>, 
> 	"gcc@gcc.gnu.org" <gcc@gcc.gnu.org>, gcc-patches <gcc-patches@gcc.gnu.org>, 
> 	"Joseph S. Myers" <joseph@codesourcery.com>
> 
> To be clear, I give links to users frequently (several times a week,
> every week, for decades) and prefer to give them a link to specific
> options. Obviously I link to the online HTML docs rather than telling
> them an 'info' command to run, because most people don't use info
> pages or know how to navigate them. That means I can't provide decent
> links, because the actual option name I'm trying to link to is always
> off the top of the page. This is simply unacceptable IMHO. Texinfo
> must go.

"Texinfo must go" is one possible conclusion from your description.
But it isn't the only one.  An alternative is "the Texinfo source of
the GCC manual must be improved to fix this problem."  And yes, this
problem does have a solution in Texinfo.

Re: Benefits of using Sphinx documentation format

[Martin Liška]

On 7/12/21 4:12 PM, Eli Zaretskii wrote:
>> From: Jonathan Wakely <jwakely.gcc@gmail.com>
>> Date: Mon, 12 Jul 2021 14:53:44 +0100
>> Cc: Martin Liška <mliska@suse.cz>,
>> 	"gcc@gcc.gnu.org" <gcc@gcc.gnu.org>, gcc-patches <gcc-patches@gcc.gnu.org>,
>> 	"Joseph S. Myers" <joseph@codesourcery.com>
>>
>> For me, these items are enough justification to switch away from
>> texinfo, which produces crap HTML pages with crap anchors.
> 
> If we want to have a serious discussion with useful conclusions, I
> suggest to avoid "loaded" terminology.
> 
> I get it that you dislike the HTML produced by Texinfo, but without
> some examples of such bad HTML it is impossible to know what exactly
> do you dislike and why.

Please follow my 1) from Benefits and *read* bullet points a) to f). That will
give you an answer.

> 
>> You can't find out the anchors without inspecting (and searching)
>> the HTML source. That's utterly stupid.
> 
> I don't think I follow: find out the anchors with which means and for
> what purposes?

Benefits, 1c).

> 
>> And even after you do that, the anchor
>> is at the wrong place:
>> https://gcc.gnu.org/onlinedocs/gcc/Overall-Options.html#index-c
> 
> IME, the anchor is where you put it.  If you show me the source of
> that HTMl, maybe we can have a more useful discussion of the issue.

Problem is that Texinfo emits poor HTML where # link points to a wrong place.
Open the given page, view source and search for <dd><a name="index-c"></a>.

> 
>> As somebody who spends a lot of time helping users on the mailing
>> list, IRC, stackoverflow, and elsewhere, this "feature" of the texinfo
>> HTML has angered me for many years.
> 
> As somebody who spends a lot of time helping users on every possible
> forum, and as someone who has wrote a lot of Texinfo, I don't
> understand what angers you.  Please elaborate.

You can't point to an option documentation.

> 
>> Yes, some people like texinfo, but some people also dislike it and
>> there are serious usability problems with the output. I support
>> replacing texinfo with anything that isn't texinfo.
> 
> "Anything"?  Even plain text?  I hope not.
> 
> See, such "arguments" don't help to have a useful discussion.
> 
>>>   4) The need to learn yet another markup language.
>>>      While this is not a problem for simple text, it does require a
>>>      serious study of RST and Sphinx to use the more advanced features.
>>
>> This is a problem with texinfo too.
> 
> Not for someone who already knows Texinfo.  We are talking about
> switching away of it, so I'm thinking about people who contributed
> patches for the manual in the past.  They already know Texinfo, at
> least to some extent, and some of them know it very well.

Yes, people will have to learn a new syntax. Similarly to transition of SVN,
people also had to learn with a more modern tool.

> 
>>>   5) Lack of macros.
>>>      AFAIK, only simple textual substitution is available, no macros
>>>      with arguments.
>>
>> Is this a problem for GCC docs though?
> 
> I don't know.  It could be, even if it isn't now.

Then it's not an argument, sorry.

Martin

Re: Benefits of using Sphinx documentation format

[Martin Liška]

On 7/12/21 4:16 PM, Eli Zaretskii wrote:
>> From: Jonathan Wakely <jwakely.gcc@gmail.com>
>> Date: Mon, 12 Jul 2021 15:05:11 +0100
>> Cc: Martin Liška <mliska@suse.cz>,
>> 	"gcc@gcc.gnu.org" <gcc@gcc.gnu.org>, gcc-patches <gcc-patches@gcc.gnu.org>,
>> 	"Joseph S. Myers" <joseph@codesourcery.com>
>>
>> To be clear, I give links to users frequently (several times a week,
>> every week, for decades) and prefer to give them a link to specific
>> options. Obviously I link to the online HTML docs rather than telling
>> them an 'info' command to run, because most people don't use info
>> pages or know how to navigate them. That means I can't provide decent
>> links, because the actual option name I'm trying to link to is always
>> off the top of the page. This is simply unacceptable IMHO. Texinfo
>> must go.
> 
> "Texinfo must go" is one possible conclusion from your description.
> But it isn't the only one.  An alternative is "the Texinfo source of
> the GCC manual must be improved to fix this problem."  And yes, this
> problem does have a solution in Texinfo.

No, the alternative is more powerful output given by Texinfo, in particular
more modern HTML pages.

Martin

Re: Benefits of using Sphinx documentation format

[Martin Liška]

On 7/12/21 3:39 PM, Eli Zaretskii wrote:
>> Cc: gcc@gcc.gnu.org, gcc-patches@gcc.gnu.org, joseph@codesourcery.com
>> From: Martin Liška <mliska@suse.cz>
>> Date: Mon, 12 Jul 2021 15:25:47 +0200
>>
>> Let's make it a separate sub-thread where we can discuss motivation why
>> do I want moving to Sphinx format.
> 
> Thanks for starting this discussion.
> 
>> Benefits:
>> 1) modern looking HTML output (before: [1], after: [2]):
>>      a) syntax highlighting for examples (code, shell commands, etc.)
>>      b) precise anchors, the current Texinfo anchors are not displayed (start with first line of an option)
>>      c) one can easily copy a link to an anchor (displayed as ¶)
>>      d) internal links are working, e.g. one can easily jump from listing of options
>>      e) left menu navigation provides better orientation in the manual
>>      f) Sphinx provides internal search capability: [3]
>> 2) internal links are also provided in PDF version of the manual
> 
> How is this different from Texinfo?

Texinfo does not emit them. See e.g. links in option listing (-O2, -Os, ...).

> 
>> 3) some existing GCC manuals are already written in Sphinx (GNAT manuals and libgccjit)
>> 4) support for various output formats, some people are interested in ePUB format
> 
> Texinfo likewise supports many output formats.  Someone presented a
> very simple package to produce epub format from it.

Good to know.

> 
>> 5) Sphinx is using RST which is quite minimal semantic markup language
> 
> Is it more minimal than Texinfo?

I would say that's pretty easy to learn, similarly complex as Texinfo.

> 
>> 6) TOC is automatically generated - no need for manual navigation like seen here: [5]
> 
> That is not needed in Texinfo as well, since long ago.  Nowadays, you
> just say
> 
>    @node Whatever
> 
> and the rest is done automatically, as long as the manual's structure
> is a proper tree (which it normally is, I know of only one manual that
> is an exception).

All right, then we likely do an extra work right now.

> 
>> Disadvantages:
>>
>> 1) info pages are currently missing Page description in TOC
>> 2) rich formatting is leading to extra wrapping in info output - beings partially addresses in [4]
>> 3) one needs e.g. Emacs support for inline links (rendered as notes)
> 
>   4) The need to learn yet another markup language.
>      While this is not a problem for simple text, it does require a
>      serious study of RST and Sphinx to use the more advanced features.

No, majority of the documentation is pretty simple: basic formatting, links, tables and
code examples.

Martin

> 
>   5) Lack of macros.
>      AFAIK, only simple textual substitution is available, no macros
>      with arguments.
> 

Re: Benefits of using Sphinx documentation format

[Jonathan Wakely]

On Mon, 12 Jul 2021 at 15:13, Eli Zaretskii <eliz@gnu.org> wrote:
>
> > From: Jonathan Wakely <jwakely.gcc@gmail.com>
> > Date: Mon, 12 Jul 2021 14:53:44 +0100
> > Cc: Martin Liška <mliska@suse.cz>,
> >       "gcc@gcc.gnu.org" <gcc@gcc.gnu.org>, gcc-patches <gcc-patches@gcc.gnu.org>,
> >       "Joseph S. Myers" <joseph@codesourcery.com>
> >
> > For me, these items are enough justification to switch away from
> > texinfo, which produces crap HTML pages with crap anchors.
>
> If we want to have a serious discussion with useful conclusions, I
> suggest to avoid "loaded" terminology.

But the results *are* crap.

>
> I get it that you dislike the HTML produced by Texinfo, but without
> some examples of such bad HTML it is impossible to know what exactly
> do you dislike and why.
>
> > You can't find out the anchors without inspecting (and searching)
> > the HTML source. That's utterly stupid.
>
> I don't think I follow: find out the anchors with which means and for
> what purposes?

I want to point a user at the documentation for the -c option. I can't
do that without examining the HTML source to find the anchor, then
manually editing the URL to append the anchor. It's a tedious process,
and the result is an anchor that doesn't even point to the option but
to the text following it. The process is unnecessarily difficult and
the results are bad.

You participated in a discussion about this very topic previously:
https://lists.gnu.org/archive/html/help-texinfo/2019-02/msg00000.html

>
> > And even after you do that, the anchor
> > is at the wrong place:
> > https://gcc.gnu.org/onlinedocs/gcc/Overall-Options.html#index-c
>
> IME, the anchor is where you put it.  If you show me the source of
> that HTMl, maybe we can have a more useful discussion of the issue.

@item -c
@opindex c
Compile or assemble the source files, but do not link.  The linking
stage simply is not done.  The ultimate output is in the form of an
object file for each source file.

Putting the @opindex before the @item causes the anchor to be placed
on the previous item, which is not desirable.


>
> > As somebody who spends a lot of time helping users on the mailing
> > list, IRC, stackoverflow, and elsewhere, this "feature" of the texinfo
> > HTML has angered me for many years.
>
> As somebody who spends a lot of time helping users on every possible
> forum, and as someone who has wrote a lot of Texinfo, I don't
> understand what angers you.  Please elaborate.

I don't know what part of my email you don't understand. The HTML
anchors that texinfo creates are in the wrong place, and not
"discoverable". If you don't understand that, then you're clearly not
using the GCC HTML docs, and so I'm not surprised you think there's no
reason to ditch texinfo. As a regular user of the HTML (for myself and
end users of GCC), the HTML output has major usability problems.


> > Yes, some people like texinfo, but some people also dislike it and
> > there are serious usability problems with the output. I support
> > replacing texinfo with anything that isn't texinfo.
>
> "Anything"?  Even plain text?  I hope not.

Plain text with a tool to generate good HTML might be better than texinfo.

> See, such "arguments" don't help to have a useful discussion.

Your insistence that texinfo is fine doesn't either. It's not fine.

> > >  4) The need to learn yet another markup language.
> > >     While this is not a problem for simple text, it does require a
> > >     serious study of RST and Sphinx to use the more advanced features.
> >
> > This is a problem with texinfo too.
>
> Not for someone who already knows Texinfo.  We are talking about
> switching away of it, so I'm thinking about people who contributed
> patches for the manual in the past.  They already know Texinfo, at
> least to some extent, and some of them know it very well.

I've contributed dozens of patches to the manual, and I don't want to
have to use texinfo to do it in future.

> > >  5) Lack of macros.
> > >     AFAIK, only simple textual substitution is available, no macros
> > >     with arguments.
> >
> > Is this a problem for GCC docs though?
>
> I don't know.  It could be, even if it isn't now.

So not a problem then.

Re: Benefits of using Sphinx documentation format

[Jonathan Wakely]

On Mon, 12 Jul 2021 at 15:52, Jonathan Wakely <jwakely.gcc@gmail.com> wrote:
>
> On Mon, 12 Jul 2021 at 15:13, Eli Zaretskii <eliz@gnu.org> wrote:
> >
> > > From: Jonathan Wakely <jwakely.gcc@gmail.com>
> > > This is a problem with texinfo too.
> >
> > Not for someone who already knows Texinfo.  We are talking about
> > switching away of it, so I'm thinking about people who contributed
> > patches for the manual in the past.  They already know Texinfo, at
> > least to some extent, and some of them know it very well.
>
> I've contributed dozens of patches to the manual, and I don't want to
> have to use texinfo to do it in future.

And some of the people already know sphinx, and some know it very
well. And it seems likely that future contributors are more likely to
know a more modern tool than they are to know texinfo.

You like texinfo. We get it.

Re: Benefits of using Sphinx documentation format

[Jonathan Wakely]

On Mon, 12 Jul 2021 at 15:52, Jonathan Wakely <jwakely.gcc@gmail.com> wrote:
>
> On Mon, 12 Jul 2021 at 15:13, Eli Zaretskii <eliz@gnu.org> wrote:
> > I get it that you dislike the HTML produced by Texinfo, but without
> > some examples of such bad HTML it is impossible to know what exactly
> > do you dislike and why.
> >
> > > You can't find out the anchors without inspecting (and searching)
> > > the HTML source. That's utterly stupid.
> >
> > I don't think I follow: find out the anchors with which means and for
> > what purposes?
>
> I want to point a user at the documentation for the -c option. I can't
> do that without examining the HTML source to find the anchor, then
> manually editing the URL to append the anchor. It's a tedious process,
> and the result is an anchor that doesn't even point to the option but
> to the text following it. The process is unnecessarily difficult and
> the results are bad.
>
> You participated in a discussion about this very topic previously:
> https://lists.gnu.org/archive/html/help-texinfo/2019-02/msg00000.html
>
> >
> > > And even after you do that, the anchor
> > > is at the wrong place:
> > > https://gcc.gnu.org/onlinedocs/gcc/Overall-Options.html#index-c
> >
> > IME, the anchor is where you put it.  If you show me the source of
> > that HTMl, maybe we can have a more useful discussion of the issue.
>
> @item -c
> @opindex c
> Compile or assemble the source files, but do not link.  The linking
> stage simply is not done.  The ultimate output is in the form of an
> object file for each source file.
>
> Putting the @opindex before the @item causes the anchor to be placed
> on the previous item, which is not desirable.

GNU Hello has the same problem with its docs:
https://www.gnu.org/software/hello/manual/hello.html#index-_002dg
That URL is garbage because of the URL-encoded %2d character, and the
fact it links to the wrong place (the description of the option, not
the option itself). The former is no longer an issue for GCC (it was
for many years) but the latter is still a problem.

If you don't know where to find it yourself, the source is visible here:
https://github.com/yugui/example/blob/master/doc/hello.texi#L208

If GNU Hello and GCC can't get this right using texinfo, maybe texinfo
is not fit for purpose?

Re: Benefits of using Sphinx documentation format

[Gavin Smith]

On Mon, Jul 12, 2021 at 4:04 PM Jonathan Wakely via Gcc <gcc@gcc.gnu.org> wrote:
> GNU Hello has the same problem with its docs:
> https://www.gnu.org/software/hello/manual/hello.html#index-_002dg
> That URL is garbage because of the URL-encoded %2d character, and the
> fact it links to the wrong place (the description of the option, not
> the option itself). The former is no longer an issue for GCC (it was
> for many years) but the latter is still a problem.
>
> If you don't know where to find it yourself, the source is visible here:
> https://github.com/yugui/example/blob/master/doc/hello.texi#L208

I downloaded the source for the "hello" manual and recreated it with
Texinfo 6.8 (running " texi2any --html hello.texi --no-split"). I've
attached the results. The current output doesn't exhibit the problem
with the scrolling being at the wrong place - this problem has
evidently resolved itself since the time when the online "hello"
manual was generated. (I don't remember many complaints about it on
the mailing list, though: if we don't know about problems, we can't
fix them.)

The URL is mangled because index entries can have more characters in
them than what is suitable for a URL. A space character becomes a "-",
so a "-" has to become something else. They have to be distinguished
because there may be two separate index entries in different places
which wouldn't be distinguishable otherwise.

However, I find that adding an extra index entry means you can use
hello.html#index-greeting instead:

@item --greeting=@var{text}
@itemx -g @var{text}
@opindex greeting
@opindex --greeting
@opindex -g
Output @var{text} instead of the default greeting.

Re: Benefits of using Sphinx documentation format

[Jonathan Wakely]

On Mon, 12 Jul 2021 at 17:01, Gavin Smith wrote:
>
> On Mon, Jul 12, 2021 at 4:04 PM Jonathan Wakely via Gcc <gcc@gcc.gnu.org> wrote:
> > GNU Hello has the same problem with its docs:
> > https://www.gnu.org/software/hello/manual/hello.html#index-_002dg
> > That URL is garbage because of the URL-encoded %2d character, and the
> > fact it links to the wrong place (the description of the option, not
> > the option itself). The former is no longer an issue for GCC (it was
> > for many years) but the latter is still a problem.
> >
> > If you don't know where to find it yourself, the source is visible here:
> > https://github.com/yugui/example/blob/master/doc/hello.texi#L208
>
> I downloaded the source for the "hello" manual and recreated it with
> Texinfo 6.8 (running " texi2any --html hello.texi --no-split"). I've
> attached the results. The current output doesn't exhibit the problem
> with the scrolling being at the wrong place - this problem has
> evidently resolved itself since the time when the online "hello"
> manual was generated. (I don't remember many complaints about it on
> the mailing list, though: if we don't know about problems, we can't
> fix them.)

The "copyable link" does work as I would expect. The #index-_002dg
anchor still seems to be in the "wrong" place, i.e. in the <dd>
element not the <dt> element. But the addition of the copyable link
nicely solves the problem of needing to easily obtain a link to the
right position.

> The URL is mangled because index entries can have more characters in
> them than what is suitable for a URL. A space character becomes a "-",
> so a "-" has to become something else.

Yes, I understand the reason.

Re: Benefits of using Sphinx documentation format

[David Malcolm]

On Mon, 2021-07-12 at 15:25 +0200, Martin Liška wrote:
> Hello.
> 
> Let's make it a separate sub-thread where we can discuss motivation
> why
> do I want moving to Sphinx format.
> 
> Benefits:
> 1) modern looking HTML output (before: [1], after: [2]):

"modern looking" is rather subjective; I'd rate Sphinx's output as
looking like it's from 2010s (last decade), whereas Texinfos' looks
like it's from the 1990s.  In theory this ought not to matter, but
every time I look at our documentation it gives me a depressing
feeling, reminiscent of a graveyard, that discourages me from fixing
things.

>     a) syntax highlighting for examples (code, shell commands, etc.)

...with support for multiple programming languages, potentially on the
same page.  For example, in the libgccjit docs:
  https://gcc.gnu.org/onlinedocs/jit/intro/tutorial02.html
we can have a mixture of C, assembler and shell on one page, and each
example is syntax-highlighted accordingly.  It's not clear to me how to
do that in texinfo, since there needs to be a way to express what
language an example is in.

>     b) precise anchors, the current Texinfo anchors are not displayed
> (start with first line of an option)

...and the URLs are sane and stable (so e.g. there is a reliable,
guessable, readable URL for the docs for say, "-Wall").

>     c) one can easily copy a link to an anchor (displayed as ¶)
>     d) internal links are working, e.g. one can easily jump from
> listing of options
>     e) left menu navigation provides better orientation in the manual
>     f) Sphinx provides internal search capability: [3]

...also (quoting myself in places here from 2015
  https://gcc.gnu.org/pipermail/gcc-patches/2015-November/434055.html 
):

* the ability to include fragments of files: libgccjit's documentation
uses directives to include code from the test suite, so that all of the
code examples are also part of the test suite, and are thus known to
compile), allowing for (almost) literate programming.  [That said, the
build of libgccjit's docs on gcc.gnu.org seems to be missing those
fragments; I wonder if there's a path or version issue?]

* a page-splitting structure that make sense, to me, at least (I have
never fathomed the way texinfo's navigation works, for HTML, at least,
and I believe I'm not the only one; I generally pick the all-in-one-
HTML-page option when viewing texinfo-html docs and do textual
searches, since otherwise I usually can't find the thing I'm looking
for (or have to resort to a brute-force depth-first search of clicking
through the links).)

* much more use of markup, with restrained and well-chosen CSS
(texinfo's HTML seems to ignore much of the inline markup in
the .texinfo file)

> 2) internal links are also provided in PDF version of the manual
> 3) some existing GCC manuals are already written in Sphinx (GNAT
> manuals and libgccjit)
> 4) support for various output formats, some people are interested in
> ePUB format
> 5) Sphinx is using RST which is quite minimal semantic markup language

Sphinx is also used by many high-profile FLOSS projects (e.g. the Linux
kernel, LLVM, and the Python community), so it reduces the barrier to
entry for new contributors, relative to texinfo.


> 6) TOC is automatically generated - no need for manual navigation
> like seen here: [5]
> 
> Disadvantages:
> 
> 1) info pages are currently missing Page description in TOC
> 2) rich formatting is leading to extra wrapping in info output -
> beings partially addresses in [4]
> 3) one needs e.g. Emacs support for inline links (rendered as notes)
> 
> I'm willing to address issue 1) in next weeks and I tend to skip
> emission of links as mentioned in 3).
> Generally speaking, I'm aware that some people still use Info, but I
> think we should more focus
> on more modern documentation formats. That's HTML (and partially
> PDF).

I think the output formats we need to support are:
- HTML
- PDF
- man page (hardly "modern", but still used)

I regared "info" as merely "nice to have" - I don't know anyone who
uses it other than some core GNU contributors.

Dave

> 
> Martin
> 
> [1]    
> https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html#index-fstrict-aliasing
> [2]    
> https://splichal.eu/gccsphinx-final/html/gcc/gcc-command-options/options-that-control-optimization.html#cmdoption-fstrict-aliasing
> [3]    
> https://splichal.eu/gccsphinx-final/html/gcc/search.html?q=-fipa-icf&check_keywords=yes&area=default#
> [4] https://github.com/sphinx-doc/sphinx/pull/9391
> [5] @comment node-name,     next,          previous, up
>      @node    Installing GCC, Binaries, , Top
> 

Re: Benefits of using Sphinx documentation format

[Eli Zaretskii]

> Cc: gcc@gcc.gnu.org, gcc-patches@gcc.gnu.org, joseph@codesourcery.com
> From: Martin Liška <mliska@suse.cz>
> Date: Mon, 12 Jul 2021 16:34:11 +0200
> 
> > "Texinfo must go" is one possible conclusion from your description.
> > But it isn't the only one.  An alternative is "the Texinfo source of
> > the GCC manual must be improved to fix this problem."  And yes, this
> > problem does have a solution in Texinfo.
> 
> No, the alternative is more powerful output given by Texinfo, in particular
> more modern HTML pages.

Please see the response by Gavin: it sounds like at least some of that
was resolved in Texinfo, sometimes long ago.

Re: Benefits of using Sphinx documentation format

[Eli Zaretskii]

> Cc: hp@bitrange.com, gcc@gcc.gnu.org, gcc-patches@gcc.gnu.org,
>  joseph@codesourcery.com
> From: Martin Liška <mliska@suse.cz>
> Date: Mon, 12 Jul 2021 16:37:00 +0200
> 
> >   4) The need to learn yet another markup language.
> >      While this is not a problem for simple text, it does require a
> >      serious study of RST and Sphinx to use the more advanced features.
> 
> No, majority of the documentation is pretty simple: basic formatting, links, tables and
> code examples.

We also have documentation of APIs (a.k.a. "functions").  I actually
tried to find in the Sphinx docs how to do that and got lost.  So, not
really "very simple".

Re: Benefits of using Sphinx documentation format

[Eli Zaretskii]

> From: Jonathan Wakely <jwakely.gcc@gmail.com>
> Date: Mon, 12 Jul 2021 15:54:49 +0100
> Cc: Martin Liška <mliska@suse.cz>, 
> 	"gcc@gcc.gnu.org" <gcc@gcc.gnu.org>, gcc-patches <gcc-patches@gcc.gnu.org>, 
> 	"Joseph S. Myers" <joseph@codesourcery.com>
> 
> You like texinfo. We get it.

Would you please drop the attitude?

Re: [PATCH] Port GCC documentation to Sphinx

[Martin Sebor]

On 6/29/21 4:09 AM, Martin Liška wrote:
> On 6/28/21 5:33 PM, Joseph Myers wrote:
>> Are formatted manuals (HTML, PDF, man, info) corresponding to this patch
>> version also available for review?
> 
> I've just uploaded them here:
> https://splichal.eu/gccsphinx-final/
> 
> Martin
> 

I think listing the -Wfoo and -Wno-foo (and analogously the -fbar
and -fno-bar) options is an improvement over prior revisions but when
the positive form is the default the text reads funny.  For example:

   -fno-inline

       Do not expand any functions inline apart from those marked
       with the always_inline attribute. This is the default when
       not optimizing.

       Single functions can be exempted from inlining by marking
       them with the noinline attribute.

   -finline

       Default option value for -fno-inline.


I.e., -finline is not what I would describe as a default value for
-fno-inline.

I would suggest to drop the option name from the text describing
the default, and also replace value with setting (it's really not
a value).  It could be as simple as:

   -finline

       Default setting.

Alternatively, to preserve the connection to the alternate setting:

   -finline

       Default setting; overrides -fno-inline.

At some point we talked about also making attributes hyperlinks
(like always_inline and noinline above) but I don't remember
the conclusion.  Are you planning to do that?  (Would handling
it as part of the transition be easier than doing it later?)

Martin

Re: Benefits of using Sphinx documentation format

[Koning, Paul]

> On Jul 12, 2021, at 12:36 PM, David Malcolm via Gcc-patches <gcc-patches@gcc.gnu.org> wrote:
> 
> On Mon, 2021-07-12 at 15:25 +0200, Martin Liška wrote:
>> ...
> 
> I think the output formats we need to support are:
> - HTML
> - PDF
> - man page (hardly "modern", but still used)

Also info format (for the Emacs info reader).  And ebook formats (epub and/or mobi).  Having good quality ebook output is a major benefit in my view; it would be very good for the standard makefiles to offer make targets for these formats.

	paul

RE: [PATCH] Port GCC documentation to Sphinx

[Tamar Christina]

Hi Martin,

> -----Original Message-----
> From: Gcc-patches <gcc-patches-
> bounces+tamar.christina=arm.com@gcc.gnu.org> On Behalf Of Martin Liška
> Sent: Tuesday, June 29, 2021 11:09 AM
> To: Joseph Myers <joseph@codesourcery.com>
> Cc: GCC Development <gcc@gcc.gnu.org>; gcc-patches@gcc.gnu.org
> Subject: Re: [PATCH] Port GCC documentation to Sphinx
> 
> On 6/28/21 5:33 PM, Joseph Myers wrote:
> > Are formatted manuals (HTML, PDF, man, info) corresponding to this
> > patch version also available for review?
> 
> I've just uploaded them here:
> https://splichal.eu/gccsphinx-final/
> 

Thanks for doing this 😊

I'm a primary user of the PDFs (easier to work offline) I do like the look and syntax highlighting of the new PDFs,
But I do prefer the way the current itemized entries are laid out.

See for instance ` vect_interleave` which before would have the description indented on the next line, vs the new docs which puts it on the same line.

The visual break in my opinion makes it easier to read.  It currently looks like a sea of text.  Also purely personal I expect, but could the weight of the bold entries be reduced a bit? They look very BOLD to me atm and when there's a lot of them I find it slightly harder to read.

Cheers,
Tamar

> Martin

Re: [PATCH] Port GCC documentation to Sphinx

[Martin Liška]

On 7/12/21 7:18 PM, Martin Sebor wrote:
> On 6/29/21 4:09 AM, Martin Liška wrote:
>> On 6/28/21 5:33 PM, Joseph Myers wrote:
>>> Are formatted manuals (HTML, PDF, man, info) corresponding to this patch
>>> version also available for review?
>>
>> I've just uploaded them here:
>> https://splichal.eu/gccsphinx-final/
>>
>> Martin
>>
> 
> I think listing the -Wfoo and -Wno-foo (and analogously the -fbar
> and -fno-bar) options is an improvement over prior revisions but when
> the positive form is the default the text reads funny.  For example:

Thank you!

> 
>    -fno-inline
> 
>        Do not expand any functions inline apart from those marked
>        with the always_inline attribute. This is the default when
>        not optimizing.
> 
>        Single functions can be exempted from inlining by marking
>        them with the noinline attribute.
> 
>    -finline
> 
>        Default option value for -fno-inline.
> 
> 
> I.e., -finline is not what I would describe as a default value for
> -fno-inline.
> 
> I would suggest to drop the option name from the text describing
> the default, and also replace value with setting (it's really not
> a value).  It could be as simple as:
> 
>    -finline
> 
>        Default setting.
> 
> Alternatively, to preserve the connection to the alternate setting:
> 
>    -finline
> 
>        Default setting; overrides -fno-inline.

I like the improvement suggestion and I've just adjusted
the generated output to 'Default setting; overrides -fno-inline'.

> 
> At some point we talked about also making attributes hyperlinks
> (like always_inline and noinline above) but I don't remember
> the conclusion.  Are you planning to do that?  (Would handling
> it as part of the transition be easier than doing it later?)

I've provided more links for attributes. However, there are still limitations
that needs to be addressed:
1) I used :option: directive which might be a bit abused (one can eventually
    come up with a custom directive.
2) Attributes like 'alias ("target")' do not correctly work with :option:`alias`,
    so the option definition needs to changed to 'alias' and we would have to provide
    syntax description on a next line. It's something I plan working on we merge it.

Martin

> 
> Martin

Re: [PATCH] Port GCC documentation to Sphinx

[Martin Liška]

On 7/13/21 4:54 PM, Tamar Christina wrote:
> Hi Martin,
> 
>> -----Original Message-----
>> From: Gcc-patches <gcc-patches-
>> bounces+tamar.christina=arm.com@gcc.gnu.org> On Behalf Of Martin Liška
>> Sent: Tuesday, June 29, 2021 11:09 AM
>> To: Joseph Myers <joseph@codesourcery.com>
>> Cc: GCC Development <gcc@gcc.gnu.org>; gcc-patches@gcc.gnu.org
>> Subject: Re: [PATCH] Port GCC documentation to Sphinx
>>
>> On 6/28/21 5:33 PM, Joseph Myers wrote:
>>> Are formatted manuals (HTML, PDF, man, info) corresponding to this
>>> patch version also available for review?
>>
>> I've just uploaded them here:
>> https://splichal.eu/gccsphinx-final/
>>
> 
> Thanks for doing this 😊
> 
> I'm a primary user of the PDFs (easier to work offline) I do like the look and syntax highlighting of the new PDFs,
> But I do prefer the way the current itemized entries are laid out.
> 
> See for instance ` vect_interleave` which before would have the description indented on the next line, vs the new docs which puts it on the same line.

Very useful comment. Generally when I look at gccint documentation, majority of documented "terms" are marked as @items in the Texinfo.
In Sphinx, we should rather use a proper concrete type, like:

:c:member:
:c:data:
:c:var:
:c:func:
:c:macro:
:c:struct:
:c:union:
:c:enum:
:c:enumerator:
:c:type:

These would improve the documentation when searching for something. I've got it listed for after migration phase.

> 
> The visual break in my opinion makes it easier to read.  It currently looks like a sea of text.  Also purely personal I expect, but could the weight of the bold entries be reduced a bit? They look very BOLD to me atm and when there's a lot of them I find it slightly harder to read.

That would be about a font selection for Latex, can you please experiment with:
https://www.sphinx-doc.org/en/master/latex.html#the-latex-elements-configuration-setting
?

Thanks,
Martin

> 
> Cheers,
> Tamar
> 
>> Martin
> 

Re: GCC documentation: porting to Sphinx

[Martin Liška]

Hello.

I've just pushed the rebased branch here:
https://gcc.gnu.org/git/gitweb.cgi?p=gcc.git;a=log;h=refs/users/marxin/heads/sphinx-v4

which I force push once I rebase it. One can fetch the branch with:
$ git fetch origin refs/users/marxin/heads/sphinx-v4

Generated output (directly made from GCC source tree) can be seen here:
https://splichal.eu/gccsphinx-final/

Changes since the previous version:

1) rebased on the current master (including addition of new target hooks, etc.)
2) two new directive/roles were added in order to not abuse 'option' directive:
    gcc-attr (function/label/... attribute) and gcc-param (--param foo=bar).
    Addition was quite straightforward and we would benefit as these attributes
    and parameters will be listed grouped in the Index:
    https://splichal.eu/gccsphinx-final/html/gcc/genindex.html
3) default syntax language was set to 'none'; made issue for e.g. chunks in license pages
    where a random Python keywords were highlighted

What needs to be done (TODO list):

1) Cross manual references are missing - we have some of them and Sphinx has nice support for it:
    https://docs.readthedocs.io/en/stable/guides/intersphinx.html
2) Remove `Texinfo Manuals` section in GCC internal manual
3) Document required packages for PDF, MAN, HTML, ..
4) Write How to write documentation, we can take inspiration from Kernel:
    https://www.kernel.org/doc/html/latest/doc-guide/index.html
5) Update update_web_docs_git - that will simply run Sphinx' Makefile
6) URL emission code needs to be changed in diagnostics.c
7) link emission should be ignored in Info builder
8) epub target should be added to Makefiles
9) function/struct/type attribute definition should be simplified as
    :gcc-attr: attr_name ("options")
    does not work with a reference to it: :gcc-atr:`attr_name`
    An attribute format should be placed into the attr description.
10) default domain should be switched to cpp, will lead to warnings as seen here:
     https://github.com/sphinx-doc/sphinx/issues/9535
11) many function and macros in gccint should promoted to '.. function::' and
     '.. macro::' directive
12) various smaller formatting issues need to be addressed

Known limitations:
1) chapter description (in TOC listing) is dropped in info pages
2) PDF output puts item description on the same line as item definition - noticed by Tamar

As previously mentioned, I'm willing to work on the majority of the points mentioned in the TODO list.
Now I see the current patchset in a reasonable shape and I'm asking the community for approval of the changes?
And I would appreciate a help with any of the items listed in the TODO list.

Thanks,
Martin

Re: GCC documentation: porting to Sphinx

[Martin Liška]

On 8/10/21 17:43, Martin Liška wrote:
> Hello.
> 
> I've just pushed the rebased branch here:
> https://gcc.gnu.org/git/gitweb.cgi?p=gcc.git;a=log;h=refs/users/marxin/heads/sphinx-v4
> 
> which I force push once I rebase it. One can fetch the branch with:
> $ git fetch origin refs/users/marxin/heads/sphinx-v4

Hello.

There's updated version of the patch set sitting here:
refs/users/marxin/heads/sphinx-v5

> 
> Generated output (directly made from GCC source tree) can be seen here:
> https://splichal.eu/gccsphinx-final/

And can be seen here.

> 
> Changes since the previous version:
> 
> 1) rebased on the current master (including addition of new target hooks, etc.)
> 2) two new directive/roles were added in order to not abuse 'option' directive:
>     gcc-attr (function/label/... attribute) and gcc-param (--param foo=bar).
>     Addition was quite straightforward and we would benefit as these attributes
>     and parameters will be listed grouped in the Index:
>     https://splichal.eu/gccsphinx-final/html/gcc/genindex.html
> 3) default syntax language was set to 'none'; made issue for e.g. chunks in license pages
>     where a random Python keywords were highlighted

Changes since the previous version:
1) Cross manual references are working. It works surprisingly well and we have much more cross references now
(for things like options, ...).
2) I have a new version of update_web_docs_git that will be very simple:
    make -C doc html/pdf SOURCEDIR=... BUILDDIR=...
3) URL link creating was updated in diagnostics.c
4) I have a patch candidate that skips links in Info format:
    https://github.com/sphinx-doc/sphinx/pull/9578
5) default domain was switched to cpp and Sphinx community fixed various issues mentioned in:
    https://github.com/sphinx-doc/sphinx/issues/9535
6) I made one round of proof-reading of the manuals where I focused on the formatting issues

> 
> What needs to be done (TODO list):
> 
> 1) Cross manual references are missing - we have some of them and Sphinx has nice support for it:
>     https://docs.readthedocs.io/en/stable/guides/intersphinx.html
> 2) Remove `Texinfo Manuals` section in GCC internal manual
> 3) Document required packages for PDF, MAN, HTML, ..
> 4) Write How to write documentation, we can take inspiration from Kernel:
>     https://www.kernel.org/doc/html/latest/doc-guide/index.html
> 5) Update update_web_docs_git - that will simply run Sphinx' Makefile
> 6) URL emission code needs to be changed in diagnostics.c
> 7) link emission should be ignored in Info builder
> 8) epub target should be added to Makefiles
> 9) function/struct/type attribute definition should be simplified as
>     :gcc-attr: attr_name ("options")
>     does not work with a reference to it: :gcc-atr:`attr_name`
>     An attribute format should be placed into the attr description.
> 10) default domain should be switched to cpp, will lead to warnings as seen here:
>      https://github.com/sphinx-doc/sphinx/issues/9535
> 11) many function and macros in gccint should promoted to '.. function::' and
>      '.. macro::' directive
> 12) various smaller formatting issues need to be addressed

What needs to be done (TODO list):

1) Remove `Texinfo Manuals` section in GCC internal manual
2) Document required packages for PDF, MAN, HTML, ..
3) Write How to write documentation, we can take inspiration from Kernel:
     https://www.kernel.org/doc/html/latest/doc-guide/index.html
4) epub target should be added to Makefiles
5) function/struct/type attribute definition should be simplified as
     :gcc-attr: attr_name ("options")
     does not work with a reference to it: :gcc-atr:`attr_name`
     An attribute format should be placed into the attr description.
6) many function and macros in gccint should promoted to '.. function::' and
    '.. macro::' directive (partialy done)

Thoughts about current status of the migration process?

Thanks,
Martin

> 
> Known limitations:
> 1) chapter description (in TOC listing) is dropped in info pages
> 2) PDF output puts item description on the same line as item definition - noticed by Tamar
> 
> As previously mentioned, I'm willing to work on the majority of the points mentioned in the TODO list.
> Now I see the current patchset in a reasonable shape and I'm asking the community for approval of the changes?
> And I would appreciate a help with any of the items listed in the TODO list.
> 
> Thanks,
> Martin