[PATCH] doc: Various fixes to makedocbook for python3.8

[PATCH] doc: Various fixes to makedocbook for python3.8

[Jon Turney]

---
 newlib/doc/makedocbook.py | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/newlib/doc/makedocbook.py b/newlib/doc/makedocbook.py
index 92d0f279d..3fab26f1a 100755
--- a/newlib/doc/makedocbook.py
+++ b/newlib/doc/makedocbook.py
@@ -199,6 +199,8 @@ def function(c, l):
     namelist = map(lambda v: re.sub('^and ', '', v.strip(), 1), namelist)
     # strip off << >> surrounding name
     namelist = map(lambda v: v.strip().lstrip('<').rstrip('>'), namelist)
+    # instantiate list to make it subscriptable
+    namelist = list(namelist)
 
     if verbose:
         print(namelist, file=sys.stderr)
@@ -262,7 +264,7 @@ def index(c, l):
     primary.text = l
 
     # to validate, it seems we need to maintain refentry elements in a certain order
-    refentry[:] = sorted(refentry, key = lambda x: x.tag)
+    refentry[:] = sorted(refentry, key = lambda x: x.tag if isinstance(x.tag, str) else '')
 
     # adds another alternate refname
     refnamediv = refentry.find('refnamediv')
@@ -807,7 +809,7 @@ def main(file):
     perform(processed)
 
     # output the XML tree
-    s = lxml.etree.tostring(rootelement, pretty_print=True)
+    s = lxml.etree.tostring(rootelement, pretty_print=True, encoding='unicode')
 
     if not s:
         print('No output produced (perhaps the input has no makedoc markup?)', file=sys.stderr)
@@ -826,7 +828,7 @@ def main(file):
 
 if __name__ == '__main__' :
     options = OptionParser()
-    options.add_option('-v', '--verbose', action='count', dest = 'verbose')
+    options.add_option('-v', '--verbose', action='count', dest = 'verbose', default = 0)
     options.add_option('-c', '--cache', action='store_true', dest = 'cache', help="just ensure PLY cache is up to date")
     (opts, args) = options.parse_args()
 
-- 
2.28.0

Re: [PATCH] doc: Various fixes to makedocbook for python3.8

[Ken Brown]

On 8/22/2020 2:45 PM, Jon Turney wrote:
> ---
>   newlib/doc/makedocbook.py | 8 +++++---
>   1 file changed, 5 insertions(+), 3 deletions(-)
> 
> diff --git a/newlib/doc/makedocbook.py b/newlib/doc/makedocbook.py
> index 92d0f279d..3fab26f1a 100755
> --- a/newlib/doc/makedocbook.py
> +++ b/newlib/doc/makedocbook.py
> @@ -199,6 +199,8 @@ def function(c, l):
>       namelist = map(lambda v: re.sub('^and ', '', v.strip(), 1), namelist)
>       # strip off << >> surrounding name
>       namelist = map(lambda v: v.strip().lstrip('<').rstrip('>'), namelist)
> +    # instantiate list to make it subscriptable
> +    namelist = list(namelist)
>   
>       if verbose:
>           print(namelist, file=sys.stderr)
> @@ -262,7 +264,7 @@ def index(c, l):
>       primary.text = l
>   
>       # to validate, it seems we need to maintain refentry elements in a certain order
> -    refentry[:] = sorted(refentry, key = lambda x: x.tag)
> +    refentry[:] = sorted(refentry, key = lambda x: x.tag if isinstance(x.tag, str) else '')
>   
>       # adds another alternate refname
>       refnamediv = refentry.find('refnamediv')
> @@ -807,7 +809,7 @@ def main(file):
>       perform(processed)
>   
>       # output the XML tree
> -    s = lxml.etree.tostring(rootelement, pretty_print=True)
> +    s = lxml.etree.tostring(rootelement, pretty_print=True, encoding='unicode')
>   
>       if not s:
>           print('No output produced (perhaps the input has no makedoc markup?)', file=sys.stderr)
> @@ -826,7 +828,7 @@ def main(file):
>   
>   if __name__ == '__main__' :
>       options = OptionParser()
> -    options.add_option('-v', '--verbose', action='count', dest = 'verbose')
> +    options.add_option('-v', '--verbose', action='count', dest = 'verbose', default = 0)
>       options.add_option('-c', '--cache', action='store_true', dest = 'cache', help="just ensure PLY cache is up to date")
>       (opts, args) = options.parse_args()
>   
> 

Would it make sense to also change the shebang line so that makedocbook.py uses 
python3?  Currently the build of Cygwin on Fedora uses python3, but the build of 
Cygwin on Cygwin uses python2.  This is of no great importance, but a recent IRC 
discussion shows that it can be confusing.

Ken

Re: [PATCH] doc: Various fixes to makedocbook for python3.8

[Jon Turney]

On 23/08/2020 16:23, Ken Brown wrote:
> On 8/22/2020 2:45 PM, Jon Turney wrote:
>> ---
>>   newlib/doc/makedocbook.py | 8 +++++---
>>   1 file changed, 5 insertions(+), 3 deletions(-)
>>
>> diff --git a/newlib/doc/makedocbook.py b/newlib/doc/makedocbook.py
>> index 92d0f279d..3fab26f1a 100755
>> --- a/newlib/doc/makedocbook.py
>> +++ b/newlib/doc/makedocbook.py
[...]
> Would it make sense to also change the shebang line so that 
> makedocbook.py uses python3?  Currently the build of Cygwin on Fedora 
> uses python3, but the build of Cygwin on Cygwin uses python2.  This is 
> of no great importance, but a recent IRC discussion shows that it can be 
> confusing.

Yeah, I guess that would make some sense, given that python2 is now EOL.

Otoh, making it gratuitously not work with python2 seems a bit harsh.

On the gripping hand, reading PEP 0394, I see it doesn't actually 
require that 'python' exist at all, so I guess changing the shebang as 
you suggest is the right thing to do.

Re: [PATCH] doc: Various fixes to makedocbook for python3.8

[Brian Inglis]

On 2020-08-23 13:41, Jon Turney wrote:
> On 23/08/2020 16:23, Ken Brown wrote:
>> On 8/22/2020 2:45 PM, Jon Turney wrote:
>>> ---
>>>   newlib/doc/makedocbook.py | 8 +++++---
>>>   1 file changed, 5 insertions(+), 3 deletions(-)
>>>
>>> diff --git a/newlib/doc/makedocbook.py b/newlib/doc/makedocbook.py
>>> index 92d0f279d..3fab26f1a 100755
>>> --- a/newlib/doc/makedocbook.py
>>> +++ b/newlib/doc/makedocbook.py
> [...]
>> Would it make sense to also change the shebang line so that makedocbook.py
>> uses python3?  Currently the build of Cygwin on Fedora uses python3, but the
>> build of Cygwin on Cygwin uses python2.  This is of no great importance, but a
>> recent IRC discussion shows that it can be confusing.
> 
> Yeah, I guess that would make some sense, given that python2 is now EOL.
> 
> Otoh, making it gratuitously not work with python2 seems a bit harsh.
> 
> On the gripping hand, reading PEP 0394, I see it doesn't actually require that
> 'python' exist at all, so I guess changing the shebang as you suggest is the
> right thing to do.

Is it not perhaps time to switch python from EOL python2 to python3, as there
are only 183 python2, 125 python27 to 199 python3, 129 python37 packages.

We are deemphasising Cygwin 32 and no longer promote it as equal to Cygwin 64,
but are far from calling it EOL, whereas not even security or severe bug fixes
will be considered, never mind distributed, for python2: its death was announced
10 years ago, confirmed 5 years ago, complete 4 months ago, it is now static and
potentially unsafe at 2.7.18, and its estimate of Napier's constant e is as
accurate as it will ever get! ;^>

-- 
Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada

This email may be disturbing to some readers as it contains
too much technical detail. Reader discretion is advised.
[Data in IEC units and prefixes, physical quantities in SI.]

Re: [PATCH] doc: Various fixes to makedocbook for python3.8

[Corinna Vinschen]

On Aug 23 20:41, Jon Turney wrote:
> On 23/08/2020 16:23, Ken Brown wrote:
> > On 8/22/2020 2:45 PM, Jon Turney wrote:
> > > ---
> > >   newlib/doc/makedocbook.py | 8 +++++---
> > >   1 file changed, 5 insertions(+), 3 deletions(-)
> > > 
> > > diff --git a/newlib/doc/makedocbook.py b/newlib/doc/makedocbook.py
> > > index 92d0f279d..3fab26f1a 100755
> > > --- a/newlib/doc/makedocbook.py
> > > +++ b/newlib/doc/makedocbook.py
> [...]
> > Would it make sense to also change the shebang line so that
> > makedocbook.py uses python3?  Currently the build of Cygwin on Fedora
> > uses python3, but the build of Cygwin on Cygwin uses python2.  This is
> > of no great importance, but a recent IRC discussion shows that it can be
> > confusing.
> 
> Yeah, I guess that would make some sense, given that python2 is now EOL.
> 
> Otoh, making it gratuitously not work with python2 seems a bit harsh.
> 
> On the gripping hand, reading PEP 0394, I see it doesn't actually require
> that 'python' exist at all, so I guess changing the shebang as you suggest
> is the right thing to do.

With this change, please push.


Thanks,
Corinna

Re: [PATCH] doc: Various fixes to makedocbook for python3.8

[Corinna Vinschen]

On Aug 23 16:49, Brian Inglis wrote:
> On 2020-08-23 13:41, Jon Turney wrote:
> > On 23/08/2020 16:23, Ken Brown wrote:
> >> On 8/22/2020 2:45 PM, Jon Turney wrote:
> >>> ---
> >>>   newlib/doc/makedocbook.py | 8 +++++---
> >>>   1 file changed, 5 insertions(+), 3 deletions(-)
> >>>
> >>> diff --git a/newlib/doc/makedocbook.py b/newlib/doc/makedocbook.py
> >>> index 92d0f279d..3fab26f1a 100755
> >>> --- a/newlib/doc/makedocbook.py
> >>> +++ b/newlib/doc/makedocbook.py
> > [...]
> >> Would it make sense to also change the shebang line so that makedocbook.py
> >> uses python3?  Currently the build of Cygwin on Fedora uses python3, but the
> >> build of Cygwin on Cygwin uses python2.  This is of no great importance, but a
> >> recent IRC discussion shows that it can be confusing.
> > 
> > Yeah, I guess that would make some sense, given that python2 is now EOL.
> > 
> > Otoh, making it gratuitously not work with python2 seems a bit harsh.
> > 
> > On the gripping hand, reading PEP 0394, I see it doesn't actually require that
> > 'python' exist at all, so I guess changing the shebang as you suggest is the
> > right thing to do.
> 
> Is it not perhaps time to switch python from EOL python2 to python3, as there
> are only 183 python2, 125 python27 to 199 python3, 129 python37 packages.

This discussion actually belongs on cygwin-apps, not on newlib ¯\_(ツ)_/¯


Thanks,
Corinna

Re: [PATCH] doc: Various fixes to makedocbook for python3.8

[Jon Turney]

[-- Attachment #1: Type: text/plain, Size: 1310 bytes --]

On 24/08/2020 11:01, Corinna Vinschen via Newlib wrote:
> On Aug 23 20:41, Jon Turney wrote:
>> On 23/08/2020 16:23, Ken Brown wrote:
>>> On 8/22/2020 2:45 PM, Jon Turney wrote:
>>>> ---
>>>>    newlib/doc/makedocbook.py | 8 +++++---
>>>>    1 file changed, 5 insertions(+), 3 deletions(-)
>>>>
>>>> diff --git a/newlib/doc/makedocbook.py b/newlib/doc/makedocbook.py
>>>> index 92d0f279d..3fab26f1a 100755
>>>> --- a/newlib/doc/makedocbook.py
>>>> +++ b/newlib/doc/makedocbook.py
>> [...]
>>> Would it make sense to also change the shebang line so that
>>> makedocbook.py uses python3?  Currently the build of Cygwin on Fedora
>>> uses python3, but the build of Cygwin on Cygwin uses python2.  This is
>>> of no great importance, but a recent IRC discussion shows that it can be
>>> confusing.
>>
>> Yeah, I guess that would make some sense, given that python2 is now EOL.
>>
>> Otoh, making it gratuitously not work with python2 seems a bit harsh.
>>
>> On the gripping hand, reading PEP 0394, I see it doesn't actually require
>> that 'python' exist at all, so I guess changing the shebang as you suggest
>> is the right thing to do.
> 
> With this change, please push.

Testing on system where '/usr/bi/python' doesn't exist, there's also 
another script which I neglected to update.

Patch attached.



[-- Attachment #2: 0001-doc-Also-update-shebang-for-chapter-texi2docbook.py.patch --]
[-- Type: text/plain, Size: 737 bytes --]

From 53a54584e06729e4c23995e9993dc1c88106c32d Mon Sep 17 00:00:00 2001
From: Jon Turney <jon.turney@dronecode.org.uk>
Date: Tue, 25 Aug 2020 17:16:36 +0100
Subject: [PATCH] doc: Also update shebang for chapter-texi2docbook.py

---
 newlib/doc/chapter-texi2docbook.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/newlib/doc/chapter-texi2docbook.py b/newlib/doc/chapter-texi2docbook.py
index e21489b9a..1a7803091 100755
--- a/newlib/doc/chapter-texi2docbook.py
+++ b/newlib/doc/chapter-texi2docbook.py
@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 #
 # python script to convert the handwritten chapter .texi files, which include
 # the generated files for each function, to DocBook XML
-- 
2.28.0

Re: [PATCH] doc: Various fixes to makedocbook for python3.8

[Corinna Vinschen]

On Aug 25 20:45, Jon Turney wrote:
> On 24/08/2020 11:01, Corinna Vinschen via Newlib wrote:
> > On Aug 23 20:41, Jon Turney wrote:
> > > On 23/08/2020 16:23, Ken Brown wrote:
> > > > On 8/22/2020 2:45 PM, Jon Turney wrote:
> > > > > ---
> > > > >    newlib/doc/makedocbook.py | 8 +++++---
> > > > >    1 file changed, 5 insertions(+), 3 deletions(-)
> > > > > 
> > > > > diff --git a/newlib/doc/makedocbook.py b/newlib/doc/makedocbook.py
> > > > > index 92d0f279d..3fab26f1a 100755
> > > > > --- a/newlib/doc/makedocbook.py
> > > > > +++ b/newlib/doc/makedocbook.py
> > > [...]
> > > > Would it make sense to also change the shebang line so that
> > > > makedocbook.py uses python3?  Currently the build of Cygwin on Fedora
> > > > uses python3, but the build of Cygwin on Cygwin uses python2.  This is
> > > > of no great importance, but a recent IRC discussion shows that it can be
> > > > confusing.
> > > 
> > > Yeah, I guess that would make some sense, given that python2 is now EOL.
> > > 
> > > Otoh, making it gratuitously not work with python2 seems a bit harsh.
> > > 
> > > On the gripping hand, reading PEP 0394, I see it doesn't actually require
> > > that 'python' exist at all, so I guess changing the shebang as you suggest
> > > is the right thing to do.
> > 
> > With this change, please push.
> 
> Testing on system where '/usr/bi/python' doesn't exist, there's also another
> script which I neglected to update.
> 
> Patch attached.

Go for it.


Thanks,
Corinna