Re: Failure during build of Python 3.8 via cygport

[Mark Geisert <mark@maxrnd.com>]

[replying to myself again...]

A similar problem happens when building 3.6 and 3.7 too.  Details at end.

On Wed, 9 Dec 2020, Mark Geisert wrote:
> Hi Marco,
> I was building Python locally so I can later submit a patch against it.  It 
> appears the local python.exe was built successfully, but a later step failed 
> with:
>
>> ./python.exe -E -S -m sysconfig --generate-posix-vars ;\
>> if test $? -ne 0 ; then \
>>         echo "generate-posix-vars failed" ; \
>>         rm -f ./pybuilddir.txt ; \
>>         exit 1 ; \
>> fi
>> Traceback (most recent call last):
>>   File 
>> "/usr/src/python38-3.8.3-1.src/python38-3.8.3-1.x86_64/src/Python-3.8.3/Lib/runpy.py", 
>> line 194, in _run_module_as_main
>>     return _run_code(code, main_globals, None,
>>   File 
>> "/usr/src/python38-3.8.3-1.src/python38-3.8.3-1.x86_64/src/Python-3.8.3/Lib/runpy.py", 
>> line 87, in _run_code
>>     exec(code, run_globals)
>>   File 
>> "/usr/src/python38-3.8.3-1.src/python38-3.8.3-1.x86_64/src/Python-3.8.3/Lib/sysconfig.py", 
>> line 711, in <module>
>>     _main()
>>   File 
>> "/usr/src/python38-3.8.3-1.src/python38-3.8.3-1.x86_64/src/Python-3.8.3/Lib/sysconfig.py", 
>> line 699, in _main
>>     _generate_posix_vars()
>>   File 
>> "/usr/src/python38-3.8.3-1.src/python38-3.8.3-1.x86_64/src/Python-3.8.3/Lib/sysconfig.py", 
>> line 416, in _generate_posix_vars
>>     f.write(pybuilddir)
>> UnicodeEncodeError: 'utf-8' codec can't encode characters in position 
>> 17-19: surrogates not allowed
>> generate-posix-vars failed
>> make: *** [Makefile:592: pybuilddir.txt] Error 1
>> *** ERROR: make failed
>
> I've seen UnicodeEncodeError before and searched and found how to fix it.. 
> but hitting the issue while building Python itself seems more fraught.  Is 
> this a known issue with known fix?

This seems to be a problem setting up a platform-specific build directory. 
The sysconfig.py script wants to use "lib." + platform + pythonversion but 
the platform string somehow gets corrupted into non-utf8 bytes.  For 
instance, building Python 3.8 comes up with:
     lib.cygwin-\365\377\377o-\377o-3.8
as the directory name.  Broken, but could work.  The build failure happens 
because the script tries to write this directory name into a file but it's 
not a valid utf8 string.  The directory name should have been:
     lib.cygwin-3.2.0-x86_64-3.8

I'm trying to debug further, learning Python as I go.  Whee....

..mark