[Bug external/32379] New: GDB debugger and my PYTHON hate each other

[Bug external/32379] New: GDB debugger and my PYTHON hate each other

[aotto1968@t-online.de]

https://sourceware.org/bugzilla/show_bug.cgi?id=32379

            Bug ID: 32379
           Summary: GDB debugger and my PYTHON hate each other
           Product: gdb
           Version: 13.1
            Status: UNCONFIRMED
          Severity: minor
          Priority: P2
         Component: external
          Assignee: unassigned at sourceware dot org
          Reporter: aotto1968@t-online.de
  Target Milestone: ---

Hi,

I get quite often a "weird" error message from a tools using the "GDB" internal
as debugger etc.

This example is from a C# mono BUG → probably mono is using GDB to get the
backtrace from a CORE-DUMP.

The "core-problem" is that GDB using an **external** python and this
**external** python does not seem to *not* work with GDB. I use my own python
(not the one from my SuSE linux distribution) because I also maintain a python
extension by my own.

→ just a "basic" rule for everyone using an **external** SW like python. 
please add a copy of the **external** SW (including a *working* setup) into
your *own* SW (GDB for example) because it makes all the users happy. 
→ python is free, seems to be no problem to get your OWN copy 🙂

=================================================================
       External Debugger Dump:
=================================================================
Fatal Python error: Py_Initialize: Unable to get the locale encoding
ModuleNotFoundError: No module named 'encodings'

Current thread 0x00007fe802241540 (most recent call first):


Fatal signal: Aborted
----- Backtrace -----
0x564ff9592a3b ???
0x564ff96bbc6a ???
0x7fe8028b3dbf ???
0x7fe8028b3d2b raise
0x7fe8028b53e4 __GI_abort
0x7fe8031567e2 Py_FatalError
0x7fe803155fef ???
0x7fe80329daa7 _Py_InitializeEx_Private
0x564ff98a03b9 ???
0x564ff96c1a70 ???
0x564ff97aca90 ???
0x564ff97adb7a ???
0x564ff94954ea ???
0x7fe80289e24c __libc_start_main
0x564ff949ebf9 ???
0xffffffffffffffff ???
---------------------
A fatal error internal to GDB has been detected, further
debugging is not possible.  GDB will now terminate.

This is a bug, please report it.  For instructions, see:
<http://bugs.opensuse.org/>.


=================================================================
       Basic Fault Address Reporting
=================================================================
Memory around native instruction pointer (0x7f7c93725d2b):0x7f7c93725d1b  d2 4c
89 ce bf 02 00 00 00 b8 0e 00 00 00 0f 05  .L..............
0x7f7c93725d2b  48 8b 8c 24 08 01 00 00 64 48 33 0c 25 28 00 00 
H..$....dH3.%(..
0x7f7c93725d3b  00 44 89 c0 75 1d 48 81 c4 10 01 00 00 5b c3 66 
.D..u.H......[.f
0x7f7c93725d4b  0f 1f 44 00 00 48 8b 15 19 01 1a 00 f7 d8 64 89 
..D..H........d.

=================================================================
       Managed Stacktrace:
=================================================================
         at <unknown> <0xffffffff>
         at csmkkernel.Mk:MkErrorStack <0x000e6>
         at csmkkernel.MkErrorC:StackFull <0x001a7>
         at csmkkernel.MkExceptionC:.ctor <0x00103>
         at csmkkernel.MkErrorC:Check <0x0008f>
         at csmqmsgque.MqContextC:LinkCreate <0x0008b>
         at csmqmsgque.MqContextC:LinkCreate <0x00077>
         at example.MyServer:Main <0x0006b>
         at <Module>:runtime_invoke_void_object <0x00091>
=================================================================

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug external/32379] GDB debugger and my PYTHON hate each other

[vries at gcc dot gnu.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=32379

Tom de Vries <vries at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |vries at gcc dot gnu.org

--- Comment #1 from Tom de Vries <vries at gcc dot gnu.org> ---
I don't see a description of how to reproduce this, so I googled the error
message and suse and found
https://stackoverflow.com/questions/71104698/no-module-named-encodings-on-opensuse,
which suggest that this can happen when PYTHONHOME is set.

So, tentative description of the problem: gdb is used with PYTHONHOME which is
incompatible (or possibly, incorrect), and aborts.

It's possible the problem is fixed by using gdb -eiex "set python
ignore-environment on" or adding that setting to one of gdb's early
initialization files (
https://sourceware.org/gdb/current/onlinedocs/gdb.html/Initialization-Files.html
).

If so, this is at least partly a duplicate of PR22503.

It would be better if gdb didn't abort.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug external/32379] GDB debugger and my PYTHON hate each other

[vries at gcc dot gnu.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=32379

--- Comment #2 from Tom de Vries <vries at gcc dot gnu.org> ---
(In reply to Tom de Vries from comment #1)
> It would be better if gdb didn't abort.

OK, actually it's python that aborts, as per spec of Py_Initialize.  Starting
python version 3.10, gdb instead calls Py_InitializeFromConfig() which doesn't
abort.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug python/32379] [gdb/python, python < 3.10] gdb aborts when python fails to initialize

[vries at gcc dot gnu.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=32379

Tom de Vries <vries at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
          Component|external                    |python
            Summary|GDB debugger and my PYTHON  |[gdb/python, python < 3.10]
                   |hate each other             |gdb aborts when python
                   |                            |fails to initialize

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug python/32379] [gdb/python, python < 3.10] gdb aborts when python fails to initialize

[vries at gcc dot gnu.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=32379

--- Comment #3 from Tom de Vries <vries at gcc dot gnu.org> ---
https://sourceware.org/pipermail/gdb-patches/2024-November/213438.html

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug python/32379] [gdb/python, python < 3.10] gdb aborts when python fails to initialize

[vries at gcc dot gnu.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=32379

--- Comment #4 from Tom de Vries <vries at gcc dot gnu.org> ---
(In reply to Tom de Vries from comment #1)
> It's possible the problem is fixed by using gdb -eiex "set python
> ignore-environment on" or adding that setting to one of gdb's early
> initialization files (
> https://sourceware.org/gdb/current/onlinedocs/gdb.html/Initialization-Files.
> html ).

I've proposed a warning to advertise this option (
https://sourceware.org/pipermail/gdb-patches/2024-November/213485.html ).

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug python/32379] [gdb/python, python < 3.10] gdb aborts when python fails to initialize

[cvs-commit at gcc dot gnu.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=32379

--- Comment #5 from Sourceware Commits <cvs-commit at gcc dot gnu.org> ---
The master branch has been updated by Tom de Vries <vries@sourceware.org>:

https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=9d3785a8ac15dcc369b8c0b1074c5ca7145cdda7

commit 9d3785a8ac15dcc369b8c0b1074c5ca7145cdda7
Author: Tom de Vries <tdevries@suse.de>
Date:   Fri Nov 22 19:34:24 2024 +0100

    [gdb/python] Fix abort on Py_Initialize

    I tried out making python initialization fail by passing an incorrect
    PYTHONHOME with python 3.6, and got:
    ...
    $ PYTHONHOME=foo gdb -q
    Fatal Python error: Py_Initialize: Unable to get the locale encoding
    ModuleNotFoundError: No module named 'encodings'

    Current thread 0x0000ffff89269c80 (most recent call first):

    Fatal signal: Aborted
      ...
    Aborted (core dumped)
    $
    ...

    This is as per spec: when Py_Initialize () fails, a fatal error is raised
    using Py_FatalError.

    This can be worked around using:
    ...
    $ PYTHONHOME=foo gdb -q -eiex "set python ignore-environment on"
    (gdb)
    ...
    but it would be better if gdb didn't abort.

    I found an article [1] describing two solutions:
    - try out Py_Initialize in a separate process, and
    - catch the abort using a signal handler.

    This patch implements the latter solution.

    Obviously we cannot call into python anymore after the abort, so we avoid
    calling Py_IsInitialized (), and instead use a new variable
py_isinitialized.

    This gets us instead:
    ...
    $ PYTHONHOME=foo gdb -q
    Fatal Python error: Py_Initialize: Unable to get the locale encoding
    ModuleNotFoundError: No module named 'encodings'

    Current thread 0x0000fffecfd49c80 (most recent call first):
    Python not initialized
    $
    ...

    Tested on aarch64-linux.

    Approved-By: Tom Tromey <tom@tromey.com>

    PR python/32379
    Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=32379

    [1]
https://stackoverflow.com/questions/7688374/how-to-i-catch-and-handle-a-fatal-error-when-py-initialize-fails

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug python/32379] [gdb/python, python < 3.10] gdb aborts when python fails to initialize

[cvs-commit at gcc dot gnu.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=32379

--- Comment #6 from Sourceware Commits <cvs-commit at gcc dot gnu.org> ---
The master branch has been updated by Tom de Vries <vries@sourceware.org>:

https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=6a02aa77d80e4040bea699a88cf0dd208df639b4

commit 6a02aa77d80e4040bea699a88cf0dd208df639b4
Author: Tom de Vries <tdevries@suse.de>
Date:   Tue Dec 3 22:58:47 2024 +0100

    [gdb/python] Issue warning if python fails to initialize

    A common problem is that python may fail to initialize if PYTHONHOME is
    set incorrectly, or points to incompatible default libraries.

    Likewise if PYTHONPATH points to incompatible modules.

    For instance, say PYTHONHOME is foo, then we get:
    ...
    $ gdb -q
    Python path configuration:
      PYTHONHOME = 'foo'
      PYTHONPATH = (not set)
      program name = '/usr/bin/python'
      isolated = 0
      environment = 1
      user site = 1
      safe_path = 0
      import site = 1
      is in build tree = 0
      stdlib dir = 'foo/lib64/python3.12'
      sys._base_executable = '/usr/bin/python'
      sys.base_prefix = 'foo'
      sys.base_exec_prefix = 'foo'
      sys.platlibdir = 'lib64'
      sys.executable = '/usr/bin/python'
      sys.prefix = 'foo'
      sys.exec_prefix = 'foo'
      sys.path = [
        'foo/lib64/python312.zip',
        'foo/lib64/python3.12',
        'foo/lib64/python3.12/lib-dynload',
      ]
    Python Exception <class 'ModuleNotFoundError'>: No module named 'encodings'
    Python not initialized
    $
    ...

    In this case, it might be easy to figure out what went wrong because of the
    obviously incorrect pathnames, but that might not be the case if PYTHONHOME
    points to an incompatible python installation.

    Fix this by adding a warning with a description of the possible cause and
what
    to do about it:
    ...
    Python initialization failed: \
      failed to get the Python codec of the filesystem encoding
    gdb: warning: Python failed to initialize with PYTHONHOME set.  Maybe
because \
      it is set incorrectly? Maybe because it points to incompatible standard \
      libraries? Consider changing or unsetting it, or ignoring it using "set \
      python ignore-environment on" at early initialization.
    ...

    Likewise for PYTHONPATH:
    ...
    Python initialization failed: \
      failed to get the Python codec of the filesystem encoding
    gdb: warning: Python failed to initialize with PYTHONPATH set.  Maybe
because \
      it points to incompatible modules? Consider changing or unsetting it, or
\
      ignoring it using "set python ignore-environment on" at early \
      initialization.
    ...

    Tested on aarch64-linux.

    Approved-By: Tom Tromey <tom@tromey.com>

    PR python/32379
    Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=32379

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug python/32379] [gdb/python, python < 3.10] gdb aborts when python fails to initialize

[vries at gcc dot gnu.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=32379

Tom de Vries <vries at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |FIXED
             Status|UNCONFIRMED                 |RESOLVED
   Target Milestone|---                         |16.1

--- Comment #7 from Tom de Vries <vries at gcc dot gnu.org> ---
Fixed.

-- 
You are receiving this mail because:
You are on the CC list for the bug.