From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 12106 invoked by alias); 9 Jun 2011 02:04:05 -0000 Received: (qmail 12093 invoked by uid 22791); 9 Jun 2011 02:04:02 -0000 X-SWARE-Spam-Status: No, hits=-2.6 required=5.0 tests=AWL,BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,RCVD_IN_DNSWL_LOW,RFC_ABUSE_POST,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: sourceware.org Received: from mail-pz0-f41.google.com (HELO mail-pz0-f41.google.com) (209.85.210.41) by sourceware.org (qpsmtpd/0.43rc1) with ESMTP; Thu, 09 Jun 2011 02:03:45 +0000 Received: by pzk4 with SMTP id 4so624002pzk.0 for ; Wed, 08 Jun 2011 19:03:44 -0700 (PDT) Received: by 10.142.43.6 with SMTP id q6mr17482wfq.421.1307585024437; Wed, 08 Jun 2011 19:03:44 -0700 (PDT) Received: from [210.32.150.166] ([210.32.150.166]) by mx.google.com with ESMTPS id z2sm1185295wfz.12.2011.06.08.19.03.39 (version=SSLv3 cipher=OTHER); Wed, 08 Jun 2011 19:03:43 -0700 (PDT) Message-ID: <4DF0292B.7030307@gmail.com> Date: Thu, 09 Jun 2011 02:04:00 -0000 From: Asm warrior User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.19) Gecko/20081209 Thunderbird/2.0.0.19 Mnenhy/0.7.6.0 MIME-Version: 1.0 To: vanboxem.ruben@gmail.com CC: gdb@sourceware.org Subject: Re: Python enabled gdb on Windows and relocation References: In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Mailing-List: contact gdb-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: gdb-owner@sourceware.org X-SW-Source: 2011-06/txt/msg00041.txt.bz2 On 2011-5-16 0:11, Ruben Van Boxem wrote: > 2011/5/15 Ruben Van Boxem: >> 2011/5/15 Ruben Van Boxem: >>> 2011/5/14 Doug Evans: >>>> On Sat, May 14, 2011 at 2:09 AM, Ruben Van Boxem >>>> wrote: >>>>> 2011/5/14 Doug Evans: >>>>>> On Thu, May 12, 2011 at 9:19 AM, Ruben Van Boxem >>>>>> wrote: >>>>>>> (now in plain-text as required by gdb mailing list) >>>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> I am currently trying to integrate Python support into my toolchain >>>>>>> build (including GDB of course). It is a sysrooted >>>>>>> binutils+GCC+GDB+mingw-w64 toolchain. >>>>>>> >>>>>>> I currently have the basic setup working: I can link gdb with my >>>>>>> manually generated import lib to the python dll from the official >>>>>>> Windows install. If there is anything I am missing or a very easy >>>>>>> solution to the problems decsribed below, please just say so. I am >>>>>>> only suggesting what I would like to happen. >>>>>>> >>>>>>> Now on to the problems I'd like to discuss: >>>>>>> >>>>>>> 1. gdb.exe won't start without me having set PYTHONPATH manually. >>>>>> >>>>>> In a properly configured/built gdb on linux this isn't necessary, even >>>>>> if python is installed in some random place. >>>>>> I'm not sure about windows though. >>>>>> Did you specify --with-python when you configured gdb, and if so did >>>>>> you specify a value? >>>>>> e.g., --with-python=SOME_VALUE >>>>> >>>>> I was cross-compiling a mingw toolchain+gdb from Linux, so I used >>>>> --with-python without a value (because gdb configure tries to find the >>>>> Python executabe), and I added -I"/path/to/python/includes" to CFLAGS >>>>> and -L"/path/to/pythondll/importlib" to LDFLAGS, which built as it >>>>> should. This is hacky though, and gdb configure should provide >>>>> --with-python-libs and --with-python-include to make it more >>>>> streamlined with any other build prerequisite (like >>>>> gmp/mpfr/mpc/cloog/ppl in GCC for example). >>>> >>>> Ah. >>>> Cross-compiling gdb with python is in need of improvement. >>>> Alas python hasn't been designed with cross-compilation in mind (e.g. >>>> build on linux, run on windows). >>>> AIUI, the way to get the parameters required for compiling with >>>> libpython is to get them from python's "distutils": kinda hard to do >>>> in a cross-compile. Done correctly there's no need to run python. >>>> >>>> I haven't done anything more to support python in gdb's configure.ac >>>> because it's not clear to me what the right thing to do is: distutils >>>> provides more than just --libs and --includes (btw, we don't use >>>> --libs though, we use --ldflags which includes all of: the directory >>>> in which to find libpython, the -l for libpython, and the -l's for all >>>> the other libraries python needs). [Which isn't to say that someone >>>> else isn't free to tackle this.] >>>> >>>> In the meantime, what I've been doing is a hack: write a script that >>>> responds to: >>>> --includes >>>> --ldflags >>>> --exec-prefix >>>> and pass that as --with-python. >>>> >>>> E.g. >>>> bash$ cat $HOME/my-python-for-config >>>> #! /bin/sh >>>> >>>> if [ $# -ne 2 ] >>>> then >>>> echo "Bad # args. Blech!">&2 >>>> exit 1 >>>> fi >>>> >>>> # The first argument is the path to python-config.py, ignore it. >>>> >>>> case "$2" in >>>> --includes) echo "-I/usr/include/python2.6 -I/usr/include/python2.6" ;; >>>> --ldflags) echo "-L/usr/lib/python2.6/config -lpthread -ldl -lutil -lm >>>> -lpython2.6" ;; >>>> --exec-prefix) echo "/usr" ;; >>>> *) echo "Bad arg $2. Blech!">&2 ; exit 1 ;; >>>> esac >>>> >>>> exit 0 >>>> bash$ ./configure --with-python=$HOME/my-python-for-config [...] >>>> [...] >>>> >>>> >>>> Note that --exec-prefix is the runtime location of python. >>>> GCC uses this to tell libpython where to find its support files. >>>> [grep for Py_SetProgramName in gdb/python/python.c] >>> >>> OK, I tried your script in a couple of variations. It gets rid of the >>> traceback I had before, but still doesn't help the PYTHONPATH problem. >>> My directory structure is as follows (this is not in root, "/" is just >>> my main build directory, there are several levels below it): >>> >>> /gdb<-- gdb build dir >>> /gdb/gdb<-- where the python configuration is done >>> /python<-- temporary install dir for python files for build, >>> extracted from the official Windows installer, also location of >>> libpython2.7.a import library >>> /python/include/python27<-- python headers, found by gdb in both cases >>> /mingw64<-- toolchain sysroot prefix >>> /mingw64/bin<-- install location of gdb and python27.dll >>> /mingw64/lib/python27<-- install location of all python scripts >>> >>> First variant: >>> --includes) echo "-I../../python/include" ;; >>> --ldflags) echo "-L../../python -lpython2.7" ;; >>> --exec-prefix) echo "../../mingw64/lib/python27" ;; >>> >>> Here exec-prefix would be the relative path from where "configure" >>> does its magic to the final location of the scripts on the build >>> system. >>> >>> Second variant: >>> --includes) echo "-I../../python/include" ;; >>> --ldflags) echo "-L../../python -lpython2.7" ;; >>> --exec-prefix) echo "../lib/python27" ;; >>> >>> I thought the second points gdb to the installed location of the >>> python scripts, but it still needed PYTHONPATH for that. I used >>> relative paths in an attempt to "do the right thing", hackwise... >>> Remember that the whole "/mingw64" directory gets zipped and moved, >>> then it gets extracted to a random location in the Windows filesystem. >>> >>> Thanks for the help so far, I think we might just get this worked out together. >>> >>> Ruben >> >> Wow, I think I have a partial solution. Delving into the Python docs, >> for example here: >> http://docs.python.org/using/windows.html#finding-modules, you can see >> that PYTHONPATH is used first, then the Windows registry, then >> PYTHONHOME, then some default relative paths. I placed the python >> scripts all in the directory structure like so: >> >> /bin/gdb >> /bin/Lib/ >> /bin/python27.dll >> >> This works, even without any manual PYTHONPATH intervention. Problem >> is though, that as soon as someone has a PYTHONPATH environment >> variable from a (incompatible) Python installation (think different >> bitness or version 3.x instead of 2.7.1), I cannot predict what will >> go wrong. This problem originates in Python's way of filling in the >> search path (sys.path). A true solution in the GDB case to prevent >> this collision of an incompatible PYTHONPATH would be that GDB sets an >> internal PYTHONPATH as directed by configure, uses that to load its >> Python internals, and allows the GDB child processes (apps being >> debugged) to use the environment PYTHONPATH. For now, I have a >> functional installation, but it will break as soon as someone installs >> Python on their system. >> >> The part your script plays in this (if I haven't misunderstood the >> intention/effects of the "--exec-prefix" part) is allowing a uniform >> way of directing the configure script to the right directories for >> includes/libs. >> >> Ruben > > I am sorry for the repeated messages that no one cares about, but I > may have discovered GDB in its current form already allows what I > want: I tried to figure out what exact paths the snake in gdb was > using to search for its modules, and came up with this: > (gdb) python import sys > (gdb) python print sys.path > ['m:\\development\\mingw64\\share\\gdb/python', > 'M:\\Development\\mingw64\\bin\\python27.zip', > 'M:\\Development\\mingw64\\bin\\DLLs', > 'M:\\Development\\mingw64\\bin\\lib', > 'M:\\Development\\mingw64\\bin\\lib\\plat-win', > 'M:\\Development\\mingw64\\bin\\lib\\lib-tk', > 'M:\\Development\\mingw64\\bin', > 'M:\\Development\\mingw64\\bin\\lib\\site-packages'] > > This means that every python command within gdb searches > /share/gdb/python FIRST (even before an environment's > PYTHONPATH), alleviating any concerns or problems I or anyone would > have with another python installation, as this apparently built-in > path comes up first. All I, or anyone interested in doing this kind of > thing, have to do is copy all the python scripts from the Windows > installation's Lib directory to the/share/gdb/python > directory. > > I don't know where this path comes from, but it is quite handy, and > makes this whole discussion moot for Python people. Only "issue" that > I'll have to work around is the --with-python-includes and > --with-python-libs that are missing, using either manual > CFLAGS/LDFLAGS or a variant of your script. > > Thanks for all the help and motivation to keep trying :) > > Ruben > >> >>> >>>> >>>>>>> I understand the need for this, but as gdb requires Python 2, and users >>>>>>> of my toolchain may have installed Python 3 or a 32-bit version python >>>>>>> they want to use from the same environment (without changing their own >>>>>>> PYTHONPATH), there is no way to run python-enabled gdb. >>>>>>> [...] >>>>>> >>>>>> Yeah. >>>>>> There is a proposal to add GDB_PYTHONPATH (or some such IIRC) and have >>>>>> gdb use that instead of PYTHONPATH if it exists, but there's been >>>>>> resistance to it. >>>>>> I think(!) what would happen is that gdb would set $PYTHONPATH to the >>>>>> value of $GDB_PYTHONPATH. >>>>>> [Inferiors started by gdb should still get the original value of >>>>>> PYTHONPATH though.] >>>>> >>>>> That way would be almost ideal, but a hardcoded *relative* path to the >>>>> python scripts (that is standardized within gdb) wouldn't hurt. >>>> >>>> See above re: --exec-prefix. >>>> >>>>> An >>>>> extra environment variable would require a lot of explaining for >>>>> Windows, and is not "plug-and-play", like the rest of a sysrooted >>>>> toolchain is supposed to be like. I think this should work on all >>>>> setups: >>>>> >>>>> 1. Check hardcoded path; my suggestion would be ">>>> executable>/../lib/python27" >>>>> 2. If this fails to find the necessary files/scripts, find it like you >>>>> described above in Linux, without PYTHONPATH set. >>>>> 3. Check PYTHONPATH. >>>> >>>> The problem being solved by the proposed GDB_PYTHONPATH is "What if >>>> the user has PYTHONPATH set and it points to an incompatible version >>>> of python?". >>>> Leaving such a value for PYTHONPATH set while gdb's python is running >>>> feels wrong (and IIRC has caused some problems). >>>> >>>> The problem of telling python where to find itself is already solved >>>> (or at least is intended to be solved) with gdb's calling >>>> Py_SetProgramName with a value derived from the python-provided >>>> --exec-prefix. >>>> >>>>> I would think only number one would change, and perhaps be only >>>>> enabled with a special configure option. Nothing else would have to >>>>> change, and Windows users would rejoice :) >>>>> Again, this is only my suggestion, if there are problems with it in >>>>> way I haven't thought of, please say so, and we can come up with >>>>> another solution. >>>> >>> >> > Hi, vanboxem. what's your finding is quite interesting. In fact, I just copy the Lib(python supporting scripts) under the MinGW/bin/ folder, then every thing works fine. I never know the details about the python search path. So, thank you very much. asmwarrior