[RFA/python:2/2] First script in GDB python library - command/pahole.py

[RFA/python:2/2] First script in GDB python library - command/pahole.py

[Joel Brobecker]

This patch makes use of the work done in the previous patch that allows
to store the GDB python library at a global location.  It updates the
Makefile.in, adding install-python and uninstall-python targets. And
it also introduces the first script in the GDB python library: pahole.py.

Pretty much 100% of the code comes from Archer, and I think that
Tom is the original author of that code. I only tweaked it a little
to make it work at the FSF level (there are some API differences
between the Archer tree and the FSF tree).

2010-05-22  Tom Tromey  <tromey@redhat.com>
            Joel Brobecker  <brobecker@adacore.com>

        * Makefile.in (GDB_PYTHONDIR, PY_FILES): New variabls.
        (install-python, uninstall-python): New rules.
        * python/lib/gdb/__init__.py, python/lib/gdb/command/__init__.py,
        python/lib/gdb/command/pahole.py: New files.

Tested on x86_64-linux.

---
 gdb/Makefile.in                        |   32 +++++++++++++
 gdb/python/lib/gdb/__init__.py         |   16 ++++++
 gdb/python/lib/gdb/command/__init__.py |   16 ++++++
 gdb/python/lib/gdb/command/pahole.py   |   81 ++++++++++++++++++++++++++++++++
 4 files changed, 145 insertions(+), 0 deletions(-)
 create mode 100644 gdb/python/lib/gdb/__init__.py
 create mode 100644 gdb/python/lib/gdb/command/__init__.py
 create mode 100644 gdb/python/lib/gdb/command/pahole.py

diff --git a/gdb/Makefile.in b/gdb/Makefile.in
index 754671f..e61ed4f 100644
--- a/gdb/Makefile.in
+++ b/gdb/Makefile.in
@@ -169,6 +169,9 @@ TARGET_SYSTEM_ROOT_DEFINE = @TARGET_SYSTEM_ROOT_DEFINE@
 # Did the user give us a --with-gdb-datadir option?
 GDB_DATADIR = @GDB_DATADIR@
 
+# The location where the GDB python library is located.
+GDB_PYTHONDIR = @GDB_PYTHONDIR@
+
 # Helper code from gnulib.
 LIBGNU = gnulib/libgnu.a
 INCGNU = -I$(srcdir)/gnulib -Ignulib
@@ -2057,6 +2060,35 @@ py-value.o: $(srcdir)/python/py-value.c
 	$(COMPILE) $(PYTHON_CFLAGS) $(srcdir)/python/py-value.c
 	$(POSTCOMPILE)
 
+# All python library files, with the "python/lib" stripped off.
+# Note that we should only install files in the "gdb" module.
+PY_FILES = gdb/command/pahole.py gdb/command/__init__.py \
+    gdb/__init__.py
+
+# Install the Python library.  Python library files go under
+# $(GDB_PYTHONDIR).
+.PHONY: install-python
+install-python:
+	files='$(PY_FILES)'; for file in $$files; do \
+	  dir=`echo "$$file" | sed 's,/[^/]*$$,,'`; \
+	  $(SHELL) $(srcdir)/../mkinstalldirs $(DESTDIR)$(GDB_PYTHONDIR)/$$dir; \
+	  $(INSTALL_DATA) $(srcdir)/python/lib/$$file $(DESTDIR)$(GDB_PYTHONDIR)/$$file; \
+	done
+
+# Other packages may have their files installed in $(GDB_PYTHONDIR).
+.PHONY: uninstall-python
+uninstall-python:
+	files='$(PY_FILES)'; for file in $$files; do \
+	  slashdir=`echo "/$$file" | sed 's,/[^/]*$$,,'`; \
+	  rm -f $(DESTDIR)$(GDB_PYTHONDIR)/$$file; \
+	  while test "x$$file" != "x$$slashdir"; do \
+	    rmdir 2>/dev/null "$(DESTDIR)$(GDB_PYTHONDIR)$$slashdir"; \
+	    file="$$slashdir"; \
+	    slashdir=`echo "$$file" | sed 's,/[^/]*$$,,'`; \
+	  done \
+	done
+
+
 #
 # Dependency tracking.  Most of this is conditional on GNU Make being
 # found by configure; if GNU Make is not found, we fall back to a
diff --git a/gdb/python/lib/gdb/__init__.py b/gdb/python/lib/gdb/__init__.py
new file mode 100644
index 0000000..4263473
--- /dev/null
+++ b/gdb/python/lib/gdb/__init__.py
@@ -0,0 +1,16 @@
+# Copyright (C) 2010 Free Software Foundation, Inc.
+
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+
diff --git a/gdb/python/lib/gdb/command/__init__.py b/gdb/python/lib/gdb/command/__init__.py
new file mode 100644
index 0000000..4263473
--- /dev/null
+++ b/gdb/python/lib/gdb/command/__init__.py
@@ -0,0 +1,16 @@
+# Copyright (C) 2010 Free Software Foundation, Inc.
+
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+
diff --git a/gdb/python/lib/gdb/command/pahole.py b/gdb/python/lib/gdb/command/pahole.py
new file mode 100644
index 0000000..32e694d
--- /dev/null
+++ b/gdb/python/lib/gdb/command/pahole.py
@@ -0,0 +1,81 @@
+# pahole command for gdb
+
+# Copyright (C) 2008, 2010 Free Software Foundation, Inc.
+
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import gdb
+
+class Pahole(gdb.Command):
+    """Show the holes in a structure.
+This command takes a single argument, a type name.
+It prints the type and displays comments showing where holes are."""
+
+    def __init__(self):
+        super(Pahole, self).__init__("pahole", gdb.COMMAND_NONE,
+                                     gdb.COMPLETE_SYMBOL)
+
+    @staticmethod
+    def strip(type):
+        while type.code == gdb.TYPE_CODE_TYPEDEF:
+            type = type.target()
+        return type
+
+    def pahole(self, type, level, name):
+        if name is None:
+            name = ''
+        tag = type.tag
+        if tag is None:
+            tag = ''
+        print '%sstruct %s {' % (' ' * (2 * level), tag)
+        bitpos = 0
+        for field in type.fields():
+            # Skip static fields.
+            if not hasattr(field, ('bitpos')):
+                continue
+
+            ftype = self.strip(field.type)
+
+            if bitpos != field.bitpos:
+                hole = field.bitpos - bitpos
+                print ' /* XXX %d bit hole, try to pack */' % hole
+                bitpos = field.bitpos
+            if field.bitsize > 0:
+                fieldsize = field.bitsize
+            else:
+                # TARGET_CHAR_BIT here...
+                fieldsize = 8 * ftype.sizeof
+
+            # TARGET_CHAR_BIT
+            print ' /* %3d %3d */' % (int(bitpos / 8), int(fieldsize / 8)),
+            bitpos = bitpos + fieldsize
+
+            if ftype.code == gdb.TYPE_CODE_STRUCT:
+                self.pahole(ftype, level + 1, field.name)
+            else:
+                print ' ' * (2 + 2 * level),
+                print '%s %s' % (str(ftype), field.name)
+
+        print ' ' * (14 + 2 * level),
+        print '} %s' % name
+
+    def invoke(self, arg, from_tty):
+        type = gdb.lookup_type(arg)
+        type = self.strip(type)
+        if type.code != gdb.TYPE_CODE_STRUCT:
+            raise TypeError, '%s is not a struct type' % arg
+        print ' ' * 14,
+        self.pahole(type, 0, '')
+
+Pahole()
-- 
1.7.1

Add support for the GDB Python Library

[Joel Brobecker]

Hello,

One of the things that we still don't have in the FSF tree is a "GDB
Python Library" - that is a collection of standard Python scripts that
come with GDB. The idea is to help future contributions but setting
things up so that it's easy to add new python-based commands/functions/
features, etc.

This is what this couple of patches provide:
  - Patch #1: Defines a pythondir where the GDB Python library is
    expected to be found, and enhances the GDB/python support to
    look for them there.
  - Path #2: is built on top of patch #1, and provides the first
    such python patch.  We selected one form project archer that
    might be useful: pahole.py.

[RFA/python:1/2] Add support for --with-pythondir.

[Joel Brobecker]

This is the location where the GDB python library should be installed.
The default is ${datadir}/python.  As with various other paths, it is
relocatable if it is a subdir of the prefix.

The code is mostly inspired from some work done in Archer.  Based on
the revision history, I think that Tom Tromey should be credited.
I did rewrite the configury part by taking advantage of a patch that
Doug Evans just submitted (and will hopefully commit soon).

     http://www.sourceware.org/ml/gdb-patches/2010-05/msg00476.html
     (Allow python to find its files if moved from original location)

2010-05-26  Tom Tromey  <tromey@adacore.com>
            Joel Brobecker  <brobecker@adacore.com>

        * configure.ac: Add handling of --with-pythondir.
        * configure, config.in: Regenerate.
        * python/python.c (_initialize_python): Add variable "pythondir"
        inside module gdb.  Initialize variable gdb.__path__ and execute
        __init__.py if found in subdirectory "gdb" of the pythondir.

Tested on x86_64-linux.  I tested this using various combinations
of prefix, and --with-pythondir, both causing relocatable and
non-relocatable paths to the pythondir...


---
 gdb/config.in       |    7 ++++++
 gdb/configure       |   52 +++++++++++++++++++++++++++++++++++++++++++++++++++
 gdb/configure.ac    |    4 +++
 gdb/python/python.c |    7 ++++++
 4 files changed, 70 insertions(+), 0 deletions(-)

diff --git a/gdb/config.in b/gdb/config.in
index afbd76d..b4f2f55 100644
--- a/gdb/config.in
+++ b/gdb/config.in
@@ -75,6 +75,13 @@
 /* Define to the default OS ABI for this configuration. */
 #undef GDB_OSABI_DEFAULT
 
+/* look for global python files in this path [GDB_DATADIR/python] */
+#undef GDB_PYTHONDIR
+
+/* Define if the gdb-pythondir directory should be relocated when GDB is
+   moved. */
+#undef GDB_PYTHONDIR_RELOCATABLE
+
 /* Define to 1 when the gnulib module memchr should be tested. */
 #undef GNULIB_TEST_MEMCHR
 
diff --git a/gdb/configure b/gdb/configure
index 02b8720..487e7b6 100755
--- a/gdb/configure
+++ b/gdb/configure
@@ -658,6 +658,7 @@ TARGET_SYSTEM_ROOT_DEFINE
 TARGET_SYSTEM_ROOT
 CONFIG_LDFLAGS
 ALLOCA
+GDB_PYTHONDIR
 PYTHON_CFLAGS
 LTLIBEXPAT
 LIBEXPAT
@@ -962,6 +963,7 @@ with_gnu_ld
 enable_rpath
 with_libexpat_prefix
 with_python
+with_gdb_pythondir
 with_included_regex
 with_sysroot
 with_system_gdbinit
@@ -1658,6 +1660,9 @@ Optional Packages:
   --with-libexpat-prefix[=DIR]  search for libexpat in DIR/include and DIR/lib
   --without-libexpat-prefix     don't search for libexpat in includedir and libdir
   --with-python           include python support (auto/yes/no/<path>)
+  --with-gdb-pythondir=PATH
+                          look for global python files in this path
+                          [GDB_DATADIR/python]
   --without-included-regex
                           don't use included regex; this is the default on
                           systems with version 2 of the GNU C library (use
@@ -10691,6 +10696,53 @@ else
 fi
 
 
+
+
+# Check whether --with-gdb-pythondir was given.
+if test "${with_gdb_pythondir+set}" = set; then :
+  withval=$with_gdb_pythondir;
+    GDB_PYTHONDIR=$withval
+else
+  GDB_PYTHONDIR=${GDB_DATADIR}/python
+fi
+
+
+  test "x$prefix" = xNONE && prefix="$ac_default_prefix"
+  test "x$exec_prefix" = xNONE && exec_prefix='${prefix}'
+  ac_define_dir=`eval echo $GDB_PYTHONDIR`
+  ac_define_dir=`eval echo $ac_define_dir`
+
+cat >>confdefs.h <<_ACEOF
+#define GDB_PYTHONDIR "$ac_define_dir"
+_ACEOF
+
+
+
+
+  if test "x$exec_prefix" = xNONE || test "x$exec_prefix" = 'x${prefix}'; then
+     if test "x$prefix" = xNONE; then
+     	test_prefix=/usr/local
+     else
+	test_prefix=$prefix
+     fi
+  else
+     test_prefix=$exec_prefix
+  fi
+  value=0
+  case ${ac_define_dir} in
+     "${test_prefix}"|"${test_prefix}/"*|\
+	'${exec_prefix}'|'${exec_prefix}/'*)
+     value=1
+     ;;
+  esac
+
+cat >>confdefs.h <<_ACEOF
+#define GDB_PYTHONDIR_RELOCATABLE $value
+_ACEOF
+
+
+
+
 # ------------------------- #
 # Checks for header files.  #
 # ------------------------- #
diff --git a/gdb/configure.ac b/gdb/configure.ac
index 5b7bc38..1f423fa 100644
--- a/gdb/configure.ac
+++ b/gdb/configure.ac
@@ -715,6 +715,10 @@ else
 fi
 AC_SUBST(PYTHON_CFLAGS)
 
+GDB_AC_WITH_DIR(GDB_PYTHONDIR, gdb-pythondir,
+    [look for global python files in this path @<:@GDB_DATADIR/python@:>@],
+    [${GDB_DATADIR}/python])
+
 # ------------------------- #
 # Checks for header files.  #
 # ------------------------- #
diff --git a/gdb/python/python.c b/gdb/python/python.c
index 0b7b7ba..00a1ecc 100644
--- a/gdb/python/python.c
+++ b/gdb/python/python.c
@@ -668,6 +668,7 @@ Enables or disables printing of Python stack traces."),
   PyModule_AddStringConstant (gdb_module, "HOST_CONFIG", (char*) host_name);
   PyModule_AddStringConstant (gdb_module, "TARGET_CONFIG", (char*) target_name);
 
+  PyModule_AddStringConstant (gdb_module, "pythondir", GDB_PYTHONDIR);
   gdbpy_gdberror_exc = PyErr_NewException ("gdb.GdbError", NULL, NULL);
   PyModule_AddObject (gdb_module, "GdbError", gdbpy_gdberror_exc);
 
@@ -718,6 +719,12 @@ class GdbOutputFile:\n\
 \n\
 sys.stderr = GdbOutputFile()\n\
 sys.stdout = GdbOutputFile()\n\
+sys.path.insert(0, gdb.pythondir)\n\
+gdb.__path__ = [gdb.pythondir + '/gdb']\n\
+from os.path import exists\n\
+ipy = gdb.pythondir + '/gdb/__init__.py'\n\
+if exists (ipy):\n\
+  execfile (ipy)\n\
 ");
 
   /* Release the GIL while gdb runs.  */
-- 
1.7.1

Re: Add support for the GDB Python Library

[Doug Evans]

On Wed, May 26, 2010 at 5:08 PM, Joel Brobecker <brobecker@adacore.com> wrote:
> Hello,
>
> One of the things that we still don't have in the FSF tree is a "GDB
> Python Library" - that is a collection of standard Python scripts that
> come with GDB. The idea is to help future contributions but setting
> things up so that it's easy to add new python-based commands/functions/
> features, etc.
>
> This is what this couple of patches provide:
>  - Patch #1: Defines a pythondir where the GDB Python library is
>    expected to be found, and enhances the GDB/python support to
>    look for them there.
>  - Path #2: is built on top of patch #1, and provides the first
>    such python patch.  We selected one form project archer that
>    might be useful: pahole.py.

Is there a reason to not make pythondir be $(GDB_DATADIR)/python or some such?
[I'm not sure to what extent .pyc files are architecture independent.
Is that it?]

Re: Add support for the GDB Python Library

[Doug Evans]

On Wed, May 26, 2010 at 6:24 PM, Doug Evans <dje@google.com> wrote:
> On Wed, May 26, 2010 at 5:08 PM, Joel Brobecker <brobecker@adacore.com> wrote:
>> Hello,
>>
>> One of the things that we still don't have in the FSF tree is a "GDB
>> Python Library" - that is a collection of standard Python scripts that
>> come with GDB. The idea is to help future contributions but setting
>> things up so that it's easy to add new python-based commands/functions/
>> features, etc.
>>
>> This is what this couple of patches provide:
>>  - Patch #1: Defines a pythondir where the GDB Python library is
>>    expected to be found, and enhances the GDB/python support to
>>    look for them there.
>>  - Path #2: is built on top of patch #1, and provides the first
>>    such python patch.  We selected one form project archer that
>>    might be useful: pahole.py.
>
> Is there a reason to not make pythondir be $(GDB_DATADIR)/python or some such?
> [I'm not sure to what extent .pyc files are architecture independent.
> Is that it?]

For reference sake,
I did a configure of python 2.6.5 with different values for --prefix
and --exec-prefix.
*.py* got put in $prefix/lib/python2.6.
lib-dynload got put in $exec_prefix/lib/python2.6.

Re: [RFA/python:1/2] Add support for --with-pythondir.

[Doug Evans]

On Wed, May 26, 2010 at 5:08 PM, Joel Brobecker <brobecker@adacore.com> wrote:
> This is the location where the GDB python library should be installed.
> The default is ${datadir}/python.  As with various other paths, it is
> relocatable if it is a subdir of the prefix.
>
> The code is mostly inspired from some work done in Archer.  Based on
> the revision history, I think that Tom Tromey should be credited.
> I did rewrite the configury part by taking advantage of a patch that
> Doug Evans just submitted (and will hopefully commit soon).
>
>     http://www.sourceware.org/ml/gdb-patches/2010-05/msg00476.html
>     (Allow python to find its files if moved from original location)

Committed.

Re: Add support for the GDB Python Library

[Joel Brobecker]

> Is there a reason to not make pythondir be $(GDB_DATADIR)/python or some such?
> [I'm not sure to what extent .pyc files are architecture independent.
> Is that it?]

It's actually an unfortunate typo. I wrote the patch so that the default
pythondir is $(GDB_DATADIR)/python.

-- 
Joel

Re: [RFA/python:1/2] Add support for --with-pythondir.

[Tom Tromey]

>>>>> "Joel" == Joel Brobecker <brobecker@adacore.com> writes:

Joel> 2010-05-26  Tom Tromey  <tromey@adacore.com>
Joel>             Joel Brobecker  <brobecker@adacore.com>
Joel>         * configure.ac: Add handling of --with-pythondir.
Joel>         * configure, config.in: Regenerate.
Joel>         * python/python.c (_initialize_python): Add variable "pythondir"
Joel>         inside module gdb.  Initialize variable gdb.__path__ and execute
Joel>         __init__.py if found in subdirectory "gdb" of the pythondir.

FWIW I read it and it seems ok to me.
Thanks for doing this.

Tom

Re: [RFA/python:2/2] First script in GDB python library - command/pahole.py

[Tom Tromey]

>>>>> "Joel" == Joel Brobecker <brobecker@adacore.com> writes:

Joel> This patch makes use of the work done in the previous patch that allows
Joel> to store the GDB python library at a global location.  It updates the
Joel> Makefile.in, adding install-python and uninstall-python targets. And
Joel> it also introduces the first script in the GDB python library: pahole.py.

Today I remembered the other reasons we haven't upstreamed any of the
new commands written in Python :-(

There is no sensible way for a user to activate such a command.  With
this patch a user would have to know the name of the command's file and
invoke "python import gdb.command.pahole".

On the branch we have this "require" system so that users can write
"require command pahole" (with completion!).  But that was just a kind
of experiment, I don't think it is a very good idea.

One idea would be to allow some kind of auto-loading when a command is
not found.  That is, load pahole.py the first time the "pahole" command
is used.  Or even better, scan the install tree first so that command
completion still works.  (This is not ideal, though, since there is not
actually a way to discover command names without loading the file.)

FWIW I did not want to load all the commands at startup, as I was
concerned about slowing down startup.


Also there is the documentation issue.  If we ship a command then it
seems like we are giving it an official stamp, and it ought to be
documented in the manual.


I realize this is a lot of problems to solve.  The command activation
solution is one that would also be useful to users, since (ideally) it
would let them install local commands alongside built-in ones.

Tom

Re: [RFA/python:2/2] First script in GDB python library - command/pahole.py

[Joel Brobecker]

> There is no sensible way for a user to activate such a command.  With
> this patch a user would have to know the name of the command's file and
> invoke "python import gdb.command.pahole".

That's what I did to test the code, and I thought that this was OK.
In my mind, I was not suggesting the addition of a new command, but
rather of a new script. The distinction is really subtle but it makes
the "python import ..." command the natural way to use that script.
But the ulterior motive is not about the pahole command, but rather
about getting things started in terms of the gdb python library.

I think it would be nice to indeed have a better way of activating
these commands.  But in the meantime, I thought that this would already
be enough of an improvement to be worthwhile on its own. We could work
on the activation as a separate patch?

> One idea would be to allow some kind of auto-loading when a command is
> not found.  That is, load pahole.py the first time the "pahole" command
> is used.  Or even better, scan the install tree first so that command
> completion still works.  (This is not ideal, though, since there is not
> actually a way to discover command names without loading the file.)
> 
> FWIW I did not want to load all the commands at startup, as I was
> concerned about slowing down startup.

I tend to agree that it's not ideal to just import every gdb.command.*
module at startup.  It seems like a nice idea 

In terms of documentation, I agree we need to write something up.
I can take care of that:
  - Document the location where the GDB scripts are;
  - Add a section where we document new commands implemented
    in python.

-- 
Joel

Re: [RFA/python:1/2] Add support for --with-pythondir.

[Doug Evans]

On Thu, May 27, 2010 at 1:13 PM, Tom Tromey <tromey@redhat.com> wrote:
>>>>>> "Joel" == Joel Brobecker <brobecker@adacore.com> writes:
>
> Joel> 2010-05-26  Tom Tromey  <tromey@adacore.com>
> Joel>             Joel Brobecker  <brobecker@adacore.com>
> Joel>         * configure.ac: Add handling of --with-pythondir.
> Joel>         * configure, config.in: Regenerate.
> Joel>         * python/python.c (_initialize_python): Add variable "pythondir"
> Joel>         inside module gdb.  Initialize variable gdb.__path__ and execute
> Joel>         __init__.py if found in subdirectory "gdb" of the pythondir.
>
> FWIW I read it and it seems ok to me.
> Thanks for doing this.

fwiw, I still have some concerns.

Support doesn't have to be included in the patch, but IWBN to have a
plan for how to support .so's associated with gdb python modules in a
way that follows Python's scheme: i.e. .py*'s go in
$prefix/lib/pythonX.Y and .so's go in
$exec_prefix/lib/pythonX.Y/lib-dynload [i.e. architecture independent
files go in $prefix and architecture dependent files go in
$exec_prefix].
If the plan is to add --with-python-sodir [or some such] if/when it's
needed, great.  Or if the plan is to decide to just not support it,
that's probably ok too.  I'd just like to put in the thought now.

Also, do we want to name this --with-gdb-pythondir? [and do we want to
rename --with-python to --with-gdb-python?]
The idea being that if gdb was ever configured with something else
that had its own with for --with-python*, then gdb's wouldn't collide.
I don't know if it's that important, just thought I'd bring it up.
Apologies if it's already been discussed and decided and I've
forgotten.

Re: [RFA/python:1/2] Add support for --with-pythondir.

[Joel Brobecker]

> Support doesn't have to be included in the patch, but IWBN to have a
> plan for how to support .so's associated with gdb python modules in a
> way that follows Python's scheme: i.e. .py*'s go in
> $prefix/lib/pythonX.Y and .so's go in
> $exec_prefix/lib/pythonX.Y/lib-dynload [i.e. architecture independent
> files go in $prefix and architecture dependent files go in
> $exec_prefix].
> If the plan is to add --with-python-sodir [or some such] if/when it's
> needed, great.  Or if the plan is to decide to just not support it,
> that's probably ok too.  I'd just like to put in the thought now.

I wasn't personally thinking about .so modules for GDB. Do you have
any example in mind where it would be more useful for a user to use
a .so rather than a .py?

I can see several options:

  (a) We do without .so support in GDB;

  (b) We plan for eventual .so support, with the location adjustable
      through a configure switch (--with-python-sodir);

  (c) Embed the gdb python modules inside the python install by
      default (in site-packages), with a configure switch that allows us
      to put the library elsewhere.

Option (c) is really radical, and I don't know that it'll really fly.
But introduces the idea that the GDB python library layout mimic
the one that python uses.

Option (b) is a good backup for option (a). So, in a way, (a) and (b)
are the same option.  I'd go with that.

> Also, do we want to name this --with-gdb-pythondir? [and do we want to
> rename --with-python to --with-gdb-python?]

I think that --with-python is a fine name, although now that I think
of it, it might have been more consistent to name it with-python-prefix.
I don't think that we should name it with a "gdb" in it, because this
is not GDB's python library, but rather the prefix where python is
installed.

I am not particularly attached to --with-gdb-pythondir, on the other
hand, but I don't seem to be able to find a better name. Perhaps,
if we want the gdb python lib to mimick the layout of a typical
python install, then --with-gdb-python-prefix might be better (?).

-- 
Joel

Re: [RFA/python:1/2] Add support for --with-pythondir.

[Doug Evans]

On Fri, May 28, 2010 at 9:08 AM, Joel Brobecker <brobecker@adacore.com> wrote:
> I wasn't personally thinking about .so modules for GDB. Do you have
> any example in mind where it would be more useful for a user to use
> a .so rather than a .py?

No example.  I just want us to have thought about it (since now seems
like the right time).
And I don't have a strong opinion on any particular solution.
I do think that we should allow for the possibility of having gdb
python modules implemented as .so's.

>> Also, do we want to name this --with-gdb-pythondir? [and do we want to
>> rename --with-python to --with-gdb-python?]
>
> I think that --with-python is a fine name, although now that I think
> of it, it might have been more consistent to name it with-python-prefix.
> I don't think that we should name it with a "gdb" in it, because this
> is not GDB's python library, but rather the prefix where python is
> installed.

For reference sake, --with-python-prefix is a worse name now.
Setting aside --with-python=yes, the preferred parameter is the name
or path of the python binary.  We fetch the needed parameters via
distutils of that binary. [Setting aside cross-compilation.]

Re: [RFA/python:1/2] Add support for --with-pythondir.

[Joel Brobecker]

> I do think that we should allow for the possibility of having gdb
> python modules implemented as .so's.

Sure.  The easiest seems to add support for that on top of what's
currently been proposed when the need arises.  AFAICT, It looks like
all we have to do to make it work is add the lib-dynload directory
to the sys.path.

> For reference sake, --with-python-prefix is a worse name now.
> Setting aside --with-python=yes, the preferred parameter is the name
> or path of the python binary.  We fetch the needed parameters via
> distutils of that binary. [Setting aside cross-compilation.]

Hmmm, yes, that's right.

Let us know if you still have some concerns...

-- 
Joel

Re: [RFA/python:1/2] Add support for --with-pythondir.

[Doug Evans]

On Tue, Jun 1, 2010 at 1:15 PM, Joel Brobecker <brobecker@adacore.com> wrote:
> Let us know if you still have some concerns...

I don't have any concerns other than those expressed.
They're not showstoppers or anything, I just want to make sure that
if/when the time comes to add --with-python-sodir (or some such), it's
not willy-nilly shot down because of the claim that there are already
too many configure options. IOW, we're preapproving adding that
configure option when the time comes.  And if we're not preapproving
that option, what *is* our plan for supporting gdb .so python modules
(in an environment where .py*'s and .so's live in different
directories, host-independent vs host-dependent).

Re: [RFA/python:1/2] Add support for --with-pythondir.

[Joel Brobecker]

> I don't have any concerns other than those expressed.
> They're not showstoppers or anything, I just want to make sure that
> if/when the time comes to add --with-python-sodir (or some such), it's
> not willy-nilly shot down because of the claim that there are already
> too many configure options.

Aaaah - I see where you are coming from now.  Personally, I certainly
would not object. The number of configure options can indeed be somewhat
of a concern (I'm slightly concerned about the distinction between
the gdb-pythonlib path, and the python path), but nothing blocking.
At worst, I think we can do without the configure option, by forcing
the lib-dynload directory to be inside the gdb-pythonlib.  WDYT?

I need to discuss the second patch with Tom anyways, so I'll wait a few
more days, to see if anyone else would like to comment on this.

-- 
Joel

Re: [RFA/python:2/2] First script in GDB python library - command/pahole.py

[Joel Brobecker]

> There is no sensible way for a user to activate such a command.  With
> this patch a user would have to know the name of the command's file and
> invoke "python import gdb.command.pahole".
[...]
> One idea would be to allow some kind of auto-loading when a command is
> not found.  That is, load pahole.py the first time the "pahole" command
> is used.  Or even better, scan the install tree first so that command
> completion still works.  (This is not ideal, though, since there is not
> actually a way to discover command names without loading the file.)

Here is an approach that should work.  The idea is to rely on the
convention that the command name is the name of the python script.
Subcommands are stored inside subdirectories.  For instance, let's
imagine that we're trying to execute the following command:

        (gdb) prefix1 prefix2 pycommand arg1 arg2

What I propose we do is that, if a python script provides an implementation
of command "prefix1 prefix2 pycommand", then it should be stored in

    GDB_PYTHON_DIR/command/prefix1/prefix2/pycommand.py

(Doug asked whether we wanted the .py extension or not; either of us
don't seem to have a strong opinion on it, although I do like it, and
most editors will likely also appreciate having it to automatically
activate the python edit mode).

pycommand.py is a python *script* (it is not expected to be a module file)
which will be executed when the "prefix1 prefix2 pycommand" command is
actually used.

I am not sure that we should search for Python commands only after
we failed to find a command. I'm thinking of the various prefix
commands that we already have such as the set command.  The current
semantics of the "set" command is that it treats the arguments as
an expression if no subcommand was found.  So if we looked at python
commands only after we failed to find a regular command, then it would
not allow us to add a set subcommand, at least not automatically.

That's why I suggest we scan the GDB_PYTHON_DIR/command directory
for all scripts, and create stub commands - the commands exist, but
their implementation is only loaded on-demand. The loading would
be performed using the execfile command (or its C equivalent if there
is one).

WDYT?  (note that having the stub command should also allow us to
have command completion)

-- 
Joel

Re: [RFA/python:2/2] First script in GDB python library - command/pahole.py

[Doug Evans]

On Wed, Jun 2, 2010 at 5:11 PM, Joel Brobecker <brobecker@adacore.com> wrote:
> (Doug asked whether we wanted the .py extension or not; either of us
> don't seem to have a strong opinion on it, although I do like it, and
> most editors will likely also appreciate having it to automatically
> activate the python edit mode).

For reference sake,
I was asking whether we needed command files for the prefixes, not
whether the files should have a .py suffix.

i.e.

instead of having .py files for each prefix and subprefix, in addition
to one for the command, as in

$dir/commands/prefix.py
$dir/commands/prefix/subprefix.py
$dir/commands/prefix/subprefix/my-command.py

just have

$dir/commands/prefix/subprefix/my-command.py

I like the consistency of the former, but I'm not sure what to do
about prefixes that already exist, e.g. enable, disable.
Or what to do if there are multiple command directories and several
have commands with the same prefix.

btw,
Prescanning the directories and creating stubs sounds reasonable.
How much of a stub though?
In addition to command completion there is help and apropos.

Re: [RFA/python:2/2] First script in GDB python library - command/pahole.py

[Joel Brobecker]

> $dir/commands/prefix.py
> $dir/commands/prefix/subprefix.py
> $dir/commands/prefix/subprefix/my-command.py
> 
> just have
> 
> $dir/commands/prefix/subprefix/my-command.py
> 
> I like the consistency of the former, but I'm not sure what to do
> about prefixes that already exist, e.g. enable, disable.

Hmm, right, I hadn't thought about that.  IMO, there should not be a
prefix.py or subprefix.py if that prefix/subprefix command is not
a valid command in itself, or if the command already exists.

> Or what to do if there are multiple command directories and several
> have commands with the same prefix.

I guess we'll have to think about that one too, eventually. I think
the thing to do when that can happen is to print a warning/error saying
that such and such py script could not be run because it collides with
another script that was run earlier.  Or print a warning saying that
a given command from such and such script was overriden by such and
such script (we might want to give precedence to user-provided scripts
over system-provided scripts for instance).

> Prescanning the directories and creating stubs sounds reasonable.
> How much of a stub though?
> In addition to command completion there is help and apropos.

I think that if we want to have help and apropos working, we might
as well just use the simpler approach of running the command scripts
right from the start.  I'd be OK with that too, and it's certainly
much simpler to work on.  Otherwise, to get the help, we'd have to
either evaluate part of the script, or find a way to provide that
information without doing the evaluation.  Either way, I think that
it would take us past the overkill point.

-- 
Joel

Re: [RFA/python:2/2] First script in GDB python library - command/pahole.py

[Tom Tromey]

>>>>> "Joel" == Joel Brobecker <brobecker@adacore.com> writes:

Joel> I think that if we want to have help and apropos working, we might
Joel> as well just use the simpler approach of running the command scripts
Joel> right from the start.

I have been thinking more about this problem recently.

One idea is KISS: just load all the commands and functions at startup.
I avoided this initially out of fear that it would be too slow -- but I
never measured it.  Maybe it isn't so bad.

A second idea is to install the Command objects eagerly, but have the
implementation lazily load the actual code.  That is, there would be a
single Command subclass that would take a help string, a module name,
and a class name (string) arguments, and then when the user invoked the
command it would import the module and instantiate an object to which
the command operations would be delegated.

This approach would work for convenience functions, too.  It is also
easy for 3rd party users to use.

Also, we can always start with KISS and move to the lazier approach if
it turns out to be too slow.

Tom