Another Python namespace pollution (was: r147958 - in /trunk/libstdc++-v3: ChangeLog Mak...)

Another Python namespace pollution (was: r147958 - in /trunk/libstdc++-v3: ChangeLog Mak...)

[Gerald Pfeifer]

On Fri, 29 May 2009, Tom Tromey wrote:
> 2009-05-29  Tom Tromey  <tromey@redhat.com>
> 
> 	PR libstdc++/40289:
> 	* python/Makefile.in: Rebuild.
> 	* python/hook.in: Compute module path relative to objfile.
> 	* python/Makefile.am (pythondir): Redefine.
> 	(gdb.py): Subst toolexeclibdir.
> 	(install-data-local): Rewrite.

Thanks, Tom!

I noticed that coming from the Java side of things we also have two
files that are installed in common namespace:

  share/python/aotcompile.py
  share/python/classfile.py

These have been added by the following patch from all can tell:

2008-06-30  Joshua Sumali  <jsumali@redhat.com>
        Andrew Haley  <aph@redhat.com>

        * configure.ac (java-home): new AC_ARG_ENABLE.
        (aot-compile-rpm): Likewise.
        :
        * contrib/aot-compile.in: New file.
        * contrib/aotcompile.py.in: Likewise.
        * contrib/aot-compile-rpm.in: Likewise.
        * contrib/classfile.py: Likewise.
        * contrib/rebuild-gcj-db.in: Likewise.

This is also http://gcc.gnu.org/bugzilla/show_bug.cgi?id=47484 .

Gerald

Re: Another Python namespace pollution (was: r147958 - in /trunk/libstdc++-v3: ChangeLog Mak...)

[Gary Benson]

Hi Gerald,

Gerald Pfeifer wrote:
> I noticed that coming from the Java side of things we also have two
> files that are installed in common namespace:
> 
>   share/python/aotcompile.py
>   share/python/classfile.py
> 
> These have been added by the following patch from all can tell:
> 
> 2008-06-30  Joshua Sumali  <jsumali@redhat.com>
>         Andrew Haley  <aph@redhat.com>
> 
>         * configure.ac (java-home): new AC_ARG_ENABLE.
>         (aot-compile-rpm): Likewise.
>         :
>         * contrib/aot-compile.in: New file.
>         * contrib/aotcompile.py.in: Likewise.
>         * contrib/aot-compile-rpm.in: Likewise.
>         * contrib/classfile.py: Likewise.
>         * contrib/rebuild-gcj-db.in: Likewise.
> 
> This is also http://gcc.gnu.org/bugzilla/show_bug.cgi?id=47484 .

I'm on the hook to fix this, but it's been a while since I worked on
this stuff so I'm a little out of touch with GCC.  Can you tell me
what you mean by "common namespace", and what the alternative should
be?

Thanks,
Gary

-- 
http://gbenson.net/

Re: Another Python namespace pollution (was: r147958 - in /trunk/libstdc++-v3: ChangeLog Mak...)

[Richard Guenther]

On Thu, Jan 27, 2011 at 11:38 AM, Gary Benson <gbenson@redhat.com> wrote:
> Hi Gerald,
>
> Gerald Pfeifer wrote:
>> I noticed that coming from the Java side of things we also have two
>> files that are installed in common namespace:
>>
>>   share/python/aotcompile.py
>>   share/python/classfile.py
>>
>> These have been added by the following patch from all can tell:
>>
>> 2008-06-30  Joshua Sumali  <jsumali@redhat.com>
>>         Andrew Haley  <aph@redhat.com>
>>
>>         * configure.ac (java-home): new AC_ARG_ENABLE.
>>         (aot-compile-rpm): Likewise.
>>         :
>>         * contrib/aot-compile.in: New file.
>>         * contrib/aotcompile.py.in: Likewise.
>>         * contrib/aot-compile-rpm.in: Likewise.
>>         * contrib/classfile.py: Likewise.
>>         * contrib/rebuild-gcj-db.in: Likewise.
>>
>> This is also http://gcc.gnu.org/bugzilla/show_bug.cgi?id=47484 .
>
> I'm on the hook to fix this, but it's been a while since I worked on
> this stuff so I'm a little out of touch with GCC.  Can you tell me
> what you mean by "common namespace", and what the alternative should
> be?

The files should reside in a directory that is private to gcc or at least
mangled with the gcc version it comes from.  Otherwise multiple
parallel gcc installations conflict.

Richard.

> Thanks,
> Gary
>
> --
> http://gbenson.net/
>

Re: Another Python namespace pollution (was: r147958 - in /trunk/libstdc++-v3: ChangeLog Mak...)

[Gary Benson]

Richard Guenther wrote:
> On Thu, Jan 27, 2011 at 11:38 AM, Gary Benson <gbenson@redhat.com> wrote:
> > Gerald Pfeifer wrote:
> > > I noticed that coming from the Java side of things we also have two
> > > files that are installed in common namespace:
> > >
> > >   share/python/aotcompile.py
> > >   share/python/classfile.py
> > >
> > > These have been added by the following patch from all can tell:
> > >
> > > 2008-06-30  Joshua Sumali  <jsumali@redhat.com>
> > >         Andrew Haley  <aph@redhat.com>
> > >
> > >         * configure.ac (java-home): new AC_ARG_ENABLE.
> > >         (aot-compile-rpm): Likewise.
> > >         :
> > >         * contrib/aot-compile.in: New file.
> > >         * contrib/aotcompile.py.in: Likewise.
> > >         * contrib/aot-compile-rpm.in: Likewise.
> > >         * contrib/classfile.py: Likewise.
> > >         * contrib/rebuild-gcj-db.in: Likewise.
> > >
> > > This is also http://gcc.gnu.org/bugzilla/show_bug.cgi?id=47484 .
> >
> > I'm on the hook to fix this, but it's been a while since I worked
> > on this stuff so I'm a little out of touch with GCC.  Can you tell
> > me what you mean by "common namespace", and what the alternative
> > should be?
> 
> The files should reside in a directory that is private to gcc or
> at least mangled with the gcc version it comes from.  Otherwise
> multiple parallel gcc installations conflict.

Ah, ok, that shouldn't be too hard.  I'll get working on a patch.

Thanks,
Gary

-- 
http://gbenson.net/

Re: Another Python namespace pollution (was: r147958 - in /trunk/libstdc++-v3: ChangeLog Mak...)

[Gerald Pfeifer]

On Thu, 27 Jan 2011, Gary Benson wrote:
>> The files should reside in a directory that is private to gcc or
>> at least mangled with the gcc version it comes from.  Otherwise
>> multiple parallel gcc installations conflict.
> Ah, ok, that shouldn't be too hard.  I'll get working on a patch.

Thanks!

In case you are wondering where to put them, there is prior art
based on Tom's fix:

  ./share/gcc-4.6.0/python/libstdcxx/__init__.py       <--- The good
  ./share/gcc-4.6.0/python/libstdcxx/v6/printers.py
  ./share/gcc-4.6.0/python/libstdcxx/v6/__init__.py
  ./share/python/classfile.py                          <--- The bad
  ./share/python/aotcompile.py
  ./lib/libstdc++.so.6-gdb.py                          <--- The ugly :-)
 
Gerald

Re: Another Python namespace pollution

[Matthias Klose]

On 27.01.2011 20:54, Gerald Pfeifer wrote:
> On Thu, 27 Jan 2011, Gary Benson wrote:
>>> The files should reside in a directory that is private to gcc or
>>> at least mangled with the gcc version it comes from.  Otherwise
>>> multiple parallel gcc installations conflict.
>> Ah, ok, that shouldn't be too hard.  I'll get working on a patch.
>
> Thanks!
>
> In case you are wondering where to put them, there is prior art
> based on Tom's fix:
>
>    ./share/gcc-4.6.0/python/libstdcxx/__init__.py<--- The good
>    ./share/gcc-4.6.0/python/libstdcxx/v6/printers.py
>    ./share/gcc-4.6.0/python/libstdcxx/v6/__init__.py
>    ./share/python/classfile.py<--- The bad
>    ./share/python/aotcompile.py
>    ./lib/libstdc++.so.6-gdb.py<--- The ugly :-)

afaiu the latter one is by design, and has to match the shared object name.

Re: Another Python namespace pollution (was: r147958 - in /trunk/libstdc++-v3: ChangeLog Mak...)

[Gary Benson]

[-- Attachment #1: Type: text/plain, Size: 1623 bytes --]

Gary Benson wrote:
> Richard Guenther wrote:
> > On Thu, Jan 27, 2011 at 11:38 AM, Gary Benson <gbenson@redhat.com> wrote:
> > > Gerald Pfeifer wrote:
> > > > I noticed that coming from the Java side of things we also
> > > > have two files that are installed in common namespace:
> > > >
> > > >   share/python/aotcompile.py
> > > >   share/python/classfile.py
> > >
> > > I'm on the hook to fix this, but it's been a while since I
> > > worked on this stuff so I'm a little out of touch with GCC.
> > > Can you tell me what you mean by "common namespace", and
> > > what the alternative should be?
> > 
> > The files should reside in a directory that is private to gcc or
> > at least mangled with the gcc version it comes from.  Otherwise
> > multiple parallel gcc installations conflict.
> 
> Ah, ok, that shouldn't be too hard.  I'll get working on a patch.

Ha ha, famous last words, I've been fighting this for days.

Ok, attached is a patch that causes the two Python modules to be
installed in a versioned directory.  Also attached is a separate
patch, orthogonal to the first, which changes how the modules are
made available to the aot-compile scripts.  What is currently in
svn adds the directory to the _end_ of the path, so if the modules
exist elsewhere then the other versions will take priority.  The
attached patch changes it to add the directory to the start of the
path.

If the patches are ok then could someone please commit them for me?
I thought I had commit access, but I couldn't seem to check out a
tree using ssh, so maybe it was revoked for inactivity :)

Cheers,
Gary

-- 
http://gbenson.net/

[-- Attachment #2: aot-compile-fix-take-1.patch --]
[-- Type: text/plain, Size: 2730 bytes --]

Index: libjava/configure.ac
===================================================================
--- libjava/configure.ac	(revision 169331)
+++ libjava/configure.ac	(working copy)
@@ -1842,21 +1842,18 @@
 # We get this from the environment.
 AC_SUBST(GCJFLAGS)
 
+default_python_dir="/share/gcc-`cat ${srcdir}/../gcc/BASE-VER`/python/libjava"
 AC_ARG_WITH([python-dir],
             AS_HELP_STRING([--with-python-dir],
                            [the location to install Python modules. This path should NOT include the prefix.]),
-            [with_python_dir=$withval], [with_python_dir=""])
+            [with_python_dir=$withval], [with_python_dir="${default_python_dir}"])
 
-if test "x${with_python_dir}" = "x"
-then
-  # Needed for installing Python modules during make install.
-  python_mod_dir="\${prefix}/share/python"
-  # Needed for substituting into aot-compile*
-  python_mod_dir_expanded="${expanded_prefix}/share/python"
-else
-  python_mod_dir="\${prefix}${with_python_dir}"
-  python_mod_dir_expanded="${expanded_prefix}${with_python_dir}"
-fi
+# Needed for installing Python modules during make install.
+python_mod_dir="\${prefix}${with_python_dir}"
+
+# Needed for substituting into aot-compile*
+python_mod_dir_expanded="${expanded_prefix}${with_python_dir}"
+
 AC_MSG_RESULT(Python modules dir: ${python_mod_dir_expanded});
 AC_SUBST(python_mod_dir)
 AC_SUBST(python_mod_dir_expanded)
Index: libjava/configure
===================================================================
--- libjava/configure	(revision 169331)
+++ libjava/configure	(working copy)
@@ -24649,25 +24649,22 @@
 # We get this from the environment.
 
 
+default_python_dir="/share/gcc-`cat ${srcdir}/../gcc/BASE-VER`/python/libjava"
 
 # Check whether --with-python-dir was given.
 if test "${with_python_dir+set}" = set; then :
   withval=$with_python_dir; with_python_dir=$withval
 else
-  with_python_dir=""
+  with_python_dir="${default_python_dir}"
 fi
 
 
-if test "x${with_python_dir}" = "x"
-then
-  # Needed for installing Python modules during make install.
-  python_mod_dir="\${prefix}/share/python"
-  # Needed for substituting into aot-compile*
-  python_mod_dir_expanded="${expanded_prefix}/share/python"
-else
-  python_mod_dir="\${prefix}${with_python_dir}"
-  python_mod_dir_expanded="${expanded_prefix}${with_python_dir}"
-fi
+# Needed for installing Python modules during make install.
+python_mod_dir="\${prefix}${with_python_dir}"
+
+# Needed for substituting into aot-compile*
+python_mod_dir_expanded="${expanded_prefix}${with_python_dir}"
+
 { $as_echo "$as_me:${as_lineno-$LINENO}: result: Python modules dir: ${python_mod_dir_expanded}" >&5
 $as_echo "Python modules dir: ${python_mod_dir_expanded}" >&6; };
 

[-- Attachment #3: aot-compile-improvement.patch --]
[-- Type: text/plain, Size: 1419 bytes --]

Index: libjava/contrib/aot-compile.in
===================================================================
--- libjava/contrib/aot-compile.in	(revision 169331)
+++ libjava/contrib/aot-compile.in	(working copy)
@@ -1,6 +1,6 @@
 #!/usr/bin/env python
 
-## Copyright (C) 2006 Free Software Foundation
+## Copyright (C) 2006, 2011 Free Software Foundation
 ## Written by Gary Benson <gbenson@redhat.com>
 ##
 ## This program is free software; you can redistribute it and/or modify
@@ -14,7 +14,7 @@
 ## GNU General Public License for more details.
 
 import sys
-sys.path.append("@python_mod_dir_expanded@")
+sys.path.insert(0, "@python_mod_dir_expanded@")
 import aotcompile
 import getopt
 import os
Index: libjava/contrib/aot-compile-rpm.in
===================================================================
--- libjava/contrib/aot-compile-rpm.in	(revision 169331)
+++ libjava/contrib/aot-compile-rpm.in	(working copy)
@@ -1,6 +1,6 @@
 #!/usr/bin/env python
 
-## Copyright (C) 2005, 2006, 2007 Free Software Foundation
+## Copyright (C) 2005, 2006, 2007, 2011 Free Software Foundation
 ## Written by Gary Benson <gbenson@redhat.com>
 ##
 ## This program is free software; you can redistribute it and/or modify
@@ -14,7 +14,7 @@
 ## GNU General Public License for more details.
 
 import sys
-sys.path.append("@python_mod_dir_expanded@")
+sys.path.insert(0, "@python_mod_dir_expanded@")
 import aotcompile
 import os
 

Re: Another Python namespace pollution (was: r147958 - in /trunk/libstdc++-v3: ChangeLog Mak...)

[Gerald Pfeifer]

On Wed, 2 Feb 2011, Gary Benson wrote:
> Ok, attached is a patch that causes the two Python modules to be
> installed in a versioned directory.

Thanks, Gary!  I see you committed this to HEAD; any chance you could
also push this to the GCC 4.5 branch which is our latest release(d)
branch?

Gerald

Re: Another Python namespace pollution

[Andrew Haley]

On 02/18/2011 02:20 PM, Gerald Pfeifer wrote:
> On Wed, 2 Feb 2011, Gary Benson wrote:
>> Ok, attached is a patch that causes the two Python modules to be
>> installed in a versioned directory.
>
> Thanks, Gary!  I see you committed this to HEAD; any chance you could
> also push this to the GCC 4.5 branch which is our latest release(d)
> branch?

4.5 is in serious regression fixes only mode.  What do you need this
for?

Andrew.