[JAVA] /bin/sh portability issues in gen-classlist.sh.in

[JAVA] /bin/sh portability issues in gen-classlist.sh.in

[Roger Sayle]

The following patch, the first in a series, fixes some shell portability
issues building libjava using /bin/sh on Solaris.  As pointed out by
Andrew Pinski, I should probably simply use CONFIG_SHELL, however I
think I've managed to resolve all the remaining shell issues preventing
bootstraping mainline GCC on Solaris/x86 with the native /bin/sh.

The first pair of issues is in libjava/classpath/lib/gen-classlist.sh.in
which uses test's "-ef" and "-e" operators which aren't available on
Solaris.  Many thanks to Ian Lance Taylor for recommending the more
portable idiom used below.

Tested on i386-pc-solaris2.10 where a top-level "make bootstrap" makes
it further with this patch than without.

Ok for mainline?



2006-07-06  Roger Sayle  <roger@eyesopen.com>
	    Ian Lance Taylor  <ian@airs.com>

	* lib/gen-classlist.sh.in:  Avoid using test's -ef operator for
	increased portability.  Likewise, use -f instead of -e.


Index: lib/gen-classlist.sh.in
===================================================================
*** lib/gen-classlist.sh.in	(revision 115223)
--- lib/gen-classlist.sh.in	(working copy)
*************** for dir in $vm_dirlist; do
*** 55,61 ****
  done

  # Only include generated files once.
! if test ! "${top_builddir}" -ef "@top_srcdir@"; then
    echo "Adding generated files in builddir '${top_builddir}'."
    # Currently the only generated files are in gnu.*.
    (cd ${top_builddir}; @FIND@ gnu -follow -name '*.java' -print) |
--- 55,61 ----
  done

  # Only include generated files once.
! if test `cd "${top_builddir}"; pwd` != `cd "@top_srcdir@"; pwd`; then
    echo "Adding generated files in builddir '${top_builddir}'."
    # Currently the only generated files are in gnu.*.
    (cd ${top_builddir}; @FIND@ gnu -follow -name '*.java' -print) |
*************** rm vm.add
*** 95,101 ****
  rm tmp.omit

  new=
! if test -e ${top_builddir}/lib/classes.2; then
    p=`diff ${top_builddir}/lib/classes.2 ${top_builddir}/lib/classes.1`
    if test "$p" != ""; then
      new="true"
--- 95,101 ----
  rm tmp.omit

  new=
! if test -f ${top_builddir}/lib/classes.2; then
    p=`diff ${top_builddir}/lib/classes.2 ${top_builddir}/lib/classes.1`
    if test "$p" != ""; then
      new="true"


Roger
--

Re: [JAVA] /bin/sh portability issues in gen-classlist.sh.in

[Andrew Haley]

These seem mostly to be patches for the upstream Classpath library
that is periodically merged into libgcj.  We need to make sure that
they are submitted there,

classpath-patches@gnu.org

Andrew.

Re: [JAVA] /bin/sh portability issues in gen-classlist.sh.in

[Eric Botcazou]

> The following patch, the first in a series, fixes some shell portability
> issues building libjava using /bin/sh on Solaris.  As pointed out by
> Andrew Pinski, I should probably simply use CONFIG_SHELL, however I
> think I've managed to resolve all the remaining shell issues preventing
> bootstraping mainline GCC on Solaris/x86 with the native /bin/sh.

I think it's a lost cause, in my experience /bin/sh randomly fails to 
configure target libraries on Solaris.  I'll personally make no effort
to support /bin/sh on Solaris even if your patches go in, i.e. I'll keep 
closing PRs related to the problem as invalid.

-- 
Eric Botcazou

Re: [JAVA] /bin/sh portability issues in gen-classlist.sh.in

[Roger Sayle]

On Fri, 7 Jul 2006, Andrew Haley wrote:
> These seem mostly to be patches for the upstream Classpath library
> that is periodically merged into libgcj.  We need to make sure that
> they are submitted there,
>
> classpath-patches@gnu.org

My apologies to the classpath-patches folks.  I did check the
recent postings to the java-patches list, but I couldn't find a
crosspost mentioning where classpatch patches should be directed.

For those classpatch folks who don't read the java list, two out
of four of my recent series of patches to support building GCC on
i386-pc-solaris2.10 using the native /bin/sh affect classpatch
code, and as mentioned by Andrew above should be reviewed
upstream.

I'm happy to repost them to the classpatch-patches list if that's
required.  But the original postings can be found at:

http://gcc.gnu.org/ml/gcc-patches/2006-07/msg00293.html
http://gcc.gnu.org/ml/gcc-patches/2006-07/msg00294.html

Sorry for the inconvenience,

Roger
--

Re: [JAVA] /bin/sh portability issues in gen-classlist.sh.in

[Andrew Haley]

Roger Sayle writes:
 > 
 > I'm happy to repost them to the classpatch-patches list if that's
 > required.  But the original postings can be found at:
 > 
 > http://gcc.gnu.org/ml/gcc-patches/2006-07/msg00293.html
 > http://gcc.gnu.org/ml/gcc-patches/2006-07/msg00294.html
 > 
 > Sorry for the inconvenience,

Hey, no need to apologize!  You are much appreciated for your work.

Andrew.

Re: [JAVA] /bin/sh portability issues in gen-classlist.sh.in

[Paolo Bonzini]

>   # Only include generated files once.
> ! if test ! "${top_builddir}" -ef "@top_srcdir@"; then
>     echo "Adding generated files in builddir '${top_builddir}'."
>     # Currently the only generated files are in gnu.*.
>     (cd ${top_builddir}; @FIND@ gnu -follow -name '*.java' -print) |
> --- 55,61 ----
>   done
> 
>   # Only include generated files once.
> ! if test `cd "${top_builddir}"; pwd` != `cd "@top_srcdir@"; pwd`; then

Might be a problem when directories have spaces in it (remember this 
script is also used on Windows):

   $ test `echo foo bar` != `echo foo bar`
   bash: test: too many arguments

Unfortunately, "`...`" is unportable if the substituted command also 
have quotes.  But a=`foo` does not break words so you can do this, which 
may also be marginally more readable:

   abs_top_builddir=`cd "${top_builddir}"; pwd`
   abs_top_srcdir=`cd "@top_srcdir@"; pwd`
   if test "$abs_top_builddir" != "$abs_top_srcdir"; then
     ...

Paolo

Re: [JAVA] /bin/sh portability issues in gen-classlist.sh.in

[Roger Sayle]

On Fri, 7 Jul 2006, Eric Botcazou wrote:
> > The following patch, the first in a series, fixes some shell portability
> > issues building libjava using /bin/sh on Solaris.  As pointed out by
> > Andrew Pinski, I should probably simply use CONFIG_SHELL, however I
> > think I've managed to resolve all the remaining shell issues preventing
> > bootstraping mainline GCC on Solaris/x86 with the native /bin/sh.
>
> I think it's a lost cause, in my experience /bin/sh randomly fails to
> configure target libraries on Solaris.  I'll personally make no effort
> to support /bin/sh on Solaris even if your patches go in, i.e. I'll keep
> closing PRs related to the problem as invalid.

I'm reminded that the GCC Development Mission Statement, explicitly
states "to ensure that GCC ... works on multiple architectures and
diverse environments".  http://gcc.gnu.org/gccmission.html

I don't disagree that there might still be fundamental bugs in Solaris'
/bin/sh or even the kernel that ultimately make this task fruitless.
However, knowing that /bin/sh has been a bane of the GCC build system
for many years, I was surprised at how easy the port was.  GCC 3.4.3 built
out of the box, and mainline three stage bootstrapped the compilers and
built most of the runtime libraries only to fail in multilibed boehmgc
and libjava.  This even makes this issue a regression.


I regularly use CONFIG_SHELL on several platforms, but I suspect most
often for performance rather than for correctness, or to workaround
portability issue.

Running "what /bin/sh" on my Solaris/x86 box shows that the executable
is dated 1st October 2005.  Perhaps after nearly/over? ten years of
brokeness SUN's code is now stable enough for GCC to meet them half
way?

There is also a fundamental difference between the problems I'm fixing
and the issues you report (that I may just not have encountered yet).
Instability issues and missing POSIX support are bugs that should be
filed with SUN Microsystems.  But using command line options and
functionality not available in the tool being invoked are bugs in
GCC's build system.  The latter I can do something about myself,
and the former may get fixed sooner or later (perhaps already?).

Roger
--

Re: [JAVA] /bin/sh portability issues in gen-classlist.sh.in

[Eric Botcazou]

> I'm reminded that the GCC Development Mission Statement, explicitly
> states "to ensure that GCC ... works on multiple architectures and
> diverse environments".  http://gcc.gnu.org/gccmission.html

Sure, GCC works just fine on Solaris with /bin/ksh.

> However, knowing that /bin/sh has been a bane of the GCC build system
> for many years, I was surprised at how easy the port was.  GCC 3.4.3 built
> out of the box, and mainline three stage bootstrapped the compilers and
> built most of the runtime libraries only to fail in multilibed boehmgc
> and libjava.

On how many versions of Solaris?

> Running "what /bin/sh" on my Solaris/x86 box shows that the executable
> is dated 1st October 2005.  Perhaps after nearly/over? ten years of
> brokeness SUN's code is now stable enough for GCC to meet them half
> way?

AFAIK Sun has essentially pledged to keep /bin/sh bug-compatible with earlier 
releases so I wouldn't depend on that.

-- 
Eric Botcazou

Re: [JAVA] /bin/sh portability issues in gen-classlist.sh.in

[Andrew Pinski]

On Jul 6, 2006, at 9:10 PM, Roger Sayle wrote:

> The first pair of issues is in libjava/classpath/lib/gen- 
> classlist.sh.in
> which uses test's "-ef" and "-e" operators which aren't available on
> Solaris.  Many thanks to Ian Lance Taylor for recommending the more
> portable idiom used below.

Actually they are available on Solaris, just not with the builtin  
inside /bin/sh.
This is just one reason why /bin/sh should not be used.  We do  
document that we
require a real POSIX shell and /bin/sh is not complaint in other  
ways, this is
just one.  In fact Sun has said many times, they are not going to  
change /bin/sh
to be a POSIX shell.

After building you should try to install the compiler, it will not  
work as some
substitutions are not done correctly.  I can point you to the bug  
reports that
were reported.

Also this really should be sent upstream and not to GCC patches.


-- Pinski

Re: [JAVA] /bin/sh portability issues in gen-classlist.sh.in

[Ralf Wildenhues]

* Andrew Pinski wrote on Sat, Jul 08, 2006 at 02:57:26AM CEST:
> On Jul 6, 2006, at 9:10 PM, Roger Sayle wrote:
> 
> >The first pair of issues is in libjava/classpath/lib/gen- 
> >classlist.sh.in
> >which uses test's "-ef" and "-e" operators which aren't available on
> >Solaris.  Many thanks to Ian Lance Taylor for recommending the more
> >portable idiom used below.
> 
> Actually they are available on Solaris, just not with the builtin
> inside /bin/sh.  This is just one reason why /bin/sh should not be
> used.  We do  document that we require a real POSIX shell and /bin/sh
> is not complaint in other  ways, this is just one.

AFAIK 'test -ef' is not POSIX.  It's mentioned in SUSv3 as 'considered
in an early proposal' but then rejected because of the non-presence in
historical sh implementations.

(This is not to say that your argument against sh on Solaris isn't
valid.  I simply haven't yet had a chance to look into the bugs it
causes elsewhere in the install.)

Cheers,
Ralf

Re: [JAVA] /bin/sh portability issues in gen-classlist.sh.in

[Tom Tromey]

>>>>> "Roger" == Roger Sayle <roger@eyesopen.com> writes:

Finally getting to these patches...

Roger> The first pair of issues is in libjava/classpath/lib/gen-classlist.sh.in
Roger> which uses test's "-ef" and "-e" operators which aren't available on
Roger> Solaris.  Many thanks to Ian Lance Taylor for recommending the more
Roger> portable idiom used below.

Roger> Ok for mainline?

It is fine by me, but I wanted to ask first whether Paolo's objection
(directories with spaces) needed to be addressed first.  Do we support
that?

Whatever we decide, I'll commit the result to Classpath...

Tom

Re: [JAVA] /bin/sh portability issues in gen-classlist.sh.in

[Roger Sayle]

Hi Tom,

On 18 Aug 2006, Tom Tromey wrote:
> Roger> Ok for mainline?
>
> It is fine by me, but I wanted to ask first whether Paolo's objection
> (directories with spaces) needed to be addressed first.  Do we support
> that?
>
> Whatever we decide, I'll commit the result to Classpath...

Sorry for the delay.  Here's a revised version of the gen-classlist.sh.in
change that incorporates Paolo's suggestions.  This patch has been tested
on i386-pc-solaris2.10 (with a tweaked boehm-gc) using the native /bin/sh
where it allows compilation of libjava to get further than previously.

Thanks,


2006-08-24  Roger Sayle  <roger@eyesopen.com>
	    Ian Lance Taylor  <ian@airs.com>
	    Paolo Bonzini <bonzini@gnu.org>

	* lib/gen-classlist.sh.in:  Avoid using test's -ef operator for
	increased portability.  Likewise, use -f instead of -e.


Index: lib/gen-classlist.sh.in
===================================================================
*** lib/gen-classlist.sh.in	(revision 116321)
--- lib/gen-classlist.sh.in	(working copy)
*************** for dir in $vm_dirlist; do
*** 55,61 ****
  done

  # Only include generated files once.
! if test ! "${top_builddir}" -ef "@top_srcdir@"; then
    echo "Adding generated files in builddir '${top_builddir}'."
    # Currently the only generated files are in gnu.*.
    (cd ${top_builddir}; @FIND@ gnu -follow -name '*.java' -print) |
--- 55,63 ----
  done

  # Only include generated files once.
! abs_top_builddir=`cd "${top_builddir}"; pwd`
! abs_top_srcdir=`cd "@top_srcdir@"; pwd`
! if test "$abs_top_builddir" != "$abs_top_srcdir"; then
    echo "Adding generated files in builddir '${top_builddir}'."
    # Currently the only generated files are in gnu.*.
    (cd ${top_builddir}; @FIND@ gnu -follow -name '*.java' -print) |
*************** rm vm.add
*** 95,101 ****
  rm tmp.omit

  new=
! if test -e ${top_builddir}/lib/classes.2; then
    p=`diff ${top_builddir}/lib/classes.2 ${top_builddir}/lib/classes.1`
    if test "$p" != ""; then
      new="true"
--- 97,103 ----
  rm tmp.omit

  new=
! if test -f ${top_builddir}/lib/classes.2; then
    p=`diff ${top_builddir}/lib/classes.2 ${top_builddir}/lib/classes.1`
    if test "$p" != ""; then
      new="true"


Roger
--

Re: [JAVA] /bin/sh portability issues in gen-classlist.sh.in

[Tom Tromey]

>>>>> "Roger" == Roger Sayle <roger@eyesopen.com> writes:

Roger> Sorry for the delay.  Here's a revised version of the
Roger> gen-classlist.sh.in change that incorporates Paolo's
Roger> suggestions.  This patch has been tested on i386-pc-solaris2.10
Roger> (with a tweaked boehm-gc) using the native /bin/sh where it
Roger> allows compilation of libjava to get further than previously.

Sorry, I don't think I ever responded to this.
This is ok for trunk; I will check it in to classpath.

Tom