[Bug testsuite/44488] New: objc test inconsistencies w/ / w/out --enable-build-with-cxx

[Bug testsuite/44488] New: objc test inconsistencies w/ / w/out --enable-build-with-cxx

[amylaar at gcc dot gnu dot org]

Sometime between revision 160389 and revision 160448,
testsuite/objc/compile/compile.exp started to use a new set of options for
overy test in testsuite/objc/compile/compile.exp, using -fnext-runtime
instead of -fgnu-runtime, thus adding 5 tests * 6 option compinations,
i.e. 30 new expected passes to the test results, for a default configuration
bootstrap/regtest on i686-pc-linux-gnu.  This is still the case at
revision 160461.
Running a bootstrap/regtest on i686-pc-linux-gnu with the configure option
--enable-build-with-cxx, I do not see these new test variants, and thus 30
fewer expected passes.

With the old trunk, or with the new trunk configured with
--enable-build-with-cxx, I see there is a test of -fnext-runtime at
the start of running compile.exp, which fails:

Executing on host: /user/inria/fsf/bldxx-20100607/gcc/xgcc
-B/user/inria/fsf/bldxx-20100607/gcc/
/user/inria/fsf/gcc/gcc/testsuite/objc/compile/trivial.m  -fnext-runtime 
-B/user/inria/fsf/bldxx-20100607/i686-pc-linux-gnu/./libobjc/.libs  
-L/user/inria/fsf/bldxx-20100607/i686-pc-linux-gnu/./libobjc/.libs  -S  -o
trivial.s    (timeout = 300)
In file included from
/user/inria/fsf/gcc/gcc/testsuite/objc/compile/trivial.m:1:0:
/user/inria/fsf/gcc/gcc/testsuite/objc/compile/../../objc-obj-c++-shared/Object1.h:21:31:
fatal error: objc/Object.h: No such file or directory
compilation terminated.
compiler exited with status 1
output is:
In file included from
/user/inria/fsf/gcc/gcc/testsuite/objc/compile/trivial.m:1:0:
/user/inria/fsf/gcc/gcc/testsuite/objc/compile/../../objc-obj-c++-shared/Object1.h:21:31:
fatal error: objc/Object.h: No such file or directory
compilation terminated.

Using the following runtimes: -fgnu-runtime

With the new trunk configured without --enable-build-with-cxx, this test
appears to succeed:

Executing on host: /user/inria/fsf/bld-20100609/gcc/xgcc
-B/user/inria/fsf/bld-20100609/gcc/
/user/inria/fsf/gcc/gcc/testsuite/objc/compile/trivial.m  -fnext-runtime 
-B/user/inria/fsf/bld-20100609/i686-pc-linux-gnu/./libobjc/.libs  
-L/user/inria/fsf/bld-20100609/i686-pc-linux-gnu/./libobjc/.libs  -S  -o
trivial.s    (timeout = 300)
Using the following runtimes: -fgnu-runtime -fnext-runtime


-- 
           Summary: objc test inconsistencies w/ / w/out --enable-build-
                    with-cxx
           Product: gcc
           Version: 4.6.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: testsuite
        AssignedTo: unassigned at gcc dot gnu dot org
        ReportedBy: amylaar at gcc dot gnu dot org
  GCC host triplet: i686-pc-linux-gnu
OtherBugsDependingO 44433
             nThis:


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44488

[Bug objc/44488] objc test inconsistencies w/ / w/out --enable-build-with-cxx

[pinskia at gcc dot gnu dot org]

------- Comment #1 from pinskia at gcc dot gnu dot org  2010-06-10 01:04 -------
-fnext-runtime should fail on x86-linux-gnu anyways.  Why it is not failing is
a good question.  If this is failing without  --enable-build-with-cxx there is
a bug somewhere.


-- 

pinskia at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
OtherBugsDependingO|44433                       |
              nThis|                            |
          Component|testsuite                   |objc


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44488

[Bug objc/44488] objc test inconsistencies w/ / w/out --enable-build-with-cxx

[pinskia at gcc dot gnu dot org]

------- Comment #2 from pinskia at gcc dot gnu dot org  2010-06-10 01:06 -------
On x86_64-linux-gnu it does the correct thing:
Executing on host: /home/pinskia/src/local/gcc/objdir/gcc/xgcc
-B/home/pinskia/src/local/gcc/objdir/gcc/
/home/pinskia/src/local/gcc/gcc/testsuite/objc/compile/trivial.m  -fgnu-runtime
-I/home/pinskia/src/local/gcc/gcc/testsuite/../../libobjc 
-B/home/pinskia/src/local/gcc/objdir/x86_64-unknown-linux-gnu/./libobjc/.libs  
-L/home/pinskia/src/local/gcc/objdir/x86_64-unknown-linux-gnu/./libobjc/.libs 
-S  -o trivial.s    (timeout = 300)
set_ld_library_path_env_vars:
ld_library_path=.::/home/pinskia/src/local/gcc/objdir/gcc:/home/pinskia/src/local/gcc/objdir/x86_64-unknown-linux-gnu/./libobjc/.libs
Executing on host: /home/pinskia/src/local/gcc/objdir/gcc/xgcc
-B/home/pinskia/src/local/gcc/objdir/gcc/
/home/pinskia/src/local/gcc/gcc/testsuite/objc/compile/trivial.m 
-fnext-runtime 
-B/home/pinskia/src/local/gcc/objdir/x86_64-unknown-linux-gnu/./libobjc/.libs  
-L/home/pinskia/src/local/gcc/objdir/x86_64-unknown-linux-gnu/./libobjc/.libs 
-S  -o trivial.s    (timeout = 300)
In file included from
/home/pinskia/src/local/gcc/gcc/testsuite/objc/compile/trivial.m:1:0:^M
/home/pinskia/src/local/gcc/gcc/testsuite/objc/compile/../../objc-obj-c++-shared/Object1.h:39:30:
fatal error: objc/objc-runtime.h: No such file or directory^M
compilation terminated.^M
compiler exited with status 1
output is:
In file included from
/home/pinskia/src/local/gcc/gcc/testsuite/objc/compile/trivial.m:1:0:^M
/home/pinskia/src/local/gcc/gcc/testsuite/objc/compile/../../objc-obj-c++-shared/Object1.h:39:30:
fatal error: objc/objc-runtime.h: No such file or directory^M
compilation terminated.^M

Using the following runtimes: -fgnu-runtime


This is without --enable-build-with-cxx .


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44488

[Bug objc/44488] objc test inconsistencies w/ / w/out --enable-build-with-cxx

[amylaar at gcc dot gnu dot org]

------- Comment #3 from amylaar at gcc dot gnu dot org  2010-06-10 05:41 -------
I see the same problem now with revision 160389.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44488

[Bug objc/44488] objc test inconsistencies w/ / w/out --enable-build-with-cxx

[amylaar at gcc dot gnu dot org]

------- Comment #4 from amylaar at gcc dot gnu dot org  2010-06-10 09:18 -------
I've found now that the runtime is picked up from the install location.
For default configuration, I use --prefix=/user/inria , and the file is
then picked up from
/user/inria/lib/gcc/i686-pc-linux-gnu/4.6.0/include/objc/Object.h ,
where it has been installed on my machine during a recent test install
of a previous build.

For --enable-build-with-cxx, I've been using a different prefix.

Is the compiler supposed to ignore an installed objc gnu runtime when testing
in the build directory with -fnext-runtime?


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44488

[Bug objc/44488] objc test inconsistencies w/ / w/out --enable-build-with-cxx

[amylaar at gcc dot gnu dot org]

------- Comment #5 from amylaar at gcc dot gnu dot org  2010-06-10 09:28 -------
(In reply to comment #4)
> I've found now that the runtime is picked up from the install location.

P.S.: That doesn't happen when I cut & paste the LD_LIBRARY_PATH settings
and the compile command from the log file - I only found out where the
runtime came from by hacking testsuite/lib/objc-torture.exp to include
--save-temps in additional_flags and inspecting the generated trivial.mi
file.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44488

[Bug objc/44488] objc test inconsistencies w/ / w/out --enable-build-with-cxx

[iains at gcc dot gnu dot org]

------- Comment #6 from iains at gcc dot gnu dot org  2010-07-09 23:40 -------
(In reply to comment #4)

> Is the compiler supposed to ignore an installed objc gnu runtime when testing
> in the build directory with -fnext-runtime?

Well, it's actually the other way round - it needs to find the pre-installed
one on a Darwin system to succeed for NeXT (which happens to be in
/usr/include/objc).  That path is built into the compiler.

I guess it's generally the case (across various parts of the test suite) that
installed versions can, unfortunately, interfere with uninstalled testing. 

I will try to refine the test in some way to make this less likely to happen. 
The "-fnext-runtime" flag should be tested on anything other than a Darwin
system (although most, if not all, of the compile tests would succeed it's
kinda pointless).


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44488

[Bug objc/44488] objc test inconsistencies w/ / w/out --enable-build-with-cxx

[iains at gcc dot gnu dot org]

------- Comment #7 from iains at gcc dot gnu dot org  2010-07-10 10:38 -------
Created an attachment (id=21173)
 --> (http://gcc.gnu.org/bugzilla/attachment.cgi?id=21173&action=view)
improve robustness of runtime option choices.

This should resolve the case where -fnext-runtime would be considered valid
(for compile tests) because the target (non-darwin) system happened to have
objc headers in the same place as a standard darwin installation.

Tested on i686-apple-darwin9 [build --enable-build-with-cxx] , cris-elf (sim)
and mipsabi64-elf (sim) with no changes in fails or counts on make
check-gcc-objc.

let me know if it solve that aspect of the problem for you.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44488

[Bug objc/44488] objc test inconsistencies w/ / w/out --enable-build-with-cxx

[amylaar at gcc dot gnu dot org]

------- Comment #8 from amylaar at gcc dot gnu dot org  2010-07-11 01:01 -------
(In reply to comment #7)
> Created an attachment (id=21173)
 --> (http://gcc.gnu.org/bugzilla/attachment.cgi?id=21173&action=view) [edit] 
...
> let me know if it solve that aspect of the problem for you.

Yes, it does.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44488

[Bug objc/44488] objc test inconsistencies w/ / w/out --enable-build-with-cxx

[iains at gcc dot gnu dot org]

------- Comment #9 from iains at gcc dot gnu dot org  2010-07-13 15:20 -------
Subject: Bug 44488

Author: iains
Date: Tue Jul 13 15:20:21 2010
New Revision: 162144

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=162144
Log:

        PR objc/44488
        * lib/objc-torture.exp (objc-set-runtime-options): Base runtime list
        on the target.  Make sure that we can assemble the emitted asm when 
        the test type is 'compile'.


Modified:
    trunk/gcc/testsuite/ChangeLog
    trunk/gcc/testsuite/lib/objc-torture.exp


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44488

[Bug objc/44488] objc test inconsistencies w/ / w/out --enable-build-with-cxx

[iains at gcc dot gnu dot org]

------- Comment #10 from iains at gcc dot gnu dot org  2010-07-13 15:32 -------
of course, there should not be different behavior with/without
--enable-build-with-cxx, so I guess that the test-suite fix is only part of the
solution.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44488

[Bug objc/44488] objc test inconsistencies w/ / w/out --enable-build-with-cxx

[iains at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44488

Iain Sandoe <iains at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |RESOLVED
         Resolution|---                         |FIXED

--- Comment #14 from Iain Sandoe <iains at gcc dot gnu.org> ---
I'm closing this as fixed for the ObjC issues, any that remain due to
installed/uninstalled versions interacting relate to GCC as a whole, rather
than ObjC.

[Bug objc/44488] objc test inconsistencies w/ / w/out --enable-build-with-cxx

[iains at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44488

--- Comment #13 from Iain Sandoe <iains at gcc dot gnu.org> 2011-01-11 08:36:42 UTC ---
(In reply to comment #12)
> (In reply to comment #11)
> > should this be closed as fixed - and, if not, what is the remaining issue?
> 
> The remaining issue is that the just built compiler under test looks in
> the install location.  I.e. bogus files from a previous borched install could
> cause failures, or a missing file in the build could be masked by having the
> file in the install location.

OK, that is agreed, but AFAIK, it applies to GCC (or the test-suite) as a
whole.
(I'm sure every dev. has at some stage been burnt by a pre-existing install).

It's also an issue when one wants to build version X.Y.Z to sit in the same
place as version P.Q.R.
 (--enable-version-specific-runtime-libs is a great option :) ).

Here, I always: 
(a) do uninstalled testing
(b) use a different (and unpopulated) install location from the final one when
testing.

(after all, one tends to debug with --enable-checking=yes and build the final
with --enable-checking=release).

---

So, unless I'm missing some finer point of your comment, this is not an
ObjC-specific problem -- perhaps a new PR could be raised for an enhancement to
greater robustness of the test-suite in the presence of installed programs?

[Bug objc/44488] objc test inconsistencies w/ / w/out --enable-build-with-cxx

[amylaar at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44488

--- Comment #12 from Jorn Wolfgang Rennecke <amylaar at gcc dot gnu.org> 2011-01-10 23:18:09 UTC ---
(In reply to comment #11)
> should this be closed as fixed - and, if not, what is the remaining issue?

The remaining issue is that the just built compiler under test looks in
the install location.  I.e. bogus files from a previous borched install could
cause failures, or a missing file in the build could be masked by having the
file in the install location.
There are ways to manage this issue, like always picking a new, un-populated
install location for test builds.  That's what I usually do now, so this
problem affects me no longer.
OTOH, if such a test methodology is generally used, we run the risk that
building with a populated install location will bitrot.

[Bug objc/44488] objc test inconsistencies w/ / w/out --enable-build-with-cxx

[iains at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44488

--- Comment #11 from Iain Sandoe <iains at gcc dot gnu.org> 2011-01-10 20:25:19 UTC ---
should this be closed as fixed - and, if not, what is the remaining issue?