Should the linker tests notice multilib variations ?

Should the linker tests notice multilib variations ?

[Nick Clifton]

Hi Guys,

  I noticed today that the linker testsuite ignores any mutlilib
  switches set up by the schedule of variants when it is run.  This
  seems a little strange to me, especially now that the testsuite is 
  testing compiler initiated features such as selective linking.

  I have appended a patch below which attempts to add support for the
  multilib switches, but I am not very happy with it.  In particular I
  have had to add some grubby code to detect if the code was compiled 
  with a big endian or little endian switch and so pass on a -EB or
  -EL switch to the linker or assembler.

  I had to do this because for the ARM toolchain at least, the
  compiler does not accept -EB only -mbig-endian, whereas the linker
  only accepts -EB not -mbig-endian.  Normally gcc would invoke the
  linker, and so it would translate a -mbig-endian command line switch
  into a -EB switch.  But in the case of the linker testsuite, the
  compiler and linker are each invoked seperately, and the switches
  passed to the linker do not get the opportunity to be translated by
  gcc. 

  This patch still breaks several of the ld tests for the ARM however,
  because the testsuite uses custom linker scripts which do not
  include a mutliple option OUTPUT_FORMAT directive, so the default
  output format (little endian arm) is used, and this generates an
  error when used to link big endian arm object files.  I am not sure
  how to solve this problem.  Perhaps the linker ought to be more
  clever about selecting an output format if it is told on the command
  line the endianness that the user desires.

Cheers
	Nick


1999-07-13  Nick Clifton  <nickc@cygnus.com>

	* lib/ld-lib.exp (proc big_or_little_endian): New proc.
	Determine if a big endian or little endian output format hass
	been selected by any of the multilib options, and if so return
	a suitable command line option for the linker/assembler.
	(proc default_ld_link): Include the result of proc
	big_or_little_endian on the command line to the linker.
	(proc ld_simple_link): Include the result of proc
	big_or_little_endian on the command line to the linker.
	(proc default_ld_compile): Append multilib flags to compiler
	switches.
	(proc default_ld_assemble): Include the result of proc
	big_or_little_endian on the command line to the linker.

Index: ld-lib.exp
===================================================================
RCS file: /cvs/cvsfiles/devo/ld/testsuite/lib/ld-lib.exp,v
retrieving revision 1.5
diff -p -r1.5 ld-lib.exp
*** ld-lib.exp	1999/06/20 17:46:35	1.5
--- ld-lib.exp	1999/07/13 10:18:36
*************** proc default_ld_relocate { ld target obj
*** 43,48 ****
--- 43,79 ----
      }
  }
  
+ # Look for big-endian or little-endian switches in the multlib
+ # options and translate these into a -EB or -EL switch.  Note
+ # we cannot rely upon proc process_multilib_options to do this
+ # for us because for some targets the compiler does not support
+ # -EB/-EL but it does support -mbig-endian/-mlittle-endian, and
+ # the site.exp file will include the switch "-mbig-endian"
+ # (rather than "big-endian") which is not detected by proc
+ # process_multilib_options.
+ 
+ proc big_or_little_endian {} {
+     
+     if [board_info [target_info name] exists multilib_flags] {
+ 	set tmp_flags " [board_info [target_info name] multilib_flags]";
+ 
+ 	foreach x $tmp_flags {
+ 	    case $x in {
+ 		{*big*endian eb EB} {
+ 		    set flags " -EB"
+ 		    return $flags
+ 		}
+ 		{*little*endian el EL} {
+ 		    set flags " -EL"
+ 		    return $flags
+ 		}
+ 	    }
+ 	}
+     }
+ 
+     set flags ""
+     return $flags
+ }
  
  #
  # default_ld_link 
*************** proc default_ld_link { ld target objects
*** 61,70 ****
  	perror "$ld does not exist"
  	return 0
      }
      
!     verbose -log "$ld $HOSTING_EMU -o $target $objs $libs"
      
!     catch "exec $ld $HOSTING_EMU -o $target $objs $libs" exec_output
      set exec_output [prune_warnings $exec_output]
      if [string match "" $exec_output] then {
  	return 1
--- 92,103 ----
  	perror "$ld does not exist"
  	return 0
      }
+     
+     set flags [big_or_little_endian]
      
!     verbose -log "$ld $HOSTING_EMU $flags -o $target $objs $libs"
      
!     catch "exec $ld $HOSTING_EMU $flags -o $target $objs $libs" exec_output
      set exec_output [prune_warnings $exec_output]
      if [string match "" $exec_output] then {
  	return 1
*************** proc default_ld_simple_link { ld target 
*** 85,94 ****
  	perror "$ld does not exist"
  	return 0
      }
      
!     verbose -log "$ld -o $target $objects"
      
!     catch "exec $ld -o $target $objects" exec_output
      set exec_output [prune_warnings $exec_output]
  
      # We don't care if we get a warning about a non-existent start
--- 118,129 ----
  	perror "$ld does not exist"
  	return 0
      }
+ 
+     set flags [big_or_little_endian]
      
!     verbose -log "$ld $flags -o $target $objects"
      
!     catch "exec $ld $flags -o $target $objects" exec_output
      set exec_output [prune_warnings $exec_output]
  
      # We don't care if we get a warning about a non-existent start
*************** proc default_ld_compile { cc source obje
*** 134,139 ****
--- 169,178 ----
  	set flags "$gcc_gas_flag $flags"
      }
  
+     if [board_info [target_info name] exists multilib_flags] {
+ 	append flags " [board_info [target_info name] multilib_flags]";
+     }
+ 
      verbose -log "$cc $flags -c $source -o $object"
  
      catch "exec $cc $flags -c $source -o $object" exec_output
*************** proc default_ld_assemble { as source obj
*** 180,188 ****
  
      if ![info exists ASFLAGS] { set ASFLAGS "" }
  
!     verbose -log "$as $ASFLAGS -o $object $source"
  
!     catch "exec $as $ASFLAGS -o $object $source" exec_output
      set exec_output [prune_warnings $exec_output]
      if [string match "" $exec_output] then {
  	return 1
--- 219,229 ----
  
      if ![info exists ASFLAGS] { set ASFLAGS "" }
  
!     set flags [big_or_little_endian]
!     
!     verbose -log "$as $flags $ASFLAGS -o $object $source"
  
!     catch "exec $as $flags $ASFLAGS -o $object $source" exec_output
      set exec_output [prune_warnings $exec_output]
      if [string match "" $exec_output] then {
  	return 1

Re: Should the linker tests notice multilib variations ?

[Ian Lance Taylor]

   Date: Tue, 13 Jul 1999 11:24:21 +0100
   From: Nick Clifton <nickc@cygnus.com>

     I noticed today that the linker testsuite ignores any mutlilib
     switches set up by the schedule of variants when it is run.  This
     seems a little strange to me, especially now that the testsuite is 
     testing compiler initiated features such as selective linking.

I've never really understood the current DejaGnu.  Does it have any
documentation yet?

     I have appended a patch below which attempts to add support for the
     multilib switches, but I am not very happy with it.  In particular I
     have had to add some grubby code to detect if the code was compiled 
     with a big endian or little endian switch and so pass on a -EB or
     -EL switch to the linker or assembler.

     I had to do this because for the ARM toolchain at least, the
     compiler does not accept -EB only -mbig-endian, whereas the linker
     only accepts -EB not -mbig-endian.  Normally gcc would invoke the
     linker, and so it would translate a -mbig-endian command line switch
     into a -EB switch.  But in the case of the linker testsuite, the
     compiler and linker are each invoked seperately, and the switches
     passed to the linker do not get the opportunity to be translated by
     gcc. 

I think this sort of hackery is inevitable.  Unfortunately, while -EB
and -EL are globally recognized options for the linker, they are not
for the assembler.  For example, the PowerPC assembler uses -be and
-le rather than -EB and -EL.  So you need a more clever hack.  Or you
need to add support for -EB and -EL to the PowerPC assembler, which
would be OK.  Perhaps we should even make -EB and -EL global options
for the assembler, and use some new md function to implement them.

     This patch still breaks several of the ld tests for the ARM however,
     because the testsuite uses custom linker scripts which do not
     include a mutliple option OUTPUT_FORMAT directive, so the default
     output format (little endian arm) is used, and this generates an
     error when used to link big endian arm object files.  I am not sure
     how to solve this problem.  Perhaps the linker ought to be more
     clever about selecting an output format if it is told on the command
     line the endianness that the user desires.

I don't think BFD provides the information required to switch between
big and little endian versions of the same format.  I'm not sure what
the best way to represent that information would be; perhaps simply a
new field in the bfd_target structure to hold the address of another
bfd_target which is the same as the current one, but reverse endian.
Then the linker could check the byteorder field to see whether it
should try to use the other endianness.

It's OK with me to check in your patch.

Ian

Re: Should the linker tests notice multilib variations ?

[Doug Evans]

   Date: 13 Jul 1999 12:02:05 -0400
   From: Ian Lance Taylor <ian@zembu.com>

   Perhaps we should even make -EB and -EL global options
   for the assembler, and use some new md function to implement them.

I think this is the way to go.
I'll even volunteer to implement it ...

Do we need a new md function though?

Studying tc-ppc.[ch], set_target_endian seems superfluous.
target_big_endian is initialized to TARGET_BYTES_BIG_ENDIAN in read.c.
Then at the end of md_begin, if !set_target_endian, target_big_endian
gets set to TARGET_BYTES_BIG_ENDIAN again.
(PPC_BIG_ENDIAN == TARGET_BYTES_BIG_ENDIAN).

Re: Should the linker tests notice multilib variations ?

[Ian Lance Taylor]

   Date: Tue, 13 Jul 1999 09:29:57 -0700
   From: Doug Evans <devans@cygnus.com>

      Perhaps we should even make -EB and -EL global options
      for the assembler, and use some new md function to implement them.

   I think this is the way to go.
   I'll even volunteer to implement it ...

Thanks.

   Do we need a new md function though?

   Studying tc-ppc.[ch], set_target_endian seems superfluous.
   target_big_endian is initialized to TARGET_BYTES_BIG_ENDIAN in read.c.
   Then at the end of md_begin, if !set_target_endian, target_big_endian
   gets set to TARGET_BYTES_BIG_ENDIAN again.
   (PPC_BIG_ENDIAN == TARGET_BYTES_BIG_ENDIAN).

As I recall, different gas targets used to implement endianness
differently.  Then somebody introduced the global target_big_endian
variable.  If you can straighten things out so that no md function is
required, so much the better.

On the other hand, we do need some code somewhere that checks whether
the option can actually be supported--we probably shouldn't just
accept and ignore -EL and generate a big endian file--so we may need
some target specific information.  Or maybe the same BFD information
that Nick wants for the linker can be used here as well.

Ian

Re: Should the linker tests notice multilib variations ?

[Doug Evans]

   Date: 13 Jul 1999 12:34:18 -0400
   From: Ian Lance Taylor <ian@zembu.com>

   On the other hand, we do need some code somewhere that checks whether
   the option can actually be supported--we probably shouldn't just
   accept and ignore -EL and generate a big endian file--so we may need
   some target specific information.  Or maybe the same BFD information
   that Nick wants for the linker can be used here as well.

Right, we may need some md support for that (but I'll try to avoid it).

Re: Should the linker tests notice multilib variations ?

[Ian Lance Taylor]

   Date: Tue, 13 Jul 1999 18:20:21 +0100
   From: Nick Clifton <nickc@cygnus.com>

   The problem however is more general than that.  If my patch were to be
   applied then we would start running into all kinds of problems with
   failed links because the switches and library search directories
   passed to the linker would not match up with the binaries being
   produced by the compiler.

   The endian problem is one example of this.  The compiler is passed the
   -mbig-endian switch, produces a big-endian object file, and then the
   link fails because it uses a linker script which only uses the default
   output format which is (say) little endian.

   But the problem is not restricted to endianism.  Any multilib option
   which has an effect at link time is going to cause problems.  Suppose
   for example that a target has two different APIs and multilibs are
   provided for both versions.  When the linker tests are run only the
   default libraries will be found (because gcc is not being used to
   drive the link process), and so there will be link time fails when
   binaries for one API are linked with libraries for the other API.

Most of the linker tests are intentionally not linked against any
other libraries.

The exceptions are tests like bootstrap and cdtest, in which the
linker generates an executable which is then run.  Those tests
naturally only work on native systems.  Most native systems don't use
multilibs, so my offhand thinking is that we need not worry about them
too much.

   The only way to be sure that correct executables are produced is to
   allow gcc to do the linking as well as the compiling, rather than
   invoking the linker directly (since gcc may have to translate some
   command line in order for the linker to understand them, and it also
   has to sort out the paths to the multilib libraries), and to always
   use the built in linker script which will support multiple output
   formats.  

I wouldn't object to doing this for tests which must build an
executable which is to be run.  However, the tests need to work if gcc
is not present, so there needs to be a reasonable fallback.

   I am not sure if the first requirement will be a problem.  Currently
   the linker testsuite invokes the linker directly, and presumably there
   is a good reason for this.

It's desirable to be able to test the linker independently from gcc.
It's OK to assume that gas is present, since they are both in the
binutils.  It's not OK to assume that gcc, or indeed any compiler, is
present.  I'm sure you yourself have been in the position of testing
the binutils for a target for which you do not have any compiler.

   The second requirement is too restrictive however, as the linker 
   tests need to be able to use their own scripts.  This is where Ian's
   suggestion comes in.

As far as I know, the linker tests which use their own scripts do not
link against any other libraries.

Ian

Re: Should the linker tests notice multilib variations ?

[Nick Clifton]

Hi Ian, Hi Doug,

: Date: 13 Jul 1999 12:02:05 -0400
: From: Ian Lance Taylor <ian@zembu.com>
: 
:    Date: Tue, 13 Jul 1999 11:24:21 +0100
:    From: Nick Clifton <nickc@cygnus.com>
: 
:      I noticed today that the linker testsuite ignores any mutlilib
:      switches set up by the schedule of variants when it is run.  This
:      seems a little strange to me, especially now that the testsuite is 
:      testing compiler initiated features such as selective linking.
: 
: I've never really understood the current DejaGnu.  Does it have any
: documentation yet?

he he he.  Documentation !  No such luck.

: I think this sort of hackery is inevitable.  Unfortunately, while -EB
: and -EL are globally recognized options for the linker, they are not
: for the assembler.  For example, the PowerPC assembler uses -be and
: -le rather than -EB and -EL.  So you need a more clever hack.  Or you
: need to add support for -EB and -EL to the PowerPC assembler, which
: would be OK.  Perhaps we should even make -EB and -EL global options
: for the assembler, and use some new md function to implement them.

It certainly would be helpful if -EL and -EB could be globally
acceptable as options for choosing endian-ness.  (Assuming that the
target supports both endian variations).  I can fix up the ARM
compiler front end, and if Doug wants to fix the assembler that is
great. 

The problem however is more general than that.  If my patch were to be
applied then we would start running into all kinds of problems with
failed links because the switches and library search directories
passed to the linker would not match up with the binaries being
produced by the compiler.

The endian problem is one example of this.  The compiler is passed the
-mbig-endian switch, produces a big-endian object file, and then the
link fails because it uses a linker script which only uses the default
output format which is (say) little endian.

But the problem is not restricted to endianism.  Any multilib option
which has an effect at link time is going to cause problems.  Suppose
for example that a target has two different APIs and multilibs are
provided for both versions.  When the linker tests are run only the
default libraries will be found (because gcc is not being used to
drive the link process), and so there will be link time fails when
binaries for one API are linked with libraries for the other API.

The only way to be sure that correct executables are produced is to
allow gcc to do the linking as well as the compiling, rather than
invoking the linker directly (since gcc may have to translate some
command line in order for the linker to understand them, and it also
has to sort out the paths to the multilib libraries), and to always
use the built in linker script which will support multiple output
formats.  

I am not sure if the first requirement will be a problem.  Currently
the linker testsuite invokes the linker directly, and presumably there
is a good reason for this.

The second requirement is too restrictive however, as the linker 
tests need to be able to use their own scripts.  This is where Ian's
suggestion comes in.

: I don't think BFD provides the information required to switch between
: big and little endian versions of the same format.  I'm not sure what
: the best way to represent that information would be; perhaps simply a
: new field in the bfd_target structure to hold the address of another
: bfd_target which is the same as the current one, but reverse endian.
: Then the linker could check the byteorder field to see whether it
: should try to use the other endianness.

This should work, now all I need is some free time to work on it...

: It's OK with me to check in your patch.

I'll hold off for now - it breaks too many tests.

Cheers
	Nick