RE: ld sysroot scripts

RE: ld sysroot scripts

[Feuerbacher, Alan]

> From: Feuerbacher, Alan 
> Sent: Friday, July 20, 2012 10:06 AM
> To: 'binutils@sourceware.org'
> Subject: ld sysroot scripts
>
> Hi,
>
> I'm having a problem with cross compiling binutils, gcc and glibc on a
> Redhat5 system. I think that the problem is addressed by Alan Modra in his post "ld sysroot > > scripts":
>
> http://sourceware.org/ml/binutils/2012-03/msg00076.html
>
> I've tried applying the supplied patch, but get a lot of errors like "Hunk #2 FAILED at 83".
>
> Is there any information available on applying this patch, or on a later version?
>
> Obviously I'm trying to compile binutils-2.22.
>
> Alan Feuerbacher


Hi again,

I seem to have figured a way to get around the above-mentioned problem, but it's not really fixed.

After looking around for some time I found the latest binutils snapshots here:

ftp://sourceware.org/pub/binutils/snapshots/

After comparing one of the files (ldfile.c) mentioned in Alan Modra's above-mentioned patch between the latest snapshot (binutils-2.22.52) and the latest official version (binutils-2.22), I saw that the patch changes were in the latest version; hence no need to apply a patch.

I then cross-compiled binutils-2.22.52, gcc-4.7.1 and glibc-2.15 successfully, using variations of ideas from the Linux From Scratch folks. But I still ended up with the same problem as before -- as described in Alan Modra's patch link ( http://sourceware.org/ml/binutils/2012-03/msg00076.html ), namely, ld seems improperly to tack the sysroot prefix onto the library path when searching for three shared library files, and then complains that it can't find the files. This problem is more or less described at the end of this link:

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35804

This says:

************************
Alan Modra 2009-01-18 23:36:48 UTC

I believe that this is a user error and that this bugzilla should be closed. 
Your problem with ld "cannot find /tools/lib/libc.so.6 inside /tools" is simply that your glibc is built wrongly for your sysroot.  (glibc is not sysroot
aware.)  You'll find that the script libc.so has a line something like

GROUP ( /tools/lib/libc.so.6 /tools/usr/lib/libc_nonshared.a  AS_NEEDED (
/tools/lib/ld-linux.so.2 ) )

The /tools sysroot prefix ought not be present on any of these paths.
************************

I've run into exactly the same problem, where the three files libc.so.6, libc_nonshared.a and ld-linux.so.2 cannot be found by ld. 

The script libc.so ends up containing the following, which mentions these shared files:

************************
/* GNU ld script
   Use the shared library, but some functions are only in
   the static library, so try that secondarily.  */
OUTPUT_FORMAT(elf64-x86-64)
GROUP ( /home/afbacher/LFS71/lfs/tools/lib/libc.so.6 /home/afbacher/LFS71/lfs/tools/lib/libc_nonshared.a  AS_NEEDED ( /home/afbacher/LFS71/lfs/tools/lib/ld-linux-x86-64.so.2 ) )
************************

I compiled both binutils and gcc using "--with-sysroot=/home/afbacher/LFS71/lfs". I fixed the problem by removing the sysroot string from the last line of libc.so:

************************
GROUP ( /tools/lib/libc.so.6 /tools/lib/libc_nonshared.a  AS_NEEDED ( /tools/lib/ld-linux-x86-64.so.2 ) )
************************

The Linux From Scratch instructions tell you to do the following test after compiling binutils, gcc and glibc:

************************
echo 'main(){}' >| dummy.c
x86_64-lfs-linux-gnu-gcc dummy.c
readelf -l a.out
************************

You should see a line like this if everything is working right:

************************
[Requesting program interpreter: /home/afbacher/LFS71/lfs/tools/lib64/ld-linux-x86-64.so.2]
************************

However, immediately after compiling the three programs, the "x86_64-lfs-linux-gnu-gcc dummy.c" line fails with something like this for each of the shared files:

************************
/home/afbacher/LFS71/lfs/tools/lib/gcc/x86_64-lfs-linux-gnu/4.7.1/../../../../x86_64-lfs-linux-gnu/bin/ld: cannot find /home/afbacher/LFS71/fum/tools/lib/libc.so.6 inside /home/afbacher/LFS71/lfs
************************

But after I change the "GROUP" line in libc.so as described above, everything works fine.

It appears to me that either the last line of libc.so is being generated improperly (i.e., it improperly contains sysroot) or that ld is improperly tacking sysroot onto the front of the library paths defined in the GROUP statement in libc.so. I'm not qualified to do more than speculate about this, but it seems to me that the patch described in Alan Modra's references above is not working.

By the way, modifying libc.so as above, and using the un-patched binutils-2.22, also fixes the problem. This again suggests that the patch is not working as intended.

Again, any help in getting this stuff fixed will be appreciated.

Alan Feuerbacher

Re: ld sysroot scripts

[Alan Modra]

On Mon, Jul 23, 2012 at 02:58:44PM +0000, Feuerbacher, Alan wrote:
> By the way, modifying libc.so as above, and using the un-patched
> binutils-2.22, also fixes the problem.

OK, you have your fix then.

> This again suggests that the patch is not working as intended.

No, you misunderstand the purpose of my patch.  ld with the patch
consistently adds the sysroot prefix to paths in scripts found in the
sysroot.  You seem to want ld to *not* add the prefix, because you're
trying to use a script that already (wrongly) has the sysroot prefix
added to paths.

-- 
Alan Modra
Australia Development Lab, IBM

Re: ld sysroot scripts

[nick clifton]

Hi Alan,
> I'm having a problem with cross compiling binutils, gcc and glibc on a
> Redhat5 system.

Since you are using a Red Hat release, you are presumably paying for 
support.  In which case I would suggest that your first port of call 
would be to Red Hat support.


> Is there any information available on applying this patch,

There are no notes specific to applying this patch.

In general patches that do not apply cleanly can be applied by hand with 
a small amount of effort.


> or on a later version?

The patch is in H.J.'s latest binutils for the kernel tarball 
(binutils-2.22.52.0.4). See:

   http://ftp.kernel.org/pub/linux/devel/binutils/

You could just use these sources as a drop in replacement for your 
current linker sources.


> Is there any further information about the problem Modra's patch addresses?

Yes, the bug report that triggered the patch:

   http://sourceware.org/bugzilla/show_bug.cgi?id=10340


> Obviously I'm trying to compile binutils-2.22.

Except that RHEL5 is based on binutils 2.17 not 2.22.  If you are using 
the current 2.22 release then I would suggest that your best option is 
to either use a tarball from H.J's site (mentioned above) or else check 
out the current binutils development sources from the CVS repository and 
use those instead.

Cheers
   Nick

ld sysroot scripts

[Feuerbacher, Alan]

Hi,

I'm having a problem with cross compiling binutils, gcc and glibc on a
Redhat5 system. I think that the problem is addressed by Alan Modra in his post "ld sysroot scripts":

http://sourceware.org/ml/binutils/2012-03/msg00076.html

I've tried applying the supplied patch, but get a lot of errors like "Hunk #2 FAILED at 83".

Is there any information available on applying this patch, or on a later version?

Is there any further information about the problem Modra's patch addresses?

Obviously I'm trying to compile binutils-2.22.

Alan Feuerbacher

ld sysroot scripts

[Alan Modra]

Current linker treatment of absolute paths in scripts found in a
sysroot is odd.  What's odd is that whether or not such paths have the
sysroot prepended depends on how the script is found rather than where
the script is found.  For instance given a script libc.so in
$sysroot/usr/lib/, "ld ... -lc" will prepend $sysroot to absolute
paths in the script, while "ld ... $sysroot/usr/lib/libc.so" won't,
and similarly for INCLUDE and a bunch of other ways scripts may be
found.  This patch fixes the inconsistency, and removes some dead
code.

I'll note that the bug reporter wants to always search the sysroot
first regardless of where the script is found.  I'm not making that
policy change with this patch.  So copying a script out of a sysroot
will mean paths need adjusting.  (A symbolic link to script in a
sysroot will work though, since ld uses realpath.)

	PR ld/10340
	* ldfile.c (is_sysrooted_pathname): Remove notsame param.
	(ldfile_add_library_path): Don't set sysrooted flag.
	(ldfile_open_file_search): Likewise, and don't copy them.
	(try_open): Delete exten and code handling such.  Add sysrooted
	param and return whether path is in sysroot.
	(ldfile_find_command_file): Delete extend param.  Add sysrooted
	param.  Rename local var.  Update try_open calls.
	(ldfile_open_command_file_1): Pass sysrooted to lex_push_file.
	* ldfile.h (search_dirs_type): Remove sysrooted field.
	* ldlang.c (new_afile): Always set sysrooted from input_flags.
	(load_symbols): Don't set input_flags.sysrooted.
	* ldlang.h (struct lang_input_statement_flags): Revise sysrooted
	comment.
	* ldlex.h (lex_push_file): Update prototype.
	* ldlex.l (sysrooted_stack): New array.
	(EOF): Pop input_flags.sysrooted.
	(lex_push_file): Add sysrooted param.  Save and set
	input_flags.sysrooted.

Index: ld/ldfile.c
===================================================================
RCS file: /cvs/src/src/ld/ldfile.c,v
retrieving revision 1.65
diff -u -p -r1.65 ldfile.c
--- ld/ldfile.c	5 Mar 2012 22:43:40 -0000	1.65
+++ ld/ldfile.c	8 Mar 2012 03:42:42 -0000
@@ -70,7 +70,7 @@ static search_arch_type **search_arch_ta
    sub-directory of the sysroot directory.  */
 
 static bfd_boolean
-is_sysrooted_pathname (const char *name, bfd_boolean notsame)
+is_sysrooted_pathname (const char *name)
 {
   char *realname;
   int len;
@@ -82,18 +82,13 @@ is_sysrooted_pathname (const char *name,
   realname = lrealpath (name);
   len = strlen (realname);
   result = FALSE;
-  if (len == ld_canon_sysroot_len)
-    result = !notsame;
-  else if (len > ld_canon_sysroot_len
-	   && IS_DIR_SEPARATOR (realname[ld_canon_sysroot_len]))
+  if (len > ld_canon_sysroot_len
+      && IS_DIR_SEPARATOR (realname[ld_canon_sysroot_len]))
     {
-      result = TRUE;
       realname[ld_canon_sysroot_len] = '\0';
+      result = FILENAME_CMP (ld_canon_sysroot, realname) == 0;
     }
 
-  if (result)
-    result = FILENAME_CMP (ld_canon_sysroot, realname) == 0;
-
   free (realname);
   return result;
 }
@@ -118,15 +113,9 @@ ldfile_add_library_path (const char *nam
   /* If a directory is marked as honoring sysroot, prepend the sysroot path
      now.  */
   if (name[0] == '=')
-    {
-      new_dirs->name = concat (ld_sysroot, name + 1, (const char *) NULL);
-      new_dirs->sysrooted = TRUE;
-    }
+    new_dirs->name = concat (ld_sysroot, name + 1, (const char *) NULL);
   else
-    {
-      new_dirs->name = xstrdup (name);
-      new_dirs->sysrooted = is_sysrooted_pathname (name, FALSE);
-    }
+    new_dirs->name = xstrdup (name);
 }
 
 /* Try to open a BFD for a lang_input_statement.  */
@@ -364,12 +353,7 @@ ldfile_open_file_search (const char *arc
 	  free (name);
 	}
       else if (ldfile_try_open_bfd (entry->filename, entry))
-	{
-	  entry->flags.sysrooted
-	    = (IS_ABSOLUTE_PATH (entry->filename)
-	       && is_sysrooted_pathname (entry->filename, TRUE));
-	  return TRUE;
-	}
+	return TRUE;
 
       if (IS_ABSOLUTE_PATH (entry->filename))
 	return FALSE;
@@ -382,10 +366,7 @@ ldfile_open_file_search (const char *arc
       if (entry->flags.dynamic && ! link_info.relocatable)
 	{
 	  if (ldemul_open_dynamic_archive (arch, search, entry))
-	    {
-	      entry->flags.sysrooted = search->sysrooted;
-	      return TRUE;
-	    }
+	    return TRUE;
 	}
 
       if (entry->flags.maybe_archive)
@@ -398,7 +379,6 @@ ldfile_open_file_search (const char *arc
       if (ldfile_try_open_bfd (string, entry))
 	{
 	  entry->filename = string;
-	  entry->flags.sysrooted = search->sysrooted;
 	  return TRUE;
 	}
 
@@ -473,15 +453,18 @@ ldfile_open_file (lang_input_statement_t
     }
 }
 
-/* Try to open NAME; if that fails, try NAME with EXTEN appended to it.  */
+/* Try to open NAME.  */
 
 static FILE *
-try_open (const char *name, const char *exten)
+try_open (const char *name, bfd_boolean *sysrooted)
 {
   FILE *result;
 
   result = fopen (name, "r");
 
+  if (result != NULL)
+    *sysrooted = is_sysrooted_pathname (name);
+
   if (trace_file_tries)
     {
       if (result == NULL)
@@ -490,26 +473,6 @@ try_open (const char *name, const char *
 	info_msg (_("opened script file %s\n"), name);
     }
 
-  if (result != NULL)
-    return result;
-
-  if (*exten)
-    {
-      char *buff;
-
-      buff = concat (name, exten, (const char *) NULL);
-      result = fopen (buff, "r");
-
-      if (trace_file_tries)
-	{
-	  if (result == NULL)
-	    info_msg (_("cannot find script file %s\n"), buff);
-	  else
-	    info_msg (_("opened script file %s\n"), buff);
-	}
-      free (buff);
-    }
-
   return result;
 }
 
@@ -570,22 +533,23 @@ find_scripts_dir (void)
 
 /* If DEFAULT_ONLY is false, try to open NAME; if that fails, look for
    it in directories specified with -L, then in the default script
-   directory, without and with EXTEND appended.  If DEFAULT_ONLY is
-   true, the search is restricted to the default script location.  */
+   directory.  If DEFAULT_ONLY is true, the search is restricted to
+   the default script location.  */
 
 static FILE *
-ldfile_find_command_file (const char *name, const char *extend,
-			  bfd_boolean default_only)
+ldfile_find_command_file (const char *name,
+			  bfd_boolean default_only,
+			  bfd_boolean *sysrooted)
 {
   search_dirs_type *search;
   FILE *result = NULL;
-  char *buffer;
+  char *path;
   static search_dirs_type *script_search;
 
   if (!default_only)
     {
       /* First try raw name.  */
-      result = try_open (name, "");
+      result = try_open (name, sysrooted);
       if (result != NULL)
 	return result;
     }
@@ -611,9 +575,9 @@ ldfile_find_command_file (const char *na
        search != NULL;
        search = search->next)
     {
-      buffer = concat (search->name, slash, name, (const char *) NULL);
-      result = try_open (buffer, extend);
-      free (buffer);
+      path = concat (search->name, slash, name, (const char *) NULL);
+      result = try_open (path, sysrooted);
+      free (path);
       if (result)
 	break;
     }
@@ -630,7 +594,9 @@ static void
 ldfile_open_command_file_1 (const char *name, bfd_boolean default_only)
 {
   FILE *ldlex_input_stack;
-  ldlex_input_stack = ldfile_find_command_file (name, "", default_only);
+  bfd_boolean sysrooted;
+
+  ldlex_input_stack = ldfile_find_command_file (name, default_only, &sysrooted);
 
   if (ldlex_input_stack == NULL)
     {
@@ -638,7 +604,7 @@ ldfile_open_command_file_1 (const char *
       einfo (_("%P%F: cannot open linker script file %s: %E\n"), name);
     }
 
-  lex_push_file (ldlex_input_stack, name);
+  lex_push_file (ldlex_input_stack, name, sysrooted);
 
   lineno = 1;
 
Index: ld/ldfile.h
===================================================================
RCS file: /cvs/src/src/ld/ldfile.h,v
retrieving revision 1.19
diff -u -p -r1.19 ldfile.h
--- ld/ldfile.h	17 Feb 2012 14:09:57 -0000	1.19
+++ ld/ldfile.h	8 Mar 2012 03:42:42 -0000
@@ -37,8 +37,6 @@ typedef struct search_dirs {
   const char *name;
   /* TRUE if this is from the command line.  */
   bfd_boolean cmdline;
-  /* true if this is from within the sys-root.  */
-  bfd_boolean sysrooted;
 } search_dirs_type;
 
 extern search_dirs_type *search_head;
Index: ld/ldlang.c
===================================================================
RCS file: /cvs/src/src/ld/ldlang.c,v
retrieving revision 1.385
diff -u -p -r1.385 ldlang.c
--- ld/ldlang.c	5 Mar 2012 22:43:40 -0000	1.385
+++ ld/ldlang.c	8 Mar 2012 03:42:43 -0000
@@ -1064,6 +1064,7 @@ new_afile (const char *name,
   p->flags.add_DT_NEEDED_for_dynamic = input_flags.add_DT_NEEDED_for_dynamic;
   p->flags.add_DT_NEEDED_for_regular = input_flags.add_DT_NEEDED_for_regular;
   p->flags.whole_archive = input_flags.whole_archive;
+  p->flags.sysrooted = input_flags.sysrooted;
 
   if (file_type == lang_input_file_is_l_enum
       && name[0] == ':' && name[1] != '\0')
@@ -1101,7 +1102,6 @@ new_afile (const char *name,
       p->local_sym_name = name;
       p->flags.real = TRUE;
       p->flags.search_dirs = TRUE;
-      p->flags.sysrooted = input_flags.sysrooted;
       break;
     case lang_input_file_is_file_enum:
       p->filename = name;
@@ -2716,7 +2716,6 @@ load_symbols (lang_input_statement_type 
       ldfile_open_command_file (entry->filename);
 
       push_stat_ptr (place);
-      input_flags.sysrooted = entry->flags.sysrooted;
       input_flags.add_DT_NEEDED_for_regular
 	= entry->flags.add_DT_NEEDED_for_regular;
       input_flags.add_DT_NEEDED_for_dynamic
@@ -2729,7 +2728,9 @@ load_symbols (lang_input_statement_type 
       yyparse ();
       ldfile_assumed_script = FALSE;
 
-      /* missing_file is sticky.  */
+      /* missing_file is sticky.  sysrooted will already have been
+	 restored when seeing EOF in yyparse, but no harm to restore
+	 again.  */
       save_flags.missing_file |= input_flags.missing_file;
       input_flags = save_flags;
       pop_stat_ptr ();
Index: ld/ldlang.h
===================================================================
RCS file: /cvs/src/src/ld/ldlang.h,v
retrieving revision 1.102
diff -u -p -r1.102 ldlang.h
--- ld/ldlang.h	5 Mar 2012 22:43:40 -0000	1.102
+++ ld/ldlang.h	8 Mar 2012 03:42:43 -0000
@@ -236,10 +236,7 @@ struct lang_input_statement_flags
   /* 1 means search a set of directories for this file.  */
   unsigned int search_dirs : 1;
 
-  /* 1 means this was found in a search directory marked as sysrooted,
-     if search_dirs is false, otherwise, that it should be
-     searched in ld_sysroot before any other location, as long as it
-     starts with a slash.  */
+  /* 1 means this was found when processing a script in the sysroot.  */
   unsigned int sysrooted : 1;
 
   /* 1 means this is base file of incremental load.
Index: ld/ldlex.h
===================================================================
RCS file: /cvs/src/src/ld/ldlex.h,v
retrieving revision 1.9
diff -u -p -r1.9 ldlex.h
--- ld/ldlex.h	17 Feb 2012 14:09:57 -0000	1.9
+++ ld/ldlex.h	8 Mar 2012 03:42:43 -0000
@@ -41,7 +41,7 @@ extern const char *lex_string;
 
 /* In ldlex.l.  */
 extern int yylex (void);
-extern void lex_push_file (FILE *, const char *);
+extern void lex_push_file (FILE *, const char *, unsigned int);
 extern void lex_redirect (const char *, const char *, unsigned int);
 extern void ldlex_script (void);
 extern void ldlex_mri_script (void);
Index: ld/ldlex.l
===================================================================
RCS file: /cvs/src/src/ld/ldlex.l,v
retrieving revision 1.54
diff -u -p -r1.54 ldlex.l
--- ld/ldlex.l	17 Feb 2012 14:09:57 -0000	1.54
+++ ld/ldlex.l	8 Mar 2012 03:42:44 -0000
@@ -68,6 +68,7 @@ const char *lex_string = NULL;
 static YY_BUFFER_STATE include_stack[MAX_INCLUDE_DEPTH];
 static const char *file_name_stack[MAX_INCLUDE_DEPTH];
 static unsigned int lineno_stack[MAX_INCLUDE_DEPTH];
+static unsigned int sysrooted_stack[MAX_INCLUDE_DEPTH];
 static unsigned int include_stack_ptr = 0;
 static int vers_node_nesting = 0;
 
@@ -454,6 +455,7 @@ V_IDENTIFIER [*?.$_a-zA-Z\[\]\-\!\^\\]([
     yy_switch_to_buffer (include_stack[include_stack_ptr]);
 
   lineno = lineno_stack[include_stack_ptr];
+  input_flags.sysrooted = sysrooted_stack[include_stack_ptr];
 
   return END;
 }
@@ -468,7 +470,7 @@ V_IDENTIFIER [*?.$_a-zA-Z\[\]\-\!\^\\]([
    saving the current input info on the include stack.  */
 
 void
-lex_push_file (FILE *file, const char *name)
+lex_push_file (FILE *file, const char *name, unsigned int sysrooted)
 {
   if (include_stack_ptr >= MAX_INCLUDE_DEPTH)
     {
@@ -476,10 +478,12 @@ lex_push_file (FILE *file, const char *n
     }
   file_name_stack[include_stack_ptr] = name;
   lineno_stack[include_stack_ptr] = lineno;
+  sysrooted_stack[include_stack_ptr] = input_flags.sysrooted;
   include_stack[include_stack_ptr] = YY_CURRENT_BUFFER;
 
   include_stack_ptr++;
   lineno = 1;
+  input_flags.sysrooted = sysrooted;
   yyin = file;
   yy_switch_to_buffer (yy_create_buffer (yyin, YY_BUF_SIZE));
 }

-- 
Alan Modra
Australia Development Lab, IBM