Reproducible builds - supporting relative paths in *-prefix-map

Reproducible builds - supporting relative paths in *-prefix-map

[Richard Purdie]

Hi,

I'm wondering if we'd be able to improve path handling in the -f*-
prefix-map compiler options to cover relative paths?

Currently it works well for absolute paths but if a file uses a
relative path or a path with a symlink in, or a non-absolute path, it
will miss those cases. For relative paths in particular it is
problematic as you can't easily construct a compiler commandline that
would cover all relative path options.

At first glance this is relatively straight forward, for example:

Index: gcc-12.1.0/gcc/file-prefix-map.cc
===================================================================
--- gcc-12.1.0.orig/gcc/file-prefix-map.cc
+++ gcc-12.1.0/gcc/file-prefix-map.cc
@@ -70,19 +70,25 @@ remap_filename (file_prefix_map *maps, c
   file_prefix_map *map;
   char *s;
   const char *name;
+  char *realname;
   size_t name_len;
 
+  realname = lrealpath (filename);
+
   for (map = maps; map; map = map->next)
-    if (filename_ncmp (filename, map->old_prefix, map->old_len) == 0)
+    if (filename_ncmp (realname, map->old_prefix, map->old_len) == 0)
       break;
-  if (!map)
+  if (!map) {
+    free (realname);
     return filename;
-  name = filename + map->old_len;
+  }
+  name = realname + map->old_len;
   name_len = strlen (name) + 1;
 
   s = (char *) ggc_alloc_atomic (name_len + map->new_len);
   memcpy (s, map->new_prefix, map->new_len);
   memcpy (s + map->new_len, name, name_len);
+  free (realname);
   return s;
 }
 
which address a realpath() call into the prefix mapping code. I did
experiment with this and found it breaks compiling ruby and xen-tools
which both have code which does:

#include __FILE__

It may be possible to make the remapping conditional of not being
directly in a #include statement but I didn't find the gcc code
responsible for that as yet. I also noticed some valgrind tests fails
after it, I've not looked into why that would be yet.

I wanted to ask if there would be any interest in adding support for
something like this? I suspect the include/__FILE__ issue is probably a
latent bug anyway. If anyone has any pointers to the code I could
improve my patch with I'm also happy to have them!

In case it helps, my background on what I'm working on and why we'd
find this useful follows: 

I work on Yocto Project which cross compiles complete software stacks,
we use gcc heavily and the resulting builds are all around us. We care
deeply about build reproducibility.

With autotools, we long ago realised that we were best off separating
the source and the build output where we could and to do it we used
relative paths, which removed a lot of problematic hardcoded paths in
our output.

More recently we've been using -fdebug-prefix-map and -fmacro-prefix-
map (we need both since some of binutils doesn't seem to support the
combined version yet). We use the debug information (split separately)
to support on target debugging but where relative paths were used, we
miss information as the remapping either doesn't happen correctly or is
mangled. We'd like to fix our debug output but we're finding that hard
without relative path support for *prefix-map.

Cheers,

Richard

Re: Reproducible builds - supporting relative paths in *-prefix-map

[Richard Purdie]

On Mon, 2022-08-15 at 12:13 +0100, Richard Purdie via Gcc wrote:
> Hi,
> 
> I'm wondering if we'd be able to improve path handling in the -f*-
> prefix-map compiler options to cover relative paths?
> 
> Currently it works well for absolute paths but if a file uses a
> relative path or a path with a symlink in, or a non-absolute path, it
> will miss those cases. For relative paths in particular it is
> problematic as you can't easily construct a compiler commandline that
> would cover all relative path options.
> 
> At first glance this is relatively straight forward, for example:
> 
> Index: gcc-12.1.0/gcc/file-prefix-map.cc
> ===================================================================
> --- gcc-12.1.0.orig/gcc/file-prefix-map.cc
> +++ gcc-12.1.0/gcc/file-prefix-map.cc
> @@ -70,19 +70,25 @@ remap_filename (file_prefix_map *maps, c
>    file_prefix_map *map;
>    char *s;
>    const char *name;
> +  char *realname;
>    size_t name_len;
>  
> +  realname = lrealpath (filename);
> +
>    for (map = maps; map; map = map->next)
> -    if (filename_ncmp (filename, map->old_prefix, map->old_len) == 0)
> +    if (filename_ncmp (realname, map->old_prefix, map->old_len) == 0)
>        break;
> -  if (!map)
> +  if (!map) {
> +    free (realname);
>      return filename;
> -  name = filename + map->old_len;
> +  }
> +  name = realname + map->old_len;
>    name_len = strlen (name) + 1;
>  
>    s = (char *) ggc_alloc_atomic (name_len + map->new_len);
>    memcpy (s, map->new_prefix, map->new_len);
>    memcpy (s + map->new_len, name, name_len);
> +  free (realname);
>    return s;
>  }
>  
> which address a realpath() call into the prefix mapping code. I did
> experiment with this and found it breaks compiling ruby and xen-tools
> which both have code which does:
> 
> #include __FILE__
> 
> It may be possible to make the remapping conditional of not being
> directly in a #include statement but I didn't find the gcc code
> responsible for that as yet. I also noticed some valgrind tests fails
> after it, I've not looked into why that would be yet.
> 
> I wanted to ask if there would be any interest in adding support for
> something like this? I suspect the include/__FILE__ issue is probably a
> latent bug anyway. If anyone has any pointers to the code I could
> improve my patch with I'm also happy to have them!

To answer my own question, something like:

+Index: gcc-12.1.0/libcpp/macro.cc
+===================================================================
+--- gcc-12.1.0.orig/libcpp/macro.cc
++++ gcc-12.1.0/libcpp/macro.cc
+@@ -563,7 +563,7 @@ _cpp_builtin_macro_text (cpp_reader *pfi
+           if (!name)
+             abort ();
+         }
+-      if (pfile->cb.remap_filename)
++      if (pfile->cb.remap_filename && !pfile->state.in_directive)
+         name = pfile->cb.remap_filename (name);
+       len = strlen (name);
+       buf = _cpp_unaligned_alloc (pfile, len * 2 + 3);

seems to do roughly what I was wondering about.

I'd be interested to understand whether some patch along the lines I've
mentioned here would stand a chance of being accepted or not.

Cheers,

Richard

Re: Reproducible builds - supporting relative paths in *-prefix-map

[David Edelsohn]

On Mon, Aug 15, 2022 at 10:28 AM Richard Purdie via Gcc <gcc@gcc.gnu.org> wrote:
>
> On Mon, 2022-08-15 at 12:13 +0100, Richard Purdie via Gcc wrote:
> > Hi,
> >
> > I'm wondering if we'd be able to improve path handling in the -f*-
> > prefix-map compiler options to cover relative paths?
> >
> > Currently it works well for absolute paths but if a file uses a
> > relative path or a path with a symlink in, or a non-absolute path, it
> > will miss those cases. For relative paths in particular it is
> > problematic as you can't easily construct a compiler commandline that
> > would cover all relative path options.
> >
> > At first glance this is relatively straight forward, for example:
> >
> > Index: gcc-12.1.0/gcc/file-prefix-map.cc
> > ===================================================================
> > --- gcc-12.1.0.orig/gcc/file-prefix-map.cc
> > +++ gcc-12.1.0/gcc/file-prefix-map.cc
> > @@ -70,19 +70,25 @@ remap_filename (file_prefix_map *maps, c
> >    file_prefix_map *map;
> >    char *s;
> >    const char *name;
> > +  char *realname;
> >    size_t name_len;
> >
> > +  realname = lrealpath (filename);
> > +
> >    for (map = maps; map; map = map->next)
> > -    if (filename_ncmp (filename, map->old_prefix, map->old_len) == 0)
> > +    if (filename_ncmp (realname, map->old_prefix, map->old_len) == 0)
> >        break;
> > -  if (!map)
> > +  if (!map) {
> > +    free (realname);
> >      return filename;
> > -  name = filename + map->old_len;
> > +  }
> > +  name = realname + map->old_len;
> >    name_len = strlen (name) + 1;
> >
> >    s = (char *) ggc_alloc_atomic (name_len + map->new_len);
> >    memcpy (s, map->new_prefix, map->new_len);
> >    memcpy (s + map->new_len, name, name_len);
> > +  free (realname);
> >    return s;
> >  }
> >
> > which address a realpath() call into the prefix mapping code. I did
> > experiment with this and found it breaks compiling ruby and xen-tools
> > which both have code which does:
> >
> > #include __FILE__
> >
> > It may be possible to make the remapping conditional of not being
> > directly in a #include statement but I didn't find the gcc code
> > responsible for that as yet. I also noticed some valgrind tests fails
> > after it, I've not looked into why that would be yet.
> >
> > I wanted to ask if there would be any interest in adding support for
> > something like this? I suspect the include/__FILE__ issue is probably a
> > latent bug anyway. If anyone has any pointers to the code I could
> > improve my patch with I'm also happy to have them!
>
> To answer my own question, something like:
>
> +Index: gcc-12.1.0/libcpp/macro.cc
> +===================================================================
> +--- gcc-12.1.0.orig/libcpp/macro.cc
> ++++ gcc-12.1.0/libcpp/macro.cc
> +@@ -563,7 +563,7 @@ _cpp_builtin_macro_text (cpp_reader *pfi
> +           if (!name)
> +             abort ();
> +         }
> +-      if (pfile->cb.remap_filename)
> ++      if (pfile->cb.remap_filename && !pfile->state.in_directive)
> +         name = pfile->cb.remap_filename (name);
> +       len = strlen (name);
> +       buf = _cpp_unaligned_alloc (pfile, len * 2 + 3);
>
> seems to do roughly what I was wondering about.
>
> I'd be interested to understand whether some patch along the lines I've
> mentioned here would stand a chance of being accepted or not.

Thanks for recognizing this issue and proposing a solution.  It's
probably more effective to submit this as an actual patch to
gcc-patches and cc David Malcolm, libcpp maintainer, than to ask
hypotheticals on the GCC mailing list.

Thanks, David

Re: Reproducible builds - supporting relative paths in *-prefix-map

[Joseph Myers]

On Mon, 15 Aug 2022, Richard Purdie via Gcc wrote:

> Currently it works well for absolute paths but if a file uses a
> relative path or a path with a symlink in, or a non-absolute path, it
> will miss those cases. For relative paths in particular it is
> problematic as you can't easily construct a compiler commandline that
> would cover all relative path options.

I'd expect a relative path to be naturally relocatable without needing to 
be remapped.  (For example, DW_AT_comp_dir would be relocated in debug 
info, but there would be no need to relocate the paths to individual files 
that are relative to DW_AT_comp_dir.)

Is the issue that you're using relative paths between two directories that 
don't have a fixed relative path between them, such as between the build 
and source directories, as opposed to relative paths within the source 
directory or within the build directory?

> which address a realpath() call into the prefix mapping code. I did

That would run the risk of breaking relocation for anyone who has 
deliberately used the paths they pass to the compiler (possibly involving 
symlinks, for example) in their remapping options - not expecting a 
further level of processing to be applied to those paths before remapping.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: Reproducible builds - supporting relative paths in *-prefix-map

[Richard Purdie]

On Mon, 2022-08-15 at 17:15 +0000, Joseph Myers wrote:
> On Mon, 15 Aug 2022, Richard Purdie via Gcc wrote:
> 
> > Currently it works well for absolute paths but if a file uses a
> > relative path or a path with a symlink in, or a non-absolute path, it
> > will miss those cases. For relative paths in particular it is
> > problematic as you can't easily construct a compiler commandline that
> > would cover all relative path options.
> 
> I'd expect a relative path to be naturally relocatable without needing to 
> be remapped.  (For example, DW_AT_comp_dir would be relocated in debug 
> info, but there would be no need to relocate the paths to individual files 
> that are relative to DW_AT_comp_dir.)
> 
> Is the issue that you're using relative paths between two directories that 
> don't have a fixed relative path between them, such as between the build 
> and source directories, as opposed to relative paths within the source 
> directory or within the build directory?

Yes, that is the issue. We build with separate build and source
directories with a relative path between them and that relation doesn't
work on target, particularly for installed binaries.

It is a general problem but particularly where we build multiple
components from the one shared source tree as those paths become
multiple levels deep which makes creating a source layout on target to
match even harder.

> > which address a realpath() call into the prefix mapping code. I did
> 
> That would run the risk of breaking relocation for anyone who has 
> deliberately used the paths they pass to the compiler (possibly involving 
> symlinks, for example) in their remapping options - not expecting a 
> further level of processing to be applied to those paths before remapping.

It would be a change in behaviour but I'm not sure how many people
would be relying on that or if they even should be. A relative path
would only be relocated if it did match something in the prefix-map
variables so this would be controllable.

Cheers,

Richard

Re: Reproducible builds - supporting relative paths in *-prefix-map

[Mark Wielaard]

Hi Richard,

I added Sergio to the CC since he was looking at debuginfo/DWARF
relative paths created by -fdebug-prefix-map in Debian and having trouble
making them work correctly. Maybe he has some feedback how to make
this work.

On Mon, Aug 15, 2022 at 12:13:28PM +0100, Richard Purdie via Gcc wrote:
> I'm wondering if we'd be able to improve path handling in the -f*-
> prefix-map compiler options to cover relative paths?
> [...]
> In case it helps, my background on what I'm working on and why we'd
> find this useful follows: 
> 
> I work on Yocto Project which cross compiles complete software stacks,
> we use gcc heavily and the resulting builds are all around us. We care
> deeply about build reproducibility.
> 
> With autotools, we long ago realised that we were best off separating
> the source and the build output where we could and to do it we used
> relative paths, which removed a lot of problematic hardcoded paths in
> our output.
> 
> More recently we've been using -fdebug-prefix-map and -fmacro-prefix-
> map (we need both since some of binutils doesn't seem to support the
> combined version yet). We use the debug information (split separately)
> to support on target debugging but where relative paths were used, we
> miss information as the remapping either doesn't happen correctly or is
> mangled. We'd like to fix our debug output but we're finding that hard
> without relative path support for *prefix-map.

I might be misinterpreting the issue you are seeing.

But one problem with debuginfo/DWARF is that relative source paths
aren't clearly defined. If you move or install the executable or
(split) debug file out of the build directory a DWARF reader has no
way to know what the paths are relative to.

So for DWARF the paths always have to be absolute (they can still be
relative to the compilation dir (DW_AT_comp_dir), but at least that
has to be absolute (and the compiler should turn any relative path
into an absolute one or make sure they are relative to an absolute
compilation directory path).

The problem with that is a) it makes the binary (debuginfo) output
dependent on the build srcdir and builddir and b) to support
debugging/tracing/profiling tools the user has to install the (debug)
sources under those exact same directories on their local machine.

One way around this is to use -fdebug-prefix-map with an absolute
paths under which you will also install any source files. Or to use
debugedit [*] after the build to rewrite the build dir path to
something like /usr/debug/src/<package-version-release-archa> which is
what some distros do.

e.g. Fedora has (one or more) binary package, a debuginfo package for
the .debug files (optionally installed under /usr/lib/debug) and
debugsource package for the debug source files (optionally installed
under /usr/debug/src). These packages can then also be used to
dynamically find any debug or source file when indexed by debuginfod
[**].

Using known absolute paths generated with debugedit or
-fdebug-prefix-map makes sure the paths used in the debuginfo/DWARF
are always the same independent from the current srcdir or builddir to
make them reproducible. And the user/tools don't have to guess what
the relative paths are relative to.

And it makes it so you can install the (debug) source files of all
packages/versions/arches in parallel without file conflicts.

Cheers,

Mark

[*] https://sourceware.org/debugedit/
[**] http://debuginfod.elfutils.org/

Re: Reproducible builds - supporting relative paths in *-prefix-map

[Richard Purdie]

Hi Mark,

Thanks for the reply.

On Mon, 2022-08-15 at 21:55 +0200, Mark Wielaard wrote:
> I added Sergio to the CC since he was looking at debuginfo/DWARF
> relative paths created by -fdebug-prefix-map in Debian and having trouble
> making them work correctly. Maybe he has some feedback how to make
> this work.
> 
> On Mon, Aug 15, 2022 at 12:13:28PM +0100, Richard Purdie via Gcc wrote:
> > I'm wondering if we'd be able to improve path handling in the -f*-
> > prefix-map compiler options to cover relative paths?
> > [...]
> > In case it helps, my background on what I'm working on and why we'd
> > find this useful follows: 
> > 
> > I work on Yocto Project which cross compiles complete software stacks,
> > we use gcc heavily and the resulting builds are all around us. We care
> > deeply about build reproducibility.
> > 
> > With autotools, we long ago realised that we were best off separating
> > the source and the build output where we could and to do it we used
> > relative paths, which removed a lot of problematic hardcoded paths in
> > our output.
> > 
> > More recently we've been using -fdebug-prefix-map and -fmacro-prefix-
> > map (we need both since some of binutils doesn't seem to support the
> > combined version yet). We use the debug information (split separately)
> > to support on target debugging but where relative paths were used, we
> > miss information as the remapping either doesn't happen correctly or is
> > mangled. We'd like to fix our debug output but we're finding that hard
> > without relative path support for *prefix-map.
> 
> I might be misinterpreting the issue you are seeing.
> 
> But one problem with debuginfo/DWARF is that relative source paths
> aren't clearly defined. If you move or install the executable or
> (split) debug file out of the build directory a DWARF reader has no
> way to know what the paths are relative to.
> 
> So for DWARF the paths always have to be absolute (they can still be
> relative to the compilation dir (DW_AT_comp_dir), but at least that
> has to be absolute (and the compiler should turn any relative path
> into an absolute one or make sure they are relative to an absolute
> compilation directory path).

It gets slightly more complicated as we build in a directory separate
to the source where we can. Some source files are generated source
files and placed in the build directory whilst many are in the source
directory. DW_AT_comp_dir can be set to one or the other but it is the
relative path between build and source which is problematic.

> The problem with that is a) it makes the binary (debuginfo) output
> dependent on the build srcdir and builddir and b) to support
> debugging/tracing/profiling tools the user has to install the (debug)
> sources under those exact same directories on their local machine.
> 
> One way around this is to use -fdebug-prefix-map with an absolute
> paths under which you will also install any source files. Or to use
> debugedit [*] after the build to rewrite the build dir path to
> something like /usr/debug/src/<package-version-release-archa> which is
> what some distros do.
> 
> e.g. Fedora has (one or more) binary package, a debuginfo package for
> the .debug files (optionally installed under /usr/lib/debug) and
> debugsource package for the debug source files (optionally installed
> under /usr/debug/src). These packages can then also be used to
> dynamically find any debug or source file when indexed by debuginfod
> [**].

We split the debuginfo into a separate package. We also look at the
sources it references and those go into a different separate package
too. We support populating a remote debuginfod server with these or
installing them onto the target.

> Using known absolute paths generated with debugedit or
> -fdebug-prefix-map makes sure the paths used in the debuginfo/DWARF
> are always the same independent from the current srcdir or builddir to
> make them reproducible. And the user/tools don't have to guess what
> the relative paths are relative to.

We have that working and set debug-prefix-map today. What is
problematic is trying to recreate the relative paths on target between
our source and build directories. Currently, most generated files in
the build directory just don't get handled correctly on target. We'd
like to fix that. There is currently no way to remap a relative path
though, at least as far as I could determine.

Cheers,

Richard

Re: Reproducible builds - supporting relative paths in *-prefix-map

[Mark Wielaard]

Hi Richard,

On Mon, Aug 15, 2022 at 09:29:03PM +0100, Richard Purdie wrote:
> On Mon, 2022-08-15 at 21:55 +0200, Mark Wielaard wrote:
> > I might be misinterpreting the issue you are seeing.
> > 
> > But one problem with debuginfo/DWARF is that relative source paths
> > aren't clearly defined. If you move or install the executable or
> > (split) debug file out of the build directory a DWARF reader has no
> > way to know what the paths are relative to.
> > 
> > So for DWARF the paths always have to be absolute (they can still be
> > relative to the compilation dir (DW_AT_comp_dir), but at least that
> > has to be absolute (and the compiler should turn any relative path
> > into an absolute one or make sure they are relative to an absolute
> > compilation directory path).
> 
> It gets slightly more complicated as we build in a directory separate
> to the source where we can. Some source files are generated source
> files and placed in the build directory whilst many are in the source
> directory. DW_AT_comp_dir can be set to one or the other but it is the
> relative path between build and source which is problematic.

Could you give an example directory structure when that creates a
problem? Any debug paths generated should be absolute or relative to
an absolute path.  In particular the DW_AT_comp_dir is normally an
absolute path (getpwd). So normally just remapping some prefix of the
pwd (either srcdir and/or builddir) should work to keep the relative
paths correct when moving the (generated) sources under that new
absolute path. I am probably missing something about the directory
setup that makes that reasoning invalid.

> We split the debuginfo into a separate package. We also look at the
> sources it references and those go into a different separate package
> too. We support populating a remote debuginfod server with these or
> installing them onto the target.

Nice. How do you lookup the referenced sources and where are they
then installed?

> > Using known absolute paths generated with debugedit or
> > -fdebug-prefix-map makes sure the paths used in the debuginfo/DWARF
> > are always the same independent from the current srcdir or builddir to
> > make them reproducible. And the user/tools don't have to guess what
> > the relative paths are relative to.
> 
> We have that working and set debug-prefix-map today. What is
> problematic is trying to recreate the relative paths on target between
> our source and build directories. Currently, most generated files in
> the build directory just don't get handled correctly on target. We'd
> like to fix that. There is currently no way to remap a relative path
> though, at least as far as I could determine.

So is the problem that when collecting the generated source files you
cannot map the debug-prefix-map back anymore? The relative paths are
relative to the original (absolute) prefix, but you only have the
remapped prefix?

Cheers,

Mark

Re: Reproducible builds - supporting relative paths in *-prefix-map

[Richard Purdie]

Hi Mark,

Thanks for the reply!

On Wed, 2022-08-17 at 13:23 +0200, Mark Wielaard wrote:
> Hi Richard,
> 
> On Mon, Aug 15, 2022 at 09:29:03PM +0100, Richard Purdie wrote:
> > On Mon, 2022-08-15 at 21:55 +0200, Mark Wielaard wrote:
> > > I might be misinterpreting the issue you are seeing.
> > > 
> > > But one problem with debuginfo/DWARF is that relative source paths
> > > aren't clearly defined. If you move or install the executable or
> > > (split) debug file out of the build directory a DWARF reader has no
> > > way to know what the paths are relative to.
> > > 
> > > So for DWARF the paths always have to be absolute (they can still be
> > > relative to the compilation dir (DW_AT_comp_dir), but at least that
> > > has to be absolute (and the compiler should turn any relative path
> > > into an absolute one or make sure they are relative to an absolute
> > > compilation directory path).
> > 
> > It gets slightly more complicated as we build in a directory separate
> > to the source where we can. Some source files are generated source
> > files and placed in the build directory whilst many are in the source
> > directory. DW_AT_comp_dir can be set to one or the other but it is the
> > relative path between build and source which is problematic.
> 
> Could you give an example directory structure when that creates a
> problem? Any debug paths generated should be absolute or relative to
> an absolute path.  In particular the DW_AT_comp_dir is normally an
> absolute path (getpwd). So normally just remapping some prefix of the
> pwd (either srcdir and/or builddir) should work to keep the relative
> paths correct when moving the (generated) sources under that new
> absolute path. I am probably missing something about the directory
> setup that makes that reasoning invalid.

An example from my build machine would be the gcc source here:

/media/build1/poky/build/tmp/work-shared/gcc-12.1.0-r0/gcc-12.1.0/

and libgcc being compiled here:

/media/build1/poky/build/tmp/work/core2-64-poky-linux/libgcc/12.1.0-r0/gcc-12.1.0/build.x86_64-poky-linux.x86_64-poky-linux

so relatively that is:

../../../work/core2-64-poky-linux/libgcc/12.1.0-r0/gcc-12.1.0/build.x86_64-poky-linux.x86_64-poky-linux

On target we put the gcc source at /usr/src/debug/gcc/12.1.0/.

To make this work we'd have to put the actual gcc source three levels
under that, and replicate the "work/core2-64-poky-linux/libgcc/12.1.0-
r0/gcc-12.1.0/build.x86_64-poky-linux.x86_64-poky-linux" structure,
neither of which are things we really want to do.

It gets more complicated as we support the idea of "external source"
which is from an arbitrary location on the system but built within
tmp/work/xxx. In the general case we can't remap all possibilities.

> > e split the debuginfo into a separate package. We also look at the
> > sources it references and those go into a different separate package
> > too. We support populating a remote debuginfod server with these or
> > installing them onto the target.
> 
> Nice. How do you lookup the referenced sources and where are they
> then installed?

We use dwarfsrcfiles which it looks like you may have written! :)

https://git.yoctoproject.org/poky/tree/meta/recipes-devtools/dwarfsrcfiles/files/dwarfsrcfiles.c

Installation is configurable but usually in
/usr/src/debug/<recipename>/<version>/

> > > Using known absolute paths generated with debugedit or
> > > -fdebug-prefix-map makes sure the paths used in the debuginfo/DWARF
> > > are always the same independent from the current srcdir or builddir to
> > > make them reproducible. And the user/tools don't have to guess what
> > > the relative paths are relative to.
> > 
> > We have that working and set debug-prefix-map today. What is
> > problematic is trying to recreate the relative paths on target between
> > our source and build directories. Currently, most generated files in
> > the build directory just don't get handled correctly on target. We'd
> > like to fix that. There is currently no way to remap a relative path
> > though, at least as far as I could determine.
> 
> So is the problem that when collecting the generated source files you
> cannot map the debug-prefix-map back anymore?

Correct, if we build with relative paths to autoconf (which we
effectively have to so we can avoid lots of full path hardcoding), we
can't collect up and remap the generated files.

>  The relative paths are relative to the original (absolute) prefix, but 
> you only have the remapped prefix?

We can't recreate the relative paths on target (in the general external
source case) and we can't remap them with the code as it stands today.

Cheers,

Richard