From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 71939 invoked by alias); 21 Jul 2017 16:16:24 -0000 Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Received: (qmail 71750 invoked by uid 89); 21 Jul 2017 16:16:12 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-14.7 required=5.0 tests=AWL,BAYES_00,GIT_PATCH_1,GIT_PATCH_2,GIT_PATCH_3,KAM_ASCII_DIVIDERS,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.2 spammy=luo, H*r:sk:mail.he, Hx-spam-relays-external:sk:mail.he, H*RU:sk:mail.he X-HELO: mail.headstrong.de Received: from mail.headstrong.de (HELO mail.headstrong.de) (81.7.4.112) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Fri, 21 Jul 2017 16:16:08 +0000 Received: from localhost (localhost.localdomain [127.0.0.1]) by mail.headstrong.de (Postfix) with ESMTP id 1B4341C007AF; Fri, 21 Jul 2017 18:16:04 +0200 (CEST) Authentication-Results: mail.headstrong.de (amavisd-new); dkim=pass (1024-bit key) reason="pass (just generated, assumed good)" header.d=headstrong.de Received: from mail.headstrong.de ([127.0.0.1]) by localhost (mail.headstrong.de [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id obGqEpSzmWCi; Fri, 21 Jul 2017 18:15:51 +0200 (CEST) Received: from infinity0 by localhost with local (Exim 4.89) (envelope-from ) id 1dYaak-0001z1-C7; Fri, 21 Jul 2017 18:15:50 +0200 From: Ximin Luo To: GCC Patches Cc: Ximin Luo Subject: [PATCH 1/3] Use BUILD_PATH_PREFIX_MAP envvar for debug-prefix-map Date: Fri, 21 Jul 2017 16:16:00 -0000 Message-Id: <20170721161538.7508-2-infinity0@pwned.gg> In-Reply-To: <20170721161538.7508-1-infinity0@pwned.gg> References: <20170721161538.7508-1-infinity0@pwned.gg> X-SW-Source: 2017-07/txt/msg01316.txt.bz2 Define the BUILD_PATH_PREFIX_MAP environment variable, and treat it as implicit -fdebug-prefix-map CLI options specified before any explicit such options. Much of the generic code for applying and parsing prefix-maps is implemented in libiberty instead of the dwarf2 parts of the code, in order to make subsequent patches unrelated to debuginfo easier. Acknowledgements ---------------- Daniel Kahn Gillmor who wrote the patch for r231835, which saved me a lot of time figuring out what to edit. HW42 for discussion on the details of the proposal, and for suggesting that we retain the ability to map the prefix to something other than ".". Other contributors to the BUILD_PATH_PREFIX_MAP specification, see https://reproducible-builds.org/specs/build-path-prefix-map/ ChangeLogs ---------- include/ChangeLog: 2017-07-21 Ximin Luo * prefix-map.h: New file implementing the BUILD_PATH_PREFIX_MAP specification; includes code from /gcc/final.c and code adapted from examples attached to the specification. libiberty/ChangeLog: 2017-07-21 Ximin Luo * prefix-map.c: New file implementing the BUILD_PATH_PREFIX_MAP specification; includes code from /gcc/final.c and code adapted from examples attached to the specification. * Makefile.in: Update for new files. gcc/ChangeLog: 2017-07-21 Ximin Luo * debug.h: Declare add_debug_prefix_map_from_envvar. * final.c: Define add_debug_prefix_map_from_envvar, and refactor prefix-map utilities to use equivalent code from libiberty instead. * opts-global.c: (handle_common_deferred_options): Call add_debug_prefix_map_from_envvar before processing options. gcc/testsuite/ChangeLog: 2017-07-21 Ximin Luo * lib/gcc-dg.exp: Allow dg-set-compiler-env-var to take only one argument in which case it unsets the given env var. * gcc.dg/debug/dwarf2/build_path_prefix_map-1.c: New test. * gcc.dg/debug/dwarf2/build_path_prefix_map-2.c: New test. Index: gcc-8-20170716/include/prefix-map.h =================================================================== --- /dev/null +++ gcc-8-20170716/include/prefix-map.h @@ -0,0 +1,94 @@ +/* Declarations for manipulating filename prefixes. + Written 2017 by Ximin Luo + This code is in the public domain. */ + +#ifndef _PREFIX_MAP_H +#define _PREFIX_MAP_H + +#ifdef __cplusplus +extern "C" { +#endif + +#ifdef HAVE_STDLIB_H +#include +#endif + +/* Linked-list of mappings from old prefixes to new prefixes. */ + +struct prefix_map +{ + const char *old_prefix; + const char *new_prefix; + size_t old_len; + size_t new_len; + struct prefix_map *next; +}; + + +/* Find a mapping suitable for the given OLD_NAME in the linked list MAP.\ + + If a mapping is found, writes a pointer to the non-matching suffix part of + OLD_NAME in SUFFIX, and its length in SUF_LEN. + + Returns NULL if there was no suitable mapping. */ +struct prefix_map * +prefix_map_find (struct prefix_map *map, const char *old_name, + const char **suffix, size_t *suf_len); + +/* Prepend a prefix map before a given SUFFIX. + + The remapped name is written to NEW_NAME and returned as a const pointer. No + allocations are performed; the caller must ensure it can hold at least + MAP->NEW_LEN + SUF_LEN + 1 characters. */ +const char * +prefix_map_prepend (struct prefix_map *map, char *new_name, + const char *suffix, size_t suf_len); + +/* Remap a filename. + + Returns OLD_NAME unchanged if there was no remapping, otherwise returns a + pointer to newly-allocated memory for the remapped filename. The memory is + allocated by the given ALLOC function, which also determines who is + responsible for freeing it. */ +#define prefix_map_remap_alloc_(map_head, old_name, alloc) \ + __extension__ \ + ({ \ + const char *__suffix; \ + size_t __suf_len; \ + struct prefix_map *__map; \ + (__map = prefix_map_find ((map_head), (old_name), &__suffix, &__suf_len)) \ + ? prefix_map_prepend (__map, \ + (char *) alloc (__map->new_len + __suf_len + 1), \ + __suffix, __suf_len) \ + : (old_name); \ + }) + +/* Remap a filename. + + Returns OLD_NAME unchanged if there was no remapping, otherwise returns a + stack-allocated pointer to the newly-remapped filename. */ +#define prefix_map_remap_alloca(map_head, old_name) \ + prefix_map_remap_alloc_ (map_head, old_name, alloca) + + +/* Parse prefix-maps according to the BUILD_PATH_PREFIX_MAP standard. + + The input string value is of the form + + dst[0]=src[0]:dst[1]=src[1]... + + Every dst[i] and src[i] has had "%", "=" and ":" characters replaced with + "%#", "%+", and "%." respectively; this function reverses this replacement. + + Rightmost entries are stored at the head of the parsed structure. + + Returns 0 on failure and 1 on success. */ +int +prefix_map_parse (struct prefix_map **map_head, const char *arg); + + +#ifdef __cplusplus +} +#endif + +#endif /* _PREFIX_MAP_H */ Index: gcc-8-20170716/libiberty/Makefile.in =================================================================== --- gcc-8-20170716.orig/libiberty/Makefile.in +++ gcc-8-20170716/libiberty/Makefile.in @@ -143,6 +143,7 @@ CFILES = alloca.c argv.c asprintf.c atex pex-common.c pex-djgpp.c pex-msdos.c pex-one.c \ pex-unix.c pex-win32.c \ physmem.c putenv.c \ + prefix-map.c \ random.c regex.c rename.c rindex.c \ rust-demangle.c \ safe-ctype.c setenv.c setproctitle.c sha1.c sigsetmask.c \ @@ -182,6 +183,7 @@ REQUIRED_OFILES = \ ./partition.$(objext) ./pexecute.$(objext) ./physmem.$(objext) \ ./pex-common.$(objext) ./pex-one.$(objext) \ ./@pexecute@.$(objext) ./vprintf-support.$(objext) \ + ./prefix-map.$(objext) \ ./rust-demangle.$(objext) \ ./safe-ctype.$(objext) \ ./simple-object.$(objext) ./simple-object-coff.$(objext) \ @@ -757,7 +759,7 @@ $(CONFIGURED_OFILES): stamp-picdir stamp $(COMPILE.c) $(srcdir)/fibheap.c $(OUTPUT_OPTION) ./filename_cmp.$(objext): $(srcdir)/filename_cmp.c config.h $(INCDIR)/ansidecl.h \ - $(INCDIR)/filenames.h $(INCDIR)/hashtab.h \ + $(INCDIR)/filenames.h $(INCDIR)/hashtab.h $(INCDIR)/libiberty.h \ $(INCDIR)/safe-ctype.h if [ x"$(PICFLAG)" != x ]; then \ $(COMPILE.c) $(PICFLAG) $(srcdir)/filename_cmp.c -o pic/$@; \ @@ -1104,7 +1106,8 @@ $(CONFIGURED_OFILES): stamp-picdir stamp $(COMPILE.c) $(srcdir)/pex-one.c $(OUTPUT_OPTION) ./pex-unix.$(objext): $(srcdir)/pex-unix.c config.h $(INCDIR)/ansidecl.h \ - $(INCDIR)/libiberty.h $(srcdir)/pex-common.h + $(INCDIR)/environ.h $(INCDIR)/libiberty.h \ + $(srcdir)/pex-common.h if [ x"$(PICFLAG)" != x ]; then \ $(COMPILE.c) $(PICFLAG) $(srcdir)/pex-unix.c -o pic/$@; \ else true; fi @@ -1143,6 +1146,15 @@ $(CONFIGURED_OFILES): stamp-picdir stamp else true; fi $(COMPILE.c) $(srcdir)/physmem.c $(OUTPUT_OPTION) +./prefix-map.$(objext): $(srcdir)/prefix-map.c config.h $(INCDIR)/prefix-map.h + if [ x"$(PICFLAG)" != x ]; then \ + $(COMPILE.c) $(PICFLAG) $(srcdir)/prefix-map.c -o pic/$@; \ + else true; fi + if [ x"$(NOASANFLAG)" != x ]; then \ + $(COMPILE.c) $(PICFLAG) $(NOASANFLAG) $(srcdir)/prefix-map.c -o noasan/$@; \ + else true; fi + $(COMPILE.c) $(srcdir)/prefix-map.c $(OUTPUT_OPTION) + ./putenv.$(objext): $(srcdir)/putenv.c config.h $(INCDIR)/ansidecl.h if [ x"$(PICFLAG)" != x ]; then \ $(COMPILE.c) $(PICFLAG) $(srcdir)/putenv.c -o pic/$@; \ @@ -1210,7 +1222,8 @@ $(CONFIGURED_OFILES): stamp-picdir stamp else true; fi $(COMPILE.c) $(srcdir)/safe-ctype.c $(OUTPUT_OPTION) -./setenv.$(objext): $(srcdir)/setenv.c config.h $(INCDIR)/ansidecl.h +./setenv.$(objext): $(srcdir)/setenv.c config.h $(INCDIR)/ansidecl.h \ + $(INCDIR)/environ.h if [ x"$(PICFLAG)" != x ]; then \ $(COMPILE.c) $(PICFLAG) $(srcdir)/setenv.c -o pic/$@; \ else true; fi @@ -1661,7 +1674,7 @@ $(CONFIGURED_OFILES): stamp-picdir stamp $(COMPILE.c) $(srcdir)/xexit.c $(OUTPUT_OPTION) ./xmalloc.$(objext): $(srcdir)/xmalloc.c config.h $(INCDIR)/ansidecl.h \ - $(INCDIR)/libiberty.h + $(INCDIR)/environ.h $(INCDIR)/libiberty.h if [ x"$(PICFLAG)" != x ]; then \ $(COMPILE.c) $(PICFLAG) $(srcdir)/xmalloc.c -o pic/$@; \ else true; fi @@ -1719,3 +1732,4 @@ $(CONFIGURED_OFILES): stamp-picdir stamp $(COMPILE.c) $(PICFLAG) $(NOASANFLAG) $(srcdir)/xvasprintf.c -o noasan/$@; \ else true; fi $(COMPILE.c) $(srcdir)/xvasprintf.c $(OUTPUT_OPTION) + Index: gcc-8-20170716/libiberty/prefix-map.c =================================================================== --- /dev/null +++ gcc-8-20170716/libiberty/prefix-map.c @@ -0,0 +1,201 @@ +/* Definitions for manipulating filename prefixes. + Written 2017 by Ximin Luo + This code is in the public domain. */ + +#ifdef HAVE_CONFIG_H +#include "config.h" +#endif + +#ifdef HAVE_STRING_H +#include +#endif + +#ifdef HAVE_STDLIB_H +#include +#endif + +#include "filenames.h" +#include "libiberty.h" +#include "prefix-map.h" + + +/* Add a new mapping. + + The input strings are duplicated and a new prefix_map struct is allocated. + Ownership of the duplicates, as well as the new prefix_map, is the same as + the ownership of the old struct. + + Returns 0 on failure and 1 on success. */ +int +prefix_map_push (struct prefix_map **map_head, + const char *new_prefix, const char *old_prefix) +{ + struct prefix_map *map = XNEW (struct prefix_map); + if (!map) + goto rewind_0; + + map->old_prefix = xstrdup (old_prefix); + if (!map->old_prefix) + goto rewind_1; + map->old_len = strlen (old_prefix); + + map->new_prefix = xstrdup (new_prefix); + if (!map->new_prefix) + goto rewind_2; + map->new_len = strlen (new_prefix); + + map->next = *map_head; + *map_head = map; + return 1; + +rewind_2: + free ((void *) map->old_prefix); +rewind_1: + free (map); +rewind_0: + return 0; +} + +/* Rewind a prefix map. + + Everything up to the given OLD_HEAD is freed. */ +void +prefix_map_pop_until (struct prefix_map **map_head, struct prefix_map *old_head) +{ + struct prefix_map *map; + struct prefix_map *next; + + for (map = *map_head; map != old_head; map = next) + { + free ((void *) map->old_prefix); + free ((void *) map->new_prefix); + next = map->next; + free (map); + } + + *map_head = map; +} + + +/* Find a mapping suitable for the given OLD_NAME in the linked list MAP.\ + + If a mapping is found, writes a pointer to the non-matching suffix part of + OLD_NAME in SUFFIX, and its length in SUF_LEN. + + Returns NULL if there was no suitable mapping. */ +struct prefix_map * +prefix_map_find (struct prefix_map *map, const char *old_name, + const char **suffix, size_t *suf_len) +{ + for (; map; map = map->next) + if (filename_ncmp (old_name, map->old_prefix, map->old_len) == 0) + { + *suf_len = strlen (*suffix = old_name + map->old_len); + break; + } + + return map; +} + +/* Prepend a prefix map before a given SUFFIX. + + The remapped name is written to NEW_NAME and returned as a const pointer. No + allocations are performed; the caller must ensure it can hold at least + MAP->NEW_LEN + SUF_LEN + 1 characters. */ +const char * +prefix_map_prepend (struct prefix_map *map, char *new_name, + const char *suffix, size_t suf_len) +{ + memcpy (new_name, map->new_prefix, map->new_len); + memcpy (new_name + map->new_len, suffix, suf_len + 1); + return new_name; +} + + +/* Parse a single part of a single prefix-map pair. + + Returns 0 on failure and 1 on success. */ +int +prefix_map_parse_unquote (char *src) +{ + for (char *dest = src; 0 != (*dest = *src); ++dest, ++src) + switch (*src) + { + case ':': + case '=': + return 0; // should have been escaped + case '%': + switch (*(src + 1)) + { + case '.': + *dest = ':'; + goto unquoted; + case '+': + *dest = '='; + unquoted: + case '#': + ++src; + break; + default: + return 0; // invalid + } + } + return 1; +} + +/* Parse a single prefix-map. + + Returns 0 on failure and 1 on success. */ +int +prefix_map_parse1 (struct prefix_map **map_head, char *arg) +{ + char *p; + p = strchr (arg, '='); + if (!p) + return 0; + *p = '\0'; + if (!prefix_map_parse_unquote (arg)) + return 0; + p++; + if (!prefix_map_parse_unquote (p)) + return 0; + + return prefix_map_push (map_head, arg, p); +} + +/* Parse a prefix-map according to the BUILD_PATH_PREFIX_MAP standard. + + The input string value is of the form + + dst[0]=src[0]:dst[1]=src[1]... + + Every dst[i] and src[i] has had "%", "=" and ":" characters replaced with + "%#", "%+", and "%." respectively; this function reverses this replacement. + + Rightmost entries are stored at the head of the parsed structure. + + Returns 0 on failure and 1 on success. */ +int +prefix_map_parse (struct prefix_map **map_head, const char *arg) +{ + struct prefix_map *old_head = *map_head; + + size_t len = strlen (arg); + char *copy = (char *) alloca (len + 1); + memcpy (copy, arg, len + 1); + + const char *sep = ":"; + char *end, *tok = strtok_r (copy, sep, &end); + while (tok != NULL) + { + if (!prefix_map_parse1 (map_head, tok)) + { + prefix_map_pop_until (map_head, old_head); + return 0; + } + + tok = strtok_r (NULL, sep, &end); + } + + return 1; +} Index: gcc-8-20170716/gcc/debug.h =================================================================== --- gcc-8-20170716.orig/gcc/debug.h +++ gcc-8-20170716/gcc/debug.h @@ -236,6 +236,7 @@ extern void dwarf2out_switch_text_sectio const char *remap_debug_filename (const char *); void add_debug_prefix_map (const char *); +void add_debug_prefix_map_from_envvar (); /* For -fdump-go-spec. */ Index: gcc-8-20170716/gcc/final.c =================================================================== --- gcc-8-20170716.orig/gcc/final.c +++ gcc-8-20170716/gcc/final.c @@ -46,6 +46,7 @@ along with GCC; see the file COPYING3. #define INCLUDE_ALGORITHM /* reverse */ #include "system.h" #include "coretypes.h" +#include "prefix-map.h" #include "backend.h" #include "target.h" #include "rtl.h" @@ -1506,22 +1507,9 @@ asm_str_count (const char *templ) return count; } -/* ??? This is probably the wrong place for these. */ -/* Structure recording the mapping from source file and directory - names at compile time to those to be embedded in debug - information. */ -struct debug_prefix_map -{ - const char *old_prefix; - const char *new_prefix; - size_t old_len; - size_t new_len; - struct debug_prefix_map *next; -}; - -/* Linked list of such structures. */ -static debug_prefix_map *debug_prefix_maps; +/* Linked list of `struct prefix_map'. */ +static prefix_map *debug_prefix_maps = NULL; /* Record a debug file prefix mapping. ARG is the argument to -fdebug-prefix-map and must be of the form OLD=NEW. */ @@ -1529,7 +1517,7 @@ static debug_prefix_map *debug_prefix_ma void add_debug_prefix_map (const char *arg) { - debug_prefix_map *map; + prefix_map *map; const char *p; p = strchr (arg, '='); @@ -1538,7 +1526,7 @@ add_debug_prefix_map (const char *arg) error ("invalid argument %qs to -fdebug-prefix-map", arg); return; } - map = XNEW (debug_prefix_map); + map = XNEW (prefix_map); map->old_prefix = xstrndup (arg, p - arg); map->old_len = p - arg; p++; @@ -1548,28 +1536,32 @@ add_debug_prefix_map (const char *arg) debug_prefix_maps = map; } +/* Add debug-prefix-maps from BUILD_PATH_PREFIX_MAP environment variable. */ + +void +add_debug_prefix_map_from_envvar () +{ + const char *arg = getenv ("BUILD_PATH_PREFIX_MAP"); + + if (!arg || prefix_map_parse (&debug_prefix_maps, arg)) + return; + + error ("environment variable BUILD_PATH_PREFIX_MAP is " + "not well formed; see the GCC documentation for more details."); +} + /* Perform user-specified mapping of debug filename prefixes. Return the new name corresponding to FILENAME. */ const char * remap_debug_filename (const char *filename) { - debug_prefix_map *map; - char *s; - const char *name; - size_t name_len; - - for (map = debug_prefix_maps; map; map = map->next) - if (filename_ncmp (filename, map->old_prefix, map->old_len) == 0) - break; - if (!map) + const char *name = prefix_map_remap_alloca (debug_prefix_maps, filename); + + if (name == filename) return filename; - name = filename + map->old_len; - name_len = strlen (name) + 1; - s = (char *) alloca (name_len + map->new_len); - memcpy (s, map->new_prefix, map->new_len); - memcpy (s + map->new_len, name, name_len); - return ggc_strdup (s); + + return ggc_strdup (name); } /* Return true if DWARF2 debug info can be emitted for DECL. */ Index: gcc-8-20170716/gcc/opts-global.c =================================================================== --- gcc-8-20170716.orig/gcc/opts-global.c +++ gcc-8-20170716/gcc/opts-global.c @@ -335,6 +335,8 @@ handle_common_deferred_options (void) if (flag_opt_info) opt_info_switch_p (NULL); + add_debug_prefix_map_from_envvar (); + FOR_EACH_VEC_ELT (v, i, opt) { switch (opt->opt_index) Index: gcc-8-20170716/gcc/testsuite/lib/gcc-dg.exp =================================================================== --- gcc-8-20170716.orig/gcc/testsuite/lib/gcc-dg.exp +++ gcc-8-20170716/gcc/testsuite/lib/gcc-dg.exp @@ -461,19 +461,24 @@ proc restore-target-env-var { } { proc dg-set-compiler-env-var { args } { global set_compiler_env_var global saved_compiler_env_var - if { [llength $args] != 3 } { - error "dg-set-compiler-env-var: need two arguments" + if { [llength $args] != 3 && [llength $args] != 2 } { + error "dg-set-compiler-env-var: need one or two arguments" return } set var [lindex $args 1] - set value [lindex $args 2] if [info exists ::env($var)] { lappend saved_compiler_env_var [list $var 1 $::env($var)] } else { lappend saved_compiler_env_var [list $var 0] } - setenv $var $value - lappend set_compiler_env_var [list $var $value] + if { [llength $args] == 3 } { + set value [lindex $args 2] + setenv $var $value + lappend set_compiler_env_var [list $var 1 $value] + } else { + catch { unsetenv $var } + lappend set_compiler_env_var [list $var 0] + } } proc restore-compiler-env-var { } { @@ -485,7 +490,7 @@ proc restore-compiler-env-var { } { if [lindex $env_var 1] { setenv $var [lindex $env_var 2] } else { - unsetenv $var + catch { unsetenv $var } } } } Index: gcc-8-20170716/gcc/testsuite/gcc.dg/debug/dwarf2/build_path_prefix_map-1.c =================================================================== --- /dev/null +++ gcc-8-20170716/gcc/testsuite/gcc.dg/debug/dwarf2/build_path_prefix_map-1.c @@ -0,0 +1,9 @@ +/* DW_AT_comp_dir should be relative if BUILD_PATH_PREFIX_MAP is a prefix of it. */ +/* { dg-do compile } */ +/* { dg-options "-gdwarf -dA" } */ +/* { dg-set-compiler-env-var BUILD_PATH_PREFIX_MAP "DWARF2TEST=[file dirname [pwd]]" } */ +/* { dg-final { scan-assembler "DW_AT_comp_dir: \"DWARF2TEST/gcc" } } */ + +void func (void) +{ +} Index: gcc-8-20170716/gcc/testsuite/gcc.dg/debug/dwarf2/build_path_prefix_map-2.c =================================================================== --- /dev/null +++ gcc-8-20170716/gcc/testsuite/gcc.dg/debug/dwarf2/build_path_prefix_map-2.c @@ -0,0 +1,9 @@ +/* DW_AT_comp_dir should be absolute if BUILD_PATH_PREFIX_MAP is not set. */ +/* { dg-do compile } */ +/* { dg-options "-gdwarf -dA" } */ +/* { dg-set-compiler-env-var BUILD_PATH_PREFIX_MAP } */ +/* { dg-final { scan-assembler "DW_AT_comp_dir: \"/" } } */ + +void func (void) +{ +}