[PATCH] Support gzip compressed exec and core files in gdb

[PATCH] Support gzip compressed exec and core files in gdb

[Michael Eager]

[-- Attachment #1: Type: text/plain, Size: 1279 bytes --]

Add support to automatically unzip compressed executable and core files.
Files will be uncompressed into temporary directory (/tmp or $TMPDIR)
and are deleted when GDB exits.  This should be transparent to users,
except for disk space requirements.  The name of the uncompressed file is
mentioned, but all references to the file in GDB messages is to the file
which the user specified.

This operation cannot be done completely by BFD because BFD allows an opened
file to be passed to it for processing.  GDB uses this functionality.

BFD:
   * bfd-in2.h: Regenerate.
   * bfd.c (struct bfd): Add uncompressed_filename.
   * bfdio.c (bfd_get_mtime): Set bfd->mtime_set to true.
   * cache.c (bfd_open): Open previously created uncompressed file.

GDB:
   * common/filestuff.c (struct compressed_file_cache_search, eq_compressed_file,
   is_gzip, decompress_gzip, do_compressed_cleanup, gdb_uncompress): New.
   * common/filestuff.h (gdb_uncompress): Declare.
   * corelow.c (core_open): Uncompress core file.
   * exec.c (exec_file_attach): Uncompress exe file.
   * symfile.c (symfile_bfd_open): Uncompress sym (exe) file.

GDB/DOC:
   * gdb.texinfo: Mention gzipped exec and core files.

-- 
Michael Eager	 eager@eagercon.com
1960 Park Blvd., Palo Alto, CA 94306  650-325-8077

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-GDB-support-compressed-exec-and-core-files.patch --]
[-- Type: text/x-patch; name="0001-GDB-support-compressed-exec-and-core-files.patch", Size: 13816 bytes --]

From b675cf554012ad1067e5bb1693fe695844cc09d6 Mon Sep 17 00:00:00 2001
From: Michael Eager <meager@cisco.com>
Date: Tue, 10 Mar 2015 15:31:11 -0700
Subject: [PATCH] GDB support compressed exec and core files.

Add support to automatically unzip compressed executable and core files.
Files will be uncompressed into temporary directory (/tmp or $TMPDIR)
and are deleted when GDB exits.  This should be transparent to users,
except for disk space requirements.  The name of the uncompressed file is
mentioned, but all references to the file in GDB messages is to the file
which the user specified.

This operation cannot be done completely by BFD because BFD allows an opened
file to be passed to it for processing.  GDB uses this functionality.

BFD:
  * bfd-in2.h: Regenerate.
  * bfd.c (struct bfd): Add uncompressed_filename.
  * bfdio.c (bfd_get_mtime): Set bfd->mtime_set to true.
  * cache.c (bfd_open): Open previously created uncompressed file.

GDB:
  * common/filestuff.c (struct compressed_file_cache_search, eq_compressed_file,
  is_gzip, decompress_gzip, do_compressed_cleanup, gdb_uncompress): New.
  * common/filestuff.h (gdb_uncompress): Declare.
  * corelow.c (core_open): Uncompress core file.
  * exec.c (exec_file_attach): Uncompress exe file.
  * symfile.c (symfile_bfd_open): Uncompress sym (exe) file.

GDB/DOC:
  * gdb.texinfo: Mention gzipped exec and core files.
---
 bfd/bfd-in2.h          |   3 +
 bfd/bfd.c              |   3 +
 bfd/bfdio.c            |   1 +
 bfd/cache.c            |   6 +-
 gdb/common/filestuff.c | 204 +++++++++++++++++++++++++++++++++++++++++++++++++
 gdb/common/filestuff.h |   6 ++
 gdb/corelow.c          |  10 +++
 gdb/doc/gdb.texinfo    |   5 ++
 gdb/exec.c             |  15 +++-
 gdb/symfile.c          |  13 +++-
 10 files changed, 263 insertions(+), 3 deletions(-)

diff --git a/bfd/bfd-in2.h b/bfd/bfd-in2.h
index 2d2e7ac..dd5d6ea 100644
--- a/bfd/bfd-in2.h
+++ b/bfd/bfd-in2.h
@@ -6294,6 +6294,9 @@ struct bfd
   /* The filename the application opened the BFD with.  */
   const char *filename;
 
+  /* The file name after being uncompressed.  */
+  const char *uncompressed_filename;
+
   /* A pointer to the target jump table.  */
   const struct bfd_target *xvec;
 
diff --git a/bfd/bfd.c b/bfd/bfd.c
index 5ae5eca..4c78e7b 100644
--- a/bfd/bfd.c
+++ b/bfd/bfd.c
@@ -56,6 +56,9 @@ CODE_FRAGMENT
 .  {* The filename the application opened the BFD with.  *}
 .  const char *filename;
 .
+.  {* The file name after being uncompressed.  *}
+.  const char *uncompressed_filename;
+.
 .  {* A pointer to the target jump table.  *}
 .  const struct bfd_target *xvec;
 .
diff --git a/bfd/bfdio.c b/bfd/bfdio.c
index 89406e3..e82c888 100644
--- a/bfd/bfdio.c
+++ b/bfd/bfdio.c
@@ -382,6 +382,7 @@ bfd_get_mtime (bfd *abfd)
     return 0;
 
   abfd->mtime = buf.st_mtime;		/* Save value in case anyone wants it */
+  abfd->mtime_set = TRUE;		/* Say that it is valid.  */
   return buf.st_mtime;
 }
 
diff --git a/bfd/cache.c b/bfd/cache.c
index 94a82da..f46e9dd 100644
--- a/bfd/cache.c
+++ b/bfd/cache.c
@@ -588,7 +588,11 @@ bfd_open_file (bfd *abfd)
     {
     case read_direction:
     case no_direction:
-      abfd->iostream = real_fopen (abfd->filename, FOPEN_RB);
+      if (abfd->uncompressed_filename)
+        /* BFD was previously uncompressed.  Reopen file.  */
+        abfd->iostream = real_fopen (abfd->uncompressed_filename, FOPEN_RB);
+      else
+        abfd->iostream = real_fopen (abfd->filename, FOPEN_RB);
       break;
     case both_direction:
     case write_direction:
diff --git a/gdb/common/filestuff.c b/gdb/common/filestuff.c
index 14d6324..b2c31fd 100644
--- a/gdb/common/filestuff.c
+++ b/gdb/common/filestuff.c
@@ -19,10 +19,16 @@
 #include "common-defs.h"
 #include "filestuff.h"
 #include "gdb_vecs.h"
+#include "hashtab.h"
 #include <fcntl.h>
 #include <unistd.h>
 #include <sys/types.h>
 #include <sys/stat.h>
+#include <stdio.h>
+
+#ifdef HAVE_ZLIB_H
+#include <zlib.h>
+#endif
 
 #ifdef USE_WIN32API
 #include <winsock2.h>
@@ -47,6 +53,27 @@
 #define SOCK_CLOEXEC 0
 #endif
 
+#ifdef HAVE_ZLIB_H
+/* Hash table of compressed files.  */
+
+static htab_t compressed_file_cache;
+
+struct compressed_file_cache_search
+{
+  const char *filename;
+  char *uncompressed_filename;
+  time_t mtime;
+};
+
+static int
+eq_compressed_file (const void *a, const void *b)
+{
+  const struct compressed_file_cache_search *entry = a;
+  const struct compressed_file_cache_search *search = b;
+
+  return (strcmp (entry->filename, search->filename) == 0);
+}
+#endif
 \f
 
 #ifndef HAVE_FDWALK
@@ -404,3 +431,180 @@ gdb_pipe_cloexec (int filedes[2])
 
   return result;
 }
+\f
+#ifndef GDBSERVER
+/* Test if file is compressed with gzip.  */
+
+static inline int
+is_gzip (unsigned char *buf)
+{
+  return (buf[0] == 037 && buf[1] == 0213);	/* From /usr/share/magic.  */
+}
+
+#define COMPRESS_BUF_SIZE (1024*1024)
+static int
+decompress_gzip (const char *filename, FILE *tmp)
+{
+#ifdef HAVE_ZLIB_H
+  char *buf = malloc (COMPRESS_BUF_SIZE);
+  gzFile compressed = gzopen (filename, "r");
+  int count, res;
+
+  if (buf == NULL || compressed == NULL)
+    {
+      printf (_("error copying gzip file\n"));
+      free (buf);
+      return 0;
+    }
+
+  while ((count = gzread (compressed, buf, COMPRESS_BUF_SIZE)))
+    {
+      res = fwrite (buf, 1, count, tmp);
+      if (res != count)
+	{
+	  printf (_("error decompressing gzip file\n"));
+          free (buf);
+	  return 0;
+	}
+    }
+
+  fflush (tmp);
+  gzclose (compressed);
+  free (buf);
+  return 1;
+#else
+  return 0;
+#endif
+}
+
+/* Delete uncompressed temp file when terminating.  */
+static void
+do_compressed_cleanup (void *filename)
+{
+  unlink (filename);
+  xfree (filename);
+}
+
+/* If file is compressed, uncompress it into a temporary.  */
+
+int
+gdb_uncompress (const char *filename, char **uncompressed_filename)
+{
+  FILE *handle;
+  struct compressed_file_cache_search search, *found;
+  struct stat st;
+  hashval_t hash;
+  void **slot;
+  static unsigned char buffer[1024];
+  size_t count;
+  enum {NONE, GZIP, BZIP2} file_compression = NONE;
+  int decomp_fd;
+  FILE *decomp_file;
+  int ret = 0;
+  char *tmpdir, *p;
+  char *template = xmalloc(128 + 12 + 7 + 1);
+  if (compressed_file_cache == NULL)
+    compressed_file_cache = htab_create_alloc (1, htab_hash_string,
+						eq_compressed_file,
+						NULL, xcalloc, xfree);
+
+  if ((stat (filename, &st) < 0))
+    return 0;
+
+  search.filename = filename;
+  search.uncompressed_filename = NULL;
+
+  hash = htab_hash_string (filename);
+  found = htab_find_with_hash (compressed_file_cache, &search, hash);
+
+  if (found)
+    {
+      /* We previously uncompressed the file.  */
+      if (found->mtime == st.st_mtime)
+        {
+	  /* Return file if compressed file not changed.  */
+	  *uncompressed_filename = found->uncompressed_filename;
+	  return 1;
+	}
+      else
+        {
+	  /* Delete old uncompressed file.  */
+	  unlink (found->uncompressed_filename);
+	  xfree ((void *)found->filename);
+	  xfree (found->uncompressed_filename);
+        }
+    }
+
+  if ((handle = fopen (filename, "rb")) == NULL)
+    return 0;
+
+  if ((count = fread (buffer, 1, sizeof buffer, handle)) > 0)
+    {
+      if (is_gzip (buffer))
+	file_compression = GZIP;
+    }
+
+  fclose (handle);
+
+  if (file_compression == NONE)
+    return 0;
+
+  /* Create temporary file name for uncompressed file.  */
+  if (!(tmpdir = getenv ("TMPDIR")))
+    tmpdir = "/tmp";
+  strncpy (template, tmpdir, 128);
+  strcat (template, "/");
+  for (p = (char *)filename + strlen (filename) - 1;
+       p >= filename && *p != '/'; p--)  /* find final slash.  */  ;
+  strncat (template, ++p, 128);
+  p = template + strlen (template);
+  if (strcmp (p - 3, ".gz") == 0)
+    *(p - 3) = '\0';
+  strcat (template, "-XXXXXX");
+
+  if ((decomp_fd = mkstemp (template)) != -1)
+    {
+      decomp_file = fdopen (decomp_fd, "w+b");
+
+      if (file_compression == GZIP)
+        {
+	  printf (_("Decompressing %s to %s\n"), filename, template);
+	  ret = decompress_gzip (filename, decomp_file);
+	}
+      else
+        {
+          xfree (template);
+          return 0;
+        }
+      fclose (decomp_file);
+
+      if (ret)
+	{
+	  if (!found)
+	    {
+	      slot = htab_find_slot_with_hash (compressed_file_cache,
+					       &search, hash, INSERT);
+	      gdb_assert (slot && !*slot);
+	      found = xmalloc (sizeof (struct compressed_file_cache_search));
+	      *slot = found;
+	    }
+	  found->filename = strdup (filename);
+	  found->mtime = st.st_mtime;
+	  found->uncompressed_filename = template;
+	}
+    }
+  else
+    {
+      warning (_("Decompression failed\n"));
+      xfree (template);
+      return 0;
+    }
+
+  *uncompressed_filename = template;
+
+  /* Schedule delete of temp file when gdb ends.  */
+  make_final_cleanup (do_compressed_cleanup, xstrdup (template));
+
+  return 1;
+}
+#endif
diff --git a/gdb/common/filestuff.h b/gdb/common/filestuff.h
index 98522a6..7a1d12d 100644
--- a/gdb/common/filestuff.h
+++ b/gdb/common/filestuff.h
@@ -67,4 +67,10 @@ extern int gdb_socket_cloexec (int domain, int style, int protocol);
 
 extern int gdb_pipe_cloexec (int filedes[2]);
 
+/* If 'filename' is compressed, uncompress it and return name of
+   uncompressed file.  */
+
+extern int gdb_uncompress (const char *filename,
+			   char **uncompressed_filename);
+
 #endif /* FILESTUFF_H */
diff --git a/gdb/corelow.c b/gdb/corelow.c
index 9218003..13662a7 100644
--- a/gdb/corelow.c
+++ b/gdb/corelow.c
@@ -279,6 +279,7 @@ core_open (const char *arg, int from_tty)
   int scratch_chan;
   int flags;
   char *filename;
+  char *uncompressed_filename;
 
   target_preopen (from_tty);
   if (!arg)
@@ -316,6 +317,15 @@ core_open (const char *arg, int from_tty)
   if (temp_bfd == NULL)
     perror_with_name (filename);
 
+  if (!write_files && gdb_uncompress (filename, &uncompressed_filename))
+    {
+      close (scratch_chan);
+      scratch_chan = gdb_open_cloexec (uncompressed_filename, flags, 0);
+      temp_bfd = gdb_bfd_fopen (uncompressed_filename, gnutarget,
+				FOPEN_RB, scratch_chan);
+      temp_bfd->uncompressed_filename = uncompressed_filename;
+    }
+
   if (!bfd_check_format (temp_bfd, bfd_core)
       && !gdb_check_format (temp_bfd))
     {
diff --git a/gdb/doc/gdb.texinfo b/gdb/doc/gdb.texinfo
index 35dbe86..3b4d389 100644
--- a/gdb/doc/gdb.texinfo
+++ b/gdb/doc/gdb.texinfo
@@ -17391,6 +17391,11 @@ via @code{gdbserver} (@pxref{Server, file, Using the @code{gdbserver}
 Program}).  In these situations the @value{GDBN} commands to specify
 new files are useful.
 
+Executable and core files may be compressed using @code{gzip}.  These
+files will be uncompressed into temporary files either in /tmp or in 
+@code{$TMPDIR} if this is set in the environment.  The files will have
+a unique name and will be deleted when @value{GDBN} terminates.
+
 @table @code
 @cindex executable file
 @kindex file
diff --git a/gdb/exec.c b/gdb/exec.c
index 696458d..6d3c1fc 100644
--- a/gdb/exec.c
+++ b/gdb/exec.c
@@ -35,6 +35,7 @@
 #include "progspace.h"
 #include "gdb_bfd.h"
 #include "gcore.h"
+#include "filestuff.h"
 
 #include <fcntl.h>
 #include "readline/readline.h"
@@ -155,6 +156,7 @@ void
 exec_file_attach (const char *filename, int from_tty)
 {
   struct cleanup *cleanups;
+  char *uncompressed_filename = NULL;
 
   /* First, acquire a reference to the current exec_bfd.  We release
      this at the end of the function; but acquiring it now lets the
@@ -209,7 +211,18 @@ exec_file_attach (const char *filename, int from_tty)
 	exec_bfd = gdb_bfd_fopen (canonical_pathname, gnutarget,
 				  FOPEN_RUB, scratch_chan);
       else
-	exec_bfd = gdb_bfd_open (canonical_pathname, gnutarget, scratch_chan);
+	{
+          if (!gdb_uncompress (canonical_pathname, &uncompressed_filename))
+	    exec_bfd = gdb_bfd_open (canonical_pathname, gnutarget, scratch_chan);
+          else
+	    {
+	      close (scratch_chan);
+	      scratch_chan = openp ("", 0, uncompressed_filename,
+				    O_RDONLY | O_BINARY, &scratch_pathname);
+
+	      exec_bfd = gdb_bfd_open (uncompressed_filename, gnutarget, scratch_chan);
+	    }
+	}
 
       if (!exec_bfd)
 	{
diff --git a/gdb/symfile.c b/gdb/symfile.c
index c2a71ec..0d173f1 100644
--- a/gdb/symfile.c
+++ b/gdb/symfile.c
@@ -56,6 +56,7 @@
 #include "stack.h"
 #include "gdb_bfd.h"
 #include "cli/cli-utils.h"
+#include "filestuff.h"
 
 #include <sys/types.h>
 #include <fcntl.h>
@@ -1739,6 +1740,7 @@ symfile_bfd_open (const char *cname)
   bfd *sym_bfd;
   int desc;
   char *name, *absolute_name;
+  char *uncompressed_filename;
   struct cleanup *back_to;
 
   if (remote_filename_p (cname))
@@ -1783,7 +1785,16 @@ symfile_bfd_open (const char *cname)
   name = absolute_name;
   back_to = make_cleanup (xfree, name);
 
-  sym_bfd = gdb_bfd_open (name, gnutarget, desc);
+  if (!gdb_uncompress (name, &uncompressed_filename))
+    sym_bfd = gdb_bfd_open (name, gnutarget, desc);
+  else
+    {
+      close (desc);
+      desc = openp ("", 0, uncompressed_filename, O_RDONLY | O_BINARY, &absolute_name);
+
+      sym_bfd = gdb_bfd_open (uncompressed_filename, gnutarget, desc);
+    }
+
   if (!sym_bfd)
     error (_("`%s': can't open to read symbols: %s."), name,
 	   bfd_errmsg (bfd_get_error ()));
-- 
2.2.1

Re: [PATCH] Support gzip compressed exec and core files in gdb

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 2773 bytes --]

On 10 Mar 2015 16:01, Michael Eager wrote:
> --- a/gdb/common/filestuff.c
> +++ b/gdb/common/filestuff.c
>
> +#define COMPRESS_BUF_SIZE (1024*1024)
> +static int
> +decompress_gzip (const char *filename, FILE *tmp)
> +{
> +#ifdef HAVE_ZLIB_H
> +  char *buf = malloc (COMPRESS_BUF_SIZE);

xmalloc ?

> +  if (buf == NULL || compressed == NULL)
> +    {
> +      printf (_("error copying gzip file\n"));

fprintf_filtered (gdb_stderr, ...) ?

> +	  printf (_("error decompressing gzip file\n"));

here too ?

> +          free (buf);

indentation is broken.  this comes up a lot, so you should scan the whole thing.

> +  fflush (tmp);

that needed ?

> +int
> +gdb_uncompress (const char *filename, char **uncompressed_filename)
> +{
> +  FILE *handle;
> +  struct compressed_file_cache_search search, *found;
> +  struct stat st;
> +  hashval_t hash;
> +  void **slot;
> +  static unsigned char buffer[1024];
> +  size_t count;
> +  enum {NONE, GZIP, BZIP2} file_compression = NONE;
> +  int decomp_fd;
> +  FILE *decomp_file;
> +  int ret = 0;
> +  char *tmpdir, *p;
> +  char *template = xmalloc(128 + 12 + 7 + 1);

probably should be a comment as to the constants you've written here

> +  if (compressed_file_cache == NULL)

style says there should be a blank line above this if statement

> +    compressed_file_cache = htab_create_alloc (1, htab_hash_string,
> +						eq_compressed_file,
> +						NULL, xcalloc, xfree);
> +
> +  if ((stat (filename, &st) < 0))

excess set of paren

> +	  /* Return file if compressed file not changed.  */
> +	  *uncompressed_filename = found->uncompressed_filename;
> +	  return 1;
> +	}
> +      else
> +        {
> +	  /* Delete old uncompressed file.  */
> +	  unlink (found->uncompressed_filename);
> +	  xfree ((void *)found->filename);

is that cast really needed ?

> +  if ((handle = fopen (filename, "rb")) == NULL)

gdb generally doesn't like to pack assignments into if statements

use FOPEN_RB instead of "rb" ?

> +  /* Create temporary file name for uncompressed file.  */
> +  if (!(tmpdir = getenv ("TMPDIR")))
> +    tmpdir = "/tmp";
> +  strncpy (template, tmpdir, 128);
> +  strcat (template, "/");
> +  for (p = (char *)filename + strlen (filename) - 1;
> +       p >= filename && *p != '/'; p--)  /* find final slash.  */  ;
> +  strncat (template, ++p, 128);
> +  p = template + strlen (template);
> +  if (strcmp (p - 3, ".gz") == 0)
> +    *(p - 3) = '\0';
> +  strcat (template, "-XXXXXX");

that's pretty messy man.  why not use mkstemp() and fdopen() instead ?

in general, there's no need for all this strcat business.  asprintf allows yout 
easily concat paths & allocate the right amount of space.
-mike

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [PATCH] Support gzip compressed exec and core files in gdb

[Alan Modra]

On Tue, Mar 10, 2015 at 04:01:42PM -0700, Michael Eager wrote:
> This operation cannot be done completely by BFD because BFD allows an opened
> file to be passed to it for processing.  GDB uses this functionality.

I'd prefer you do this entirely outside of BFD, without adding another
field to struct bfd.  I think that can be done by simply clearing
abfd->cacheable on files you uncompress.  This prevents BFD from
closing the file, so you won't need to open it again.

-- 
Alan Modra
Australia Development Lab, IBM

Re: [PATCH] Support gzip compressed exec and core files in gdb

[Gary Benson]

Michael Eager wrote:
> Add support to automatically unzip compressed executable and core
> files.  Files will be uncompressed into temporary directory (/tmp or
> $TMPDIR) and are deleted when GDB exits.  This should be transparent
> to users, except for disk space requirements.  The name of the
> uncompressed file is mentioned, but all references to the file in
> GDB messages is to the file which the user specified.
...
> diff --git a/gdb/common/filestuff.c b/gdb/common/filestuff.c
> index 14d6324..b2c31fd 100644
> --- a/gdb/common/filestuff.c
> +++ b/gdb/common/filestuff.c
...
> +#ifndef GDBSERVER

Please do not add GDBSERVER conditionals to gdb/common, I spent half
a year removing them all.

It looks like this code is only used by GDB, not gdbserver, so the
fix is simple, just put the code somewhere GDB-specific.

Thanks,
Gary

-- 
http://gbenson.net/

Re: [PATCH] Support gzip compressed exec and core files in gdb

[Michael Eager]

On 03/11/15 01:14, Alan Modra wrote:
> On Tue, Mar 10, 2015 at 04:01:42PM -0700, Michael Eager wrote:
>> This operation cannot be done completely by BFD because BFD allows an opened
>> file to be passed to it for processing.  GDB uses this functionality.
>
> I'd prefer you do this entirely outside of BFD, without adding another
> field to struct bfd.  I think that can be done by simply clearing
> abfd->cacheable on files you uncompress.  This prevents BFD from
> closing the file, so you won't need to open it again.

GDB closes the exec file, then uses BFD to seek (I think when reading
syms).  BFD then re-opens the file, so it needs the name of the
uncompressed file.

-- 
Michael Eager	 eager@eagercon.com
1960 Park Blvd., Palo Alto, CA 94306  650-325-8077

Re: [PATCH] Support gzip compressed exec and core files in gdb

[Michael Eager]

On 03/11/15 03:07, Gary Benson wrote:
> Michael Eager wrote:
>> Add support to automatically unzip compressed executable and core
>> files.  Files will be uncompressed into temporary directory (/tmp or
>> $TMPDIR) and are deleted when GDB exits.  This should be transparent
>> to users, except for disk space requirements.  The name of the
>> uncompressed file is mentioned, but all references to the file in
>> GDB messages is to the file which the user specified.
> ...
>> diff --git a/gdb/common/filestuff.c b/gdb/common/filestuff.c
>> index 14d6324..b2c31fd 100644
>> --- a/gdb/common/filestuff.c
>> +++ b/gdb/common/filestuff.c
> ...
>> +#ifndef GDBSERVER
>
> Please do not add GDBSERVER conditionals to gdb/common, I spent half
> a year removing them all.
>
> It looks like this code is only used by GDB, not gdbserver, so the
> fix is simple, just put the code somewhere GDB-specific.

Sure.  Any suggestions?  gdb_bfd.c seems like the wrong place.


-- 
Michael Eager	 eager@eagercon.com
1960 Park Blvd., Palo Alto, CA 94306  650-325-8077

Re: [PATCH] Support gzip compressed exec and core files in gdb

[Michael Eager]

On 03/10/15 19:37, Mike Frysinger wrote:
> On 10 Mar 2015 16:01, Michael Eager wrote:
>> --- a/gdb/common/filestuff.c
>> +++ b/gdb/common/filestuff.c
>>
>> +#define COMPRESS_BUF_SIZE (1024*1024)
>> +static int
>> +decompress_gzip (const char *filename, FILE *tmp)
>> +{
>> +#ifdef HAVE_ZLIB_H
>> +  char *buf = malloc (COMPRESS_BUF_SIZE);
>
> xmalloc ?
>
>> +  if (buf == NULL || compressed == NULL)
>> +    {
>> +      printf (_("error copying gzip file\n"));
>
> fprintf_filtered (gdb_stderr, ...) ?
>
>> +	  printf (_("error decompressing gzip file\n"));
>
> here too ?
>
>> +          free (buf);
>
> indentation is broken.  this comes up a lot, so you should scan the whole thing.
>
>> +  fflush (tmp);
>
> that needed ?
>
>> +int
>> +gdb_uncompress (const char *filename, char **uncompressed_filename)
>> +{
>> +  FILE *handle;
>> +  struct compressed_file_cache_search search, *found;
>> +  struct stat st;
>> +  hashval_t hash;
>> +  void **slot;
>> +  static unsigned char buffer[1024];
>> +  size_t count;
>> +  enum {NONE, GZIP, BZIP2} file_compression = NONE;
>> +  int decomp_fd;
>> +  FILE *decomp_file;
>> +  int ret = 0;
>> +  char *tmpdir, *p;
>> +  char *template = xmalloc(128 + 12 + 7 + 1);
>
> probably should be a comment as to the constants you've written here
>
>> +  if (compressed_file_cache == NULL)
>
> style says there should be a blank line above this if statement
>
>> +    compressed_file_cache = htab_create_alloc (1, htab_hash_string,
>> +						eq_compressed_file,
>> +						NULL, xcalloc, xfree);
>> +
>> +  if ((stat (filename, &st) < 0))
>
> excess set of paren
>
>> +	  /* Return file if compressed file not changed.  */
>> +	  *uncompressed_filename = found->uncompressed_filename;
>> +	  return 1;
>> +	}
>> +      else
>> +        {
>> +	  /* Delete old uncompressed file.  */
>> +	  unlink (found->uncompressed_filename);
>> +	  xfree ((void *)found->filename);
>
> is that cast really needed ?
>
>> +  if ((handle = fopen (filename, "rb")) == NULL)
>
> gdb generally doesn't like to pack assignments into if statements
>
> use FOPEN_RB instead of "rb" ?
>
>> +  /* Create temporary file name for uncompressed file.  */
>> +  if (!(tmpdir = getenv ("TMPDIR")))
>> +    tmpdir = "/tmp";
>> +  strncpy (template, tmpdir, 128);
>> +  strcat (template, "/");
>> +  for (p = (char *)filename + strlen (filename) - 1;
>> +       p >= filename && *p != '/'; p--)  /* find final slash.  */  ;
>> +  strncat (template, ++p, 128);
>> +  p = template + strlen (template);
>> +  if (strcmp (p - 3, ".gz") == 0)
>> +    *(p - 3) = '\0';
>> +  strcat (template, "-XXXXXX");
>
> that's pretty messy man.  why not use mkstemp() and fdopen() instead ?
>
> in general, there's no need for all this strcat business.  asprintf allows yout
> easily concat paths & allocate the right amount of space.
> -mike

Thanks for the review.  I'll clean up and resubmit.


-- 
Michael Eager	 eager@eagercon.com
1960 Park Blvd., Palo Alto, CA 94306  650-325-8077

Re: [PATCH] Support gzip compressed exec and core files in gdb

[Eli Zaretskii]

> Date: Tue, 10 Mar 2015 16:01:42 -0700
> From: Michael Eager <eager@eagerm.com>
> 
> GDB/DOC:
>    * gdb.texinfo: Mention gzipped exec and core files.

This should mention the name of the node where you make changes; see
the other ChangeLog entries in gdb/doc/.

Do we need a NEWS entry for this?

> --- a/gdb/doc/gdb.texinfo
> +++ b/gdb/doc/gdb.texinfo
> @@ -17391,6 +17391,11 @@ via @code{gdbserver} (@pxref{Server, file, Using the @code{gdbserver}
>  Program}).  In these situations the @value{GDBN} commands to specify
>  new files are useful.
>  
> +Executable and core files may be compressed using @code{gzip}.  These

'gzip' is a command, so please use @command, not @code.

> +files will be uncompressed into temporary files either in /tmp

'/tmp' is a file name, so please use @file for its markup.

>                                                                 or in 
> +@code{$TMPDIR} if this is set in the environment.

I'd prefer to say

  in the system-wide temporary directory

and mention neither /tmp nor $TMPDIR, as both are platform-dependent.

Otherwise, the documentation parts are OK.

I agree with Mike that it's better to use library functions for
creating temporary files, since that hides platform dependencies.

Thanks.

Re: [PATCH] Support gzip compressed exec and core files in gdb

[Gary Benson]

Michael Eager wrote:
> On 03/11/15 03:07, Gary Benson wrote:
> >Michael Eager wrote:
> > > Add support to automatically unzip compressed executable and core
> > > files.  Files will be uncompressed into temporary directory (/tmp or
> > > $TMPDIR) and are deleted when GDB exits.  This should be transparent
> > > to users, except for disk space requirements.  The name of the
> > > uncompressed file is mentioned, but all references to the file in
> > > GDB messages is to the file which the user specified.
> >...
> > > diff --git a/gdb/common/filestuff.c b/gdb/common/filestuff.c
> > > index 14d6324..b2c31fd 100644
> > > --- a/gdb/common/filestuff.c
> > > +++ b/gdb/common/filestuff.c
> >...
> > > +#ifndef GDBSERVER
> >
> > Please do not add GDBSERVER conditionals to gdb/common, I spent
> > half a year removing them all.
> > 
> > It looks like this code is only used by GDB, not gdbserver, so
> > the fix is simple, just put the code somewhere GDB-specific.
> 
> Sure.  Any suggestions?  gdb_bfd.c seems like the wrong place.

gdb/utils.c maybe?

Cheers,
Gary

-- 
http://gbenson.net/

Re: [PATCH] Support gzip compressed exec and core files in gdb

[Doug Evans]

On Wed, Mar 11, 2015 at 10:42 AM, Gary Benson <gbenson@redhat.com> wrote:
> Michael Eager wrote:
>> On 03/11/15 03:07, Gary Benson wrote:
>> >Michael Eager wrote:
>> > > Add support to automatically unzip compressed executable and core
>> > > files.  Files will be uncompressed into temporary directory (/tmp or
>> > > $TMPDIR) and are deleted when GDB exits.  This should be transparent
>> > > to users, except for disk space requirements.  The name of the
>> > > uncompressed file is mentioned, but all references to the file in
>> > > GDB messages is to the file which the user specified.
>> >...
>> > > diff --git a/gdb/common/filestuff.c b/gdb/common/filestuff.c
>> > > index 14d6324..b2c31fd 100644
>> > > --- a/gdb/common/filestuff.c
>> > > +++ b/gdb/common/filestuff.c
>> >...
>> > > +#ifndef GDBSERVER
>> >
>> > Please do not add GDBSERVER conditionals to gdb/common, I spent
>> > half a year removing them all.
>> >
>> > It looks like this code is only used by GDB, not gdbserver, so
>> > the fix is simple, just put the code somewhere GDB-specific.
>>
>> Sure.  Any suggestions?  gdb_bfd.c seems like the wrong place.
>
> gdb/utils.c maybe?

Fine by me.
Though, OTOH, there's nothing intrinsic in what's being done there
that gdbserver (or anything else that might want to use common)
couldn't use it tomorrow.
I'd be ok with common/compression.c or some such.

Re: [PATCH] Support gzip compressed exec and core files in gdb

[Cary Coutant]

How will this affect split DWARF?

If you uncompress a foo.gz binary, and the binary has relative paths
to .dwo files, will GDB look for the .dwo files relative to the
original binary, or to the uncompressed binary?

When it looks for a .dwp file, will it look for foo.dwp or foo.gz.dwp,
and will it look in the same directory as the original, or in /tmp?

If foo.dwp is also compressed, will it uncompress that?

Would it make sense to support .tar.gz/.tgz files containing both a
binary and its .dwp?

-cary



On Tue, Mar 10, 2015 at 4:01 PM, Michael Eager <eager@eagerm.com> wrote:
> Add support to automatically unzip compressed executable and core files.
> Files will be uncompressed into temporary directory (/tmp or $TMPDIR)
> and are deleted when GDB exits.  This should be transparent to users,
> except for disk space requirements.  The name of the uncompressed file is
> mentioned, but all references to the file in GDB messages is to the file
> which the user specified.
>
> This operation cannot be done completely by BFD because BFD allows an opened
> file to be passed to it for processing.  GDB uses this functionality.
>
> BFD:
>   * bfd-in2.h: Regenerate.
>   * bfd.c (struct bfd): Add uncompressed_filename.
>   * bfdio.c (bfd_get_mtime): Set bfd->mtime_set to true.
>   * cache.c (bfd_open): Open previously created uncompressed file.
>
> GDB:
>   * common/filestuff.c (struct compressed_file_cache_search,
> eq_compressed_file,
>   is_gzip, decompress_gzip, do_compressed_cleanup, gdb_uncompress): New.
>   * common/filestuff.h (gdb_uncompress): Declare.
>   * corelow.c (core_open): Uncompress core file.
>   * exec.c (exec_file_attach): Uncompress exe file.
>   * symfile.c (symfile_bfd_open): Uncompress sym (exe) file.
>
> GDB/DOC:
>   * gdb.texinfo: Mention gzipped exec and core files.
>
> --
> Michael Eager    eager@eagercon.com
> 1960 Park Blvd., Palo Alto, CA 94306  650-325-8077

Re: [PATCH] Support gzip compressed exec and core files in gdb

[Gary Benson]

Doug Evans wrote:
> On Wed, Mar 11, 2015 at 10:42 AM, Gary Benson <gbenson@redhat.com> wrote:
> > Michael Eager wrote:
> > > On 03/11/15 03:07, Gary Benson wrote:
> > > >Michael Eager wrote:
> > > > > Add support to automatically unzip compressed executable and
> > > > > core files.  Files will be uncompressed into temporary
> > > > > directory (/tmp or $TMPDIR) and are deleted when GDB exits.
> > > > > This should be transparent to users, except for disk space
> > > > > requirements.  The name of the uncompressed file is
> > > > > mentioned, but all references to the file in GDB messages is
> > > > > to the file which the user specified.
> > > >...
> > > > > diff --git a/gdb/common/filestuff.c b/gdb/common/filestuff.c
> > > > > index 14d6324..b2c31fd 100644
> > > > > --- a/gdb/common/filestuff.c
> > > > > +++ b/gdb/common/filestuff.c
> > > >...
> > > > > +#ifndef GDBSERVER
> > > >
> > > > Please do not add GDBSERVER conditionals to gdb/common, I spent
> > > > half a year removing them all.
> > > >
> > > > It looks like this code is only used by GDB, not gdbserver, so
> > > > the fix is simple, just put the code somewhere GDB-specific.
> > >
> > > Sure.  Any suggestions?  gdb_bfd.c seems like the wrong place.
> >
> > gdb/utils.c maybe?
> 
> Fine by me.
> Though, OTOH, there's nothing intrinsic in what's being done there
> that gdbserver (or anything else that might want to use common)
> couldn't use it tomorrow.
> I'd be ok with common/compression.c or some such.

Do we really want to be putting things in common speculatively?

Cheers,
Gary

-- 
http://gbenson.net/

Re: [PATCH] Support gzip compressed exec and core files in gdb

[Doug Evans]

On Wed, Mar 11, 2015 at 11:21 AM, Gary Benson <gbenson@redhat.com> wrote:
> Do we really want to be putting things in common speculatively?

I don't have a strong enough opinion to spend too many emails on it.
But I would ask that such a rule be written down in an easy to find place.

Re: [PATCH] Support gzip compressed exec and core files in gdb

[Michael Eager]

On 03/11/15 09:15, Eli Zaretskii wrote:
>> Date: Tue, 10 Mar 2015 16:01:42 -0700
>> From: Michael Eager <eager@eagerm.com>
>>
>> GDB/DOC:
>>     * gdb.texinfo: Mention gzipped exec and core files.
>
> This should mention the name of the node where you make changes; see
> the other ChangeLog entries in gdb/doc/.
>
> Do we need a NEWS entry for this?
>
>> --- a/gdb/doc/gdb.texinfo
>> +++ b/gdb/doc/gdb.texinfo
>> @@ -17391,6 +17391,11 @@ via @code{gdbserver} (@pxref{Server, file, Using the @code{gdbserver}
>>   Program}).  In these situations the @value{GDBN} commands to specify
>>   new files are useful.
>>
>> +Executable and core files may be compressed using @code{gzip}.  These
>
> 'gzip' is a command, so please use @command, not @code.
>
>> +files will be uncompressed into temporary files either in /tmp
>
> '/tmp' is a file name, so please use @file for its markup.
>
>>                                                                  or in
>> +@code{$TMPDIR} if this is set in the environment.
>
> I'd prefer to say
>
>    in the system-wide temporary directory
>
> and mention neither /tmp nor $TMPDIR, as both are platform-dependent.

I think it might be good to tell people how to specify where the
uncompressed file is located.

>
> Otherwise, the documentation parts are OK.

Thanks.  I'll make the changes.

> I agree with Mike that it's better to use library functions for
> creating temporary files, since that hides platform dependencies.

The code uses mkstemp() to create a temporary file.  Is there a
GDB or BFD wrapper for this function?


-- 
Michael Eager	 eager@eagercon.com
1960 Park Blvd., Palo Alto, CA 94306  650-325-8077

Re: [PATCH] Support gzip compressed exec and core files in gdb

[Michael Eager]

On 03/11/15 11:24, Cary Coutant wrote:
> How will this affect split DWARF?
>
> If you uncompress a foo.gz binary, and the binary has relative paths
> to .dwo files, will GDB look for the .dwo files relative to the
> original binary, or to the uncompressed binary?

GDB should look for .dwo files relative to the original file.  The
uncompressed file is essentially hidden.

If you can send me a test case creating .dwo files, I'll make sure
that it works.

> When it looks for a .dwp file, will it look for foo.dwp or foo.gz.dwp,
> and will it look in the same directory as the original, or in /tmp?
>
> If foo.dwp is also compressed, will it uncompress that?

The file search for exec and core are unchanged, so gdb will find
whatever file it is asked to look for, using the current search path,
exactly as it does now.  No file names are modified.

There's specific code to uncompress exec and core files.  I might
need to add code for dwp.

I originally looked at doing decompression in BFD, so that it would
simply be transparent.  If a component opened a gzipped file, BFD
would transparently decompress it.  This didn't work.  GDB does
some file operations, like closing and reopening files, which I think
should be done in BFD.

> Would it make sense to support .tar.gz/.tgz files containing both a
> binary and its .dwp?

Perhaps, but I don't want to extend this (conceptually) simple extension
to also handling tar files.  Maybe as a follow-on.

>
> -cary
>
>
>
> On Tue, Mar 10, 2015 at 4:01 PM, Michael Eager <eager@eagerm.com> wrote:
>> Add support to automatically unzip compressed executable and core files.
>> Files will be uncompressed into temporary directory (/tmp or $TMPDIR)
>> and are deleted when GDB exits.  This should be transparent to users,
>> except for disk space requirements.  The name of the uncompressed file is
>> mentioned, but all references to the file in GDB messages is to the file
>> which the user specified.
>>
>> This operation cannot be done completely by BFD because BFD allows an opened
>> file to be passed to it for processing.  GDB uses this functionality.
>>
>> BFD:
>>    * bfd-in2.h: Regenerate.
>>    * bfd.c (struct bfd): Add uncompressed_filename.
>>    * bfdio.c (bfd_get_mtime): Set bfd->mtime_set to true.
>>    * cache.c (bfd_open): Open previously created uncompressed file.
>>
>> GDB:
>>    * common/filestuff.c (struct compressed_file_cache_search,
>> eq_compressed_file,
>>    is_gzip, decompress_gzip, do_compressed_cleanup, gdb_uncompress): New.
>>    * common/filestuff.h (gdb_uncompress): Declare.
>>    * corelow.c (core_open): Uncompress core file.
>>    * exec.c (exec_file_attach): Uncompress exe file.
>>    * symfile.c (symfile_bfd_open): Uncompress sym (exe) file.
>>
>> GDB/DOC:
>>    * gdb.texinfo: Mention gzipped exec and core files.
>>
>> --
>> Michael Eager    eager@eagercon.com
>> 1960 Park Blvd., Palo Alto, CA 94306  650-325-8077
>


-- 
Michael Eager	 eager@eagercon.com
1960 Park Blvd., Palo Alto, CA 94306  650-325-8077

Re: [PATCH] Support gzip compressed exec and core files in gdb

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 494 bytes --]

On 11 Mar 2015 12:55, Michael Eager wrote:
> On 03/11/15 09:15, Eli Zaretskii wrote:
> > I agree with Mike that it's better to use library functions for
> > creating temporary files, since that hides platform dependencies.
> 
> The code uses mkstemp() to create a temporary file.  Is there a
> GDB or BFD wrapper for this function?

if you're worried about portability, gdb uses gnulib now, so it should be easy 
to import a gnulib module that provides that func as a fallback
-mike

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [PATCH] Support gzip compressed exec and core files in gdb

[Doug Evans]

On Wed, Mar 11, 2015 at 1:12 PM, Michael Eager <eager@eagerm.com> wrote:
>> Would it make sense to support .tar.gz/.tgz files containing both a
>> binary and its .dwp?
>
>
> Perhaps, but I don't want to extend this (conceptually) simple extension
> to also handling tar files.  Maybe as a follow-on.

Is there a fuse-based wrapper for tarballs?
A little googling suggests there is but I didn't dig for details.

How useful would it be to make that work from gdb more easily?

E.g., "file tarfuse:myexec"

and then provide a way to register a handler for "tarfuse".

Or some such.
Just thinking out loud ...

Re: [PATCH] Support gzip compressed exec and core files in gdb

[Eli Zaretskii]

> Date: Wed, 11 Mar 2015 12:55:19 -0700
> From: Michael Eager <eager@eagerm.com>
> CC: gdb-patches@sourceware.org, binutils@sourceware.org
> 
> > and mention neither /tmp nor $TMPDIR, as both are platform-dependent.
> 
> I think it might be good to tell people how to specify where the
> uncompressed file is located.

Saying "system-wide temporary directory" does tell that, doesn't it?
Users of each platform know where that is, I think.

We could, of course, describe all the details for each supported
platform, but then the description needs to be much longer than what
you wrote.  Is it really that important?

> The code uses mkstemp() to create a temporary file.  Is there a
> GDB or BFD wrapper for this function?

Not that I know of.  Why do you need a wrapper?

Re: [PATCH] Support gzip compressed exec and core files in gdb

[Jan Kratochvil]

On Wed, 11 Mar 2015 00:01:42 +0100, Michael Eager wrote:
> Add support to automatically unzip compressed executable and core files.
> Files will be uncompressed into temporary directory (/tmp or $TMPDIR)
> and are deleted when GDB exits.

Such feature has been requested to support xz-compressed core files as
currently being stored by systemd.  But to make it more convenient one should
decompress on-demand only the blocks of file that are really accessed by GDB
- expecting by bfd_iovec.  Obviously GDB usually needs to access only small
part of the whole core file.

I did not check how it is supported by gzip but for xz one needs to use
--block-size, otherwise the file blocks cannot be decompressed independently
in random access way.

ISTM libz-gzip and liblzma-xz compatibility is mutually exclusive.


Jan

Re: [PATCH] Support gzip compressed exec and core files in gdb

[Doug Evans]

On Wed, Mar 11, 2015 at 3:13 PM, Jan Kratochvil
<jan.kratochvil@redhat.com> wrote:
> ISTM libz-gzip and liblzma-xz compatibility is mutually exclusive.

Can you elaborate?

Re: [PATCH] Support gzip compressed exec and core files in gdb

[Alan Modra]

On Wed, Mar 11, 2015 at 07:56:30AM -0700, Michael Eager wrote:
> On 03/11/15 01:14, Alan Modra wrote:
> >On Tue, Mar 10, 2015 at 04:01:42PM -0700, Michael Eager wrote:
> >>This operation cannot be done completely by BFD because BFD allows an opened
> >>file to be passed to it for processing.  GDB uses this functionality.
> >
> >I'd prefer you do this entirely outside of BFD, without adding another
> >field to struct bfd.  I think that can be done by simply clearing
> >abfd->cacheable on files you uncompress.  This prevents BFD from
> >closing the file, so you won't need to open it again.
> 
> GDB closes the exec file, then uses BFD to seek (I think when reading
> syms).  BFD then re-opens the file, so it needs the name of the
> uncompressed file.

Really?  I think it quite unclean if gdb expects BFD to reopen a file
that gdb has closed!

-- 
Alan Modra
Australia Development Lab, IBM

Re: [PATCH] Support gzip compressed exec and core files in gdb

[Michael Eager]

On 03/11/15 15:13, Jan Kratochvil wrote:
> On Wed, 11 Mar 2015 00:01:42 +0100, Michael Eager wrote:
>> Add support to automatically unzip compressed executable and core files.
>> Files will be uncompressed into temporary directory (/tmp or $TMPDIR)
>> and are deleted when GDB exits.
>
> Such feature has been requested to support xz-compressed core files as
> currently being stored by systemd.  But to make it more convenient one should
> decompress on-demand only the blocks of file that are really accessed by GDB
> - expecting by bfd_iovec.  Obviously GDB usually needs to access only small
> part of the whole core file.
>
> I did not check how it is supported by gzip but for xz one needs to use
> --block-size, otherwise the file blocks cannot be decompressed independently
> in random access way.

gzip is not compressed block-by-block.  As far as I can tell, you need to
decompress starting from the beginning of the file.

> ISTM libz-gzip and liblzma-xz compatibility is mutually exclusive.

I don't know why they would be incompatible, but support for an on-demand
block-compression scheme would be significantly different.  Decompressing
an xz file by making a copy (as is done for gzip) would be a simple extension
to the current patch.

-- 
Michael Eager	 eager@eagercon.com
1960 Park Blvd., Palo Alto, CA 94306  650-325-8077

Re: [PATCH] Support gzip compressed exec and core files in gdb

[Michael Eager]

On 03/11/15 17:08, Alan Modra wrote:
> On Wed, Mar 11, 2015 at 07:56:30AM -0700, Michael Eager wrote:
>> On 03/11/15 01:14, Alan Modra wrote:
>>> On Tue, Mar 10, 2015 at 04:01:42PM -0700, Michael Eager wrote:
>>>> This operation cannot be done completely by BFD because BFD allows an opened
>>>> file to be passed to it for processing.  GDB uses this functionality.
>>>
>>> I'd prefer you do this entirely outside of BFD, without adding another
>>> field to struct bfd.  I think that can be done by simply clearing
>>> abfd->cacheable on files you uncompress.  This prevents BFD from
>>> closing the file, so you won't need to open it again.
>>
>> GDB closes the exec file, then uses BFD to seek (I think when reading
>> syms).  BFD then re-opens the file, so it needs the name of the
>> uncompressed file.
>
> Really?  I think it quite unclean if gdb expects BFD to reopen a file
> that gdb has closed!

Agreed.

GDB doesn't expect BFD to reopen the file, per se.  But it does a seek
on an exec file (IIRC, while reading symbols) which it previously closed
and when BFD notices that the file is closed, it opens it.  I don't think
that it is feasible to remove calls to exec_close() so this doesn't happen.

There's overlapping code between BFD and GDB.  It would be much cleaner if
GDB let BFD do everything with files, and BFD had support for opening files
with additional flags like O_CLOEXEC.

-- 
Michael Eager	 eager@eagercon.com
1960 Park Blvd., Palo Alto, CA 94306  650-325-8077

Re: [PATCH] Support gzip compressed exec and core files in gdb

[Alan Modra]

On Wed, Mar 11, 2015 at 05:45:28PM -0700, Michael Eager wrote:
> On 03/11/15 17:08, Alan Modra wrote:
> >On Wed, Mar 11, 2015 at 07:56:30AM -0700, Michael Eager wrote:
> >>On 03/11/15 01:14, Alan Modra wrote:
> >>>On Tue, Mar 10, 2015 at 04:01:42PM -0700, Michael Eager wrote:
> >>>>This operation cannot be done completely by BFD because BFD allows an opened
> >>>>file to be passed to it for processing.  GDB uses this functionality.
> >>>
> >>>I'd prefer you do this entirely outside of BFD, without adding another
> >>>field to struct bfd.  I think that can be done by simply clearing
> >>>abfd->cacheable on files you uncompress.  This prevents BFD from
> >>>closing the file, so you won't need to open it again.
> >>
> >>GDB closes the exec file, then uses BFD to seek (I think when reading
> >>syms).  BFD then re-opens the file, so it needs the name of the
> >>uncompressed file.
> >
> >Really?  I think it quite unclean if gdb expects BFD to reopen a file
> >that gdb has closed!
> 
> Agreed.
> 
> GDB doesn't expect BFD to reopen the file, per se.  But it does a seek
> on an exec file (IIRC, while reading symbols) which it previously closed
> and when BFD notices that the file is closed, it opens it.  I don't think
> that it is feasible to remove calls to exec_close() so this doesn't happen.

It looks to me that exec_close() calls bfd_close().  You won't be able
to do anything with the bfd after bfd_close(), so I think your
analysis is faulty and very much doubt your statement that "GDB closes
the exec file, then uses BFD..".

-- 
Alan Modra
Australia Development Lab, IBM

Re: [PATCH] Support gzip compressed exec and core files in gdb

[Pedro Alves]

On 03/12/2015 12:40 AM, Michael Eager wrote:
> On 03/11/15 15:13, Jan Kratochvil wrote:
>> On Wed, 11 Mar 2015 00:01:42 +0100, Michael Eager wrote:
>>> Add support to automatically unzip compressed executable and core files.
>>> Files will be uncompressed into temporary directory (/tmp or $TMPDIR)
>>> and are deleted when GDB exits.
>>
>> Such feature has been requested to support xz-compressed core files as
>> currently being stored by systemd.  But to make it more convenient one should
>> decompress on-demand only the blocks of file that are really accessed by GDB
>> - expecting by bfd_iovec.  Obviously GDB usually needs to access only small
>> part of the whole core file.
>>
>> I did not check how it is supported by gzip but for xz one needs to use
>> --block-size, otherwise the file blocks cannot be decompressed independently
>> in random access way.
> 
> gzip is not compressed block-by-block.  As far as I can tell, you need to
> decompress starting from the beginning of the file.
> 
>> ISTM libz-gzip and liblzma-xz compatibility is mutually exclusive.
> 
> I don't know why they would be incompatible, but support for an on-demand
> block-compression scheme would be significantly different.  Decompressing
> an xz file by making a copy (as is done for gzip) would be a simple extension
> to the current patch.

I won't strongly object if others want to approve it, but IMO, having GDB
decompress the whole file to temp doesn't add that much convenience over
decompressing it outside GDB.  Let me explore and expand (below).

We need to weigh the convenience of having gdb do this, over maintaining
this inside BFD+GDB going forward.

If you loading the core just once to extract a backtrace, in an
automated fashion, you can simply decompress before loading the core
with a trivial script.  So the convenience added for this use case
is not significant.

If OTOH you're doing interactive debugging of the core dump, then it's
convenient to be able to skip manual/laborious steps.

However, if people are compressing cores, it's because they are big.

As a quick experiment, I ran 'gcore `pidof firefox`' and generated
a core dump of the firefox process that I have running.  That resulted
in a 4.5GB core dump.  I gzipped it, which shrinked it to 4.5MB.
A ~1000/1 factor.  Then I timed gunzipping it.  It took almost
2 (two) minutes.

If you're doing interactive debugging (either CLI, or GUI) you'll
likely end up starting gdb multiple times, and thus load the core multiple
times into gdb, and each of those invocations will result in a
slow decompression of the whole file.

Waiting for GDB to decompress that once is already painful.  Waiting for it
multiple times likely results in cursing and swearing at gdb's slow start
up.  Smart users will realize that and end up decompressing the file manually
outside gdb, just once, anyway, thus saving time.

We could "fix" the "multiple times" issue by adding even more smarts,
based on an already-decompressed-files cache or some such.  Though of
course, more smarts, more code to maintain.

I agree with Jan -- The real convenience would be being able to skip the
long whole-file decompression step altogether, with an on-demand
block-decompress scheme, because gdb in reality doesn't need to touch
most of the vast majority of the core dump's contents.  That would
be a solution that I'd be happy to see implemented.

If we're just decompressing to /tmp, then we also need to
compare the benefits of a built-in solution against having users
do the same with a user-provided gdb command implemented in one
of gdb's extensions languages (python, scheme), e.g., a command
that adds a "decompress-core" command that does the same:
decompresses whatever compression format, and loads the result
with "core /tmp/FILE".

IMO, whatever the solution, if built in, this is best implemented
in BFD, so that objdump can dump the same files gdb can.

Thanks,
Pedro Alves

Re: [PATCH] Support gzip compressed exec and core files in gdb

[Jan Kratochvil]

On Thu, 12 Mar 2015 00:14:21 +0100, Doug Evans wrote:
> On Wed, Mar 11, 2015 at 3:13 PM, Jan Kratochvil
> <jan.kratochvil@redhat.com> wrote:
> > ISTM libz-gzip and liblzma-xz compatibility is mutually exclusive.
> 
> Can you elaborate?

That gzip decompression can be done by libz but libz cannot decompress xz.

The xz decompression can be done by liblzma but liblzma cannot decompress
gzip.

Therefore supporting both gzip and xz formats needs two functions / two
libraries / two APIs support in GDB.  But that may not be needed:


On Thu, 12 Mar 2015 01:40:20 +0100, Michael Eager wrote:
> gzip is not compressed block-by-block.  As far as I can tell, you need to
> decompress starting from the beginning of the file.

I also think so.

Then I do not understand why to support gzip in the first place. One testfile
does not represent all testcases but what I randomly tried now:

uncompressed : 342549479
gzip -9      :  26053431  0m14.839s
xz   -9  -T32:  15135468  0m13.415s (--block-size=10000000)
xz   -9e -T32:  12825220  0m38.119s (--block-size=10000000)
xz   -1      :  18114936  0m 8.495s
xz   -2      :  17632160  0m12.248s
xz   -9      :  15490372  3m13.554s
xz   -9e     :  12606128 18m35.478s

gzip is irrelevant, xz is about twice size or time better by every metric one
can find.


> support for an on-demand block-compression scheme would be significantly
> different.  Decompressing an xz file by making a copy (as is done for gzip)
> would be a simple extension to the current patch.

The on-demand block decompression would bring a new functionality, the whole
file decompression is one command saving convenience function.

I do not plan to implement it, just if you aware of both the xz and block
decompression advantages.


Jan

Re: [PATCH] Support gzip compressed exec and core files in gdb

[Michael Eager]

On 03/11/15 21:54, Jan Kratochvil wrote:
> On Thu, 12 Mar 2015 00:14:21 +0100, Doug Evans wrote:
>> On Wed, Mar 11, 2015 at 3:13 PM, Jan Kratochvil
>> <jan.kratochvil@redhat.com> wrote:
>>> ISTM libz-gzip and liblzma-xz compatibility is mutually exclusive.
>>
>> Can you elaborate?
>
> That gzip decompression can be done by libz but libz cannot decompress xz.
>
> The xz decompression can be done by liblzma but liblzma cannot decompress
> gzip.
>
> Therefore supporting both gzip and xz formats needs two functions / two
> libraries / two APIs support in GDB.

A different decompression library is needed for each compression type.
Adding another decompression method would be simple.

> Then I do not understand why to support gzip in the first place.

We have exec and core files which are compressed with gzip.

One testfile
> does not represent all testcases but what I randomly tried now:
>
> uncompressed : 342549479
> gzip -9      :  26053431  0m14.839s
> xz   -9  -T32:  15135468  0m13.415s (--block-size=10000000)
> xz   -9e -T32:  12825220  0m38.119s (--block-size=10000000)
> xz   -1      :  18114936  0m 8.495s
> xz   -2      :  17632160  0m12.248s
> xz   -9      :  15490372  3m13.554s
> xz   -9e     :  12606128 18m35.478s

This is not a comparison of compression methods.  For that you
can do an online search.  This is a patch to support gzipped
exec and core files.

> gzip is irrelevant, xz is about twice size or time better by every metric one
> can find.

Actually, this statement is irrelevant.  This is a patch to support the
gzipped files which we have, not files which use some other method which
we do not have.

>> support for an on-demand block-compression scheme would be significantly
>> different.  Decompressing an xz file by making a copy (as is done for gzip)
>> would be a simple extension to the current patch.
>
> The on-demand block decompression would bring a new functionality, the whole
> file decompression is one command saving convenience function.
>
> I do not plan to implement it, just if you aware of both the xz and block
> decompression advantages.

I'm certainly aware of different compression methods.  This patch brings
new functionality to gdb.  The patch has a framework which can be extended
to support other methods, if and when someone is interested in implementing it.

-- 
Michael Eager	 eager@eagercon.com
1960 Park Blvd., Palo Alto, CA 94306  650-325-8077

Re: [PATCH] Support gzip compressed exec and core files in gdb

[Michael Eager]

On 03/12/15 03:41, Pedro Alves wrote:

> Waiting for GDB to decompress that once is already painful.  Waiting for it
> multiple times likely results in cursing and swearing at gdb's slow start
> up.  Smart users will realize that and end up decompressing the file manually
> outside gdb, just once, anyway, thus saving time.
>
> We could "fix" the "multiple times" issue by adding even more smarts,
> based on an already-decompressed-files cache or some such.  Though of
> course, more smarts, more code to maintain.

I had considered adding a command or command line option to specify
the name of the uncompress file, so that it could be reused.

> I agree with Jan -- The real convenience would be being able to skip the
> long whole-file decompression step altogether, with an on-demand
> block-decompress scheme, because gdb in reality doesn't need to touch
> most of the vast majority of the core dump's contents.  That would
> be a solution that I'd be happy to see implemented.

That's a solution to a different problem.

> If we're just decompressing to /tmp, then we also need to
> compare the benefits of a built-in solution against having users
> do the same with a user-provided gdb command implemented in one
> of gdb's extensions languages (python, scheme), e.g., a command
> that adds a "decompress-core" command that does the same:
> decompresses whatever compression format, and loads the result
> with "core /tmp/FILE".

This requires that users manually decompress files, and makes it
impossible to put the compressed file name on the command line.

It also looks to me like a wart and kludge, rather than having
GDB automatically identify the compression method and do the
operation transparently for the user.

> IMO, whatever the solution, if built in, this is best implemented
> in BFD, so that objdump can dump the same files gdb can.

I took that approach initially.  But GDB finds and opens files,
not BFD.  Moving what GDB is doing into BFD, where it should have
been in the first place (IMO), seemed more problematic.

-- 
Michael Eager	 eager@eagercon.com
1960 Park Blvd., Palo Alto, CA 94306  650-325-8077

Re: [PATCH] Support gzip compressed exec and core files in gdb

[Pedro Alves]

On 03/12/2015 03:34 PM, Michael Eager wrote:
> On 03/12/15 03:41, Pedro Alves wrote:
> 
>> Waiting for GDB to decompress that once is already painful.  Waiting for it
>> multiple times likely results in cursing and swearing at gdb's slow start
>> up.  Smart users will realize that and end up decompressing the file manually
>> outside gdb, just once, anyway, thus saving time.
>>
>> We could "fix" the "multiple times" issue by adding even more smarts,
>> based on an already-decompressed-files cache or some such.  Though of
>> course, more smarts, more code to maintain.
> 
> I had considered adding a command or command line option to specify
> the name of the uncompress file, so that it could be reused.

What's the point then?  If you need to do that, then you alread
lost all the convenience.  Just type "gunzip core.gz && gdb core"
instead of "gdb -tmp-core /tmp/core core.gz".

So I think my point still stands, and IMO, it's a crucial point.

> 
>> I agree with Jan -- The real convenience would be being able to skip the
>> long whole-file decompression step altogether, with an on-demand
>> block-decompress scheme, because gdb in reality doesn't need to touch
>> most of the vast majority of the core dump's contents.  That would
>> be a solution that I'd be happy to see implemented.
> 
> That's a solution to a different problem.

I don't think it is.  What's the real problem you're solving?

>> If we're just decompressing to /tmp, then we also need to
>> compare the benefits of a built-in solution against having users
>> do the same with a user-provided gdb command implemented in one
>> of gdb's extensions languages (python, scheme), e.g., a command
>> that adds a "decompress-core" command that does the same:
>> decompresses whatever compression format, and loads the result
>> with "core /tmp/FILE".
> 
> This requires that users manually decompress files, and makes it
> impossible to put the compressed file name on the command line.

No it doesn't: there's '-ex' to run commands on the command
line: gdb -ex "decompress-and-load-core foo.gz"

> 
> It also looks to me like a wart and kludge, rather than having
> GDB automatically identify the compression method and do the
> operation transparently for the user.

What I'm saying is that it seems to me that you're doing
automatically in GDB, it can be done automatically in a
script.  A gdb command implemented in python can of course
also identify the compression method and support a multiple
number of compression formats, by just calling the
appropriate decompression tool.

As I said, I won't strongly object, though I still don't
see the compelling use case that warrants doing this in GDB.
I do envision ahead the usability problems and support
requests this will lead to.

> 
>> IMO, whatever the solution, if built in, this is best implemented
>> in BFD, so that objdump can dump the same files gdb can.
> 
> I took that approach initially.  But GDB finds and opens files,
> not BFD.  Moving what GDB is doing into BFD, where it should have
> been in the first place (IMO), seemed more problematic.

If there's problems, let's fix them.  From Alan's response,
the problem you mention doesn't really exist in the form you
described.

Thanks,
Pedro Alves

Re: [PATCH] Support gzip compressed exec and core files in gdb

[Michael Eager]

On 03/11/15 19:50, Alan Modra wrote:
> On Wed, Mar 11, 2015 at 05:45:28PM -0700, Michael Eager wrote:
>> On 03/11/15 17:08, Alan Modra wrote:
>>> On Wed, Mar 11, 2015 at 07:56:30AM -0700, Michael Eager wrote:
>>>> On 03/11/15 01:14, Alan Modra wrote:
>>>>> On Tue, Mar 10, 2015 at 04:01:42PM -0700, Michael Eager wrote:
>>>>>> This operation cannot be done completely by BFD because BFD allows an opened
>>>>>> file to be passed to it for processing.  GDB uses this functionality.
>>>>>
>>>>> I'd prefer you do this entirely outside of BFD, without adding another
>>>>> field to struct bfd.  I think that can be done by simply clearing
>>>>> abfd->cacheable on files you uncompress.  This prevents BFD from
>>>>> closing the file, so you won't need to open it again.
>>>>
>>>> GDB closes the exec file, then uses BFD to seek (I think when reading
>>>> syms).  BFD then re-opens the file, so it needs the name of the
>>>> uncompressed file.
>>>
>>> Really?  I think it quite unclean if gdb expects BFD to reopen a file
>>> that gdb has closed!
>>
>> Agreed.
>>
>> GDB doesn't expect BFD to reopen the file, per se.  But it does a seek
>> on an exec file (IIRC, while reading symbols) which it previously closed
>> and when BFD notices that the file is closed, it opens it.  I don't think
>> that it is feasible to remove calls to exec_close() so this doesn't happen.
>
> It looks to me that exec_close() calls bfd_close().  You won't be able
> to do anything with the bfd after bfd_close(), so I think your
> analysis is faulty and very much doubt your statement that "GDB closes
> the exec file, then uses BFD..".

The file opened in exec_file_attach() is closed in bfd_cache_close_all(),
before the function returns, not in the call to exec_close().  The bfd
is not deleted.

Later, in bfd_get_section_contents(), while sniffing the OSABI, GDB calls
bfd_seek() to do a seek on the same bfd.  This notices that bfd->iostream
is zero and re-opens the file.

-- 
Michael Eager	 eager@eagercon.com
1960 Park Blvd., Palo Alto, CA 94306  650-325-8077

Re: [PATCH] Support gzip compressed exec and core files in gdb

[Michael Eager]

On 03/12/15 09:12, Pedro Alves wrote:
> On 03/12/2015 03:34 PM, Michael Eager wrote:
>> On 03/12/15 03:41, Pedro Alves wrote:
>>
>>> Waiting for GDB to decompress that once is already painful.  Waiting for it
>>> multiple times likely results in cursing and swearing at gdb's slow start
>>> up.  Smart users will realize that and end up decompressing the file manually
>>> outside gdb, just once, anyway, thus saving time.
>>>
>>> We could "fix" the "multiple times" issue by adding even more smarts,
>>> based on an already-decompressed-files cache or some such.  Though of
>>> course, more smarts, more code to maintain.
>>
>> I had considered adding a command or command line option to specify
>> the name of the uncompress file, so that it could be reused.
>
> What's the point then?  If you need to do that, then you alread
> lost all the convenience.  Just type "gunzip core.gz && gdb core"
> instead of "gdb -tmp-core /tmp/core core.gz".
>
> So I think my point still stands, and IMO, it's a crucial point.
>
>>
>>> I agree with Jan -- The real convenience would be being able to skip the
>>> long whole-file decompression step altogether, with an on-demand
>>> block-decompress scheme, because gdb in reality doesn't need to touch
>>> most of the vast majority of the core dump's contents.  That would
>>> be a solution that I'd be happy to see implemented.
>>
>> That's a solution to a different problem.
>
> I don't think it is.  What's the real problem you're solving?

Lots of users, most who are not very conversant with gdb, lots of
compressed exec and core files, all compressed with gzip.  The
goal is to make using compressed files simple and transparent,
without users having to enter additional commands.

A wonderful patch supporting xz compressed files with an on-demand
block-decompress scheme would be a great solution for some other
use case, one which seems hypothetical at the moment.

>>> If we're just decompressing to /tmp, then we also need to
>>> compare the benefits of a built-in solution against having users
>>> do the same with a user-provided gdb command implemented in one
>>> of gdb's extensions languages (python, scheme), e.g., a command
>>> that adds a "decompress-core" command that does the same:
>>> decompresses whatever compression format, and loads the result
>>> with "core /tmp/FILE".
>>
>> This requires that users manually decompress files, and makes it
>> impossible to put the compressed file name on the command line.
>
> No it doesn't: there's '-ex' to run commands on the command
> line: gdb -ex "decompress-and-load-core foo.gz"

As I said, this requires users to know whether an exec or core
is compressed and manually enter a command to uncompress it.

>> It also looks to me like a wart and kludge, rather than having
>> GDB automatically identify the compression method and do the
>> operation transparently for the user.
>
> What I'm saying is that it seems to me that you're doing
> automatically in GDB, it can be done automatically in a
> script.  A gdb command implemented in python can of course
> also identify the compression method and support a multiple
> number of compression formats, by just calling the
> appropriate decompression tool.

We have many wrappers around gdb, some of which handle compressed
files, some of which don't.  They are all different and
problematic to debug and maintain.  Putting support for compressed
files into gdb means that it simply works, and users don't have
to remember which wrapper accepts compressed files and which
don't.

> As I said, I won't strongly object, though I still don't
> see the compelling use case that warrants doing this in GDB.
> I do envision ahead the usability problems and support
> requests this will lead to.
>
>>
>>> IMO, whatever the solution, if built in, this is best implemented
>>> in BFD, so that objdump can dump the same files gdb can.
>>
>> I took that approach initially.  But GDB finds and opens files,
>> not BFD.  Moving what GDB is doing into BFD, where it should have
>> been in the first place (IMO), seemed more problematic.
>
> If there's problems, let's fix them.  From Alan's response,
> the problem you mention doesn't really exist in the form you
> described.

I misspoke/misremembered.  It isn't exec_close() which closes the
file, it is bfd_cache_close_all().  The bfd is not closed, only
the file.

While BFD does most of the file operations, there isn't a clear
functional boundary between GDB file operations and BFD file
operations.  Allowing an opened fd to be passed into BFD makes
doing the decompression in BFD problematic, since BFD doesn't
know where the opened file was found, and the decompress libraries
(at least gzopen()) expects a path, not an opened fd.  BFD
wouldn't be know how the fd was opened so that it could use the
same flags to open the uncompressed file.

Fixing the problem would mean eliminating functions in BFD
which accepted an open fd and replacing calls to these functions
with modified versions of bfd_open() which accepted additional
flags or did whatever other operations the caller did when opening
the files.  GDB opens files a number of places, such as in
exec_file_attach() using openp() which searches the path for
the file.  This would have to be migrated into BFD.

-- 
Michael Eager	 eager@eagercon.com
1960 Park Blvd., Palo Alto, CA 94306  650-325-8077

Re: [PATCH] Support gzip compressed exec and core files in gdb

[Jan Kratochvil]

On Thu, 12 Mar 2015 17:58:02 +0100, Michael Eager wrote:
> I misspoke/misremembered.  It isn't exec_close() which closes the
> file, it is bfd_cache_close_all().  The bfd is not closed, only
> the file.

This is problematic, I have already posted a pending patch for it:
	[patch] Do not close BFDs, breaking deleted inferior shlibs
	https://sourceware.org/ml/gdb-patches/2015-02/msg00367.html


> Allowing an opened fd to be passed into BFD makes
> doing the decompression in BFD problematic, since BFD doesn't
> know where the opened file was found, and the decompress libraries
> (at least gzopen()) expects a path, not an opened fd.

One can readlink(/proc/self/fd/%d) although I haven't checked now how exactly
to use it.


Jan

Re: [PATCH] Support gzip compressed exec and core files in gdb

[Michael Eager]

On 03/12/15 10:11, Jan Kratochvil wrote:
> On Thu, 12 Mar 2015 17:58:02 +0100, Michael Eager wrote:
>> I misspoke/misremembered.  It isn't exec_close() which closes the
>> file, it is bfd_cache_close_all().  The bfd is not closed, only
>> the file.
>
> This is problematic, I have already posted a pending patch for it:
> 	[patch] Do not close BFDs, breaking deleted inferior shlibs
> 	https://sourceware.org/ml/gdb-patches/2015-02/msg00367.html

Thanks.  I'll see if this eliminates the need for the BFD patch.

>> Allowing an opened fd to be passed into BFD makes
>> doing the decompression in BFD problematic, since BFD doesn't
>> know where the opened file was found, and the decompress libraries
>> (at least gzopen()) expects a path, not an opened fd.
>
> One can readlink(/proc/self/fd/%d) although I haven't checked now how exactly
> to use it.

This won't tell what flags were used to open the file,
so that the same flags can be used on the decompressed file.

-- 
Michael Eager	 eager@eagercon.com
1960 Park Blvd., Palo Alto, CA 94306  650-325-8077

Re: [PATCH] Support gzip compressed exec and core files in gdb

[Jan Kratochvil]

On Thu, 12 Mar 2015 18:37:45 +0100, Michael Eager wrote:
> On 03/12/15 10:11, Jan Kratochvil wrote:
> > One can readlink(/proc/self/fd/%d) although I haven't checked now how exactly
> > to use it.
> 
> This won't tell what flags were used to open the file,
> so that the same flags can be used on the decompressed file.

If you mean the O_RDONLY etc. flags then read(/proc/self/fdinfo/%d):
	pos:	0
	flags:	0102002
	mnt_id:	20


Jan

Re: [PATCH] Support gzip compressed exec and core files in gdb

[Sergio Durigan Junior]

On Wednesday, March 11 2015, Eli Zaretskii wrote:

>> I think it might be good to tell people how to specify where the
>> uncompressed file is located.
>
> Saying "system-wide temporary directory" does tell that, doesn't it?
> Users of each platform know where that is, I think.

Why not say:

  system-wide temporary directory (e.g., @file{/tmp} on some systems)

?

-- 
Sergio
GPG key ID: 0x65FC5E36
Please send encrypted e-mail if possible
http://sergiodj.net/

Re: [PATCH] Support gzip compressed exec and core files in gdb

[Alan Modra]

On Thu, Mar 12, 2015 at 09:14:12AM -0700, Michael Eager wrote:
> On 03/11/15 19:50, Alan Modra wrote:
> >On Wed, Mar 11, 2015 at 05:45:28PM -0700, Michael Eager wrote:
> >>On 03/11/15 17:08, Alan Modra wrote:
> >>>On Wed, Mar 11, 2015 at 07:56:30AM -0700, Michael Eager wrote:
> >>>>On 03/11/15 01:14, Alan Modra wrote:
> >>>>>On Tue, Mar 10, 2015 at 04:01:42PM -0700, Michael Eager wrote:
> >>>>>>This operation cannot be done completely by BFD because BFD allows an opened
> >>>>>>file to be passed to it for processing.  GDB uses this functionality.
> >>>>>
> >>>>>I'd prefer you do this entirely outside of BFD, without adding another
> >>>>>field to struct bfd.  I think that can be done by simply clearing
> >>>>>abfd->cacheable on files you uncompress.  This prevents BFD from
> >>>>>closing the file, so you won't need to open it again.
> >>>>
> >>>>GDB closes the exec file, then uses BFD to seek (I think when reading
> >>>>syms).  BFD then re-opens the file, so it needs the name of the
> >>>>uncompressed file.
> >>>
> >>>Really?  I think it quite unclean if gdb expects BFD to reopen a file
> >>>that gdb has closed!
> >>
> >>Agreed.
> >>
> >>GDB doesn't expect BFD to reopen the file, per se.  But it does a seek
> >>on an exec file (IIRC, while reading symbols) which it previously closed
> >>and when BFD notices that the file is closed, it opens it.  I don't think
> >>that it is feasible to remove calls to exec_close() so this doesn't happen.
> >
> >It looks to me that exec_close() calls bfd_close().  You won't be able
> >to do anything with the bfd after bfd_close(), so I think your
> >analysis is faulty and very much doubt your statement that "GDB closes
> >the exec file, then uses BFD..".
> 
> The file opened in exec_file_attach() is closed in bfd_cache_close_all(),
> before the function returns, not in the call to exec_close().  The bfd
> is not deleted.

Thanks for tracking it down.  Not a gdb problem then, but likely a BFD
bug that bfd_cache_close{,_all} closes !cacheable bfds.

-- 
Alan Modra
Australia Development Lab, IBM

Re: [PATCH] Support gzip compressed exec and core files in gdb

[Eli Zaretskii]

> From: Sergio Durigan Junior <sergiodj@redhat.com>
> Cc: Michael Eager <eager@eagerm.com>, gdb-patches@sourceware.org,
>         binutils@sourceware.org
> Date: Thu, 12 Mar 2015 17:16:55 -0400
> 
> On Wednesday, March 11 2015, Eli Zaretskii wrote:
> 
> >> I think it might be good to tell people how to specify where the
> >> uncompressed file is located.
> >
> > Saying "system-wide temporary directory" does tell that, doesn't it?
> > Users of each platform know where that is, I think.
> 
> Why not say:
> 
>   system-wide temporary directory (e.g., @file{/tmp} on some systems)
> 
> ?

That'd be fine with me, thanks.

Re: [PATCH] Support gzip compressed exec and core files in gdb

[Ed Maste]

On 12 March 2015 at 13:11, Jan Kratochvil <jan.kratochvil@redhat.com> wrote:
>
> One can readlink(/proc/self/fd/%d) although I haven't checked now how exactly
> to use it.

That isn't portable though.

Re: [PATCH] Support gzip compressed exec and core files in gdb

[Michael Eager]

[-- Attachment #1: Type: text/plain, Size: 2017 bytes --]

On 03/10/15 16:01, Michael Eager wrote:
> Add support to automatically unzip compressed executable and core files.
> Files will be uncompressed into temporary directory (/tmp or $TMPDIR)
> and are deleted when GDB exits.  This should be transparent to users,
> except for disk space requirements.  The name of the uncompressed file is
> mentioned, but all references to the file in GDB messages is to the file
> which the user specified.
>
> This operation cannot be done completely by BFD because BFD allows an opened
> file to be passed to it for processing.  GDB uses this functionality.
>
> BFD:
>    * bfd-in2.h: Regenerate.
>    * bfd.c (struct bfd): Add uncompressed_filename.
>    * bfdio.c (bfd_get_mtime): Set bfd->mtime_set to true.
>    * cache.c (bfd_open): Open previously created uncompressed file.
>
> GDB:
>    * common/filestuff.c (struct compressed_file_cache_search, eq_compressed_file,
>    is_gzip, decompress_gzip, do_compressed_cleanup, gdb_uncompress): New.
>    * common/filestuff.h (gdb_uncompress): Declare.
>    * corelow.c (core_open): Uncompress core file.
>    * exec.c (exec_file_attach): Uncompress exe file.
>    * symfile.c (symfile_bfd_open): Uncompress sym (exe) file.
>
> GDB/DOC:
>    * gdb.texinfo: Mention gzipped exec and core files.
>

Revised patch attached.  After Jan's patch to not close cached files,
I was able to eliminate the changes to BFD.  Moved uncompress code to
utils.c.  Added NEWS entry.

gdb/ChangeLog:
   * utils.c (struct compressed_file_cache_search, eq_compressed_file,
   is_gzip, decompress_gzip, do_compressed_cleanup, gdb_uncompress): New.
   * utils.h (gdb_uncompress): Declare.
   * corelow.c (core_open): Uncompress core file.
   * exec.c (exec_file_attach): Uncompress exec file.
   * symfile.c (symfile_bfd_open): Uncompress sym (exec) file.
   * NEWS: Mention new functionality.

gdb/doc:
   * gdb.texinfo (Files): Mention gzipped exec and core files.

-- 
Michael Eager	 eager@eagercon.com
1960 Park Blvd., Palo Alto, CA 94306  650-325-8077

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-GDB-support-compressed-exec-and-core-files.patch --]
[-- Type: text/x-patch; name="0001-GDB-support-compressed-exec-and-core-files.patch", Size: 11303 bytes --]

From e0946586d235a45b5c314e7af35970ed79317e43 Mon Sep 17 00:00:00 2001
From: Michael Eager <eager@eagercon.com>
Date: Wed, 18 Mar 2015 11:08:38 -0700
Subject: [PATCH] GDB support compressed exec and core files.

Add support to automatically unzip compressed executable and core files.
Files will be uncompressed into temporary directory (/tmp or $TMPDIR)
and are deleted when GDB exits.  This should be transparent to users,
except for disk space requirements.  The name of the uncompressed file is
mentioned, but all references to the file in GDB messages is to the file
which the user specified.

gdb/ChangeLog:
  * utils.c (struct compressed_file_cache_search, eq_compressed_file,
  is_gzip, decompress_gzip, do_compressed_cleanup, gdb_uncompress): New.
  * utils.h (gdb_uncompress): Declare.
  * corelow.c (core_open): Uncompress core file.
  * exec.c (exec_file_attach): Uncompress exec file.
  * symfile.c (symfile_bfd_open): Uncompress sym (exec) file.
  * NEWS: Mention new functionality.

gdb/doc:
  * gdb.texinfo (Files): Mention gzipped exec and core files.
---
 gdb/NEWS            |   3 +
 gdb/corelow.c       |   9 +++
 gdb/doc/gdb.texinfo |   5 ++
 gdb/exec.c          |  14 +++-
 gdb/symfile.c       |  13 +++-
 gdb/utils.c         | 199 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 gdb/utils.h         |   4 ++
 7 files changed, 245 insertions(+), 2 deletions(-)

diff --git a/gdb/NEWS b/gdb/NEWS
index bda4a35..b132196 100644
--- a/gdb/NEWS
+++ b/gdb/NEWS
@@ -89,6 +89,9 @@ vFile:fstat:
 
 * Removed targets and native configurations
 
+* GDB will automatically uncompress executable and core files
+  which have been compressed using gzip.
+
 HP/PA running HP-UX           hppa*-*-hpux*
 Itanium running HP-UX         ia64-*-hpux*
 
diff --git a/gdb/corelow.c b/gdb/corelow.c
index 9218003..0314d1f 100644
--- a/gdb/corelow.c
+++ b/gdb/corelow.c
@@ -279,6 +279,7 @@ core_open (const char *arg, int from_tty)
   int scratch_chan;
   int flags;
   char *filename;
+  char *uncompressed_filename;
 
   target_preopen (from_tty);
   if (!arg)
@@ -316,6 +317,14 @@ core_open (const char *arg, int from_tty)
   if (temp_bfd == NULL)
     perror_with_name (filename);
 
+  if (!write_files && gdb_uncompress (filename, &uncompressed_filename))
+    {
+      close (scratch_chan);
+      scratch_chan = gdb_open_cloexec (uncompressed_filename, flags, 0);
+      temp_bfd = gdb_bfd_fopen (uncompressed_filename, gnutarget,
+				FOPEN_RB, scratch_chan);
+    }
+
   if (!bfd_check_format (temp_bfd, bfd_core)
       && !gdb_check_format (temp_bfd))
     {
diff --git a/gdb/doc/gdb.texinfo b/gdb/doc/gdb.texinfo
index 9e71642..4c9c3f3 100644
--- a/gdb/doc/gdb.texinfo
+++ b/gdb/doc/gdb.texinfo
@@ -17375,6 +17375,11 @@ via @code{gdbserver} (@pxref{Server, file, Using the @code{gdbserver}
 Program}).  In these situations the @value{GDBN} commands to specify
 new files are useful.
 
+Executable and core files may be compressed using @command{gzip}.  These
+files will be uncompressed into temporary files in the system-wide
+temporary directory (e.g., @file{/tmp} on some systems).  The files will
+have a unique name and will be deleted when @value{GDBN} terminates.
+
 @table @code
 @cindex executable file
 @kindex file
diff --git a/gdb/exec.c b/gdb/exec.c
index b1f6157..4e550e1 100644
--- a/gdb/exec.c
+++ b/gdb/exec.c
@@ -155,6 +155,7 @@ void
 exec_file_attach (const char *filename, int from_tty)
 {
   struct cleanup *cleanups;
+  char *uncompressed_filename = NULL;
 
   /* First, acquire a reference to the current exec_bfd.  We release
      this at the end of the function; but acquiring it now lets the
@@ -209,7 +210,18 @@ exec_file_attach (const char *filename, int from_tty)
 	exec_bfd = gdb_bfd_fopen (canonical_pathname, gnutarget,
 				  FOPEN_RUB, scratch_chan);
       else
-	exec_bfd = gdb_bfd_open (canonical_pathname, gnutarget, scratch_chan);
+	{
+	  if (!gdb_uncompress (canonical_pathname, &uncompressed_filename))
+	    exec_bfd = gdb_bfd_open (canonical_pathname, gnutarget, scratch_chan);
+          else
+	    {
+	      close (scratch_chan);
+	      scratch_chan = openp ("", 0, uncompressed_filename,
+				    O_RDONLY | O_BINARY, &scratch_pathname);
+
+	      exec_bfd = gdb_bfd_open (uncompressed_filename, gnutarget, scratch_chan);
+	    }
+	}
 
       if (!exec_bfd)
 	{
diff --git a/gdb/symfile.c b/gdb/symfile.c
index bd3a366..e24517f 100644
--- a/gdb/symfile.c
+++ b/gdb/symfile.c
@@ -56,6 +56,7 @@
 #include "stack.h"
 #include "gdb_bfd.h"
 #include "cli/cli-utils.h"
+#include "utils.h"
 
 #include <sys/types.h>
 #include <fcntl.h>
@@ -1744,6 +1745,7 @@ symfile_bfd_open (const char *cname)
   bfd *sym_bfd;
   int desc;
   char *name, *absolute_name;
+  char *uncompressed_filename;
   struct cleanup *back_to;
 
   if (remote_filename_p (cname))
@@ -1788,7 +1790,16 @@ symfile_bfd_open (const char *cname)
   name = absolute_name;
   back_to = make_cleanup (xfree, name);
 
-  sym_bfd = gdb_bfd_open (name, gnutarget, desc);
+  if (!gdb_uncompress (name, &uncompressed_filename))
+    sym_bfd = gdb_bfd_open (name, gnutarget, desc);
+  else
+    {
+      close (desc);
+      desc = openp ("", 0, uncompressed_filename, O_RDONLY | O_BINARY, &absolute_name);
+
+      sym_bfd = gdb_bfd_open (uncompressed_filename, gnutarget, desc);
+    }
+
   if (!sym_bfd)
     error (_("`%s': can't open to read symbols: %s."), name,
 	   bfd_errmsg (bfd_get_error ()));
diff --git a/gdb/utils.c b/gdb/utils.c
index 7172bba..35e7c02 100644
--- a/gdb/utils.c
+++ b/gdb/utils.c
@@ -29,6 +29,10 @@
 #include <sys/resource.h>
 #endif /* HAVE_SYS_RESOURCE_H */
 
+#ifdef HAVE_ZLIB_H
+#include <zlib.h>
+#endif
+
 #ifdef TUI
 #include "tui/tui.h"		/* For tui_get_command_dimension.   */
 #endif
@@ -3486,6 +3490,201 @@ gdb_filename_fnmatch (const char *pattern, const char *string, int flags)
 
   return fnmatch (pattern, string, flags);
 }
+\f
+#ifdef HAVE_ZLIB_H
+/* Hash table of compressed files.  */
+
+static htab_t compressed_file_cache;
+
+struct compressed_file_cache_search
+{
+  char *filename;
+  char *uncompressed_filename;
+  time_t mtime;
+};
+
+static int
+eq_compressed_file (const void *a, const void *b)
+{
+  const struct compressed_file_cache_search *entry = a;
+  const struct compressed_file_cache_search *search = b;
+
+  return (strcmp (entry->filename, search->filename) == 0);
+}
+#endif
+
+/* Test if file is compressed with gzip.  */
+
+static inline int
+is_gzip (unsigned char *buf)
+{
+  return (buf[0] == 037 && buf[1] == 0213);	/* From /usr/share/magic.  */
+}
+
+#define COMPRESS_BUF_SIZE (1024*1024)
+static int
+decompress_gzip (const char *filename, FILE *tmp)
+{
+#ifdef HAVE_ZLIB_H
+  char *buf = xmalloc (COMPRESS_BUF_SIZE);
+  gzFile compressed = gzopen (filename, "r");
+  int count, res;
+
+  if (buf == NULL || compressed == NULL)
+    {
+      fprintf_filtered (gdb_stderr, _("error copying gzip file\n"));
+      free (buf);
+      return 0;
+    }
+
+  while ((count = gzread (compressed, buf, COMPRESS_BUF_SIZE)))
+    {
+      res = fwrite (buf, 1, count, tmp);
+      if (res != count)
+	{
+	  fprintf_filtered (gdb_stderr, _("error decompressing gzip file\n"));
+          free (buf);
+	  return 0;
+	}
+    }
+
+  gzclose (compressed);
+  free (buf);
+  return 1;
+#else
+  return 0;
+#endif
+}
+
+/* Delete uncompressed temp file when terminating.  */
+static void
+do_compressed_cleanup (void *filename)
+{
+  unlink (filename);
+  xfree (filename);
+}
+
+/* If file is compressed, uncompress it into a temporary.  */
+
+int
+gdb_uncompress (const char *filename, char **uncompressed_filename)
+{
+  FILE *handle;
+  struct compressed_file_cache_search search, *found;
+  struct stat st;
+  hashval_t hash;
+  void **slot;
+  static unsigned char buffer[1024];
+  size_t count;
+  enum {NONE, GZIP, BZIP2} file_compression = NONE;
+  int decomp_fd;
+  FILE *decomp_file;
+  int ret = 0;
+  char *tmpdir, *p;
+  char *template;
+
+  if (compressed_file_cache == NULL)
+    compressed_file_cache = htab_create_alloc (1, htab_hash_string,
+					       eq_compressed_file,
+					       NULL, xcalloc, xfree);
+
+  if (stat (filename, &st) < 0)
+    return 0;
+
+  search.filename = (char *) filename;
+  search.uncompressed_filename = NULL;
+
+  hash = htab_hash_string (filename);
+  found = htab_find_with_hash (compressed_file_cache, &search, hash);
+
+  if (found)
+    {
+      /* We previously uncompressed the file.  */
+      if (found->mtime == st.st_mtime)
+        {
+	  /* Return file if compressed file not changed.  */
+	  *uncompressed_filename = found->uncompressed_filename;
+	  return 1;
+	}
+      else
+        {
+	  /* Delete old uncompressed file.  */
+	  unlink (found->uncompressed_filename);
+	  xfree (found->filename);
+	  xfree (found->uncompressed_filename);
+        }
+    }
+
+  handle = fopen (filename, FOPEN_RB);
+  if (handle == NULL)
+    return 0;
+
+  count = fread (buffer, 1, sizeof buffer, handle);
+  if (count > 0)
+    {
+      if (is_gzip (buffer))
+	file_compression = GZIP;
+    }
+
+  fclose (handle);
+
+  if (file_compression == NONE)
+    return 0;
+
+  /* Create temporary file name for uncompressed file.  */
+  if (!(tmpdir = getenv ("TMPDIR")))
+    tmpdir = "/tmp";
+
+  if (!asprintf (&template, "%s/%s-XXXXXX", tmpdir, basename (filename)))
+    return 0;
+
+  decomp_fd = mkstemp (template);
+  if (decomp_fd != -1)
+    {
+      decomp_file = fdopen (decomp_fd, "w+b");
+
+      if (file_compression == GZIP)
+        {
+	  printf (_("Decompressing %s to %s\n"), filename, template);
+	  ret = decompress_gzip (filename, decomp_file);
+	}
+      else
+        {
+          xfree (template);
+          return 0;
+        }
+      fclose (decomp_file);
+
+      if (ret)
+	{
+	  if (!found)
+	    {
+	      slot = htab_find_slot_with_hash (compressed_file_cache,
+					       &search, hash, INSERT);
+	      gdb_assert (slot && !*slot);
+	      found = xmalloc (sizeof (struct compressed_file_cache_search));
+	      *slot = found;
+	    }
+	  found->filename = strdup (filename);
+	  found->mtime = st.st_mtime;
+	  found->uncompressed_filename = template;
+	}
+    }
+  else
+    {
+      warning (_("Decompression failed\n"));
+      xfree (template);
+      return 0;
+    }
+
+  *uncompressed_filename = template;
+
+  /* Schedule delete of temp file when gdb ends.  */
+  make_final_cleanup (do_compressed_cleanup, xstrdup (template));
+
+  return 1;
+}
+\f
 
 /* Provide a prototype to silence -Wmissing-prototypes.  */
 extern initialize_file_ftype _initialize_utils;
diff --git a/gdb/utils.h b/gdb/utils.h
index b8e1aff..10d9c08 100644
--- a/gdb/utils.h
+++ b/gdb/utils.h
@@ -367,4 +367,8 @@ extern void dump_core (void);
 
 extern char *make_hex_string (const gdb_byte *data, size_t length);
 
+/* Uncompress file if compressed.  */
+
+int gdb_uncompress (const char *filename, char **uncompressed_filename);
+
 #endif /* UTILS_H */
-- 
1.8.1.4

Re: [PATCH] Support gzip compressed exec and core files in gdb

[Eli Zaretskii]

> Date: Wed, 18 Mar 2015 17:58:53 -0700
> From: Michael Eager <eager@eagerm.com>
> 
> Revised patch attached.  After Jan's patch to not close cached files,
> I was able to eliminate the changes to BFD.  Moved uncompress code to
> utils.c.  Added NEWS entry.

The NEWS entry is OK.

Re: [PATCH] Support gzip compressed exec and core files in gdb

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 2819 bytes --]

On 18 Mar 2015 17:58, Michael Eager wrote:
> --- a/gdb/utils.c
> +++ b/gdb/utils.c
>
> +#define COMPRESS_BUF_SIZE (1024*1024)

space around the *

would be nice to have a comment noting where the 1MiB comes from

> +static int
> +decompress_gzip (const char *filename, FILE *tmp)
> +{
> +#ifdef HAVE_ZLIB_H
> +  char *buf = xmalloc (COMPRESS_BUF_SIZE);
> +  gzFile compressed = gzopen (filename, "r");
> +  int count, res;
> +
> +  if (buf == NULL || compressed == NULL)
> +    {
> +      fprintf_filtered (gdb_stderr, _("error copying gzip file\n"));
> +      free (buf);
> +      return 0;
> +    }
> +
> +  while ((count = gzread (compressed, buf, COMPRESS_BUF_SIZE)))
> +    {
> +      res = fwrite (buf, 1, count, tmp);
> +      if (res != count)
> +	{
> +	  fprintf_filtered (gdb_stderr, _("error decompressing gzip file\n"));
> +          free (buf);

this line needs to use a tab to indent

> +/* If file is compressed, uncompress it into a temporary.  */
> +
> +int
> +gdb_uncompress (const char *filename, char **uncompressed_filename)
> +{
> +  FILE *handle;
> +  struct compressed_file_cache_search search, *found;
> +  struct stat st;
> +  hashval_t hash;
> +  void **slot;
> +  static unsigned char buffer[1024];
> +  size_t count;
> +  enum {NONE, GZIP, BZIP2} file_compression = NONE;

shouldn't this enum be declared outside this func ?

> +  if (found)
> +    {
> +      /* We previously uncompressed the file.  */
> +      if (found->mtime == st.st_mtime)
> +        {

this brace needs to indent w/a tab

> +	  /* Return file if compressed file not changed.  */
> +	  *uncompressed_filename = found->uncompressed_filename;
> +	  return 1;
> +	}
> +      else
> +        {

same here

> +	  /* Delete old uncompressed file.  */
> +	  unlink (found->uncompressed_filename);
> +	  xfree (found->filename);
> +	  xfree (found->uncompressed_filename);
> +        }

and here

> +  /* Create temporary file name for uncompressed file.  */
> +  if (!(tmpdir = getenv ("TMPDIR")))
> +    tmpdir = "/tmp";
> +
> +  if (!asprintf (&template, "%s/%s-XXXXXX", tmpdir, basename (filename)))
> +    return 0;
> +
> +  decomp_fd = mkstemp (template);

ignoring that assigningments in if statements are frowned upon, looks like you 
can just use make_temp_file(NULL) from libiberty

you would also leak the fopen() handle here.  probably want to go through this 
code again looking for such leaks.  and maybe try breaking the func up into more 
utility related chunks when possible.

> +  if (decomp_fd != -1)
> +    {
> +      decomp_file = fdopen (decomp_fd, "w+b");

FOPEN_WUB

> +      if (file_compression == GZIP)
> +        {

you've got more missing tabs in this file.  i'm going to stop checking.
-mike

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [PATCH] Support gzip compressed exec and core files in gdb

[Michael Eager]

[-- Attachment #1: Type: text/plain, Size: 2448 bytes --]

Removed binutils@sourceware.org.

On 03/20/15 15:16, Mike Frysinger wrote:
> On 18 Mar 2015 17:58, Michael Eager wrote:
>> --- a/gdb/utils.c
>> +++ b/gdb/utils.c
>>
>> +#define COMPRESS_BUF_SIZE (1024*1024)
>
> space around the *
>
> would be nice to have a comment noting where the 1MiB comes from

I added a comment, but the buffer size is arbitrary.

>> +  /* Create temporary file name for uncompressed file.  */
>> +  if (!(tmpdir = getenv ("TMPDIR")))
>> +    tmpdir = "/tmp";
>> +
>> +  if (!asprintf (&template, "%s/%s-XXXXXX", tmpdir, basename (filename)))
>> +    return 0;
>> +
>> +  decomp_fd = mkstemp (template);
>
> ignoring that assigningments in if statements are frowned upon, looks like you
> can just use make_temp_file(NULL) from libiberty

Fixed assignment.  make_temp_file() generates an anonymous file; I want to
keep the user-specified file name as part of the uncompressed file name.
(I could update libiberty to add an alternate to make_temp_file() which
takes a basename, I guess.)  I replaced getenv("TMPDIR") with choose_tmpdir()
from libiberty.

> you would also leak the fopen() handle here.  probably want to go through this
> code again looking for such leaks.  and maybe try breaking the func up into more
> utility related chunks when possible.

I refactored the gdb_uncompress() function and broke it into 3 functions.

> you've got more missing tabs in this file.  i'm going to stop checking.

I cleaned up the white space.  I hope I got all the tabs this time.

gdb/ChangeLog:
   * utils.c (struct compressed_file_cache_search, eq_compressed_file,
   is_gzip, decompress_gzip, do_compressed_cleanup, identify_compression,
   uncompress_to_temporary, gdb_uncompress): New.
   * utils.h (gdb_uncompress): Declare.
   * corelow.c (core_open): Uncompress core file.
   * exec.c (exec_file_attach): Uncompress exec file.
   * symfile.c (symfile_bfd_open): Uncompress sym (exec) file.
   * NEWS: Mention new functionality.

gdb/doc:
   * gdb.texinfo (Files): Mention gzipped exec and core files.


I'm not fond of whitespace patches, but there were many whitespace errors
in these files unrelated to my patch.  I've attached a second patch to clean
this up.

gdb/ChangeLog:
* corelow.c: Replace leading whitespace with tabs, eliminate trailing blanks.
* exec.c: Ditto.
* symfile.c: Ditto.
* utils.c: Ditto
* utils.h: Ditto



-- 
Michael Eager	 eager@eagercon.com
1960 Park Blvd., Palo Alto, CA 94306  650-325-8077

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-GDB-support-compressed-exec-and-core-files.patch --]
[-- Type: text/x-patch; name="0001-GDB-support-compressed-exec-and-core-files.patch", Size: 12454 bytes --]

From a2a0b14bda48e9e2af9842029d7774001e2e9132 Mon Sep 17 00:00:00 2001
From: Michael Eager <eager@eagercon.com>
Date: Thu, 26 Mar 2015 16:55:59 -0700
Subject: [PATCH 1/2] GDB support compressed exec and core files.

Add support to automatically unzip compressed executable and core files.
Files will be uncompressed into temporary directory (/tmp or $TMPDIR)
and are deleted when GDB exits.  This should be transparent to users,
except for disk space requirements.  The name of the uncompressed file is
mentioned, but all references to the file in GDB messages is to the file
which the user specified.

gdb/ChangeLog:
  * utils.c (struct compressed_file_cache_search, eq_compressed_file,
  is_gzip, decompress_gzip, do_compressed_cleanup, identify_compression,
  uncompress_to_temporar, gdb_uncompress): New.
  * utils.h (gdb_uncompress): Declare.
  * corelow.c (core_open): Uncompress core file.
  * exec.c (exec_file_attach): Uncompress exec file.
  * symfile.c (symfile_bfd_open): Uncompress sym (exec) file.
  * NEWS: Mention new functionality.

gdb/doc:
  * gdb.texinfo (Files): Mention gzipped exec and core files.
---
 gdb/NEWS            |   3 +
 gdb/corelow.c       |  11 ++-
 gdb/doc/gdb.texinfo |   5 ++
 gdb/exec.c          |  14 +++-
 gdb/symfile.c       |  13 +++-
 gdb/utils.c         | 221 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 gdb/utils.h         |   4 +
 7 files changed, 268 insertions(+), 3 deletions(-)

diff --git a/gdb/NEWS b/gdb/NEWS
index bda4a35..b132196 100644
--- a/gdb/NEWS
+++ b/gdb/NEWS
@@ -89,6 +89,9 @@ vFile:fstat:
 
 * Removed targets and native configurations
 
+* GDB will automatically uncompress executable and core files
+  which have been compressed using gzip.
+
 HP/PA running HP-UX           hppa*-*-hpux*
 Itanium running HP-UX         ia64-*-hpux*
 
diff --git a/gdb/corelow.c b/gdb/corelow.c
index 9218003..8211595 100644
--- a/gdb/corelow.c
+++ b/gdb/corelow.c
@@ -279,6 +279,7 @@ core_open (const char *arg, int from_tty)
   int scratch_chan;
   int flags;
   char *filename;
+  char *uncompressed_filename;
 
   target_preopen (from_tty);
   if (!arg)
@@ -310,12 +311,20 @@ core_open (const char *arg, int from_tty)
   if (scratch_chan < 0)
     perror_with_name (filename);
 
-  temp_bfd = gdb_bfd_fopen (filename, gnutarget, 
+  temp_bfd = gdb_bfd_fopen (filename, gnutarget,
 			    write_files ? FOPEN_RUB : FOPEN_RB,
 			    scratch_chan);
   if (temp_bfd == NULL)
     perror_with_name (filename);
 
+  if (!write_files && gdb_uncompress (filename, &uncompressed_filename))
+    {
+      close (scratch_chan);
+      scratch_chan = gdb_open_cloexec (uncompressed_filename, flags, 0);
+      temp_bfd = gdb_bfd_fopen (uncompressed_filename, gnutarget,
+				FOPEN_RB, scratch_chan);
+    }
+
   if (!bfd_check_format (temp_bfd, bfd_core)
       && !gdb_check_format (temp_bfd))
     {
diff --git a/gdb/doc/gdb.texinfo b/gdb/doc/gdb.texinfo
index 9e71642..4c9c3f3 100644
--- a/gdb/doc/gdb.texinfo
+++ b/gdb/doc/gdb.texinfo
@@ -17375,6 +17375,11 @@ via @code{gdbserver} (@pxref{Server, file, Using the @code{gdbserver}
 Program}).  In these situations the @value{GDBN} commands to specify
 new files are useful.
 
+Executable and core files may be compressed using @command{gzip}.  These
+files will be uncompressed into temporary files in the system-wide
+temporary directory (e.g., @file{/tmp} on some systems).  The files will
+have a unique name and will be deleted when @value{GDBN} terminates.
+
 @table @code
 @cindex executable file
 @kindex file
diff --git a/gdb/exec.c b/gdb/exec.c
index b1f6157..4b84d65 100644
--- a/gdb/exec.c
+++ b/gdb/exec.c
@@ -155,6 +155,7 @@ void
 exec_file_attach (const char *filename, int from_tty)
 {
   struct cleanup *cleanups;
+  char *uncompressed_filename = NULL;
 
   /* First, acquire a reference to the current exec_bfd.  We release
      this at the end of the function; but acquiring it now lets the
@@ -209,7 +210,18 @@ exec_file_attach (const char *filename, int from_tty)
 	exec_bfd = gdb_bfd_fopen (canonical_pathname, gnutarget,
 				  FOPEN_RUB, scratch_chan);
       else
-	exec_bfd = gdb_bfd_open (canonical_pathname, gnutarget, scratch_chan);
+	{
+	  if (!gdb_uncompress (canonical_pathname, &uncompressed_filename))
+	    exec_bfd = gdb_bfd_open (canonical_pathname, gnutarget, scratch_chan);
+	  else
+	    {
+	      close (scratch_chan);
+	      scratch_chan = openp ("", 0, uncompressed_filename,
+				    O_RDONLY | O_BINARY, &scratch_pathname);
+
+	      exec_bfd = gdb_bfd_open (uncompressed_filename, gnutarget, scratch_chan);
+	    }
+	}
 
       if (!exec_bfd)
 	{
diff --git a/gdb/symfile.c b/gdb/symfile.c
index bd3a366..e24517f 100644
--- a/gdb/symfile.c
+++ b/gdb/symfile.c
@@ -56,6 +56,7 @@
 #include "stack.h"
 #include "gdb_bfd.h"
 #include "cli/cli-utils.h"
+#include "utils.h"
 
 #include <sys/types.h>
 #include <fcntl.h>
@@ -1744,6 +1745,7 @@ symfile_bfd_open (const char *cname)
   bfd *sym_bfd;
   int desc;
   char *name, *absolute_name;
+  char *uncompressed_filename;
   struct cleanup *back_to;
 
   if (remote_filename_p (cname))
@@ -1788,7 +1790,16 @@ symfile_bfd_open (const char *cname)
   name = absolute_name;
   back_to = make_cleanup (xfree, name);
 
-  sym_bfd = gdb_bfd_open (name, gnutarget, desc);
+  if (!gdb_uncompress (name, &uncompressed_filename))
+    sym_bfd = gdb_bfd_open (name, gnutarget, desc);
+  else
+    {
+      close (desc);
+      desc = openp ("", 0, uncompressed_filename, O_RDONLY | O_BINARY, &absolute_name);
+
+      sym_bfd = gdb_bfd_open (uncompressed_filename, gnutarget, desc);
+    }
+
   if (!sym_bfd)
     error (_("`%s': can't open to read symbols: %s."), name,
 	   bfd_errmsg (bfd_get_error ()));
diff --git a/gdb/utils.c b/gdb/utils.c
index 7172bba..e9f86e5 100644
--- a/gdb/utils.c
+++ b/gdb/utils.c
@@ -29,6 +29,10 @@
 #include <sys/resource.h>
 #endif /* HAVE_SYS_RESOURCE_H */
 
+#ifdef HAVE_ZLIB_H
+#include <zlib.h>
+#endif
+
 #ifdef TUI
 #include "tui/tui.h"		/* For tui_get_command_dimension.   */
 #endif
@@ -126,6 +130,10 @@ int immediate_quit;
    as octal escapes.  Zero means just print the value (e.g. it's an
    international character, and the terminal or window can cope.)  */
 
+/* Type of file compression.  */
+
+enum file_compression_t {NONE, GZIP, BZIP2};
+
 int sevenbit_strings = 0;
 static void
 show_sevenbit_strings (struct ui_file *file, int from_tty,
@@ -3486,6 +3494,219 @@ gdb_filename_fnmatch (const char *pattern, const char *string, int flags)
 
   return fnmatch (pattern, string, flags);
 }
+\f
+#ifdef HAVE_ZLIB_H
+/* Hash table of compressed files.  */
+
+static htab_t compressed_file_cache;
+
+struct compressed_file_cache_search
+{
+  char *filename;
+  char *uncompressed_filename;
+  time_t mtime;
+};
+
+static int
+eq_compressed_file (const void *a, const void *b)
+{
+  const struct compressed_file_cache_search *entry = a;
+  const struct compressed_file_cache_search *search = b;
+
+  return (strcmp (entry->filename, search->filename) == 0);
+}
+#endif
+
+/* Test if file is compressed with gzip.  */
+
+static inline int
+is_gzip (unsigned char *buf)
+{
+  return (buf[0] == 037 && buf[1] == 0213);	/* From /usr/share/magic.  */
+}
+
+/* Alloc 1Mb buffer to uncompress data.  */
+#define COMPRESS_BUF_SIZE (1024 * 1024)
+static int
+decompress_gzip (const char *filename, FILE *tmp)
+{
+#ifdef HAVE_ZLIB_H
+  char *buf = xmalloc (COMPRESS_BUF_SIZE);
+  gzFile compressed = gzopen (filename, "r");
+  int count, res;
+
+  if (buf == NULL || compressed == NULL)
+    {
+      fprintf_filtered (gdb_stderr, _("error copying gzip file\n"));
+      free (buf);
+      return 0;
+    }
+
+  while ((count = gzread (compressed, buf, COMPRESS_BUF_SIZE)))
+    {
+      res = fwrite (buf, 1, count, tmp);
+      if (res != count)
+	{
+	  fprintf_filtered (gdb_stderr, _("error decompressing gzip file\n"));
+	  free (buf);
+	  return 0;
+	}
+    }
+
+  gzclose (compressed);
+  free (buf);
+  return 1;
+#else
+  return 0;
+#endif
+}
+
+/* Delete uncompressed temp file when terminating.  */
+static void
+do_compressed_cleanup (void *filename)
+{
+  unlink (filename);
+  xfree (filename);
+}
+
+/* Identify type of file compression used.  */
+static enum file_compression_t
+identify_compression (const char *filename)
+{
+  FILE *handle;
+  enum file_compression_t file_compression = NONE;
+  unsigned char buffer[1024];
+  size_t count;
+
+  handle = fopen (filename, FOPEN_RB);
+  if (handle == NULL)
+    return 0;
+
+  count = fread (buffer, 1, sizeof buffer, handle);
+  if (count > 0)
+    {
+      if (is_gzip (buffer))
+	file_compression = GZIP;
+    }
+
+  fclose (handle);
+
+  return file_compression;
+}
+
+/* Uncompress into temporary file.  Return temp file name.  */
+static char *
+uncompress_to_temporary (const char *filename)
+{
+  enum file_compression_t file_compression;
+  char *template;
+  int ret = 0;
+  int decomp_fd;
+  FILE *decomp_file;
+
+  file_compression = identify_compression (filename);
+  if (file_compression != GZIP)
+    /* Only gzip supported at this time.  */
+    return NULL;
+
+  /* Create temporary file name for uncompressed file.  */
+  if (!asprintf (&template, "%s%s-XXXXXX", choose_tmpdir (),
+		 basename (filename)))
+    return NULL;
+
+  decomp_fd = mkstemp (template);
+  if (decomp_fd == -1)
+    return NULL;
+
+  decomp_file = fdopen (decomp_fd, FOPEN_WUB);
+
+  if (file_compression == GZIP)
+    {
+      printf (_("Decompressing %s to %s\n"), filename, template);
+      ret = decompress_gzip (filename, decomp_file);
+    }
+
+  if (!ret)
+    {
+      xfree (template);
+      template = NULL;
+    }
+
+  fclose (decomp_file);
+  return template;
+}
+
+/* If file is compressed, uncompress it into a temporary.  */
+int
+gdb_uncompress (const char *filename, char **uncompressed_filename)
+{
+  struct compressed_file_cache_search search, *found;
+  struct stat st;
+  hashval_t hash;
+  void **slot;
+  char *temp_filename;
+
+  if (compressed_file_cache == NULL)
+    compressed_file_cache = htab_create_alloc (1, htab_hash_string,
+					       eq_compressed_file,
+					       NULL, xcalloc, xfree);
+  if (stat (filename, &st) < 0)
+    return 0;
+
+  search.filename = (char *) filename;
+  search.uncompressed_filename = NULL;
+
+  hash = htab_hash_string (filename);
+  found = htab_find_with_hash (compressed_file_cache, &search, hash);
+
+  if (found)
+    {
+      /* We previously uncompressed the file.  */
+      if (found->mtime == st.st_mtime)
+	{
+	  /* Return file if compressed file not changed.  */
+	  *uncompressed_filename = found->uncompressed_filename;
+	  return 1;
+	}
+      else
+	{
+	  /* Delete old uncompressed file.  */
+	  unlink (found->uncompressed_filename);
+	  xfree (found->filename);
+	  xfree (found->uncompressed_filename);
+	}
+    }
+
+  temp_filename = uncompress_to_temporary (filename);
+  if (temp_filename)
+    {
+      if (!found)
+	{
+	  slot = htab_find_slot_with_hash (compressed_file_cache,
+					   &search, hash, INSERT);
+	  gdb_assert (slot && !*slot);
+	  found = xmalloc (sizeof (struct compressed_file_cache_search));
+	  *slot = found;
+	}
+
+      found->filename = strdup (filename);
+      found->mtime = st.st_mtime;
+      found->uncompressed_filename = temp_filename;
+    }
+  else
+    {
+      warning (_("Decompression failed\n"));
+      return 0;
+    }
+
+  /* Schedule delete of temp file when gdb ends.  */
+  make_final_cleanup (do_compressed_cleanup, xstrdup (temp_filename));
+
+  *uncompressed_filename = temp_filename;
+
+  return 1;
+}
+\f
 
 /* Provide a prototype to silence -Wmissing-prototypes.  */
 extern initialize_file_ftype _initialize_utils;
diff --git a/gdb/utils.h b/gdb/utils.h
index b8e1aff..10d9c08 100644
--- a/gdb/utils.h
+++ b/gdb/utils.h
@@ -367,4 +367,8 @@ extern void dump_core (void);
 
 extern char *make_hex_string (const gdb_byte *data, size_t length);
 
+/* Uncompress file if compressed.  */
+
+int gdb_uncompress (const char *filename, char **uncompressed_filename);
+
 #endif /* UTILS_H */
-- 
1.8.1.4


[-- Attachment #3: 0002-GDB-Fix-leading-trailing-whitespace.patch --]
[-- Type: text/x-patch, Size: 21452 bytes --]

From 1cee9b02f6821921de761a33c7ee7e78b73f85b9 Mon Sep 17 00:00:00 2001
From: Michael Eager <eager@eagercon.com>
Date: Thu, 26 Mar 2015 17:18:02 -0700
Subject: [PATCH 2/2] GDB - Fix leading & trailing whitespace

* corelow.c: Replace leading whitespace with tabs, eliminate trailing blanks.
* exec.c: Ditto.
* symfile.c: Ditto.
* utils.c: Ditto
* utils.h: Ditto
---
 gdb/corelow.c | 10 +++++-----
 gdb/exec.c    | 22 ++++++++++-----------
 gdb/symfile.c | 62 +++++++++++++++++++++++++++++------------------------------
 gdb/utils.c   | 60 ++++++++++++++++++++++++++++-----------------------------
 gdb/utils.h   |  4 ++--
 5 files changed, 79 insertions(+), 79 deletions(-)

diff --git a/gdb/corelow.c b/gdb/corelow.c
index 8211595..ee5b872 100644
--- a/gdb/corelow.c
+++ b/gdb/corelow.c
@@ -198,7 +198,7 @@ core_close (struct target_ops *self)
 	exit_inferior_silent (pid);
 
       /* Clear out solib state while the bfd is still open.  See
-         comments in clear_solib in solib.c.  */
+	 comments in clear_solib in solib.c.  */
       clear_solib ();
 
       if (core_data)
@@ -330,8 +330,8 @@ core_open (const char *arg, int from_tty)
     {
       /* Do it after the err msg */
       /* FIXME: should be checking for errors from bfd_close (for one
-         thing, on error it does not free all the storage associated
-         with the bfd).  */
+	 thing, on error it does not free all the storage associated
+	 with the bfd).  */
       make_cleanup_bfd_unref (temp_bfd);
       error (_("\"%s\" is not a core dump: %s"),
 	     filename, bfd_errmsg (bfd_get_error ()));
@@ -954,7 +954,7 @@ core_read_description (struct target_ops *target)
     {
       const struct target_desc *result;
 
-      result = gdbarch_core_read_description (core_gdbarch, 
+      result = gdbarch_core_read_description (core_gdbarch,
 					      target, core_bfd);
       if (result != NULL)
 	return result;
@@ -1055,7 +1055,7 @@ init_core_ops (void)
   core_ops.to_magic = OPS_MAGIC;
 
   if (core_target)
-    internal_error (__FILE__, __LINE__, 
+    internal_error (__FILE__, __LINE__,
 		    _("init_core_ops: core target already exists (\"%s\")."),
 		    core_target->to_longname);
   core_target = &core_ops;
diff --git a/gdb/exec.c b/gdb/exec.c
index 4b84d65..b3c1459 100644
--- a/gdb/exec.c
+++ b/gdb/exec.c
@@ -1,4 +1,4 @@
-/* Work with executable files, for GDB. 
+/* Work with executable files, for GDB.
 
    Copyright (C) 1988-2015 Free Software Foundation, Inc.
 
@@ -171,7 +171,7 @@ exec_file_attach (const char *filename, int from_tty)
   if (!filename)
     {
       if (from_tty)
-        printf_unfiltered (_("No executable file now.\n"));
+	printf_unfiltered (_("No executable file now.\n"));
 
       set_gdbarch_from_file (NULL);
     }
@@ -283,7 +283,7 @@ exec_file_attach (const char *filename, int from_tty)
    Note that we have to explicitly ignore additional args, since we can
    be called from file_command(), which also calls symbol_file_command()
    which can take multiple args.
-   
+
    If ARGS is NULL, we just want to close the exec file.  */
 
 static void
@@ -302,16 +302,16 @@ exec_file_command (char *args, int from_tty)
       struct cleanup *cleanups;
 
       /* Scan through the args and pick up the first non option arg
-         as the filename.  */
+	 as the filename.  */
 
       argv = gdb_buildargv (args);
       cleanups = make_cleanup_freeargv (argv);
 
       for (; (*argv != NULL) && (**argv == '-'); argv++)
-        {;
-        }
+	{;
+	}
       if (*argv == NULL)
-        error (_("No executable file name was specified"));
+	error (_("No executable file name was specified"));
 
       filename = tilde_expand (*argv);
       make_cleanup (xfree, filename);
@@ -716,7 +716,7 @@ section_table_xfer_memory_partial (gdb_byte *readbuf, const gdb_byte *writebuf,
       if (section_name && strcmp (section_name, asect->name) != 0)
 	continue;		/* not the section we need.  */
       if (memaddr >= p->addr)
-        {
+	{
 	  if (memend <= p->endaddr)
 	    {
 	      /* Entire transfer is within this section.  */
@@ -762,7 +762,7 @@ section_table_xfer_memory_partial (gdb_byte *readbuf, const gdb_byte *writebuf,
 	      else
 		return TARGET_XFER_EOF;
 	    }
-        }
+	}
     }
 
   return TARGET_XFER_EOF;		/* We can't help.  */
@@ -832,8 +832,8 @@ print_section_info (struct target_section_table *t, bfd *abfd)
 	warning (_("Cannot find section for the entry point of %s."),
 		 bfd_get_filename (abfd));
 
-      entry_point = gdbarch_addr_bits_remove (gdbarch, 
-					      bfd_get_start_address (abfd) 
+      entry_point = gdbarch_addr_bits_remove (gdbarch,
+					      bfd_get_start_address (abfd)
 						+ displacement);
       printf_filtered (_("\tEntry point: %s\n"),
 		       paddress (gdbarch, entry_point));
diff --git a/gdb/symfile.c b/gdb/symfile.c
index e24517f..37ea11c 100644
--- a/gdb/symfile.c
+++ b/gdb/symfile.c
@@ -265,7 +265,7 @@ alloc_section_addr_info (size_t num_sections)
 
 extern struct section_addr_info *
 build_section_addr_info_from_section_table (const struct target_section *start,
-                                            const struct target_section *end)
+					    const struct target_section *end)
 {
   struct section_addr_info *sap;
   const struct target_section *stp;
@@ -507,7 +507,7 @@ relative_addr_info_to_section_offsets (struct section_offsets *section_offsets,
 
       /* Record all sections in offsets.  */
       /* The section_offsets in the objfile are here filled in using
-         the BFD index.  */
+	 the BFD index.  */
       section_offsets->offsets[osp->sectindex] = osp->addr;
     }
 }
@@ -918,7 +918,7 @@ init_entry_point_info (struct objfile *objfile)
   if (bfd_get_file_flags (objfile->obfd) & EXEC_P)
     {
       /* Executable file -- record its entry point so we'll recognize
-         the startup file because it contains the entry point.  */
+	 the startup file because it contains the entry point.  */
       ei->entry_point = bfd_get_start_address (objfile->obfd);
       ei->entry_point_p = 1;
     }
@@ -1017,7 +1017,7 @@ syms_from_objfile_1 (struct objfile *objfile,
 
       objfile->num_sections = num_sections;
       objfile->section_offsets
-        = obstack_alloc (&objfile->objfile_obstack, size);
+	= obstack_alloc (&objfile->objfile_obstack, size);
       memset (objfile->section_offsets, 0, size);
       return;
     }
@@ -1039,7 +1039,7 @@ syms_from_objfile_1 (struct objfile *objfile,
   if (mainline)
     {
       /* We will modify the main symbol table, make sure that all its users
-         will be cleaned up if an error occurs during symbol reading.  */
+	 will be cleaned up if an error occurs during symbol reading.  */
       make_cleanup (clear_symtab_users_cleanup, 0 /*ignore*/);
 
       /* Since no error yet, throw away the old symbol table.  */
@@ -1051,9 +1051,9 @@ syms_from_objfile_1 (struct objfile *objfile,
 	}
 
       /* Currently we keep symbols from the add-symbol-file command.
-         If the user wants to get rid of them, they should do "symbol-file"
-         without arguments first.  Not sure this is the best behavior
-         (PR 2207).  */
+	 If the user wants to get rid of them, they should do "symbol-file"
+	 without arguments first.  Not sure this is the best behavior
+	 (PR 2207).  */
 
       (*objfile->sf->sym_new_init) (objfile);
     }
@@ -1279,8 +1279,8 @@ symbol_file_add_separate (bfd *bfd, const char *name, int symfile_flags,
 
 struct objfile *
 symbol_file_add_from_bfd (bfd *abfd, const char *name, int add_flags,
-                          struct section_addr_info *addrs,
-                          int flags, struct objfile *parent)
+			  struct section_addr_info *addrs,
+			  int flags, struct objfile *parent)
 {
   return symbol_file_add_with_addrs (abfd, name, add_flags, addrs, flags,
 				     parent);
@@ -2137,9 +2137,9 @@ generic_load (const char *args, int from_tty)
       cbdata.load_offset = strtoulst (argv[1], &endptr, 0);
 
       /* If the last word was not a valid number then
-         treat it as a file name with spaces in.  */
+	 treat it as a file name with spaces in.  */
       if (argv[1] == endptr)
-        error (_("Invalid download offset:%s."), argv[1]);
+	error (_("Invalid download offset:%s."), argv[1]);
 
       if (argv[2] != NULL)
 	error (_("Too many parameters."));
@@ -2392,7 +2392,7 @@ add_symbol_file_command (char *args, int from_tty)
       addr = parse_and_eval_address (val);
 
       /* Here we store the section offsets in the order they were
-         entered on the command line.  */
+	 entered on the command line.  */
       section_addrs->other[sec_num].name = sec;
       section_addrs->other[sec_num].addr = addr;
       printf_unfiltered ("\t%s_addr = %s\n", sec,
@@ -2751,7 +2751,7 @@ reread_symbols (void)
 	observer_notify_new_objfile (objfile);
 
       /* At least one objfile has changed, so we can consider that
-         the executable we're debugging has changed too.  */
+	 the executable we're debugging has changed too.  */
       observer_notify_executable_changed ();
     }
 
@@ -3181,7 +3181,7 @@ section_is_mapped (struct obj_section *osect)
       return 0;			/* overlay debugging off */
     case ovly_auto:		/* overlay debugging automatic */
       /* Unles there is a gdbarch_overlay_update function,
-         there's really nothing useful to do here (can't really go auto).  */
+	 there's really nothing useful to do here (can't really go auto).  */
       gdbarch = get_objfile_arch (osect->objfile);
       if (gdbarch_overlay_update_p (gdbarch))
 	{
@@ -3434,7 +3434,7 @@ map_overlay_command (char *args, int from_tty)
       sec->ovly_mapped = 1;
 
       /* Next, make a pass and unmap any sections that are
-         overlapped by this new section: */
+	 overlapped by this new section: */
       ALL_OBJSECTIONS (objfile2, sec2)
 	if (sec2->ovly_mapped && sec != sec2 && sections_overlap (sec, sec2))
 	{
@@ -3630,8 +3630,8 @@ simple_read_overlay_table (void)
   if (! novlys_msym.minsym)
     {
       error (_("Error reading inferior's overlay table: "
-             "couldn't find `_novlys' variable\n"
-             "in inferior.  Use `overlay manual' mode."));
+	     "couldn't find `_novlys' variable\n"
+	     "in inferior.  Use `overlay manual' mode."));
       return 0;
     }
 
@@ -3639,8 +3639,8 @@ simple_read_overlay_table (void)
   if (! ovly_table_msym.minsym)
     {
       error (_("Error reading inferior's overlay table: couldn't find "
-             "`_ovly_table' array\n"
-             "in inferior.  Use `overlay manual' mode."));
+	     "`_ovly_table' array\n"
+	     "in inferior.  Use `overlay manual' mode."));
       return 0;
     }
 
@@ -3654,8 +3654,8 @@ simple_read_overlay_table (void)
     = (void *) xmalloc (cache_novlys * sizeof (*cache_ovly_table));
   cache_ovly_table_base = BMSYMBOL_VALUE_ADDRESS (ovly_table_msym);
   read_target_long_array (cache_ovly_table_base,
-                          (unsigned int *) cache_ovly_table,
-                          cache_novlys * 4, word_size, byte_order);
+			  (unsigned int *) cache_ovly_table,
+			  cache_novlys * 4, word_size, byte_order);
 
   return 1;			/* SUCCESS */
 }
@@ -3779,7 +3779,7 @@ symfile_dummy_outputs (bfd *abfd, asection *sectp, void *dummy)
 
 bfd_byte *
 default_symfile_relocate (struct objfile *objfile, asection *sectp,
-                          bfd_byte *buf)
+			  bfd_byte *buf)
 {
   /* Use sectp->owner instead of objfile->obfd.  sectp may point to a
      DWO file.  */
@@ -3813,7 +3813,7 @@ default_symfile_relocate (struct objfile *objfile, asection *sectp,
 
 bfd_byte *
 symfile_relocate_debug_section (struct objfile *objfile,
-                                asection *sectp, bfd_byte *buf)
+				asection *sectp, bfd_byte *buf)
 {
   gdb_assert (objfile->sf->sym_relocate);
 
@@ -3881,17 +3881,17 @@ symfile_map_offsets_to_segments (bfd *abfd,
       gdb_assert (0 <= which && which <= data->num_segments);
 
       /* Don't bother computing offsets for sections that aren't
-         loaded as part of any segment.  */
+	 loaded as part of any segment.  */
       if (! which)
-        continue;
+	continue;
 
       /* Use the last SEGMENT_BASES entry as the address of any extra
-         segments mentioned in DATA->segment_info.  */
+	 segments mentioned in DATA->segment_info.  */
       if (which > num_segment_bases)
-        which = num_segment_bases;
+	which = num_segment_bases;
 
       offsets->offsets[i] = (segment_bases[which - 1]
-                             - data->segment_bases[which - 1]);
+			     - data->segment_bases[which - 1]);
     }
 
   return 1;
@@ -4087,9 +4087,9 @@ Set printing of symbol loading messages."), _("\
 Show printing of symbol loading messages."), _("\
 off   == turn all messages off\n\
 brief == print messages for the executable,\n\
-         and brief messages for shared libraries\n\
+	 and brief messages for shared libraries\n\
 full  == print messages for the executable,\n\
-         and messages for each shared library."),
+	 and messages for each shared library."),
 			NULL,
 			NULL,
 			&setprintlist, &showprintlist);
diff --git a/gdb/utils.c b/gdb/utils.c
index e9f86e5..1bc4fa7 100644
--- a/gdb/utils.c
+++ b/gdb/utils.c
@@ -696,13 +696,13 @@ internal_vproblem (struct internal_problem *problem,
 	abort_with_message (msg);
       default:
 	dejavu = 3;
-        /* Newer GLIBC versions put the warn_unused_result attribute
-           on write, but this is one of those rare cases where
-           ignoring the return value is correct.  Casting to (void)
-           does not fix this problem.  This is the solution suggested
-           at http://gcc.gnu.org/bugzilla/show_bug.cgi?id=25509.  */
+	/* Newer GLIBC versions put the warn_unused_result attribute
+	   on write, but this is one of those rare cases where
+	   ignoring the return value is correct.  Casting to (void)
+	   does not fix this problem.  This is the solution suggested
+	   at http://gcc.gnu.org/bugzilla/show_bug.cgi?id=25509.  */
 	if (write (STDERR_FILENO, msg, sizeof (msg)) != sizeof (msg))
-          abort (); /* NOTE: GDB has only three calls to abort().  */
+	  abort (); /* NOTE: GDB has only three calls to abort().  */
 	exit (1);
       }
   }
@@ -751,7 +751,7 @@ internal_vproblem (struct internal_problem *problem,
       if (!confirm || !filtered_printing_initialized ())
 	quit_p = 1;
       else
-        quit_p = query (_("%s\nQuit this debugging session? "), reason);
+	quit_p = query (_("%s\nQuit this debugging session? "), reason);
     }
   else if (problem->should_quit == internal_problem_yes)
     quit_p = 1;
@@ -1058,7 +1058,7 @@ quit (void)
 #else
   if (job_control
       /* If there is no terminal switching for this target, then we can't
-         possibly get screwed by the lack of job control.  */
+	 possibly get screwed by the lack of job control.  */
       || !target_supports_terminal_ours ())
     throw_quit ("Quit");
   else
@@ -1299,15 +1299,15 @@ defaulted_query (const char *ctlstr, const char defchar, va_list args)
       if (answer >= 'a')
 	answer -= 040;
       /* Check answer.  For the non-default, the user must specify
-         the non-default explicitly.  */
+	 the non-default explicitly.  */
       if (answer == not_def_answer)
 	{
 	  retval = !def_value;
 	  break;
 	}
       /* Otherwise, if a default was specified, the user may either
-         specify the required input or have it default by entering
-         nothing.  */
+	 specify the required input or have it default by entering
+	 nothing.  */
       if (answer == def_answer
 	  || (defchar != '\0' && answer == '\0'))
 	{
@@ -1323,7 +1323,7 @@ defaulted_query (const char *ctlstr, const char defchar, va_list args)
   gettimeofday (&prompt_ended, NULL);
   timeval_sub (&prompt_delta, &prompt_ended, &prompt_started);
   timeval_add (&prompt_for_continue_wait_time,
-               &prompt_for_continue_wait_time, &prompt_delta);
+	       &prompt_for_continue_wait_time, &prompt_delta);
 
   xfree (prompt);
   if (annotation_level > 1)
@@ -1686,8 +1686,8 @@ init_page_info (void)
       chars_per_line = cols;
 
       /* Readline should have fetched the termcap entry for us.
-         Only try to use tgetnum function if rl_get_screen_size
-         did not return a useful value. */
+	 Only try to use tgetnum function if rl_get_screen_size
+	 did not return a useful value. */
       if (((rows <= 0) && (tgetnum ("li") < 0))
 	/* Also disable paging if inside EMACS.  */
 	  || getenv ("EMACS"))
@@ -1745,7 +1745,7 @@ struct cleanup *
 set_batch_flag_and_make_cleanup_restore_page_info (void)
 {
   struct cleanup *back_to = make_cleanup_restore_page_info ();
-  
+
   make_cleanup_restore_integer (&batch_flag);
   batch_flag = 1;
   init_page_info ();
@@ -1826,7 +1826,7 @@ prompt_for_continue (void)
     strcat (cont_prompt, "\n\032\032prompt-for-continue\n");
 
   /* We must do this *before* we call gdb_readline, else it will eventually
-     call us -- thinking that we're trying to print beyond the end of the 
+     call us -- thinking that we're trying to print beyond the end of the
      screen.  */
   reinitialize_more_filter ();
 
@@ -1852,7 +1852,7 @@ prompt_for_continue (void)
   gettimeofday (&prompt_ended, NULL);
   timeval_sub (&prompt_delta, &prompt_ended, &prompt_started);
   timeval_add (&prompt_for_continue_wait_time,
-               &prompt_for_continue_wait_time, &prompt_delta);
+	       &prompt_for_continue_wait_time, &prompt_delta);
 
   if (annotation_level > 1)
     printf_unfiltered (("\n\032\032post-prompt-for-continue\n"));
@@ -1960,11 +1960,11 @@ wrap_here (char *indent)
     }
 }
 
-/* Print input string to gdb_stdout, filtered, with wrap, 
+/* Print input string to gdb_stdout, filtered, with wrap,
    arranging strings in columns of n chars.  String can be
-   right or left justified in the column.  Never prints 
+   right or left justified in the column.  Never prints
    trailing spaces.  String should never be longer than
-   width.  FIXME: this could be useful for the EXAMINE 
+   width.  FIXME: this could be useful for the EXAMINE
    command, which currently doesn't tabulate very well.  */
 
 void
@@ -2075,8 +2075,8 @@ fputs_maybe_filtered (const char *linebuffer, struct ui_file *stream,
 	      else
 		fputc_unfiltered ('\t', stream);
 	      /* Shifting right by 3 produces the number of tab stops
-	         we have already passed, and then adding one and
-	         shifting left 3 advances to the next tab stop.  */
+		 we have already passed, and then adding one and
+		 shifting left 3 advances to the next tab stop.  */
 	      chars_printed = ((chars_printed >> 3) + 1) << 3;
 	      lineptr++;
 	    }
@@ -2097,8 +2097,8 @@ fputs_maybe_filtered (const char *linebuffer, struct ui_file *stream,
 	      chars_printed = 0;
 	      lines_printed++;
 	      /* If we aren't actually wrapping, don't output newline --
-	         if chars_per_line is right, we probably just overflowed
-	         anyway; if it's wrong, let us keep going.  */
+		 if chars_per_line is right, we probably just overflowed
+		 anyway; if it's wrong, let us keep going.  */
 	      if (wrap_column)
 		fputc_unfiltered ('\n', stream);
 
@@ -2638,7 +2638,7 @@ strcmp_iw_ordered (const char *string1, const char *string2)
 
       if (case_pass == case_sensitive_on)
 	return 0;
-      
+
       /* Otherwise the strings were equal in case insensitive way, make
 	 a more fine grained comparison in a case sensitive way.  */
 
@@ -2918,7 +2918,7 @@ gdb_realpath_keepfile (const char *filename)
   char *real_path;
   char *result;
 
-  /* Extract the basename of filename, and return immediately 
+  /* Extract the basename of filename, and return immediately
      a copy of filename if it does not contain any directory prefix.  */
   if (base_name == filename)
     return xstrdup (filename);
@@ -3196,7 +3196,7 @@ gdb_bfd_errmsg (bfd_error_type error_tag, char **matching)
     return bfd_errmsg (error_tag);
 
   ret_len = strlen (bfd_errmsg (error_tag)) + strlen (AMBIGUOUS_MESS1)
-            + strlen (AMBIGUOUS_MESS2);
+	    + strlen (AMBIGUOUS_MESS2);
   for (p = matching; *p; p++)
     ret_len += strlen (*p) + 1;
   ret = xmalloc (ret_len + 1);
@@ -3302,9 +3302,9 @@ producer_is_gcc (const char *producer, int *major, int *minor)
       */
       cs = &producer[strlen ("GNU ")];
       while (*cs && !isspace (*cs))
-        cs++;
+	cs++;
       if (*cs && isspace (*cs))
-        cs++;
+	cs++;
       if (sscanf (cs, "%d.%d", major, minor) == 2)
 	return 1;
     }
@@ -3356,7 +3356,7 @@ substitute_path_component (char **stringp, const char *from, const char *to)
 
       if ((s == string || IS_DIR_SEPARATOR (s[-1])
 	   || s[-1] == DIRNAME_SEPARATOR)
-          && (s[from_len] == '\0' || IS_DIR_SEPARATOR (s[from_len])
+	  && (s[from_len] == '\0' || IS_DIR_SEPARATOR (s[from_len])
 	      || s[from_len] == DIRNAME_SEPARATOR))
 	{
 	  char *string_new;
diff --git a/gdb/utils.h b/gdb/utils.h
index 10d9c08..33ed914 100644
--- a/gdb/utils.h
+++ b/gdb/utils.h
@@ -77,8 +77,8 @@ extern struct cleanup *
   make_cleanup_ui_out_redirect_pop (struct ui_out *uiout);
 
 struct section_addr_info;
-extern struct cleanup *(make_cleanup_free_section_addr_info 
-                        (struct section_addr_info *));
+extern struct cleanup *(make_cleanup_free_section_addr_info
+			(struct section_addr_info *));
 
 extern struct cleanup *make_cleanup_close (int fd);
 
-- 
1.8.1.4

Re: [PATCH] Support gzip compressed exec and core files in gdb

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 246 bytes --]

On 27 Mar 2015 08:26, Michael Eager wrote:
> I'm not fond of whitespace patches, but there were many whitespace errors
> in these files unrelated to my patch.  I've attached a second patch to clean
> this up.

looks obvious enough to me :)
-mike

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [PATCH] Support gzip compressed exec and core files in gdb

[Michael Eager]

On 03/27/15 08:26, Michael Eager wrote:
> Removed binutils@sourceware.org.
>
> On 03/20/15 15:16, Mike Frysinger wrote:
>> On 18 Mar 2015 17:58, Michael Eager wrote:
>>> --- a/gdb/utils.c
>>> +++ b/gdb/utils.c
>>>
>>> +#define COMPRESS_BUF_SIZE (1024*1024)
>>
>> space around the *
>>
>> would be nice to have a comment noting where the 1MiB comes from
>
> I added a comment, but the buffer size is arbitrary.
>
>>> +  /* Create temporary file name for uncompressed file.  */
>>> +  if (!(tmpdir = getenv ("TMPDIR")))
>>> +    tmpdir = "/tmp";
>>> +
>>> +  if (!asprintf (&template, "%s/%s-XXXXXX", tmpdir, basename (filename)))
>>> +    return 0;
>>> +
>>> +  decomp_fd = mkstemp (template);
>>
>> ignoring that assigningments in if statements are frowned upon, looks like you
>> can just use make_temp_file(NULL) from libiberty
>
> Fixed assignment.  make_temp_file() generates an anonymous file; I want to
> keep the user-specified file name as part of the uncompressed file name.
> (I could update libiberty to add an alternate to make_temp_file() which
> takes a basename, I guess.)  I replaced getenv("TMPDIR") with choose_tmpdir()
> from libiberty.
>
>> you would also leak the fopen() handle here.  probably want to go through this
>> code again looking for such leaks.  and maybe try breaking the func up into more
>> utility related chunks when possible.
>
> I refactored the gdb_uncompress() function and broke it into 3 functions.
>
>> you've got more missing tabs in this file.  i'm going to stop checking.
>
> I cleaned up the white space.  I hope I got all the tabs this time.
>
> gdb/ChangeLog:
>    * utils.c (struct compressed_file_cache_search, eq_compressed_file,
>    is_gzip, decompress_gzip, do_compressed_cleanup, identify_compression,
>    uncompress_to_temporary, gdb_uncompress): New.
>    * utils.h (gdb_uncompress): Declare.
>    * corelow.c (core_open): Uncompress core file.
>    * exec.c (exec_file_attach): Uncompress exec file.
>    * symfile.c (symfile_bfd_open): Uncompress sym (exec) file.
>    * NEWS: Mention new functionality.
>
> gdb/doc:
>    * gdb.texinfo (Files): Mention gzipped exec and core files.
>
>
> I'm not fond of whitespace patches, but there were many whitespace errors
> in these files unrelated to my patch.  I've attached a second patch to clean
> this up.
>
> gdb/ChangeLog:
> * corelow.c: Replace leading whitespace with tabs, eliminate trailing blanks.
> * exec.c: Ditto.
> * symfile.c: Ditto.
> * utils.c: Ditto
> * utils.h: Ditto

Pedro, Doug -- Can you approve this patch?


-- 
Michael Eager	 eager@eagercon.com
1960 Park Blvd., Palo Alto, CA 94306  650-325-8077

Re: [PATCH] Support gzip compressed exec and core files in gdb

[Pedro Alves]

On 03/28/2015 04:56 PM, Michael Eager wrote:
>> > I'm not fond of whitespace patches, but there were many whitespace errors
>> > in these files unrelated to my patch.  I've attached a second patch to clean
>> > this up.
>> >
>> > gdb/ChangeLog:
>> > * corelow.c: Replace leading whitespace with tabs, eliminate trailing blanks.
>> > * exec.c: Ditto.
>> > * symfile.c: Ditto.
>> > * utils.c: Ditto
>> > * utils.h: Ditto

OK.

Thanks,
Pedro Alves

Re: [PATCH] Support gzip compressed exec and core files in gdb

[Michael Eager]

On 04/01/15 05:41, Pedro Alves wrote:
> On 03/28/2015 04:56 PM, Michael Eager wrote:
>>>> I'm not fond of whitespace patches, but there were many whitespace errors
>>>> in these files unrelated to my patch.  I've attached a second patch to clean
>>>> this up.
>>>>
>>>> gdb/ChangeLog:
>>>> * corelow.c: Replace leading whitespace with tabs, eliminate trailing blanks.
>>>> * exec.c: Ditto.
>>>> * symfile.c: Ditto.
>>>> * utils.c: Ditto
>>>> * utils.h: Ditto
>
> OK.

Is this OK for both the gzip support and the trivial whitespace patch?


-- 
Michael Eager	 eager@eagercon.com
1960 Park Blvd., Palo Alto, CA 94306  650-325-8077

Re: [PATCH] Support gzip compressed exec and core files in gdb

[Pedro Alves]

On 04/01/2015 07:52 PM, Michael Eager wrote:
> On 04/01/15 05:41, Pedro Alves wrote:
>> On 03/28/2015 04:56 PM, Michael Eager wrote:
>>>>> I'm not fond of whitespace patches, but there were many whitespace errors
>>>>> in these files unrelated to my patch.  I've attached a second patch to clean
>>>>> this up.
>>>>>
>>>>> gdb/ChangeLog:
>>>>> * corelow.c: Replace leading whitespace with tabs, eliminate trailing blanks.
>>>>> * exec.c: Ditto.
>>>>> * symfile.c: Ditto.
>>>>> * utils.c: Ditto
>>>>> * utils.h: Ditto
>>
>> OK.
> 
> Is this OK for both the gzip support and the trivial whitespace patch?

Trivial whitespace patch.

Thanks,
Pedro Alves

Re: [PATCH] Support gzip compressed exec and core files in gdb

[Michael Eager]

[-- Attachment #1: Type: text/plain, Size: 1075 bytes --]

On 03/10/2015 04:01 PM, Michael Eager wrote:
> Add support to automatically unzip compressed executable and core files.
> Files will be uncompressed into temporary directory (/tmp or $TMPDIR)
> and are deleted when GDB exits.  This should be transparent to users,
> except for disk space requirements.  The name of the uncompressed file is
> mentioned, but all references to the file in GDB messages is to the file
> which the user specified.

Attached is a revised patch.  I discovered a logic error after refactoring
the code.

gdb/ChangeLog:
   * utils.c (struct compressed_file_cache_search, eq_compressed_file,
   is_gzip, decompress_gzip, do_compressed_cleanup, identify_compression,
   uncompress_to_temporar, gdb_uncompress): New.
   * utils.h (gdb_uncompress): Declare.
   * corelow.c (core_open): Uncompress core file.
   * exec.c (exec_file_attach): Uncompress exec file.
   * symfile.c (symfile_bfd_open): Uncompress sym (exec) file.
   * NEWS: Mention new functionality.


-- 
Michael Eager	 eager@eagercon.com
1960 Park Blvd., Palo Alto, CA 94306  650-325-8077

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-GDB-support-compressed-exec-and-core-files.patch --]
[-- Type: text/x-patch; name="0001-GDB-support-compressed-exec-and-core-files.patch", Size: 12354 bytes --]

From 9e8ffc3d0a5472cc6e1c48ebf90641f79157fc8a Mon Sep 17 00:00:00 2001
From: Michael Eager <meager@cisco.com>
Date: Fri, 10 Apr 2015 11:55:02 -0700
Subject: [PATCH] GDB support compressed exec and core files.

Add support to automatically unzip compressed executable and core files.
Files will be uncompressed into temporary directory (/tmp or $TMPDIR)
and are deleted when GDB exits.  This should be transparent to users,
except for disk space requirements.  The name of the uncompressed file is
mentioned, but all references to the file in GDB messages is to the file
which the user specified.

gdb/ChangeLog:
  * utils.c (struct compressed_file_cache_search, eq_compressed_file,
  is_gzip, decompress_gzip, do_compressed_cleanup, identify_compression,
  uncompress_to_temporar, gdb_uncompress): New.
  * utils.h (gdb_uncompress): Declare.
  * corelow.c (core_open): Uncompress core file.
  * exec.c (exec_file_attach): Uncompress exec file.
  * symfile.c (symfile_bfd_open): Uncompress sym (exec) file.
  * NEWS: Mention new functionality.

gdb/doc:
  * gdb.texinfo (Files): Mention gzipped exec and core files.
---
 gdb/NEWS            |   3 +
 gdb/corelow.c       |  11 ++-
 gdb/doc/gdb.texinfo |   5 ++
 gdb/exec.c          |  14 +++-
 gdb/symfile.c       |  14 +++-
 gdb/utils.c         | 211 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 gdb/utils.h         |   4 +
 7 files changed, 259 insertions(+), 3 deletions(-)

diff --git a/gdb/NEWS b/gdb/NEWS
index 651401d..2c59236 100644
--- a/gdb/NEWS
+++ b/gdb/NEWS
@@ -139,6 +139,9 @@ qXfer:exec-file:read
 
 * Removed targets and native configurations
 
+* GDB will automatically uncompress executable and core files
+  which have been compressed using gzip.
+
 HP/PA running HP-UX           hppa*-*-hpux*
 Itanium running HP-UX         ia64-*-hpux*
 
diff --git a/gdb/corelow.c b/gdb/corelow.c
index 9218003..8211595 100644
--- a/gdb/corelow.c
+++ b/gdb/corelow.c
@@ -279,6 +279,7 @@ core_open (const char *arg, int from_tty)
   int scratch_chan;
   int flags;
   char *filename;
+  char *uncompressed_filename;
 
   target_preopen (from_tty);
   if (!arg)
@@ -310,12 +311,20 @@ core_open (const char *arg, int from_tty)
   if (scratch_chan < 0)
     perror_with_name (filename);
 
-  temp_bfd = gdb_bfd_fopen (filename, gnutarget, 
+  temp_bfd = gdb_bfd_fopen (filename, gnutarget,
 			    write_files ? FOPEN_RUB : FOPEN_RB,
 			    scratch_chan);
   if (temp_bfd == NULL)
     perror_with_name (filename);
 
+  if (!write_files && gdb_uncompress (filename, &uncompressed_filename))
+    {
+      close (scratch_chan);
+      scratch_chan = gdb_open_cloexec (uncompressed_filename, flags, 0);
+      temp_bfd = gdb_bfd_fopen (uncompressed_filename, gnutarget,
+				FOPEN_RB, scratch_chan);
+    }
+
   if (!bfd_check_format (temp_bfd, bfd_core)
       && !gdb_check_format (temp_bfd))
     {
diff --git a/gdb/doc/gdb.texinfo b/gdb/doc/gdb.texinfo
index 2ce2e57..a82a55b 100644
--- a/gdb/doc/gdb.texinfo
+++ b/gdb/doc/gdb.texinfo
@@ -17504,6 +17504,11 @@ via @code{gdbserver} (@pxref{Server, file, Using the @code{gdbserver}
 Program}).  In these situations the @value{GDBN} commands to specify
 new files are useful.
 
+Executable and core files may be compressed using @command{gzip}.  These
+files will be uncompressed into temporary files in the system-wide
+temporary directory (e.g., @file{/tmp} on some systems).  The files will
+have a unique name and will be deleted when @value{GDBN} terminates.
+
 @table @code
 @cindex executable file
 @kindex file
diff --git a/gdb/exec.c b/gdb/exec.c
index 4e079fd..2eba1bb 100644
--- a/gdb/exec.c
+++ b/gdb/exec.c
@@ -196,6 +196,7 @@ void
 exec_file_attach (const char *filename, int from_tty)
 {
   struct cleanup *cleanups;
+  char *uncompressed_filename = NULL;
 
   /* First, acquire a reference to the current exec_bfd.  We release
      this at the end of the function; but acquiring it now lets the
@@ -280,7 +281,18 @@ exec_file_attach (const char *filename, int from_tty)
 	exec_bfd = gdb_bfd_fopen (canonical_pathname, gnutarget,
 				  FOPEN_RUB, scratch_chan);
       else
-	exec_bfd = gdb_bfd_open (canonical_pathname, gnutarget, scratch_chan);
+	{
+	  if (!gdb_uncompress (canonical_pathname, &uncompressed_filename))
+	    exec_bfd = gdb_bfd_open (canonical_pathname, gnutarget, scratch_chan);
+	  else
+	    {
+	      close (scratch_chan);
+	      scratch_chan = openp ("", 0, uncompressed_filename,
+				    O_RDONLY | O_BINARY, &scratch_pathname);
+
+	      exec_bfd = gdb_bfd_open (uncompressed_filename, gnutarget, scratch_chan);
+	    }
+	}
 
       if (!exec_bfd)
 	{
diff --git a/gdb/symfile.c b/gdb/symfile.c
index 0c35ffa..43c0522 100644
--- a/gdb/symfile.c
+++ b/gdb/symfile.c
@@ -56,6 +56,7 @@
 #include "stack.h"
 #include "gdb_bfd.h"
 #include "cli/cli-utils.h"
+#include "utils.h"
 
 #include <sys/types.h>
 #include <fcntl.h>
@@ -1723,6 +1724,8 @@ symfile_bfd_open (const char *name)
   bfd *sym_bfd;
   int desc = -1;
   struct cleanup *back_to = make_cleanup (null_cleanup, 0);
+  char *absolute_name;
+  char *uncompressed_filename;
 
   if (!is_target_filename (name))
     {
@@ -1756,7 +1759,16 @@ symfile_bfd_open (const char *name)
       name = absolute_name;
     }
 
-  sym_bfd = gdb_bfd_open (name, gnutarget, desc);
+  if (!gdb_uncompress (name, &uncompressed_filename))
+    sym_bfd = gdb_bfd_open (name, gnutarget, desc);
+  else
+    {
+      close (desc);
+      desc = openp ("", 0, uncompressed_filename, O_RDONLY | O_BINARY, &absolute_name);
+
+      sym_bfd = gdb_bfd_open (uncompressed_filename, gnutarget, desc);
+    }
+
   if (!sym_bfd)
     error (_("`%s': can't open to read symbols: %s."), name,
 	   bfd_errmsg (bfd_get_error ()));
diff --git a/gdb/utils.c b/gdb/utils.c
index aaaf9c5..4160c54 100644
--- a/gdb/utils.c
+++ b/gdb/utils.c
@@ -29,6 +29,8 @@
 #include <sys/resource.h>
 #endif /* HAVE_SYS_RESOURCE_H */
 
+#include <zlib.h>
+
 #ifdef TUI
 #include "tui/tui.h"		/* For tui_get_command_dimension.   */
 #endif
@@ -126,6 +128,10 @@ int immediate_quit;
    as octal escapes.  Zero means just print the value (e.g. it's an
    international character, and the terminal or window can cope.)  */
 
+/* Type of file compression.  */
+
+enum file_compression_t {NONE, GZIP, BZIP2};
+
 int sevenbit_strings = 0;
 static void
 show_sevenbit_strings (struct ui_file *file, int from_tty,
@@ -3479,6 +3485,211 @@ gdb_filename_fnmatch (const char *pattern, const char *string, int flags)
 
   return fnmatch (pattern, string, flags);
 }
+\f
+/* Hash table of compressed files.  */
+
+static htab_t compressed_file_cache;
+
+struct compressed_file_cache_search
+{
+  char *filename;
+  char *uncompressed_filename;
+  time_t mtime;
+};
+
+static int
+eq_compressed_file (const void *a, const void *b)
+{
+  const struct compressed_file_cache_search *entry = a;
+  const struct compressed_file_cache_search *search = b;
+
+  return (strcmp (entry->filename, search->filename) == 0);
+}
+
+/* Test if file is compressed with gzip.  */
+
+static inline int
+is_gzip (unsigned char *buf)
+{
+  return (buf[0] == 037 && buf[1] == 0213);	/* From /usr/share/magic.  */
+}
+
+/* Alloc 1Mb buffer to uncompress data.  */
+#define COMPRESS_BUF_SIZE (1024 * 1024)
+static int
+decompress_gzip (const char *filename, FILE *tmp)
+{
+  char *buf = xmalloc (COMPRESS_BUF_SIZE);
+  gzFile compressed = gzopen (filename, "r");
+  int count, res;
+
+  if (buf == NULL || compressed == NULL)
+    {
+      fprintf_filtered (gdb_stderr, _("error copying gzip file\n"));
+      free (buf);
+      return 0;
+    }
+
+  while ((count = gzread (compressed, buf, COMPRESS_BUF_SIZE)))
+    {
+      res = fwrite (buf, 1, count, tmp);
+      if (res != count)
+	{
+	  fprintf_filtered (gdb_stderr, _("error decompressing gzip file\n"));
+	  free (buf);
+	  return 0;
+	}
+    }
+
+  gzclose (compressed);
+  free (buf);
+  return 1;
+}
+
+/* Delete uncompressed temp file when terminating.  */
+static void
+do_compressed_cleanup (void *filename)
+{
+  unlink (filename);
+  xfree (filename);
+}
+
+/* Identify type of file compression used.  */
+static enum file_compression_t
+identify_compression (const char *filename)
+{
+  FILE *handle;
+  enum file_compression_t file_compression = NONE;
+  unsigned char buffer[1024];
+  size_t count;
+
+  handle = fopen (filename, FOPEN_RB);
+  if (handle == NULL)
+    return 0;
+
+  count = fread (buffer, 1, sizeof buffer, handle);
+  if (count > 0)
+    {
+      if (is_gzip (buffer))
+	file_compression = GZIP;
+    }
+
+  fclose (handle);
+
+  return file_compression;
+}
+
+/* Uncompress into temporary file.  Return temp file name.  */
+static char *
+uncompress_to_temporary (const char *filename)
+{
+  enum file_compression_t file_compression;
+  char *template;
+  int ret = 0;
+  int decomp_fd;
+  FILE *decomp_file;
+
+  file_compression = identify_compression (filename);
+  if (file_compression != GZIP)
+    /* Only gzip supported at this time.  */
+    return NULL;
+
+  /* Create temporary file name for uncompressed file.  */
+  if (!asprintf (&template, "%s%s-XXXXXX", choose_tmpdir (),
+		 basename (filename)))
+    return NULL;
+
+  decomp_fd = mkstemp (template);
+  if (decomp_fd == -1)
+    return NULL;
+
+  decomp_file = fdopen (decomp_fd, FOPEN_WUB);
+
+  if (file_compression == GZIP)
+    {
+      printf (_("Decompressing %s to %s\n"), filename, template);
+      ret = decompress_gzip (filename, decomp_file);
+    }
+
+  if (!ret)
+    {
+      xfree (template);
+      template = NULL;
+    }
+
+  fclose (decomp_file);
+  return template;
+}
+
+/* If file is compressed, uncompress it into a temporary.  */
+int
+gdb_uncompress (const char *filename, char **uncompressed_filename)
+{
+  struct compressed_file_cache_search search, *found;
+  struct stat st;
+  hashval_t hash;
+  void **slot;
+  char *temp_filename;
+
+  if (compressed_file_cache == NULL)
+    compressed_file_cache = htab_create_alloc (1, htab_hash_string,
+					       eq_compressed_file,
+					       NULL, xcalloc, xfree);
+  if (stat (filename, &st) < 0)
+    return 0;
+
+  search.filename = (char *) filename;
+  search.uncompressed_filename = NULL;
+
+  hash = htab_hash_string (filename);
+  found = htab_find_with_hash (compressed_file_cache, &search, hash);
+
+  if (found)
+    {
+      /* We previously uncompressed the file.  */
+      if (found->mtime == st.st_mtime)
+	{
+	  /* Return file if compressed file not changed.  */
+	  *uncompressed_filename = found->uncompressed_filename;
+	  return 1;
+	}
+      else
+	{
+	  /* Delete old uncompressed file.  */
+	  unlink (found->uncompressed_filename);
+	  xfree (found->filename);
+	  xfree (found->uncompressed_filename);
+	}
+    }
+
+  temp_filename = uncompress_to_temporary (filename);
+
+  if (!temp_filename)
+    /* Not uncompressed.  */
+    return 0;
+
+  if (!found)
+    {
+      slot = htab_find_slot_with_hash (compressed_file_cache,
+				       &search, hash, INSERT);
+      gdb_assert (slot && !*slot);
+      found = xmalloc (sizeof (struct compressed_file_cache_search));
+      *slot = found;
+    }
+
+  found->filename = strdup (filename);
+  found->mtime = st.st_mtime;
+  found->uncompressed_filename = temp_filename;
+
+  /* Tell caller name of uncompressed file.  */
+  *uncompressed_filename = temp_filename;
+
+  /* Schedule delete of temp file when gdb ends.  */
+  make_final_cleanup (do_compressed_cleanup, xstrdup (temp_filename));
+
+  return 1;
+}
+\f
 
 /* Provide a prototype to silence -Wmissing-prototypes.  */
 extern initialize_file_ftype _initialize_utils;
diff --git a/gdb/utils.h b/gdb/utils.h
index cae1ac0..883e4c6 100644
--- a/gdb/utils.h
+++ b/gdb/utils.h
@@ -371,4 +371,8 @@ extern void dump_core (void);
 
 extern char *make_hex_string (const gdb_byte *data, size_t length);
 
+/* Uncompress file if compressed.  */
+
+int gdb_uncompress (const char *filename, char **uncompressed_filename);
+
 #endif /* UTILS_H */
-- 
2.2.1

Re: [PATCH] Support gzip compressed exec and core files in gdb

[Eli Zaretskii]

> Date: Fri, 01 May 2015 07:40:26 -0700
> From: Michael Eager <eager@eagerm.com>
> 
> On 03/10/2015 04:01 PM, Michael Eager wrote:
> > Add support to automatically unzip compressed executable and core files.
> > Files will be uncompressed into temporary directory (/tmp or $TMPDIR)
> > and are deleted when GDB exits.  This should be transparent to users,
> > except for disk space requirements.  The name of the uncompressed file is
> > mentioned, but all references to the file in GDB messages is to the file
> > which the user specified.
> 
> Attached is a revised patch.  I discovered a logic error after refactoring
> the code.

OK for the documentation parts.

> +  decomp_fd = mkstemp (template);

Do we have mkstemp on all supported platforms?

Thanks.

Re: [PATCH] Support gzip compressed exec and core files in gdb

[Michael Eager]

On 05/01/2015 10:53 AM, Eli Zaretskii wrote:
>> Date: Fri, 01 May 2015 07:40:26 -0700
>> From: Michael Eager <eager@eagerm.com>
>>
>> On 03/10/2015 04:01 PM, Michael Eager wrote:
>>> Add support to automatically unzip compressed executable and core files.
>>> Files will be uncompressed into temporary directory (/tmp or $TMPDIR)
>>> and are deleted when GDB exits.  This should be transparent to users,
>>> except for disk space requirements.  The name of the uncompressed file is
>>> mentioned, but all references to the file in GDB messages is to the file
>>> which the user specified.
>>
>> Attached is a revised patch.  I discovered a logic error after refactoring
>> the code.
>
> OK for the documentation parts.
>
>> +  decomp_fd = mkstemp (template);
>
> Do we have mkstemp on all supported platforms?

It looks like there is a version in gnulib, if not supported natively.

-- 
Michael Eager	 eager@eagercon.com
1960 Park Blvd., Palo Alto, CA 94306  650-325-8077

Re: [PATCH] Support gzip compressed exec and core files in gdb

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 1082 bytes --]

On 01 May 2015 15:03, Michael Eager wrote:
> On 05/01/2015 10:53 AM, Eli Zaretskii wrote:
> >> Date: Fri, 01 May 2015 07:40:26 -0700
> >> From: Michael Eager <eager@eagerm.com>
> >>
> >> On 03/10/2015 04:01 PM, Michael Eager wrote:
> >>> Add support to automatically unzip compressed executable and core files.
> >>> Files will be uncompressed into temporary directory (/tmp or $TMPDIR)
> >>> and are deleted when GDB exits.  This should be transparent to users,
> >>> except for disk space requirements.  The name of the uncompressed file is
> >>> mentioned, but all references to the file in GDB messages is to the file
> >>> which the user specified.
> >>
> >> Attached is a revised patch.  I discovered a logic error after refactoring
> >> the code.
> >
> > OK for the documentation parts.
> >
> >> +  decomp_fd = mkstemp (template);
> >
> > Do we have mkstemp on all supported platforms?
> 
> It looks like there is a version in gnulib, if not supported natively.

libiberty limits itself to "mkstemps" since it provides a fallback for that ...
-mike

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [PATCH] Support gzip compressed exec and core files in gdb

[Michael Eager]

On 05/04/2015 12:50 AM, Mike Frysinger wrote:
> On 01 May 2015 15:03, Michael Eager wrote:
>> On 05/01/2015 10:53 AM, Eli Zaretskii wrote:
>>>> Date: Fri, 01 May 2015 07:40:26 -0700
>>>> From: Michael Eager <eager@eagerm.com>
>>>>
>>>> On 03/10/2015 04:01 PM, Michael Eager wrote:
>>>>> Add support to automatically unzip compressed executable and core files.
>>>>> Files will be uncompressed into temporary directory (/tmp or $TMPDIR)
>>>>> and are deleted when GDB exits.  This should be transparent to users,
>>>>> except for disk space requirements.  The name of the uncompressed file is
>>>>> mentioned, but all references to the file in GDB messages is to the file
>>>>> which the user specified.
>>>>
>>>> Attached is a revised patch.  I discovered a logic error after refactoring
>>>> the code.
>>>
>>> OK for the documentation parts.
>>>
>>>> +  decomp_fd = mkstemp (template);
>>>
>>> Do we have mkstemp on all supported platforms?
>>
>> It looks like there is a version in gnulib, if not supported natively.
>
> libiberty limits itself to "mkstemps" since it provides a fallback for that ...
> -mike
>

Any comment on this patch?  Or approval?

-- 
Michael Eager	 eager@eagercon.com
1960 Park Blvd., Palo Alto, CA 94306  650-325-8077