[PATCH] linux_nat_target::xfer_partial: Fallback to ptrace

[PATCH] linux_nat_target::xfer_partial: Fallback to ptrace

[Keith Seitz]

Commit 05c06f318fd9a112529dfc313e6512b399a645e4 enabled GDB
to access memory while threads are running. It did this by accessing
/proc/PID/task/LWP/mem.

Unfortunatley, this interface is not implemented for writing in older kernels
(such as RHEL6). This means that GDB is unable to insert breakpoints on
these hosts:

$ ./gdb -q gdb -ex start
Reading symbols from gdb...
Temporary breakpoint 1 at 0x40fdd5: file ../../src/gdb/gdb.c, line 28.
Starting program: /home/rhel6/fsf/linux/gdb/gdb
Warning:
Cannot insert breakpoint 1.
Cannot access memory at address 0x40fdd5

(gdb)

Before this patch, linux_proc_xfer_memory_partial (previously called
linux_proc_xfer_partial) would return TARGET_XFER_EOF if the write
to /proc/PID/mem failed. [More specifically, linux_proc_xfer_partial would
not "bother for one word," but the effect is the essentially same.]

This status was checked by linux_nat_target::xfer_partial, which would then
fallback to using ptrace to perform the operation.

This is the specific hunk that removed the fallback:

-  xfer = linux_proc_xfer_partial (object, annex, readbuf, writebuf,
-                                 offset, len, xfered_len);
-  if (xfer != TARGET_XFER_EOF)
-    return xfer;
+      return linux_proc_xfer_memory_partial (readbuf, writebuf,
+                                            offset, len, xfered_len);
+    }

   return inf_ptrace_target::xfer_partial (object, annex, readbuf, writebuf,
                                          offset, len, xfered_len);

This patch restores this fallback mechanism, enabling GDB to insert
breakpoints on these older kernels.

Tested on {unix,native-gdbserver,native-extended-gdbserver}/-m{32,64}
on x86_64, s390x, aarch64, and ppc64le.
---
 gdb/linux-nat.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/gdb/linux-nat.c b/gdb/linux-nat.c
index 740cc0ddfc0..f5490c27bf5 100644
--- a/gdb/linux-nat.c
+++ b/gdb/linux-nat.c
@@ -3706,8 +3706,12 @@ linux_nat_target::xfer_partial (enum target_object object,
       if (addr_bit < (sizeof (ULONGEST) * HOST_CHAR_BIT))
 	offset &= ((ULONGEST) 1 << addr_bit) - 1;
 
-      return linux_proc_xfer_memory_partial (readbuf, writebuf,
-					     offset, len, xfered_len);
+      enum target_xfer_status xfer
+	= linux_proc_xfer_memory_partial (readbuf, writebuf,
+					  offset, len, xfered_len);
+      if (xfer != TARGET_XFER_EOF)
+	return xfer;
+      /* Fallthrough to ptrace.  */
     }
 
   return inf_ptrace_target::xfer_partial (object, annex, readbuf, writebuf,
-- 
2.36.1

Re: [PATCH] linux_nat_target::xfer_partial: Fallback to ptrace

[Pedro Alves]

On 2022-05-12 19:15, Keith Seitz via Gdb-patches wrote:
> Commit 05c06f318fd9a112529dfc313e6512b399a645e4 enabled GDB
> to access memory while threads are running. It did this by accessing
> /proc/PID/task/LWP/mem.
> 
> Unfortunatley, this interface is not implemented for writing in older kernels

Unfortunatley -> Unfortunately

> (such as RHEL6). This means that GDB is unable to insert breakpoints on
> these hosts:
> 
> $ ./gdb -q gdb -ex start
> Reading symbols from gdb...
> Temporary breakpoint 1 at 0x40fdd5: file ../../src/gdb/gdb.c, line 28.
> Starting program: /home/rhel6/fsf/linux/gdb/gdb
> Warning:
> Cannot insert breakpoint 1.
> Cannot access memory at address 0x40fdd5
> 
> (gdb)
> 

Oh man.  I thought such kernels were already older than the oldest version
we support, but looks like not.  :-/  I don't suppose you could instead
convince the kernel team to backport the patches that made /proc/pid/mem
writable (https://lore.kernel.org/lkml/20110314151320.GG21770@outflux.net/T/).. :-P

Both gdb and gdbserver are now relying on this to access memory of running threads.
This never worked for gdb, but it did for gdbserver, by stopping all threads temporarily.
I would really-really-really prefer not to add that code back for ancient
kernels...

> --- a/gdb/linux-nat.c
> +++ b/gdb/linux-nat.c
> @@ -3706,8 +3706,12 @@ linux_nat_target::xfer_partial (enum target_object object,
>        if (addr_bit < (sizeof (ULONGEST) * HOST_CHAR_BIT))
>  	offset &= ((ULONGEST) 1 << addr_bit) - 1;
>  
> -      return linux_proc_xfer_memory_partial (readbuf, writebuf,
> -					     offset, len, xfered_len);
> +      enum target_xfer_status xfer
> +	= linux_proc_xfer_memory_partial (readbuf, writebuf,
> +					  offset, len, xfered_len);
> +      if (xfer != TARGET_XFER_EOF)
> +	return xfer;
> +      /* Fallthrough to ptrace.  */

Seems fine, but I'd like a comment here giving a hint that we'll be able to
remove this once we stop supporting such old kernels.  Something like:

      /* Fallthrough to ptrace.  /proc/pid/mem wasn't writable before Linux 2.6.39.  */

I got that number by finding commit 198214a7ee50, and looking at git tag --contains 198214a7ee50.

AFAICT, RHEL 6 is on 2.6.32.

Re: [PATCH] linux_nat_target::xfer_partial: Fallback to ptrace

[Keith Seitz]

On 5/20/22 11:51, Pedro Alves wrote:
> On 2022-05-12 19:15, Keith Seitz via Gdb-patches wrote:
>> Commit 05c06f318fd9a112529dfc313e6512b399a645e4 enabled GDB
>> to access memory while threads are running. It did this by accessing
>> /proc/PID/task/LWP/mem.
>>
>> Unfortunatley, this interface is not implemented for writing in older kernels
> 
> Unfortunatley -> Unfortunately

Fixed.

> Oh man.  I thought such kernels were already older than the oldest version
> we support, but looks like not.  :-/  I don't suppose you could instead
> convince the kernel team to backport the patches that made /proc/pid/mem
> writable (https://lore.kernel.org/lkml/20110314151320.GG21770@outflux.net/T/).. :-P

:-)

> Both gdb and gdbserver are now relying on this to access memory of running threads.
> This never worked for gdb, but it did for gdbserver, by stopping all threads temporarily.
> I would really-really-really prefer not to add that code back for ancient
> kernels...

I did not observe any issues with gdbserver. As to whether we need to support
kernels as old as RHEL6? I don't really know. I noticed problems when I was
running through some internal testing which still uses RHEL6.

I figured (maybe incorrectly) that the fallthrough was otherwise harmless.
I'm fine if we'd prefer not to include this patch, though. I honestly
haven't a clue how widespread RHEL6-vintage kernels are in the wild.

>> --- a/gdb/linux-nat.c
>> +++ b/gdb/linux-nat.c
>> @@ -3706,8 +3706,12 @@ linux_nat_target::xfer_partial (enum target_object object,
>>         if (addr_bit < (sizeof (ULONGEST) * HOST_CHAR_BIT))
>>   	offset &= ((ULONGEST) 1 << addr_bit) - 1;
>>   
>> -      return linux_proc_xfer_memory_partial (readbuf, writebuf,
>> -					     offset, len, xfered_len);
>> +      enum target_xfer_status xfer
>> +	= linux_proc_xfer_memory_partial (readbuf, writebuf,
>> +					  offset, len, xfered_len);
>> +      if (xfer != TARGET_XFER_EOF)
>> +	return xfer;
>> +      /* Fallthrough to ptrace.  */
> 
> Seems fine, but I'd like a comment here giving a hint that we'll be able to
> remove this once we stop supporting such old kernels.  Something like:
> 
>        /* Fallthrough to ptrace.  /proc/pid/mem wasn't writable before Linux 2.6.39.  */
> 
> I got that number by finding commit 198214a7ee50, and looking at git tag --contains 198214a7ee50.

I've updated that comment.

> AFAICT, RHEL 6 is on 2.6.32.

As far as I can tell, that is correct.

I will wait before pushing this to give others the opportunity to chime in.

Thank you for taking a look at this,
Keith

Re: [PATCH] linux_nat_target::xfer_partial: Fallback to ptrace

[Pedro Alves]

On 2022-05-24 19:56, Keith Seitz wrote:
> On 5/20/22 11:51, Pedro Alves wrote:
>> On 2022-05-12 19:15, Keith Seitz via Gdb-patches wrote:
>>> Commit 05c06f318fd9a112529dfc313e6512b399a645e4 enabled GDB
>>> to access memory while threads are running. It did this by accessing
>>> /proc/PID/task/LWP/mem.
>>>
>>> Unfortunatley, this interface is not implemented for writing in older kernels
>>
>> Unfortunatley -> Unfortunately
> 
> Fixed.
> 
>> Oh man.  I thought such kernels were already older than the oldest version
>> we support, but looks like not.  :-/  I don't suppose you could instead
>> convince the kernel team to backport the patches that made /proc/pid/mem
>> writable (https://lore.kernel.org/lkml/20110314151320.GG21770@outflux.net/T/).. :-P
> 
> :-)
> 
>> Both gdb and gdbserver are now relying on this to access memory of running threads.
>> This never worked for gdb, but it did for gdbserver, by stopping all threads temporarily.
>> I would really-really-really prefer not to add that code back for ancient
>> kernels...
> 
> I did not observe any issues with gdbserver. As to whether we need to support
> kernels as old as RHEL6? I don't really know. I noticed problems when I was
> running through some internal testing which still uses RHEL6.
> 
> I figured (maybe incorrectly) that the fallthrough was otherwise harmless.
> I'm fine if we'd prefer not to include this patch, though. I honestly
> haven't a clue how widespread RHEL6-vintage kernels are in the wild.
> 
>>> --- a/gdb/linux-nat.c
>>> +++ b/gdb/linux-nat.c
>>> @@ -3706,8 +3706,12 @@ linux_nat_target::xfer_partial (enum target_object object,
>>>         if (addr_bit < (sizeof (ULONGEST) * HOST_CHAR_BIT))
>>>       offset &= ((ULONGEST) 1 << addr_bit) - 1;
>>>   -      return linux_proc_xfer_memory_partial (readbuf, writebuf,
>>> -                         offset, len, xfered_len);
>>> +      enum target_xfer_status xfer
>>> +    = linux_proc_xfer_memory_partial (readbuf, writebuf,
>>> +                      offset, len, xfered_len);
>>> +      if (xfer != TARGET_XFER_EOF)
>>> +    return xfer;
>>> +      /* Fallthrough to ptrace.  */
>>
>> Seems fine, but I'd like a comment here giving a hint that we'll be able to
>> remove this once we stop supporting such old kernels.  Something like:
>>
>>        /* Fallthrough to ptrace.  /proc/pid/mem wasn't writable before Linux 2.6.39.  */
>>
>> I got that number by finding commit 198214a7ee50, and looking at git tag --contains 198214a7ee50.
> 
> I've updated that comment.
> 
>> AFAICT, RHEL 6 is on 2.6.32.
> 
> As far as I can tell, that is correct.
> 
> I will wait before pushing this to give others the opportunity to chime in.

I'd like to chime in again myself. :-P

Lancelot's patch here:

 https://sourceware.org/pipermail/gdb-patches/2022-May/189388.html

made me realize that checking for TARGET_XFER_EOF isn't what we want here.
If /proc/pid/mem failed with TARGET_XFER_EOF, it means the program exited,
or execed.  If the latter, we want to propagate the error out, not fallback
to ptrace, as that would result in read/writing memory out of the post-exec
address space, while gdb thought it was reading/writing the pre-exec address space.
I.e., the kernel failing with EOF is nice in that it avoids a race condition
where we e.g., poke a breakpoint (or poke back the original instruction) in the
post-exec binary at a completely wrong address/instruction.  The other case
for EOF is when the process exits, and falling back to ptrace in that case
is just pointless.

I guess the write to /proc/pid/mem fails with EIO for you, and there's nothing
else we can use to detect the scenario.  So we probably want to check TARGET_XFER_E_IO
instead.  And, maybe only do the fallback if writing.

Re: [PATCH] linux_nat_target::xfer_partial: Fallback to ptrace

[Pedro Alves]

[-- Attachment #1: Type: text/plain, Size: 1841 bytes --]

On 2022-07-26 6:24 p.m., Keith Seitz wrote:

> I've taken your patch and updated mine (now trivial):
> 
> diff --git a/gdb/linux-nat.c b/gdb/linux-nat.c
> index 0a93ab5..95732a6 100644
> --- a/gdb/linux-nat.c
> +++ b/gdb/linux-nat.c
> @@ -3710,8 +3712,10 @@ enum target_xfer_status
>        if (addr_bit < (sizeof (ULONGEST) * HOST_CHAR_BIT))
>         offset &= ((ULONGEST) 1 << addr_bit) - 1;
>  
> -      return linux_proc_xfer_memory_partial (readbuf, writebuf,
> -                                            offset, len, xfered_len);
> +      if (proc_mem_file_is_writable ())
> +       return linux_proc_xfer_memory_partial (readbuf, writebuf,
> +                                              offset, len, xfered_len);
> +      /* Fallthrough to ptrace  */
>      }
>  
>    return inf_ptrace_target::xfer_partial (object, annex, readbuf, writebuf,
> 
> I've re-run this through all my testing, and all looks good.

Great, thanks.

> 
> However, I suspect you already knew that. :-)

I was actually curious whether that would work (or rather fail quietly) on the
older kernels!

> 
> If/when you push your patch, and there are no further concerns, I will push mine,
> with your approval.
> 

As discussed off list, I merged both patches to master at once, to avoid "unused"
warnings.

I've attached both patches, as merged.  In my patch, I fixed a bad usage
of gdb::optional (need to emplace once before doing "*opt" ...), and removed
the "static" from the written-to variable (I don't know why I added it in the
first place).  In your patch, I added a comment explaining why not to fallback
to ptrace, in case someone changes this code in the future.

> Thank you for your follow-up!

np.

Pedro Alves.

[-- Attachment #2: 0001-gdb-linux-nat-Check-whether-proc-pid-mem-is-writable.patch --]
[-- Type: text/x-patch, Size: 5716 bytes --]

From 1bcb0708f22956d5128a2e75df6eba5a18327892 Mon Sep 17 00:00:00 2001
From: Pedro Alves <pedro@palves.net>
Date: Thu, 21 Jul 2022 19:11:16 +0100
Subject: [PATCH 1/2] gdb/linux-nat: Check whether /proc/pid/mem is writable

Probe whether /proc/pid/mem is writable, by using it to write to a GDB
variable.  This will be used in the following patch to avoid falling
back to writing to inferior memory with ptrace if /proc/pid/mem _is_
writable.

Change-Id: If87eff0b46cbe5e32a583e2977a9e17d29d0ed3e
---
 gdb/linux-nat.c | 105 ++++++++++++++++++++++++++++++++++++++++--------
 1 file changed, 88 insertions(+), 17 deletions(-)

diff --git a/gdb/linux-nat.c b/gdb/linux-nat.c
index a2bbd3cbfc8..b641e88b1ef 100644
--- a/gdb/linux-nat.c
+++ b/gdb/linux-nat.c
@@ -244,6 +244,7 @@ static int lwp_status_pending_p (struct lwp_info *lp);
 
 static void save_stop_reason (struct lwp_info *lp);
 
+static bool proc_mem_file_is_writable ();
 static void close_proc_mem_file (pid_t pid);
 static void open_proc_mem_file (ptid_t ptid);
 
@@ -3882,25 +3883,19 @@ open_proc_mem_file (ptid_t ptid)
 			  fd, ptid.pid (), ptid.lwp ());
 }
 
-/* Implement the to_xfer_partial target method using /proc/PID/mem.
-   Because we can use a single read/write call, this can be much more
-   efficient than banging away at PTRACE_PEEKTEXT.  Also, unlike
-   PTRACE_PEEKTEXT/PTRACE_POKETEXT, this works with running
-   threads.  */
+/* Helper for linux_proc_xfer_memory_partial and
+   proc_mem_file_is_writable.  FD is the already opened /proc/pid/mem
+   file, and PID is the pid of the corresponding process.  The rest of
+   the arguments are like linux_proc_xfer_memory_partial's.  */
 
 static enum target_xfer_status
-linux_proc_xfer_memory_partial (gdb_byte *readbuf, const gdb_byte *writebuf,
-				ULONGEST offset, LONGEST len,
-				ULONGEST *xfered_len)
+linux_proc_xfer_memory_partial_fd (int fd, int pid,
+				   gdb_byte *readbuf, const gdb_byte *writebuf,
+				   ULONGEST offset, LONGEST len,
+				   ULONGEST *xfered_len)
 {
   ssize_t ret;
 
-  auto iter = proc_mem_file_map.find (inferior_ptid.pid ());
-  if (iter == proc_mem_file_map.end ())
-    return TARGET_XFER_EOF;
-
-  int fd = iter->second.fd ();
-
   gdb_assert (fd != -1);
 
   /* Use pread64/pwrite64 if available, since they save a syscall and can
@@ -3919,8 +3914,7 @@ linux_proc_xfer_memory_partial (gdb_byte *readbuf, const gdb_byte *writebuf,
   if (ret == -1)
     {
       linux_nat_debug_printf ("accessing fd %d for pid %d failed: %s (%d)",
-			      fd, inferior_ptid.pid (),
-			      safe_strerror (errno), errno);
+			      fd, pid, safe_strerror (errno), errno);
       return TARGET_XFER_E_IO;
     }
   else if (ret == 0)
@@ -3928,7 +3922,7 @@ linux_proc_xfer_memory_partial (gdb_byte *readbuf, const gdb_byte *writebuf,
       /* EOF means the address space is gone, the whole process exited
 	 or execed.  */
       linux_nat_debug_printf ("accessing fd %d for pid %d got EOF",
-			      fd, inferior_ptid.pid ());
+			      fd, pid);
       return TARGET_XFER_EOF;
     }
   else
@@ -3938,6 +3932,81 @@ linux_proc_xfer_memory_partial (gdb_byte *readbuf, const gdb_byte *writebuf,
     }
 }
 
+/* Implement the to_xfer_partial target method using /proc/PID/mem.
+   Because we can use a single read/write call, this can be much more
+   efficient than banging away at PTRACE_PEEKTEXT.  Also, unlike
+   PTRACE_PEEKTEXT/PTRACE_POKETEXT, this works with running
+   threads.  */
+
+static enum target_xfer_status
+linux_proc_xfer_memory_partial (gdb_byte *readbuf, const gdb_byte *writebuf,
+				ULONGEST offset, LONGEST len,
+				ULONGEST *xfered_len)
+{
+  int pid = inferior_ptid.pid ();
+
+  auto iter = proc_mem_file_map.find (pid);
+  if (iter == proc_mem_file_map.end ())
+    return TARGET_XFER_EOF;
+
+  int fd = iter->second.fd ();
+
+  return linux_proc_xfer_memory_partial_fd (fd, pid, readbuf, writebuf, offset,
+					    len, xfered_len);
+}
+
+/* Check whether /proc/pid/mem is writable in the current kernel, and
+   return true if so.  It wasn't writable before Linux 2.6.39, but
+   there's no way to know whether the feature was backported to older
+   kernels.  So we check to see if it works.  The result is cached,
+   and this is garanteed to be called once early at startup.  */
+
+static bool
+proc_mem_file_is_writable ()
+{
+  static gdb::optional<bool> writable;
+
+  if (writable.has_value ())
+    return *writable;
+
+  writable.emplace (false);
+
+  /* We check whether /proc/pid/mem is writable by trying to write to
+     one of our variables via /proc/self/mem.  */
+
+  int fd = gdb_open_cloexec ("/proc/self/mem", O_RDWR | O_LARGEFILE, 0).release ();
+
+  if (fd == -1)
+    {
+      warning (_("opening /proc/self/mem file failed: %s (%d)"),
+	       safe_strerror (errno), errno);
+      return *writable;
+    }
+
+  SCOPE_EXIT { close (fd); };
+
+  /* This is the variable we try to write to.  Note OFFSET below.  */
+  volatile gdb_byte test_var = 0;
+
+  gdb_byte writebuf[] = {0x55};
+  ULONGEST offset = (uintptr_t) &test_var;
+  ULONGEST xfered_len;
+
+  enum target_xfer_status res
+    = linux_proc_xfer_memory_partial_fd (fd, getpid (), nullptr, writebuf,
+					 offset, 1, &xfered_len);
+
+  if (res == TARGET_XFER_OK)
+    {
+      gdb_assert (xfered_len == 1);
+      gdb_assert (test_var == 0x55);
+      /* Success.  */
+      *writable = true;
+    }
+
+  return *writable;
+}
+
 /* Parse LINE as a signal set and add its set bits to SIGS.  */
 
 static void
@@ -4437,6 +4506,8 @@ Enables printf debugging output."),
   sigemptyset (&blocked_mask);
 
   lwp_lwpid_htab_create ();
+
+  proc_mem_file_is_writable ();
 }
 \f
 

base-commit: ecbff28a4457d0ebe11023fa9671d62251e7463d
-- 
2.36.0


[-- Attachment #3: 0002-gdb-linux_nat-Write-memory-using-ptrace-if-proc-pid-.patch --]
[-- Type: text/x-patch, Size: 3327 bytes --]

From dd09fe0d53242a5f6a86d2822b0cfdeb3f5baa8f Mon Sep 17 00:00:00 2001
From: Keith Seitz <keiths@redhat.com>
Date: Tue, 26 Jul 2022 19:11:04 +0100
Subject: [PATCH 2/2] gdb/linux_nat: Write memory using ptrace if /proc/pid/mem
 is not writable

Commit 05c06f318fd9a112529dfc313e6512b399a645e4 enabled GDB to access
memory while threads are running.  It did this by accessing
/proc/PID/task/LWP/mem.

Unfortunately, this interface is not implemented for writing in older
kernels (such as RHEL6).  This means that GDB is unable to insert
breakpoints on these hosts:

 $ ./gdb -q gdb -ex start
 Reading symbols from gdb...
 Temporary breakpoint 1 at 0x40fdd5: file ../../src/gdb/gdb.c, line 28.
 Starting program: /home/rhel6/fsf/linux/gdb/gdb
 Warning:
 Cannot insert breakpoint 1.
 Cannot access memory at address 0x40fdd5

 (gdb)

Before this patch, linux_proc_xfer_memory_partial (previously called
linux_proc_xfer_partial) would return TARGET_XFER_EOF if the write to
/proc/PID/mem failed. [More specifically, linux_proc_xfer_partial
would not "bother for one word," but the effect is the essentially
same.]

This status was checked by linux_nat_target::xfer_partial, which would
then fallback to using ptrace to perform the operation.

This is the specific hunk that removed the fallback:

-  xfer = linux_proc_xfer_partial (object, annex, readbuf, writebuf,
-                                 offset, len, xfered_len);
-  if (xfer != TARGET_XFER_EOF)
-    return xfer;
+      return linux_proc_xfer_memory_partial (readbuf, writebuf,
+                                            offset, len, xfered_len);
+    }

   return inf_ptrace_target::xfer_partial (object, annex, readbuf, writebuf,
                                          offset, len, xfered_len);

This patch makes linux_nat_target::xfer_partial go straight to writing
memory via ptrace if writing via /proc/pid/mem is not possible in the
running kernel, enabling GDB to insert breakpoints on these older
kernels.  Note that a recent patch changed the return status from
TARGET_XFER_EOF to TARGET_XFER_E_IO.

Tested on {unix,native-gdbserver,native-extended-gdbserver}/-m{32,64}
on x86_64, s390x, aarch64, and ppc64le.

Change-Id: If1d884278e8c4ea71d8836bedd56e6a6c242a415
---
 gdb/linux-nat.c | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/gdb/linux-nat.c b/gdb/linux-nat.c
index b641e88b1ef..e638e8ad04e 100644
--- a/gdb/linux-nat.c
+++ b/gdb/linux-nat.c
@@ -3711,8 +3711,15 @@ linux_nat_target::xfer_partial (enum target_object object,
       if (addr_bit < (sizeof (ULONGEST) * HOST_CHAR_BIT))
 	offset &= ((ULONGEST) 1 << addr_bit) - 1;
 
-      return linux_proc_xfer_memory_partial (readbuf, writebuf,
-					     offset, len, xfered_len);
+      /* If /proc/pid/mem is writable, don't fallback to ptrace.  If
+	 the write via /proc/pid/mem fails because the inferior execed
+	 (and we haven't seen the exec event yet), a subsequent ptrace
+	 poke would incorrectly write memory to the post-exec address
+	 space, while the core was trying to write to the pre-exec
+	 address space.  */
+      if (proc_mem_file_is_writable ())
+	return linux_proc_xfer_memory_partial (readbuf, writebuf,
+					       offset, len, xfered_len);
     }
 
   return inf_ptrace_target::xfer_partial (object, annex, readbuf, writebuf,
-- 
2.36.0

Re: [PATCH] linux_nat_target::xfer_partial: Fallback to ptrace

[Keith Seitz]

On 7/21/22 13:07, Pedro Alves wrote:
> 
> Appologies for the delay, there was something here that I wanted to think about, and then it
> just fell through the cracks.

I have quite a few cracks myself through which things fall!

>> On 6/3/22 08:18, Keith Seitz via Gdb-patches wrote:
>>> Pedro Alves wrote:
>>> diff --git a/gdb/linux-nat.c b/gdb/linux-nat.c
>>> index 3b5400896bc..571c97137c2 100644
>>> --- a/gdb/linux-nat.c
>>> +++ b/gdb/linux-nat.c
>>> @@ -3706,8 +3706,12 @@ linux_nat_target::xfer_partial (enum target_object object,
>>>          if (addr_bit < (sizeof (ULONGEST) * HOST_CHAR_BIT))
>>>        offset &= ((ULONGEST) 1 << addr_bit) - 1;
>>>    -      return linux_proc_xfer_memory_partial (readbuf, writebuf,
>>> -                         offset, len, xfered_len);
>>> +      enum target_xfer_status xfer
>>> +    = linux_proc_xfer_memory_partial (readbuf, writebuf,
>>> +                      offset, len, xfered_len);
>>> +      if (xfer != TARGET_XFER_E_IO || readbuf != nullptr)
>>> +    return xfer;
>>> +      /* Fallthrough to ptrace.  /proc/pid/mem wasn't writable before Linux 2.6.39.  */
>>>        }
>>>        return inf_ptrace_target::xfer_partial (object, annex, readbuf, writebuf,
>>
> So I was wondering about mitigating this by only falling back to ptrace if writing to
> /proc/pid/mem doesn't really work.  Checking the kernel version itself seems a bit
> fragile, so I thought we could make gdb probe once at started up whether writing
> to itself via /proc/self/mem works.  It turns out that actually works.  With this, you'd
> just add an extra proc_mem_file_is_writable() check in your patch before falling
> back, or even, skip straight to ptrace if !proc_mem_file_is_writable().
> 
> WDYT?

I think this is quite clever, and an obvious step up in reliability. Thanks
for doing this for me.

>  From 56622b9cadff4b62a0b05861015ce06cf9d6e8f2 Mon Sep 17 00:00:00 2001
> From: Pedro Alves <pedro@palves.net>
> Date: Thu, 21 Jul 2022 19:11:16 +0100
> Subject: [PATCH] gdb/linux-nat: Check whether /proc/pid/mem is writable
> 
> Probe whether /proc/pid/mem is writable, by using it to write to a GDB
> variable.

[snip]

I've taken your patch and updated mine (now trivial):

diff --git a/gdb/linux-nat.c b/gdb/linux-nat.c
index 0a93ab5..95732a6 100644
--- a/gdb/linux-nat.c
+++ b/gdb/linux-nat.c
@@ -3710,8 +3712,10 @@ enum target_xfer_status
        if (addr_bit < (sizeof (ULONGEST) * HOST_CHAR_BIT))
         offset &= ((ULONGEST) 1 << addr_bit) - 1;
  
-      return linux_proc_xfer_memory_partial (readbuf, writebuf,
-                                            offset, len, xfered_len);
+      if (proc_mem_file_is_writable ())
+       return linux_proc_xfer_memory_partial (readbuf, writebuf,
+                                              offset, len, xfered_len);
+      /* Fallthrough to ptrace  */
      }
  
    return inf_ptrace_target::xfer_partial (object, annex, readbuf, writebuf,

I've re-run this through all my testing, and all looks good.

However, I suspect you already knew that. :-)

If/when you push your patch, and there are no further concerns, I will push mine,
with your approval.

Thank you for your follow-up!
Keith

Re: [PATCH] linux_nat_target::xfer_partial: Fallback to ptrace

[Pedro Alves]

Hi Keith,

On 2022-07-21 4:03 p.m., Keith Seitz wrote:
> Ping

Appologies for the delay, there was something here that I wanted to think about, and then it
just fell through the cracks.

See below.

> On 6/3/22 08:18, Keith Seitz via Gdb-patches wrote:
>> Pedro Alves wrote:
>>
>>> I guess the write to /proc/pid/mem fails with EIO for you, and there's nothing
>>> else we can use to detect the scenario.  So we probably want to check TARGET_XFER_E_IO
>>> instead.  And, maybe only do the fallback if writing.
>>
>> I've updated the patch to include these changes and retested on all the "usual"
>> target boards on Fedora 36 x86_64, s390x, ppc64le, and aarch64 and on RHEL6
>> x86_64.

Thanks, that's a lot of testing.

>> Commit 05c06f318fd9a112529dfc313e6512b399a645e4 enabled GDB
>> to access memory while threads are running. It did this by accessing
>> /proc/PID/task/LWP/mem.
>>
>> Unfortunately, this interface is not implemented for writing in older kernels
>> (such as RHEL6). This means that GDB is unable to insert breakpoints on
>> these hosts:
>>
>> $ ./gdb -q gdb -ex start
>> Reading symbols from gdb...
>> Temporary breakpoint 1 at 0x40fdd5: file ../../src/gdb/gdb.c, line 28.
>> Starting program: /home/rhel6/fsf/linux/gdb/gdb
>> Warning:
>> Cannot insert breakpoint 1.
>> Cannot access memory at address 0x40fdd5
>>
>> (gdb)
>>
>> Before this patch, linux_proc_xfer_memory_partial (previously called
>> linux_proc_xfer_partial) would return TARGET_XFER_EOF if the write
>> to /proc/PID/mem failed. [More specifically, linux_proc_xfer_partial would
>> not "bother for one word," but the effect is the essentially same.]
>>
>> This status was checked by linux_nat_target::xfer_partial, which would then
>> fallback to using ptrace to perform the operation.
>>
>> This is the specific hunk that removed the fallback:
>>
>> -  xfer = linux_proc_xfer_partial (object, annex, readbuf, writebuf,
>> -                                 offset, len, xfered_len);
>> -  if (xfer != TARGET_XFER_EOF)
>> -    return xfer;
>> +      return linux_proc_xfer_memory_partial (readbuf, writebuf,
>> +                                            offset, len, xfered_len);
>> +    }
>>
>>     return inf_ptrace_target::xfer_partial (object, annex, readbuf, writebuf,
>>                                            offset, len, xfered_len);
>>
>> This patch restores this fallback mechanism, enabling GDB to insert
>> breakpoints on these older kernels. Note that a recent patch changed
>> the return status from TARGET_XFER_EOF to TARGET_XFER_E_IO.
>>
>> Tested on {unix,native-gdbserver,native-extended-gdbserver}/-m{32,64}
>> on x86_64, s390x, aarch64, and ppc64le.
>> ---
>>   gdb/linux-nat.c | 8 ++++++--
>>   1 file changed, 6 insertions(+), 2 deletions(-)
>>
>> diff --git a/gdb/linux-nat.c b/gdb/linux-nat.c
>> index 3b5400896bc..571c97137c2 100644
>> --- a/gdb/linux-nat.c
>> +++ b/gdb/linux-nat.c
>> @@ -3706,8 +3706,12 @@ linux_nat_target::xfer_partial (enum target_object object,
>>         if (addr_bit < (sizeof (ULONGEST) * HOST_CHAR_BIT))
>>       offset &= ((ULONGEST) 1 << addr_bit) - 1;
>>   -      return linux_proc_xfer_memory_partial (readbuf, writebuf,
>> -                         offset, len, xfered_len);
>> +      enum target_xfer_status xfer
>> +    = linux_proc_xfer_memory_partial (readbuf, writebuf,
>> +                      offset, len, xfered_len);
>> +      if (xfer != TARGET_XFER_E_IO || readbuf != nullptr)
>> +    return xfer;
>> +      /* Fallthrough to ptrace.  /proc/pid/mem wasn't writable before Linux 2.6.39.  */
>>       }
>>       return inf_ptrace_target::xfer_partial (object, annex, readbuf, writebuf,
> 

So my worry here is this bringing back a race with more modern kernels, one which the
always writing via /proc/pid/mem plugged -- when the inferior execs, writes through /proc/pid/mem fail,
as the old pre-exec address space is gone.  Only when we see the exec even out of ptrace, will
we close the file and reopen a new one to access the post-exec address space.  Up until recent
kernels, ptrace calls didn't have a similar protection.  I.e., if the inferior execs, and the
write through /proc/pid/mem fails because of an exec, and then we fallback to ptrace, that write
will succeed, but it will be writing bytes in the post-exec address space that were meant
for the pre-exec address space.

So I was wondering about mitigating this by only falling back to ptrace if writing to
/proc/pid/mem doesn't really work.  Checking the kernel version itself seems a bit
fragile, so I thought we could make gdb probe once at started up whether writing
to itself via /proc/self/mem works.  It turns out that actually works.  With this, you'd
just add an extra proc_mem_file_is_writable() check in your patch before falling
back, or even, skip straight to ptrace if !proc_mem_file_is_writable().

WDYT?

From 56622b9cadff4b62a0b05861015ce06cf9d6e8f2 Mon Sep 17 00:00:00 2001
From: Pedro Alves <pedro@palves.net>
Date: Thu, 21 Jul 2022 19:11:16 +0100
Subject: [PATCH] gdb/linux-nat: Check whether /proc/pid/mem is writable

Probe whether /proc/pid/mem is writable, by using it to write to a GDB
variable.

Change-Id: If87eff0b46cbe5e32a583e2977a9e17d29d0ed3e
---
 gdb/linux-nat.c | 106 ++++++++++++++++++++++++++++++++++++++++--------
 1 file changed, 89 insertions(+), 17 deletions(-)

diff --git a/gdb/linux-nat.c b/gdb/linux-nat.c
index 0a93ab5c6ae..0916b841ec7 100644
--- a/gdb/linux-nat.c
+++ b/gdb/linux-nat.c
@@ -3674,6 +3674,8 @@ static enum target_xfer_status
 linux_proc_xfer_memory_partial (gdb_byte *readbuf, const gdb_byte *writebuf,
 				ULONGEST offset, LONGEST len, ULONGEST *xfered_len);
 
+static bool proc_mem_file_is_writable ();
+
 enum target_xfer_status
 linux_nat_target::xfer_partial (enum target_object object,
 				const char *annex, gdb_byte *readbuf,
@@ -3882,25 +3884,19 @@ open_proc_mem_file (ptid_t ptid)
 			  fd, ptid.pid (), ptid.lwp ());
 }
 
-/* Implement the to_xfer_partial target method using /proc/PID/mem.
-   Because we can use a single read/write call, this can be much more
-   efficient than banging away at PTRACE_PEEKTEXT.  Also, unlike
-   PTRACE_PEEKTEXT/PTRACE_POKETEXT, this works with running
-   threads.  */
+/* Helper for linux_proc_xfer_memory_partial and
+   proc_mem_file_is_writable.  FD is the already opened /proc/pid/mem
+   file, and PID is the pid of the corresponding process.  The rest of
+   the arguments are like linux_proc_xfer_memory_partial's.  */
 
 static enum target_xfer_status
-linux_proc_xfer_memory_partial (gdb_byte *readbuf, const gdb_byte *writebuf,
-				ULONGEST offset, LONGEST len,
-				ULONGEST *xfered_len)
+linux_proc_xfer_memory_partial_fd (int fd, int pid,
+				   gdb_byte *readbuf, const gdb_byte *writebuf,
+				   ULONGEST offset, LONGEST len,
+				   ULONGEST *xfered_len)
 {
   ssize_t ret;
 
-  auto iter = proc_mem_file_map.find (inferior_ptid.pid ());
-  if (iter == proc_mem_file_map.end ())
-    return TARGET_XFER_EOF;
-
-  int fd = iter->second.fd ();
-
   gdb_assert (fd != -1);
 
   /* Use pread64/pwrite64 if available, since they save a syscall and can
@@ -3919,8 +3915,7 @@ linux_proc_xfer_memory_partial (gdb_byte *readbuf, const gdb_byte *writebuf,
   if (ret == -1)
     {
       linux_nat_debug_printf ("accessing fd %d for pid %d failed: %s (%d)",
-			      fd, inferior_ptid.pid (),
-			      safe_strerror (errno), errno);
+			      fd, pid, safe_strerror (errno), errno);
       return TARGET_XFER_E_IO;
     }
   else if (ret == 0)
@@ -3928,7 +3923,7 @@ linux_proc_xfer_memory_partial (gdb_byte *readbuf, const gdb_byte *writebuf,
       /* EOF means the address space is gone, the whole process exited
 	 or execed.  */
       linux_nat_debug_printf ("accessing fd %d for pid %d got EOF",
-			      fd, inferior_ptid.pid ());
+			      fd, pid);
       return TARGET_XFER_EOF;
     }
   else
@@ -3938,6 +3933,81 @@ linux_proc_xfer_memory_partial (gdb_byte *readbuf, const gdb_byte *writebuf,
     }
 }
 
+/* Implement the to_xfer_partial target method using /proc/PID/mem.
+   Because we can use a single read/write call, this can be much more
+   efficient than banging away at PTRACE_PEEKTEXT.  Also, unlike
+   PTRACE_PEEKTEXT/PTRACE_POKETEXT, this works with running
+   threads.  */
+
+static enum target_xfer_status
+linux_proc_xfer_memory_partial (gdb_byte *readbuf, const gdb_byte *writebuf,
+				ULONGEST offset, LONGEST len,
+				ULONGEST *xfered_len)
+{
+  int pid = inferior_ptid.pid ();
+
+  auto iter = proc_mem_file_map.find (pid);
+  if (iter == proc_mem_file_map.end ())
+    return TARGET_XFER_EOF;
+
+  int fd = iter->second.fd ();
+
+  return linux_proc_xfer_memory_partial_fd (fd, pid, readbuf, writebuf, offset,
+					    len, xfered_len);
+}
+
+/* Check whether /proc/pid/mem is writable in the current kernel, and
+   return true if so.  It wasn't writable before Linux 2.6.39, but
+   there's no way to know whether the feature was backported to older
+   kernels.  So we check to see if it works.  The result is cached,
+   and this is garanteed to be called once early at startup.  */
+
+static bool
+proc_mem_file_is_writable ()
+{
+  static gdb::optional<bool> writable;
+
+  if (writable.has_value ())
+    return *writable;
+
+  *writable = false;
+
+  /* We check whether /proc/pid/mem is writable by trying to write to
+     one of our variables via /proc/self/mem.  */
+
+  int fd = gdb_open_cloexec ("/proc/self/mem", O_RDWR | O_LARGEFILE, 0).release ();
+
+  if (fd == -1)
+    {
+      warning (_("opening /proc/self/mem file failed: %s (%d)"),
+	       safe_strerror (errno), errno);
+      return *writable;
+    }
+
+  SCOPE_EXIT { close (fd); };
+
+  /* This is the variable we try to write to.  Note OFFSET below.  */
+  volatile static gdb_byte test_var = 0;
+
+  gdb_byte writebuf[] = {0x55};
+  ULONGEST offset = (uintptr_t) &test_var;
+  ULONGEST xfered_len;
+
+  enum target_xfer_status res
+    = linux_proc_xfer_memory_partial_fd (fd, getpid (), nullptr, writebuf,
+					 offset, 1, &xfered_len);
+
+  if (res == TARGET_XFER_OK)
+    {
+      gdb_assert (xfered_len == 1);
+      gdb_assert (test_var == 0x55);
+      /* Success.  */
+      *writable = true;
+    }
+
+  return *writable;
+}
+
 /* Parse LINE as a signal set and add its set bits to SIGS.  */
 
 static void
@@ -4437,6 +4507,8 @@ Enables printf debugging output."),
   sigemptyset (&blocked_mask);
 
   lwp_lwpid_htab_create ();
+
+  proc_mem_file_is_writable ();
 }
 \f
 

base-commit: c07ec968f7342a2386969bc192ff0b42e33991ef
-- 
2.36.0

Re: [PATCH] linux_nat_target::xfer_partial: Fallback to ptrace

[Keith Seitz]

Ping

On 6/3/22 08:18, Keith Seitz via Gdb-patches wrote:
> Pedro Alves wrote:
> 
>> I guess the write to /proc/pid/mem fails with EIO for you, and there's nothing
>> else we can use to detect the scenario.  So we probably want to check TARGET_XFER_E_IO
>> instead.  And, maybe only do the fallback if writing.
> 
> I've updated the patch to include these changes and retested on all the "usual"
> target boards on Fedora 36 x86_64, s390x, ppc64le, and aarch64 and on RHEL6
> x86_64.
> 
> Keith
> 
> --
> 
> Commit 05c06f318fd9a112529dfc313e6512b399a645e4 enabled GDB
> to access memory while threads are running. It did this by accessing
> /proc/PID/task/LWP/mem.
> 
> Unfortunately, this interface is not implemented for writing in older kernels
> (such as RHEL6). This means that GDB is unable to insert breakpoints on
> these hosts:
> 
> $ ./gdb -q gdb -ex start
> Reading symbols from gdb...
> Temporary breakpoint 1 at 0x40fdd5: file ../../src/gdb/gdb.c, line 28.
> Starting program: /home/rhel6/fsf/linux/gdb/gdb
> Warning:
> Cannot insert breakpoint 1.
> Cannot access memory at address 0x40fdd5
> 
> (gdb)
> 
> Before this patch, linux_proc_xfer_memory_partial (previously called
> linux_proc_xfer_partial) would return TARGET_XFER_EOF if the write
> to /proc/PID/mem failed. [More specifically, linux_proc_xfer_partial would
> not "bother for one word," but the effect is the essentially same.]
> 
> This status was checked by linux_nat_target::xfer_partial, which would then
> fallback to using ptrace to perform the operation.
> 
> This is the specific hunk that removed the fallback:
> 
> -  xfer = linux_proc_xfer_partial (object, annex, readbuf, writebuf,
> -                                 offset, len, xfered_len);
> -  if (xfer != TARGET_XFER_EOF)
> -    return xfer;
> +      return linux_proc_xfer_memory_partial (readbuf, writebuf,
> +                                            offset, len, xfered_len);
> +    }
> 
>     return inf_ptrace_target::xfer_partial (object, annex, readbuf, writebuf,
>                                            offset, len, xfered_len);
> 
> This patch restores this fallback mechanism, enabling GDB to insert
> breakpoints on these older kernels. Note that a recent patch changed
> the return status from TARGET_XFER_EOF to TARGET_XFER_E_IO.
> 
> Tested on {unix,native-gdbserver,native-extended-gdbserver}/-m{32,64}
> on x86_64, s390x, aarch64, and ppc64le.
> ---
>   gdb/linux-nat.c | 8 ++++++--
>   1 file changed, 6 insertions(+), 2 deletions(-)
> 
> diff --git a/gdb/linux-nat.c b/gdb/linux-nat.c
> index 3b5400896bc..571c97137c2 100644
> --- a/gdb/linux-nat.c
> +++ b/gdb/linux-nat.c
> @@ -3706,8 +3706,12 @@ linux_nat_target::xfer_partial (enum target_object object,
>         if (addr_bit < (sizeof (ULONGEST) * HOST_CHAR_BIT))
>   	offset &= ((ULONGEST) 1 << addr_bit) - 1;
>   
> -      return linux_proc_xfer_memory_partial (readbuf, writebuf,
> -					     offset, len, xfered_len);
> +      enum target_xfer_status xfer
> +	= linux_proc_xfer_memory_partial (readbuf, writebuf,
> +					  offset, len, xfered_len);
> +      if (xfer != TARGET_XFER_E_IO || readbuf != nullptr)
> +	return xfer;
> +      /* Fallthrough to ptrace.  /proc/pid/mem wasn't writable before Linux 2.6.39.  */
>       }
>   
>     return inf_ptrace_target::xfer_partial (object, annex, readbuf, writebuf,

Re: [PATCH] linux_nat_target::xfer_partial: Fallback to ptrace

[Keith Seitz]

Pedro Alves wrote:

> I guess the write to /proc/pid/mem fails with EIO for you, and there's nothing
> else we can use to detect the scenario.  So we probably want to check TARGET_XFER_E_IO
> instead.  And, maybe only do the fallback if writing.

I've updated the patch to include these changes and retested on all the "usual"
target boards on Fedora 36 x86_64, s390x, ppc64le, and aarch64 and on RHEL6
x86_64.

Keith

--

Commit 05c06f318fd9a112529dfc313e6512b399a645e4 enabled GDB
to access memory while threads are running. It did this by accessing
/proc/PID/task/LWP/mem.

Unfortunately, this interface is not implemented for writing in older kernels
(such as RHEL6). This means that GDB is unable to insert breakpoints on
these hosts:

$ ./gdb -q gdb -ex start
Reading symbols from gdb...
Temporary breakpoint 1 at 0x40fdd5: file ../../src/gdb/gdb.c, line 28.
Starting program: /home/rhel6/fsf/linux/gdb/gdb
Warning:
Cannot insert breakpoint 1.
Cannot access memory at address 0x40fdd5

(gdb)

Before this patch, linux_proc_xfer_memory_partial (previously called
linux_proc_xfer_partial) would return TARGET_XFER_EOF if the write
to /proc/PID/mem failed. [More specifically, linux_proc_xfer_partial would
not "bother for one word," but the effect is the essentially same.]

This status was checked by linux_nat_target::xfer_partial, which would then
fallback to using ptrace to perform the operation.

This is the specific hunk that removed the fallback:

-  xfer = linux_proc_xfer_partial (object, annex, readbuf, writebuf,
-                                 offset, len, xfered_len);
-  if (xfer != TARGET_XFER_EOF)
-    return xfer;
+      return linux_proc_xfer_memory_partial (readbuf, writebuf,
+                                            offset, len, xfered_len);
+    }

   return inf_ptrace_target::xfer_partial (object, annex, readbuf, writebuf,
                                          offset, len, xfered_len);

This patch restores this fallback mechanism, enabling GDB to insert
breakpoints on these older kernels. Note that a recent patch changed
the return status from TARGET_XFER_EOF to TARGET_XFER_E_IO.

Tested on {unix,native-gdbserver,native-extended-gdbserver}/-m{32,64}
on x86_64, s390x, aarch64, and ppc64le.
---
 gdb/linux-nat.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/gdb/linux-nat.c b/gdb/linux-nat.c
index 3b5400896bc..571c97137c2 100644
--- a/gdb/linux-nat.c
+++ b/gdb/linux-nat.c
@@ -3706,8 +3706,12 @@ linux_nat_target::xfer_partial (enum target_object object,
       if (addr_bit < (sizeof (ULONGEST) * HOST_CHAR_BIT))
 	offset &= ((ULONGEST) 1 << addr_bit) - 1;
 
-      return linux_proc_xfer_memory_partial (readbuf, writebuf,
-					     offset, len, xfered_len);
+      enum target_xfer_status xfer
+	= linux_proc_xfer_memory_partial (readbuf, writebuf,
+					  offset, len, xfered_len);
+      if (xfer != TARGET_XFER_E_IO || readbuf != nullptr)
+	return xfer;
+      /* Fallthrough to ptrace.  /proc/pid/mem wasn't writable before Linux 2.6.39.  */
     }
 
   return inf_ptrace_target::xfer_partial (object, annex, readbuf, writebuf,
-- 
2.36.1