[PATCHv4 3/4] gdb/python: add gdb.RemoteTargetConnection.send_packet

[Andrew Burgess <andrew.burgess@embecosm.com>]

This commits adds a new sub-class of gdb.TargetConnection,
gdb.RemoteTargetConnection.  This sub-class is created for all
'remote' and 'extended-remote' targets.

This new sub-class has one additional method over its base class,
'send_packet'.  This new method is equivalent to the 'maint
packet' CLI command, it allows a custom packet to be sent to a remote
target.

The result of calling RemoteTargetConnection.send_packet is a string
containing the reply that came from the remote.
---
 gdb/NEWS                                    |   8 +-
 gdb/doc/gdb.texinfo                         |   1 +
 gdb/doc/python.texi                         |  40 ++++-
 gdb/python/py-connection.c                  | 174 +++++++++++++++++++-
 gdb/testsuite/gdb.python/py-connection.exp  |  14 +-
 gdb/testsuite/gdb.python/py-send-packet.c   |  22 +++
 gdb/testsuite/gdb.python/py-send-packet.exp |  56 +++++++
 gdb/testsuite/gdb.python/py-send-packet.py  |  81 +++++++++
 8 files changed, 385 insertions(+), 11 deletions(-)
 create mode 100644 gdb/testsuite/gdb.python/py-send-packet.c
 create mode 100644 gdb/testsuite/gdb.python/py-send-packet.exp
 create mode 100644 gdb/testsuite/gdb.python/py-send-packet.py

diff --git a/gdb/NEWS b/gdb/NEWS
index 3bd13993005..a2d3505d14e 100644
--- a/gdb/NEWS
+++ b/gdb/NEWS
@@ -46,7 +46,9 @@ maint show internal-warning backtrace
      before GDB starts to clean up its internal state.
 
   ** New gdb.TargetConnection object type that represents a connection
-     (as displayed by the 'info connections' command).
+     (as displayed by the 'info connections' command).  A sub-class,
+     gdb.RemoteTargetConnection, is used to represent 'remote' and
+     'extended-remote' connections.
 
   ** The gdb.Inferior type now has a 'connection' property which is an
      instance of gdb.TargetConnection, the connection used by this
@@ -60,6 +62,10 @@ maint show internal-warning backtrace
   ** New gdb.connections() function that returns a list of all
      currently active connections.
 
+  ** New gdb.RemoteTargetConnection.send_packet(STRING) method.  This
+     is equivalent to the existing 'maint packet' CLI command; it
+     allows a user specified packet to be sent to the remote target.
+
 *** Changes in GDB 11
 
 * The 'set disassembler-options' command now supports specifying options
diff --git a/gdb/doc/gdb.texinfo b/gdb/doc/gdb.texinfo
index 631a7c03b31..89cf86210ab 100644
--- a/gdb/doc/gdb.texinfo
+++ b/gdb/doc/gdb.texinfo
@@ -39269,6 +39269,7 @@
 error stream.  This is @samp{on} by default for @code{internal-error}
 and @samp{off} by default for @code{internal-warning}.
 
+@anchor{maint packet}
 @kindex maint packet
 @item maint packet @var{text}
 If @value{GDBN} is talking to an inferior via the serial protocol,
diff --git a/gdb/doc/python.texi b/gdb/doc/python.texi
index e62e2703b85..587cc934ca4 100644
--- a/gdb/doc/python.texi
+++ b/gdb/doc/python.texi
@@ -5980,9 +5980,9 @@
 Examples of different connection types are @samp{native} and
 @samp{remote}.  @xref{Inferiors Connections and Programs}.
 
-@value{GDBN} uses the @code{gdb.TargetConnection} object type to
-represent a connection in Python code.  To get a list of all
-connections use @code{gdb.connections}
+Connections in @value{GDBN} are represented as instances of
+@code{gdb.TargetConnection}, or as one of its sub-classes.  To get a
+list of all connections use @code{gdb.connections}
 (@pxref{gdbpy_connections,,gdb.connections}).
 
 To get the connection for a single @code{gdb.Inferior} read its
@@ -6035,6 +6035,40 @@
 to the remote target.
 @end defvar
 
+The @code{gdb.RemoteTargetConnection} class is a sub-class of
+@code{gdb.TargetConnection}, and is used to represent @samp{remote}
+and @samp{extended-remote} connections.  In addition to the attributes
+and methods available from the @code{gdb.TargetConnection} base class,
+a @code{gdb.RemoteTargetConnection} has the following method:
+
+@kindex maint packet
+@defun RemoteTargetConnection.send_packet (@var{packet})
+This method sends @var{packet}, which should be a non-empty string, to
+the remote target and returns the response.  If @var{packet} is not a
+string, or is the empty string, then an exception of type
+@code{ValueError} is thrown.
+
+In Python 3, the response is returned as a @code{bytes} buffer, if it
+is known that the response can be represented as a string then this
+can be decoded from the buffer.  For example, if it is known that the
+response is an @code{"ascii"} string:
+
+@smallexample
+remote_connection.send_packet("some_packet").decode("ascii")
+@end smallexample
+
+In Python 2, the response is returned as a @code{str}, which may
+contain non-printable characters.
+
+The prefix, suffix, and checksum (as required by the remote serial
+protocol) are automatically added to the outgoing packet, and removed
+from the incoming packet before the contents of the reply are
+returned.
+
+This is equivalent to the @code{maintenance packet} command
+(@pxref{maint packet}).
+@end defun
+
 @node TUI Windows In Python
 @subsubsection Implementing new TUI windows
 @cindex Python TUI Windows
diff --git a/gdb/python/py-connection.c b/gdb/python/py-connection.c
index f1dfa26e39c..f47182d720e 100644
--- a/gdb/python/py-connection.c
+++ b/gdb/python/py-connection.c
@@ -26,6 +26,8 @@
 #include "py-events.h"
 #include "py-event.h"
 #include "arch-utils.h"
+#include "remote.h"
+#include "charset.h"
 
 #include <map>
 
@@ -47,6 +49,9 @@ struct connection_object
 extern PyTypeObject connection_object_type
   CPYCHECKER_TYPE_OBJECT_FOR_TYPEDEF ("connection_object");
 
+extern PyTypeObject remote_connection_object_type
+  CPYCHECKER_TYPE_OBJECT_FOR_TYPEDEF ("remote_connection_object");
+
 /* Require that CONNECTION be valid.  */
 #define CONNPY_REQUIRE_VALID(connection)			\
   do {								\
@@ -81,8 +86,15 @@ target_to_connection_object (process_stratum_target *target)
   auto conn_obj_iter = all_connection_objects.find (target);
   if (conn_obj_iter == all_connection_objects.end ())
     {
-      conn_obj.reset (PyObject_New (connection_object,
-				    &connection_object_type));
+      PyTypeObject *type;
+
+      std::string connection_type = target->shortname ();
+      if (connection_type == "remote" || connection_type == "extended-remote")
+	type = &remote_connection_object_type;
+      else
+	type = &connection_object_type;
+
+      conn_obj.reset (PyObject_New (connection_object, type));
       if (conn_obj == nullptr)
 	return nullptr;
       conn_obj->target = target;
@@ -284,9 +296,111 @@ gdbpy_initialize_connection (void)
 			      (PyObject *) &connection_object_type) < 0)
     return -1;
 
+  if (PyType_Ready (&remote_connection_object_type) < 0)
+    return -1;
+
+  if (gdb_pymodule_addobject (gdb_module, "RemoteTargetConnection",
+			      (PyObject *) &remote_connection_object_type) < 0)
+    return -1;
+
   return 0;
 }
 
+/* Set of callbacks used to implement gdb.send_packet.  */
+
+struct py_send_packet_callbacks : public send_remote_packet_callbacks
+{
+  /* Constructor, initialise the result to None.  */
+
+  py_send_packet_callbacks ()
+    : m_result (Py_None)
+  { /* Nothing.  */ }
+
+  /* There's nothing to do when the packet is sent.  */
+
+  void sending (const char *args) override
+  { /* Nothing.  */ }
+
+  /* When the result is returned create a Python string and assign this
+     into the result member variable.  */
+
+  void received (const gdb::char_vector &buf) override
+  {
+    /* m_result is initialized to None, leave it untouched unless we got
+       back a useful result.  */
+    if (buf.data ()[0] != '\0')
+      {
+	PyObject *result;
+
+#ifdef IS_PY3K
+	result = Py_BuildValue("y#", buf.data (), strlen (buf.data ()));
+#else
+	result = PyString_FromStringAndSize (buf.data (),
+					     strlen (buf.data ()));
+#endif
+
+	if (result != nullptr)
+	  m_result = gdbpy_ref<> (result);
+      }
+  }
+
+  /* Get a reference to the result as a Python object.  */
+
+  gdbpy_ref<> result () const
+  {
+    return m_result;
+  }
+
+private:
+
+  /* A reference to a valid result value.  This is initialized in the
+     constructor, and so will always point to a valid value, even if this
+     is just None.  */
+
+  gdbpy_ref<> m_result;
+};
+
+/* Implement RemoteTargetConnection.send_packet function.  Send a packet to
+   the target identified by SELF.  The connection must still be valid, and
+   the packet to be sent must be non-empty, otherwise an exception will be
+   thrown.  */
+
+static PyObject *
+connpy_send_packet (PyObject *self, PyObject *args, PyObject *kw)
+{
+  connection_object *conn = (connection_object *) self;
+
+  CONNPY_REQUIRE_VALID (conn);
+
+  static const char *keywords[] = {"packet", nullptr};
+  const char *packet_str = nullptr;
+
+  if (!gdb_PyArg_ParseTupleAndKeywords (args, kw, "s", keywords,
+					&packet_str))
+    return nullptr;
+
+  if (packet_str == nullptr || *packet_str == '\0')
+    {
+      PyErr_SetString (PyExc_ValueError, _("Invalid remote packet"));
+      return nullptr;
+    }
+
+  try
+    {
+      scoped_restore_current_thread restore_thread;
+      switch_to_target_no_thread (conn->target);
+
+      py_send_packet_callbacks callbacks;
+      send_remote_packet (packet_str, &callbacks);
+      return callbacks.result ().release ();
+    }
+  catch (const gdb_exception &except)
+    {
+      gdbpy_convert_exception (except);
+      return nullptr;
+    }
+}
+
 /* Global initialization for this file.  */
 
 void _initialize_py_connection ();
@@ -307,6 +421,17 @@ Return true if this TargetConnection is valid, false if not." },
   { NULL }
 };
 
+/* Methods for the gdb.RemoteTargetConnection object type.  */
+
+static PyMethodDef remote_connection_object_methods[] =
+{
+  { "send_packet", (PyCFunction) connpy_send_packet,
+    METH_VARARGS | METH_KEYWORDS,
+    "send_packet (PACKET) -> String\n\
+Send PACKET to a remote target, return the reply as a string." },
+  { NULL }
+};
+
 /* Attributes for the gdb.TargetConnection object type.  */
 
 static gdb_PyGetSetDef connection_object_getset[] =
@@ -345,7 +470,7 @@ PyTypeObject connection_object_type =
   0,				  /* tp_getattro */
   0,				  /* tp_setattro */
   0,				  /* tp_as_buffer */
-  Py_TPFLAGS_DEFAULT,		  /* tp_flags */
+  Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE,	/* tp_flags */
   "GDB target connection object", /* tp_doc */
   0,				  /* tp_traverse */
   0,				  /* tp_clear */
@@ -364,3 +489,46 @@ PyTypeObject connection_object_type =
   0,				  /* tp_init */
   0				  /* tp_alloc */
 };
+
+/* Define the gdb.RemoteTargetConnection object type.  */
+
+PyTypeObject remote_connection_object_type =
+{
+  PyVarObject_HEAD_INIT (NULL, 0)
+  "gdb.RemoteTargetConnection",	  /* tp_name */
+  sizeof (connection_object),	  /* tp_basicsize */
+  0,				  /* tp_itemsize */
+  connpy_connection_dealloc,	  /* tp_dealloc */
+  0,				  /* tp_print */
+  0,				  /* tp_getattr */
+  0,				  /* tp_setattr */
+  0,				  /* tp_compare */
+  connpy_repr,			  /* tp_repr */
+  0,				  /* tp_as_number */
+  0,				  /* tp_as_sequence */
+  0,				  /* tp_as_mapping */
+  0,				  /* tp_hash  */
+  0,				  /* tp_call */
+  0,				  /* tp_str */
+  0,				  /* tp_getattro */
+  0,				  /* tp_setattro */
+  0,				  /* tp_as_buffer */
+  Py_TPFLAGS_DEFAULT,		  /* tp_flags */
+  "GDB remote target connection object",	  /* tp_doc */
+  0,				  /* tp_traverse */
+  0,				  /* tp_clear */
+  0,				  /* tp_richcompare */
+  0,				  /* tp_weaklistoffset */
+  0,				  /* tp_iter */
+  0,				  /* tp_iternext */
+  remote_connection_object_methods,	  /* tp_methods */
+  0,				  /* tp_members */
+  0,				  /* tp_getset */
+  &connection_object_type,	  /* tp_base */
+  0,				  /* tp_dict */
+  0,				  /* tp_descr_get */
+  0,				  /* tp_descr_set */
+  0,				  /* tp_dictoffset */
+  0,				  /* tp_init */
+  0				  /* tp_alloc */
+};
diff --git a/gdb/testsuite/gdb.python/py-connection.exp b/gdb/testsuite/gdb.python/py-connection.exp
index b805b052f73..96c83781839 100644
--- a/gdb/testsuite/gdb.python/py-connection.exp
+++ b/gdb/testsuite/gdb.python/py-connection.exp
@@ -33,12 +33,18 @@ if ![runto_main] then {
     return 0
 }
 
+if { [target_info exists gdb_protocol] } {
+    set connection_type "RemoteTargetConnection"
+} else {
+    set connection_type "TargetConnection"
+}
+
 # Create a gdb.TargetConnection object and check it is initially
 # valid.
 gdb_test_no_output "python conn = gdb.selected_inferior().connection"
 gdb_test "python print(conn)" \
-    "<gdb.TargetConnection num=1, what=\"\[^\"\]+\">" \
-    "print gdb.TargetConnection while it is still valid"
+    "<gdb.${connection_type} num=1, what=\"\[^\"\]+\">" \
+    "print gdb.${connection_type} while it is still valid"
 gdb_test "python print(conn.is_valid())" "True" "is_valid returns True"
 
 # Get the connection again, and ensure we get the exact same object.
@@ -53,8 +59,8 @@ gdb_test "disconnect" "" "kill the inferior" \
     "A program is being debugged already\\.  Kill it\\? .*y or n. $" "y"
 gdb_test "info connections" "No connections\\." \
     "info connections now all the connections have gone"
-gdb_test "python print(conn)" "<gdb.TargetConnection \\(invalid\\)>" \
-    "print gdb.TargetConnection now its invalid"
+gdb_test "python print(conn)" "<gdb.${connection_type} \\(invalid\\)>" \
+    "print gdb.${connection_type} now its invalid"
 gdb_test "python print(conn.is_valid())" "False" "is_valid returns False"
 
 # Now check that accessing properties of the invalid connection cases
diff --git a/gdb/testsuite/gdb.python/py-send-packet.c b/gdb/testsuite/gdb.python/py-send-packet.c
new file mode 100644
index 00000000000..bfe52c018d4
--- /dev/null
+++ b/gdb/testsuite/gdb.python/py-send-packet.c
@@ -0,0 +1,22 @@
+/* This testcase is part of GDB, the GNU debugger.
+
+   Copyright 2021 Free Software Foundation, Inc.
+
+   This program is free software; you can redistribute it and/or modify
+   it under the terms of the GNU General Public License as published by
+   the Free Software Foundation; either version 3 of the License, or
+   (at your option) any later version.
+
+   This program is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+   GNU General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with this program.  If not, see <http://www.gnu.org/licenses/>.  */
+
+int
+main (void)
+{
+  return 0;
+}
diff --git a/gdb/testsuite/gdb.python/py-send-packet.exp b/gdb/testsuite/gdb.python/py-send-packet.exp
new file mode 100644
index 00000000000..dde5c270f20
--- /dev/null
+++ b/gdb/testsuite/gdb.python/py-send-packet.exp
@@ -0,0 +1,56 @@
+# Copyright (C) 2021 Free Software Foundation, Inc.
+
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# Test the gdb.RemoteTargetConnection.send_packet API.  This is done
+# by connecting to a remote target and fetching the thread list in two
+# ways, first, we manually send the packets required to read the
+# thread list using gdb.TargetConnection.send_packet, then we compare
+# the results to the thread list using the standard API calls.
+
+load_lib gdb-python.exp
+load_lib gdbserver-support.exp
+
+standard_testfile
+
+if {[skip_gdbserver_tests]} {
+    return 0
+}
+
+if { [prepare_for_testing "failed to prepare" ${testfile} ${srcfile}] } {
+    return -1
+}
+
+if { [skip_python_tests] } {
+    return 0
+}
+
+# Make sure we're disconnected, in case we're testing with an
+# extended-remote board, therefore already connected.
+gdb_test "disconnect" ".*"
+
+gdbserver_run ""
+
+# Source the python script.
+set remote_python_file [gdb_remote_download host \
+			    ${srcdir}/${subdir}/${testfile}.py]
+gdb_test "source $remote_python_file" "Sourcing complete\\." \
+    "source ${testfile}.py script"
+
+# The test is actually written in the Python script.  Run it now.
+gdb_test "python run_send_packet_test()" "Send packet test passed"
+
+# Check the string representation of a remote target connection.
+gdb_test "python print(gdb.selected_inferior().connection)" \
+    "<gdb.RemoteTargetConnection num=$decimal, what=\".*\">"
diff --git a/gdb/testsuite/gdb.python/py-send-packet.py b/gdb/testsuite/gdb.python/py-send-packet.py
new file mode 100644
index 00000000000..35754164a98
--- /dev/null
+++ b/gdb/testsuite/gdb.python/py-send-packet.py
@@ -0,0 +1,81 @@
+# Copyright (C) 2021 Free Software Foundation, Inc.
+
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import xml.etree.ElementTree as ET
+import gdb
+
+# Make use of gdb.RemoteTargetConnection.send_packet to fetch the
+# thread list from the remote target.
+#
+# Sending existing serial protocol packets like this is not a good
+# idea, there should be better ways to get this information using an
+# official API, this is just being used as a test case.
+#
+# Really, the send_packet API would be used to send target
+# specific packets to the target, but these are, by definition, target
+# specific, so hard to test in a general testsuite.
+def get_thread_list_str():
+    start_pos = 0
+    thread_desc = ""
+    conn = gdb.selected_inferior().connection
+    if not isinstance(conn, gdb.RemoteTargetConnection):
+        raise gdb.GdbError("connection is the wrong type")
+    while True:
+        str = conn.send_packet("qXfer:threads:read::%d,200" % start_pos).decode("ascii")
+        start_pos += 200
+        c = str[0]
+        str = str[1:]
+        thread_desc += str
+        if c == "l":
+            break
+    return thread_desc
+
+
+# Use gdb.RemoteTargetConnection.send_packet to manually fetch the
+# thread list, then extract the thread list using the gdb.Inferior and
+# gdb.InferiorThread API.  Compare the two results to ensure we
+# managed to successfully read the thread list from the remote.
+def run_send_packet_test():
+    # Find the IDs of all current threads.
+    all_threads = {}
+    for inf in gdb.inferiors():
+        for thr in inf.threads():
+            id = "p%x.%x" % (thr.ptid[0], thr.ptid[1])
+            all_threads[id] = False
+
+    # Now fetch the thread list from the remote, and parse the XML.
+    str = get_thread_list_str()
+    threads_xml = ET.fromstring(str)
+
+    # Look over all threads in the XML list and check we expected to
+    # find them, mark the ones we do find.
+    for thr in threads_xml:
+        id = thr.get("id")
+        if not id in all_threads:
+            raise "found unexpected thread in remote thread list"
+        else:
+            all_threads[id] = True
+
+    # Check that all the threads were found in the XML list.
+    for id in all_threads:
+        if not all_threads[id]:
+            raise "thread missingt from remote thread list"
+
+    # Test complete.
+    print("Send packet test passed")
+
+
+# Just to indicate the file was sourced correctly.
+print("Sourcing complete.")
-- 
2.25.4